Die cutter defect inspection method, device, medium and equipment

Abstract

The present invention relates to a die-cutting knife defect inspection method, device, medium and equipment, comprising: inputting each high-definition image of the surface of a die-cutting knife to be inspected, which is captured by multiple high-definition cameras, into a pre-trained image stitching model, so that each high-definition image is subjected to multiple mask sliding by the image stitching model, and attention values ​​are added to each mask image obtained, and stitching is performed according to the attention values ​​of multiple mask images in each high-definition image to obtain a die-cutting knife image to be identified; obtaining an artificial annotation type of the die-cutting knife defect in the die-cutting knife image to be identified, and determining a target die-cutting knife auxiliary identification image from a die-cutting knife auxiliary identification image according to the annotation type of the die-cutting knife defect in the die-cutting knife image to be identified; inputting both the die-cutting knife image to be identified and the target die-cutting knife identification auxiliary image into the die-cutting knife defect recognition model, and obtaining a die-cutting knife defect inspection result output by the die-cutting knife defect recognition model.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a die-cutting knife defect inspection method, device, medium and equipment. Background Art

[0002] Die-cutting technology is widely used in packaging, printing and other fields. As a key tool in the die-cutting process, the quality of the die-cutting knife directly affects the precision and quality of the die-cut products. At present, with the continuous improvement of the market's requirements for the quality of die-cut products, defect inspection of die-cutting knives has become very important. However, the existing die-cutting knife defect inspection methods have many shortcomings.

[0003] On the one hand, the traditional manual inspection method is inefficient, and the inspection results are easily affected by the subjective factors of the inspectors, lacking accuracy and consistency. Inspectors need to observe and judge the die-cutting knives with their naked eyes and experience, which is not only time-consuming and laborious, but also difficult to detect some subtle defects.

[0004] On the other hand, although some automated inspection equipment has improved inspection efficiency to a certain extent, it often has the problem of low detection accuracy. These devices may not accurately identify small defects on the die-cutting knife, such as small cracks and notches, etc., which may lead to defective die-cutting knives being put into use, affecting the quality of die-cut products.

[0005] In addition, existing inspection methods also face great difficulties when inspecting die cutters with complex shapes. Due to the different shapes of die cutters, some parts with special shapes are difficult to be effectively inspected and are prone to missed inspections. Summary of the invention

[0006] The purpose of the present invention is to provide a die-cutting knife defect inspection method, device, medium and equipment, aiming to efficiently, accurately and reliably detect various defects on the die-cutting knife, overcome the problems of low efficiency of traditional manual inspection methods and the results being greatly affected by subjective factors, and at the same time solve the technical problems of low detection accuracy of existing automated inspection equipment and difficulty in dealing with complex-shaped die-cutting knives.

[0007] In order to achieve the above-mentioned purpose, a first aspect of an embodiment of the present disclosure provides a die cutting knife defect inspection method, the method comprising:

[0008] Inputting each high-definition image of the surface of the die-cutting knife to be inspected, which is captured by multiple high-definition cameras located at different positions of the die-cutting knife to be inspected and having different focal lengths, into a pre-trained image stitching model, so that each of the high-definition images is subjected to multiple mask sliding by the image stitching model, and attention values ​​are added to each mask image obtained, and stitching is performed according to the attention values ​​of the multiple mask images in each of the high-definition images to obtain a die-cutting knife image to be identified;

[0009] Obtaining the manually labeled type of the die-cutting knife defect in the die-cutting knife image to be identified, and determining a target die-cutting knife auxiliary identification image from the die-cutting knife auxiliary identification image according to the labeled type of the die-cutting knife defect in the die-cutting knife image to be identified, wherein the die-cutting knife auxiliary identification image is obtained by sequentially performing multiple scaling, multiple-direction pixel-level translation, and multiple-angle rotation on the training sample image for training the die-cutting knife defect identification model, the training sample image and the die-cutting knife auxiliary identification image are used together as input sample images to train the die-cutting knife defect identification model, and the labeled type and quantity of the die-cutting knife defect in the target die-cutting knife auxiliary identification image are respectively the same as the labeled type and quantity of the die-cutting knife defect in the die-cutting knife image to be identified;

[0010] The die-cutting knife image to be identified and the target die-cutting knife identification auxiliary image are both input into the die-cutting knife defect recognition model to obtain the die-cutting knife defect inspection result output by the die-cutting knife defect recognition model, wherein the die-cutting knife defect recognition model fine-tunes the number of layers and the learning rate settings through the target die-cutting knife identification auxiliary image before identifying the die-cutting knife image to be identified.

[0011] In a possible implementation, the step of fine-tuning the number of layers and the setting of the learning rate by identifying the auxiliary image of the target die-cutting knife includes:

[0012] The following steps are repeated until the loss value is minimized:

[0013] Inputting the target die-cutting knife recognition auxiliary image into the die-cutting knife defect recognition model, calculating the loss through a cross entropy loss function, and obtaining a loss value;

[0014] Determine the number of fine-tuning layers for fine-tuning the learning rate in the die-cutting knife defect recognition model according to the loss value, and determine the target fine-tuning residual layers in sequence from the residual layers close to the fully connected layer in the die-cutting knife defect recognition model according to the number of fine-tuning layers;

[0015] Determine the target fine-tuning learning rate for this cycle based on the difference between the loss value of this cycle and the loss value of the previous cycle.

[0016] According to the target fine-tuning learning rate, the learning rate of the target fine-tuning residual layer in the die-cutting knife defect recognition model is fine-tuned.

[0017] In a possible implementation, the die-cutting knife auxiliary recognition image is obtained by sequentially performing multiple scaling, multiple direction pixel-level translation, and multiple angle rotation on the training sample image for training the die-cutting knife defect recognition model, including:

[0018] According to the number of the annotated types of the die-cutting knife defect corresponding to the training sample image, the training sample image is enlarged and reduced at multiple ratios to obtain a first extended image of the die-cutting knife defect in the training sample image at different sizes;

[0019] Determine the pixel-level translation offset of the training sample image according to the resolution of the high-definition camera, translate the training sample image in multiple directions according to the pixel-level translation offset, and obtain a translation sample image corresponding to each direction;

[0020] The translation sample images corresponding to each direction are rotated at multiple angles to obtain multiple die-cutting knife auxiliary recognition images.

[0021] In a possible implementation, the annotation type includes any one of wear, crack, notch, and deformation, and the training sample image is enlarged and reduced in multiple proportions according to the number of annotation types of the die-cutting knife defect corresponding to the training sample image to obtain a first extended image of the die-cutting knife defect in the training sample image at different sizes, including:

[0022] In a case where the die-cutter defect corresponding to the training sample image is of any single annotation type, determining a first magnification ratio and a first reduction ratio of the training sample image, and magnifying and reducing the training sample image according to the first magnification ratio and the first reduction ratio, respectively, to obtain a first extended image of the die-cutter defect in the training sample image at different sizes;

[0023] In the case where the die-cutter defect corresponding to the training sample image is any combination of multiple annotation types, determining a second magnification ratio and a second reduction ratio of the training sample image, and magnifying and reducing the training sample image according to the second magnification ratio and the second reduction ratio, respectively, to obtain a first extended image of the die-cutter defect in the training sample image at different sizes;

[0024] The first enlargement ratio is greater than the second enlargement ratio, and the first reduction ratio is less than the second reduction ratio.

[0025] In a possible implementation, performing mask sliding multiple times on each of the high-definition images through the image stitching model and then adding an attention value to each mask image obtained include:

[0026] Determining a mask for performing multiple mask sliding on each of the high-definition images according to a size relationship between the resolution of each of the high-definition images and an average resolution of the high-definition images and a resolution difference;

[0027] The high-definition image is respectively subjected to multiple mask sliding by the mask corresponding to the high-definition image to obtain multiple mask images corresponding to the high-definition image;

[0028] Multiplying the vector corresponding to each of the mask images by three different weight matrices corresponding to the focal length of the high-definition camera corresponding to the mask image, respectively, to obtain a query vector, a key vector and a value vector corresponding to the mask image;

[0029] The attention value of each mask image is determined according to the image gradient of each high-definition image and the query vector, the key vector and the value vector corresponding to each mask image in the high-definition image.

[0030] In a possible implementation, determining the attention value of each mask image according to the image gradient of each high-definition image and the query vector, the key vector, and the value vector corresponding to each mask image in the high-definition image includes:

[0031] The following steps are performed for each high-definition image until any mask image in the high-definition image is used as a query to perform the following steps:

[0032] According to the image gradient of the high-definition image, along the direction in which the image gradient descends, determining, from the mask images included in the high-definition image, a dot product mask image of each mask image corresponding to the high-definition image;

[0033] Taking any of the mask images in each of the high-definition images as a query, determining an attention score of the mask image as the query according to a query vector of the mask image and a key vector of the dot product mask image;

[0034] According to the attention score of the mask image and the value vector of the mask image, a weighted sum is performed on the value vector of the mask image to obtain the attention value of the mask image.

[0035] In a possible implementation, determining a mask for performing multiple mask sliding on each of the high-definition images according to a size relationship between a resolution of each of the high-definition images and an average resolution of the high-definition images and a resolution difference includes:

[0036] For any of the high-definition images, when the size relationship indicates that the resolution of the high-definition image is greater than the average resolution of the high-definition images, determining that the scaling strategy for the standard mask is reduction, and determining a reduction ratio according to a resolution difference between the resolution of the high-definition image and the average resolution of the high-definition images, and determining a mask obtained by reducing the standard mask according to the reduction ratio as a mask for performing multiple mask sliding on the high-definition image;

[0037] For any of the high-definition images, when the size relationship indicates that the resolution of the high-definition image is less than the average resolution of the high-definition images, determining that the scaling strategy for the standard mask is magnification, and determining the magnification ratio according to the resolution difference between the resolution of the high-definition image and the average resolution of the high-definition images, and determining the mask obtained by magnifying the standard mask according to the magnification ratio as the mask used for performing multiple mask sliding on the high-definition image;

[0038] For any of the high-definition images, when the size relationship indicates that the resolution of the high-definition image is equal to the average resolution of each of the high-definition images, the standard mask is determined as a mask for performing multiple mask sliding on the high-definition image.

[0039] According to a second aspect of the embodiments of the present disclosure, there is provided a die cutter defect inspection device, comprising:

[0040] A first input module is configured to input each high-definition image of the surface of the die-cutting knife to be inspected, which is captured by multiple high-definition cameras located at different positions of the die-cutting knife to be inspected and have different focal lengths, into a pre-trained image stitching model, so as to add an attention value to each mask image obtained after performing multiple mask sliding on each of the high-definition images through the image stitching model, and stitching the multiple mask images in each of the high-definition images according to the attention values ​​to obtain a die-cutting knife image to be identified;

[0041] an acquisition and determination module, configured to acquire the type of manual annotation of the die-cutting knife defect in the die-cutting knife image to be identified, and determine the target die-cutting knife auxiliary identification image from the die-cutting knife auxiliary identification image according to the type of annotation of the die-cutting knife defect in the die-cutting knife image to be identified, wherein the die-cutting knife auxiliary identification image is obtained after sequentially performing multiple scaling, pixel-level translation in multiple directions and rotation at multiple angles on the training sample image for training the die-cutting knife defect identification model, the training sample image and the die-cutting knife auxiliary identification image are used together as input sample images to train the die-cutting knife defect identification model, and the annotation type and quantity of the die-cutting knife defect in the target die-cutting knife auxiliary identification image are respectively the same as the annotation type and quantity of the die-cutting knife defect in the die-cutting knife image to be identified;

[0042] The second input module is configured to input the die-cutting knife image to be identified and the target die-cutting knife identification auxiliary image into the die-cutting knife defect recognition model to obtain the die-cutting knife defect inspection result output by the die-cutting knife defect recognition model, wherein the die-cutting knife defect recognition model fine-tunes the number of layers and fine-tunes the learning rate settings through the target die-cutting knife identification auxiliary image before identifying the die-cutting knife image to be identified.

[0043] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0044] According to a fourth aspect of the embodiments of the present disclosure, an electronic device is provided, including:

[0045] a memory having a computer program stored thereon;

[0046] A processor is used to execute the computer program in the memory to implement the steps of any one of the methods in the first aspect.

[0047] The present invention provides a die cutter defect inspection method, device, medium and equipment. Compared with the prior art, it has the following beneficial effects:

[0048] The above technical solution captures high-definition images of the die-cutting knife surface through multiple high-definition cameras located in different directions and with different focal lengths, and then inputs these images into a pre-trained image stitching model. This model stitches the images by sliding the mask multiple times and adding attention values ​​to generate a complete and clear die-cutting knife image to be identified. It can ensure that the image information captured from different angles and focal lengths is fully utilized, improving the integrity of the image and the clarity of the details. The types of manual annotations of defects in the die-cutting knife image to be identified are obtained, and according to these annotation types, the target die-cutting knife auxiliary identification image is determined from the die-cutting knife auxiliary identification image. These auxiliary images are obtained by scaling, translating and rotating the training sample images, simulating different detection conditions. More training samples can be provided for the model to enhance the model's ability to identify defects under different conditions. The training sample images and the die-cutting knife auxiliary identification images are used as input sample images to train the die-cutting knife defect recognition model. The model can identify defects on the die-cutting knife and fine-tune according to the defect annotation type and number in the target die-cutting knife auxiliary identification image to adapt to different detection tasks. The die-cutting knife image to be identified and the target die-cutting knife identification auxiliary image are input into the die-cutting knife defect recognition model, and the model will output the die-cutting knife defect inspection result. Before recognition, the number of layers and the learning rate settings are fine-tuned according to the target die-cutting knife identification auxiliary image to improve the accuracy and efficiency of recognition. Through automated image capture, splicing, auxiliary image generation, model training and defect recognition, efficient and accurate detection of die-cutting knife defects is achieved. It can improve production efficiency, reduce the labor intensity of manual inspection, and improve the accuracy and reliability of defect detection.

[0049] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0051] Figure 1 It is a flow chart of a die cutting knife defect inspection method shown according to an embodiment of the specification.

[0052] Figure 2 It is a block diagram of a die cutting knife defect inspection device shown according to an embodiment of the specification.

[0053] Figure 3 It is a block diagram of another die cutting knife defect inspection device shown according to an embodiment of the specification. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0056] The present invention provides a die cutting knife defect inspection method. Figure 1 The flowchart of a die cutter defect inspection method according to an embodiment is shown. The method comprises:

[0057] In step S11, each high-definition image of the surface of the die-cutting knife to be inspected, captured by multiple high-definition cameras located at different positions of the die-cutting knife to be inspected and having different focal lengths, is input into a pre-trained image stitching model, so that each of the high-definition images is subjected to multiple mask sliding by the image stitching model, and an attention value is added to each mask image obtained, and the attention values ​​of the multiple mask images in each of the high-definition images are stitched together to obtain a die-cutting knife image to be identified;

[0058] In the disclosed embodiments, a die cutter is a tool used in the printing, packaging and other industries to cut or punch materials (such as paper, plastic film, printed circuit boards, etc.) into preset shapes. A high-definition camera is a camera device with high resolution that can capture clear and detailed images. An image stitching model is an algorithm model based on deep learning or computer vision technology, which is used to merge multiple images with overlapping areas into a wide-viewing angle, high-resolution image. Mask sliding is a technology that covers different parts of an image by sliding a window (i.e., a mask) and processing the image in each window separately. The attention value refers to the degree of attention the model pays to different parts of the input data, which is reflected in a weighted manner.

[0059] In the disclosed embodiment, a plurality of high-definition cameras located at different positions and with different focal lengths are used to photograph the surface of the die-cutting knife to be inspected from different angles and distances to obtain rich image data. These image data are then input into a pre-trained image stitching model. For example: suppose there are four high-definition cameras, located in front of, on the left side, above, and above the right of the die-cutting knife, and each camera has a specific focal length setting. These four cameras simultaneously photograph the surface of the die-cutting knife and obtain four images containing different perspectives and details. These images are then fed into the image stitching model for further processing.

[0060] Furthermore, the image stitching model first performs a mask sliding operation on each HD image, that is, sliding a window (mask) of a certain size on an image, and processing the image blocks in each window separately. During the processing, the model assigns an attention value to each mask image according to the characteristics of the image block (such as edges, textures, etc.), indicating the importance of the image block in the final stitched image. Based on the attention values ​​of the mask images in all HD images, the model fuses the effective information in multiple images and stitches them into a complete die-cutting knife image to be identified.

[0061] For example, during the mask sliding process, the model may find that the edge of the die cutter is clearly displayed in multiple images, so the mask images of these areas will be given a higher attention value. On the contrary, non-critical areas on the surface of the die cutter (such as flat and featureless parts) may receive a lower attention value. Finally, by weighted fusion of these mask images, the model generates a stitched image containing the complete shape and details of the die cutter, which not only retains the resolution advantage of the high-definition camera, but also improves the comprehensiveness and accuracy of the image through multi-view fusion.

[0062] In step S12, the manually labeled type of the die-cutting knife defect in the die-cutting knife image to be identified is obtained, and according to the labeled type of the die-cutting knife defect in the die-cutting knife image to be identified, a target die-cutting knife auxiliary identification image is determined from the die-cutting knife auxiliary identification image, wherein the die-cutting knife auxiliary identification image is obtained after sequentially performing multiple scaling, pixel-level translation in multiple directions, and rotation at multiple angles on the training sample image for training the die-cutting knife defect identification model, the training sample image and the die-cutting knife auxiliary identification image are used together as input sample images to train the die-cutting knife defect identification model, and the labeled type and quantity of the die-cutting knife defect in the target die-cutting knife auxiliary identification image are respectively the same as the labeled type and quantity of the die-cutting knife defect in the die-cutting knife image to be identified;

[0063] In the disclosed embodiments, the annotation type is the manual classification or labeling of specific objects (such as defects) in the image, which is used to train or evaluate the accuracy of the recognition model. Pixel-level translation is the operation of moving an image a certain distance in a specified direction (such as horizontally or vertically) at the pixel level, which can perform data enhancement or image transformation on the image.

[0064] Scaling can adjust the size of an image by changing its width and height while keeping the relative proportions of the image content unchanged. Rotation is the process of rotating an image around a certain point (usually the center of the image) to change the orientation of the image, which is also commonly used for data enhancement. Training sample images are a collection of image data used to train machine learning or deep learning models. These images usually contain known labels or annotations to guide model learning.

[0065] In the disclosed embodiment, the die-cutting knife defects in the die-cutting knife image to be identified obtained in step S11 are manually annotated to clarify the types of defects (such as cracks, wear, stains, etc.). Based on these annotated types, images that match the types and quantities of defects in the image to be identified are screened from the die-cutting knife auxiliary identification image set as target die-cutting knife auxiliary identification images. These auxiliary identification images are obtained by performing a series of transformations (such as scaling, pixel-level translation, and rotation) on the training sample images, aiming to increase the generalization ability and recognition accuracy of the model.

[0066] For example: suppose that in the die-cutting knife image to be identified, two defects are manually annotated: a crack and a wear. Next, based on these annotation types, images with the same type and number of defects are searched in the die-cutting knife auxiliary identification image set. These auxiliary identification images may be obtained by scaling the original training sample images (such as normal die-cutting knife images or die-cutting knife images with known defects) at different ratios (such as enlarging to 1.2 times, reducing to 0.8 times), pixel-level translation in different directions (such as moving 10 pixels to the left, moving 5 pixels up), and rotating at different angles (such as rotating 10 degrees clockwise, rotating 5 degrees counterclockwise). Finally, find the auxiliary identification image that is exactly the same as the defect type and quantity in the image to be identified as the target image for subsequent defect identification or comparison analysis.

[0067] In step S13, the die-cutting knife image to be identified and the target die-cutting knife identification auxiliary image are both input into the die-cutting knife defect recognition model to obtain the die-cutting knife defect inspection result output by the die-cutting knife defect recognition model, wherein the die-cutting knife defect recognition model fine-tunes the number of layers and the learning rate settings through the target die-cutting knife identification auxiliary image before identifying the die-cutting knife image to be identified.

[0068] In the disclosed embodiment, fine-tuning refers to the process of using a model pre-trained on a large dataset and then adjusting parameters for a small dataset of die-cutter defect recognition. This helps to speed up training and improve the performance of the model on specific tasks. Fine-tuning the number of layers means that the parameters of all layers are not adjusted during the fine-tuning process. Instead, one or several network layers (such as the last few fully connected layers or convolutional layers) are selected for parameter update, while the parameters of other layers remain unchanged. These selected layers are the number of fine-tuning layers. Fine-tuning the learning rate means adjusting the learning rate during the fine-tuning process. The learning rate is an important hyperparameter that determines the step size of the parameter update. Compared with training a model from scratch, a smaller learning rate is usually used during fine-tuning to avoid destroying useful features that the pre-trained model has learned.

[0069] In the disclosed embodiment, the die-cutting knife image to be identified and the target die-cutting knife identification auxiliary image are used as input and simultaneously fed into the trained die-cutting knife defect recognition model. Before the model recognizes the die-cutting knife image to be identified, the target die-cutting knife identification auxiliary image is used to fine-tune the model. This step is intended to make the model better adapt to the data distribution and characteristics of the current task, thereby improving the accuracy of recognition.

[0070] The number of fine-tuning layers is to select a part of the network layers for fine-tuning based on the similarity between the target die-cutter recognition auxiliary image and the training sample image and the complexity of the model. These layers are usually the layers close to the model output because they have a greater impact on specific tasks. The fine-tuning learning rate is to set a smaller learning rate for the fine-tuning process to avoid destroying the important features learned by the model in the pre-training stage. The choice of learning rate may need to be determined through experiments to achieve the best fine-tuning effect.

[0071] Furthermore, the fine-tuned model recognizes the die-cutting knife image to be identified and outputs the inspection results of the die-cutting knife defects. These results may include information such as the type, location, and size of the defects.

[0072] For example: suppose there is a die-cutter defect recognition model that has been pre-trained on a large-scale die-cutter dataset. Now, fine-tune this model to improve its recognition performance on a specific batch of die-cutter images (including the die-cutter images to be recognized and the target die-cutter recognition auxiliary images). First, select the last two convolutional layers and the following fully connected layers of the die-cutter defect recognition model for fine-tuning. This is because these layers are closer to the output layer and have a greater impact on the recognition results of a specific task.

[0073] Next, set a small learning rate (such as 0.001) to prevent the global features that the model has learned from being destroyed during the fine-tuning process. The die-cutting knife image to be identified and the target die-cutting knife identification auxiliary image are sent as input to the fine-tuned model. The model uses the feature information in the target die-cutting knife identification auxiliary image to fine-tune its own parameters to adapt to the data distribution of the current task.

[0074] Finally, the model outputs the defect inspection results of the die-cutting knife image to be identified, including information such as the defect type (such as cracks, wear), location (such as coordinates in the image), and size (such as the area or perimeter of the defect area). This information can be used for subsequent quality control or troubleshooting.

[0075] The above technical solution captures high-definition images of the die-cutting knife surface through multiple high-definition cameras located in different directions and with different focal lengths, and then inputs these images into a pre-trained image stitching model. This model stitches the images by sliding the mask multiple times and adding attention values ​​to generate a complete and clear die-cutting knife image to be identified. It can ensure that the image information captured from different angles and focal lengths is fully utilized, improving the integrity of the image and the clarity of the details. The types of manual annotations of defects in the die-cutting knife image to be identified are obtained, and according to these annotation types, the target die-cutting knife auxiliary identification image is determined from the die-cutting knife auxiliary identification image. These auxiliary images are obtained by scaling, translating and rotating the training sample images, simulating different detection conditions. More training samples can be provided for the model to enhance the model's ability to identify defects under different conditions. The training sample images and the die-cutting knife auxiliary identification images are used as input sample images to train the die-cutting knife defect recognition model. The model can identify defects on the die-cutting knife and fine-tune according to the defect annotation type and number in the target die-cutting knife auxiliary identification image to adapt to different detection tasks. The die-cutting knife image to be identified and the target die-cutting knife identification auxiliary image are input into the die-cutting knife defect recognition model, and the model will output the die-cutting knife defect inspection result. Before recognition, the number of layers and the learning rate settings are fine-tuned according to the target die-cutting knife identification auxiliary image to improve the accuracy and efficiency of recognition. Through automated image capture, splicing, auxiliary image generation, model training and defect recognition, efficient and accurate detection of die-cutting knife defects is achieved. It can improve production efficiency, reduce the labor intensity of manual inspection, and improve the accuracy and reliability of defect detection.

[0076] In a possible implementation, in step S13, the step of fine-tuning the number of layers and the setting of the learning rate by using the target die-cutting knife recognition auxiliary image includes:

[0077] The following steps are repeated until the loss value is minimized:

[0078] In step S21, the target die-cutting knife recognition auxiliary image is input into the die-cutting knife defect recognition model, and the loss is calculated by a cross entropy loss function to obtain a loss value;

[0079] In the embodiment of the present disclosure, a cross entropy loss function is used to calculate the difference between the predicted result of the target die-cutter recognition auxiliary image after passing through the die-cutter defect recognition model and the true label to obtain a loss value. The loss value reflects the performance of the model under the current parameter settings.

[0080] For example, suppose the target die-cutter recognition auxiliary image contains a die-cutter image, whose true label is "no defect". After this image is input into the die-cutter defect recognition model, the model predicts that the result is "defective". The loss value calculated by the cross entropy loss function reflects the magnitude of this prediction error.

[0081] In step S22, the number of fine-tuning layers for fine-tuning the learning rate in the die-cutting knife defect recognition model is determined according to the loss value, and according to the number of fine-tuning layers, the target fine-tuning residual layers are sequentially determined from the residual layers close to the fully connected layer in the die-cutting knife defect recognition model;

[0082] In the disclosed embodiment, the number of layers that need to be fine-tuned is determined based on the change trend of the loss value and the model structure. In a residual network, the target fine-tuning residual layer is usually selected starting from the residual layer close to the fully connected layer.

[0083] For example: Assume that the die-cutter defect recognition model is a ResNet model containing multiple residual blocks. In the early stage of training, it is found that the loss value decreases slowly, which may indicate that the latter layers of the model have not learned the characteristics of the target die-cutter well. Therefore, it is decided to start from the first residual block close to the fully connected layer and gradually select the residual layers forward for fine-tuning.

[0084] In step S23, the target fine-tuning learning rate corresponding to the current cycle is determined according to the difference between the loss value corresponding to the current cycle and the loss value of the previous cycle of the current cycle;

[0085] In the disclosed embodiment, the learning rate is dynamically adjusted according to the change of the loss value (such as the difference between the loss value of the current cycle and the previous cycle). If the loss value decreases quickly, it means that the current learning rate may be too large and should be appropriately reduced; if the loss value decreases slowly or even increases, it means that the learning rate may be too small and should be appropriately increased.

[0086] For example: If the loss value drops from 0.5 to 0.4 (a fast drop) in two consecutive cycles, it may be considered that the current learning rate is too large and the target fine-tuning learning rate should be appropriately reduced. On the contrary, if the loss value changes from 0.5 to 0.51 (almost unchanged or slightly increased), the learning rate should be increased.

[0087] In step S24, the learning rate of the target fine-tuning residual layer in the die-cutting knife defect recognition model is fine-tuned according to the target fine-tuning learning rate.

[0088] In the disclosed embodiment, the parameters in the selected target fine-tuning residual layer are updated according to the target fine-tuning learning rate determined in step S23. The adjustment of the learning rate will affect the step size of the parameter update, thereby affecting the training speed and final performance of the model.

[0089] For example: Assume that the target fine-tuning learning rate is 0.001 after calculation in step S23. In the current loop, this learning rate will be used to update the parameters in the target fine-tuning residual layer selected in step S22. Through multiple loop iterations, the learning rate and the number of fine-tuning layers are continuously adjusted until the loss value reaches the minimum or other stopping conditions are met.

[0090] The above technical solution can more finely control the training process of the model by dynamically determining the number of fine-tuning layers and the target fine-tuning learning rate according to the change of the loss value. When the loss value decreases slowly or stagnates, by adjusting the learning rate and selecting a more appropriate number of fine-tuning layers, the model parameters can be further optimized, thereby improving the recognition accuracy of the model. By fine-tuning the residual layer close to the fully connected layer, the general features learned by the model at the shallow layer can be retained, while the deep features are adjusted for specific tasks (such as die-cutting knife defect recognition). This strategy helps the model better adapt to recognition tasks in specific fields while maintaining the ability to understand general image features, thereby improving the generalization ability of the model. By dynamically adjusting the learning rate, a larger learning rate can be used in the early stage of training to quickly reduce the loss value, and the learning rate can be gradually reduced as the training progresses to avoid overfitting and oscillation. This strategy helps to accelerate the convergence process of the model and reduce the training time. During the training process, the change of the loss value may be affected by many factors, such as data distribution, model structure, etc. By looping the above steps and continuously monitoring the change of the loss value, potential training problems (such as gradient disappearance, gradient explosion, etc.) can be discovered and responded to in a timely manner, thereby enhancing the stability of the model.

[0091] In summary, by cyclically executing loss calculation, fine-tuning the number of layers, and adjusting the learning rate, the die-cutting knife defect recognition model is refined and optimized. This not only improves the recognition accuracy and generalization ability of the model, but also accelerates the convergence process of the model and enhances its stability.

[0092] In a possible implementation, in step S12, the training sample image according to the training die-cutting knife defect recognition model is sequentially subjected to multiple scaling, multiple direction pixel-level translation, and multiple angle rotation to obtain the die-cutting knife auxiliary recognition image, including:

[0093] In step S121, the training sample image is enlarged and reduced at multiple ratios according to the number of annotated types of the die-cutting knife defect corresponding to the training sample image, so as to obtain a first extended image of the die-cutting knife defect in the training sample image at different sizes;

[0094] In the embodiment of the present disclosure, the first extended image is a set of images obtained by enlarging and reducing the training sample image at multiple ratios. The die-cutting knife defects in these images are presented in different sizes, which are used to enhance the model's recognition ability for defects of different sizes.

[0095] In the disclosed embodiment, the image is enlarged or reduced in different proportions according to the number of annotated types of die cutter defects in the training sample image. This allows the model to be exposed to more defects of varying sizes during the training process, thereby improving the robustness of the model to defects of different sizes.

[0096] For example, suppose the training sample image contains a type of die cutter defect (such as a crack). Based on the annotation of the defect, you may choose to enlarge the image to 1.2 times, 1.5 times, and reduce it to 0.8 times, 0.6 times, to obtain four first expanded images of different sizes. The width and length of the cracks in these images will change accordingly, which helps the model learn how to identify cracks of different sizes.

[0097] In step S122, according to the resolution of the high-definition camera, the pixel-level translation offset of the training sample image is determined, and the training sample image is translated in multiple directions according to the pixel-level translation offset to obtain a translation sample image corresponding to each direction;

[0098] In the disclosed embodiment, the pixel-level translation offset is the distance that the image moves in a specific direction at the pixel level. This offset is determined according to the resolution of the high-definition camera and the desired data enhancement effect. The translation sample image is an image obtained by performing a translation operation on the training sample image according to the pixel-level translation offset, and is used to simulate the slight position deviation that may occur when the camera is shooting.

[0099] In the disclosed embodiment, the pixel-level translation offset is determined according to the resolution of the high-definition camera and the desired data enhancement effect. Then, the training sample image is translated in multiple directions (such as up, down, left, and right) according to these offsets to simulate the position deviation that may exist in actual shooting.

[0100] For example, if the resolution of the HD camera is 1920x1080, a small pixel-level translation offset (such as 10 pixels) may be set, and then the training sample image is translated by 10 pixels up, down, left, and right, respectively, to obtain four translated sample images. The die cutter and defect positions in these images will be slightly offset, which helps the model learn how to identify defects in different positions.

[0101] In step S123, the translation sample images corresponding to each direction are rotated at multiple angles to obtain multiple die-cutting knife auxiliary recognition images.

[0102] In the disclosed embodiment, the translation sample images corresponding to each direction are rotated at multiple angles (such as 0°, 90°, 180°, 270°, etc.) to further increase the diversity of the data. The rotated images are used as auxiliary recognition images for die cutting knives to train or evaluate die cutting knife defect recognition models.

[0103] For example, for the four translated sample images obtained in step S122 (one in each direction), you may choose to rotate 0°, 90°, 180° and 270° respectively, so that each translated sample image will generate four rotated images. Therefore, multiple die-cutting knife auxiliary recognition images will be generated in the end (the specific number depends on the number of translation directions and rotation angles). These images contain the manifestations of die-cutting knife defects in different positions, sizes and angles, which help improve the recognition performance of the model.

[0104] The above technical solution obtains the first extended image containing the die-cutting knife defect at different sizes by enlarging and reducing the training sample image at multiple ratios. This helps the model learn the characteristics of the defect size change, thereby improving the recognition accuracy of defects of different sizes. The pixel-level translation offset is determined according to the resolution of the high-definition camera, and the training sample image is translated in multiple directions. This simulates the position deviation that may exist in actual shooting, so that the model can learn how to effectively identify defects at different positions during the training process. Therefore, the model has higher robustness to changes in the position of defects in the image. The translated sample image is rotated at multiple angles to generate multiple die-cutting knife auxiliary recognition images. These images vary in size, position and angle, which greatly enriches the diversity of the training data set. Using such a data set to train the model helps reduce the risk of overfitting of the model during the training process and improve the generalization ability of the model.

[0105] In summary, through image processing techniques such as scaling, pixel-level translation and rotation, the training sample images are effectively expanded to generate diversified die-cutting knife auxiliary recognition images. These images not only enhance the model's ability to recognize defects of different sizes and positions, but also reduce the risk of overfitting by increasing data diversity, thereby improving the overall performance and robustness of the die-cutting knife defect recognition model.

[0106] In a possible implementation, the annotation type includes any one of wear, crack, notch, and deformation. In step S121, the training sample image is enlarged and reduced in multiple proportions according to the number of annotation types of the die-cutting knife defect corresponding to the training sample image, to obtain a first extended image of the die-cutting knife defect in the training sample image at different sizes, including:

[0107] In step S1211, when the die-cutter defect corresponding to the training sample image is of any single annotation type, a first magnification ratio and a first reduction ratio of the training sample image are determined, and the training sample image is magnified and reduced according to the first magnification ratio and the first reduction ratio respectively, to obtain a first extended image of the die-cutter defect in the training sample image at different sizes;

[0108] In the disclosed embodiment, when the die cutter defect in the training sample image corresponds to only one annotation type (such as only wear), the appropriate magnification and reduction ratios are determined according to the characteristics of this single defect. The selection of these ratios is usually based on experience or experiments, aiming to ensure that the defect can still be effectively identified by the model at different sizes.

[0109] For example: suppose there is a training sample image in which there is an obvious wear defect on the die cutter. First, a first magnification ratio (such as 1.5 times) and a first reduction ratio (such as 0.7 times) are determined. Then, the original image is magnified and reduced according to these two ratios, respectively, to obtain two new images. In these two images, the wear defect is presented in the form of magnification and reduction, respectively, forming the first extended images at different sizes.

[0110] In the case of a single annotation type and multiple annotation types, different magnification and reduction ratios are determined and the corresponding first extended images are generated, which helps the model learn more comprehensive and detailed defect features. This processing method enables the model to not only accurately identify a single type of defect, but also handle complex situations involving multiple defects. Therefore, the generalization ability of the model in complex scenarios is improved.

[0111] In step S1212, when the die-cutter defect corresponding to the training sample image is any composite of multiple annotation types, a second magnification ratio and a second reduction ratio of the training sample image are determined, and the training sample image is magnified and reduced according to the second magnification ratio and the second reduction ratio respectively, to obtain a first extended image of the die-cutter defect in the training sample image at different sizes;

[0112] The first enlargement ratio is greater than the second enlargement ratio, and the first reduction ratio is less than the second reduction ratio.

[0113] In the disclosed embodiment, when the die cutter defects in the training sample image include multiple types (such as wear and cracks), a more complex strategy is used to determine the magnification and reduction ratios. This is because different types of defects may show different characteristics at different scaling ratios. Therefore, a wider scaling range may be selected, or the scaling ratio may be determined separately according to the characteristics of each defect, and then a compromise value may be taken.

[0114] For example: suppose there is a training sample image in which both wear and cracks exist on the die cutter. To handle this situation, first consider the scaling requirements for each defect separately. For wear, a smaller magnification ratio (such as 1.2 times) and a larger reduction ratio (such as 0.8 times) may be selected; for cracks, a larger magnification ratio (such as 1.8 times) and a smaller reduction ratio (such as 0.6 times) may be selected. However, since these two defects exist in the same image at the same time, a second magnification ratio (such as 1.6 times) and a second reduction ratio (such as 0.75 times) may be selected in the end to ensure that both defects can be properly represented at different sizes. Then, the original image is enlarged and reduced according to these two ratios to obtain two new images as the first extended image.

[0115] The above technical solution can generate first extended images containing die cutter defects at different sizes by enlarging and reducing the training sample images at different ratios. These extended images increase the diversity of training data, allowing the model to learn the characteristic performance of defects at different scales during the training process. Therefore, when the model encounters defects of different sizes in actual applications, it can identify and classify them more accurately, thereby enhancing the robustness of the model.

[0116] By generating the first extended image, the size of the training dataset is actually increased. In the field of deep learning, a larger dataset usually means that the model can learn more feature information, thereby accelerating the training process and reducing the risk of overfitting. Therefore, it helps to obtain a die-cutter defect recognition model with better performance in a limited time. During the training process, the model will learn and optimize based on the defect features in the extended image. Since the extended image contains the manifestations of defects at different sizes, the model can learn more refined and accurate feature representations. This optimization process helps to improve the accuracy and efficiency of the model in identifying die-cutter defects, thereby optimizing the overall performance of the model.

[0117] In summary, by generating the first extended images containing die-cutting knife defects at different sizes, the robustness and generalization ability of the model are enhanced, the model training process is accelerated, and the model performance is optimized.

[0118] In a possible implementation, in step S11, performing mask sliding for each of the high-definition images multiple times through the image stitching model and then adding an attention value to each mask image obtained include:

[0119] In step S111, a mask for performing multiple mask sliding on each of the high-definition images is determined according to a size relationship between the resolution of each of the high-definition images and an average resolution of the high-definition images and a resolution difference;

[0120] In the disclosed embodiment, during the image stitching process, since the resolutions of different high-definition images may be different, masks of different sizes or shapes are required to adapt to images of different resolutions. The purpose of this step is to dynamically determine the appropriate mask size and shape based on the size relationship between the resolution of the high-definition image and the average resolution and the resolution difference. The mask should be designed to cover the key areas of the image while reducing the stitching error caused by the resolution difference.

[0121] For example, suppose there is a set of high-definition images, one of which has a resolution of 1920x1080, and the average resolution of all images is 1280x720. Based on the resolution difference (640x360), a relatively small mask can be determined, such as 640x360 or smaller (depending on the specific stitching strategy and requirements). This mask will be used in subsequent mask sliding operations to ensure that the key areas of the image are covered during the stitching process, while reducing the stitching artifacts caused by different resolutions.

[0122] In step S112, the mask corresponding to the high-definition image is used to slide the high-definition image for multiple times to obtain multiple mask images corresponding to the high-definition image;

[0123] In the disclosed embodiment, after determining a suitable mask, the mask is slid multiple times on the high-definition image. Each time it slides, the mask moves according to a preset step size (such as pixels or percentages), and the moved image area is processed (although the specific processing operation is not explicitly stated in this step, it usually includes extracting features, calculating similarities, etc.). Through multiple slides, multiple mask images can be obtained, which contain local information of the image at different positions, providing a basis for subsequent image stitching and attention value addition.

[0124] For example: Assume that a 640x360 mask is used, and the step size is set to 50% of the mask size (i.e. 320x180). Start the mask from the upper left corner of the HD image and slide it to the right according to the step size until it covers the entire width of the image. Then, move the mask to the next row (also according to the step size) and repeat the above process until the entire image is covered. Through this process, multiple mask images will be obtained, each of which corresponds to a local area in the original image.

[0125] In step S113, the vector corresponding to each mask image is multiplied by three different weight matrices corresponding to the focal length of the high-definition camera corresponding to the mask image to obtain a query vector, a key vector and a value vector corresponding to the mask image;

[0126] In the disclosed embodiments, query vector, key vector, and value vector are concepts used in attention mechanisms (such as the self-attention mechanism in the Transformer model). The query vector is used to match the key vector to determine which information in the value vector is important and calculate the attention weight accordingly.

[0127] In the disclosed embodiment, the vector corresponding to each mask image (which may be a feature vector or a representation vector) is converted into a query vector, a key vector, and a value vector through different weight matrices. These weight matrices are determined according to the focal length of the high-definition camera, because the focal length may affect the clarity and details of the image, thereby affecting the effect of the attention mechanism. By setting a different weight matrix for each focal length, it can better adapt to images taken at different focal lengths.

[0128] For example: suppose there is a high-definition camera with an adjustable focal length. For a high-definition image taken by the camera, a mask is slid and multiple mask images are obtained. For the vector corresponding to each mask image, the corresponding weight matrix is ​​selected from the preset weight matrix set according to the focal length of the camera when shooting. Then, this vector is multiplied by three different weight matrices (corresponding to query, key and value respectively) to obtain the query vector, key vector and value vector of the mask image.

[0129] In step S114, the attention value of each mask image is determined according to the image gradient of each high-definition image and the query vector, the key vector and the value vector corresponding to each mask image in the high-definition image.

[0130] In the disclosed embodiment, the image gradient is the rate and direction of brightness change of the high-definition image. It reflects the change of pixel values ​​in the image, and uses the image gradient as well as the query vector, key vector and value vector to calculate the attention value of each mask image. The image gradient can provide information about the change of pixel values ​​in the image, which helps to identify features such as edges and textures. The query vector, key vector and value vector evaluate the importance of different mask image areas through the attention mechanism. By combining the two, a more accurate attention value can be obtained.

[0131] For example: First, the image gradient of the HD image is calculated to obtain information about the change in pixel value. Then, for each mask image, its query vector and key vector are used to calculate the attention weight. This weight reflects the similarity between the query vector and the key vector, that is, the relevance of the mask image area to the current focus point. Next, this weight is used to weight the value vector to obtain a weighted value vector. Finally, the weighted value vector is aggregated (such as summed or averaged) to obtain the attention value of the mask image.

[0132] The above technical solution can dynamically determine the appropriate mask size and shape by considering the size relationship between the resolution of the high-definition image and the average resolution as well as the resolution difference. This adaptive mask design can ensure that when processing images of different resolutions, the mask can cover the key areas of the image while reducing processing errors caused by resolution differences. This helps to improve the accuracy of subsequent image stitching and feature extraction.

[0133] Then, by sliding the mask multiple times on the HD image, multiple mask images can be obtained, each of which corresponds to a local area in the original image. This multi-scale feature extraction method helps capture detailed information of the image at different scales.

[0134] Furthermore, by setting different weight matrices for high-definition cameras with different focal lengths and multiplying the vectors corresponding to the mask images by these weight matrices, the query vector, key vector and value vector are obtained. This focal-sensitive attention mechanism enables the model to more accurately evaluate the importance of different areas when processing images taken at different focal lengths, and adjust the model's focus accordingly. It helps to improve the recognition ability and robustness of the model in complex scenes. Combining the image gradient of the high-definition image and the query vector, key vector and value vector corresponding to the mask image, the attention value of each mask image is calculated. This image gradient-based attention value calculation method can make full use of feature information such as edges and textures in the image, further improving the accuracy and reliability of the attention value. Guided by the attention value, the model can pay more attention to the key areas in the image, thereby improving the efficiency and quality of image processing and recognition.

[0135] In summary, through the steps of adaptive mask design, multi-scale feature extraction, focus-sensitive attention mechanism and image gradient-based attention value calculation, efficient processing and accurate recognition of high-definition images are achieved.

[0136] In a possible implementation, in step S114, determining the attention value of each mask image according to the image gradient of each high-definition image and the query vector, the key vector, and the value vector corresponding to each mask image in the high-definition image includes:

[0137] The following steps are performed for each high-definition image until any mask image in the high-definition image is used as a query to perform the following steps:

[0138] In step S1141, according to the image gradient of the high-definition image, a dot product mask image of each mask image corresponding to the high-definition image is determined from the mask images included in the high-definition image along the direction in which the image gradient descends;

[0139] In the disclosed embodiment, the image gradient information of the high-definition image is used to guide the selection of the mask image. The image gradient describes the direction and rate of change of the pixel values ​​in the image, especially the edge area usually has a larger gradient value. By selecting the mask image along the direction of the image gradient descent, it is possible to focus on the areas in the image where the changes are most significant, and these areas often contain more important information. Therefore, the correlation between each mask image and the surrounding area in the image gradient is calculated to obtain a "dot product mask image", which is actually a weighted set of mask images that reflects the importance along the gradient direction.

[0140] For example: suppose there is a high-definition image, and multiple mask images have been generated by sliding windows or other methods. Now, select a mask image as a reference point, and find and determine other mask images in the gradient direction based on the image gradient at the image position. Specifically, the dot product (or similar correlation measure) of the reference mask image and the surrounding mask images in the gradient direction can be calculated to obtain a weighting coefficient. Then, the contribution of the surrounding mask images is adjusted according to this weighting coefficient, and finally a comprehensive dot product mask image is obtained.

[0141] In step S1142, taking any mask image in each of the high-definition images as a query, determining an attention score of the mask image as the query according to the query vector of the mask image and the key vector of the dot product mask image;

[0142] In the disclosed embodiment, the query vector and key vector in the attention mechanism are used to calculate the correlation between mask images, that is, the attention score. The query vector represents the current area or feature of interest, while the key vector represents other candidate areas or features. By calculating the dot product (or other similarity measure) of the query vector and each candidate key vector, a score can be obtained that reflects the correlation or importance between the query area and the candidate area.

[0143] For example, a mask image is used as a query, and its query vector is dot-producted with the key vector of each mask image in the dot-product mask image obtained in step S1141. The result of this operation is a numerical value, which represents the similarity or attention score between the query mask image and each candidate mask image. The higher the score, the more similar the two mask images are in terms of features, or the more the query mask image should pay attention to the candidate mask image.

[0144] In step S1143, based on the attention score of the mask image and the value vector of the mask image, a weighted sum is performed on the value vector of the mask image to obtain the attention value of the mask image.

[0145] In the disclosed embodiment, the value vector of the mask image is weighted and summed according to the attention score obtained in the previous step to obtain the attention value of the mask image. The attention value is a vector that integrates information of multiple candidate regions, which reflects the importance and contribution of the query mask image in the global or local context.

[0146] For example: For each mask image, its value vector is weighted according to its attention score obtained in step S1142. Specifically, the attention score is used as a weight, multiplied by the corresponding value vector, and then all weighted value vectors are added. The result of this weighted summation is the attention value of the mask image. This attention value not only contains the information of the mask image itself, but also integrates the information of other mask images related to it, thereby providing a more comprehensive and accurate feature representation.

[0147] The above technical solution determines the dot product mask image by using the image gradient information of the high-definition image. This method can capture the areas in the image where the pixel values ​​change most significantly, that is, features such as edges and textures. These areas often contain important information of the image. Therefore, by selecting the mask image along the direction of image gradient descent, the correlation between features can be enhanced, so that subsequent processing can focus more on the key parts of the image.

[0148] Furthermore, by taking each mask image as a query and calculating the attention score using its query vector and the key vector of the dot product mask image, this method can accurately evaluate the similarity and importance between different mask images. The level of attention score directly reflects the degree of association between the query mask image and the candidate mask image, thus providing a more accurate attention weight for subsequent processing.

[0149] Furthermore, the attention value is obtained by weighted summing up the value vector of the mask image according to its attention score. This process actually integrates and fuses the feature information of multiple candidate mask images. Since each mask image represents the features of the image at different positions or scales, the attention value obtained by weighted summing can contain more comprehensive and rich feature information, which helps to improve the performance of subsequent image processing and recognition.

[0150] Furthermore, by introducing the attention mechanism, computing resources can be allocated more efficiently when processing high-definition images. Specifically, mask images with high attention scores will receive more attention and processing, while mask images with low attention scores will be relatively ignored. This mechanism not only improves computational efficiency, but also enhances the system's robustness to noise and interference, because the attention mechanism can automatically suppress unimportant information, thereby reducing the impact on overall performance. In summary, by combining methods such as image gradient, attention mechanism and weighted summation, accurate extraction and fusion of key features in high-definition images are achieved, improving the performance of image processing and recognition.

[0151] In a possible implementation, in step S11, determining a mask for performing multiple mask sliding on each of the high-definition images according to a size relationship between a resolution of each of the high-definition images and an average resolution of the high-definition images and a resolution difference includes:

[0152] For any of the high-definition images, when the size relationship indicates that the resolution of the high-definition image is greater than the average resolution of the high-definition images, determining that the scaling strategy for the standard mask is reduction, and determining a reduction ratio according to a resolution difference between the resolution of the high-definition image and the average resolution of the high-definition images, and determining a mask obtained by reducing the standard mask according to the reduction ratio as a mask for performing multiple mask sliding on the high-definition image;

[0153] For any of the high-definition images, when the size relationship indicates that the resolution of the high-definition image is less than the average resolution of the high-definition images, determining that the scaling strategy for the standard mask is magnification, and determining the magnification ratio according to the resolution difference between the resolution of the high-definition image and the average resolution of the high-definition images, and determining the mask obtained by magnifying the standard mask according to the magnification ratio as the mask used for performing multiple mask sliding on the high-definition image;

[0154] For any of the high-definition images, when the size relationship indicates that the resolution of the high-definition image is equal to the average resolution of each of the high-definition images, the standard mask is determined as a mask for performing multiple mask sliding on the high-definition image.

[0155] In the disclosed embodiment, for each high-definition image, the size relationship between its resolution and the average resolution of all high-definition images is calculated. This size relationship determines the scaling strategy of the mask. If the resolution of the high-definition image is greater than the average resolution, it means that the image contains more detailed information, so a smaller mask is required to capture this information in detail. On the contrary, if the resolution of the high-definition image is less than the average resolution, a larger mask may be required to ensure sufficient coverage. If the resolution is exactly equal to the average resolution, the standard mask can be used directly.

[0156] For example, suppose there is a set of high-definition images, one of which has a resolution of 2048x1024, while the average resolution of all images is 1920x1080. Since the resolution of this image is greater than the average resolution, the scaling strategy for the standard mask is determined to be downscaling.

[0157] After the scaling strategy is determined, the specific scaling ratio is calculated based on the difference between the resolution of the HD image and the average resolution. This scaling ratio is used to adjust the size of the standard mask to ensure that the mask can fit the HD image currently being processed. The scaling ratio is usually calculated based on the width and height dimensions of the resolution, but other more complex strategies can also be used to maintain the aspect ratio of the mask or consider other factors.

[0158] Continuing with the example above, assume that the standard mask has a resolution of 512x256. To scale this mask down to fit an image with a resolution of 2048x1024, the scaling ratios for each dimension can be calculated. For the width, the scaling ratio is 2048 / 1920 (approximately 1.067), but to simplify the calculation and maintain integer bounds, a close smaller integer ratio (such as 1:1 or slightly smaller) might be chosen. For the height, the scaling ratio is similarly calculated as 1024 / 1080 (approximately 0.948), and a close smaller integer ratio might also be chosen. In practical applications, one might choose a scaling ratio that combines the effects of width and height, or adjust the scaling of the width and height separately to maintain the aspect ratio of the mask.

[0159] Finally, the standard mask is scaled according to the calculated scaling ratio to obtain a mask suitable for the current high-definition image. The scaling process can be implemented by scaling functions in the image processing library, which usually support multiple scaling algorithms, such as bilinear interpolation, bicubic interpolation, etc., to ensure the quality of the scaled mask.

[0160] For example: Assume that the width scaling ratio is 1 (i.e., no width scaling) and the height scaling ratio is 0.9 (i.e., 10% height reduction). Then, the width of the standard mask (512x256) will remain unchanged, while the height will be reduced to 230 (i.e., 256*0.9, rounded to the nearest integer). In this way, a mask (512x230) suitable for the current high-definition image is obtained, and this mask can be used for subsequent multiple sliding operations.

[0161] The present disclosure also provides a die cutter defect inspection device, see Figure 2 As shown, including:

[0162] The first input module 210 is configured to input each high-definition image of the surface of the die-cutting knife to be inspected, which is captured by multiple high-definition cameras located at different positions of the die-cutting knife to be inspected and have different focal lengths, into a pre-trained image stitching model, so as to add an attention value to each mask image obtained after performing multiple mask sliding on each of the high-definition images through the image stitching model, and stitching the multiple mask images in each of the high-definition images according to the attention values ​​to obtain a die-cutting knife image to be identified;

[0163] The acquisition and determination module 220 is configured to acquire the manually labeled type of the die-cutting knife defect in the die-cutting knife image to be identified, and determine the target die-cutting knife auxiliary identification image from the die-cutting knife auxiliary identification image according to the labeled type of the die-cutting knife defect in the die-cutting knife image to be identified, wherein the die-cutting knife auxiliary identification image is obtained after performing multiple scaling, pixel-level translation in multiple directions and rotation at multiple angles in sequence according to the training sample image for training the die-cutting knife defect recognition model, the training sample image and the die-cutting knife auxiliary identification image are used together as input sample images to train the die-cutting knife defect recognition model, and the labeled type and quantity of the die-cutting knife defect in the target die-cutting knife auxiliary identification image are respectively the same as the labeled type and quantity of the die-cutting knife defect in the die-cutting knife image to be identified;

[0164] The second input module 230 is configured to input the die-cutting knife image to be identified and the target die-cutting knife identification auxiliary image into the die-cutting knife defect recognition model to obtain the die-cutting knife defect inspection result output by the die-cutting knife defect recognition model, wherein the die-cutting knife defect recognition model fine-tunes the number of layers and the fine-tunes the learning rate settings through the target die-cutting knife identification auxiliary image before identifying the die-cutting knife image to be identified.

[0165] In a possible implementation, the second input module 230 is configured as follows:

[0166] The following steps are repeated until the loss value is minimized:

[0167] Inputting the target die-cutting knife recognition auxiliary image into the die-cutting knife defect recognition model, calculating the loss through a cross entropy loss function, and obtaining a loss value;

[0168] Determine the number of fine-tuning layers for fine-tuning the learning rate in the die-cutting knife defect recognition model according to the loss value, and determine the target fine-tuning residual layers in sequence from the residual layers close to the fully connected layer in the die-cutting knife defect recognition model according to the number of fine-tuning layers;

[0169] Determine the target fine-tuning learning rate for this cycle based on the difference between the loss value of this cycle and the loss value of the previous cycle.

[0170] According to the target fine-tuning learning rate, the learning rate of the target fine-tuning residual layer in the die-cutting knife defect recognition model is fine-tuned.

[0171] In a possible implementation, the acquisition and determination module 220 is configured to:

[0172] According to the number of the annotated types of the die-cutting knife defect corresponding to the training sample image, the training sample image is enlarged and reduced at multiple ratios to obtain a first extended image of the die-cutting knife defect in the training sample image at different sizes;

[0173] Determine the pixel-level translation offset of the training sample image according to the resolution of the high-definition camera, translate the training sample image in multiple directions according to the pixel-level translation offset, and obtain a translation sample image corresponding to each direction;

[0174] The translation sample images corresponding to each direction are rotated at multiple angles to obtain multiple die-cutting knife auxiliary recognition images.

[0175] In a possible implementation, the annotation type includes any one of wear, crack, notch, and deformation, and the acquisition and determination module 220 is configured to:

[0176] In a case where the die-cutter defect corresponding to the training sample image is of any single annotation type, determining a first magnification ratio and a first reduction ratio of the training sample image, and magnifying and reducing the training sample image according to the first magnification ratio and the first reduction ratio, respectively, to obtain a first extended image of the die-cutter defect in the training sample image at different sizes;

[0177] In the case where the die-cutter defect corresponding to the training sample image is any combination of multiple annotation types, determining a second magnification ratio and a second reduction ratio of the training sample image, and magnifying and reducing the training sample image according to the second magnification ratio and the second reduction ratio, respectively, to obtain a first extended image of the die-cutter defect in the training sample image at different sizes;

[0178] The first enlargement ratio is greater than the second enlargement ratio, and the first reduction ratio is less than the second reduction ratio.

[0179] In a possible implementation, the first input module 210 is configured as follows:

[0180] Determining a mask for performing multiple mask sliding on each of the high-definition images according to a size relationship between the resolution of each of the high-definition images and an average resolution of the high-definition images and a resolution difference;

[0181] The high-definition image is respectively subjected to multiple mask sliding by the mask corresponding to the high-definition image to obtain multiple mask images corresponding to the high-definition image;

[0182] Multiplying the vector corresponding to each of the mask images by three different weight matrices corresponding to the focal length of the high-definition camera corresponding to the mask image, respectively, to obtain a query vector, a key vector and a value vector corresponding to the mask image;

[0183] The attention value of each mask image is determined according to the image gradient of each high-definition image and the query vector, the key vector and the value vector corresponding to each mask image in the high-definition image.

[0184] In a possible implementation, the first input module 210 is configured as follows:

[0185] The following steps are performed for each high-definition image until any mask image in the high-definition image is used as a query to perform the following steps:

[0186] According to the image gradient of the high-definition image, along the direction in which the image gradient descends, determining, from the mask images included in the high-definition image, a dot product mask image of each mask image corresponding to the high-definition image;

[0187] Taking any of the mask images in each of the high-definition images as a query, determining an attention score of the mask image as the query according to a query vector of the mask image and a key vector of the dot product mask image;

[0188] According to the attention score of the mask image and the value vector of the mask image, a weighted sum is performed on the value vector of the mask image to obtain the attention value of the mask image.

[0189] In a possible implementation, the first input module 210 is configured as follows:

[0190] For any of the high-definition images, when the size relationship indicates that the resolution of the high-definition image is greater than the average resolution of the high-definition images, determining that the scaling strategy for the standard mask is reduction, and determining a reduction ratio according to a resolution difference between the resolution of the high-definition image and the average resolution of the high-definition images, and determining a mask obtained by reducing the standard mask according to the reduction ratio as a mask for performing multiple mask sliding on the high-definition image;

[0191] For any of the high-definition images, when the size relationship indicates that the resolution of the high-definition image is less than the average resolution of the high-definition images, determining that the scaling strategy for the standard mask is magnification, and determining the magnification ratio according to the resolution difference between the resolution of the high-definition image and the average resolution of the high-definition images, and determining the mask obtained by magnifying the standard mask according to the magnification ratio as the mask used for performing multiple mask sliding on the high-definition image;

[0192] For any of the high-definition images, when the size relationship indicates that the resolution of the high-definition image is equal to the average resolution of each of the high-definition images, the standard mask is determined as a mask for performing multiple mask sliding on the high-definition image.

[0193] The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method described in any one of the aforementioned embodiments are implemented.

[0194] The present disclosure also provides an electronic device, including:

[0195] a memory having a computer program stored thereon;

[0196] A processor is used to execute the computer program in the memory to implement the steps of the method in any one of the aforementioned embodiments.

[0197] Figure 3 The die-cutting knife defect inspection device 100 shown includes: a processor 1001 and a memory 1003. The processor 1001 and the memory 1003 are connected, such as through a bus 1002. Optionally, the die-cutting knife defect inspection device 100 may also include a communication component 1004, which can be used for data interaction between the device 100 and other devices, such as data transmission and / or data reception. It should be noted that in actual scheduling, the communication component 1004 is not limited to one, and the structure of the die-cutting knife defect inspection device 100 does not constitute a limitation on the embodiments of the present application.

[0198] Processor 1001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 1001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0199] The bus 1002 may include a path to transmit information between the above components. The bus 1002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 1002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0200] The memory 1003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium, other magnetic storage devices, or any other medium that can be used to carry or store program code and can be read by a computer, without limitation herein.

[0201] The memory 1003 is used to store program codes for executing the embodiments of the present disclosure, and the execution is controlled by the processor 1001. The processor 1001 is used to execute the program codes stored in the memory 1003 to implement the steps shown in the above die cutter defect inspection method embodiment.

[0202] The embodiment of the present disclosure also provides a computer-readable storage medium having program codes stored thereon. When the program codes are executed by a processor, the steps and corresponding contents of the aforementioned die-cutting knife defect inspection method embodiment can be implemented.

[0203] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments; within the technical concept of the present disclosure, various changes, modifications, substitutions and variations may be made to these embodiments, and these changes, modifications, substitutions and variations all fall within the protection scope of the present disclosure.

[0204] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction, and they should also be regarded as the contents disclosed in this disclosure. In order to avoid unnecessary repetition, this disclosure will not further describe various possible combinations. The technical scope of this application is not limited to the contents in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A die cutting knife defect inspection method, characterized in that: The method comprises: Inputting each high-definition image of the surface of the die-cutting knife to be inspected, which is captured by multiple high-definition cameras located at different positions of the die-cutting knife to be inspected and having different focal lengths, into a pre-trained image stitching model, so that each of the high-definition images is subjected to multiple mask sliding by the image stitching model, and attention values ​​are added to each mask image obtained, and stitching is performed according to the attention values ​​of the multiple mask images in each of the high-definition images to obtain a die-cutting knife image to be identified; Obtaining the manually labeled type of the die-cutting knife defect in the die-cutting knife image to be identified, and determining a target die-cutting knife auxiliary identification image from the die-cutting knife auxiliary identification image according to the labeled type of the die-cutting knife defect in the die-cutting knife image to be identified, wherein the die-cutting knife auxiliary identification image is obtained by sequentially performing multiple scaling, multiple-direction pixel-level translation, and multiple-angle rotation on the training sample image for training the die-cutting knife defect identification model, the training sample image and the die-cutting knife auxiliary identification image are used together as input sample images to train the die-cutting knife defect identification model, and the labeled type and quantity of the die-cutting knife defect in the target die-cutting knife auxiliary identification image are respectively the same as the labeled type and quantity of the die-cutting knife defect in the die-cutting knife image to be identified; The die-cutting knife image to be identified and the target die-cutting knife auxiliary identification image are input into the die-cutting knife defect recognition model to obtain the die-cutting knife defect inspection result output by the die-cutting knife defect recognition model, wherein the die-cutting knife defect recognition model fine-tunes the number of layers and the setting of the learning rate through the target die-cutting knife auxiliary identification image before identifying the die-cutting knife image to be identified; The step of fine-tuning the number of layers and the setting of the learning rate by using the target die-cutter to assist in recognizing the image comprises: The following steps are repeated until the loss value is minimized: Inputting the target die-cutting knife auxiliary recognition image into the die-cutting knife defect recognition model, calculating the loss through a cross entropy loss function, and obtaining a loss value; Determine the number of fine-tuning layers for fine-tuning the learning rate in the die-cutting knife defect recognition model according to the loss value, and determine the target fine-tuning residual layers in sequence from the residual layers close to the fully connected layer in the die-cutting knife defect recognition model according to the number of fine-tuning layers; Determine the target fine-tuning learning rate for this cycle based on the difference between the loss value of this cycle and the loss value of the previous cycle. According to the target fine-tuning learning rate, the learning rate of the target fine-tuning residual layer in the die-cutting knife defect recognition model is fine-tuned.

2. The die cutting knife defect inspection method according to claim 1, characterized in that: The die-cutting knife auxiliary recognition image is obtained by sequentially performing multiple scaling, multiple direction pixel-level translation and multiple angle rotation on the training sample image of the training die-cutting knife defect recognition model, including: According to the number of the annotated types of the die-cutting knife defect corresponding to the training sample image, the training sample image is enlarged and reduced at multiple ratios to obtain a first extended image of the die-cutting knife defect in the training sample image at different sizes; Determine the pixel-level translation offset of the training sample image according to the resolution of the high-definition camera, translate the training sample image in multiple directions according to the pixel-level translation offset, and obtain a translation sample image corresponding to each direction; The translation sample images corresponding to each direction are rotated at multiple angles to obtain multiple die-cutting knife auxiliary recognition images.

3. The die cutting knife defect inspection method according to claim 2, characterized in that: The annotation type includes one or more of wear, crack, notch, and deformation. The training sample image is enlarged and reduced in multiple proportions according to the number of annotation types of the die-cutting knife defect corresponding to the training sample image to obtain a first extended image of the die-cutting knife defect in the training sample image at different sizes, including: In a case where the die-cutter defect corresponding to the training sample image is of any single annotation type, determining a first magnification ratio and a first reduction ratio of the training sample image, and magnifying and reducing the training sample image according to the first magnification ratio and the first reduction ratio, respectively, to obtain a first extended image of the die-cutter defect in the training sample image at different sizes; In the case where the die-cutter defect corresponding to the training sample image is any combination of multiple annotation types, determining a second magnification ratio and a second reduction ratio of the training sample image, and magnifying and reducing the training sample image according to the second magnification ratio and the second reduction ratio, respectively, to obtain a first extended image of the die-cutter defect in the training sample image at different sizes; The first enlargement ratio is greater than the second enlargement ratio, and the first reduction ratio is less than the second reduction ratio.

4. The die cutter defect inspection method according to any one of claims 1 to 3, characterized in that: The method of performing mask sliding for multiple times on each of the high-definition images through the image stitching model and then adding an attention value to each mask image obtained includes: Determining a mask for performing multiple mask sliding on each of the high-definition images according to a size relationship between the resolution of each of the high-definition images and an average resolution of the high-definition images and a resolution difference; The high-definition image is respectively subjected to multiple mask sliding by the mask corresponding to the high-definition image to obtain multiple mask images corresponding to the high-definition image; Multiplying the vector corresponding to each of the mask images by three different weight matrices corresponding to the focal length of the high-definition camera corresponding to the mask image, respectively, to obtain a query vector, a key vector and a value vector corresponding to the mask image; The attention value of each mask image is determined according to the image gradient of each high-definition image and the query vector, the key vector and the value vector corresponding to each mask image in the high-definition image.

5. The die cutting knife defect inspection method according to claim 4, characterized in that: The determining the attention value of each mask image according to the image gradient of each high-definition image and the query vector, the key vector, and the value vector corresponding to each mask image in the high-definition image comprises: The following steps are performed for each high-definition image until any mask image in the high-definition image is used as a query to perform the following steps: According to the image gradient of the high-definition image, along the direction in which the image gradient descends, determining, from the mask images included in the high-definition image, a dot product mask image of each mask image corresponding to the high-definition image; Taking any of the mask images in each of the high-definition images as a query, determining an attention score of the mask image as the query according to a query vector of the mask image and a key vector of the dot product mask image; According to the attention score of the mask image and the value vector of the mask image, a weighted sum is performed on the value vector of the mask image to obtain the attention value of the mask image.

6. The die cutting knife defect inspection method according to claim 4, characterized in that: The determining of a mask for performing multiple mask sliding on each of the high-definition images according to a size relationship between the resolution of each of the high-definition images and an average resolution of the high-definition images and a resolution difference includes: For any of the high-definition images, when the size relationship indicates that the resolution of the high-definition image is greater than the average resolution of the high-definition images, determining that the scaling strategy for the standard mask is reduction, and determining a reduction ratio according to a resolution difference between the resolution of the high-definition image and the average resolution of the high-definition images, and determining a mask obtained by reducing the standard mask according to the reduction ratio as a mask for performing multiple mask sliding on the high-definition image; For any of the high-definition images, when the size relationship indicates that the resolution of the high-definition image is less than the average resolution of the high-definition images, determining that the scaling strategy for the standard mask is magnification, and determining the magnification ratio according to the resolution difference between the resolution of the high-definition image and the average resolution of the high-definition images, and determining the mask obtained by magnifying the standard mask according to the magnification ratio as the mask used for performing multiple mask sliding on the high-definition image; For any of the high-definition images, when the size relationship indicates that the resolution of the high-definition image is equal to the average resolution of each of the high-definition images, the standard mask is determined as a mask for performing multiple mask sliding on the high-definition image.

7. A die cutting knife defect inspection device, characterized in that: include: A first input module is configured to input each high-definition image of the surface of the die-cutting knife to be inspected, which is captured by multiple high-definition cameras located at different positions of the die-cutting knife to be inspected and have different focal lengths, into a pre-trained image stitching model, so as to add an attention value to each mask image obtained after performing multiple mask sliding on each of the high-definition images through the image stitching model, and stitching the multiple mask images in each of the high-definition images according to the attention values ​​to obtain a die-cutting knife image to be identified; an acquisition and determination module, configured to acquire the type of manual annotation of the die-cutting knife defect in the die-cutting knife image to be identified, and determine the target die-cutting knife auxiliary identification image from the die-cutting knife auxiliary identification image according to the type of annotation of the die-cutting knife defect in the die-cutting knife image to be identified, wherein the die-cutting knife auxiliary identification image is obtained after sequentially performing multiple scaling, pixel-level translation in multiple directions and rotation at multiple angles on the training sample image for training the die-cutting knife defect identification model, the training sample image and the die-cutting knife auxiliary identification image are used together as input sample images to train the die-cutting knife defect identification model, and the annotation type and quantity of the die-cutting knife defect in the target die-cutting knife auxiliary identification image are respectively the same as the annotation type and quantity of the die-cutting knife defect in the die-cutting knife image to be identified; A second input module is configured to input the die-cutting knife image to be identified and the target die-cutting knife auxiliary identification image into the die-cutting knife defect recognition model to obtain the die-cutting knife defect inspection result output by the die-cutting knife defect recognition model, wherein the die-cutting knife defect recognition model fine-tunes the number of layers and the fine-tunes the learning rate setting through the target die-cutting knife auxiliary identification image before identifying the die-cutting knife image to be identified; The second input module is configured to loop through the following steps until the loss value reaches a minimum: Inputting the target die-cutting knife auxiliary recognition image into the die-cutting knife defect recognition model, calculating the loss through a cross entropy loss function, and obtaining a loss value; Determine the number of fine-tuning layers for fine-tuning the learning rate in the die-cutting knife defect recognition model according to the loss value, and determine the target fine-tuning residual layers in sequence from the residual layers close to the fully connected layer in the die-cutting knife defect recognition model according to the number of fine-tuning layers; Determine the target fine-tuning learning rate for this cycle based on the difference between the loss value of this cycle and the loss value of the previous cycle. According to the target fine-tuning learning rate, the learning rate of the target fine-tuning residual layer in the die-cutting knife defect recognition model is fine-tuned.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

9. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 6.

Images