Diffusion model-based industrial vision occluded object recognition and repair system and method

Abstract

The application discloses an industrial visual occlusion object recognition and repair system and method based on a diffusion model, wherein the system comprises an image preprocessing module, a diffusion repair module and a recognition module; the image preprocessing module comprises a grayscale conversion unit, an image cropping unit and a pixel normalization unit, and is used for preprocessing an input industrial image containing occlusion; the diffusion repair module comprises a noise prediction unit, a mask-guided repair unit and a resampling unit, and is used for repairing an occlusion area of the preprocessed industrial image; the recognition module comprises a residual network classifier, and is used for performing object recognition and classification on the repaired complete industrial image to obtain an output result, wherein the class label in the output result at least comprises one of a bearing, a gear or a screw. Through the diffusion model with the mask guidance and resampling mechanism, high-quality and high-detail restoration and repair of the image are realized in the industrial visual occlusion scene, and the accuracy of subsequent object recognition and the overall stability of the system are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of visual processing technology, and more specifically, to an industrial visual occlusion object recognition and repair system and method based on a diffusion model. Background Technology

[0002] With the rapid development of industry and intelligent manufacturing, machine vision systems are playing an increasingly important role in industrial production, especially in automated inspection, product quality control, and robot guidance and localization. In these applications, vision systems need to quickly and accurately identify and locate target objects. However, in real industrial environments, due to factors such as equipment layout, lighting changes, object stacking, and robotic arm movements, the objects being inspected are often partially occluded, preventing the vision system from fully acquiring object features, thus affecting recognition accuracy and system stability.

[0003] Traditional visual recognition algorithms, such as feature-point matching algorithms (e.g., SIFT, ORB), template matching, or classification and detection models based on convolutional neural networks (CNNs), often perform poorly when faced with occlusion. This is because occlusion disrupts the structural integrity and feature continuity of an object, making it difficult for the model to infer overall features from local information. Especially in high-speed, high-precision industrial applications, occlusion has become one of the key bottlenecks restricting the performance improvement of machine vision systems. Summary of the Invention

[0004] The purpose of this invention is to provide an industrial visual occluded object recognition and repair system and method based on a diffusion model, which solves the problems of inconsistent repair structures, poor detail restoration, unstable training, and difficulty in meeting real-time requirements in the processing of occluded objects in industrial images by existing technologies.

[0005] The first aspect of this invention provides an industrial visual occlusion object recognition and repair system based on a diffusion model, comprising:

[0006] The image preprocessing module includes a grayscale conversion unit, an image cropping unit, and a pixel normalization unit, which are used to preprocess the input industrial images containing occlusions.

[0007] The diffusion repair module includes a noise prediction unit, a mask-guided repair unit, and a resampling unit, which are used to repair occluded areas in preprocessed industrial images.

[0008] The recognition module includes a residual network classifier for object recognition and classification of the restored complete industrial image to obtain output results, wherein the category labels in the output results include at least one of bearings, gears or screws.

[0009] In this solution, the grayscale conversion unit is used to convert a color industrial image into a grayscale image; the image cropping unit is used to extract a region of interest containing a target object from the grayscale image, the target object including one of a bearing, gear, or screw; and the pixel normalization unit is used to scale the pixel values ​​of the image from the original range to a preset standard range.

[0010] In this scheme, the diffusion repair module uses a U-Net network as the prediction network in the noise prediction unit to predict the added noise based on the current industrial image and time step information during the forward and backward processes of the diffusion model.

[0011] In this solution, the specific execution steps of the mask-guided repair unit include:

[0012] Provide a binary mask of the same size as the input industrial image, wherein the area with a mask value of a first preset value represents the occluded area to be repaired, and the area with a mask value of a second preset value represents the known area;

[0013] In the reverse denoising process, the original image is initially estimated based on the current industrial image and the predicted noise;

[0014] The denoising result of the current step is divided into a known part and a part to be repaired. The known part is obtained by multiplying the binary mask with the noisy known region data, and the part to be repaired is obtained by multiplying the intermediate value calculated by the binary mask with the data reconstructed based on the mean and standard deviation calculated based on the predicted noise.

[0015] The known portion is added to the portion to be repaired, and the two are combined into the noise reduction input for the next step.

[0016] In this scheme, the resampling unit refers to sampling and fusing the part to be repaired a preset number of times every preset number of steps during the reverse denoising process.

[0017] In this scheme, the recognition module is trained in two ways: training it separately and training it jointly with the diffusion repair module in an end-to-end manner. During the joint training process, a public dataset is used in conjunction with occlusion-complete image pairs synthesized from industrial part images to train the diffusion repair module.

[0018] A second aspect of the present invention provides an industrial visual occluded object recognition and repair method based on a diffusion model, applicable to the industrial visual occluded object recognition and repair system based on a diffusion model described in any of the preceding claims, wherein the method includes the following steps:

[0019] Preprocess the input industrial images containing occlusions;

[0020] Repairing occluded areas in preprocessed industrial images;

[0021] The restored complete industrial image is subjected to object recognition and classification to obtain the output results, wherein the category labels in the output results include at least one of bearings, gears or screws.

[0022] In this scheme, noise prediction is included in the occlusion repair of the preprocessed industrial image. Specifically, the U-Net network is used as the prediction network to predict the added noise based on the current industrial image and time step information during the forward and backward processes of the diffusion model.

[0023] In this solution, the occlusion repair process for preprocessed industrial images includes mask-guided repair, and the specific process is as follows:

[0024] Provide a binary mask of the same size as the input industrial image, wherein the area with a mask value of a first preset value represents the occluded area to be repaired, and the area with a mask value of a second preset value represents the known area;

[0025] In the reverse denoising process, the original image is initially estimated based on the current industrial image and the predicted noise;

[0026] The denoising result of the current step is divided into a known part and a part to be repaired. The known part is obtained by multiplying the binary mask with the noisy known region data, and the part to be repaired is obtained by multiplying the intermediate value calculated by the binary mask with the data reconstructed based on the mean and standard deviation calculated based on the predicted noise.

[0027] The known portion is added to the portion to be repaired, and the two are combined into the noise reduction input for the next step.

[0028] A third aspect of the present invention provides a computer-readable storage medium comprising a machine program for an industrial visual occluded object recognition and repair method based on a diffusion model, wherein when the diffusion model-based industrial visual occluded object recognition and repair method program is executed by a processor, it implements the steps of the diffusion model-based industrial visual occluded object recognition and repair method as described in any of the preceding claims.

[0029] This invention discloses an industrial visual occluded object recognition and restoration system and method based on a diffusion model. By introducing a diffusion model with mask guidance and resampling mechanisms, it achieves high-quality, high-detail restoration of images in industrial visual occlusion scenarios, significantly improving the accuracy of subsequent object recognition and the overall stability of the system. Specific beneficial effects are as follows:

[0030] 1. The restoration quality is high and the detail restoration capability is strong. Thanks to the powerful generation capability of the diffusion model and the mask guidance mechanism introduced in this invention, the restored image is highly consistent with the real image in terms of structure and texture, especially in the details such as the edges and holes of industrial parts.

[0031] 2. Excellent structural consistency: Through resampling mechanism and mask fusion strategy, the repaired area and the known area transition naturally at the boundary, without obvious artifacts or distortions;

[0032] 3. Stable training and strong adaptability: The diffusion model training process is stable and not prone to mode collapse, and it can adapt to various industrial parts and occlusion conditions.

[0033] 4. Significantly improved recognition accuracy: Even when the repaired image is input into the ResNet network, it can still maintain a high recognition rate under occlusion conditions. Experiments show that the recognition accuracy is improved by more than 15% on AR datasets.

[0034] 5. Highly scalable: This method is not only applicable to human faces or natural images, but can also be widely used in various industrial vision scenarios such as industrial parts, electronic components, and textiles. Attached Figure Description

[0035] Figure 1 A block diagram of an industrial visual occlusion object recognition and repair system based on a diffusion model is shown in the present invention.

[0036] Figure 2 The flowchart of an industrial visual occlusion object recognition and repair method based on a diffusion model according to the present invention is shown. Detailed Implementation

[0037] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0038] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0039] Currently, various technical solutions have been proposed to address the problem of visual recognition of occluded objects, mainly including the following categories:

[0040] 1. Image inpainting methods based on Generative Adversarial Networks (GANs) utilize adversarial training between a generator and a discriminator to generate image content that closely resembles the distribution of real images. GANs are widely used in image inpainting tasks to fill in missing regions. For example, Context-AwareGAN, by introducing a context-aware module, considers global semantic information during the inpainting process, improving the naturalness of the results. Furthermore, methods such as CycleGAN and StyleGAN have demonstrated strong capabilities in style transfer and image generation tasks, and can be used to generate plausible content for occluded areas.

[0041] However, GAN models suffer from problems during training, such as pattern collapse, training instability, and uncontrollable generated results. Especially in industrial vision, where the requirements for image detail reproduction and structural consistency are extremely high, GAN-generated images often suffer from texture blurring and structural distortion, making it difficult to meet the accuracy requirements of industrial applications.

[0042] 2. Generative models based on Variational Autoencoders (VAEs) and Normalizing Flow (NFs) employ a VAE model, which maps the input image to the latent space through an encoder and then reconstructs the image through a decoder. This approach offers a degree of controllability in image generation tasks. Normalizing Flow models, on the other hand, model the image distribution through a series of invertible transformations, theoretically allowing for precise calculation of likelihood probabilities. However, images generated by VAEs are often overly smooth and lack detail; while NF models have high computational complexity and struggle to handle high-resolution images, limiting their practical application in industrial vision.

[0043] 3. Transformer-based attention mechanisms: In recent years, the Transformer structure has also been applied in image processing, capturing long-range dependencies in images through a self-attention mechanism. For example, some studies have proposed using Transformers for image inpainting, guiding the generation of content in missing regions through masking positional encoding. However, Transformers have high computational resource requirements and are not as flexible as convolutional structures when processing local details, especially in detailed regions of industrial images, where their inpainting effect is limited.

[0044] 4. Image generation methods based on diffusion models: Diffusion models are a type of generative model that has emerged in recent years. They generate images step by step through a process of forward noise addition and backward denoising, offering advantages such as high generation quality and good diversity. DDPM (Denoising Diffusion Probabilistic Models), as a typical representative, has demonstrated superior performance in tasks such as image generation, super-resolution, and image inpainting. For example, the RePaint method utilizes diffusion models for image inpainting, improving the inpainting quality through multiple sampling and resampling strategies.

[0045] Nevertheless, existing diffusion-based methods are mostly focused on natural image inpainting and have not yet been systematically applied to occluded object recognition tasks in industrial vision, especially in industrial scenarios with extremely high requirements for structural consistency, detail restoration, and real-time performance. Many problems still need to be solved, among which the main drawbacks of existing technologies are:

[0046] 1. Inconsistent generated image structure: When repairing occluded areas, models such as GAN often have difficulty maintaining consistency with the surrounding image structure, resulting in distorted object contours or unnatural textures after repair, which affects the accuracy of subsequent recognition.

[0047] 2. Insufficient ability to restore details: Industrial vision has extremely high requirements for details such as the edges of parts, holes, and markings. Existing methods do not perform well in restoring details, especially the repair effect of small-scale features.

[0048] 3. Unstable training and monotonous patterns: GAN models are prone to gradient vanishing or pattern collapse during training, resulting in monotonous repair results that cannot adapt to the diverse object shapes and occlusion situations in industry.

[0049] 4. High computational complexity, making it difficult to apply in real time: Although the diffusion model generates high-quality data, its iterative sampling process is time-consuming, making it difficult to meet the real-time requirements of industrial vision.

[0050] 5. Unnatural handling of occluded area boundaries: Existing methods are prone to artifacts or blurring when repairing boundary areas, resulting in an unnatural transition between the repaired area and the known area.

[0051] Therefore, to address the shortcomings of the existing technologies, this invention proposes an industrial visual occluded object recognition and restoration system and method based on a diffusion model, aiming to achieve the following objectives: high-quality restoration of object images in occluded industrial images, ensuring that the restored image is consistent with the real object in structure and texture; improved detail restoration capability of the restored image, especially achieving higher accuracy in the restoration of key industrial features such as edges, holes, and markings; improved restoration efficiency by refining the sampling and restoration mechanism of the diffusion model, making it suitable for industrial scenarios with certain real-time requirements; a restoration process integrating mask guidance and resampling mechanisms to improve the naturalness of the fusion between the restored area and the known area; and finally, inputting the restored image into a recognition network (such as ResNet) to achieve high-precision occluded object recognition.

[0052] Specifically, Figure 1 A block diagram of an industrial visual occlusion object recognition and repair system based on a diffusion model is shown in this application.

[0053] like Figure 1 As shown, this application discloses an industrial visual occlusion object recognition and repair system based on a diffusion model, comprising:

[0054] The image preprocessing module includes a grayscale conversion unit, an image cropping unit, and a pixel normalization unit, which are used to preprocess the input industrial images containing occlusions.

[0055] The diffusion repair module includes a noise prediction unit, a mask-guided repair unit, and a resampling unit, which are used to repair occluded areas in preprocessed industrial images.

[0056] The recognition module includes a residual network classifier for object recognition and classification of the restored complete industrial image to obtain output results, wherein the category labels in the output results include at least one of bearings, gears or screws.

[0057] It should be noted that, in this embodiment, the system includes three core modules: an image preprocessing module, a diffusion repair module, and a recognition module. In the image preprocessing stage, the system first converts the input color industrial image into a grayscale image to reduce computational complexity and highlight structural features. Subsequently, the image is cropped and normalized to adapt it to the model input size. In the diffusion repair stage, the core of this invention lies in the improvement of DDPM (Denoising Diffusion Probabilistic Models) to make it suitable for occlusion repair tasks in industrial vision. DDPM includes two stages: a forward process and a backward process. This invention uses a unified U-Net structure as the noise prediction network in both stages.

[0058] Specifically, the forward pass is a process of gradually adding noise, assuming the original industrial image is... At every step (From 1 to T), based on preset noise scheduling parameters Add Gaussian noise to the image Obtain a noisy image Its mathematical expression is:

[0059] ;

[0060] in, , , Using standard Gaussian noise, this process aims to gradually transform the image into pure noise while preserving its underlying structural information.

[0061] During the forward pass, the U-Net network receives noisy images. With time step Predict the added noise The network is trained by minimizing the mean square error between the predicted noise and the actual noise. .

[0062] The reverse process is the gradual recovery of an image from noise. Traditional DDPM directly denoises the noise predicted by U-Net to generate an image during the reverse process. This invention introduces a mask-guided restoration mechanism on this basis, with the specific steps as follows:

[0063] Assumption A binary mask of the same size as the image, where "0" represents the occluded area (to be repaired) and "1" represents the known area;

[0064] At every step from generate First, the original image is estimated using the following formula:

[0065] ;

[0066] in, Noise predicted by U-Net;

[0067] Next, the known parts and the repaired parts are calculated separately:

[0068] ;

[0069] ;

[0070] in, for Noisy images at any given time for The known part of the time, for The unknown part of time, This represents point-by-point multiplication. and These are the mean and standard deviation, calculated from the predicted noise and scheduling parameters, respectively. It is standard Gaussian noise.

[0071] Finally, the two parts were merged: .

[0072] According to an embodiment of the present invention, the grayscale conversion unit is used to convert a color industrial image into a grayscale image; the image cropping unit is used to extract a region of interest containing a target object from the grayscale image, the target object including one of a bearing, a gear, or a screw; the pixel normalization unit is used to scale the pixel values ​​of the image from the original range to a preset standard range.

[0073] It should be noted that, in this embodiment, grayscale conversion transforms a color industrial RGB image into a grayscale image. Its main purpose is to reduce data dimensionality and computational complexity. Furthermore, since many industrial inspection tasks rely more on texture and shape features than color information, grayscale conversion can highlight these structural features. Image cropping involves extracting the region of interest containing the target object from the original image. This eliminates interference from irrelevant backgrounds and fixes the image size to the input size required by the model, reducing unnecessary computation. Correspondingly, pixel normalization scales the pixel values ​​of the image from the original range (e.g., 0-255) to a preset standard range (e.g., [-1, 1] or [0, 1]). This step helps improve the training stability of the model, accelerates convergence, and makes it more adaptable to images under different lighting conditions.

[0074] According to an embodiment of the present invention, the diffusion repair module uses a U-Net network as the prediction network in the noise prediction unit to predict the added noise based on the current industrial image and time step information during the forward and backward processes of the diffusion model.

[0075] It should be noted that, in this embodiment, during the forward and backward processes of the diffusion model, the encoder-decoder structure is responsible for predicting the added noise based on the current noisy image and time step information. Its powerful encoder-decoder structure can capture the global semantics and local details of the image, which is the basis for high-quality generation and restoration.

[0076] According to an embodiment of the present invention, the specific execution steps of the masked boot repair unit include:

[0077] Provide a binary mask of the same size as the input industrial image, wherein the area with a mask value of a first preset value represents the occluded area to be repaired, and the area with a mask value of a second preset value represents the known area;

[0078] In the reverse denoising process, the original image is initially estimated based on the current industrial image and the predicted noise;

[0079] The denoising result of the current step is divided into a known part and a part to be repaired. The known part is obtained by multiplying the binary mask with the noisy known region data, and the part to be repaired is obtained by multiplying the intermediate value calculated by the binary mask with the data reconstructed based on the mean and standard deviation calculated based on the predicted noise.

[0080] The known portion is added to the portion to be repaired, and the two are combined into the noise reduction input for the next step.

[0081] It should be noted that, in this embodiment, mask-guided restoration is key to solving the occlusion problem. This invention uses a binary mask to explicitly inform the model which parts of the image are known and need to be preserved, and which parts are occluded and need to be generated. In each step of the reverse denoising process, it forces the pixel values ​​of known areas to approach the original clean data, while only creatively generating data for occluded areas. This ensures that the restored image maintains a high degree of structural consistency with the real object. Specifically, iterative denoising, for each step... (From T to 1), the specific steps have been described in the above steps, and will not be repeated in this embodiment.

[0082] According to an embodiment of the present invention, the resampling unit refers to sampling and fusing the part to be repaired a preset number of times every preset number of steps during the reverse denoising process.

[0083] It should be noted that, in this embodiment, in order to further improve the naturalness of the fusion between the repaired area and the known area, the present invention introduces a resampling mechanism, that is, in certain steps (such as every j steps), the repaired area is sampled and fused multiple times to optimize the consistency of details.

[0084] According to an embodiment of the present invention, the training method of the recognition module includes individual training and joint training with the diffusion repair module in an end-to-end manner. In the joint training process, the diffusion repair module is trained using a public dataset and combined with occlusion-complete image pairs synthesized from industrial part images.

[0085] It should be noted that, in this embodiment, during the recognition stage, the repaired image is input into a pre-trained ResNet classification network for recognition. ResNet alleviates the gradient vanishing problem of deep networks through its residual structure, making it suitable for high-precision classification of industrial parts, products, etc. Furthermore, this invention employs an end-to-end approach during training, jointly optimizing the repair module and the recognition module to further improve recognition accuracy.

[0086] Figure 2 The flowchart of an industrial visual occlusion object recognition and repair method based on a diffusion model according to this application is shown.

[0087] like Figure 2 As shown, this application discloses an industrial visual occluded object recognition and repair method based on a diffusion model, applicable to any of the industrial visual occluded object recognition and repair systems based on a diffusion model, wherein the method includes the following steps:

[0088] S202, preprocess the input industrial image containing occlusion;

[0089] S204, Repairing occluded areas in the preprocessed industrial image;

[0090] S206, Perform object recognition and classification on the restored complete industrial image to obtain the output results.

[0091] It should be noted that, in this embodiment, the input occluded industrial image is preprocessed by the image preprocessing module, the occluded area is repaired by the diffusion repair module, and the object recognition and classification are performed on the repaired complete industrial image by the recognition module to obtain the output result. In the image preprocessing stage, the system first converts the input color industrial image into a grayscale image to reduce computational complexity and highlight structural features. Subsequently, the image is cropped and normalized to adapt it to the model input size. In the diffusion repair stage, the core of this invention lies in the improvement of DDPM (Denoising Diffusion Probabilistic Models) to make it suitable for occlusion repair tasks in industrial vision. DDPM includes two stages: a forward process and a backward process. This invention uses a unified U-Net structure as the noise prediction network in both stages.

[0092] Specifically, the forward pass is a process of gradually adding noise, assuming the original industrial image is... At every step (From 1 to T), based on preset noise scheduling parameters Add Gaussian noise to the image Obtain a noisy image Its mathematical expression is:

[0093] ;

[0094] in, , , Using standard Gaussian noise, this process aims to gradually transform the image into pure noise while preserving its underlying structural information.

[0095] During the forward pass, the U-Net network receives noisy images. With time step Predict the added noise The network is trained by minimizing the mean square error between the predicted noise and the actual noise. .

[0096] The reverse process is the gradual recovery of an image from noise. Traditional DDPM directly denoises the noise predicted by U-Net to generate an image during the reverse process. This invention introduces a mask-guided restoration mechanism on this basis, with the specific steps as follows:

[0097] Assumption A binary mask of the same size as the image, where "0" represents the occluded area (to be repaired) and "1" represents the known area;

[0098] At every step from generate First, the original image is estimated using the following formula:

[0099] ;

[0100] in, Noise predicted by U-Net;

[0101] Next, the known parts and the repaired parts are calculated separately:

[0102] ;

[0103] ;

[0104] in, for Noisy images at any given time for The known part of the time, for The unknown part of time, This represents point-by-point multiplication. and These are the mean and standard deviation, calculated from the predicted noise and scheduling parameters, respectively. It is standard Gaussian noise.

[0105] Finally, the two parts were merged: .

[0106] According to an embodiment of the present invention, the grayscale conversion unit is used to convert a color industrial image into a grayscale image; the image cropping unit is used to extract a region of interest containing a target object from the grayscale image, the target object including one of a bearing, a gear, or a screw; the pixel normalization unit is used to scale the pixel values ​​of the image from the original range to a preset standard range.

[0107] It should be noted that, in this embodiment, grayscale conversion transforms a color industrial RGB image into a grayscale image. Its main purpose is to reduce data dimensionality and computational complexity. Furthermore, since many industrial inspection tasks rely more on texture and shape features than color information, grayscale conversion can highlight these structural features. Image cropping involves extracting the region of interest containing the target object from the original image. This eliminates interference from irrelevant backgrounds and fixes the image size to the input size required by the model, reducing unnecessary computation. Correspondingly, pixel normalization scales the pixel values ​​of the image from the original range (e.g., 0-255) to a preset standard range (e.g., [-1, 1] or [0, 1]). This step helps improve the training stability of the model, accelerates convergence, and makes it more adaptable to images under different lighting conditions.

[0108] According to an embodiment of the present invention, the diffusion repair module uses a U-Net network as the prediction network in the noise prediction unit to predict the added noise based on the current industrial image and time step information during the forward and backward processes of the diffusion model.

[0109] It should be noted that, in this embodiment, during the forward and backward processes of the diffusion model, the encoder-decoder structure is responsible for predicting the added noise based on the current noisy image and time step information. Its powerful encoder-decoder structure can capture the global semantics and local details of the image, which is the basis for high-quality generation and restoration.

[0110] According to an embodiment of the present invention, the specific execution steps of the masked boot repair unit include:

[0111] Provide a binary mask of the same size as the input industrial image, wherein the area with a mask value of a first preset value represents the occluded area to be repaired, and the area with a mask value of a second preset value represents the known area;

[0112] In the reverse denoising process, the original image is initially estimated based on the current industrial image and the predicted noise;

[0113] The denoising result of the current step is divided into a known part and a part to be repaired. The known part is obtained by multiplying the binary mask with the noisy known region data, and the part to be repaired is obtained by multiplying the intermediate value calculated by the binary mask with the data reconstructed based on the mean and standard deviation calculated based on the predicted noise.

[0114] The known portion is added to the portion to be repaired, and the two are combined into the noise reduction input for the next step.

[0115] It should be noted that, in this embodiment, mask-guided restoration is key to solving the occlusion problem. This invention uses a binary mask to explicitly inform the model which parts of the image are known and need to be preserved, and which parts are occluded and need to be generated. In each step of the reverse denoising process, it forces the pixel values ​​of known areas to approach the original clean data, while only creatively generating data for occluded areas. This ensures that the restored image maintains a high degree of structural consistency with the real object. Specifically, iterative denoising, for each step... (From T to 1), the specific steps have been described in the above steps, and will not be repeated in this embodiment.

[0116] According to an embodiment of the present invention, the resampling unit refers to sampling and fusing the part to be repaired a preset number of times every preset number of steps during the reverse denoising process.

[0117] It should be noted that, in this embodiment, in order to further improve the naturalness of the fusion between the repaired area and the known area, the present invention introduces a resampling mechanism, that is, in certain steps (such as every j steps), the repaired area is sampled and fused multiple times to optimize the consistency of details.

[0118] According to an embodiment of the present invention, the training method of the recognition module includes individual training and joint training with the diffusion repair module in an end-to-end manner. In the joint training process, the diffusion repair module is trained using a public dataset and combined with occlusion-complete image pairs synthesized from industrial part images.

[0119] It should be noted that, in this embodiment, during the recognition stage, the repaired image is input into a pre-trained ResNet classification network for recognition. ResNet alleviates the gradient vanishing problem of deep networks through its residual structure, making it suitable for high-precision classification of industrial parts, products, etc. Furthermore, this invention employs an end-to-end approach during training, jointly optimizing the repair module and the recognition module to further improve recognition accuracy.

[0120] A third aspect of the present invention provides a computer-readable storage medium comprising a program for an industrial visual occluded object recognition and repair method based on a diffusion model. When executed by a processor, the program implements the steps of the industrial visual occluded object recognition and repair method based on a diffusion model as described in any of the preceding claims.

[0121] This invention discloses an industrial visual occlusion object recognition and repair system and method based on a diffusion model. By introducing a diffusion model with mask guidance and resampling mechanism, high-quality, high-detail restoration of images is achieved in industrial visual occlusion scenarios, significantly improving the accuracy of subsequent object recognition and the overall stability of the system.

[0122] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0123] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0124] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0125] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. An industrial visual occluded object recognition and repair system based on a diffusion model, characterized in that, The system includes: The image preprocessing module includes a grayscale conversion unit, an image cropping unit, and a pixel normalization unit, which are used to preprocess the input industrial images containing occlusions. The diffusion repair module includes a noise prediction unit, a mask-guided repair unit, and a resampling unit, which are used to repair occluded areas in preprocessed industrial images. The recognition module includes a residual network classifier for object recognition and classification of the restored complete industrial image to obtain output results, wherein the category labels in the output results include at least one of bearings, gears or screws.

2. The industrial visual occlusion object recognition and repair system based on a diffusion model according to claim 1, characterized in that, The grayscale conversion unit is used to convert a color industrial image into a grayscale image; the image cropping unit is used to extract a region of interest containing a target object from the grayscale image, the target object including one of a bearing, gear, or screw; the pixel normalization unit is used to scale the pixel values ​​of the image from the original range to a preset standard range.

3. The industrial visual occlusion object recognition and repair system based on a diffusion model according to claim 1, characterized in that, The diffusion repair module uses a U-Net network as the prediction network in the noise prediction unit to predict the added noise based on the current industrial image and time step information during the forward and backward processes of the diffusion model.

4. The industrial visual occlusion object recognition and repair system based on a diffusion model according to claim 3, characterized in that, The specific execution steps of the mask-guided repair unit include: Provide a binary mask of the same size as the input industrial image, wherein the area with a mask value of a first preset value represents the occluded area to be repaired, and the area with a mask value of a second preset value represents the known area; In the reverse denoising process, the original image is initially estimated based on the current industrial image and the predicted noise; The denoising result of the current step is divided into a known part and a part to be repaired. The known part is obtained by multiplying the binary mask with the noisy known region data, and the part to be repaired is obtained by multiplying the intermediate value calculated by the binary mask with the data reconstructed based on the mean and standard deviation calculated based on the predicted noise. The known portion is added to the portion to be repaired, and the two are combined into the noise reduction input for the next step.

5. The industrial visual occlusion object recognition and repair system based on a diffusion model according to claim 1, characterized in that, The resampling unit refers to the sampling and fusion of the part to be repaired a preset number of times every preset number of steps during the reverse denoising process.

6. The industrial visual occlusion object recognition and repair system based on a diffusion model according to claim 1, characterized in that, The recognition module is trained in two ways: training it separately and training it jointly with the diffusion repair module in an end-to-end manner. In the joint training process, the diffusion repair module is trained using a public dataset and occlusion-complete image pairs synthesized from industrial part images.

7. A method for industrial visual occlusion object recognition and repair based on a diffusion model, characterized in that, The industrial visual occlusion object recognition and repair system based on the diffusion model, as described in any one of claims 1-6, comprises the following steps: Preprocess the input industrial images containing occlusions; Repairing occluded areas in preprocessed industrial images; The restored complete industrial image is subjected to object recognition and classification to obtain the output results, wherein the category labels in the output results include at least one of bearings, gears or screws.

8. The industrial visual occlusion object recognition and repair method based on a diffusion model according to claim 7, characterized in that, Noise prediction is included in the occlusion restoration of preprocessed industrial images. Specifically, the U-Net network is used as the prediction network to predict the added noise based on the current industrial image and time step information during the forward and backward processes of the diffusion model.

9. The industrial visual occlusion object recognition and repair method based on a diffusion model according to claim 8, characterized in that, The occlusion restoration process for preprocessed industrial images includes mask-guided restoration, and the specific steps are as follows: Provide a binary mask of the same size as the input industrial image, wherein the area with a mask value of a first preset value represents the occluded area to be repaired, and the area with a mask value of a second preset value represents the known area; In the reverse denoising process, the original image is initially estimated based on the current industrial image and the predicted noise; The denoising result of the current step is divided into a known part and a part to be repaired. The known part is obtained by multiplying the binary mask with the noisy known region data, and the part to be repaired is obtained by multiplying the intermediate value calculated by the binary mask with the data reconstructed based on the mean and standard deviation calculated based on the predicted noise. The known portion is added to the portion to be repaired, and the two are combined into the noise reduction input for the next step.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a program for an industrial visual occlusion object recognition and repair method based on a diffusion model. When the program is executed by a processor, it implements the steps of the industrial visual occlusion object recognition and repair method based on a diffusion model as described in any one of claims 7 to 9.

Images