Model training method, oil mist removal method, device, equipment and storage medium

Abstract

The present disclosure provides a model training method, an oil mist removal method, a device, equipment and a storage medium. The method comprises: inputting a first sample image into a to-be-trained noise prediction model to obtain predicted noise; determining a first loss value based on the gap information between the predicted noise and the labeled noise; adjusting the parameters of the to-be-trained noise prediction model based on the first loss value, and obtaining a target noise prediction model under the condition that the to-be-trained noise prediction model meets a first convergence condition, the target noise prediction model being used to remove oil mist in a to-be-processed image to obtain an oil mist-removed image, the to-be-processed image being an image with oil mist obtained by image acquisition of a spinning box by an inspection robot. According to the target noise prediction model trained in the scheme of the present disclosure, the oil mist as noise can be accurately predicted, so as to accurately remove the oil mist from the image collected by the inspection robot in the spinning workshop.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision technology, and more particularly to the fields of artificial intelligence and image processing technology. Background Technology

[0002] To address potential anomalies in the spinning box during the spinning workshop production process, inspection robots can be used to check the spinning box. However, due to the high ambient temperature, the inspection robot's camera may become covered in oil mist, resulting in unclear photos and hindering subsequent monitoring of the spinning box's working status. Summary of the Invention

[0003] This disclosure provides a model training method, an oil mist removal method, an apparatus, a device, and a storage medium to solve or alleviate one or more technical problems in the prior art.

[0004] Firstly, this disclosure provides a model training method, including:

[0005] The first sample image is input into the noise prediction model to be trained to obtain the predicted noise. The noise prediction model to be trained is constructed based on the target oil mist distribution model. The noise prediction model to be trained is used to construct the reverse process of the diffusion model. The target oil mist distribution model learns the data distribution characteristics of the second sample image set. The second sample image set includes multiple second sample images with different oil mist contents. The second sample images are obtained by the inspection robot from image acquisition of the spinning box.

[0006] The first loss value is determined based on the difference between the predicted noise and the labeled noise.

[0007] The parameters of the noise prediction model to be trained are adjusted based on the first loss value. When the noise prediction model to be trained meets the first convergence condition, the target noise prediction model is obtained. The target noise prediction model is used to remove oil mist from the image to be processed to obtain an oil mist-free image. The image to be processed is an image with oil mist obtained by the inspection robot from the image acquisition of the spinning box.

[0008] Secondly, this disclosure provides an oil mist removal method, applied to a target noise prediction model trained as described in the first aspect, comprising:

[0009] Images of the spinning box in its working state are acquired to obtain the image to be processed;

[0010] Repeat the following steps until the oil fog content of the new image to be processed is lower than a preset threshold, in order to obtain a defogging image:

[0011] The image to be processed is input into the target noise prediction model to obtain the predicted noise;

[0012] Remove the predicted noise from the image to be processed to obtain a new image to be processed;

[0013] Evaluate the oil mist content of the new image to be processed.

[0014] Thirdly, this disclosure provides a model training apparatus, including:

[0015] The input module is used to input the first sample image into the noise prediction model to be trained to obtain the predicted noise. The noise prediction model to be trained is constructed based on the target oil mist distribution model. The noise prediction model to be trained is used to construct the reverse process of the diffusion model. The target oil mist distribution model learns the data distribution characteristics of the second sample image set. The second sample image set includes multiple second sample images with different oil mist contents. The second sample images are obtained by the inspection robot from image acquisition of the spinning box.

[0016] The comparison module is used to determine the first loss value based on the difference between the predicted noise and the labeled noise;

[0017] The adjustment module is used to adjust the parameters of the noise prediction model to be trained based on the first loss value. When the noise prediction model to be trained meets the first convergence condition, the target noise prediction model is obtained. The target noise prediction model is used to remove oil mist from the image to be processed to obtain an oil mist-free image. The image to be processed is an image with oil mist obtained by the inspection robot from the image acquisition of the spinning box.

[0018] Fourthly, this disclosure provides an oil mist removal device applied to the target noise prediction model trained in the third aspect, comprising:

[0019] The acquisition module is used to acquire images of the spinning box in its working state to obtain the image to be processed;

[0020] The removal module performs the following operations repeatedly until the oil fog content of a new image to be processed is lower than a preset threshold, in order to obtain a defogging image:

[0021] The image to be processed is input into the target noise prediction model to obtain the predicted noise;

[0022] Remove the predicted noise from the image to be processed to obtain a new image to be processed;

[0023] Evaluate the oil mist content of the new image to be processed.

[0024] Fifthly, an electronic device is provided, comprising:

[0025] At least one processor; and

[0026] The memory is communicatively connected to the at least one processor; wherein,

[0027] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0028] In a sixth aspect, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of the present disclosure.

[0029] In a seventh aspect, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of the present disclosure.

[0030] The beneficial effects of the technical solution provided in this disclosure include at least the following: based on a well-trained target noise prediction model, it is possible to accurately predict oil mist as noise, so as to accurately remove oil mist from images collected by inspection robots in the spinning workshop.

[0031] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0032] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments provided according to this disclosure and should not be construed as limiting the scope of this disclosure.

[0033] Figure 1 This is a flowchart illustrating a model training method according to an embodiment of the present disclosure;

[0034] Figure 2 This is a schematic diagram of a target oil mist distribution model according to an embodiment of the present disclosure;

[0035] Figure 3 This is a flowchart illustrating a training method for a target oil mist distribution model according to an embodiment of the present disclosure;

[0036] Figure 4 This is a schematic flowchart of an oil mist removal method according to an embodiment of the present disclosure;

[0037] Figure 5 This is a schematic diagram of the structure of a model training device according to an embodiment of the present disclosure;

[0038] Figure 6 This is a schematic diagram of the structure of a model training device according to another embodiment of the present disclosure;

[0039] Figure 7 This is a schematic diagram of the structure of an oil mist removal device according to an embodiment of the present disclosure;

[0040] Figure 8 This is a schematic diagram of the structure of an oil mist removal device according to another embodiment of the present disclosure;

[0041] Figure 9 This is a block diagram of an electronic device used to implement the model training method / oil mist removal method of the embodiments of this disclosure. Detailed Implementation

[0042] The present disclosure will now be described in further detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0043] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0044] In this embodiment of the disclosure, during the production process in the spinning workshop, abnormalities such as broken yarn, misaligned yarn, drifting yarn, unoiled single yarn (i.e., the yarn does not enter the oil nozzle), and unhooked single yarn (i.e., the yarn does not enter the guide hook) may occur in the spinning box. On average, the management personnel conduct manual inspections of the on-site production every two hours. The on-site ambient temperature is high, and the detection is all done by human eyes, which is labor-intensive and inefficient.

[0045] To address anomalies during production, inspection robots are used to patrol the workshop, improving error detection rate and timeliness. However, high workshop temperatures can cause oil mist to evaporate from the spinning box nozzles, which then diffuses into the air, potentially obscuring the inspection robot's camera. In the presence of oil mist, the images captured by the inspection robot may be unclear, hindering timely monitoring of the workshop's operational status. To effectively remove oil mist from images captured by the inspection robot, a model training method is proposed. In this embodiment, oil mist is treated as noise. The noise prediction capability of the inverse process of a diffusion model is used to predict noise, thereby removing oil mist from the image and optimizing image quality. The inverse process of the diffusion model requires constructing a noise prediction model, such as... Figure 1 The training method for this model is shown below, including:

[0046] S101, input the first sample image into the noise prediction model to be trained to obtain the predicted noise.

[0047] The noise prediction model to be trained is constructed based on the target oil mist distribution model. The noise prediction model to be trained is used to construct the reverse process of the diffusion model. The target oil mist distribution model learns the data distribution characteristics of the second sample image set. The second sample image set includes multiple second sample images with different oil mist contents. The second sample images are obtained by the inspection robot from image acquisition of the spinning box.

[0048] Since the target oil mist distribution model includes a feature extraction module and an oil quantity prediction module, the noise prediction model to be trained can be composed of the feature extraction module and the oil quantity prediction module of the target oil mist distribution model, or it can be composed of the feature extraction module and the prediction model of the target oil mist distribution model. Any model that can obtain the predicted noise based on image features can be used as a prediction model.

[0049] The first sample image can be either an image with oil mist or an image without oil mist. The image without oil mist can be used as a negative sample to train the noise prediction model to improve the model's learning ability.

[0050] The diffusion model includes a forward process and a backward process. The forward process involves progressively adding noise to a noise-free image based on a set of noise sequences, ultimately resulting in an image with added noise. The backward process involves denoising the noise-added image step by step, ultimately obtaining a noise-free image.

[0051] S102, determine the first loss value based on the difference information between the predicted noise and the labeled noise.

[0052] S103, the parameters of the noise prediction model to be trained are adjusted based on the first loss value. If the noise prediction model to be trained satisfies the first convergence condition, the target noise prediction model is obtained. This target noise prediction model is used to remove oil mist from the image to be processed, resulting in a de-fogging image. The image to be processed is an image containing oil mist obtained by the inspection robot from image acquisition of the spinning box.

[0053] The first convergence condition is that the first loss value tends to stabilize or the number of iterations of the noise prediction model to be trained meets the preset number.

[0054] In this embodiment, the target oil mist distribution model pre-learns the data distribution of oil mist-containing images collected by the inspection robot, i.e., it learns the oil mist distribution characteristics of the oil mist-containing images. The reverse process of constructing a diffusion model based on the target oil mist distribution model is completed after learning the oil mist-containing images. Therefore, by fully learning the impact of oil mist in the spinning workshop on the images collected by the inspection robot, the noise prediction model to be trained can be constructed. This allows the noise prediction model to predict and train based on certain knowledge and experience, which is beneficial for improving the noise prediction capability and achieving training convergence more quickly. Based on the trained target noise prediction model, the oil mist as noise can be accurately predicted, facilitating the accurate removal of oil mist from the images collected by the inspection robot in the spinning workshop.

[0055] Since the structure of the spinning box and the inspection method of the inspection robot in the spinning workshop are relatively stable, the target noise prediction model can be trained based on the images collected by the inspection robot in the spinning workshop. This makes it easy to learn specific knowledge in a specific scenario, so as to achieve the goal of accurately predicting and denoising noise.

[0056] In summary, the main aspects involved in the embodiments of this disclosure include obtaining a target oil mist distribution model and obtaining labeled noise, thereby training a target noise prediction model. These important aspects will be described in detail below.

[0057] 1) Training to obtain the target oil mist distribution model

[0058] In some embodiments, in order to learn the data distribution of images collected by the spinning workshop inspection robot, the target oil mist distribution model includes a feature extraction module and an oil quantity prediction module. A schematic diagram of the model is shown below. Figure 2 As shown, the image is input into the feature extraction module of the target oil mist distribution model to obtain the feature information of the image. The feature information is then input into the oil quantity prediction module to obtain the oil mist content of the image.

[0059] In some embodiments, the target oil mist distribution model can be obtained based on the following method, such as Figure 3 As shown, it includes:

[0060] S301, input the second sample image into the feature extraction module of the oil mist distribution model to be trained, and obtain the feature information of the second sample image.

[0061] This feature extraction module can use the VGG (Visual Geometry Group) model for feature extraction. By inputting the second sample image into the VGG model, the features corresponding to the image can be obtained.

[0062] It should be noted that all models capable of feature extraction are applicable to the embodiments of this disclosure, and the embodiments of this disclosure do not limit them.

[0063] S302, input the feature information of the second sample image into the oil quantity prediction module of the oil mist distribution model to be trained, and obtain the predicted content of the second sample image.

[0064] The oil mist prediction module can be a classification model, categorizing the predicted content of the second sample image into types with varying oil mist concentrations. For example, the predicted content could be categorized as 5% oil mist, 10% oil mist, 15% oil mist, 20% oil mist, etc. Based on the feature information of the second sample image, the corresponding oil mist content, i.e., the predicted content of that sample image, can be obtained.

[0065] The oil mist prediction module can be a regression model, meaning it can represent the relationship between the predicted content of the second sample images and the feature information of multiple second sample images. Therefore, it can achieve precise prediction of the oil mist content in the second sample images.

[0066] Of course, it should be noted that any method that can obtain the predicted content of the second sample image can be used as an oil quantity prediction module, and this disclosure does not limit this.

[0067] S303. Based on the comparison between the predicted content and the labeled content of the second sample image, determine the second loss value.

[0068] S304. Adjust the parameters of the oil mist distribution model to be trained based on the second loss value. If the oil mist distribution model to be trained satisfies the second convergence condition, the target oil mist distribution model is obtained.

[0069] The second convergence condition is that the second loss value tends to stabilize or the number of model iterations meets the preset number.

[0070] In this embodiment of the disclosure, an oil mist distribution model to be trained is trained based on images containing different oil mist contents, so that the obtained target oil mist distribution model can obtain feature information of different oil mists, providing a strong foundation for subsequent oil mist removal of the image.

[0071] 2) Determine the labeling noise

[0072] As explained above, labeled noise is needed to train the target noise prediction model. To obtain accurate labeled noise, a target noise-adding model can be trained based on the forward process of a diffusion model to obtain labeled noise. This disclosure provides two methods for training the target noise-adding model, one possible method including the following steps A1-A5:

[0073] Step A1: Input the third sample image into the noise-adding model to be trained to obtain the noise sequence to be processed corresponding to the target oil mist content; the noise-adding model to be trained is constructed based on the target oil mist distribution model, and the noise-adding model to be trained is used in the forward process of constructing the diffusion model.

[0074] The third sample image is the image without oil mist. The target oil mist content can be 5%, 10%, 15%, 20%, etc., and this disclosure does not limit it in any way.

[0075] Step A2: Iterate the noise sequence to be processed sequentially to the third sample image to obtain the predicted image.

[0076] When the target oil mist content is 5%, the corresponding noise sequence to be processed can be represented as [x1, x2, ..., xt], which means that the noise sequence to be processed is superimposed on the third sample image after t steps to obtain the predicted image, where t is a positive integer greater than or equal to 1.

[0077] Step A3: Input the predicted image into the target oil mist distribution model to obtain the oil mist content of the predicted image.

[0078] Step A4: Determine the third loss value based on the difference between the oil mist content in the predicted image and the target oil mist content.

[0079] Step A5: Adjust the parameters of the training model based on the third loss value. If the training model satisfies the third convergence condition, the target model is obtained.

[0080] The third convergence condition can be that the third loss value tends to stabilize or the number of model iterations meets a preset number.

[0081] In this embodiment, the aim is to reduce the gap between the predicted image's oil mist content and the target oil mist content, so that the predicted image gradually converges towards the target oil mist content and the target image. The target noise-adding model obtained based on this method can generate images with arbitrary oil mist content, providing a foundation for subsequent oil mist removal.

[0082] In addition to the methods mentioned above, the target noise model can also be obtained based on the following steps B1-B4:

[0083] Step B1: Input the third sample image into the noise-adding model to be trained to obtain the noise sequence to be processed corresponding to the target oil mist content; the noise-adding model to be trained is used for the forward process of constructing the diffusion model.

[0084] Step B2: Iterate the noise sequence to be processed sequentially to the third sample image to obtain the predicted image.

[0085] Step B3 compares the predicted image and the actual image to determine the fourth loss value; the actual image is the image corresponding to the target oil mist content. That is, the actual image is the real sample image collected by the inspection robot.

[0086] The process involves first acquiring feature information from the predicted image and the actual image, calculating the similarity based on their feature information, and then determining the fourth loss based on their similarity. The similarity can be calculated using cosine similarity or the Pearson correlation coefficient. Any method that compares the predicted image and the actual image is applicable to this embodiment, and this embodiment does not limit the specific method used.

[0087] Step B4: Adjust the parameters of the training model based on the fourth loss value. If the training model satisfies the fourth convergence condition, the target model is obtained.

[0088] The fourth convergence condition can be either the fourth loss value stabilizing or the number of model iterations meeting a preset number.

[0089] In this embodiment of the disclosure, with the aim of reducing the gap between the predicted image and the actual image, the real image of the target oil mist content is used as a constraint to accurately train the noise model, so that the model can accurately predict the labeled noise, laying the foundation for subsequent accurate removal of oil mist in the image.

[0090] Based on the target noise model obtained using the aforementioned method, the labeled noise can be obtained using the following method, which can be implemented as follows:

[0091] Step C1: Input the oil-free image into the target noise model to obtain the target noise sequence.

[0092] Step C2: Determine the labeled noise based on the target noise sequence.

[0093] The labeled noise is the noise information at each step in the target noise sequence. For example, in the case of a target noise sequence of [x1, x2, ..., xt], since the image is denoised step by step in this embodiment of the disclosure, the labeled noise can be either x1 or x2, determined based on the comparison steps.

[0094] In this embodiment of the disclosure, based on the aforementioned trained target noise model, labeled noise is obtained to achieve gradual denoising of the image containing oil fog, laying the foundation for obtaining an accurate oil fog-free image in the future.

[0095] The labeled noise can also be customized noise data based on actual conditions. For example, the labeled noise can be set to a%, b%, etc. This disclosure does not limit the method for determining the labeled noise.

[0096] Based on the same technical concept, and building upon the target noise prediction model obtained using the aforementioned training method, the images of the spinning box under working conditions collected by the inspection robot are processed to remove noise, such as... Figure 4 As shown, it includes:

[0097] S401, acquire images of the spinning box in its working state to obtain the image to be processed.

[0098] In some embodiments, a solid-color background board can be placed at a preset position in the spinning box. When the inspection robot's camera is free of oil fog, the background board is photographed to obtain a standard image. When the inspection robot reaches the preset position, it acquires an image of the solid-color background board and compares it with the standard image. If the similarity between the two is less than a preset threshold, it indicates that the difference is too large, suggesting the presence of oil fog in the camera. Based on this, an image of the spinning box in its working state is then acquired and used as the image to be processed.

[0099] Of course, the images captured during the operation of the spinning box can also include part of the background panel. The portion of the image containing the background panel is cropped to obtain a cropped image. This cropped image is then compared to a standard image. If the similarity between the two is less than a preset threshold, it indicates a significant difference, suggesting the presence of oil fog in the camera. This cropped image can then be used as the image to be processed.

[0100] In another embodiment, acquiring an image of the spinning box in its working state to obtain an image to be processed can also be implemented as follows: determining the clarity of the image in the working state of the spinning box; if the clarity is less than a second threshold, determining the image in the working state of the spinning box as the image to be processed.

[0101] In this embodiment of the disclosure, the presence of oil mist in the camera is determined directly based on the image clarity. This determination method is relatively simple and easy to operate.

[0102] S402, perform the following operations repeatedly until the oil fog content of the new image to be processed is lower than a preset threshold, so as to obtain a defogging image:

[0103] S4021, Input the image to be processed into the target noise prediction model to obtain the predicted noise;

[0104] S4022, Remove the predicted noise from the image to be processed to obtain a new image to be processed;

[0105] S4023, Evaluate the oil mist content of the new image to be processed.

[0106] Since the training process of the target noise prediction model is a gradual denoising process, the application of the target noise prediction model can also be understood as a gradual denoising process. Therefore, the image is processed in a loop.

[0107] In some embodiments, when the preset threshold is m%, the image to be processed is input into the target noise prediction model to obtain predicted noise; the predicted noise is removed from the image to be processed to obtain a new image to be processed; when the oil mist content of the new image to be processed is n%, where n is a natural number greater than m, it is input into the target noise prediction model again for noise removal until the oil mist content of the new image to be processed is lower than m%.

[0108] In some embodiments, denoising the image based on the aforementioned trained target noise prediction model can automatically remove oil mist from the image, thereby improving work efficiency.

[0109] In some embodiments, evaluating the oil mist content of a new image to be processed can be implemented by inputting the new image to be processed into a target oil mist distribution model to obtain the oil mist content of the new image to be processed.

[0110] The target oil mist distribution model includes a feature extraction module and an oil quantity prediction module. Based on the feature extraction module, the feature information of the new image to be processed is obtained. Based on the oil quantity prediction module, the feature information is evaluated to obtain the oil mist content of the new image to be processed.

[0111] In this embodiment of the disclosure, the oil mist content of a new image to be processed can be obtained automatically and accurately based on the target oil mist distribution model, so as to facilitate subsequent oil mist processing.

[0112] In some embodiments, if there is too much oil mist information on the camera, the method further includes: sending an alarm message when the oil mist content of the image to be processed exceeds a preset threshold.

[0113] In some embodiments, if the oil mist content of the image to be processed is greater than a preset threshold, an alarm message can be sent to the manager of the spinning box so that the manager can wipe the camera of the inspection robot to remove the oil mist.

[0114] In this embodiment, the present disclosure provides a method to promptly send alarm information to prompt the management personnel to perform degreasing treatment on the inspection robot, thereby obtaining a clear image of the textile machine so as to understand the working status of the spinning box in a timely manner.

[0115] Based on the same technical concept, this disclosure provides a model training device 500, such as... Figure 5 As shown, it includes:

[0116] The input module 501 is used to input the first sample image into the noise prediction model to be trained to obtain the predicted noise; the noise prediction model to be trained is constructed based on the target oil mist distribution model, and the noise prediction model to be trained is used to construct the reverse process of the diffusion model. The target oil mist distribution model learns the data distribution characteristics of the second sample image set. The second sample image set includes multiple second sample images with different oil mist contents. The second sample images are obtained by the inspection robot from image acquisition of the spinning box.

[0117] The comparison module 502 is used to determine a first loss value based on the difference information between the predicted noise and the labeled noise;

[0118] The adjustment module 503 is used to adjust the parameters of the noise prediction model to be trained based on the first loss value. When the noise prediction model to be trained satisfies the first convergence condition, a target noise prediction model is obtained. The target noise prediction model is used to remove oil mist from the image to be processed to obtain an oil mist-free image. The image to be processed is an image with oil mist obtained by the inspection robot from the image acquisition of the spinning box.

[0119] In some embodiments, the target oil mist distribution model includes a feature extraction module and an oil quantity prediction module. Figure 5 Based on this, the model training device 600, such as Figure 6 As shown, it also includes a first training module 601, used to obtain the target oil mist distribution model based on the following method:

[0120] The second sample image is input into the feature extraction module of the oil mist distribution model to be trained to obtain the feature information of the second sample image;

[0121] The feature information of the second sample image is input into the oil quantity prediction module of the oil mist distribution model to be trained, so as to obtain the predicted content of the second sample image.

[0122] The second loss value is determined by comparing the predicted content and the labeled content of the second sample image.

[0123] The parameters of the oil mist distribution model to be trained are adjusted based on the second loss value. When the oil mist distribution model to be trained satisfies the second convergence condition, the target oil mist distribution model is obtained.

[0124] In some embodiments, Figure 5 Based on this, the model training device 600, such as Figure 6 As shown, it also includes a labeling module 602, used to obtain the labeling noise based on the following method:

[0125] The oil-free image is input into the target noise model to obtain the target noise sequence;

[0126] The labeled noise is determined based on the target noise sequence.

[0127] In some embodiments, Figure 5 Based on this, the model training device 600, such as Figure 6 As shown, it also includes a second training module 603, used to obtain the target noise-adding model based on the following method:

[0128] The third sample image is input into the noise-adding model to be trained to obtain the noise sequence to be processed corresponding to the target oil mist content; the noise-adding model to be trained is used to construct the forward process of the diffusion model;

[0129] The noise sequence to be processed is iterated sequentially to the third sample image to obtain the predicted image;

[0130] The predicted image is input into the target oil mist distribution model to obtain the oil mist content of the predicted image;

[0131] Based on the difference between the oil mist content in the predicted image and the target oil mist content, a third loss value is determined.

[0132] The parameters of the noise-adding model to be trained are adjusted based on the third loss value. When the noise-adding model to be trained satisfies the third convergence condition, the target noise-adding model is obtained.

[0133] In some embodiments, Figure 5 Based on this, the model training device 600, such as Figure 6 As shown, it also includes a third training module 604, used to obtain the target noise-adding model based on the following method:

[0134] The third sample image is input into the noise-adding model to be trained to obtain the noise sequence to be processed corresponding to the target oil mist content; the noise-adding model to be trained is used to construct the forward process of the diffusion model;

[0135] The noise sequence to be processed is iterated sequentially to the third sample image to obtain the predicted image;

[0136] The predicted image and the actual image are compared to determine the fourth loss value; the actual image is the image corresponding to the target oil mist content.

[0137] The parameters of the noise-adding model to be trained are adjusted based on the fourth loss value. When the noise-adding model to be trained satisfies the fourth convergence condition, the target noise-adding model is obtained.

[0138] Based on the same technical concept, this disclosure provides an oil mist removal device 700, such as... Figure 7 As shown, the target noise prediction model obtained from the aforementioned training includes:

[0139] The acquisition module 701 is used to acquire images of the spinning box in its working state to obtain the image to be processed;

[0140] The removal module 702 is used to repeatedly perform the following operations until the oil fog content of the new image to be processed is lower than a preset threshold, so as to obtain a defogging image:

[0141] The image to be processed is input into the target noise prediction model to obtain the predicted noise;

[0142] Remove the predicted noise from the image to be processed to obtain a new image to be processed;

[0143] Evaluate the oil mist content of the new image to be processed.

[0144] In some embodiments, the removal module 702 is specifically used for:

[0145] The new image to be processed is input into the target oil mist distribution model to obtain the oil mist content of the new image to be processed.

[0146] In some embodiments, the acquisition module 701 is specifically used for:

[0147] Determine the image clarity of the spinning box under working conditions;

[0148] If the resolution is less than the second threshold, the image of the spinning box in its working state is determined as the image to be processed.

[0149] In some embodiments, Figure 7 Based on this, the oil mist removal device 800, such as Figure 8 As shown, it also includes a sending module 801, which is used to send alarm information when the oil mist content of the image to be processed is greater than a preset threshold.

[0150] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0151] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0152] Figure 9 This is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Figure 9As shown, the electronic device includes a memory 910 and a processor 920. The memory 910 stores a computer program that can run on the processor 920. The number of memories 910 and processors 920 can be one or more. The memory 910 can store one or more computer programs, which, when executed by the electronic device, cause the electronic device to perform the methods provided in the above-described method embodiments. The electronic device may also include a communication interface 930 for communicating with external devices and performing data exchange and transmission.

[0153] If the memory 910, processor 920, and communication interface 930 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0154] Optionally, in a specific implementation, if the memory 910, processor 920, and communication interface 930 are integrated on a single chip, then the memory 910, processor 920, and communication interface 930 can communicate with each other through an internal interface.

[0155] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0156] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct RAMBUS RAM (DR RAM).

[0157] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line, DSL) or wireless (e.g., infrared, Bluetooth, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)). It is worth noting that the computer-readable storage media mentioned in this disclosure can be non-volatile storage media; in other words, it can be non-transient storage media.

[0158] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0159] In the description of the embodiments of this disclosure, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0160] In the description of the embodiments disclosed herein, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0161] In the description of embodiments of this disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0162] The above description is merely an exemplary embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A method for training a target noise prediction model, comprising: Input the first sample image into the noise prediction model to be trained to obtain the predicted noise; The noise prediction model to be trained is constructed based on the target oil mist distribution model. The noise prediction model to be trained is used to construct the inverse process of the diffusion model. The target oil mist distribution model learns the data distribution characteristics of the second sample image set. The second sample image set includes multiple second sample images with different oil mist contents. The second sample images are obtained by the inspection robot through image acquisition of the spinning box. Based on the difference information between the predicted noise and the labeled noise, a first loss value is determined; The labeled noise is obtained based on the target noise model, which is obtained based on the forward process of the diffusion model; Based on the first loss value, the parameters of the noise prediction model to be trained are adjusted. When the noise prediction model to be trained satisfies the first convergence condition, a target noise prediction model is obtained. The target noise prediction model is used to remove oil mist from the image to be processed to obtain an oil mist-free image. The image to be processed is an image with oil mist obtained by the inspection robot from the image acquisition of the spinning box. The target oil mist distribution model includes a feature extraction module and an oil quantity prediction module, and further includes obtaining the target oil mist distribution model based on the following method: The second sample image is input into the feature extraction module of the oil mist distribution model to be trained to obtain the feature information of the second sample image; The feature information of the second sample image is input into the oil quantity prediction module of the oil mist distribution model to be trained, so as to obtain the predicted content of the second sample image. A second loss value is determined by comparing the predicted content and the labeled content of the second sample image; the labeled content represents the labeled oil mist content corresponding to the second sample image. The parameters of the oil mist distribution model to be trained are adjusted based on the second loss value. When the oil mist distribution model to be trained satisfies the second convergence condition, the target oil mist distribution model is obtained.

2. The method according to claim 1 further comprises obtaining the labeled noise based on the following method: The oil-free image is input into the target noise model to obtain the target noise sequence; The labeled noise is determined based on the target noise sequence.

3. The method according to claim 2 further includes obtaining the target noise model based on the following method: The third sample image is input into the noise-adding model to be trained to obtain the noise sequence to be processed corresponding to the target oil mist content; the noise-adding model to be trained is used to construct the forward process of the diffusion model; The noise sequence to be processed is iterated sequentially to the third sample image to obtain the predicted image; The predicted image is input into the target oil mist distribution model to obtain the oil mist content of the predicted image; Based on the difference between the oil mist content in the predicted image and the target oil mist content, a third loss value is determined. The parameters of the noise-adding model to be trained are adjusted based on the third loss value. When the noise-adding model to be trained satisfies the third convergence condition, the target noise-adding model is obtained.

4. The method according to claim 2 further comprises obtaining the target noise model based on the following manner: The third sample image is input into the noise-adding model to be trained to obtain the noise sequence to be processed corresponding to the target oil mist content; the noise-adding model to be trained is used to construct the forward process of the diffusion model; The noise sequence to be processed is iterated sequentially to the third sample image to obtain the predicted image; The predicted image and the actual image are compared to determine the fourth loss value; the actual image is the image corresponding to the target oil mist content. The parameters of the noise-adding model to be trained are adjusted based on the fourth loss value. When the noise-adding model to be trained satisfies the fourth convergence condition, the target noise-adding model is obtained.

5. A method for removing oil mist, comprising: Images of the spinning box in its working state are acquired to obtain the image to be processed; Repeat the following steps until the oil fog content of the new image to be processed is lower than a preset threshold, in order to obtain a defogging image: The image to be processed is input into the target noise prediction model to obtain the predicted noise; wherein the target noise prediction model is the model trained according to claims 1-4; Remove the predicted noise from the image to be processed to obtain a new image to be processed; Evaluate the oil mist content of the new image to be processed.

6. The method according to claim 5, wherein, The evaluation of the oil fog content of the new image to be processed includes: The new image to be processed is input into the target oil mist distribution model to obtain the oil mist content of the new image to be processed.

7. The method according to claim 5, wherein, Images of the spinning box in its working state are acquired to obtain the image to be processed, including: Determine the image clarity of the spinning box under working conditions; If the resolution is less than the second threshold, the image of the spinning box in its working state is determined as the image to be processed.

8. The method according to any one of claims 5-7, further comprising: If the oil mist content of the image to be processed exceeds a preset threshold, an alarm message is sent.

9. A model training device, comprising: The input module is used to input the first sample image into the noise prediction model to be trained, and obtain the predicted noise; The noise prediction model to be trained is constructed based on the target oil mist distribution model. The noise prediction model to be trained is used to construct the inverse process of the diffusion model. The target oil mist distribution model learns the data distribution characteristics of the second sample image set. The second sample image set includes multiple second sample images with different oil mist contents. The second sample images are obtained by the inspection robot through image acquisition of the spinning box. The comparison module is used to determine a first loss value based on the difference information between the predicted noise and the labeled noise; The labeled noise is obtained based on the target noise model, which is obtained based on the forward process of the diffusion model; An adjustment module is used to adjust the parameters of the noise prediction model to be trained based on the first loss value. When the noise prediction model to be trained satisfies the first convergence condition, a target noise prediction model is obtained. The target noise prediction model is used to remove oil mist from the image to be processed to obtain an oil mist-free image. The image to be processed is an image with oil mist obtained by the inspection robot from the image acquisition of the spinning box. The target oil mist distribution model includes a feature extraction module and an oil quantity prediction module, and also includes a first training module for obtaining the target oil mist distribution model based on the following method: The second sample image is input into the feature extraction module of the oil mist distribution model to be trained to obtain the feature information of the second sample image; The feature information of the second sample image is input into the oil quantity prediction module of the oil mist distribution model to be trained, so as to obtain the predicted content of the second sample image. A second loss value is determined by comparing the predicted content and the labeled content of the second sample image; the labeled content represents the labeled oil mist content corresponding to the second sample image. The parameters of the oil mist distribution model to be trained are adjusted based on the second loss value. When the oil mist distribution model to be trained satisfies the second convergence condition, the target oil mist distribution model is obtained.

10. The apparatus of claim 9, further comprising a labeling module for obtaining the labeling noise in the following manner: The oil-free image is input into the target noise model to obtain the target noise sequence; The labeled noise is determined based on the target noise sequence.

11. The apparatus of claim 10, further comprising a second training module for obtaining the target noise model in the following manner: The third sample image is input into the noise-adding model to be trained to obtain the noise sequence to be processed corresponding to the target oil mist content; the noise-adding model to be trained is used to construct the forward process of the diffusion model; The noise sequence to be processed is iterated sequentially to the third sample image to obtain the predicted image; The predicted image is input into the target oil mist distribution model to obtain the oil mist content of the predicted image; Based on the difference between the oil mist content in the predicted image and the target oil mist content, a third loss value is determined. The parameters of the noise-adding model to be trained are adjusted based on the third loss value. When the noise-adding model to be trained satisfies the third convergence condition, the target noise-adding model is obtained.

12. The apparatus of claim 10, further comprising a third training module for obtaining the target noise model in the following manner: The third sample image is input into the noise-adding model to be trained to obtain the noise sequence to be processed corresponding to the target oil mist content; the noise-adding model to be trained is used to construct the forward process of the diffusion model; The noise sequence to be processed is iterated sequentially to the third sample image to obtain the predicted image; The predicted image and the actual image are compared to determine the fourth loss value; the actual image is the image corresponding to the target oil mist content. The parameters of the noise-adding model to be trained are adjusted based on the fourth loss value. When the noise-adding model to be trained satisfies the fourth convergence condition, the target noise-adding model is obtained.

13. An oil mist removal device, comprising: The acquisition module is used to acquire images of the spinning box in its working state to obtain the image to be processed; The removal module performs the following operations repeatedly until the oil fog content of a new image to be processed is lower than a preset threshold, in order to obtain a defogging image: The image to be processed is input into the target noise prediction model to obtain the predicted noise; wherein the target noise prediction model is the model trained according to claims 1-4; Remove the predicted noise from the image to be processed to obtain a new image to be processed; Evaluate the oil mist content of the new image to be processed.

14. The apparatus according to claim 13, wherein, The removal module is specifically used for: The new image to be processed is input into the target oil mist distribution model to obtain the oil mist content of the new image to be processed.

15. The apparatus according to claim 13, wherein, The acquisition module is specifically used for: Determine the image clarity of the spinning box under working conditions; If the resolution is less than the second threshold, the image of the spinning box in its working state is determined as the image to be processed.

16. The apparatus according to any one of claims 13-15 further includes a transmitting module for transmitting an alarm message when the oil mist content of the image to be processed is greater than a preset threshold.

17. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

Images