Webpage ICON image defogging classification method based on patch diffusion model

Abstract

The invention relates to the technical field of image defogging classification, in particular to a webpage ICON image defogging classification method based on a patch diffusion model, and the method comprises the steps: carrying out the patch extraction from a foggy webpage ICON image based on the image features contained in the foggy webpage ICON image, and obtaining a plurality of target ICON patches; all target ICON patches corresponding to the foggy webpage ICON image are input into the improved patch diffusion model for defogging processing, and a defogged ICON patch corresponding to each target ICON patch is obtained; all the defogged ICON patches are integrated, and a defogged webpage ICON image is obtained; and identifying target classification features contained in the defogged webpage ICON image, and inputting the target classification features into a preset ICON classifier for classification to obtain a classification result. According to the method and the device, the defogging effect can be improved conveniently, so that the accuracy of classifying and identifying the foggy webpage ICON image is improved.

Description

Technical Field

[0001] The present application relates to the technical field of image dehazing and classification, and in particular to a web page ICON image dehazing and classification method based on a patch diffusion model. Background Art

[0002] The generation of foggy web icon images is often related to a variety of factors during image acquisition, transmission, storage, and display. As a core visual element of the user interface, web icons play a key role in brand identity, functional navigation, and interactive guidance. If a web icon image contains features such as optical blur and noise interference, it can be characterized as foggy. Because foggy web icon images may exhibit blurred edges and loss of texture detail, subsequent classification and recognition based on foggy web icon images may result in low accuracy.

[0003] In conventional technical means, the foggy web page ICON image is usually denoised through a diffusion model before subsequent classification and recognition operations are performed. However, when processing foggy web page ICON images, the conventional diffusion model generally adopts a traditional dehazing algorithm, that is, it may not be optimized for the characteristics of foggy web page ICON, or it may use a general image recovery model that ignores the domain knowledge of foggy web page ICON. Therefore, it may lead to poor dehazing effect, or the dehazed image still cannot effectively improve the classification performance. Summary of the Invention

[0004] In order to improve the defogging effect and thus improve the accuracy of classification and identification of foggy web page ICON images, the present application provides a web page ICON image defogging classification method based on a patch diffusion model.

[0005] In the first aspect, the present application provides a webpage icon image dehazing and classification method based on a patch diffusion model, which adopts the following technical solutions: A webpage ICON image dehazing and classification method based on a patch diffusion model includes: Acquire a foggy webpage ICON image, and extract patches from the foggy webpage ICON image based on image features contained in the foggy webpage ICON image to obtain a plurality of target ICON patches; Input all target ICON patches corresponding to the foggy webpage ICON image into the improved patch diffusion model for defogging, and obtain the defogging ICON patch corresponding to each target ICON patch; Integrate all dehazed ICON patches to get the dehazed webpage ICON image; Identify target classification features contained in the dehazed webpage ICON image, and input the target classification features into a preset ICON classifier for classification to obtain a classification result.

[0006] By adopting the above technical solution, patches are extracted from the foggy web ICON image, so that each area in the foggy web ICON image can be processed separately, avoiding edge blur or detail loss caused by global defogging. By using the improved patch diffusion model for defogging, the local details and complex structures of the foggy web ICON image can be better captured. Then, each defogging ICON patch is integrated to eliminate the splicing traces of adjacent patch boundaries and generate a smooth transition defogging web ICON image. Finally, by extracting target classification features from the defogging ICON image and combining it with the preset ICON classifier, the accuracy of classification and recognition of the foggy web ICON image can be improved.

[0007] In one possible implementation, extracting patches from the foggy webpage ICON image based on image features contained in the foggy webpage ICON image to obtain a plurality of target ICON patches includes: Performing image preprocessing on the foggy web page ICON image to obtain a processed web page ICON image; Performing edge detection on the processed webpage ICON image, and setting an initial patch size and an initial repetition rate based on the edge detection result; Performing patch extraction on the processed webpage ICON image based on the initial patch size and the initial repetition rate to obtain a plurality of initial ICON patches; Semantic features are extracted for each initial ICON patch, and patches are merged based on the semantic feature extraction results corresponding to each initial ICON patch to obtain the target ICON patch.

[0008] By adopting the above technical solution, by performing image preprocessing on the foggy web page ICON image, it is convenient to reduce interference such as fog and noise in the foggy web page ICON image, and improve the clarity and contrast of the image. By performing edge detection on the foggy web page ICON image, it is convenient to accurately locate the contour information of the web page features in the foggy web page ICON image. By analyzing the edge detection results to set the initial patch size and repetition rate, it can adapt to the size and distribution density of different ICONs, and it is convenient to avoid crossing different web page features when extracting patches. In addition, by performing semantic feature extraction on each initial ICON patch, it is convenient to analyze the semantic correlation between each initial ICON patch. By merging the initial ICON patches with semantic correlation, it is convenient to reduce the number of patches and avoid over-segmentation.

[0009] In one possible implementation, the method further includes: Identify the patch feature pixel value corresponding to each target ICON patch in the adjacent target ICON patches; Determining a repetition rate ratio between the adjacent target ICON patches based on the characteristic pixel values ​​of each patch and the initial repetition rate between the adjacent target ICON patches; Each target ICON patch among the adjacent target ICON patches is optimized based on the repetition rate ratio.

[0010] By adopting the above technical solution, the visual features of each target ICON patch in adjacent target ICON patches are quantified to obtain patch feature pixel values, which facilitates intuitive comparison of the visual features between adjacent target ICON patches. The initial repetition rate between adjacent target ICON patches is adjusted based on the comparison results to obtain the repetition rate ratio, which provides a basis for dynamic adjustment for subsequent patch optimization and avoids over-merging or under-merging caused by a fixed repetition rate.

[0011] In one possible implementation, the patch diffusion model improvement process includes: Obtain sample data and classification requirements corresponding to the foggy web page ICON image; Identifying ICON priori feature information and sample noise feature information corresponding to the sample data, and determining an adaptive noise scheduling strategy based on the sample noise feature information; Determine a denoising loss term and a classification loss term based on classification requirements, and construct a joint loss function based on the denoising loss term and the classification loss term; Performing inverse diffusion improvement based on the ICON prior feature information, the sample noise feature information, and the constructed joint loss function to obtain an improved inverse diffusion model; determining a target noise schedule based on the sample data, and performing diffusion improvement based on the target noise schedule to obtain an improved diffusion model; Based on the improved inverse diffusion model and the improved diffusion model, the improvement process of the patch diffusion model is completed.

[0012] By adopting the above technical solution, by analyzing the noise characteristics of web page ICON images and setting an adaptive noise scheduling strategy, it is convenient to dynamically adjust the appropriate noise step size according to the degree of detail of each area, so as to reduce the collection of invalid information while retaining the detail information. By introducing ICON prior feature information in the model improvement process, it is convenient to enable the improved model to focus on important structures and edge information, so as to guide the improved model to generate clear images that are more in line with the corresponding ICON semantics. When training the diffusion model, in addition to the traditional denoising loss, a loss related to the classification task is also introduced. By using a joint loss function, it is convenient to guide the improved model to generate dehazing results that are more conducive to classification.

[0013] In one possible implementation, integrating all defogging ICON patches to obtain a defogging webpage ICON image includes: Identify the patch parameter information corresponding to each dehazing ICON patch, which includes the patch size and patch position coordinates; Based on the patch parameter information corresponding to each dehazing ICON patch, all the dehazing ICON patches are integrated to obtain an initial integrated web page image, and overlapping areas corresponding to the initial integrated web page image are identified; Based on the overlapping defogging ICON patch corresponding to each overlapping area and the patch parameter information corresponding to each overlapping defogging ICON patch, weighted elimination processing is performed on each overlapping area to obtain the defogging web page ICON image.

[0014] By adopting the above technical solution, by identifying the size and position coordinates of each dehazed ICON patch, the position of the patch in the original image can be accurately located, and the size matching between patches can be ensured, thereby facilitating the improvement of the accuracy of the dehazed ICON patches during splicing and fusion. The weighted elimination processing facilitates the smoothing of the pixel values ​​in the overlapping areas, thereby facilitating the elimination of splicing traces at the patch boundaries and improving the image quality of the dehazed web ICON image.

[0015] In one possible implementation, based on the overlapping defogging ICON patches corresponding to the overlapping areas and the patch parameter information corresponding to the overlapping defogging ICON patches, weighted elimination processing is performed on the overlapping areas, including: Identify the overlapping pixel value and overlapping pixel coordinate point of each overlapping pixel point in the overlapping area; Determine the overlapping patch Gaussian weight of each overlapping defogging ICON patch corresponding to the overlapping area based on the overlapping patch parameter information of the overlapping defogging ICON patch corresponding to the overlapping area and the overlapping pixel value and overlapping pixel coordinate point of each overlapping pixel point; A weighted elimination process is performed on each overlapping pixel point according to the Gaussian weights of each overlapping patch corresponding to the overlapping area.

[0016] By adopting the above technical solution, by identifying the overlapping pixel value of each pixel point in the overlapping area, the visual features of the overlapping area can be accurately quantified to ensure that the details of the overlapping area are preserved. In addition, based on the overlapping patch parameter information and the overlapping pixel value, the Gaussian weight of each overlapping patch is calculated, and weighted elimination is performed based on the Gaussian weights of each overlapping patch, so as to ensure a smooth transition of the pixel values ​​in the overlapping area and avoid color mutations or edge blurring caused by simple averaging.

[0017] In one possible implementation, identifying target classification features contained in the defogging webpage ICON image includes: Identifying threshold pixel points and threshold pixel coordinates corresponding to each threshold pixel point from the dehazed webpage ICON image, where the threshold pixel point is a pixel point having a pixel value higher than a preset threshold value; Determine the predicted path of the web page feature corresponding to the dehazed web page ICON image based on the coordinates of each threshold pixel point; An initial feature extraction strategy is optimized based on the predicted direction path to obtain a target feature extraction strategy, and the target classification feature is identified from the defogging webpage ICON image based on the target feature extraction strategy.

[0018] By adopting the above technical solution, the coordinate position of the threshold pixel point is analyzed to facilitate the prediction of the direction of the web page features in the dehazed web page ICON image, and then the target classification features in the dehazed web page ICON image are extracted in a targeted manner by predicting the direction path, so as to reduce redundant calculations when extracting target classification features, and at the same time facilitate improving the accuracy of determining the classification results.

[0019] In a second aspect, the present application provides a processing system, which adopts the following technical solution: A processing system, the processing system comprising: at least one processor; Memory; At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the above-mentioned web page ICON image dehazing classification method based on the patch diffusion model.

[0020] In a third aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium includes: a computer program that can be loaded by a processor and execute the above-mentioned web page ICON image defogging classification method based on the patch diffusion model.

[0021] In a fourth aspect, the present application provides a computer program product that adopts the following technical solution: A computer program product includes a computer program, which, when executed by a processor, implements the above-mentioned web page ICON image dehazing and classification method based on the patch diffusion model.

[0022] In summary, this application includes at least one of the following beneficial technical effects: By performing patch extraction on the foggy web ICON image, it is convenient to process each area in the foggy web ICON image separately, avoiding edge blur or detail loss caused by global dehazing. By using the improved patch diffusion model for dehazing, it is convenient to better capture the local details and complex structures of the foggy web ICON image. Then, by integrating each dehazed ICON patch, it is convenient to eliminate the splicing traces of adjacent patch boundaries and generate a smooth transition dehazed web ICON image. Finally, by extracting target classification features from the dehazed ICON image and combining it with the preset ICON classifier, it is convenient to improve the accuracy of classification and recognition of foggy web ICON images.

[0023] By identifying the size and position coordinates of each dehazed ICON patch, the position of the patch in the original image can be accurately located, and the size matching between patches can be ensured, thereby improving the accuracy of the dehazed ICON patches during splicing and fusion. Weighted elimination processing is used to smooth the pixel values ​​of overlapping areas, thereby eliminating the splicing traces at the patch boundaries and improving the image quality of the dehazed web ICON image. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flow chart of a webpage ICON image dehazing and classification method based on a patch diffusion model in an embodiment of the present application; Figure 2 This is a schematic diagram of a process for determining a defogging web page ICON image in an embodiment of the present application; Figure 3 It is a structural diagram of a processing system in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following is combined with Figures 1 to 3 This application is described in further detail.

[0026] After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.

[0027] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0028] It should be noted that in the optional embodiments of the present application, the object information and other related data involved, when the embodiments in the present application are applied to specific products or technologies, need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, it needs to be obtained through the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information requires the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained. The embodiments also need to be implemented with the authorization and consent of the object.

[0029] Specifically, the embodiments of the present application provide a webpage icon image dehazing and classification method based on a patch diffusion model, which is executed by a processing system, which can be a server or a terminal device. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smartphone, tablet computer, laptop computer, desktop computer, etc., but is not limited to this. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0030] refer to Figure 1 , Figure 1 : This is a flow chart of a webpage ICON image dehazing and classification method based on a patch diffusion model in an embodiment of the present application. The method includes steps S110 to S140, wherein: Step S110: obtaining a foggy web page ICON image, and extracting patches from the foggy web page ICON image based on image features contained in the foggy web page ICON image to obtain a plurality of target ICON patches.

[0031] Specifically, foggy web ICON images refer to images that appear blurry, low contrast, and lost details due to fog effects or similar visual interference. There are multiple reasons for their generation, including but not limited to compression distortion, noise interference, storage damage, etc. Foggy web ICON images often show blurred edges, lost details, low contrast between foreground and background, grayish colors, and visual graininess. Generally, foggy web ICON images need to be dehazed in order to reduce the probability of misjudgment in subsequent classification tasks. Since the fog effect of a foggy ICON is usually uneven, for example, the edges of a foggy web ICON image are blurred due to the fog effect, but the center part is still clear. If the overall defogging is performed, the edges may be over-sharpened, and noise will appear in the center part. Therefore, the technical method provided in this application does not treat the foggy web ICON image as a whole for defogging when defogging the foggy web ICON image, but rather performs patch extraction on the foggy web ICON image, that is, the foggy web ICON image is divided into multiple region images, and then defogging is performed on each region image. A foggy web ICON image can be divided into multiple region images, that is, a foggy web ICON image can extract multiple target ICON patches.

[0032] When extracting patches from a foggy webpage icon image, the foggy webpage icon image may be equally segmented according to a fixed size. During the segmentation process, overlap between adjacent target icon patches is allowed to ensure continuity during subsequent reassembly. Furthermore, to avoid over-segmentation, when extracting patches from the foggy webpage icon image based on image features contained in the foggy webpage icon image to obtain multiple target icon patches, the following may be specifically included: Image preprocessing is performed on the foggy web page ICON image to obtain a processed web page ICON image; edge detection is performed on the processed web page ICON image, and an initial patch size and an initial repetition rate are set based on the edge detection results; patch extraction is performed on the processed web page ICON image based on the initial patch size and the initial repetition rate to obtain multiple initial ICON patches; semantic feature extraction is performed on each initial ICON patch, and patches are merged based on the semantic feature extraction results corresponding to each initial ICON patch to obtain a target ICON patch.

[0033] Specifically, the image preprocessing of the foggy web page ICON image includes but is not limited to denoising, contrast enhancement, etc., wherein a preset filtering algorithm can be used to remove noise interference in the foggy web page ICON image to reduce the impact of noise on subsequent patch extraction. The preset filtering algorithm can be Gaussian filtering, median filtering, etc. The specific preset filtering algorithm is not specifically limited in the embodiment of the present application; histogram equalization, contrast-limited adaptive histogram equalization and other methods can be applied to enhance the contrast of the foggy web page ICON image to enhance the details in the foggy web page ICON image. The specific image preprocessing method is not specifically limited in the embodiment of the present application. The foggy web page ICON image that has undergone image preprocessing can be regarded as a processed web page ICON image.

[0034] A preset edge detection algorithm can be used to perform edge detection on the processed web page ICON image, wherein the preset edge detection algorithm can be a detection algorithm such as Canny edge detection, Sobel operator or Laplacian operator. The specific preset edge detection algorithm is not specifically limited in the embodiment of this application. By performing edge detection on the foggy web page ICON image, it is convenient to accurately locate the contour information of the web page feature in the foggy web page ICON image, and determine the initial patch size based on the contour information of the web page feature in the foggy web page ICON image, so as to reduce the probability of the same web page feature being divided into multiple initial ICON patches. When determining the initial patch size, the contour information of each web page feature in the foggy web page ICON image can be counted, and the average contour width and average contour height of all web page features can be calculated based on this. Finally, the initial patch size is determined based on the average contour width and average contour height to ensure that when extracting patches from the processed web page ICON image based on the initial patch size, each initial ICON patch can completely contain the corresponding web page feature. After determining the initial patch size, the initial repetition rate corresponding to the initial patch size can be determined based on the preset repetition rate mapping relationship, wherein the preset repetition rate mapping relationship is the correspondence between the initial patch size and the initial repetition rate. The specific content is not specifically limited in the embodiment of this application and can be determined by relevant staff based on historical experimental data and uploaded to the processing system.

[0035] Based on the initial patch size and the initial repetition rate, the initial segmentation of the web page ICON image can be achieved. Since different initial ICON patches may have semantic associations, after obtaining multiple initial ICON patches, the semantic features of each initial ICON patch can be extracted according to a preset deep learning model, wherein the preset deep learning model can be ResNet, VGG, etc. The specific semantic feature extraction method is not specifically limited in the embodiment of this application. After extracting the semantic features of each initial ICON patch, the semantic feature similarity between the initial ICON patches can be calculated using methods such as cosine similarity and Euclidean distance, and the initial ICON patches whose semantic feature similarity is higher than the preset similarity threshold are merged. The specific preset similarity threshold is not specifically limited in the embodiment of this application. By merging the initial ICON patches with semantic associations, it is convenient to reduce the number of patches and avoid over-segmentation.

[0036] Furthermore, in order to avoid over-merging or under-merging caused by a fixed repetition rate, the method provided in the embodiment of the present application further includes: Identify the patch feature pixel value corresponding to each target ICON patch in the adjacent target ICON patches; determine the repetition rate ratio between the adjacent target ICON patches based on the individual patch feature pixel values ​​and the initial repetition rate between the adjacent target ICON patches; and optimize each target ICON patch in the adjacent target ICON patches based on the repetition rate ratio.

[0037] Specifically, the embodiment of the present application performs patch extraction based on the initial repetition rate corresponding to the initial patch size. However, the multiple target ICON patches obtained after merging the initial ICON patches based on the semantic features of each initial ICON patch may have different sizes and feature richness. At this time, the initial repetition rate of the two can be further adjusted according to the patch features corresponding to each target ICON patch in the adjacent target ICON patches.

[0038] For any group of adjacent target ICON patches, each group of adjacent target ICON patches contains at least two target ICON patches. The patch feature pixel value corresponding to each target ICON patch in the adjacent target ICON patches can be identified based on a preset feature recognition algorithm. The patch feature pixel value is the average value of the pixel values ​​corresponding to all pixels in the target ICON patch. When counting the patch feature pixel values, statistical analysis can be performed from aspects such as color, texture, and gradient to determine the patch feature pixel value corresponding to the target ICON patch, that is, a quantitative analysis of the feature distribution in the target ICON patch is performed. The specific quantitative analysis method is not specifically limited in the embodiment of this application. After determining the patch feature pixel value corresponding to each target ICON patch, the patch feature pixel values ​​corresponding to the target ICON patches can be compared to determine the feature ratio between the target ICON patches, and then the repetition rate ratio between adjacent target ICON patches can be determined based on the feature ratio and the initial repetition rate. For example, the initial repetition rate between two adjacent target ICON patches a and target ICON patch b is 20%, that is, when performing patch extraction, in addition to all the features corresponding to itself, the target ICON patch a also contains 20% of the features at the connection with the target ICON patch b. Similarly, in addition to all the features corresponding to itself, the target ICON patch b also contains 20% of the features at the connection with the target ICON patch a.

[0039] By quantitatively analyzing the features corresponding to target ICON patches a and b, and determining that the feature ratio between target ICON patches a and b is 2:3, the target ICON patch with the lower feature ratio can be maintained at its original initial repetition rate. Based on the feature ratio, the initial repetition rate of the other target ICON patch can be adjusted to ensure that the adjusted repetition rate ratio is consistent with the feature ratio. For example, if target ICON patch a maintains its original initial repetition rate of 20%, based on the feature ratio of 2:3, it can be determined that target ICON patch b can be determined at a repetition rate of 30%. Since the patch feature pixel value is directly proportional to the feature complexity contained in the target ICON patch, when the patch feature pixel value is low, the corresponding target ICON patch is less prominent, and the difficulty of overlapping the corresponding patches may be greater. Therefore, by quantifying the features between adjacent target ICON patches and adjusting the repetition rate ratio based on the feature ratio, a dynamic adjustment basis is provided for subsequent patch optimization, avoiding over-merging or under-merging caused by a fixed repetition rate. Based on the above method, the repetition rate ratios corresponding to all adjacent target ICON patches can be determined, and optimization operations on all target ICON patches can be achieved.

[0040] Step S120: input all target ICON patches corresponding to the foggy webpage ICON image into the improved patch diffusion model for defogging, and obtain a defogging ICON patch corresponding to each target ICON patch.

[0041] Specifically, the patch diffusion model consists of two phases: the diffusion phase and the inverse diffusion phase. For any target icon patch, after the target icon patch is imported into the improved patch diffusion model, in the diffusion phase, the model gradually adds Gaussian noise to the target icon patch until the target icon patch becomes random noise. In the inverse diffusion phase, the model iteratively applies a preset denoising function to gradually restore a clear target icon patch from the random noise, i.e., the dehazed icon patch. The preset denoising function is usually implemented by a neural network. Using this method, the dehazed icon patch corresponding to each target icon patch can be obtained.

[0042] The method provided in the embodiment of the present application mainly includes the following steps when improving the patch diffusion model: Obtain sample data and classification requirements corresponding to the foggy web page ICON image; identify the ICON prior feature information and sample noise feature information corresponding to the sample data, and determine the adaptive noise scheduling strategy based on the sample noise feature information; determine the denoising loss term and the classification loss term based on the classification requirements, and construct a joint loss function based on the denoising loss term and the classification loss term; perform inverse diffusion improvement based on the ICON prior feature information, the sample noise feature information and the constructed joint loss function to obtain an improved inverse diffusion model; determine the target noise scheduling based on the sample data, perform diffusion improvement based on the target noise scheduling to obtain an improved diffusion model; complete the improvement process of the patch diffusion model based on the improved inverse diffusion model and the improved diffusion model.

[0043] Specifically, the sample data is a large dataset containing clear and foggy web icon images and annotated classification labels. For example, the dataset can be 500 foggy web icon images and 500 clear web icon images. Each image is annotated with a classification label, such as "social media", "tools", "games", etc., which can be determined by relevant staff based on historical dehazing processing data and uploaded to the processing system. Classification requirements can provide a clear direction for model improvement, such as prioritizing the improvement of the recognition accuracy of specific categories to avoid blind optimization. For example, if the classification requirement is to identify "social media" and "tools", then the model improvement needs to focus on optimizing the feature extraction and classification capabilities of these two categories. The specific classification requirements can be determined by relevant staff based on actual conditions and uploaded to the processing system.

[0044] Since the patch diffusion model mainly consists of two stages, namely, the inverse diffusion stage and the diffusion stage, the improvements performed in the inverse diffusion stage include the introduction of ICON prior feature information, adaptive noise scheduling, and a joint loss function, where: The introduction of ICON prior feature information refers to prior knowledge related to the semantics, structure or visual attributes of the target ICON patch, which can help the model better understand the characteristics of the target ICON patch, thereby facilitating the retention of its key information during the denoising or generation process. For example, by adding an attention mechanism to the model, it can focus on important structural and edge information in the target ICON patch, which is more effective for retaining local details and processing structural blur. At the same time, the pre-trained ICON feature extractor is used to extract the semantic features of the target ICON patch as an additional conditional input, and the dehazing process of the model is guided based on these semantic features to generate a clearer image that is more in line with the ICON semantics. The ICON prior feature information can be the category, function, symbolic meaning, geometric structure, edge, contour or key point of the ICON, etc. The specific ICON prior feature information is not specifically limited in the embodiments of this application.

[0045] The introduction of adaptive noise scheduling refers to dynamically adjusting the noise step size in the inverse diffusion process according to the local or global characteristics of the ICON image, such as detail richness, edge density, semantic importance, etc., so as to retain the key information in the target ICON patch while denoising. Specifically, a smaller noise step size can be used in areas with rich details, a larger noise step size can be used in smooth areas, a smaller noise step size can be used for semantically more important areas, and a larger noise step size can be used for secondary areas. The specific form can be setting the noise step size to 0.08 for the character pattern area in the "game" ICON and setting the noise step size to 0.2 for the background area. The specific adaptive noise scheduling is not specifically limited in the embodiments of this application.

[0046] The introduction of a joint loss function means that in the process of model improvement, it is necessary to consider both the denoising loss term and the classification loss term. The denoising loss term is used to measure the pixel-level or feature-level difference between the intermediate image in the process of model improvement and the original clear web page ICON image, and is used to measure the dehazing quality. The classification loss term is used to determine whether the intermediate image in the process of model improvement conforms to the semantic category or function corresponding to the original clear web page ICON image. It mainly compares the classification prediction result with the corresponding sample image and the corresponding annotated classification label. In the embodiment of the present application, the cross entropy function can be used to evaluate the classification loss term. The intermediate image refers to the image generated at one or more intermediate time steps in the dehazing process of the diffusion model, which is between pure noise and the final dehazed image. The denoising loss term mainly calculates the parameter gradient of the model during the improvement process through the back-propagation algorithm, and uses optimization algorithms such as gradient descent to update the model parameters. The classification loss term can be applied to the model improvement process by cooperating with the differentiable classifier. The classification loss term provides a clear semantic optimization goal for model improvement, and the integration of differentiable classifiers further refines the generation process through dynamic feedback. For example, in the joint loss function, the classification loss term may not be able to fully capture the subtle semantic differences of web page features. By integrating the ICON classifier into the model improvement process, it is easier for the model to perceive the needs of the downstream classifier, input the predicted intermediate image into the differentiable classifier in the intermediate step of the back diffusion, and adjust the back diffusion process according to the classification results. After the introduction of the classification loss term, a bias term can be introduced in the sampling process to make the model more likely to sample images belonging to the predicted category. Specifically, the gradient information of the differentiable classifier can be used to guide the sampling: at each step t of the back diffusion, an intermediate image x is first predicted. t ; x t Input into the differentiable classifier and calculate the probability or score of it belonging to the target category; calculate the output of the differentiable classifier relative to x t The gradient of x, which indicates how to change t This increases the likelihood that the image belongs to the target category; this gradient information is added to the denoising direction predicted by the model, making it more likely that an image that matches the target category will be generated in the next sampling step. This process can be iterated to gradually guide the generated image closer to the target category.

[0047] Among them, the requirements of the downstream classifier refer to image features that can maximize its classification performance, that is, image characteristics that enable it to make accurate and robust predictions. Specifically, for brand logo classification, the diffusion model needs to learn to retain and enhance the clear logo shape, color, and specific detailed patterns during the dehazing process, while suppressing noise or blur that is useless for classification. For small changes in the image, such as lighting, angle, and slight deformation, the diffusion model needs to learn to retain and enhance these features during the dehazing process, thereby improving the generalization ability of the classifier. Through integrated training with a differentiable classifier, the diffusion model can learn to generate dehazed images containing image features that maximize its classification performance, thereby achieving a dual improvement in dehazing and classification performance. If the downstream classifier uses a preset pre-trained feature extractor, such as ResNet, ViT, etc., the diffusion model needs to learn to generate images that can be effectively encoded by the preset pre-trained feature extractor. This may mean that certain texture, edge, or color information needs to be retained during the dehazing process.

[0048] The improvement process in the diffusion phase can be expressed as follows: identifying multiple noise schedules corresponding to the sample data, determining the noise schedule with the highest number in the sample data as the target noise schedule, and then calculating the intermediate noise schedule of the intermediate image during the model improvement process based on the noise schedule formula until the intermediate noise schedule is no less than the target noise schedule. The noise schedule formula is as follows: , where x t is the noise image at time step t, x t-1 is the image at the previous time step, β t is a predefined noise schedule, ϵ t is noise sampled from a standard Gaussian distribution.

[0049] By integrating the improved diffusion stage and the improved inverse diffusion stage, the improved patch diffusion model can be obtained.

[0050] Step S130: Integrate all dehazed ICON patches to obtain a dehazed web page ICON image.

[0051] Specifically, after dehazing processing is performed using the improved patch diffusion model, a dehazed ICON patch corresponding to each target ICON patch can be obtained. By splicing all the dehazed ICON patches, a dehazed web ICON image can be obtained. Since some features of adjacent target ICON patches are retained in each target ICON patch when the foggy web ICON image is segmented, obvious splicing marks may appear on the patch boundaries after all the dehazed ICON patches are recombined. In order to eliminate the splicing marks on the patch boundaries and improve the image quality of the dehazed web ICON image, the method provided in the embodiment of the present application specifically includes steps S210-S230 when integrating all the dehazed ICON patches to obtain the dehazed web ICON image. Figure 2 As shown, where: Step S210: Identify patch parameter information corresponding to each dehazing ICON patch, where the patch parameter information includes patch size and patch position coordinates.

[0052] Specifically, the patch size of each dehazed ICON patch and the position coordinates of each dehazed ICON patch in the foggy web page ICON image are identified based on the preset feature recognition algorithm. In order to facilitate unified analysis, the patch position coordinates can be specified as the upper left corner coordinates or the center point coordinates of the dehazed ICON patch. The specific settings of the patch position coordinates and the preset feature recognition algorithm can be set by relevant staff according to actual needs.

[0053] Step S220: Based on the patch parameter information corresponding to each dehazing ICON patch, all the dehazing ICON patches are integrated to obtain an initial integrated web page image, and overlapping areas corresponding to the initial integrated web page image are identified.

[0054] Specifically, an initial canvas corresponding to the foggy web page icon image can be constructed based on the patch parameter information corresponding to all dehazed icon patches. Specifically, the minimum required size of the initial canvas can be calculated by traversing the patch sizes and position coordinates of all dehazed icon patches. The width of the initial canvas should at least cover the right edge of the rightmost patch of all dehazed icon patches, and the height of the initial canvas should at least cover the bottom edge of the bottommost patch of all dehazed icon patches. Then, based on the patch position coordinates of each dehazed icon patch, each dehazed icon patch is superimposed on the initial canvas to obtain the initial integrated web page image. By identifying the edge information of each dehazed icon patch, the overlapping area can be identified from the initial integrated web page image. The overlapping area is at least the splicing area of ​​two adjacent icon patches.

[0055] Step S230: Based on the overlapping defogging ICON patch corresponding to each overlapping area and the patch parameter information corresponding to each overlapping defogging ICON patch, weighted elimination processing is performed on each overlapping area to obtain a defogging web page ICON image.

[0056] Specifically, for any overlapping area, each pixel in the overlapping area can be weighted based on a preset weighting strategy, wherein the preset weighting strategy includes but is not limited to average weighting, distance weighting, and transparency weighting. For example, when using distance weighting, weights can be assigned based on the distance between each pixel and the boundary of the overlapping area. The closer the pixel is to the boundary, the smaller the weight. The calculated weighted average value is applied to each pixel in the overlapping area to eliminate the splicing traces corresponding to each overlapping area. Finally, the overlapping areas without splicing traces are re-spliced ​​to obtain a defogged web page ICON image. Furthermore, in order to ensure a smooth transition of pixel values ​​in the overlapping area and avoid color mutations or edge blurring caused by simple averaging, the overlapping area is weighted based on the overlapping defogging ICON patch corresponding to the overlapping area and the patch parameter information corresponding to the overlapping defogging ICON patch. Specifically, it may include: Identify the overlapping pixel value and overlapping pixel coordinate point of each overlapping pixel point in the overlapping area; determine the overlapping patch Gaussian weight of each overlapping dehazing ICON patch corresponding to the overlapping area based on the overlapping patch parameter information of the overlapping dehazing ICON patch corresponding to the overlapping area and the overlapping pixel value and overlapping pixel coordinate point of each overlapping pixel point; perform weighted elimination processing on each overlapping pixel point according to the Gaussian weight of each overlapping patch corresponding to the overlapping area.

[0057] Specifically, the overlapping pixel values ​​and overlapping pixel coordinates of each overlapping pixel in the overlapping area can be directly identified according to the preset feature recognition algorithm. Each overlapping pixel involves at least two defogging ICON patches. The specific preset feature recognition algorithm is not specifically limited in the embodiment of this application. Gaussian weight is a distance-based weight distribution method. The weight gradually decreases with the increase of distance, forming a smooth transition. When using Gaussian weight for weighted elimination processing, a center point can be selected from the overlapping area as the calculation basis of the Gaussian weight. This center point is generally the geometric center point of the overlapping area. For each overlapping pixel in the overlapping area, the distance to the center point is calculated according to the overlapping pixel coordinate point, and then the preset Gaussian function is used to calculate the Gaussian weight corresponding to each overlapping pixel. At the same time, all Gaussian weights are normalized. Finally, the normalized Gaussian weight and the overlapping pixel value corresponding to each overlapping pixel are used for weighted calculation to obtain the weighted pixel value corresponding to each overlapping pixel.

[0058] For the embodiments of the present application, by identifying the overlapping pixel values ​​of each pixel point in the overlapping area, the visual features of the overlapping area can be accurately quantified to ensure that the details of the overlapping area are retained. In addition, based on the overlapping patch parameter information and the overlapping pixel value, the Gaussian weight of each overlapping patch is calculated, and weighted elimination is performed based on the Gaussian weights of each overlapping patch, so as to ensure a smooth transition of the pixel values ​​in the overlapping area and avoid color mutations or edge blurring caused by simple averaging.

[0059] Step S140: identifying target classification features contained in the dehazed web page ICON image, and inputting the target classification features into a preset ICON classifier for classification to obtain a classification result.

[0060] Specifically, target classification features for classification are extracted from the dehazed web page ICON image according to a preset feature recognition algorithm. The preset feature recognition algorithm may be SIFT, HOG, or a deep learning method. The specific preset feature recognition algorithm is not specifically limited in the embodiments of this application. The extracted target classification features are input into a preset ICON classifier for classification, and the classification results of the preset ICON classifier are output. The preset ICON classifier may be a support vector machine, a random forest, or a fully connected neural network, etc. The specific preset ICON classifier may be set by relevant staff according to actual needs.

[0061] For the embodiments of the present application, by performing patch extraction on the foggy web page ICON image, it is convenient to process each area in the foggy web page ICON image separately, avoiding edge blur or detail loss caused by global dehazing, and by using the improved patch diffusion model for dehazing, it is convenient to better capture the local details and complex structures of the foggy web page ICON image, and then by integrating each dehazed ICON patch, it is convenient to eliminate the splicing traces of adjacent patch boundaries and generate a smooth transition dehazed web page ICON image. Finally, by extracting target classification features from the dehazed ICON image and combining it with a preset ICON classifier, it is convenient to improve the accuracy of classifying and identifying the foggy web page ICON image.

[0062] Furthermore, in order to improve the accuracy of determining the classification result, when identifying the target classification features contained in the dehazed web page icon image, the following may be specifically included: Identify threshold pixels and threshold pixel coordinates corresponding to each threshold pixel from the dehazed web page ICON image, where the threshold pixel is a pixel whose pixel value is higher than a preset threshold; determine the predicted direction path of the web page feature corresponding to the dehazed web page ICON image based on the coordinates of each threshold pixel; optimize the initial feature extraction strategy based on the predicted direction path to obtain the target feature extraction strategy, and identify the target classification features from the dehazed web page ICON image based on the target feature extraction strategy.

[0063] Specifically, when identifying target classification features from the defogging web page ICON image, not all pixels of the defogging web page ICON are collected and analyzed. Instead, threshold pixels and threshold pixel coordinates corresponding to the threshold pixels are first identified from the defogging web page ICON image based on a preset threshold. The pixel value corresponding to any threshold pixel is higher than the preset threshold. The specific preset threshold is not specifically limited in the embodiment of this application and can be determined by relevant staff based on historical experimental data and uploaded to the processing system. The direction path of the web page features in the defogging web page ICON image can be predicted based on the preset clustering algorithm and the coordinates of each threshold pixel. The preset clustering algorithm can be a principal component analysis algorithm, a directional histogram algorithm, etc. The specific preset clustering algorithm is not specifically limited in the embodiment of this application and can be determined by relevant staff based on historical experimental data and uploaded to the processing system.

[0064] The initial feature extraction strategy is the corresponding interval step size when collecting pixels from the defogging webpage ICON image. That is, after interval collection is performed from the defogging webpage ICON image based on the interval step size and the predicted direction path is determined, the initial feature extraction strategy can be adjusted based on the predicted direction path. For example, the interval step size of the pixels corresponding to the predicted direction path can be reduced for dense collection, and the interval compensation of the pixels outside the predicted direction path can be expanded for sparse collection. The specific adjustment value when adjusting the initial feature enhancement strategy is not specifically limited in the embodiment of this application, as long as it can reduce the redundant calculation when extracting the target classification feature while ensuring the accuracy of the classification result. The processing system includes an image input module, a patch extraction module, a defogging processing module, a patch reconstruction module, a feature extraction module, and an ICON classification module, wherein: Image input module, used to obtain foggy web page ICON images; A patch extraction module is used to extract patches from the foggy webpage ICON image to obtain multiple target ICON patches; A defogging processing module is used to input all target ICON patches corresponding to the foggy webpage ICON image into the improved patch diffusion model for defogging, so as to obtain a defogging ICON patch corresponding to each target ICON patch; The patch reassembly module is used to integrate all the dehazed ICON patches to obtain the dehazed web ICON image; A feature extraction module is used to identify target classification features contained in the dehazed web page ICON image; The ICON classification module is used to input the target classification features into a preset ICON classifier for classification to obtain a classification result.

[0065] The present application provides a processing system, such as Figure 3 As shown, Figure 3 The processing system 300 shown includes a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302. Optionally, the processing system 300 may further include a transceiver 304. It should be noted that in practical applications, the number of transceivers 304 is not limited to one, and the structure of the processing system 300 does not constitute a limitation on the embodiments of the present application.

[0066] Processor 301 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0067] Bus 302 may include a path for transmitting information between the above components. Bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 3 The fact that only one line is used does not mean that there is only one bus or one type of bus.

[0068] The memory 303 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0069] The memory 303 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the content shown in the above method embodiment.

[0070] The processing system includes, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. It may also include servers, etc. Figure 3 The processing system shown is only an example and should not limit the functions and scope of use of the embodiments of the present application.

[0071] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding contents of the aforementioned method embodiment.

[0072] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method in any of the above embodiments is implemented.

[0073] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0074] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A webpage ICON image dehazing and classification method based on patch diffusion model, characterized in that: include: Acquire a foggy webpage ICON image, and extract patches from the foggy webpage ICON image based on image features contained in the foggy webpage ICON image to obtain a plurality of target ICON patches; Input all target ICON patches corresponding to the foggy webpage ICON image into the improved patch diffusion model for defogging, and obtain the defogging ICON patch corresponding to each target ICON patch; Integrate all dehazed ICON patches to get the dehazed web ICON image; Identify target classification features contained in the dehazed webpage ICON image, and input the target classification features into a preset ICON classifier for classification to obtain a classification result.

2. A webpage ICON image dehazing and classification method based on a patch diffusion model according to claim 1, characterized in that: The extracting patches from the foggy webpage ICON image based on the image features contained in the foggy webpage ICON image to obtain a plurality of target ICON patches includes: Performing image preprocessing on the foggy web page ICON image to obtain a processed web page ICON image; Performing edge detection on the processed webpage ICON image, and setting an initial patch size and an initial repetition rate based on the edge detection result; Performing patch extraction on the processed webpage ICON image based on the initial patch size and the initial repetition rate to obtain a plurality of initial ICON patches; Semantic features are extracted for each initial ICON patch, and patches are merged based on the semantic feature extraction results corresponding to each initial ICON patch to obtain the target ICON patch.

3. The webpage ICON image defogging classification method based on the patch diffusion model according to claim 2 is characterized in that: Also includes: Identify the patch feature pixel value corresponding to each target ICON patch in the adjacent target ICON patches; Determining a repetition rate ratio between the adjacent target ICON patches based on the characteristic pixel values ​​of each patch and the initial repetition rate between the adjacent target ICON patches; Each target ICON patch among the adjacent target ICON patches is optimized based on the repetition rate ratio.

4. The webpage ICON image dehazing and classification method based on the patch diffusion model according to claim 1 is characterized in that: The improvement process of the patch diffusion model includes: Obtain sample data and classification requirements corresponding to the foggy web page ICON image; Identifying ICON priori feature information and sample noise feature information corresponding to the sample data, and determining an adaptive noise scheduling strategy based on the sample noise feature information; Determine a denoising loss term and a classification loss term based on classification requirements, and construct a joint loss function based on the denoising loss term and the classification loss term; Performing inverse diffusion improvement based on the ICON prior feature information, the sample noise feature information, and the constructed joint loss function to obtain an improved inverse diffusion model; determining a target noise schedule based on the sample data, and performing diffusion improvement based on the target noise schedule to obtain an improved diffusion model; Based on the improved inverse diffusion model and the improved diffusion model, the improvement process of the patch diffusion model is completed.

5. The webpage ICON image dehazing and classification method based on the patch diffusion model according to claim 1 is characterized in that: The defogging icon image is obtained by integrating all the defogging icon patches, including: Identify the patch parameter information corresponding to each dehazing ICON patch, which includes the patch size and patch position coordinates; Based on the patch parameter information corresponding to each dehazing ICON patch, all the dehazing ICON patches are integrated to obtain an initial integrated web page image, and overlapping areas corresponding to the initial integrated web page image are identified; Based on the overlapping defogging ICON patch corresponding to each overlapping area and the patch parameter information corresponding to each overlapping defogging ICON patch, weighted elimination processing is performed on each overlapping area to obtain the defogging web page ICON image.

6. The webpage ICON image dehazing and classification method based on the patch diffusion model according to claim 5 is characterized in that: Based on the overlapping defogging ICON patches corresponding to the overlapping areas and the patch parameter information corresponding to the overlapping defogging ICON patches, the overlapping areas are weightedly eliminated, including: Identify the overlapping pixel value and overlapping pixel coordinate point of each overlapping pixel point in the overlapping area; Determine the overlapping patch Gaussian weight of each overlapping defogging ICON patch corresponding to the overlapping area based on the overlapping patch parameter information of the overlapping defogging ICON patch corresponding to the overlapping area and the overlapping pixel value and overlapping pixel coordinate point of each overlapping pixel point; A weighted elimination process is performed on each overlapping pixel point according to the Gaussian weights of each overlapping patch corresponding to the overlapping area.

7. A processing system, characterized in that The processing system includes: at least one processor; Memory; At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute a web page ICON image dehazing and classification method based on a patch diffusion model according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that include: The invention stores a computer program that can be loaded by a processor and executes a webpage ICON image defogging classification method based on a patch diffusion model as described in any one of claims 1 to 6.

9. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the steps of a web page ICON image defogging and classification method based on a patch diffusion model according to any one of claims 1 to 6.

Images