Impurity detection method, device and equipment based on deep learning, medium and product

Abstract

The embodiment of the invention discloses an impurity detection method and device based on deep learning, equipment, a medium and a product. A specific embodiment of the method comprises the following steps: acquiring a multi-light-source image group set of renewable resources corresponding to each angle; preprocessing the multi-light-source image set to obtain a preprocessed multi-light-source image set; performing channel-level fusion on the preprocessed multi-light-source image group set to obtain a fused image group; inputting the fused image group into a pre-trained feature extraction model to obtain a multi-scale feature image group set; generating a mask data set according to the multi-scale feature map set; generating a marked image group according to the mask data group and the preprocessed multi-light-source image group set; and according to the marked image group, impurity detection information corresponding to the renewable resources is generated. According to the embodiment, efficient, accurate and intelligent impurity detection is realized by using a deep learning method.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of renewable resource processing, and a method, apparatus, device, medium, and product for impurity detection based on deep learning. Background Art

[0002] At present, with the booming development of the renewable resource recycling and reuse industry, the presence of impurities seriously affects the processing efficiency and recycling quality of renewable resources. Currently, traditional impurity detection methods mainly rely on manual inspection. However, manual inspection has significant drawbacks: on the one hand, subjective errors are inevitably present in manual inspection, and it is difficult for different inspectors to achieve completely consistent judgment criteria for impurities, which greatly reduces the objectivity and stability of the detection results. On the other hand, manual inspection is slow, difficult to meet the high-efficiency requirements of large-scale renewable resource processing, and the long-term high-intensity detection work is prone to cause manual fatigue, further reducing the detection accuracy.

[0003] Some automated detection methods have emerged in the prior art, but most of them still rely on traditional image processing methods or shallow algorithms. Traditional image processing methods often have difficulty in accurately identifying and distinguishing complex impurity types and tiny impurities. Shallow algorithms also have limitations, lacking the ability to mine deep features of data, and it is difficult to achieve efficient and accurate detection for the characteristics of various impurity forms and complex compositions in renewable resources. Moreover, such methods generally lack intelligence and self-adaptability and are difficult to adapt to the complex situations of diverse renewable resources, various impurity characteristics, and continuous changes.

[0004] The above information disclosed in this background art section is only used to enhance the understanding of the background of the concept of the present disclosure, and thus, it may include information that does not form the prior art known to those of ordinary skill in the art in this country. Summary of the Invention

[0005] This summary part of the present disclosure is used to introduce concepts in a brief form, and these concepts will be described in detail in the subsequent detailed implementation part. This summary part of the present disclosure is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

[0006] Some embodiments of the present disclosure propose a method, apparatus, electronic device, computer-readable medium, and computer program product for impurity detection based on deep learning to explain one or more of the technical problems mentioned in the above background art section.

[0007] In a first aspect, some embodiments of the present disclosure propose an impurity detection method based on deep learning. The method includes: obtaining a multi-light-source image set corresponding to each angle of the renewable resource, where each multi-light-source image set in the multi-light-source image set corresponds to an angle; preprocessing the multi-light-source image set to obtain a preprocessed multi-light-source image set, where the preprocessed multi-light-source image set includes each preprocessed multi-light-source image set; performing channel-level fusion on the preprocessed multi-light-source image set to obtain a fused image set, where each fused image in the fused image set corresponds to an angle; inputting the fused image set into a pre-trained feature extraction model to obtain a multi-scale feature map set, where each multi-scale feature map set in the multi-scale feature map set corresponds to an angle; generating a mask data set according to the multi-scale feature map set, where each mask data in the mask data set corresponds to an angle; generating an annotated image set according to the mask data set and the preprocessed multi-light-source image set, where each annotated image in the annotated image set corresponds to an angle; generating impurity detection information corresponding to the renewable resource according to the annotated image set.

[0008] In a second aspect, some embodiments of the present disclosure propose an impurity detection device based on deep learning. The device includes: an acquisition unit configured to obtain a multi-light-source image set corresponding to each angle of the renewable resource, where each multi-light-source image set in the multi-light-source image set corresponds to an angle; a preprocessing unit configured to preprocess the multi-light-source image set to obtain a preprocessed multi-light-source image set, where the preprocessed multi-light-source image set includes each preprocessed multi-light-source image set; a fusion unit configured to perform channel-level fusion on the preprocessed multi-light-source image set to obtain a fused image set, where each fused image in the fused image set corresponds to an angle; an extraction unit configured to input the fused image set into a pre-trained feature extraction model to obtain a multi-scale feature map set, where each multi-scale feature map set in the multi-scale feature map set corresponds to an angle; a first generation unit configured to generate a mask data set according to the multi-scale feature map set, where each mask data in the mask data set corresponds to an angle; a second generation unit configured to generate an annotated image set according to the mask data set and the preprocessed multi-light-source image set, where each annotated image in the annotated image set corresponds to an angle; a third generation unit configured to generate impurity detection information corresponding to the renewable resource according to the annotated image set.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device, including: one or more processors; a storage device storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the method described in any implementation manner of the above first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium storing a computer program, wherein the program, when executed by a processor, implements the method described in any implementation manner of the above first aspect.

[0011] In a fifth aspect, some embodiments of the present disclosure provide a computer program product including a computer program, which, when executed by a processor, implements the method described in any implementation manner of the above first aspect.

[0012] The above-described embodiments of the present disclosure have the following beneficial effects: Through the impurity detection method based on deep learning in some embodiments of the present disclosure, more efficient, accurate, and intelligent impurity detection can be achieved, which improves the processing efficiency and regeneration quality of recycled resources to a certain extent. Specifically: Traditional manual inspection has problems of subjective errors and slow speed, while existing automated detection methods are based on traditional image processing methods or shallow algorithms, which are difficult to accurately identify complex impurities and lack intelligence and adaptability. Based on this, in some embodiments of the present disclosure, the impurity detection method based on deep learning first obtains a multi-light-source image set corresponding to each angle of the recycled resources, where each multi-light-source image set in the above multi-light-source image set corresponds to an angle. Thus, a multi-light-source image set of each angle of the recycled resources is comprehensively obtained, providing rich information for subsequent detection and avoiding feature omission or misjudgment caused by a single perspective and light source. Then, the above multi-light-source image set is preprocessed to obtain a preprocessed multi-light-source image set, where the above preprocessed multi-light-source image set includes each preprocessed multi-light-source image set. Thus, the multi-light-source image set is preprocessed to remove noise and normalize data, improving the image quality and laying a foundation for subsequent processing. Next, channel-level fusion is performed on the above preprocessed multi-light-source image set to obtain a fused image set, where each fused image in the above fused image set corresponds to an angle. Thus, a fused image set is obtained through channel-level fusion, highlighting the impurity feature differences and enhancing the detection sensitivity and accuracy. Then, the above fused image set is input into a pre-trained feature extraction model to obtain a multi-scale feature map set, where each multi-scale feature map set in the above multi-scale feature map set corresponds to an angle. Thus, the fused image set is input into a pre-trained feature extraction model to generate a multi-scale feature map set, realizing multi-dimensional and deep-level feature mining of recycled resources and impurities and adapting to the forms and compositions of different impurities. Then, according to the above multi-scale feature map set, a mask data set is generated, where each mask data in the above mask data set corresponds to an angle. Thus, the impurities are more accurately located through the mask data. Then, according to the above mask data set and the above preprocessed multi-light-source image set, an image set with annotations is generated, where each image with annotations in the above image set with annotations corresponds to an angle. Thus, using the generated mask data set, an image set with annotations is generated in combination with the preprocessed multi-light-source image set, thereby realizing a more obvious and intuitive annotation of the impurities and providing an intuitive basis for subsequent analysis. Finally, according to the above image set with annotations, the impurity detection information corresponding to the above recycled resources is generated. Thus, the impurity detection information of the recycled resources is finally generated, providing relatively strong data support for the quality evaluation and processing decision-making of the recycled resources.In summary, the impurity detection method based on deep learning of the present disclosure overcomes the deficiencies of traditional methods to a certain extent, provides a relatively efficient, accurate and intelligent impurity detection solution for the renewable resource recycling and reuse industry, and helps to improve the processing efficiency and recycling quality of renewable resource recycling and reuse. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the elements and elements are not necessarily drawn to scale.

[0014] Figure 1 is a flowchart of some embodiments of the impurity detection method based on deep learning according to the present disclosure; Figure 2 is a schematic structural diagram of some embodiments of the impurity detection device based on deep learning according to the present disclosure; Figure 3 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0015] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0016] In addition, it should be noted that, for the sake of convenience of description, only the parts related to the relevant invention are shown in the drawings. Without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0017] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or the interdependence relationship therebetween.

[0018] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise clearly specified in the context, it should be understood as "one or more".

[0019] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0020] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0021] Figure 1 Flow 100 of some embodiments of an impurity detection method based on deep learning according to the present disclosure is shown. The impurity detection method based on deep learning includes the following steps: Step 101, obtaining a multi-light-source image set corresponding to each angle of the renewable resources.

[0022] In some embodiments, the execution subject (such as a computing device) of the impurity detection method based on deep learning can obtain a multi-light-source image set corresponding to each angle of the renewable resources. Among them, renewable resources generally refer to various waste materials, scraps, and items that are generated during the production and living processes, have lost all or part of their original use value, but can regain their use value after recycling, sorting, or processing. For example, the above-mentioned renewable resources can be a pile of waste materials, including waste metals, waste plastics, waste rubbers, or waste electrical and electronic products. The above-mentioned multi-light-source image set includes each multi-light-source image group. Each multi-light-source image group in the above-mentioned multi-light-source image set corresponds to an angle. A multi-light-source image group can be a set of images corresponding to different light sources of the above-mentioned renewable resources captured through each light source at the same angle. The above-mentioned each light source can include a visible light source, a near-infrared light source, and a ultraviolet light source. The multi-light-source image group includes a visible light image, a near-infrared single-channel image, and a ultraviolet single-channel image. In practice, first, a high-definition industrial camera can be used to capture the above-mentioned renewable resources at any angle through a visible light source, a near-infrared light source, and a ultraviolet light source to obtain a visible light acquisition image, a near-infrared acquisition image, and a ultraviolet light acquisition image corresponding to this angle. Then, the visible light acquisition image, the near-infrared acquisition image, and the ultraviolet light acquisition image corresponding to this angle are determined as the visible light image, the near-infrared single-channel image, and the ultraviolet single-channel image corresponding to this angle. Next, the visible light image, the near-infrared single-channel image, and the ultraviolet single-channel image corresponding to this angle are integrated into a multi-light-source image group corresponding to this angle. According to this method, each multi-light-source image group corresponding to each angle can be obtained. Finally, the above-mentioned each multi-light-source image group is integrated into a multi-light-source image set. For example, the above-mentioned high-definition industrial camera can be selected from Sony IMX series or Basler cameras, etc.

[0023] Optionally, the above-mentioned execution subject can also perform the following steps: Step 1: Perform a forward transformation on each visible light acquisition image corresponding to the above-mentioned renewable resources obtained through a visible light source to obtain each image to be corrected. Among them, each visible light acquisition image corresponds to each of the above-mentioned angles. The above-mentioned forward transformation can be a transformation operation that converts an RGB-format image into an HSV-format image. For example, perform pixel-by-pixel conversion using the RGB-to-HSV formula. In practice, the above-mentioned execution entity can perform a forward transformation on each visible light acquisition image to obtain each forward transformation image in the HSV format corresponding to each visible light acquisition image. Then, determine each forward transformation image in the HSV format corresponding to each visible light acquisition image as each image to be corrected.

[0024] Step 2: Perform correction processing on each of the above-mentioned images to be corrected to obtain each corrected image. Among them, the above-mentioned correction processing can be an adaptive gamma correction based on the local mean for the V (brightness) channel of the HSV image. In practice, for each image to be corrected among the above-mentioned images to be corrected, the above-mentioned execution entity can extract the V channel from the image to be corrected. Then, for each pixel point in the V-channel image, calculate the brightness mean of the neighborhood centered on this pixel point to obtain the brightness mean of the neighborhood of this pixel point. For pixel points at the image edge where a complete neighborhood cannot be formed, the strategy of copying the outermost pixel value can be used to approximately calculate the brightness mean of the neighborhood. Then, according to the preset gamma value generation formula, calculate the gamma value applicable to this pixel point. Next, use the gamma value applicable to this pixel point and the brightness mean of the neighborhood of this pixel point to generate the corrected brightness value of this pixel point. In this way, the corrected brightness values of each pixel point in the image to be corrected can be obtained. Finally, integrate the corrected brightness values of each pixel point into the updated V channel, and combine the above-mentioned updated V channel, the original H channel, and the original S channel into the corrected image corresponding to the image to be corrected. Therefore, through the above correction processing method, each image to be corrected can be corrected to obtain each corrected image. For example, the brightness mean of the neighborhood centered on this pixel point can be the brightness mean of the 8 surrounding pixels centered on this pixel point. The above-mentioned preset gamma value generation formula can be: γ = 1.5 - L, where the above-mentioned γ is the gamma value applicable to this pixel point, and the above-mentioned L is the brightness mean of the neighborhood of this pixel point. When generating the corrected brightness value of this pixel point, the above-mentioned execution entity can use the gamma value applicable to this pixel point as the exponent of the brightness mean of the neighborhood of this pixel point, and the corrected brightness value of this pixel point can be obtained through calculation.

[0025] Step 3: Perform inverse transformation on each of the above corrected images to obtain each visible light image. Among them, the above inverse transformation can be a transformation operation that converts an image in HSV format to an image in RGB format. For example, use the mathematical formula for HSV to RGB conversion for pixel-by-pixel conversion. Each of the above visible light images corresponds to each of the above angles. In practice, the above execution entity can perform inverse transformation on each of the above corrected images to obtain each inverse transformation image in RGB format corresponding to each of the above corrected images. Then, determine each inverse transformation image in RGB format corresponding to each of the above corrected images as each visible light image.

[0026] Step 102: Preprocess the multi-light source image set to obtain a preprocessed multi-light source image set.

[0027] In some embodiments, the above execution entity can preprocess the above multi-light source image set to obtain a preprocessed multi-light source image set.

[0028] In some optional implementation manners of some embodiments, the above execution entity can preprocess the above multi-light source image set through the following steps to obtain a preprocessed multi-light source image set: Step 1: Perform image enhancement processing on the above multi-light source image set to obtain an enhanced multi-light source image set. Among them, the above enhancement processing can be to perform image enhancement processing on each visible light image included in each multi-light source image group in the above multi-light source image set. For example, use the CLAHE algorithm to enhance the image contrast of each visible light image, where the contrast limit amplitude parameter is set to 2 and the grid size parameter is 8 8. In practice, the above execution entity can perform image enhancement processing on each visible light image included in each multi-light source image group, and use each visible light image after image enhancement processing to update each visible light image in each multi-light source image group to obtain each multi-light source image group after image enhancement processing. Finally, integrate each multi-light source image group after the above image enhancement processing into an enhanced multi-light source image set.

[0029] Step 2: Perform denoising processing on the above enhanced multi-light source image set to obtain a preprocessed multi-light source image set. Among them, the above preprocessed multi-light source image set includes each preprocessed multi-light source image group. The above denoising processing can be to perform denoising processing on each multi-light source image group after the above image enhancement processing. For example, on the one hand, the median filter or Gaussian filter method can be used to perform denoising processing on each visible light image in each multi-light source image group after the above image enhancement processing. Among them, the window size of the median filter and the kernel size of the Gaussian filter are both 5 5. Edge filling is performed using the outermost pixel points. On the other hand, the wavelet transform method can be used to denoise each near-infrared single-channel image and each ultraviolet single-channel image in each multi-light-source image group after the above image enhancement processing. Among them, the parameters of the wavelet transform can be configured as the basis function "db4", the decomposition level is 3, and the BayesShrink soft threshold method is used. In practice, the above execution entity can denoise each multi-light-source image group after the above image enhancement processing to obtain each multi-light-source image group after denoising processing. Then, each multi-light-source image group after the above denoising processing is determined as each preprocessed multi-light-source image group. Among them, each preprocessed multi-light-source image group corresponds to an angle. Each preprocessed multi-light-source image group corresponding to an angle includes a preprocessed visible light image, a preprocessed near-infrared single-channel image, and a preprocessed ultraviolet single-channel image corresponding to the angle. Finally, the above preprocessed multi-light-source image groups are integrated into a set of preprocessed multi-light-source image groups.

[0030] Step 103: Perform channel-level fusion on the set of preprocessed multi-light-source image groups to obtain a fused image group.

[0031] In some embodiments, the above execution entity can perform channel-level fusion on the set of preprocessed multi-light-source image groups to obtain a fused image group.

[0032] In some optional implementation manners of some embodiments, the above execution entity can perform channel-level fusion on the set of preprocessed multi-light-source image groups through the following steps to obtain a fused image group: Step 1: For each preprocessed multi-light-source image group corresponding to each angle in the set of preprocessed multi-light-source image groups, perform the following steps: Sub-step 1: Recombine the channels of the pre-processed multi-light-source image group corresponding to the above angle to obtain a channel tensor group. Among them, the above channel recombination can be a process of converting the pre-processed multi-light-source image group corresponding to any angle into a standardized four-dimensional data structure. The above channel recombination includes single-channel replication and dimension stacking. The above channel tensor group can be a four-dimensional tensor data structure formed by integrating each channel tensor. The above channel tensor group can include each channel tensor corresponding to each of the above light sources. In practice, first, the above execution entity can copy the pre-processed near-infrared single-channel image in the pre-processed multi-light-source image group corresponding to the above angle into a three-channel image to obtain an extended near-infrared image. Among them, each channel value of the above extended near-infrared image is the same as the original single-channel value. Then, the pre-processed ultraviolet single-channel image in the pre-processed multi-light-source image group corresponding to the above angle can be copied into a three-channel image to obtain an extended ultraviolet image. Among them, each channel value of the above extended ultraviolet image is the same as the original single-channel value. Next, the channel tensor corresponding to the pre-processed visible light image, the channel tensor corresponding to the above extended near-infrared image, and the channel tensor corresponding to the above extended ultraviolet image in the pre-processed multi-light-source image group corresponding to the above angle can be stacked along a predefined dimension to form a four-dimensional tensor data structure, that is, the channel tensor group. Among them, the above predefined dimension can be: [image height, image width, RGB channel, light source type]. The order of the light source type can be configured as: visible light (index 0), near-infrared (index 1), ultraviolet (index 2).

[0033] Sub-step 2: Normalize each channel tensor in the above channel tensor group to obtain a normalized channel tensor group. Among them, the above normalization can be to convert each channel tensor in the above channel tensor group into a zero-centered distribution, and the numerical range is compressed to a stable interval. For example, use the Z-score normalization method to process each channel tensor in the above channel tensor group. In practice, the above execution entity can normalize each channel tensor in the above channel tensor group to obtain each normalized channel tensor. Then, the above each normalized channel tensor is integrated into a normalized channel tensor group. Among them, the above each normalized channel tensor can include a normalized visible light channel tensor, a normalized near-infrared channel tensor, and a normalized ultraviolet channel tensor.

[0034] Sub-step 3: Perform weighted channel fusion on the above normalized channel tensor group to obtain the fusion image corresponding to the above angle. Among them, the fusion image can be a single enhanced RGB image obtained by performing weighted channel fusion on the normalized channel tensor group corresponding to any angle.

[0035] Step 2: Integrate the fused images corresponding to the above respective angles into a fused image group. Each fused image in the above fused image group corresponds to one angle. In practice, according to the above method, the fused images corresponding to the above respective angles can be obtained. Then, integrate the fused images corresponding to the above respective angles into a fused image group.

[0036] In some alternative implementation manners of some embodiments, the above execution subject may perform weighted channel fusion on the above normalized channel tensor group through the following steps to obtain the fused image corresponding to the above angle: First step: Perform reflection characteristic detection on the preprocessed visible light image corresponding to the above angle in the above preprocessed multi-light-source image group to obtain the metal area corresponding to the above angle and the non-metal area corresponding to the above angle. The above reflection characteristic detection may be a method for detecting the reflection of light on the surface of renewable resources in the preprocessed visible light image, and then distinguishing the metal area from the non-metal area. For example, a joint detection method based on HSV color space threshold segmentation and local binary pattern texture analysis. In practice, the above execution subject may perform reflection characteristic detection on the preprocessed visible light image corresponding to the above angle according to the preset determination conditions for the metal area and the non-metal area, to obtain the metal area and the non-metal area on the preprocessed visible light image corresponding to the above angle. And determine the metal area and the non-metal area on the preprocessed visible light image corresponding to the above angle as the metal area corresponding to the above angle and the non-metal area corresponding to the above angle respectively. The above preset determination conditions for the metal area and the non-metal area may include a metal area determination condition and a non-metal area determination condition. For example, the above metal area determination condition may be: the low color saturation is less than 0.3 and the high brightness gradient is greater than 30; the above non-metal area determination condition may be: the warm color hue is in the closed interval of 40° to 80° and the low texture variance is less than 15.

[0037] Second step: Generate the respective area proportion coefficients corresponding to the above angle based on the metal area corresponding to the above angle and the non-metal area corresponding to the above angle. The respective area proportion coefficients corresponding to the above angle include a metal area proportion coefficient and a non-metal area proportion coefficient. In practice, the above execution subject may integrate the metal area corresponding to the above angle and the non-metal area corresponding to the above angle into an effective area. Then, respectively calculate the proportion of the metal area corresponding to the above angle in the above effective area, and use it as the metal area proportion coefficient. Similarly, the non-metal area proportion coefficient can be obtained. Finally, determine the metal area proportion coefficient and the non-metal area proportion coefficient as the respective area proportion coefficients corresponding to the above angle.

[0038] Step 3: According to the proportion coefficients of each region corresponding to the above angles and the predefined dynamic weight fusion conditions, generate the weight coefficients corresponding to each normalized channel tensor in the above normalized channel tensor group. Among them, the above predefined dynamic weight fusion conditions can be the judgment conditions for the weight coefficients corresponding to each normalized channel tensor set in advance. For example, the above predefined dynamic weight fusion conditions can be set as follows: when the proportion coefficient of the metal region is greater than 40%, the weight coefficient corresponding to the normalized near-infrared channel tensor is 0.7, the weight coefficient corresponding to the normalized ultraviolet channel tensor is 0.1, and the weight coefficient corresponding to the normalized visible light channel tensor is 0.2; when the proportion coefficient of the non-metal region is greater than 60%, the weight coefficient corresponding to the normalized ultraviolet channel tensor is 0.6, the weight coefficient corresponding to the normalized visible light channel tensor is 0.3, and the weight coefficient corresponding to the normalized near-infrared channel tensor is 0.1; when the proportion coefficient of the metal region is less than or equal to 40% and the proportion coefficient of the non-metal region is less than or equal to 60%, the weight coefficient corresponding to the normalized near-infrared channel tensor is 0.4, the weight coefficient corresponding to the normalized ultraviolet channel tensor is 0.4, and the weight coefficient corresponding to the normalized visible light channel tensor is 0.2. In practice, the above execution entity can obtain the weight coefficients corresponding to the normalized visible light channel tensor, the normalized near-infrared channel tensor, and the normalized ultraviolet channel tensor in the above normalized channel tensor group as the weight coefficients corresponding to each normalized channel tensor in the above normalized channel tensor group according to the above predefined dynamic weight fusion conditions and the proportion coefficients of each region corresponding to the above angles.

[0039] Step 4: According to the weight coefficients corresponding to each of the above normalized channel tensors, perform channel enhancement processing on the above normalized channel tensor group to obtain an enhanced channel tensor group. In practice, the above execution entity can perform channel enhancement processing on each normalized channel tensor in the above normalized channel tensor group according to the weight coefficients corresponding to each of the above normalized channel tensors to obtain each enhanced channel tensor. Then, integrate the above enhanced channel tensors into an enhanced channel tensor group. For example, the above execution entity can perform Laplacian sharpening on the normalized channel tensor with a weight coefficient greater than 0.5 and perform Gaussian noise reduction on the normalized channel tensor with a weight coefficient less than 0.3. Thus, each enhanced channel tensor can be obtained.

[0040] In the fifth step, based on the respective weight coefficients corresponding to the above-mentioned normalized channel tensors, the above-mentioned enhanced channel tensor group is weighted and fused to obtain the fused image corresponding to the above-mentioned angle. In practice, the above-mentioned execution entity may perform weighted summation on the respective channels corresponding to the respective enhanced channel tensors in the above-mentioned enhanced channel tensor group according to the respective weight coefficients corresponding to the above-mentioned normalized channel tensors to obtain the fused image corresponding to the above-mentioned angle.

[0041] The first to fifth steps of the embodiments of the present disclosure are an inventive point of the embodiments of the present disclosure, which solve the technical problem that "traditional multi-light-source image fusion technology is difficult to perform adaptive channel fusion according to the characteristics of image material regions, resulting in poor quality of fused images, inability to accurately distinguish metal and non-metal regions, and inability to reasonably utilize the advantages of each channel tensor". The prior art has the following deficiencies in multi-light-source image fusion processing: on the one hand, it is difficult to accurately distinguish metal regions and non-metal regions in images; on the other hand, during the image fusion process, the weights of each channel tensor cannot be dynamically adjusted according to the proportion of metal and non-metal regions, resulting in defects in the details, contrast, and overall visual effect of the fused image, and usually difficult to meet the requirements of high-quality image fusion. If the above problems are solved, the effects of improving the quality of image fusion, enhancing the detail performance of the fused image, and achieving adaptive channel fusion can be achieved. To achieve this effect, the present disclosure proposes a multi-light-source image fusion method based on material region adaptability. The specific steps are as follows: The first step is to perform reflection characteristic detection on the preprocessed visible light images corresponding to the above angles in the preprocessed multi-light-source image group set, to obtain the metal region corresponding to the above angle and the non-metal region corresponding to the above angle, laying a foundation for subsequent fusion processing. The second step is to generate a metal region proportion coefficient and a non-metal region proportion coefficient based on the metal region and non-metal region obtained in the first step, realizing a quantitative description of the distribution of different material regions, and providing key data support for subsequent dynamic allocation of channel weights. The third step is to determine the weight coefficients corresponding to each normalized channel tensor in the normalized channel tensor group according to the predefined dynamic weight fusion conditions, in combination with the metal region proportion coefficient and the non-metal region proportion coefficient obtained in the second step. The purpose of dynamically adjusting the weights of each channel according to the proportion coefficients of different material regions is achieved. The fourth step is to perform channel enhancement processing on the normalized channel tensor group according to the weight coefficients corresponding to each normalized channel tensor, to obtain an enhanced channel tensor group, further improving the image quality. The fifth step is to perform weighted summation on each enhanced channel tensor in the enhanced channel tensor group according to the respective weight coefficients corresponding to each normalized channel tensor, finally obtaining the fused image corresponding to the above angle. In summary, the first to fifth steps of the embodiments of the present disclosure cooperate with each other, starting from distinguishing metal and non-metal regions, quantifying the region proportion, dynamically determining the channel weight coefficients, enhancing the channel tensors, and finally weighted fusion to generate high-quality images, realizing the adaptive collaborative optimization from region division to image fusion. By integrating the advantages of each channel tensor in imaging different material regions, the quality and effect of multi-light-source image fusion are improved, and to a certain extent, the requirements for high-quality image fusion in complex multi-light-source scenarios are met.

[0042] Step 104: Input the fused image group into a pre-trained feature extraction model to obtain a multi-scale feature map group set.

[0043] In some embodiments, the above-mentioned execution entity may input the above-mentioned fused image group into a pre-trained feature extraction model to obtain a multi-scale feature map set. Among them, the above-mentioned pre-trained feature extraction model may be a deep convolutional neural network pre-trained on a large-scale general dataset. The above-mentioned pre-trained feature extraction model may extract multi-scale semantic features from the input fused images through hierarchical convolution operations. Each multi-scale feature map group in the above-mentioned multi-scale feature map set corresponds to an angle. For example, the above-mentioned large-scale general dataset may include 200,000 pictures with an unbalanced class distribution, such as pictures with more paint, tape, and oil stains, and fewer asphalt and foam. The above-mentioned pre-trained feature extraction model may be a model based on the InternImage architecture. Each multi-scale feature map group in the above-mentioned multi-scale feature map set may include feature maps of 4 scales. In practice, the above-mentioned execution entity may input each fused image in the above-mentioned fused image group into the pre-trained feature extraction model to obtain feature maps of each scale corresponding to each angle. Then, the feature maps of each scale corresponding to each angle are integrated into each multi-scale feature map group. Finally, the above-mentioned multi-scale feature map groups are integrated into a multi-scale feature map set.

[0044] Step 105, generate a mask data set according to the multi-scale feature map set.

[0045] In some embodiments, the above-mentioned execution entity may generate a mask data set according to the above-mentioned multi-scale feature map set.

[0046] In some optional implementation manners of some embodiments, the above-mentioned execution entity may generate a mask data set according to the above-mentioned multi-scale feature map set through the following steps: Step 1, for each multi-scale feature map group corresponding to each angle in the above-mentioned multi-scale feature map set, perform the following steps: Sub-step 1, perform multi-scale fusion processing on the multi-scale feature map group corresponding to the above-mentioned angle to obtain an enhanced feature map. Among them, the above-mentioned multi-scale fusion processing may be a method of fusing feature maps of each scale corresponding to any angle. For example, the above-mentioned multi-scale fusion processing may be a feature pyramid fusion method. In practice, the above-mentioned execution entity may perform multi-scale fusion processing on the feature maps of each scale in the multi-scale feature map group corresponding to the above-mentioned angle to obtain a multi-scale fused feature map. Then, the above-mentioned multi-scale fused feature map is determined as the enhanced feature map corresponding to the above-mentioned angle.

[0047] Sub-step 2: Generate a group of candidate regions based on the above enhanced feature map. Among them, the above group of candidate regions includes each candidate region. The candidate region can be an area with impurities. The candidate region can be characterized as a bounding box containing the position of the impurities. In practice, the above execution entity can make predictions at each position in the above enhanced feature map to obtain each candidate region. Then, integrate the above candidate regions into a group of candidate regions. For example, the above execution entity can use a Region Proposal Network (RPN) to make predictions at each position in the above enhanced feature map to obtain each candidate region.

[0048] Sub-step 3: Screen the above group of candidate regions according to the pre-set region screening conditions to obtain a screened group of candidate regions. Among them, the above pre-set region screening conditions can be that the overlap degree of the candidate box of any candidate region is less than or equal to the pre-set overlap degree threshold. The overlap degree can be the proportion of the overlapping part of any candidate box and other candidate boxes in this candidate box. As an example, the pre-set overlap degree threshold can be 0.7. In practice, the above execution entity can screen out each candidate region that meets the above pre-set region screening conditions in the above group of candidate regions according to the above pre-set region screening conditions. Then, determine the above candidate regions that meet the above pre-set region screening conditions as each screened candidate region. Finally, integrate the above screened candidate regions into a screened group of candidate regions. For example, the above execution entity can use the method of Non-Maximum Suppression (NMS) to screen the above candidate regions using the above pre-set overlap degree threshold. Scale constraints can also be introduced, such as removing candidate regions with a width and height less than 8 pixels.

[0049] Sub-step 4: Perform feature alignment on the above screened group of candidate regions to obtain a first group of feature regions. Among them, the above feature alignment can be a method of mapping each of the above screened candidate regions from the image space to the feature space and generating feature blocks. For example, the above feature alignment can be the ROI Align method. The above first group of feature regions includes each first feature region. Each of the above first feature regions corresponds one-to-one with each of the above screened candidate regions. The first feature region can be a standardized feature block output after feature alignment of the corresponding screened candidate region. The first feature region can be the expression of the corresponding screened candidate region in the feature space. In practice, the above execution entity can perform feature alignment on each of the above screened candidate regions in the above screened group of candidate regions to obtain each first feature region. Then, integrate the above first feature regions into a group of feature regions.

[0050] Sub-step five: Generate an adjustment information group according to the above-mentioned first feature region group. Among them, the above-mentioned adjustment information group includes each adjustment information. Each adjustment information corresponds to each first candidate region one by one. The adjustment information includes a classification result and bounding box adjustment parameters. The classification result is the impurity category corresponding to any first candidate region. The bounding box adjustment parameters can be a set of geometric offset data used to adjust the position and size of the corresponding filtered candidate region. In practice, the above-mentioned execution entity can process each first feature region in the above-mentioned first feature region group through a classification result generation method to obtain the classification result corresponding to each first feature region. The bounding box adjustment parameters corresponding to each first feature region in the above-mentioned first feature region group can be obtained by processing each first feature region in the above-mentioned first feature region group through a bounding box adjustment parameter generation method. Then, the classification results corresponding to the above-mentioned each first feature region and the bounding box adjustment parameters corresponding to the above-mentioned each first feature region are determined as the respective adjustment information corresponding to each first feature region. Then, the above-mentioned each adjustment information is integrated into an adjustment information group. As an example, the above-mentioned classification result generation method can be a method combining a deep fully connected network and category probability modeling. The above-mentioned bounding box adjustment parameter generation method can be a method combining a regression network and geometric transformation modeling.

[0051] Sub-step six: Adjust the above-mentioned filtered candidate region group according to the above-mentioned adjustment information group to obtain an adjusted candidate region group. In practice, the above-mentioned execution entity can use the above-mentioned each adjustment information to adjust the above-mentioned each filtered candidate region to obtain each adjusted candidate region. Then, the above-mentioned each adjusted candidate region is integrated into an adjusted candidate region group. For example, the above-mentioned execution entity can use the classification results corresponding to the above-mentioned each first feature region to screen the above-mentioned each filtered candidate region, and remove the each filtered candidate region corresponding to each first feature region whose corresponding classification result probability is less than 0.5 from the above-mentioned filtered candidate region group to obtain a preliminarily adjusted filtered candidate region group. Then, using the respective bounding box adjustment parameters corresponding to each filtered candidate region in the above-mentioned preliminarily adjusted filtered candidate region group, geometric refinement is performed on each filtered candidate region in the above-mentioned preliminarily adjusted filtered candidate region group, such as center point adjustment and size adjustment, to obtain each adjusted candidate region. Finally, the above-mentioned each adjusted candidate region is integrated into an adjusted candidate region group.

[0052] Sub-step seven: Generate initial mask data based on the above-mentioned adjusted candidate region group. Among them, the above-mentioned initial mask data includes the respective segmentation masks corresponding to the above-mentioned adjusted candidate regions. In practice, the above-mentioned execution entity can perform convolution processing on the respective first feature regions corresponding to the above-mentioned adjusted candidate regions to obtain respective mask matrices. Then, the coordinate space transformation technology is used for the respective mask matrices and the above-mentioned adjusted candidate regions to obtain the respective segmentation masks corresponding to the above-mentioned adjusted candidate regions. Finally, the above-mentioned respective segmentation masks are integrated into initial mask data.

[0053] Sub-step eight: Optimize the above-mentioned initial mask data to obtain the mask data corresponding to the above-mentioned angle. Among them, the above-mentioned optimization process can be a method for edge optimization processing of the above-mentioned respective segmentation masks. For example, the above-mentioned optimization process can be a conditional random field optimization (CRF) processing method or an edge smoothing processing method. In practice, the above-mentioned execution entity can perform optimization processing on the respective segmentation masks in the above-mentioned initial mask data to obtain respective optimized segmentation masks. Finally, the above-mentioned respective optimized segmentation masks are integrated into the mask data corresponding to the above-mentioned angle. As an example, the above-mentioned execution entity can use morphological closing operation processing and hole filling in the edge smoothing processing method to optimize the respective segmentation masks in the above-mentioned initial mask data.

[0054] Step two: Integrate the respective mask data corresponding to the above-mentioned respective angles into a mask data group. Among them, each mask data in the above-mentioned mask data group corresponds to an angle. In practice, according to the above method, the respective mask data corresponding to the respective angles can be obtained. Then, the respective mask data corresponding to the above-mentioned respective angles are integrated into a mask data group.

[0055] Step 106: Generate an image group with annotations according to the mask data group and the preprocessed multi-light source image group set.

[0056] In some embodiments, the above-mentioned execution entity may generate an image group with annotations according to the above-mentioned mask data group and the above-mentioned preprocessed multi-light-source image group set. Each image with an annotation in the above-mentioned image group with annotations corresponds to an angle. In practice, the above-mentioned execution entity may, according to each angle corresponding to each mask data in the above-mentioned mask data group, first, for each pixel in any preprocessed visible light image, determine the mask value of the pixel point corresponding to the above-mentioned pixel in the corresponding mask data, and update the pixel value of the above-mentioned pixel to the above-mentioned mask value. In this way, the pixel values of each pixel in each preprocessed visible light image can be updated to obtain each updated visible light image. Then, according to each classification result of each first feature region corresponding to each segmentation mask in each mask data, each segmentation mask in each mask data is annotated on the above-mentioned updated visible light images to obtain each image with an annotation. Finally, the above-mentioned images with annotations are integrated into an image group with annotations.

[0057] Step 107: Generate impurity detection information corresponding to the renewable resources according to the image group with annotations.

[0058] In some embodiments, the above-mentioned execution entity may generate impurity detection information corresponding to the above-mentioned renewable resources according to the above-mentioned image group with annotations. Among them, the above-mentioned impurity detection information includes the types, positions, area ratios, treatment suggestions of the impurities corresponding to the above-mentioned renewable resources, and the above-mentioned image group with annotations. The types of the above-mentioned impurities may be the respective categories corresponding to the respective classification results. The positions of the above-mentioned impurities may be the centroid coordinates of the respective segmentation masks in the respective annotated images. The area ratio of the above-mentioned impurities may be the area ratio corresponding to the respective classification results. The above-mentioned treatment suggestions may be pre-set text prompt information representing treatment suggestions for impurities. For example, the above-mentioned treatment suggestions may be pre-set as follows: for goods containing explosive items (such as sealed tanks, etc.), it is recommended to conduct manual verification of the goods; for metal impurities, it is recommended to recycle, sort, or further classify; for non-metal impurities, it is recommended to conduct further experimental detection or recycling treatment. In practice, first, the above-mentioned execution entity may calculate the respective area ratios of the respective segmentation masks in the annotated images corresponding to each angle. Then, according to the respective classification results corresponding to the respective segmentation masks in each mask data, the respective area ratios of the respective segmentation masks corresponding to the same classification result in the annotated images corresponding to each angle are summed to obtain the respective area ratios corresponding to the respective classification results in each angle. Next, the respective area ratios corresponding to the same classification result in each angle are averaged according to the categories of the classification results to obtain the area ratios of the impurities corresponding to the respective classification results. In addition, based on the camera calibration parameters, the 2D coordinate data of the respective segmentation masks in the respective annotated images may be converted into 3D physical coordinate data, and the centroid coordinates of the respective segmentation masks in the respective annotated images may be obtained. Finally, the respective categories corresponding to the respective classification results, the centroid coordinates of the respective segmentation masks in the respective annotated images, the area ratios of the impurities corresponding to the respective classification results, the pre-set treatment suggestions, and the above-mentioned image group with annotations are integrated into the impurity detection information corresponding to the above-mentioned renewable resources.

[0059] Optionally, the above-mentioned execution entity may further perform the following steps: In the first step, according to the above-mentioned mask data group and the respective ultraviolet single-channel images in the above-mentioned multi-light-source image group set, generate the respective mask regions corresponding to the respective ultraviolet single-channel images. Among them, the above-mentioned respective ultraviolet single-channel images correspond to the above-mentioned respective angles. In practice, the above-mentioned execution entity may map the respective mask data corresponding to the above-mentioned respective angles onto the above-mentioned respective ultraviolet single-channel images according to the pixel-level correspondence relationship to obtain the respective mask regions corresponding to the above-mentioned respective ultraviolet single-channel images.

[0060] Second, from each mask region corresponding to each of the above ultraviolet single-channel images, extract the ultraviolet reflection intensity data corresponding to each of the above angles. Among them, the above ultraviolet reflection intensity data can be each spectral curve representing the reflection intensity of each ultraviolet single-channel image on the corresponding mask region. In practice, the above execution entity can use an ultraviolet reflection intensity data extraction method to extract the ultraviolet reflection intensity data corresponding to each of the above angles from each mask region corresponding to each of the above ultraviolet single-channel images. For example, the above ultraviolet reflection intensity data extraction method can be a region statistical analysis method or a combined method of block integration algorithm and characteristic spectrum extraction.

[0061] Third, perform denoising processing on the ultraviolet reflection intensity data corresponding to each of the above angles to obtain the denoised spectra corresponding to each of the above angles. Among them, the above denoising processing can be a method for denoising spectral data. For example, a combined method of wavelet denoising, SG filtering, and baseline correction. In practice, the above execution entity can perform spectral denoising on the ultraviolet reflection intensity data corresponding to each of the above angles to obtain the denoised spectra corresponding to each of the above angles.

[0062] Fourth, perform multi-angle spectral fusion on the denoised spectra corresponding to each of the above angles to obtain an average spectrum and a central wavelength. Among them, the above multi-angle spectral fusion can be a method for fusing the denoised spectra corresponding to each of the above angles. For example, the above multi-angle spectral fusion can be a combined method of weighted Gaussian spectral fusion and double-peak collaborative determination. In practice, the above execution entity can perform weighted fusion on the denoised spectra corresponding to each of the above angles to obtain an average spectrum. Then, using the above average spectrum, the centroid wavelength and peak wavelength can be obtained. Next, the centroid wavelength and peak wavelength are weighted and fused according to a predefined wavelength weight distribution rule to obtain a central wavelength. As an example, the above predefined wavelength weight distribution rule can be: the wavelength weight of the centroid wavelength is 0.6, and the wavelength weight of the peak wavelength is 0.4.

[0063] Fifth, generate a parameter group according to the above average spectrum and the above central wavelength. Among them, the above parameter group includes multi-angle reflectance and reflectance volatility. In practice, the above execution entity can use the above average spectrum and the above central wavelength for calculation, and generate multi-angle reflectance and reflectance volatility after solving. Then, the above multi-angle reflectance and the above reflectance volatility are integrated into a parameter group. For example, the above execution entity can use the regional reflectance feature quantization method to solve the above average spectrum and the above central wavelength, and finally generate a parameter group.

[0064] Step 6: According to the predefined mask screening conditions, screen out the target mask data from the above mask data group. Among them, the above predefined mask screening conditions can be the conditions predefined for screening the target mask data. For example, the above predefined mask screening conditions can be the mask data with the largest area in the above mask data group. In practice, the above execution entity can screen out the mask data that meets the above predefined mask screening conditions from the above mask data group according to the above predefined mask screening conditions, and determine it as the target mask data.

[0065] Step 7: Perform morphological dynamics processing on the above target mask data to obtain the spatial gradient magnitude. Among them, the above morphological dynamics processing can be a method for generating the spatial gradient magnitude. For example, the above morphological dynamics processing can be the distance transform gradient magnitude calculation method. In practice, the above execution entity can perform morphological dynamics processing on the above target mask data to obtain the spatial gradient magnitude.

[0066] Step 8: Input the above multi-angle reflectance, the reflectance volatility, and the above spatial gradient magnitude into a pre-constructed toxicity detection model to obtain a risk coefficient. Among them, the above pre-constructed toxicity detection model can be a linear regression model with the multi-angle reflectance, the reflectance volatility, and the spatial gradient magnitude as inputs and the risk coefficient as the output. For example, the above toxicity detection model can be a gradient boosting decision tree (GBDT). The above risk coefficient can be a quantitative index characterizing the degree of harm in the impurity. In practice, the above execution entity can input the above multi-angle reflectance, the above reflectance volatility, and the above spatial gradient magnitude into the pre-constructed toxicity detection model, and obtain the risk coefficient through linear regression prediction and normalization output processing in the above pre-constructed toxicity detection model.

[0067] Step 9: Generate a risk level label according to the predefined risk level label generation conditions and the above risk coefficient. Among them, the above predefined risk level label generation conditions can be the conditions predefined for generating the risk level label corresponding to the risk coefficient according to the size of the risk coefficient. For example, the above predefined risk level label generation conditions can be set as follows: when the risk coefficient is greater than or equal to 0 and less than 0.3, generate a risk level label of "safe"; when the risk coefficient is greater than or equal to 0.3 and less than 0.6, generate a risk level label of "low risk"; when the risk coefficient is greater than or equal to 0.6 and less than 0.8, generate a risk level label of "medium risk"; when the risk coefficient is greater than or equal to 0.8 and less than or equal to 1, generate a risk level label of "high risk". In practice, the above execution entity can generate a risk level label according to the size of the above risk coefficient according to the above predefined risk level label generation conditions.

[0068] Step 10: Add the above-mentioned hazard level label to the above-mentioned impurity detection information. In practice, after generating the above-mentioned hazard level label, the above-mentioned execution entity may add the above-mentioned hazard level label to the above-mentioned impurity detection information.

[0069] The first step to the tenth step of the embodiments of the present disclosure are an inventive point of the embodiments of the present disclosure, which solve the technical problem that "traditional toxicity detection techniques are difficult to comprehensively determine the toxicity accurately from multiple perspectives, multi-dimensional spectral features and spatial information, resulting in inaccurate classification of danger levels and inability to effectively meet the needs of complex impurity detection". The prior art has the following deficiencies in toxicity detection: on the one hand, only analyzing based on single spectral data or limited features, it is impossible to comprehensively integrate multi-angle spectral information and spatial features, and it is difficult to accurately quantify the harm degree of impurities; on the other hand, the lack of effective screening of mask data and processing of spatial dimension features leads to low accuracy in classifying danger levels and inability to provide a reliable basis for subsequent decision-making. If the above problems are solved, the effects of improving the accuracy of toxicity detection, optimizing the accuracy of danger level classification, and realizing multi-dimensional feature fusion detection can be achieved. To achieve this effect, the present disclosure proposes an impurity toxicity detection method based on multi-dimensional feature fusion. The specific steps are as follows: The first step is to map the mask data corresponding to each angle to the corresponding ultraviolet single-channel image based on the pixel-level correspondence relationship for each ultraviolet single-channel image in the preprocessed mask data group and multi-light source image group, and generate a mask region corresponding to each ultraviolet single-channel image. Thus, it lays a foundation for subsequent extraction of spectral data for different regions. The second step is to extract the ultraviolet reflection intensity data corresponding to each angle from each mask region generated in the first step by using the ultraviolet reflection intensity data extraction method. Thus, it provides an accurate spectral feature basis for subsequent spectral processing. The third step is to perform denoising processing on the ultraviolet reflection intensity data corresponding to each angle obtained in the second step to obtain the denoised spectrum corresponding to each angle. Thus, it improves the quality and reliability of spectral data and creates a good data basis for subsequent spectral fusion. The fourth step is to fuse the denoised spectra corresponding to each angle obtained in the third step to obtain an average spectrum. Then, the centroid wavelength and peak wavelength are obtained according to the average spectrum, and weighted fusion is performed according to the predefined wavelength weights to obtain the central wavelength. Thus, the effective integration of multi-angle spectral information is realized. The fifth step is to solve and generate a parameter group including multi-angle reflectance and reflectance volatility based on the average spectrum and central wavelength obtained in the fourth step. Thus, it provides characteristic parameters for subsequent steps. The sixth step is to accurately screen out the target mask data from the mask data group according to the predefined mask screening conditions. Thus, it selects a key region for subsequent spatial feature analysis. The seventh step is to use the morphological dynamics processing method for the target mask data screened out in the sixth step to obtain the spatial gradient modulus length. Thus, it provides an important basis for spatial distribution feature analysis. The eighth step is to input the multi-angle reflectance, reflectance volatility and spatial gradient modulus length into a pre-constructed toxicity detection model, and output to obtain a danger coefficient. Thus, the quantitative prediction of the harm degree of impurities is realized. The ninth step is to generate the danger level label corresponding to the renewable resource according to the predefined danger level label generation conditions and the danger coefficient obtained in the eighth step.Thus, accurate classification of the degree of danger is achieved, making the detection results more intuitive and practical. The tenth step is to add the danger level labels generated in the ninth step to the impurity detection information to form a relatively complete and systematic impurity detection report. Thus, it provides comprehensive and accurate information support for subsequent processing links such as impurity treatment and risk assessment, and to a certain extent ensures the scientificity and effectiveness of impurity detection. In summary, steps one to ten of the embodiments of the present disclosure cooperate closely. From the generation of the mask area, the extraction and denoising of spectral data, to the multi-angle spectral fusion, the generation of characteristic parameters, the screening of target mask data and the calculation of spatial features, and then to the quantitative representation of the degree of danger, the division of danger levels and the improvement of detection information, it realizes the full-process optimization from multi-dimensional feature analysis to accurate danger classification. By fully integrating multi-dimensional information such as spectrum and space, the accuracy and reliability of toxicity detection are effectively improved, and to a certain extent, it meets the high-precision requirements in complex impurity detection scenarios, providing relatively strong technical support and decision-making assistance for impurity detection and risk management in related fields.

[0070] Optionally, the above-mentioned execution entity can also perform the following steps: Step one, in response to detecting an editing operation on the above-mentioned annotated image group, generate editing data according to the editing operation record corresponding to the above-mentioned editing operation. Among them, the above-mentioned editing operation can be an interactive operation in which the user subjectively modifies each annotation or each mask data in each annotated image in the above-mentioned annotated image group. For example, marking the missed detection area or deleting the misdetected area. The above-mentioned editing data can include each annotated image after the editing operation. In practice, the above-mentioned execution entity can record and store each annotated image on which the user performs the editing operation. Then, each annotated image after the above-mentioned editing operation is determined as the editing data.

[0071] Step two, based on the above-mentioned editing data, perform incremental learning training on the above-mentioned pre-trained feature extraction model to obtain an updated pre-trained feature extraction model. In practice, the above-mentioned execution entity can use each annotated image after the above-mentioned editing operation as a new training set to perform iterative training on the above-mentioned pre-trained feature extraction model, and can also adopt methods to prevent overfitting to ensure that overfitting does not occur during the iterative training process and at the same time accelerate the learning process. Finally, an updated pre-trained feature extraction model can be obtained, and the above-mentioned updated pre-trained feature extraction model is used to update the above-mentioned pre-trained feature extraction model for subsequent use. As an example, the above-mentioned methods to prevent overfitting can be methods such as learning rate decay and batch normalization.

[0072] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: Through the impurity detection method based on deep learning in some embodiments of the present disclosure, more efficient, accurate, and intelligent impurity detection can be achieved, which can improve the processing efficiency and regeneration quality of recycled resources to a certain extent. Specifically: Traditional manual inspection has problems of subjective errors and slow speed, while existing automated detection methods are based on traditional image processing methods or shallow algorithms, which are difficult to accurately identify complex impurities and lack intelligence and adaptability. Based on this, in some embodiments of the present disclosure, the impurity detection method based on deep learning first obtains a multi-light source image set corresponding to each angle of the recycled resources, where each multi-light source image set in the above multi-light source image set corresponds to an angle. Thus, a multi-light source image set corresponding to each angle of the recycled resources is comprehensively obtained, providing rich information for subsequent detection and avoiding feature omission or misjudgment caused by a single perspective and light source. Then, the above multi-light source image set is preprocessed to obtain a preprocessed multi-light source image set, where the above preprocessed multi-light source image set includes each preprocessed multi-light source image set. Thus, the multi-light source image set is preprocessed to remove noise and normalize the data, improving the image quality and laying a foundation for subsequent processing. Next, channel-level fusion is performed on the above preprocessed multi-light source image set to obtain a fused image set, where each fused image in the above fused image set corresponds to an angle. Thus, a fused image set is obtained through channel-level fusion, highlighting the impurity feature differences and enhancing the detection sensitivity and accuracy. Then, the above fused image set is input into a pre-trained feature extraction model to obtain a multi-scale feature map set, where each multi-scale feature map set in the above multi-scale feature map set corresponds to an angle. Thus, the fused image set is input into a pre-trained feature extraction model to generate a multi-scale feature map set, realizing multi-dimensional and deep-level feature mining of recycled resources and impurities and adapting to the morphology and composition of different impurities. Then, according to the above multi-scale feature map set, a mask data set is generated, where each mask data in the above mask data set corresponds to an angle. Thus, the impurities are more accurately located through the mask data. Then, according to the above mask data set and the above preprocessed multi-light source image set, an image set with annotations is generated, where each image with annotations in the above image set with annotations corresponds to an angle. Thus, by using the generated mask data set and combining it with the preprocessed multi-light source image set, an image set with annotations is generated, thereby realizing a more obvious and intuitive annotation of the impurities and providing an intuitive basis for subsequent analysis. Finally, according to the above image set with annotations, the impurity detection information corresponding to the above recycled resources is generated. Thus, the impurity detection information of the recycled resources is finally generated, providing relatively strong data support for the quality assessment and processing decision-making of the recycled resources.In summary, the impurity detection method based on deep learning disclosed in this disclosure overcomes the deficiencies of traditional methods to a certain extent, provides a relatively efficient, accurate, and intelligent impurity detection solution for the renewable resource recycling and reuse industry, and helps improve the processing efficiency and recycling quality of renewable resource recycling and reuse.

[0073] Further referring to Figure 2 , as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an impurity detection device based on deep learning. These device embodiments correspond to Figure 1 the method embodiments shown, and the device can be specifically applied to various electronic devices.

[0074] As Figure 2 shown, an impurity detection device 200 based on deep learning in some embodiments includes: an acquisition unit 201, a preprocessing unit 202, a fusion unit 203, an extraction unit 204, a first generation unit 205, a second generation unit 206, and a third generation unit 207. Among them, the acquisition unit 201 is configured to acquire a multi-light source image set corresponding to each angle of the renewable resource, where each multi-light source image set in the above multi-light source image set corresponds to one angle; the preprocessing unit 202 is configured to preprocess the above multi-light source image set to obtain a preprocessed multi-light source image set, where the above preprocessed multi-light source image set includes each preprocessed multi-light source image set; the fusion unit 203 is configured to perform channel-level fusion on the above preprocessed multi-light source image set to obtain a fused image set, where each fused image in the above fused image set corresponds to one angle; the extraction unit 204 is configured to input the above fused image set into a pre-trained feature extraction model to obtain a multi-scale feature map set, where each multi-scale feature map set in the above multi-scale feature map set corresponds to one angle; the first generation unit 205 is configured to generate a mask data set according to the above multi-scale feature map set, where each mask data in the above mask data set corresponds to one angle; the second generation unit 206 is configured to generate an image set with annotations according to the above mask data set and the above preprocessed multi-light source image set, where each image with annotations in the above image set with annotations corresponds to one angle; the third generation unit 207 is configured to generate the impurity detection information corresponding to the above renewable resource according to the above image set with annotations.

[0075] It can be understood that the units described in the device 200 correspond to the respective steps in the method described with reference to Figure 1 . Therefore, the operations, features, and beneficial effects described above for the method also apply to the device 200 and the units included therein, and will not be repeated here.

[0076] Next, referring toFigure 3 , which shows a schematic structural diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The illustrated electronic device is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0077] As Figure 3 shown, the electronic device 300 may include a processing device 301 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0078] Generally, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 3 the electronic device 300 with various devices is shown, it should be understood that it is not required to implement or include all the shown devices. More or fewer devices may be alternatively implemented or included. Figure 3 Each block shown in

[0079] Specifically, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing the methods shown in the flowcharts. In such some embodiments, the computer program may be downloaded and installed from a network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the methods of some embodiments of the present disclosure are executed.

[0080] It should be noted that the computer-readable medium described in some embodiments of the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0081] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0082] The above computer-readable medium may be included in the above electronic device; or it may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by the electronic device, the electronic device is caused to: obtain a multi-light-source image set corresponding to each angle of the renewable resource, wherein each multi-light-source image group in the above multi-light-source image set corresponds to one angle; preprocess the above multi-light-source image set to obtain a preprocessed multi-light-source image set, wherein the above preprocessed multi-light-source image set includes each preprocessed multi-light-source image group; perform channel-level fusion on the above preprocessed multi-light-source image set to obtain a fused image set, wherein each fused image in the above fused image set corresponds to one angle; input the above fused image set into a pre-trained feature extraction model to obtain a multi-scale feature map set, wherein each multi-scale feature map group in the above multi-scale feature map set corresponds to one angle; generate a mask data set according to the above multi-scale feature map set, wherein each mask data in the above mask data set corresponds to one angle; generate an image set with annotations according to the above mask data set and the above preprocessed multi-light-source image set, wherein each annotated image in the above image set with annotations corresponds to one angle; generate the impurity detection information corresponding to the above renewable resource according to the above image set with annotations.

[0083] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++; and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or may be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0084] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0085] The units described in some embodiments of the present disclosure can be implemented in software or in hardware. The described units can also be provided in a processor. For example, it can be described as: a processor includes an acquisition unit, a preprocessing unit, a fusion unit, an extraction unit, a first generation unit, a second generation unit, and a third generation unit. Among them, the names of these units do not constitute a limitation on the unit itself in some cases. For example, the acquisition unit can also be described as "the unit for acquiring a multi-light-source image set corresponding to each angle of the renewable resources".

[0086] The functions described above herein can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Product (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), and so on.

[0087] Some embodiments of the present disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements any of the above-described impurity detection methods based on deep learning.

[0088] The above description is only some preferred embodiments of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features having similar functions disclosed in the embodiments of the present disclosure.

Claims

1. An impurity detection method based on deep learning, comprising: Obtaining a multi-light source image set corresponding to each angle of the renewable resources, wherein each multi-light source image group in the multi-light source image set corresponds to one angle; Preprocessing the multi-light source image set to obtain a preprocessed multi-light source image set, wherein the preprocessed multi-light source image set includes each preprocessed multi-light source image group; Performing channel-level fusion on the preprocessed multi-light source image set to obtain a fused image set, wherein each fused image in the fused image set corresponds to one angle; Inputting the fused image set into a pre-trained feature extraction model to obtain a multi-scale feature map set, wherein each multi-scale feature map group in the multi-scale feature map set corresponds to one angle; Generating a mask data set according to the multi-scale feature map set, wherein each mask data in the mask data set corresponds to one angle; Generating an annotated image set according to the mask data set and the preprocessed multi-light source image set, wherein each annotated image in the annotated image set corresponds to one angle; Generating the impurity detection information corresponding to the renewable resources according to the annotated image set.

2. The method according to claim 1, wherein The impurity detection information includes: the type, quantity, position, area ratio, processing suggestions of the impurities corresponding to the renewable resources, and the annotated image set.

3. The method according to claim 1, wherein, The method further includes: Performing forward conversion on each visible light acquisition image corresponding to the renewable resources obtained through the visible light source to obtain each image to be corrected, wherein each visible light acquisition image corresponds to each angle; Performing correction processing on each image to be corrected to obtain each corrected image; Performing reverse conversion on each corrected image to obtain each visible light image, wherein each visible light image corresponds to each angle.

4. The method according to claim 1, wherein, The method further includes: In response to detecting an editing operation on the annotated image set, generating editing data according to the editing operation record corresponding to the editing operation; Performing incremental learning training on the pre-trained feature extraction model based on the editing data to obtain an updated pre-trained feature extraction model.

5. The method according to claim 1, wherein The preprocessing the multi-light source image set to obtain a preprocessed multi-light source image set includes: Performing image enhancement processing on the multi-light source image set to obtain an enhanced multi-light source image set; Performing denoising processing on the enhanced multi-light source image set to obtain a preprocessed multi-light source image set.

6. The method according to claim 1, wherein The performing channel-level fusion on the preprocessed multi-light source image set to obtain a fused image set includes: Performing the following steps on the preprocessed multi-light source image group corresponding to each angle in the preprocessed multi-light source image set: Performing channel recombination on the preprocessed multi-light source image group corresponding to the angle to obtain a channel tensor group; Normalizing each channel tensor in the channel tensor group to obtain a normalized channel tensor group; Performing weighted channel fusion on the normalized channel tensor group to obtain the fused image corresponding to the angle; Integrate the respective fused images corresponding to the respective angles into a fused image group.

7. The method according to claim 1, wherein Generating a mask data group according to the multi-scale feature map set includes: For each multi-scale feature map set corresponding to each angle in the multi-scale feature map set, perform the following steps: Perform multi-scale fusion processing on the multi-scale feature map set corresponding to the angle to obtain an enhanced feature map; Generate a candidate region group according to the enhanced feature map; Filter the candidate region group according to preset region filtering conditions to obtain a filtered candidate region group; Perform feature alignment on the filtered candidate region group to obtain a first feature region group; Generate an adjustment information group according to the first feature region group, where the adjustment information group includes respective classification results and respective bounding box adjustment parameters; Adjust the filtered candidate region group according to the adjustment information group to obtain an adjusted candidate region group; Generate initial mask data based on the adjusted candidate region group; Perform optimization processing on the initial mask data to obtain the mask data corresponding to the angle; Integrate the respective mask data corresponding to the respective angles into a mask data group.

8. An impurity detection device based on deep learning, comprising: An acquisition unit configured to acquire a multi-light source image set corresponding to each angle of the renewable resource, where each multi-light source image set in the multi-light source image set corresponds to an angle; A preprocessing unit configured to preprocess the multi-light source image set to obtain a preprocessed multi-light source image set, where the preprocessed multi-light source image set includes respective preprocessed multi-light source image sets; A fusion unit configured to perform channel-level fusion on the preprocessed multi-light source image set to obtain a fused image group, where each fused image in the fused image group corresponds to an angle; An extraction unit configured to input the fused image group into a pre-trained feature extraction model to obtain a multi-scale feature map set, where each multi-scale feature map set in the multi-scale feature map set corresponds to an angle; A first generation unit configured to generate a mask data group according to the multi-scale feature map set, where each mask data in the mask data group corresponds to an angle; A second generation unit configured to generate an image group with annotations according to the mask data group and the preprocessed multi-light source image set, where each image with annotations in the image group with annotations corresponds to an angle; A third generation unit configured to generate impurity detection information corresponding to the renewable resource according to the image group with annotations.

9. An electronic device, comprising: One or more processors; A storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, wherein, The program, when executed by the processor, implements the method according to any one of claims 1 to 7.

Images