A smart garbage identification and classification method

Through the intelligent garbage identification and classification method, the image optimization network is used to restore spam image information and identify it, which solves the problem of low reliability of garbage identification and achieves more reliable garbage type recognition.

CN118608880BActive Publication Date: 2024-11-08BEIJING HUIMENG XINAN SOFTWARE TECH CO LTD +1
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202411084050.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-11-08
Estimated Expiration
2044-08-08

AI Technical Summary

Technical Problem

The reliability of garbage identification in the prior art is not high, resulting in errors in garbage classification and disposal, and it is impossible to reliably classify and identify.

Method used

The intelligent garbage identification and classification method is adopted to optimize the original garbage images through the target garbage image optimization network, recover lost image information, and use the target garbage type identification network to identify garbage type, improving the reliability of recognition.

Benefits of technology

Through image optimization processing, the integrity of spam image information can be ensured, the reliability of spam type recognition is improved, and the problem of low reliability of spam recognition in the prior art is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118608880B_ABST
    Figure CN118608880B_ABST
Patent Text Reader

Abstract

The present invention provides an intelligent garbage identification and classification method, which relates to the field of image recognition technology. In the present invention, firstly, the garbage transported by the target garbage transport vehicle is subjected to image acquisition and processing to obtain the original garbage image; secondly, the target garbage image optimization network is used to perform image optimization processing on the original garbage image to obtain the optimized garbage image; then, the target garbage type recognition network is used to perform garbage type recognition processing on the optimized garbage image to obtain the target garbage type information. Based on the above method, the problem of low reliability of garbage recognition existing in the prior art can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to an intelligent garbage recognition and classification method. Background Art

[0002] Transferring garbage by garbage transfer trucks is a common process in the field of sanitation. Among them, garbage classification plays an important role in the subsequent treatment of garbage. However, during garbage transfer and before garbage transfer, due to various factors, the effect of garbage classification and placement is not reliable, that is, there is a common problem of incorrect garbage classification and placement. Therefore, during the transfer process, classification and identification need to be performed again. However, in the prior art, there is still a problem of low reliability of garbage identification, which makes it impossible to perform reliable classification and identification. Summary of the invention

[0003] In view of this, an object of the present invention is to provide an intelligent garbage identification and classification method to improve the problem of low reliability of garbage identification in the prior art.

[0004] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0005] An intelligent garbage identification and classification method, comprising:

[0006] Perform image acquisition and processing on the garbage transferred by the target garbage transfer vehicle to obtain the original garbage image;

[0007] Using a target junk image optimization network, the original junk image is subjected to image optimization processing to obtain an optimized junk image, wherein the target junk image optimization network is a trained neural network, and the image optimization processing refers to restoring lost image information of the junk to be identified in the original junk image, wherein the lost image information at least includes image information of the junk to be identified that is lost due to occlusion by other junk or other objects, and the image optimization processing includes mining multiple levels of key information of the image information and restoring the corresponding optimized junk image based on the result of the mining operation, wherein the multiple levels of key information include key information of multiple color channels and key information of extracted contours;

[0008] The optimized garbage image is subjected to garbage type recognition processing by utilizing a target garbage type recognition network to obtain target garbage type information of the garbage to be recognized, wherein the target garbage type recognition network is a trained neural network, and the target garbage type information is used as a basis for classifying and monitoring the garbage transferred by the target garbage transfer vehicle, and the classification monitoring processing at least includes performing corresponding classification abnormality warning operations when it is determined that misclassified garbage exists.

[0009] In some preferred embodiments, in the above-mentioned intelligent garbage identification and classification method, the step of using the target garbage image optimization network to perform image optimization processing on the original garbage image to obtain the optimized garbage image includes:

[0010] Using a target junk image optimization network, performing an image information mining operation on a junk image block in the original junk image, and outputting a first number of junk image feature representations corresponding to the junk image block, wherein the first number is greater than 1, and the first number of junk image feature representations are used to reflect key information of the junk image block at different first number levels, the junk image block is formed by performing image segmentation processing on the original junk image, and the key information of the junk image block at different first number levels includes key information of multiple color channels and key information of contours extracted from the junk image block;

[0011] Using the target junk image optimization network, fusing the first number of junk image feature representations, outputting a fused junk image feature representation corresponding to the junk image block, and performing focused feature mining on the fused junk image feature representation, outputting a focused junk image feature representation corresponding to the fused junk image feature representation;

[0012] Using the target junk image optimization network, performing multi-level aggregation processing on the image block association parameter distribution corresponding to the original junk image and the target junk image feature representation corresponding to the junk image block, and outputting the aggregated junk image feature representation corresponding to the junk image block, wherein the target junk image feature representation is obtained based on the focused junk image feature representation and the fused junk image feature representation, and the image block association parameter distribution is used to reflect the association relationship between each of the junk image blocks;

[0013] The target junk image optimization network is used to perform image optimization processing on the original junk image based on the aggregated junk image feature representation to obtain an optimized junk image.

[0014] In some preferred embodiments, in the above-mentioned intelligent garbage identification and classification method, the first number of garbage image feature representations include garbage image color feature representations;

[0015] The step of using the target junk image optimization network to perform image information mining operations on junk image blocks in the original junk image and outputting a first number of junk image feature representations corresponding to the junk image blocks includes:

[0016] Loading the original junk image so that the image information mining model in the target junk image optimization network can obtain the original junk image;

[0017] Determine a second number of color channel values ​​of a junk image block in the original junk image by using the image information mining model, wherein the second number is greater than 1, and the second number of color channel values ​​includes a value of each pixel in the junk image block in a color channel;

[0018] Determine color channel value feature representations corresponding to each of the second number of color channel values, and perform feature extraction processing on each of the second number of color channel value feature representations, and output extracted color feature representations corresponding to each of the second number of color channel value feature representations;

[0019] The output second number of the extracted color feature representations are subjected to feature compression processing, and the garbage image color feature representation corresponding to the garbage image block is output.

[0020] In some preferred embodiments, in the above-mentioned intelligent garbage identification and classification method, the first number of garbage image feature representations include garbage image contour feature representations;

[0021] The step of using the target junk image optimization network to perform image information mining operations on junk image blocks in the original junk image and outputting a first number of junk image feature representations corresponding to the junk image blocks includes:

[0022] Performing contour extraction processing on the original garbage image to determine whether each pixel in the original garbage image belongs to the contour, and based on the result of whether it belongs to the contour, updating the pixel value of each pixel in the original garbage image to obtain a corresponding updated garbage image, wherein in the updated garbage image, if a pixel belongs to the contour, the pixel value of the pixel is 1, and if a pixel does not belong to the contour, the pixel value of the pixel is 0;

[0023] Loading the updated junk image so that the image information mining model in the target junk image optimization network acquires the updated junk image;

[0024] Determine a second number of contour representation values ​​of the junk image block in the updated junk image by using the image information mining model, wherein the second number is greater than 1, and the second number of contour representation values ​​includes a representation value reflecting whether each pixel point in the junk image block belongs to a contour;

[0025] Determine the contour characterization value feature representations corresponding to each of the second number of contour characterization values, and perform feature extraction processing on each of the second number of contour characterization value feature representations, and output the extracted contour feature representations corresponding to each of the second number of contour characterization value feature representations;

[0026] The output second number of the extracted contour feature representations are subjected to feature compression processing, and the garbage image contour feature representation corresponding to the garbage image block is output.

[0027] In some preferred embodiments, in the above-mentioned intelligent garbage identification and classification method, the step of using the target garbage image optimization network to fuse the first number of garbage image feature representations, outputting the fused garbage image feature representation corresponding to the garbage image block, and performing focused feature mining on the fused garbage image feature representation, and outputting the focused garbage image feature representation corresponding to the fused garbage image feature representation, comprises:

[0028] Loading the first number of junk image feature representations so that the image feature focusing model in the target junk image optimization network acquires the first number of junk image feature representations;

[0029] Using the image feature focusing model, cascade combining the first number of junk image feature representations, and outputting a fused junk image feature representation corresponding to the junk image block;

[0030] Performing matrix mapping processing on the fused garbage image feature representation, and outputting a third number of focus mapping image feature representations corresponding to the fused garbage image feature representation;

[0031] The third number of focus map image feature representations are subjected to similarity fusion processing, and a focused garbage image feature representation corresponding to the fused garbage image feature representation is output.

[0032] In some preferred embodiments, in the above-mentioned intelligent garbage identification and classification method, the third number of focus mapping image feature representations includes a first mapping image feature representation, a second mapping image feature representation and a third mapping image feature representation;

[0033] The step of performing similarity fusion processing on the third number of focus map image feature representations and outputting the focused garbage image feature representation corresponding to the fused garbage image feature representation comprises:

[0034] Performing parameter adjustment processing on the first mapping image feature representation so that the distribution of parameters in the first mapping image feature representation is adjusted to obtain an adjusted mapping image feature representation;

[0035] multiplying the third mapped image feature representation and the adjusted mapped image feature representation to output a corresponding correlation parameter distribution, wherein the correlation parameter distribution is used to reflect the correlation relationship between the first number of junk image feature representations;

[0036] Performing parameter mapping processing on the correlation parameter distribution, and outputting a mapped correlation parameter distribution corresponding to the correlation parameter distribution;

[0037] multiplying the mapping correlation parameter distribution and the second mapping image feature representation, and outputting the correlation image feature representations corresponding to the first number of junk image feature representations;

[0038] A linear mapping operation is performed on the associated image feature representation to output a focused junk image feature representation corresponding to the fused junk image feature representation.

[0039] In some preferred embodiments, in the above-mentioned intelligent garbage identification and classification method, the step of using the target garbage image optimization network to perform multi-level aggregation processing on the image block association parameter distribution corresponding to the original garbage image and the target garbage image feature representation corresponding to the garbage image block, and outputting the aggregated garbage image feature representation corresponding to the garbage image block includes:

[0040] Loading and processing the image block associated parameter distribution corresponding to the original junk image and the target junk image feature representation corresponding to the junk image block, so that the deep aggregation model in the target junk image optimization network obtains the image block associated parameter distribution and the target junk image feature representation, wherein the deep aggregation model includes a deep feature mining unit, a fourth number of deep feature aggregation units and an aggregation output unit, and the fourth number is greater than or equal to 1;

[0041] Using the deep feature mining unit, performing a deep feature mining operation on the target junk image feature representation corresponding to the junk image block, and outputting the deep junk image feature representation corresponding to the junk image block;

[0042] Outputting, using the fourth number of deep feature aggregation units, intermediate junk image feature representations corresponding to each of the fourth number of deep feature aggregation units according to the image block association parameter distribution corresponding to the original junk image and the deep junk image feature representations corresponding to the junk image blocks;

[0043] Marking the intermediate junk image feature representation corresponding to the target deep feature aggregation unit in the fourth number of deep feature aggregation units to form a target deep image feature representation corresponding to the junk image block, wherein the target deep feature aggregation unit belongs to the last deep feature aggregation unit in the fourth number of deep feature aggregation units, and the fourth number of deep feature aggregation units have a cascade connection order relationship;

[0044] By using the aggregation output unit, after the target depth image feature representation corresponding to the junk image block and the depth junk image feature representation corresponding to the junk image block are fused with weight coefficients, an aggregated junk image feature representation corresponding to the junk image block is output by performing superposition aggregation processing.

[0045] In some preferred embodiments, in the above-mentioned intelligent garbage identification and classification method, the fourth number of deep feature aggregation units includes any deep feature aggregation unit;

[0046] The step of using the fourth number of deep feature aggregation units to output the intermediate junk image feature representations corresponding to each of the fourth number of deep feature aggregation units according to the image block association parameter distribution corresponding to the original junk image and the deep junk image feature representations corresponding to the junk image blocks comprises:

[0047] When the arbitrary deep feature aggregation unit belongs to the deep feature aggregation unit ranked first among the fourth number of deep feature aggregation units, using the arbitrary deep feature aggregation unit, the image block association parameter distribution corresponding to the original junk image and the deep junk image feature representation corresponding to the junk image block are associated and aggregated, and an intermediate junk image feature representation corresponding to the arbitrary deep feature aggregation unit is output;

[0048] When the arbitrary deep feature aggregation unit belongs to other depth feature aggregation units other than the depth feature aggregation unit ranked first among the fourth number of depth feature aggregation units, the arbitrary deep feature aggregation unit is used to associate and aggregate the image block association parameter distribution corresponding to the original junk image and the intermediate junk image feature representation corresponding to the previous depth feature aggregation unit of the arbitrary deep feature aggregation unit, and output the intermediate junk image feature representation corresponding to the arbitrary deep feature aggregation unit.

[0049] In some preferred embodiments, in the above-mentioned intelligent garbage identification and classification method, the step of using the target garbage image optimization network to perform image optimization processing on the original garbage image based on the aggregated garbage image feature representation to obtain an optimized garbage image includes:

[0050] Using the image reconstruction model included in the target junk image optimization network, based on the aggregated junk image feature representation corresponding to each junk image block in the original junk image, performing image reconstruction processing on the original junk image to obtain a corresponding reconstructed junk image, and using the reconstructed junk image as the corresponding optimized junk image; or

[0051] The aggregated junk image feature representation is linearly mapped to output the mapped junk image feature representation corresponding to the junk image block, and the mapped junk image feature representation is mapped and output to output the probability parameter distribution corresponding to the mapped junk image feature representation, and the probability parameter with the maximum value is determined in the probability parameter distribution, and based on the image pixel adjustment method corresponding to the probability parameter, the junk image block is pixel adjusted to obtain the corresponding optimized junk image block, and the optimized junk image blocks corresponding to each junk image block in the original junk image are spliced ​​to output the optimized junk image.

[0052] In some preferred embodiments, in the above-mentioned intelligent garbage identification and classification method, the intelligent garbage identification and classification method further includes a training step of the target garbage image optimization network, and the training step includes:

[0053] Determining a candidate junk image optimization network, a training first junk image, and a training second junk image corresponding to the training first junk image, wherein the training first junk image is formed by performing image information adjustment processing on the training second junk image;

[0054] Using the candidate junk image optimization network, performing an image information mining operation on the junk image blocks in the first training junk image, and outputting a first number of junk image feature representations in training corresponding to the junk image blocks in the first training junk image;

[0055] Using the candidate junk image optimization network, fusing the first number of junk image feature representations in training, outputting the fused junk image feature representations in training corresponding to the junk image blocks in the first junk image in training, and performing focused feature mining on the fused junk image feature representations in training, outputting the focused junk image feature representations in training of the fused junk image feature representations in training;

[0056] Using the candidate junk image optimization network, performing multi-level aggregation processing on the image block association parameter distribution in training corresponding to the first junk image in training and the target junk image feature representation in training corresponding to the junk image block in the first junk image in training, and outputting the aggregated junk image feature representation in training corresponding to the junk image block in the first junk image in training, wherein the target junk image feature representation in training is obtained based on the focused junk image feature representation in training and the fused junk image feature representation in training;

[0057] Using the candidate junk image optimization network, based on the aggregated junk image feature representation in the training, the image optimization process is performed on the training first junk image to obtain an optimized first junk image;

[0058] Based on the optimized first junk image and the trained second junk image, the candidate junk image optimization network is trained and updated, and a corresponding target junk image optimization network is output.

[0059] An intelligent garbage identification and classification method provided by an embodiment of the present invention firstly collects and processes images of the garbage transported by the target garbage transfer vehicle to obtain an original garbage image; secondly, uses a target garbage image optimization network to perform image optimization processing on the original garbage image to obtain an optimized garbage image; then, uses a target garbage type recognition network to perform garbage type recognition processing on the optimized garbage image to obtain target garbage type information. Based on the above method, since the original garbage image is optimized before the garbage type recognition process is performed, that is, the lost image information of the garbage to be recognized in the original garbage image is restored, the lost image information can be restored to ensure the integrity of the image information of the garbage to be recognized, thus, the reliability of the garbage type recognition process based on the complete image information can be guaranteed, and therefore, the problem of low reliability of garbage recognition in the prior art can be improved.

[0060] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a structural block diagram of an electronic device provided by an embodiment of the present invention.

[0062] Figure 2 A flowchart of the steps of the intelligent garbage identification and classification method provided in an embodiment of the present invention.

[0063] Figure 3 A network architecture diagram of a deep aggregation model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0065] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] like Figure 1 As shown, an embodiment of the present invention provides an electronic device, wherein the electronic device may include a memory and a processor.

[0067] In detail, the memory and the processor are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, they can be electrically connected to each other through one or more communication buses or signal lines. The memory can store at least one software function module (computer program) that can exist in the form of software or firmware. The processor can be used to execute the executable computer program stored in the memory, thereby realizing the intelligent garbage identification and classification method provided by the embodiment of the present invention (as described later).

[0068] Combination Figure 2 The embodiment of the present invention further provides an intelligent garbage identification and classification method, which can be applied to the above electronic device. The method steps defined in the process related to the intelligent garbage identification and classification method can be implemented by the electronic device.

[0069] The following will Figure 2 The specific process shown is explained in detail.

[0070] Step S110, performing image acquisition and processing on the garbage transferred by the target garbage transfer vehicle to obtain an original garbage image.

[0071] In an embodiment of the present invention, the electronic device can perform image acquisition processing on the garbage transferred by the target garbage transfer vehicle to obtain an original garbage image. Exemplarily, the electronic device can perform image acquisition processing on the garbage through a connected image acquisition device (such as a camera, etc.).

[0072] Step S120, using the target junk image optimization network, performing image optimization processing on the original junk image to obtain an optimized junk image.

[0073] In an embodiment of the present invention, the electronic device can use a target junk image optimization network to perform image optimization processing on the original junk image to obtain an optimized junk image. The target junk image optimization network is a trained neural network, and the image optimization processing refers to restoring the lost image information of the junk to be identified in the original junk image. The lost image information at least includes the image information of the junk to be identified that is lost due to occlusion by other junk or other objects. In other words, the image information of other objects or junk needs to be adjusted so that it is adjusted to the lost image information of the junk to be identified. In addition, the image optimization processing includes mining multiple levels of key information on the image information and restoring the corresponding optimized junk image based on the results of the mining operation. The key information at multiple levels includes key information of multiple color channels and key information of extracted contours.

[0074] Step S130 , using a target garbage type recognition network, performs garbage type recognition processing on the optimized garbage image to obtain target garbage type information of the garbage to be recognized.

[0075] In an embodiment of the present invention, the electronic device can use a target garbage type recognition network to perform garbage type recognition processing on the optimized garbage image to obtain target garbage type information of the garbage to be recognized. The target garbage type recognition network is a trained neural network, and the target garbage type information is used as a basis for classifying and monitoring the garbage transferred by the target garbage transfer vehicle. The classification monitoring processing at least includes performing corresponding classification abnormality warning operations when it is determined that there is misclassified garbage, such as reminding to reclassify and place it, or controlling the corresponding mechanical arm to grab the wrongly placed garbage.

[0076] Based on the above method, since the original garbage image will be optimized before the garbage type recognition process is performed, that is, the lost image information of the garbage to be identified in the original garbage image will be restored, so that the lost image information can be restored to ensure the integrity of the image information of the garbage to be identified. In this way, the reliability of the garbage type recognition process based on the complete image information can be guaranteed. Therefore, the problem of low reliability of garbage recognition in the prior art can be improved, and a reliable basis can be provided for subsequent garbage processing.

[0077] On the basis of the above implementation, it should be noted that the target garbage type identification network can be any neural network with garbage classification and identification function, or in other words, it is formed by training with garbage images and corresponding classification labels on the basis of a neural network with image classification and identification function. Therefore, in the embodiments of the present invention, no specific limitation is made.

[0078] In the embodiment of the present invention, the specific process of performing image optimization processing on the target junk image optimization network and the specific process of performing corresponding training to form the target junk image optimization network are described in detail, which can be specifically described through the following two aspects.

[0079] In the first aspect, in step S120, the specific manner of performing image optimization processing on the original junk image using the target junk image optimization network is not limited and can be selected according to actual needs.

[0080] For example, in a specific implementation, in order to ensure that the basis for image optimization processing is sufficient and reliable, so that the obtained optimized garbage image is also sufficient and reliable, the above-mentioned step S120 may include steps S121 to S124.

[0081] Step S121, using a target junk image optimization network, performing an image information mining operation on junk image blocks in the original junk image, and outputting a first number of junk image feature representations corresponding to the junk image blocks.

[0082] In an embodiment of the present invention, a target junk image optimization network can be used to perform image information mining operations on junk image blocks in the original junk image, and a first number of junk image feature representations corresponding to the junk image blocks can be output. The first number is greater than 1, and the first number of junk image feature representations are used to reflect key information of the junk image blocks at different first number levels. The junk image blocks are formed by performing image segmentation processing on the original junk image (exemplarily, the original junk image can be segmented according to a predetermined size to form multiple junk image blocks of the same size, such as 2*3, 4*4, 16*16, 32*32, etc.), and the key information of the junk image blocks at different first number levels includes key information of multiple color channels (such as RGB three color channels) and key information of contours extracted from the junk image blocks, that is, the information at each level is mined separately.

[0083] Step S122, using the target junk image optimization network, the first number of junk image feature representations are fused, and the fused junk image feature representation corresponding to the junk image block is output; the fused junk image feature representation is subjected to focused feature mining, and a focused junk image feature representation corresponding to the fused junk image feature representation is output.

[0084] In an embodiment of the present invention, the target junk image optimization network can be used to fuse the first number of junk image feature representations, output the fused junk image feature representation corresponding to the junk image block, and perform focused feature mining on the fused junk image feature representation, output the focused junk image feature representation corresponding to the fused junk image feature representation. In this way, after fusing image information at multiple levels, corresponding focusing is performed, so that important information can be mined and the representation capability of the focused junk image feature representation is improved.

[0085] Step S123, using the target junk image optimization network, performing multi-level aggregation processing on the image block association parameter distribution corresponding to the original junk image and the target junk image feature representation corresponding to the junk image block, and outputting the aggregated junk image feature representation corresponding to the junk image block.

[0086] In an embodiment of the present invention, the target junk image optimization network can be used to perform multi-level aggregation processing on the image block association parameter distribution corresponding to the original junk image and the target junk image feature representation corresponding to the junk image block, and output the aggregated junk image feature representation corresponding to the junk image block. The target junk image feature representation is obtained based on the focused junk image feature representation and the fused junk image feature representation; exemplarily, the focused junk image feature representation and the fused junk image feature representation can be added to obtain the target junk image feature representation, and the image block association parameter distribution is used to reflect the association relationship between each of the junk image blocks. In this way, the aggregated junk image feature representation can not only reflect the information of the junk image block itself, but also reflect the association relationship between the junk image blocks, so that the representation ability of the aggregated junk image feature representation can be better.

[0087] Step S124, using the target junk image optimization network, performing image optimization processing on the original junk image based on the aggregated junk image feature representation to obtain an optimized junk image.

[0088] In an embodiment of the present invention, the target junk image optimization network can be used to perform image optimization processing on the original junk image based on the aggregated junk image feature representation to obtain an optimized junk image, that is, an image that restores the lost image information. In addition, in an embodiment of the present invention, the specific expression form of various feature representations can be a vector, such as:

[0089] {(1,5,3,7,6,9,5,6,2,0,0,4);

[0090] (2, 4, 6, 3, 3, 2, 5, 7, 1, 0, 1, 2);

[0091] (5,5,3,7,6,0,0,6,2,0,0,3);

[0092] (7, 7, 3, 0, 6, 9, 0, 6, 2, 0, 0, 8);

[0093] (9, 5, 3, 0, 6, 9, 5, 0, 2, 1, 0, 7)}.

[0094] Alternatively, in a specific implementation, the first number of junk image feature representations include junk image color feature representations (it should be noted that information of each color channel may be mined separately). Based on this, in order to reliably mine the junk image color feature representations, the above step S121 may include the following contents:

[0095] First, the original junk image may be loaded so that the image information mining model in the target junk image optimization network can obtain the original junk image;

[0096] Secondly, the image information mining model can be used to determine a second number of color channel values ​​of the junk image block in the original junk image, wherein the second number is greater than 1, and the second number of color channel values ​​includes a value of each pixel in the junk image block in a color channel, such as all R channel values, all G channel values, or all B channel values, such as a 4*4 junk image block corresponds to 16 color channel values ​​(255, 100, 90, 0, 0, 0, 99, 80, 60, 158, 220, 178, 96, 12, 15, 55);

[0097] Then, the color channel value feature representations corresponding to each of the second number of color channel values ​​can be determined (exemplarily, the mapping relationship in the image information mining model can be used to map each color channel value, such as mapping to a 100-dimensional vector, such as (0.1, 0.3, -0.2, 06, 0.7, 0, ..., 0, -0.25)), and the second number of color channel value feature representations are respectively subjected to feature extraction processing, and the extracted color feature representations corresponding to each of the second number of color channel value feature representations are output. Exemplarily, the convolution kernel or convolution unit in the image information mining model can be used to perform convolution processing on each color channel value feature representation, so as to obtain the corresponding extracted color feature representation;

[0098] Finally, the second number of the extracted color feature representations outputted may be subjected to feature compression processing, and the garbage image color feature representation corresponding to the garbage image block may be outputted; illustratively, the second number of the extracted color feature representations may be cascaded and combined (also referred to as splicing, such as {extracted color feature representation 1; extracted color feature representation 2; extracted color feature representation 3; ...}), and then, the cascaded combined feature representation may be subjected to feature compression processing, such as performing mean pooling or maximum pooling or the like through the pooling unit in the image information mining model.

[0099] Alternatively, in a specific implementation, the first number of junk image feature representations include junk image contour feature representations. Based on this, in order to reliably mine junk image contour feature representations, the above step S121 may include the following contents:

[0100] First, the original garbage image can be subjected to contour extraction processing (the contour extraction processing can use any existing extraction method, which is not specifically limited here) to determine whether each pixel point in the original garbage image belongs to the contour, and based on the result of whether it belongs to the contour, the pixel value of each pixel point in the original garbage image is updated to obtain a corresponding updated garbage image, wherein, in the updated garbage image, if a pixel point belongs to the contour, the pixel value of the pixel point is 1, and if a pixel point does not belong to the contour, the pixel value of the pixel point is 0; that is, in the updated garbage image, the pixel value actually represents whether the pixel point at the corresponding position is on the contour;

[0101] Secondly, the updated junk image can be loaded so that the image information mining model in the target junk image optimization network can obtain the updated junk image;

[0102] Then, the image information mining model may be used to determine a second number of contour representation values ​​of the junk image block in the updated junk image, wherein the second number is greater than 1 (such as a number of 4*4), and the second number of contour representation values ​​includes a representation value reflecting whether each pixel point in the junk image block belongs to the contour, such as (0, 0, 1, 0, 0, 0, 0, ...);

[0103] Afterwards, the contour characterization value feature representations corresponding to each of the second number of contour characterization values ​​may be determined (as described above, i.e., mapped through a corresponding mapping relationship), and the second number of contour characterization value feature representations may be subjected to feature extraction processing (as described above, i.e., convolution processing may be performed through a corresponding convolution kernel or convolution unit), and the extracted contour feature representations corresponding to each of the second number of contour characterization value feature representations may be output;

[0104] Finally, the second number of the extracted contour feature representations outputted may be subjected to feature compression processing to output the junk image contour feature representation corresponding to the junk image block; illustratively, the second number of the extracted contour feature representations may be cascaded and combined, and then the result of the cascaded combination may be subjected to processing such as mean pooling or maximum pooling.

[0105] Optionally, in a specific implementation, in order to improve the representation capability of the focused garbage image feature representation obtained by fusion and focusing processing, the above-mentioned step S122 may include steps S122a to S122d.

[0106] Step S122a: loading the first number of junk image feature representations so that the image feature focusing model in the target junk image optimization network acquires the first number of junk image feature representations.

[0107] In the embodiment of the present invention, the first number of junk image feature representations may be loaded so that the image feature focusing model in the target junk image optimization network acquires the first number of junk image feature representations.

[0108] Step S122b: using the image feature focusing model, cascade combining the first number of junk image feature representations, and outputting a fused junk image feature representation corresponding to the junk image block.

[0109] In the embodiment of the present invention, the image feature focusing model can be used to perform cascade combination processing on the first number of junk image feature representations, and output a fused junk image feature representation corresponding to the junk image block. That is, the fusion of junk image feature representations can be achieved by performing cascade combination processing, such as the fused junk image feature representation is (jam image feature representation 1; junk image feature representation 2; junk image feature representation 3; ...)

[0110] Step S122c: performing matrix mapping processing on the fused garbage image feature representation, and outputting a third number of focus mapping image feature representations corresponding to the fused garbage image feature representation.

[0111] In an embodiment of the present invention, the fused junk image feature representation can be subjected to matrix mapping processing to output a third number of focused mapping image feature representations corresponding to the fused junk image feature representation. Exemplarily, the third number can be equal to 3. For example, the first mapping matrix, the second mapping matrix and the third mapping matrix included in the image feature focusing model (which can be used as model parameters of the image feature focusing model to be updated and formed in the corresponding training process) can be used to perform matrix mapping processing (such as multiplication processing) on ​​the fused junk image feature representation. In this way, three focused mapping image feature representations can be obtained.

[0112] Step S122d: performing similarity fusion processing on the third number of focus map image feature representations, and outputting a focused garbage image feature representation corresponding to the fused garbage image feature representation.

[0113] In the embodiment of the present invention, the third number of focused map image feature representations may be subjected to similarity fusion processing to output a focused junk image feature representation corresponding to the fused junk image feature representation, that is, correlation mining may be performed on the fused junk image feature representation.

[0114] Alternatively, in a specific implementation, in order to enable reliable association mining of the fused junk image feature representation, that is, to mine out important information or significant information therein, the third number of focus mapping image feature representations includes a first mapping image feature representation, a second mapping image feature representation, and a third mapping image feature representation. Based on this, the above step S122d may further include:

[0115] First, the first mapped image feature representation may be subjected to parameter adjustment processing (such as converting rows into columns, and using columns as rows), so that the distribution of parameters in the first mapped image feature representation is adjusted to obtain a corresponding adjusted mapped image feature representation;

[0116] Secondly, the third mapped image feature representation and the adjusted mapped image feature representation may be multiplied to output a corresponding correlation parameter distribution, wherein the correlation parameter distribution is used to reflect the correlation relationship between the first number of junk image feature representations;

[0117] Then, the correlation parameter distribution may be subjected to parameter mapping processing (such as mapping the parameter to an interval of 0-1), and a mapped correlation parameter distribution corresponding to the correlation parameter distribution may be output;

[0118] Afterwards, the mapping correlation parameter distribution and the second mapping image feature representation may be multiplied to output the correlation image feature representation corresponding to the first number of junk image feature representations. That is, the second mapping image feature representation may be weighted based on the mapping correlation parameter distribution, so that important information may be mined.

[0119] Finally, a linear mapping operation may be performed on the associated image feature representation to output a focused junk image feature representation corresponding to the fused junk image feature representation. Exemplarily, the linear mapping operation may refer to a fully connected (FC) operation.

[0120] Optionally, in a specific implementation, in order to ensure the reliability of the multi-level aggregation processing so that the output aggregated junk image feature representation has a better representation capability for key information in the image, the above-mentioned step S123 may include steps S123a to S123e.

[0121] Step S123a, loading and processing the image block associated parameter distribution corresponding to the original junk image and the target junk image feature representation corresponding to the junk image block, so that the deep aggregation model in the target junk image optimization network obtains the image block associated parameter distribution and the target junk image feature representation.

[0122] In an embodiment of the present invention, the image block associated parameter distribution corresponding to the original junk image and the target junk image feature representation corresponding to the junk image block can be loaded and processed, so that the deep aggregation model in the target junk image optimization network can obtain the image block associated parameter distribution and the target junk image feature representation. The deep aggregation model includes a deep feature mining unit, a fourth number of deep feature aggregation units and an aggregation output unit (such as Figure 3 As shown), the fourth number is greater than or equal to 1, that is, there is at least one feature aggregation unit.

[0123] Step S123b: using the deep feature mining unit, performing a deep feature mining operation on the target junk image feature representation corresponding to the junk image block, and outputting the deep junk image feature representation corresponding to the junk image block.

[0124] In an embodiment of the present invention, the deep feature mining unit can be used to perform a deep feature mining operation on the target junk image feature representation corresponding to the junk image block, and output the deep junk image feature representation corresponding to the junk image block. Exemplarily, the deep feature mining unit can be a convolutional neural network.

[0125] Step S123c: using the fourth number of deep feature aggregation units, outputting the intermediate junk image feature representations corresponding to each of the fourth number of deep feature aggregation units according to the image block association parameter distribution corresponding to the original junk image and the deep junk image feature representations corresponding to the junk image blocks.

[0126] In the embodiment of the present invention, the fourth number of deep feature aggregation units can be used to output the intermediate junk image feature representations corresponding to each of the fourth number of deep feature aggregation units according to the image block association parameter distribution corresponding to the original junk image and the deep junk image feature representations corresponding to the junk image blocks. That is, each deep feature aggregation unit can output the corresponding intermediate junk image feature representation after corresponding processing.

[0127] Step S123d: marking the intermediate garbage image feature representations corresponding to the target deep feature aggregation units in the fourth number of deep feature aggregation units to form the target deep image feature representations corresponding to the garbage image block.

[0128] In an embodiment of the present invention, the intermediate junk image feature representation corresponding to the target deep feature aggregation unit in the fourth number of deep feature aggregation units can be marked to form the target deep image feature representation corresponding to the junk image block. The target deep feature aggregation unit belongs to the last deep feature aggregation unit in the fourth number of deep feature aggregation units, and there is a cascade connection between the fourth number of deep feature aggregation units. In other words, the output of the previous deep feature aggregation unit can be used as the input of the next deep feature aggregation unit, so that the aggregation depth of the features can be gradually improved.

[0129] Step S123e, using the aggregation output unit, after fusing the target depth image feature representation corresponding to the junk image block and the depth junk image feature representation corresponding to the junk image block with the weight coefficient, output the aggregated junk image feature representation corresponding to the junk image block by performing superposition aggregation processing.

[0130] In an embodiment of the present invention, the aggregation output unit can be used to perform a superposition aggregation process on the target depth image feature representation corresponding to the junk image block and the depth junk image feature representation corresponding to the junk image block after the weight coefficients are fused. Exemplarily, the weight coefficient of the target depth image feature representation can be greater than the weight coefficient of the depth junk image feature representation. For example, the weight coefficient of the target depth image feature representation can be 1, and the weight coefficient of the depth junk image feature representation can be 0.4, 0.5, 0.6, etc. In this way, the aggregated junk image feature representation = target depth image feature representation * weight coefficient corresponding to the target depth image feature representation + depth junk image feature representation * weight coefficient corresponding to the depth junk image feature representation.

[0131] Alternatively, in a specific implementation, in order to make the aggregation effect of the image block association parameter distribution and the deep garbage image feature representation more reliable, the fourth number of deep feature aggregation units includes any deep feature aggregation unit (that is, any one deep feature aggregation unit). Based on this, the above step S123c may further include the following specific implementation content:

[0132] First, when the arbitrary deep feature aggregation unit belongs to the deep feature aggregation unit ranked first among the fourth number of deep feature aggregation units, the arbitrary deep feature aggregation unit can be used to associate and aggregate the image block association parameter distribution corresponding to the original junk image and the deep junk image feature representation corresponding to the junk image block, and output the intermediate junk image feature representation corresponding to the arbitrary deep feature aggregation unit;

[0133] Secondly, when the arbitrary deep feature aggregation unit belongs to other depth feature aggregation units other than the depth feature aggregation unit ranked first among the fourth number of depth feature aggregation units (such as the depth feature aggregation unit ranked second, third or last), the arbitrary deep feature aggregation unit can be used to perform association aggregation processing on the image block association parameter distribution corresponding to the original junk image and the intermediate junk image feature representation corresponding to the previous depth feature aggregation unit of the arbitrary deep feature aggregation unit (exemplarily, the association aggregation processing method can be consistent with the previous step, except that the processing object is changed from the image block association parameter distribution and the deep junk image feature representation to the image block association parameter distribution and the intermediate junk image feature representation), and output the intermediate junk image feature representation corresponding to the arbitrary deep feature aggregation unit.

[0134] Wherein, it can be selected that, in a specific implementation manner, the number of the junk image blocks is equal to a fifth number, the fifth number of junk image blocks includes any junk image block (i.e., any one junk image block), and the fifth number is greater than 1. The image block association parameter distribution corresponding to the original junk image includes image block association parameters respectively possessed by the arbitrary junk image block relative to the fifth number of junk image blocks (such as similarities between junk image blocks, which can be obtained by calculating according to conventional image similarities, or by calculating cosine similarity of corresponding junk image feature representations), and the deep junk image feature representation corresponding to the junk image block includes the deep junk image feature representation corresponding to the arbitrary junk image block.

[0135] Based on this, when the arbitrary deep feature aggregation unit belongs to the deep feature aggregation unit ranked first among the fourth number of deep feature aggregation units, the image block association parameter distribution corresponding to the original junk image and the deep junk image feature representation corresponding to the junk image block are associated and aggregated by using the arbitrary deep feature aggregation unit, and the step of outputting the intermediate junk image feature representation corresponding to the arbitrary deep feature aggregation unit may further include the following contents:

[0136] First, when the arbitrary deep feature aggregation unit belongs to the deep feature aggregation unit ranked first among the fourth number of deep feature aggregation units, the correlation feature representations of the arbitrary junk image blocks in the arbitrary deep feature aggregation unit relative to the fifth number of junk image blocks can be obtained; illustratively, the correlation feature representations can be used as parameters of the deep feature aggregation unit and updated in the corresponding training process;

[0137] Secondly, the arbitrary deep feature aggregation unit can be used to perform cascade combination processing on the deep junk image feature representation corresponding to the arbitrary junk image block and the fifth number of the correlation feature representations, and output the cascade correlation feature representations corresponding to each of the fifth number of junk image blocks, for example, the deep junk image feature representation is cascaded and combined with the first correlation feature representation to obtain a first cascade correlation feature representation; the deep junk image feature representation is cascaded and combined with the second correlation feature representation to obtain a second cascade correlation feature representation;

[0138] Then, based on the fifth number of the image block association parameters, the fifth number of the cascade association feature representations may be weighted superposition calculated to output the image association feature representation corresponding to the arbitrary junk image block; illustratively, the image association feature representation=the first image block association parameter*the first cascade association feature representation+the second image block association parameter*the second cascade association feature representation+the third image block association parameter*the third cascade association feature representation+...;

[0139] Finally, the image correlation feature representation corresponding to the arbitrary junk image block can be subjected to an excitation mapping operation (the specific excitation function is not limited and can be selected according to actual needs, such as a Parametric ReLU function), and the excitation junk image feature representation corresponding to the arbitrary junk image block can be output, and based on the excitation junk image feature representation corresponding to the arbitrary junk image block, the intermediate junk image feature representation corresponding to the arbitrary deep feature aggregation unit can be determined; illustratively, the excitation junk image feature representation can be directly used as the corresponding intermediate junk image feature representation.

[0140] Alternatively, in a specific implementation, in order to reliably perform image optimization processing on the original junk image based on the aggregated junk image feature representation, the above step S124 may further include the following contents:

[0141] The image reconstruction model included in the target junk image optimization network can be used to perform image reconstruction processing on the original junk image based on the aggregated junk image feature representation corresponding to each junk image block in the original junk image to obtain a corresponding reconstructed junk image, and the reconstructed junk image is used as the corresponding optimized junk image; illustratively, the image reconstruction model can be a decoding neural network, so that the aggregated junk image feature representation can be decoded by a decoding neural network (such as a generator in a generative adversarial network (GAN) or a decoder in an autoencoder) to obtain an optimized junk image; or

[0142] The aggregated junk image feature representation can be subjected to a linear mapping operation (such as full connection, and the parameters of the full connection can be formed during the training process) to output the mapped junk image feature representation corresponding to the junk image block, and the mapped junk image feature representation can be subjected to a mapping output operation (such as excitation mapping) to output the probability parameter distribution corresponding to the mapped junk image feature representation, and the probability parameter with the maximum value is determined in the probability parameter distribution, and based on the image pixel adjustment method corresponding to the probability parameter (that is, the probability parameter distribution refers to the probabilities corresponding to various image pixel adjustment methods), the junk image block is subjected to a pixel adjustment operation to obtain the corresponding optimized junk image block, and the optimized junk image blocks corresponding to each junk image block in the original junk image are spliced ​​to output the optimized junk image.

[0143] In a second aspect, the intelligent garbage identification and classification method may further include a training step of the target garbage image optimization network, and the training step may include the following contents:

[0144] First, a candidate junk image optimization network, a training first junk image, and a training second junk image corresponding to the training first junk image may be determined, wherein the training first junk image is formed by adjusting image information of the training second junk image, such as partially covering the training second junk image with other junk images;

[0145] Secondly, the candidate junk image optimization network may be used to perform image information mining operations on junk image blocks in the first training junk image, and output a first number of junk image feature representations in training corresponding to the junk image blocks in the first training junk image. For details, reference may be made to the explanation of step S121 in the previous text.

[0146] Then, the candidate junk image optimization network may be used to fuse the first number of junk image feature representations in training, output the fused junk image feature representations in training corresponding to the junk image blocks in the first junk image in training, and perform focused feature mining on the fused junk image feature representations in training, outputting the focused junk image feature representations in training of the fused junk image feature representations in training, for details, refer to the explanation of step S122 in the previous text;

[0147] Afterwards, the candidate junk image optimization network may be used to perform multi-level aggregation processing on the image block association parameter distribution in training corresponding to the first junk image in training and the target junk image feature representation in training corresponding to the junk image block in the first junk image in training, and output the aggregated junk image feature representation in training corresponding to the junk image block in the first junk image in training, wherein the target junk image feature representation in training is obtained based on the focused junk image feature representation in training and the fused junk image feature representation in training, and the details may refer to the explanation of step S123 in the previous text;

[0148] Further, the candidate junk image optimization network can be used to perform image optimization processing on the training first junk image based on the feature representation of the aggregated junk image in the training to obtain an optimized first junk image. For details, reference can be made to the explanation of step S124 in the previous text.

[0149] Finally, the candidate junk image optimization network can be trained and updated based on the optimized first junk image and the trained second junk image, and a corresponding target junk image optimization network can be output; illustratively, the difference (such as mean square error) between the optimized first junk image and the trained second junk image can be calculated, and then the candidate junk image optimization network can be trained and updated in a direction of reducing the difference until the difference is reduced to a preset value or the number of training updates reaches a preset number, and then the training is stopped, thereby obtaining a target junk image optimization network; in addition, in other embodiments, as described above, a probability parameter distribution can be obtained in step S124, and accordingly, in Before obtaining the optimized first junk image, the corresponding training probability parameter distribution is also obtained. In this way, the corresponding image pixel adjustment method in training can be determined based on the training probability parameter distribution. Then, the image pixel adjustment method in training and the actual image pixel adjustment method (the actual image pixel adjustment method can be obtained based on the difference between the training first junk image and the training second junk image) can be calculated to obtain the corresponding error. In this way, the candidate junk image optimization network can also be trained and updated in the direction of reducing the error until the error is reduced to a preset value or the number of training updates reaches a preset number, and then it is stopped, thereby obtaining the target junk image optimization network.

[0150] In summary, the present invention provides an intelligent garbage identification and classification method. First, the garbage transported by the target garbage transfer vehicle is imaged and processed to obtain an original garbage image; second, the target garbage image optimization network is used to optimize the original garbage image to obtain an optimized garbage image; then, the target garbage type recognition network is used to identify the optimized garbage image to obtain target garbage type information. Based on the above method, the original garbage image is optimized before the garbage type recognition process is performed, that is, the lost image information of the garbage to be identified in the original garbage image is restored, so that the lost image information can be restored to ensure the integrity of the image information of the garbage to be identified. In this way, the reliability of the garbage type recognition process based on the complete image information can be guaranteed. Therefore, the problem of low reliability of garbage recognition in the prior art can be improved.

[0151] In several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the apparatus, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0152] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0153] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code. It should be noted that in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0154] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An intelligent garbage identification and classification method, characterized in that: include: Perform image acquisition and processing on the garbage transferred by the target garbage transfer vehicle to obtain the original garbage image; Using a target junk image optimization network, the original junk image is subjected to image optimization processing to obtain an optimized junk image, wherein the target junk image optimization network is a trained neural network, and the image optimization processing refers to restoring lost image information of the junk to be identified in the original junk image, wherein the lost image information at least includes image information of the junk to be identified that is lost due to occlusion by other junk or other objects, and the image optimization processing includes mining multiple levels of key information of the image information and restoring the corresponding optimized junk image based on the result of the mining operation, wherein the multiple levels of key information include key information of multiple color channels and key information of extracted contours; Using a target garbage type recognition network, the optimized garbage image is subjected to garbage type recognition processing to obtain target garbage type information of the garbage to be recognized, wherein the target garbage type recognition network is a trained neural network, and the target garbage type information is used as a basis for classifying and monitoring the garbage transferred by the target garbage transfer vehicle, and the classification monitoring processing at least includes performing a corresponding classification abnormality warning operation when it is determined that there is misclassified garbage; The step of using the target junk image optimization network to perform image optimization processing on the original junk image to obtain an optimized junk image includes: Using a target junk image optimization network, performing an image information mining operation on a junk image block in the original junk image, and outputting a first number of junk image feature representations corresponding to the junk image block, wherein the first number is greater than 1, and the first number of junk image feature representations are used to reflect key information of the junk image block at different first number levels, the junk image block is formed by performing image segmentation processing on the original junk image, and the key information of the junk image block at different first number levels includes key information of multiple color channels and key information of contours extracted from the junk image block; Using the target junk image optimization network, fusing the first number of junk image feature representations, outputting a fused junk image feature representation corresponding to the junk image block, and performing focused feature mining on the fused junk image feature representation, outputting a focused junk image feature representation corresponding to the fused junk image feature representation; Using the target junk image optimization network, performing multi-level aggregation processing on the image block association parameter distribution corresponding to the original junk image and the target junk image feature representation corresponding to the junk image block, and outputting the aggregated junk image feature representation corresponding to the junk image block, wherein the target junk image feature representation is obtained based on the focused junk image feature representation and the fused junk image feature representation, and the image block association parameter distribution is used to reflect the association relationship between each of the junk image blocks; Using the target junk image optimization network, performing image optimization processing on the original junk image based on the aggregated junk image feature representation to obtain an optimized junk image; The step of using the target junk image optimization network to fuse the first number of junk image feature representations, outputting the fused junk image feature representation corresponding to the junk image block, and performing focused feature mining on the fused junk image feature representation, and outputting the focused junk image feature representation corresponding to the fused junk image feature representation, comprises: Loading the first number of junk image feature representations so that the image feature focusing model in the target junk image optimization network acquires the first number of junk image feature representations; Using the image feature focusing model, cascade combining the first number of junk image feature representations, and outputting a fused junk image feature representation corresponding to the junk image block; Performing matrix mapping processing on the fused garbage image feature representation, and outputting a third number of focus mapping image feature representations corresponding to the fused garbage image feature representation; Performing similarity fusion processing on the third number of focus map image feature representations, and outputting a focused garbage image feature representation corresponding to the fused garbage image feature representation; Wherein, the third number of focus mapping image feature representations includes a first mapping image feature representation, a second mapping image feature representation and a third mapping image feature representation; The step of performing similarity fusion processing on the third number of focus map image feature representations and outputting the focused garbage image feature representation corresponding to the fused garbage image feature representation comprises: Performing parameter adjustment processing on the first mapping image feature representation so that the distribution of parameters in the first mapping image feature representation is adjusted to obtain an adjusted mapping image feature representation; multiplying the third mapped image feature representation and the adjusted mapped image feature representation to output a corresponding correlation parameter distribution, wherein the correlation parameter distribution is used to reflect the correlation relationship between the first number of junk image feature representations; Performing parameter mapping processing on the correlation parameter distribution, and outputting a mapped correlation parameter distribution corresponding to the correlation parameter distribution; multiplying the mapping correlation parameter distribution and the second mapping image feature representation, and outputting the correlation image feature representations corresponding to the first number of junk image feature representations; A linear mapping operation is performed on the associated image feature representation to output a focused garbage image feature representation corresponding to the fused garbage image feature representation.

2. The intelligent garbage identification and classification method according to claim 1, characterized in that: The first number of junk image feature representations include junk image color feature representations; The step of using the target junk image optimization network to perform image information mining operations on junk image blocks in the original junk image and outputting a first number of junk image feature representations corresponding to the junk image blocks includes: Loading the original junk image so that the image information mining model in the target junk image optimization network can obtain the original junk image; Determine a second number of color channel values ​​of a junk image block in the original junk image by using the image information mining model, wherein the second number is greater than 1, and the second number of color channel values ​​includes a value of each pixel in the junk image block in a color channel; Determine color channel value feature representations corresponding to each of the second number of color channel values, and perform feature extraction processing on each of the second number of color channel value feature representations, and output extracted color feature representations corresponding to each of the second number of color channel value feature representations; The output second number of the extracted color feature representations are subjected to feature compression processing, and the garbage image color feature representation corresponding to the garbage image block is output.

3. The intelligent garbage identification and classification method according to claim 1, characterized in that: The first number of junk image feature representations include junk image contour feature representations; The step of using the target junk image optimization network to perform image information mining operations on junk image blocks in the original junk image and outputting a first number of junk image feature representations corresponding to the junk image blocks includes: Performing contour extraction processing on the original garbage image to determine whether each pixel in the original garbage image belongs to the contour, and based on the result of whether it belongs to the contour, updating the pixel value of each pixel in the original garbage image to obtain a corresponding updated garbage image, wherein in the updated garbage image, if a pixel belongs to the contour, the pixel value of the pixel is 1, and if a pixel does not belong to the contour, the pixel value of the pixel is 0; Loading the updated junk image so that the image information mining model in the target junk image optimization network acquires the updated junk image; Determine a second number of contour representation values ​​of the junk image block in the updated junk image by using the image information mining model, wherein the second number is greater than 1, and the second number of contour representation values ​​includes a representation value reflecting whether each pixel point in the junk image block belongs to the contour; Determine the contour characterization value feature representations corresponding to each of the second number of contour characterization values, and perform feature extraction processing on each of the second number of contour characterization value feature representations, and output the extracted contour feature representations corresponding to each of the second number of contour characterization value feature representations; The output second number of the extracted contour feature representations are subjected to feature compression processing, and the garbage image contour feature representation corresponding to the garbage image block is output.

4. The intelligent garbage identification and classification method according to claim 1, characterized in that: The step of using the target junk image optimization network to perform multi-level aggregation processing on the image block association parameter distribution corresponding to the original junk image and the target junk image feature representation corresponding to the junk image block, and outputting the aggregated junk image feature representation corresponding to the junk image block includes: Loading and processing the image block associated parameter distribution corresponding to the original junk image and the target junk image feature representation corresponding to the junk image block, so that the deep aggregation model in the target junk image optimization network obtains the image block associated parameter distribution and the target junk image feature representation, wherein the deep aggregation model includes a deep feature mining unit, a fourth number of deep feature aggregation units and an aggregation output unit, and the fourth number is greater than or equal to 1; Using the deep feature mining unit, performing a deep feature mining operation on the target junk image feature representation corresponding to the junk image block, and outputting the deep junk image feature representation corresponding to the junk image block; Outputting, using the fourth number of deep feature aggregation units, intermediate junk image feature representations corresponding to each of the fourth number of deep feature aggregation units according to the image block association parameter distribution corresponding to the original junk image and the deep junk image feature representations corresponding to the junk image blocks; Marking the intermediate junk image feature representation corresponding to the target deep feature aggregation unit in the fourth number of deep feature aggregation units to form a target deep image feature representation corresponding to the junk image block, wherein the target deep feature aggregation unit belongs to the last deep feature aggregation unit in the fourth number of deep feature aggregation units, and the fourth number of deep feature aggregation units have a cascade connection order relationship; By using the aggregation output unit, after the target depth image feature representation corresponding to the junk image block and the depth junk image feature representation corresponding to the junk image block are fused with weight coefficients, an aggregated junk image feature representation corresponding to the junk image block is output by performing superposition aggregation processing.

5. The intelligent garbage identification and classification method according to claim 4, characterized in that: The fourth number of deep feature aggregation units includes any deep feature aggregation unit; The step of using the fourth number of deep feature aggregation units to output the intermediate junk image feature representations corresponding to each of the fourth number of deep feature aggregation units according to the image block association parameter distribution corresponding to the original junk image and the deep junk image feature representations corresponding to the junk image blocks comprises: When the arbitrary deep feature aggregation unit belongs to the deep feature aggregation unit ranked first among the fourth number of deep feature aggregation units, using the arbitrary deep feature aggregation unit, the image block association parameter distribution corresponding to the original junk image and the deep junk image feature representation corresponding to the junk image block are associated and aggregated, and an intermediate junk image feature representation corresponding to the arbitrary deep feature aggregation unit is output; When the arbitrary deep feature aggregation unit belongs to other depth feature aggregation units other than the depth feature aggregation unit ranked first among the fourth number of depth feature aggregation units, the arbitrary deep feature aggregation unit is used to associate and aggregate the image block association parameter distribution corresponding to the original junk image and the intermediate junk image feature representation corresponding to the previous depth feature aggregation unit of the arbitrary deep feature aggregation unit, and output the intermediate junk image feature representation corresponding to the arbitrary deep feature aggregation unit.

6. The intelligent garbage identification and classification method according to claim 1, characterized in that: The step of using the target junk image optimization network to perform image optimization processing on the original junk image based on the aggregated junk image feature representation to obtain an optimized junk image comprises: Using the image reconstruction model included in the target junk image optimization network, based on the aggregated junk image feature representation corresponding to each junk image block in the original junk image, performing image reconstruction processing on the original junk image to obtain a corresponding reconstructed junk image, and using the reconstructed junk image as the corresponding optimized junk image; or The aggregated junk image feature representation is linearly mapped to output the mapped junk image feature representation corresponding to the junk image block, and the mapped junk image feature representation is mapped and output to output the probability parameter distribution corresponding to the mapped junk image feature representation, and the probability parameter with the maximum value is determined in the probability parameter distribution, and based on the image pixel adjustment method corresponding to the probability parameter, the junk image block is pixel adjusted to obtain the corresponding optimized junk image block, and the optimized junk image blocks corresponding to each junk image block in the original junk image are spliced ​​to output the optimized junk image.

7. The intelligent garbage identification and classification method according to any one of claims 1 to 6, characterized in that: The intelligent garbage identification and classification method further includes a training step of the target garbage image optimization network, and the training step includes: Determining a candidate junk image optimization network, a training first junk image, and a training second junk image corresponding to the training first junk image, wherein the training first junk image is formed by performing image information adjustment processing on the training second junk image; Using the candidate junk image optimization network, performing an image information mining operation on the junk image blocks in the first training junk image, and outputting a first number of junk image feature representations in training corresponding to the junk image blocks in the first training junk image; Using the candidate junk image optimization network, fusing the first number of junk image feature representations in training, outputting the fused junk image feature representations in training corresponding to the junk image blocks in the first junk image in training, and performing focused feature mining on the fused junk image feature representations in training, outputting the focused junk image feature representations in training of the fused junk image feature representations in training; Using the candidate junk image optimization network, performing multi-level aggregation processing on the image block association parameter distribution in training corresponding to the first junk image in training and the target junk image feature representation in training corresponding to the junk image block in the first junk image in training, and outputting the aggregated junk image feature representation in training corresponding to the junk image block in the first junk image in training, wherein the target junk image feature representation in training is obtained based on the focused junk image feature representation in training and the fused junk image feature representation in training; Using the candidate junk image optimization network, based on the aggregated junk image feature representation in the training, the image optimization process is performed on the training first junk image to obtain an optimized first junk image; Based on the optimized first junk image and the trained second junk image, the candidate junk image optimization network is trained and updated, and a corresponding target junk image optimization network is output.

Citation Information

Patent Citations

  • Human face image shading removing method based on deep learning

    CN106910176A

  • Method for automatically classifying garbage

    CN112241679A

  • Supervision method for recoverable garbage, transportation equipment and storage medium

    CN113850124A

  • Waste image classification methods, devices, electronic equipment and storage media

    CN114937179A

  • Garbage classification digital management method based on vision

    CN117392465A