Intelligent monitoring method and system for garbage classification and delivery point of environmental sanitation

Abstract

The application provides an intelligent monitoring method and system for a sanitation garbage classification and delivery point. The first garbage image semantic features corresponding to a garbage collection image are calculated, the second garbage image semantic features corresponding to each network branch of a trained dangerous garbage prediction network are determined based on the first garbage image semantic features, the second garbage image semantic features are then loaded to the network branches corresponding to the dangerous garbage prediction network, the dangerous garbage prediction branch parameters generated by each network branch are used to determine dangerous garbage prediction data, and then the dangerous garbage prediction data is used to perform a dangerous garbage early warning on the garbage collection image. Thus, the dangerous garbage prediction network based on multiple network branches can improve the accuracy of dangerous garbage prediction by determining the target dangerous garbage prediction data based on the dangerous garbage prediction branch parameters generated by each network branch.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to an intelligent monitoring method and system for a garbage classification and disposal point. BACKGROUND

[0002] Dangerous garbage has certain dangerous characteristics, including corrosiveness, toxicity, flammability, reactivity, and infectivity. At present, for each garbage disposal area for providing garbage disposal services, there is a lack of intelligent monitoring and identification of dangerous garbage. How to combine an intelligent garbage disposal terminal to perform image acquisition and analysis during garbage disposal, thereby improving the prediction accuracy of dangerous garbage, is a technical problem that needs to be solved at present. SUMMARY

[0003] Therefore, the purpose of the present application is to provide an intelligent monitoring method and system for a garbage classification and disposal point.

[0004] According to a first aspect of the present application, an intelligent monitoring method for a garbage classification and disposal point is provided, applied to a cloud server, and the method comprises:

[0005] obtaining garbage collection images uploaded by each garbage disposal terminal for dangerous garbage analysis, and calculating first garbage image semantic features corresponding to the garbage collection images based on different image semantic encoding networks;

[0006] determining second garbage image semantic features corresponding to each network branch of a trained dangerous garbage prediction network based on the first garbage image semantic features;

[0007] loading the second garbage image semantic features corresponding to each network branch to the network branch corresponding to the dangerous garbage prediction network, the dangerous garbage prediction network comprising a plurality of network branches, each network branch having a corresponding unit dangerous garbage prediction network, the dangerous garbage prediction network being configured to determine dangerous garbage prediction data based on dangerous garbage prediction branch parameters generated by each unit dangerous garbage prediction network, the dangerous garbage prediction branch parameters representing dangerous garbage prediction data of the corresponding network branch, the dangerous garbage prediction branch parameters comprising a heat value corresponding to each dangerous garbage label, the unit dangerous garbage prediction network and the dangerous garbage prediction network being an integral whole, when the dangerous garbage prediction branch parameters generated by the unit dangerous garbage prediction networks are the same, the dangerous garbage prediction data of the network branch corresponding to each unit dangerous garbage prediction network is taken as the dangerous garbage prediction data generated by the dangerous garbage prediction network, and when the dangerous garbage prediction branch parameters generated by the unit dangerous garbage prediction networks are different, the dangerous garbage prediction data of the network branch corresponding to the unit dangerous garbage prediction network with the largest heat value is taken as the dangerous garbage prediction data generated by the dangerous garbage prediction network.

[0008] obtain dangerous garbage prediction data generated by the dangerous garbage prediction network, and perform a dangerous garbage early warning on the garbage collection image based on the dangerous garbage prediction data.

[0009] In a possible implementation of the first aspect, the step of calculating the first garbage image semantic feature corresponding to the garbage collection image comprises:

[0010] performing image segmentation on the garbage collection image to generate a plurality of target garbage image segments;

[0011] extracting an image segment semantic feature corresponding to each target garbage image segment, and determining the first garbage image semantic feature corresponding to the garbage collection image based on the image segment semantic feature corresponding to each target garbage image segment.

[0012] In a possible implementation of the first aspect, the step of performing image segmentation on the garbage collection image to generate a plurality of target garbage image segments is followed by:

[0013] obtaining a noise image template feature sequence, and filtering out the garbage collection image when any one of the plurality of target garbage image segments matches a noise image template feature in the noise image template feature sequence;

[0014] when any one of the plurality of target garbage image segments does not match a noise image template feature in the noise image template feature sequence, proceeding to the step of extracting an image segment semantic feature corresponding to each target garbage image segment.

[0015] In a possible implementation of the first aspect, the dangerous garbage prediction network comprises a first network branch and a second network branch.

[0016] The step of extracting an image segment semantic feature corresponding to each target garbage image segment, and determining the first garbage image semantic feature corresponding to the garbage collection image based on the image segment semantic feature corresponding to each target garbage image segment comprises:

[0017] extracting a first image segment semantic feature corresponding to each target garbage image segment based on a first image semantic encoding network, and obtaining a first first garbage image semantic feature based on the first image segment semantic feature;

[0018] extracting a second image segment semantic feature corresponding to each target garbage image segment based on a second image semantic encoding network, and obtaining a second first garbage image semantic feature based on the second image segment semantic feature;

[0019] The step of determining the second garbage image semantic feature corresponding to each network branch of the trained dangerous garbage prediction network based on the first garbage image semantic feature comprises:

[0020] The first first garbage image semantic feature is taken as the second garbage image semantic feature of the first network branch, and the second first garbage image semantic feature is taken as the second garbage image semantic feature of the second network branch.

[0021] In a possible implementation of the first aspect, the step of extracting the first image block semantic feature corresponding to each target garbage image block based on the first image semantic encoding network and obtaining the first first garbage image semantic feature based on the first image block semantic feature comprises:

[0022] A block space covering the plurality of target garbage image blocks is established, and the first image block semantic feature of each target garbage image block is determined based on the feature repetition frequency of the target garbage image block in the garbage collection image;

[0023] The first first garbage image semantic feature corresponding to the garbage collection image is determined based on the feature weight position of each target garbage image block in the block space and the first image block semantic feature of each target garbage image block;

[0024] The step of extracting the second image block semantic feature corresponding to each target garbage image block based on the second image semantic encoding network and obtaining the second garbage image semantic feature based on the second image block semantic feature comprises:

[0025] An image block semantic feature library corresponding to the second image semantic encoding network is obtained, and the second image block semantic feature corresponding to each target garbage image block is obtained from the image block semantic feature library;

[0026] The second first garbage image semantic feature corresponding to the garbage collection image is constructed based on the second image block semantic feature corresponding to each target garbage image block.

[0027] In a possible implementation of the first aspect, the step of determining the second garbage image semantic feature corresponding to each network branch of the trained dangerous garbage prediction network based on the first garbage image semantic feature further comprises:

[0028] A sample garbage collection image is obtained, and a first sample garbage image semantic feature corresponding to the sample garbage collection image is calculated;

[0029] The second sample garbage image semantic feature corresponding to each network branch of the initial dangerous garbage prediction network is determined based on the first sample garbage image semantic feature.

[0030] obtain the dangerous annotation label data corresponding to the sample garbage collection image;

[0031] load the second sample garbage image semantic features corresponding to each network branch to the network branch corresponding to the initial dangerous garbage prediction network, update the weight parameters of the initial dangerous garbage prediction network by taking the dangerous annotation label data as the learning output data of the initial dangerous garbage prediction network, and generate a target dangerous garbage prediction network, which is the trained dangerous garbage prediction network, and which includes a plurality of unit dangerous garbage prediction networks, each of which includes different function weight definition information.

[0032] In a possible implementation of the first aspect, before the step of determining the second garbage image semantic features corresponding to each network branch of the trained dangerous garbage prediction network based on the first garbage image semantic features, the method further includes: obtaining a sample garbage collection image, performing image blocking on the sample garbage collection image, and generating a plurality of sample garbage image blocks.

[0033] extract first learning image block semantic features corresponding to each sample garbage image block based on a first image semantic encoding network, and obtain first sample garbage image semantic features based on the first learning image block semantic features; extract second learning image block semantic features corresponding to each sample garbage image block based on a second image semantic encoding network, and obtain second sample garbage image semantic features based on the second learning image block semantic features; obtain the dangerous annotation label data corresponding to the sample garbage collection image;

[0034] take the first sample garbage image semantic features as the network input of a first network branch in an initial dangerous garbage prediction network, take the second sample garbage image semantic features as the network input of a second network branch in the initial dangerous garbage prediction network, update the weight parameters of the dangerous garbage prediction network by taking the dangerous annotation label data as the learning output data of the initial dangerous garbage prediction network, and generate a target dangerous garbage prediction network, which is the trained dangerous garbage prediction network, and which includes a member prediction unit corresponding to the first network branch and a member prediction unit corresponding to the second network branch.

[0035] In a possible implementation of the first aspect, the unit dangerous garbage prediction network includes a feature item selection unit based on expected cross-entropy, a feature dimension reduction unit, and a fully connected unit.

[0036] The second garbage image semantic features corresponding to each network branch are loaded into the network branch corresponding to the dangerous garbage prediction network, the dangerous garbage prediction network includes a plurality of network branches, and a corresponding unit dangerous garbage prediction network exists in each network branch. The step of determining the dangerous garbage prediction data based on the dangerous garbage prediction branch parameters generated by each unit dangerous garbage prediction network includes:

[0037] The second garbage image semantic features corresponding to the network branch are taken as the network input of the feature item selection unit based on the expected cross-entropy in the corresponding network branch. The feature item selection unit based on the expected cross-entropy is used for feature item selection based on the expected cross-entropy to obtain a first vector selection array. The first vector selection array is fused and transposed as the network input of the neuron input mapping unit to obtain a second vector selection array.

[0038] The second vector selection array is taken as the network input of the feature dimension reduction unit. The feature dimension reduction unit is used for performing feature dimension reduction processing on each selection vector in the second vector selection array to obtain a corresponding third vector selection array.

[0039] The third vector selection array is taken as the network input of the fully connected unit. The fully connected unit is used for fully connected output based on the third vector selection array to obtain dangerous garbage prediction branch parameters.

[0040] The dangerous garbage prediction branch parameters obtained by each network branch are taken as the network input of the convergence unit in the dangerous garbage prediction network. The convergence unit is used for converging and weighting based on the dangerous garbage prediction branch parameters generated by each network branch to obtain target dangerous garbage prediction data.

[0041] In a possible implementation of the first aspect, the step of taking the dangerous garbage prediction branch parameters obtained by each network branch as the network input of the convergence unit in the dangerous garbage prediction network, and the convergence unit is used for converging and weighting based on the dangerous garbage prediction branch parameters generated by each network branch to obtain target dangerous garbage prediction data includes:

[0042] The dangerous garbage prediction branch parameters are calculated based on each network branch. The dangerous garbage prediction branch parameters include a heat value corresponding to each dangerous garbage label. The branch dangerous garbage prediction data corresponding to the network branch is determined.

[0043] If the branch dangerous garbage prediction data of the plurality of network branches are the same, the branch dangerous garbage prediction data is taken as the target dangerous garbage prediction data.

[0044] If branch dangerous garbage prediction data of the plurality of network branches are different, branch dangerous garbage prediction data with the largest heat value is taken as target dangerous garbage prediction data.

[0045] According to a second aspect of the present application, a cloud server is provided, the cloud server comprising a machine readable storage medium and a processor, the machine readable storage medium storing machine executable instructions, and the processor, when executing the machine executable instructions, implements the intelligent monitoring method of the garbage classification disposal point.

[0046] According to a third aspect of the present application, a computer readable storage medium is provided, the computer readable storage medium storing computer executable instructions, and when the computer executable instructions are executed, the intelligent monitoring method of the garbage classification disposal point is implemented.

[0047] According to any of the above aspects, in the present application, a garbage collection image to be analyzed for dangerous garbage is obtained, and then a first garbage image semantic feature corresponding to the garbage collection image is calculated, a second garbage image semantic feature corresponding to each network branch of the trained dangerous garbage prediction network is determined based on the first garbage image semantic feature, and then the second garbage image semantic feature is loaded to the network branch corresponding to the dangerous garbage prediction network. Each network branch has a corresponding unit dangerous garbage prediction network, and the dangerous garbage prediction network is used to determine dangerous garbage prediction data based on dangerous garbage prediction branch parameters generated by each unit dangerous garbage prediction network, and then the garbage collection image is warned for dangerous garbage based on the dangerous garbage prediction data. The dangerous garbage warning method for the garbage collection image is based on the dangerous garbage prediction network with multiple network branches, and the target dangerous garbage prediction data is determined by comprehensively generating dangerous garbage prediction branch parameters of each network branch, which can improve the accuracy of dangerous garbage prediction. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be considered as limiting the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0049] Figure 1 The flowchart of the intelligent monitoring method of the garbage classification disposal point provided by the embodiments of the present application is shown.

[0050] Figure 2 The component structure schematic diagram of the cloud server for implementing the intelligent monitoring method of the garbage classification disposal point provided by the embodiments of the present application is shown. DETAILED DESCRIPTION

[0051] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described clearly and completely below in accordance with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn according to the actual proportions. The flowcharts used in the present application show the operations implemented according to some embodiments of the embodiments of the present application. It should be understood that the operations of the flowcharts can not be implemented in sequence, and the steps without logical contextual relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flowcharts or one or more operations can be removed from the flowcharts under the guidance of the content of the present application.

[0052] In addition, the described embodiments are only some of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art without creative labor in accordance with the embodiments of the present application are within the scope of protection of the present application.

[0053] Figure 1 A flowchart of the intelligent monitoring method of the garbage classification and disposal point provided by the embodiments of the present application is shown. It should be understood that in other embodiments, the order of some steps of the intelligent monitoring method of the garbage classification and disposal point of the present embodiment can be shared according to actual needs, or some steps can be omitted or maintained. The intelligent monitoring method of the garbage classification and disposal point includes in detail:

[0054] In step S102, garbage collection images uploaded by each garbage disposal terminal for hazardous waste analysis are obtained, and a first garbage image semantic feature corresponding to the garbage collection images is calculated based on different image semantic coding networks.

[0055] The garbage image semantic feature refers to converting the garbage collection image into a semantic feature, which can include color features, texture features, shape features, and spatial relationship features of the garbage collection image, for example. The embodiment first needs to image block the garbage collection image to be analyzed for dangerous garbage to generate a plurality of garbage image blocks. Image blocking refers to cutting the garbage collection image into a plurality of individual garbage image blocks. Then, the image block semantic features of each garbage image block are extracted, and finally, the first garbage image semantic feature corresponding to the garbage collection image is determined based on the image block semantic features corresponding to each garbage image block.

[0056] In step S104, the second garbage image semantic features corresponding to each network branch of the trained dangerous garbage prediction network are determined based on the first garbage image semantic features.

[0057] The trained dangerous garbage prediction network refers to a network for predicting the dangerous garbage category of the garbage collection image that meets the network convergence condition. The dangerous garbage prediction network includes a plurality of network branches, and each network branch corresponds to a unit dangerous garbage prediction network. The second garbage image semantic features of each network branch are determined based on the obtained first garbage image semantic features. In an alternative embodiment, there is only one type of first garbage image semantic feature obtained, and the same first garbage image semantic feature can be used as the second garbage image semantic feature of each network branch. In another alternative embodiment, there are multiple types of first garbage image semantic features obtained, and different first garbage image semantic features can be used as the second garbage image semantic features of different network branches.

[0058] In step S106, the second garbage image semantic features corresponding to each network branch are loaded into the network branches corresponding to the dangerous garbage prediction network. The dangerous garbage prediction network includes a plurality of network branches, and each network branch corresponds to a unit dangerous garbage prediction network. The dangerous garbage prediction network is used to determine the dangerous garbage prediction data based on the dangerous garbage prediction branch parameters generated by each unit dangerous garbage prediction network.

[0059] The dangerous garbage prediction network comprises a plurality of network branches, and the network branches are in parallel relationship. Each network branch corresponds to a unit dangerous garbage prediction network. The unit dangerous garbage prediction network can be a network branch. The unit dangerous garbage prediction network is used for preliminary dangerous garbage analysis of the garbage collection image. The dangerous garbage prediction network is used for integrating the dangerous garbage prediction branch parameters generated by each unit dangerous garbage prediction network, i.e., comprehensively determining the final dangerous garbage prediction data of the garbage collection image for dangerous garbage warning based on the preliminary dangerous garbage analysis of each unit dangerous garbage prediction network. In an alternative embodiment, the second garbage image semantic features corresponding to each network branch are used as the network input of the corresponding network branch in the dangerous garbage prediction network. The dangerous garbage prediction network obtains the dangerous garbage prediction branch parameters generated by each network branch, and then determines the final dangerous garbage warning result by combining the dangerous garbage prediction branch parameters corresponding to each unit dangerous garbage prediction network.

[0060] In step S108, the dangerous garbage prediction data generated by the dangerous garbage prediction network is obtained, and the garbage collection image is warned of dangerous garbage based on the dangerous garbage prediction data.

[0061] The dangerous garbage prediction network is used for warning the dangerous garbage label of the garbage collection image, and outputs dangerous garbage prediction data, based on which the garbage collection image is warned.

[0062] Based on the above steps, the garbage collection image to be analyzed for dangerous garbage is obtained, the first garbage image semantic features corresponding to the garbage collection image are calculated, the second garbage image semantic features corresponding to each network branch of the trained dangerous garbage prediction network are determined based on the first garbage image semantic features, and then the second garbage image semantic features are loaded into the network branch corresponding to the dangerous garbage prediction network. Each network branch has a corresponding unit dangerous garbage prediction network. The dangerous garbage prediction network is used for determining the dangerous garbage prediction data based on the dangerous garbage prediction branch parameters generated by each unit dangerous garbage prediction network, and then warning the garbage collection image of dangerous garbage based on the dangerous garbage prediction data. The garbage collection image is warned of dangerous garbage based on the dangerous garbage prediction network with multiple network branches, and the target dangerous garbage prediction data is determined by comprehensively analyzing the dangerous garbage prediction branch parameters generated by each network branch, which can improve the accuracy of dangerous garbage prediction.

[0063] In an alternative embodiment, the step S102 of calculating the first garbage image semantic features corresponding to the garbage collection image comprises:

[0064] In step S102A, the garbage collection image is divided into a plurality of target garbage image blocks.

[0065] Step S102B, extracting the image block semantic features corresponding to each target garbage image block, and determining the first garbage image semantic features corresponding to the garbage collection image based on the image block semantic features corresponding to each target garbage image block.

[0066] In the embodiment, the garbage collection image to be analyzed for dangerous garbage is divided into multiple target garbage image blocks. Then, the image block semantic features corresponding to each target garbage image block are extracted, and the first garbage image semantic features corresponding to the garbage collection image are obtained after the image block semantic features corresponding to each target garbage image block are extracted.

[0067] In an alternative embodiment, after the step of dividing the garbage collection image into multiple target garbage image blocks, the method further includes: obtaining a sequence of noise image template features, and when any one of the multiple target garbage image blocks matches a noise image template feature in the sequence of noise image template features, filtering out the garbage collection image; and when any one of the multiple target garbage image blocks does not match a noise image template feature in the sequence of noise image template features, entering the step of extracting the image block semantic features corresponding to each target garbage image block.

[0068] In an alternative embodiment, the dangerous garbage prediction network includes a first network branch and a second network branch. The step S102B of extracting the image block semantic features corresponding to each target garbage image block and determining the first garbage image semantic features corresponding to the garbage collection image based on the image block semantic features corresponding to each target garbage image block includes: extracting the first image block semantic features corresponding to each target garbage image block based on a first image semantic encoding network, and obtaining the first first garbage image semantic features based on the first image block semantic features. Extracting the second image block semantic features corresponding to each target garbage image block based on a second image semantic encoding network, and obtaining the second first garbage image semantic features based on the second image block semantic features.

[0069] The dangerous garbage prediction network includes two network branches, i.e., a first network branch and a second network branch. The first network branch corresponds to a first unit dangerous garbage prediction network, and the second network branch corresponds to a second unit dangerous garbage prediction network. There are multiple models for calculating image block semantic features. Based on a first image semantic encoding network, a first image block semantic feature corresponding to each target garbage image block is extracted, and then a first garbage image semantic feature is obtained based on the first image block semantic feature aggregation. The second image semantic encoding network is different from the first image semantic encoding network. Based on the second image semantic encoding network, a second image block semantic feature corresponding to each target garbage image block is extracted, and then a second garbage image semantic feature is obtained based on the second image block semantic feature corresponding to each target garbage image block. The garbage image semantic feature is composed of multiple image block semantic features.

[0070] The step S104 of determining the second garbage image semantic feature corresponding to each network branch of the trained dangerous garbage prediction network based on the first garbage image semantic feature includes: taking the first garbage image semantic feature as the second garbage image semantic feature of the first network branch, and taking the second garbage image semantic feature as the second garbage image semantic feature of the second network branch.

[0071] The different garbage image semantic features are loaded into different network branches of the dangerous garbage prediction network. The first garbage image semantic feature is loaded into the first network branch of the dangerous garbage prediction network, and the second garbage image semantic feature is loaded into the second network branch of the dangerous garbage prediction network. By converting the garbage collection image into different image block semantic feature representations based on different image semantic encoding networks, and then respectively substituting into different network branches for parallel processing, the target dangerous garbage prediction data is obtained by synthesizing the dangerous garbage prediction branch parameters generated by the two network branches. By synthesizing the advantages of different image semantic encoding networks, the dangerous garbage prediction network can obtain more accurate dangerous garbage prediction data.

[0072] In an alternative embodiment, based on the first image semantic encoding network, a first image block semantic feature corresponding to each target garbage image block is extracted, and based on the first image block semantic feature aggregation, a first garbage image semantic feature is obtained. The specific operation can be: a block space covering multiple target garbage image blocks is established, the first image block semantic feature of each target garbage image block is determined based on the feature repetition frequency of each target garbage image block in the garbage collection image, and the first garbage image semantic feature corresponding to the garbage collection image is determined based on the feature weight position of each target garbage image block in the block space and the first image block semantic feature of each target garbage image block.

[0073] Wherein, after the plurality of target garbage image blocks are obtained by cutting the garbage collection image, a block space covering the plurality of target garbage image blocks is established, then the feature repetition frequency of each target garbage image block in the garbage collection image is counted, and then the first image block semantic feature corresponding to the target garbage image block is determined based on the feature repetition frequency. After the first image block semantic feature corresponding to each target garbage image block is determined, the first garbage image semantic feature corresponding to the garbage collection image is determined based on the feature weight position of the target garbage image block in the block space and the first image block semantic feature corresponding to the target garbage image block.

[0074] In an alternative embodiment, the second image block semantic feature corresponding to each target garbage image block is extracted based on the second image semantic encoding network, and the second garbage image semantic feature is obtained based on the second image block semantic feature. The specific operation can be: obtaining an image block semantic feature library corresponding to the second image semantic encoding network, obtaining the second image block semantic feature corresponding to each target garbage image block from the image block semantic feature library, and constructing the second first garbage image semantic feature corresponding to the garbage collection image based on the second image block semantic feature corresponding to each target garbage image block.

[0075] In an alternative embodiment, before the step of obtaining the dangerous garbage prediction parameter generated by each network branch of the dangerous garbage prediction network by taking the garbage image semantic feature as the network input of each network branch of the dangerous garbage prediction network, the dangerous garbage prediction network is established. The establishment of the dangerous garbage prediction network specifically includes the following steps:

[0076] Step S202, obtaining a sample garbage collection image, and calculating a first sample garbage image semantic feature corresponding to the sample garbage collection image.

[0077] Wherein, before using the dangerous garbage prediction network, the weight parameter of the dangerous garbage prediction network needs to be updated first. The sample garbage collection image in the sample garbage collection image sequence is obtained, and then the first sample garbage image semantic feature corresponding to the sample garbage collection image is calculated. First, the sample garbage collection image is image-blocked to generate a training target garbage image block, then the image block semantic feature corresponding to the training target garbage image block is extracted based on the image semantic encoding network, and then the first sample garbage image semantic feature corresponding to the sample garbage collection image is determined based on the image block semantic feature.

[0078] Step S204, determining the second sample garbage image semantic feature corresponding to each network branch of the dangerous garbage prediction network based on the first sample garbage image semantic feature.

[0079] The initial dangerous garbage prediction network includes multiple network branches, each network branch corresponds to a unit dangerous garbage prediction network, and the weight parameter of each network branch needs to be updated simultaneously. Therefore, the second sample garbage image semantic features corresponding to each network branch need to be determined simultaneously.

[0080] In step S206, the dangerous label data corresponding to the sample garbage collection image is obtained.

[0081] In step S208, the second sample garbage image semantic features corresponding to each network branch are loaded into the network branch corresponding to the dangerous garbage prediction network, the dangerous label data is used as the learning output data of the dangerous garbage prediction network, the weight parameter of the dangerous garbage prediction network is updated, and a target dangerous garbage prediction network is generated. The target dangerous garbage prediction network includes multiple unit dangerous garbage prediction networks, and the function weight definition information included in each unit dangerous garbage prediction network is different.

[0082] The second sample garbage image semantic features corresponding to each network branch are determined as the network input of the dangerous garbage prediction network, the dangerous label data is used as the learning output data of the dangerous garbage prediction network, the weight parameter of the dangerous garbage prediction network is updated, and a target dangerous garbage prediction network is generated. The dangerous garbage prediction network can be updated based on the convolutional neural network model during training. The target dangerous garbage prediction network includes multiple unit dangerous garbage prediction networks, and the function weight definition information included in each unit dangerous garbage prediction network is different, that is, each unit dangerous garbage prediction network is a network with different function weight definition information.

[0083] In an alternative embodiment, before the step of using the garbage image semantic features as the network input of each network branch of the dangerous garbage prediction network and obtaining the dangerous garbage prediction parameters generated by each network branch, the following steps are further included:

[0084] In step S302, a sample garbage collection image is obtained, and the sample garbage collection image is divided into blocks to generate multiple sample garbage image blocks.

[0085] The sample garbage collection image in the sample garbage collection image sequence is obtained, and the sample garbage collection image is divided into blocks to generate multiple sample garbage image blocks.

[0086] In step S304, the first learning image block semantic features corresponding to each sample garbage image block are extracted based on the first image semantic encoding network, and the first sample garbage image semantic features are obtained based on the first learning image block semantic features.

[0087] The initial dangerous garbage prediction network includes two network branches, which are a first network branch and a second network branch. The first network branch corresponds to a first unit dangerous garbage prediction network, and the second network branch corresponds to a second unit dangerous garbage prediction network. There are multiple models for calculating the image block semantic features. Based on the first image semantic encoding network, the first learning image block semantic features corresponding to each sample garbage image block are extracted, and then the first sample garbage image semantic features are obtained by aggregation based on the first learning image block semantic features. The first sample garbage image semantic features are composed of multiple first learning image block semantic features.

[0088] In step S306, the second learning image block semantic features corresponding to each sample garbage image block are extracted based on the second image semantic encoding network, and the second sample garbage image semantic features are obtained by aggregation based on the second learning image block semantic features.

[0089] The second image semantic encoding network is different from the first image semantic encoding network. The second learning image block semantic features corresponding to each sample garbage image block are extracted based on the second image semantic encoding network, and then the second sample garbage image semantic features are obtained by aggregation based on the second learning image block semantic features corresponding to each sample garbage image block.

[0090] In step S308, the dangerous label data corresponding to the sample garbage collection image is obtained.

[0091] In step S310, the first sample garbage image semantic features are input into the first network branch of the initial dangerous garbage prediction network, the second sample garbage image semantic features are input into the second network branch of the initial dangerous garbage prediction network, the dangerous label data is used as the learning output data of the dangerous garbage prediction network, the weight parameters of the dangerous garbage prediction network are updated, and a target dangerous garbage prediction network is generated. The target dangerous garbage prediction network includes a member prediction unit corresponding to the first network branch and a member prediction unit corresponding to the second network branch.

[0092] The first sample garbage image semantic features are input into the first network branch of the initial dangerous garbage prediction network, the second sample garbage image semantic features are input into the second network branch of the initial dangerous garbage prediction network, the dangerous label data is used as the learning output data of the dangerous garbage prediction network, the weight parameters of the dangerous garbage prediction network are updated, and a target dangerous garbage prediction network is generated. The target dangerous garbage prediction network includes two unit dangerous garbage prediction networks, and the function weight definition information included in the two unit dangerous garbage prediction networks is different, that is, the two unit dangerous garbage prediction networks are networks with different function weight definition information.

[0093] In an alternative implementation, the unit dangerous garbage prediction network includes an expected cross-entropy-based feature item selection unit, a feature dimension reduction unit, and a full connection unit. The second garbage image semantic features corresponding to each network branch are loaded into the network branch corresponding to the dangerous garbage prediction network, the dangerous garbage prediction network includes a plurality of network branches, and each network branch has a corresponding unit dangerous garbage prediction network. The dangerous garbage prediction network is used to determine the dangerous garbage prediction data based on the dangerous garbage prediction branch parameters generated by each unit dangerous garbage prediction network. The specific operation can be:

[0094] In step S402, the second garbage image semantic features corresponding to the network branch are taken as the network input of the expected cross-entropy-based feature item selection unit in the corresponding network branch. The expected cross-entropy-based feature item selection unit is used to perform expected cross-entropy-based feature item selection on the garbage image semantic features to obtain a first vector selection array. The first vector selection array is fused with a transposed array as the network input of the neuron input mapping unit to obtain a second vector selection array.

[0095] Each network branch corresponds to a unit dangerous garbage prediction network, and the unit dangerous garbage prediction network includes an expected cross-entropy-based feature item selection unit, a feature dimension reduction unit, and a full connection unit. First, the second garbage image semantic features corresponding to each network branch are taken as the network input of the expected cross-entropy-based feature item selection unit. The expected cross-entropy-based feature item selection unit performs expected cross-entropy-based feature item selection on the second garbage image semantic features to obtain a first vector selection array.

[0096] In step S404, the second vector selection array is taken as the network input of the feature dimension reduction unit. The feature dimension reduction unit is used to perform feature dimension reduction processing on each selection vector in the second vector selection array to obtain a corresponding third vector selection array.

[0097] The second vector selection array generated by the expected cross-entropy-based feature item selection unit is taken as the network input of the feature dimension reduction unit.

[0098] In step S406, the third vector selection array is taken as the network input of the full connection unit. The full connection unit is used to perform full connection output based on the third vector selection array to obtain the dangerous garbage prediction branch parameter.

[0099] In step S408, the dangerous garbage prediction branch parameter obtained by each network branch is taken as the network input of the convergence unit in the dangerous garbage prediction network. The convergence unit is used to perform convergence weighting based on the dangerous garbage prediction branch parameters generated by each network branch to obtain the target dangerous garbage prediction data.

[0100] The dangerous garbage prediction network further includes a converging unit, which is configured to receive the dangerous garbage prediction branch parameters generated by each network branch, and then perform converging weighting based on the dangerous garbage prediction branch parameters to obtain target dangerous garbage prediction data.

[0101] In an alternative implementation, the dangerous garbage prediction branch parameters obtained by each network branch are taken as network input of the converging unit in the dangerous garbage prediction network, and the converging unit is configured to perform converging weighting based on the dangerous garbage prediction branch parameters generated by each network branch to obtain target dangerous garbage prediction data. Specifically, the dangerous garbage prediction branch parameters obtained by each network branch include a heat value corresponding to each dangerous garbage label, and branch dangerous garbage prediction data corresponding to the network branch is determined. If the branch dangerous garbage prediction data of multiple network branches are the same, the branch dangerous garbage prediction data is taken as the target dangerous garbage prediction data. If the branch dangerous garbage prediction data of multiple network branches are different, the branch dangerous garbage prediction data with the largest heat value is taken as the target dangerous garbage prediction data.

[0102] The dangerous garbage prediction branch parameters include a heat value corresponding to each dangerous garbage label, and the dangerous garbage label with the largest heat value is taken as the branch dangerous garbage prediction data. After obtaining the branch dangerous garbage prediction data corresponding to multiple network branches, if the branch dangerous garbage prediction data of multiple network branches are the same, the branch dangerous garbage prediction data is taken as the target dangerous garbage prediction data. If the branch dangerous garbage prediction data of multiple network branches are different, the branch dangerous garbage prediction data with the largest heat value is taken as the target dangerous garbage prediction data.

[0103] In an alternative implementation, the other embodiment of the present application specifically includes the following steps:

[0104] In step S501, a sample garbage collection image is obtained, and the sample garbage collection image is divided into blocks to generate multiple sample garbage image blocks.

[0105] In step S502, first learning image block semantic features corresponding to each sample garbage image block are extracted based on a first image semantic encoding network, and first sample garbage image semantic features are obtained by converging the first learning image block semantic features.

[0106] In step S503, second learning image block semantic features corresponding to each sample garbage image block are extracted based on a second image semantic encoding network, and second sample garbage image semantic features are obtained by converging the second learning image block semantic features.

[0107] In step S504, dangerous label data corresponding to the sample garbage collection image is obtained.

[0108] Step S505, taking the first sample garbage image semantic feature as the network input of the first network branch, taking the second sample garbage image semantic feature as the network input of the second network branch, taking the dangerous label data as the learning output data of the dangerous garbage prediction network, updating the weight parameters of the dangerous garbage prediction network, generating a target dangerous garbage prediction network, and the target dangerous garbage prediction network includes a member prediction unit corresponding to the first network branch and a member prediction unit corresponding to the second network branch.

[0109] Step S506, acquiring a garbage collection image to be analyzed for dangerous garbage, performing image blocking on the garbage collection image, and generating a plurality of target garbage image blocks.

[0110] Step S507, acquiring a noise image template feature sequence, determining whether there is any noise image template feature in the noise image template feature sequence in the plurality of target garbage image blocks, if yes, proceeding to step S508, and if no, proceeding to step S509.

[0111] Step S508, filtering out the garbage collection image.

[0112] Step S509, extracting a first image block semantic feature corresponding to each target garbage image block based on the first image semantic encoding network, and obtaining a first garbage image semantic feature based on the first image block semantic feature.

[0113] Step S510, extracting a second image block semantic feature corresponding to each target garbage image block based on the second image semantic encoding network, and obtaining a second first garbage image semantic feature based on the second image block semantic feature.

[0114] Step S511, taking the first first garbage image semantic feature as the second garbage image semantic feature of the first network branch, and taking the second first garbage image semantic feature as the second garbage image semantic feature of the second network branch.

[0115] Step S512, acquiring dangerous garbage prediction data generated by the dangerous garbage prediction network, and performing dangerous garbage early warning on the garbage collection image based on the dangerous garbage prediction data.

[0116] Figure 2 A cloud server 100 that can be used to implement various embodiments described in the present application is schematically shown.

[0117] For one embodiment, Figure 2A cloud server 100 is shown having one or more processors 102, a control module (chipset) 104 coupled to one or more of the processor(s) 102, a memory 106 coupled to the control module 104, a non-volatile memory (NVM) / storage device 108 coupled to the control module 104, one or more input / output devices 110 coupled to the control module 104, and a network interface 112 coupled to the control module 104.

[0118] The processor(s) 102 can include one or more single core or multicore processors, which can include any combination of general-purpose processors or dedicated processors (e.g., graphics processors, application processors, baseband processors, etc.). In some example design scenarios, the cloud server 100 can be capable of acting as a server device such as a gateway as described in embodiments of the present application.

[0119] In some example design scenarios, the cloud server 100 can include one or more computer-readable media (e.g., the memory 106 or the NVM / storage device 108) having instructions 114 and one or more processors 102 integrated with the one or more computer-readable media configured to execute the instructions 114 to implement modules to perform the actions described in the present disclosure.

[0120] For one embodiment, the control module 104 can include any suitable interface controllers to provide for any suitable interface to one or more of the processor(s) 102 and / or any suitable device or component in communication with the control module 104.

[0121] The control module 104 can include a memory controller module to provide an interface to the memory 106. The memory controller module can be a hardware module, a software module, and / or a firmware module.

[0122] The memory 106 can be used, for example, to load and store data and / or instructions 114 for the cloud server 100. For one embodiment, the memory 106 can include any suitable volatile memory, such as suitable DRAM. In some example design scenarios, the memory 106 can include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0123] For one embodiment, the control module 104 can include one or more input / output controllers to provide an interface to the NVM / storage device 108 and the input / output device(s) 110.

[0124] For example, the NVM / storage device 108 can be used to store data and / or instructions 114. The NVM / storage device 108 can include any suitable non-volatile memory (e.g., flash memory) and / or can include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0125] The NVM / storage device 108 can include storage resources that are physically part of the device on which the cloud server 100 is installed, or it can be accessed by the device remotely and / or can not necessarily be part of the device. For example, the NVM / storage device 108 can be accessed via the input / output device(s) 110 over a network.

[0126] The input / output device(s) 110 can provide an interface for the cloud server 100 to communicate with any other suitable device(s), and the input / output device(s) 110 can include communication components, input components, sensor components, etc. The network interface 112 can provide an interface for the cloud server 100 to communicate over one or more networks, and the cloud server 100 can wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, such as accessing a wireless network according to a communication standard, such as 2G, 3G, 4G, 5G, etc., or combinations thereof.

[0127] For one embodiment, one or more of the processor(s) 102 can be loaded with logic of one or more controllers of the control module 104 (e.g., a memory controller module). For one embodiment, one or more of the processor(s) 102 can be loaded with logic of one or more controllers of the control module 104 to form a system-in-a-package (SwP). For one embodiment, one or more of the processor(s) 102 can be integrated on the same die as logic of one or more controllers of the control module 104. For one embodiment, one or more of the processor(s) 102 can be integrated on the same die as logic of one or more controllers of the control module 104 to form a system-on-a-chip (SoC).

[0128] In various embodiments, cloud server 100 can be, but is not limited to, a cloud server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet, a netbook, etc.), among other end devices. In various embodiments, cloud server 100 can have more or fewer components, and / or different architectures. For example, in some example design considerations, cloud server 100 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including touch screen displays), a non-removable memory port, multiple antennas, a graphics chip, an application specific integrated circuit (ASIC), and a speaker.

[0129] The above detailed description of the embodiments of the present application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the method and application to the precise form disclosed. Many modifications and variations will be apparent to practitioners skilled in this art. The concept of the present application is intended to embrace all such modifications and variations.

Claims

1. An intelligent monitoring method for sanitation waste sorting and disposal points, characterized in that, Applied to cloud servers, the method includes: Obtain waste collection images uploaded by each waste disposal terminal for hazardous waste analysis, and calculate the first waste image semantic features corresponding to the waste collection images based on different image semantic coding networks; Based on the first garbage image semantic features, determine the second garbage image semantic features corresponding to each network branch of the trained hazardous garbage prediction network; The second garbage image semantic features corresponding to each network branch are loaded into the network branch corresponding to the hazardous waste prediction network. The hazardous waste prediction network includes multiple network branches, and each network branch has a corresponding unit hazardous waste prediction network. The hazardous waste prediction network is used to determine hazardous waste prediction data based on the hazardous waste prediction branch parameters generated by each unit hazardous waste prediction network. The hazardous waste prediction branch parameters represent the hazardous waste prediction data of the corresponding network branch. The hazardous waste prediction branch parameters include the heat value corresponding to each hazardous waste label. The unit hazardous waste prediction network and the hazardous waste prediction network are a whole. When the hazardous waste prediction branch parameters generated by each unit hazardous waste prediction network are the same, the hazardous waste prediction data of the network branch corresponding to each unit hazardous waste prediction network is used as the hazardous waste prediction data generated by the hazardous waste prediction network. When the hazardous waste prediction branch parameters generated by each unit hazardous waste prediction network are different, the hazardous waste prediction data of the network branch corresponding to the unit hazardous waste prediction network with the largest heat value is used as the hazardous waste prediction data generated by the hazardous waste prediction network. Obtain hazardous waste prediction data generated by the hazardous waste prediction network, and conduct hazardous waste early warning based on the waste collection images; The unit hazardous waste prediction network includes a feature selection unit based on expected cross-entropy, a feature dimensionality reduction unit, and a fully connected unit. The step of loading the second garbage image semantic features corresponding to each network branch into the network branch corresponding to the hazardous waste prediction network, wherein the hazardous waste prediction network includes multiple network branches, each network branch has a corresponding unit hazardous waste prediction network, and the hazardous waste prediction network is used to determine hazardous waste prediction data based on the hazardous waste prediction branch parameters generated by each unit hazardous waste prediction network includes: The second garbage image semantic feature corresponding to the network branch is used as the network input of the feature selection unit based on expected cross-entropy in the corresponding network branch. The feature selection unit based on expected cross-entropy is used to select the garbage image semantic feature based on expected cross-entropy to obtain a first vector selection array. The first vector selection array is fused and transposed as the network input of the neuron input mapping unit to obtain a second vector selection array. The second vector selection array is used as the network input of the feature dimensionality reduction unit, which is used to perform feature dimensionality reduction processing on each selected vector in the second vector selection array to obtain the corresponding third vector selection array. The third vector selection array is used as the network input of the fully connected unit, which is used to perform a fully connected output based on the third vector selection array to obtain the hazardous waste prediction branch parameters. The hazardous waste prediction branch parameters obtained from each network branch are used as the network input of the aggregation unit in the hazardous waste prediction network. The aggregation unit is used to perform aggregation and weighting based on the hazardous waste prediction branch parameters generated by each network branch to obtain the target hazardous waste prediction data.

2. The intelligent monitoring method for sanitation waste sorting and disposal points according to claim 1, characterized in that, The step of calculating the first semantic feature of the garbage image corresponding to the garbage collection image includes: The collected garbage images are segmented into blocks to generate multiple target garbage image blocks; Extract the semantic features of each target garbage image block, and determine the first garbage image semantic feature corresponding to the garbage collection image based on the semantic features of each target garbage image block.

3. The intelligent monitoring method for sanitation waste sorting and disposal points according to claim 2, characterized in that, Following the step of segmenting the collected garbage image into multiple target garbage image blocks, the method further includes: Obtain a noisy image template feature sequence. If any one of the multiple target garbage image blocks matches the noisy image template feature in the noisy image template feature sequence, then the garbage collection image is filtered out. If any one of the multiple target garbage image blocks does not match the noise image template features in the noise image template feature sequence, then the step of extracting the semantic features of the image block corresponding to each target garbage image block is initiated.

4. The intelligent monitoring method for sanitation waste sorting and disposal points according to claim 2, characterized in that, The hazardous waste prediction network includes a first network branch and a second network branch; The step of extracting the semantic features of each target garbage image block and determining the first garbage image semantic feature corresponding to the garbage collection image based on the semantic features of each target garbage image block includes: Based on the first image semantic coding network, the semantic features of the first image block corresponding to each target garbage image block are extracted, and the semantic features of the first image block are aggregated to obtain the first garbage image semantic features; The second image block semantic features corresponding to each target garbage image block are extracted based on the second image semantic coding network, and the second first garbage image semantic features are obtained by converging the second image block semantic features. The step of determining the second garbage image semantic features corresponding to each network branch of the trained hazardous waste prediction network based on the first garbage image semantic features includes: The first garbage image semantic features are used as the second garbage image semantic features of the first network branch, and the second garbage image semantic features are used as the second garbage image semantic features of the second network branch.

5. The intelligent monitoring method for sanitation waste sorting and disposal points according to claim 4, characterized in that, The step of extracting the first image block semantic features corresponding to each target garbage image block based on the first image semantic coding network, and converging the first image block semantic features to obtain the first garbage image semantic features includes: A block space is established that covers the multiple target waste image blocks, and the first image block semantic features of the target waste image blocks are determined based on the feature repetition frequency of each target waste image block in the waste collection image; Based on the feature weight position of each target garbage image block in the block space and the semantic features of the first image block of each target garbage image block, the first garbage image semantic features corresponding to the garbage collection image are determined; The steps of extracting second image block semantic features corresponding to each target garbage image block based on the second image semantic coding network, and obtaining second garbage image semantic features by converging the second image block semantic features include: Obtain the image block semantic feature library corresponding to the second image semantic coding network, and obtain the second image block semantic features corresponding to each target garbage image block from the image block semantic feature library; Based on the semantic features of the second image block corresponding to each target garbage image block, a second first garbage image semantic feature corresponding to the garbage collection image is constructed.

6. The intelligent monitoring method for sanitation waste sorting and disposal points according to claim 1, characterized in that, Before the step of determining the second garbage image semantic features corresponding to each network branch of the trained hazardous waste prediction network based on the first garbage image semantic features, the method further includes: Acquire sample garbage collection images and calculate the semantic features of the first sample garbage image corresponding to the sample garbage collection images; Based on the semantic features of the first sample garbage image, determine the semantic features of the second sample garbage image corresponding to each network branch of the initial hazardous waste prediction network; Obtain the hazard label data corresponding to the sample waste collection image; The semantic features of the second sample garbage images corresponding to each network branch are loaded into the network branches corresponding to the initial hazardous waste prediction network. The hazardous label data is used as the learning output data of the initial hazardous waste prediction network to update the weight parameters of the initial hazardous waste prediction network, thereby generating a target hazardous waste prediction network. The target hazardous waste prediction network is the trained hazardous waste prediction network, which includes multiple unit hazardous waste prediction networks, and the function weight definition information covered in each unit hazardous waste prediction network is different.

7. The intelligent monitoring method for sanitation waste sorting and disposal points according to claim 4, characterized in that, Before the step of determining the second garbage image semantic features corresponding to each network branch of the trained hazardous waste prediction network based on the first garbage image semantic features, the method further includes: Acquire sample waste collection images, and divide the sample waste collection images into image blocks to generate multiple sample waste image blocks; Based on the first image semantic coding network, the semantic features of the first learned image block corresponding to each sample garbage image block are extracted, and the semantic features of the first sample garbage image are obtained by pooling the semantic features of the first learned image block. The semantic features of the second sample garbage image are extracted based on the second image semantic coding network, and the semantic features of the second sample garbage image are obtained by converging the semantic features of the second learned image block; the hazard label data corresponding to the sample garbage collection image is obtained; The semantic features of the first sample garbage image are used as the network input of the first network branch in the initial hazardous waste prediction network, and the semantic features of the second sample garbage image are used as the network input of the second network branch in the initial hazardous waste prediction network. The hazardous label data is used as the learning output data of the initial hazardous waste prediction network to update the weight parameters of the hazardous waste prediction network, thereby generating a target hazardous waste prediction network. The target hazardous waste prediction network is the trained hazardous waste prediction network, and the target hazardous waste prediction network includes member prediction units corresponding to the first network branch and member prediction units corresponding to the second network branch.

8. The intelligent monitoring method for sanitation waste sorting and disposal points according to claim 1, characterized in that, The step of using the hazardous waste prediction branch parameters obtained from each network branch as the network input of the aggregation unit in the hazardous waste prediction network, and the aggregation unit performing aggregation and weighting based on the hazardous waste prediction branch parameters generated from each network branch to obtain target hazardous waste prediction data, includes: Hazardous waste prediction branch parameters are calculated based on each network branch. The hazardous waste prediction branch parameters include the thermal value corresponding to each hazardous waste label. Branch hazardous waste prediction data corresponding to the network branch are then determined. If the branch hazardous waste prediction data of the multiple network branches are the same, then the branch hazardous waste prediction data shall be used as the target hazardous waste prediction data. If the hazardous waste prediction data of the multiple network branches are different, the hazardous waste prediction data of the branch with the highest heat value shall be used as the target hazardous waste prediction data.

9. An intelligent monitoring system for sanitation waste sorting and disposal points, characterized in that, The intelligent monitoring system for the sanitation waste sorting and disposal point includes a cloud server and a waste disposal terminal that is communicatively connected to the cloud server. The cloud server is specifically used for: Obtain waste collection images uploaded by each waste disposal terminal for hazardous waste analysis, and calculate the first waste image semantic features corresponding to the waste collection images based on different image semantic coding networks; Based on the first garbage image semantic features, determine the second garbage image semantic features corresponding to each network branch of the trained hazardous garbage prediction network; The second garbage image semantic features corresponding to each network branch are loaded into the network branch corresponding to the hazardous waste prediction network. The hazardous waste prediction network includes multiple network branches, and each network branch has a corresponding unit hazardous waste prediction network. The hazardous waste prediction network is used to determine hazardous waste prediction data based on the hazardous waste prediction branch parameters generated by each unit hazardous waste prediction network. The hazardous waste prediction branch parameters represent the hazardous waste prediction data of the corresponding network branch. The hazardous waste prediction branch parameters include the heat value corresponding to each hazardous waste label. The unit hazardous waste prediction network and the hazardous waste prediction network are a whole. When the hazardous waste prediction branch parameters generated by each unit hazardous waste prediction network are the same, the hazardous waste prediction data of the network branch corresponding to each unit hazardous waste prediction network is used as the hazardous waste prediction data generated by the hazardous waste prediction network. When the hazardous waste prediction branch parameters generated by each unit hazardous waste prediction network are different, the hazardous waste prediction data of the network branch corresponding to the unit hazardous waste prediction network with the largest heat value is used as the hazardous waste prediction data generated by the hazardous waste prediction network. Obtain hazardous waste prediction data generated by the hazardous waste prediction network, and conduct hazardous waste early warning based on the waste collection images; The unit hazardous waste prediction network includes a feature selection unit based on expected cross-entropy, a feature dimensionality reduction unit, and a fully connected unit. The step involves loading the second garbage image semantic features corresponding to each network branch into the network branch corresponding to the hazardous waste prediction network. The hazardous waste prediction network includes multiple network branches, each with a corresponding unit hazardous waste prediction network. The hazardous waste prediction network is used to determine hazardous waste prediction data based on the hazardous waste prediction branch parameters generated by each unit hazardous waste prediction network, including: The second garbage image semantic feature corresponding to the network branch is used as the network input of the feature selection unit based on expected cross-entropy in the corresponding network branch. The feature selection unit based on expected cross-entropy is used to select the garbage image semantic feature based on expected cross-entropy to obtain a first vector selection array. The first vector selection array is fused and transposed as the network input of the neuron input mapping unit to obtain a second vector selection array. The second vector selection array is used as the network input of the feature dimensionality reduction unit, which is used to perform feature dimensionality reduction processing on each selected vector in the second vector selection array to obtain the corresponding third vector selection array. The third vector selection array is used as the network input of the fully connected unit, which is used to perform a fully connected output based on the third vector selection array to obtain the hazardous waste prediction branch parameters. The hazardous waste prediction branch parameters obtained from each network branch are used as the network input of the aggregation unit in the hazardous waste prediction network. The aggregation unit is used to perform aggregation and weighting based on the hazardous waste prediction branch parameters generated by each network branch to obtain the target hazardous waste prediction data.

Images