Detection method for abnormal conditions in waste incineration power plant pits, edge deployment method for detection method, and equipment

Abstract

The present application discloses a method for detecting abnormal conditions in a waste incineration power plant pit, an edge deployment method for the detection method, and equipment. The abnormal condition detection method includes: receiving an image to be detected sent by a camera, wherein the image to be detected is obtained by the camera photographing the waste incineration power plant pit; determining the required operator type during the processing of the image to be detected, and calling the corresponding operator for processing to obtain a processing result; detecting abnormal conditions in the image to be detected based on the processing result and the upper layer of a preset convolutional neural network. The present application classifies the operator types, and then determines the operator type for processing the image during the image processing process, and performs corresponding image processing by calling the operator. This can make the neural network algorithm lightweight and lay the foundation for the edge deployment of the algorithm.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for detecting abnormal conditions in a waste incineration power plant pit, an edge deployment method for the detection method, and equipment. Background Art

[0002] With global economic development and urbanization, reports indicate that municipal waste is experiencing explosive growth. Under the current circumstances, waste-to-energy incineration has become the optimal method for reducing, harmlessly treating, and recycling municipal waste. Compared to landfilling, incineration not only avoids groundwater contamination from leachate but also mitigates the potential safety hazards posed by methane and other gases produced by fermentation. Furthermore, compared to thermal power generation, incineration-based power generation utilizes municipal waste instead of fossil fuels, reducing greenhouse gas emissions.

[0003] With the expansion of waste-to-energy plants, problems in industrial production have also emerged. For example, abnormalities such as the mixing of large pieces of garbage and fires at waste dumps can severely impact the efficiency and safe operation of power plants. Currently, foreign matter in waste-to-energy plants is often identified manually. However, due to the complex environment of waste dumps, manual identification cannot effectively distinguish foreign matter from garbage, posing a serious safety hazard and resulting in low efficiency. Summary of the Invention

[0004] The main purpose of this application is to provide a method and equipment for detecting abnormal conditions in the waste incineration power plant pit, and to solve the technical problem of low efficiency of existing manual detection methods.

[0005] In a first aspect, to achieve the above-mentioned objectives, the present application provides a method for detecting abnormal conditions in a waste incineration power plant pit, the method comprising:

[0006] Receive the image to be detected sent by the camera;

[0007] Determining the required operator type during the processing of the image to be detected, and calling the corresponding operator to perform processing to obtain a processing result;

[0008] Anomalies in the image to be detected are detected based on the processing results and the upper layer of the preset convolutional neural network.

[0009] Optionally, determining the required operator type during processing of the image to be detected, calling the corresponding operator for processing, and obtaining the processing result includes:

[0010] Determine the convolution layer in the convolution operator that contributes the least to the detection result, which is defined as the weakened convolution layer;

[0011] Obtaining an average value of the data in the weakened convolution layer, and using the average value as a processing result of the weakened convolution layer;

[0012] The detecting of abnormalities in the image to be detected based on the processing result and the upper layer of the preset convolutional neural network includes:

[0013] Restoring the average value through the upper layer of the preset convolutional neural network to obtain restored data;

[0014] Anomalies in the image to be detected are detected based on the restored data.

[0015] Optionally, obtaining an average value of the data in the weakened convolution layer and using the average value as a processing result of the weakened convolution layer includes:

[0016] Determine the L1 norm of the convolution kernels in other convolution layers, where other convolution kernels are convolution layers other than the weakened convolution layer;

[0017] Sort the L1 norm values ​​of the convolution kernels in other convolutional layers;

[0018] Determine the average value of the channels corresponding to the sorted convolution kernels.

[0019] Optionally, the step of receiving the image to be detected sent by the camera includes:

[0020] Receive images to be detected sent by multiple cameras through a double buffer mechanism;

[0021] The step of determining the required operator type during the processing of the image to be detected, calling the corresponding operator for processing, and obtaining the processing result includes:

[0022] The images to be detected in the double buffer areas are processed alternately to obtain the processing results.

[0023] Optionally, the step of determining the required operator type during processing of the image to be detected, calling the corresponding operator for processing, and obtaining the processing result includes:

[0024] The image to be detected is subjected to convolution and normalization processing simultaneously to obtain the processing result.

[0025] Optionally, the operators include convolution operators, activation operators, pooling operators, upsampling operators and splicing operators of multiple scales.

[0026] Optionally, the weights and / or activation functions in the detection method are of fixed-point type.

[0027] In a second aspect, the present application further provides an edge deployment method for detecting abnormal conditions in a waste incineration power plant pit, the edge deployment method comprising:

[0028] Extracting operators involved in a preset convolutional neural network algorithm and deploying the operators to a programmable logic end;

[0029] The upper-layer code script of the preset convolutional neural network algorithm is extracted and burned into the processing system end, and the upper-layer code script is used to call the operator in the programmable logic end to perform corresponding processing on the image to be detected.

[0030] Optionally, the edge deployment method further includes:

[0031] Two data buffer areas are set up, and the two data buffer areas alternately receive the image to be detected sent by the camera. When one of the data buffer areas receives the image to be detected sent by the camera, the image to be detected in the other data buffer area is processed by the upper-level code script by calling the corresponding operator.

[0032] Optionally, before the step of extracting operators involved in the preset convolutional neural network algorithm and deploying the operators to the programmable logic end, the step includes:

[0033] An initial convolutional neural network algorithm is trained according to a preset image set to obtain the preset convolutional neural network algorithm.

[0034] Optionally, before the step of training an initial convolutional neural network algorithm according to a preset image set to obtain the preset convolutional neural network algorithm, the method further includes:

[0035] Deleting the i-th convolutional layer in the preset convolutional neural network algorithm each time to obtain a pruned convolutional neural network algorithm;

[0036] Performing recognition using the pruned convolutional neural network algorithm to obtain a recognition accuracy rate of the pruned convolutional neural network algorithm;

[0037] The recognition contribution of the i-th convolutional layer is determined according to the recognition accuracy of the pruned convolutional neural network algorithm.

[0038] In a third aspect, the present application also provides a computer device, comprising a programmable logic end and a processing system end, wherein the programmable logic end comprises: a convolution kernel composed of multiple operators, the convolution kernel is communicatively connected to the processing system end, and the processing system end is burned with an upper-level code script of a preset convolutional neural network algorithm, and the upper-level code script is used to call the operators in the programmable logic end to perform corresponding processing on the image to be detected.

[0039] In the technical solution of the present application, an image to be detected is received from a camera, and the image to be detected is obtained by photographing the waste incineration power plant pit by the camera; the required operator type is determined during the processing of the image to be detected, and the corresponding operator is called for processing to obtain a processing result; abnormal conditions in the image to be detected are detected based on the processing result and the upper layer of the preset convolutional neural network. This solution classifies the operator types, and then determines the operator type for processing the image during the image processing process, and performs corresponding image processing by calling the operator. This can make the neural network algorithm lightweight, lay the foundation for the edge deployment of the algorithm, and meet the needs of some special scenarios; at the same time, there is no need to design and adjust each similar operator separately, which greatly reduces the complexity and workload of model design, and reduces the design cost and error probability. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a schematic diagram of the hardware structure of the terminal involved in the embodiment of the present application;

[0041] Figure 2 This is a flow chart of a first embodiment of a method for detecting abnormal conditions in a waste incineration power plant pit according to the present application;

[0042] Figure 3 Schematic diagram of the branch reduction operation in the convolutional layer in the prior art of this application;

[0043] Figure 4 This is a schematic diagram of the operations in the convolutional layer of this application;

[0044] Figure 5 A flow chart of the edge deployment method for detecting abnormal conditions in a waste incineration power plant pit in this application;

[0045] Figure 6 A schematic diagram of the internal circuit of a computer deployed at the edge for this application;

[0046] Figure 7 This is a schematic diagram of the power consumption results after deployment in an embodiment of the present application.

[0047] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0049] The method for detecting abnormal conditions in a waste incineration power plant pit involved in the embodiment of the present application is mainly applied to a terminal, which can be a PC, a portable computer, a mobile terminal, or other device with processing capabilities.

[0050] Reference Figure 1 , Figure 1 This is a schematic diagram of the terminal structure involved in the embodiments of the present application. The terminal in this embodiment of the present application is a computer device, typically located in special scenarios such as waste incineration power plants. These scenarios typically require lightweight or fast processing. In this embodiment of the present application, the computer device may include a processor 1001 (e.g., a CPU), a communication bus 1002, a programmable logic terminal (module) 1003, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The memory 1005 may be a high-speed RAM memory or a non-volatile memory, such as a disk drive. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001 and the programmable logic terminal (module) 1003. It is understood that the terminal in this embodiment of the present application may also include a camera, which is used to capture images within the application scenario and transmit the captured images to the processing processor 1001 via a wired connection.

[0051] Those skilled in the art will understand that Figure 1 The hardware structure shown in the figure does not constitute a limitation of the device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0052] Continue to refer to Figure 1 , Figure 1 The memory 1005 as a computer-readable storage medium may include an operating system, a programmable logic terminal (module), and an abnormality detection program.

[0053] exist Figure 1 In the example, the programmable logic end (module) is deployed with operators required by the abnormality detection program, and the processor 1001 can call the abnormality detection program for the waste incineration power plant pit stored in the memory 1005 and perform the following operations:

[0054] receiving an image to be detected sent by a camera, where the image to be detected is obtained by photographing a waste incineration power plant pit by the camera;

[0055] Determining the required operator type during the processing of the image to be detected, and calling the corresponding operator to perform processing to obtain a processing result;

[0056] Anomalies in the image to be detected are detected based on the processing results and the upper layer of the preset convolutional neural network.

[0057] Furthermore, the processor 1001 may call the abnormality detection program stored in the memory 1005 and perform the following operations:

[0058] The method of determining the required operator type during the processing of the image to be detected, and calling the corresponding operator for processing to obtain the processing result includes:

[0059] Determine the convolution layer in the convolution operator that contributes the least to the detection result, which is defined as the weakened convolution layer;

[0060] Obtaining an average value of the data in the weakened convolution layer, and using the average value as a processing result of the weakened convolution layer;

[0061] The detecting of abnormalities in the image to be detected based on the processing result and the upper layer of the preset convolutional neural network includes:

[0062] Restoring the average value through the upper layer of the preset convolutional neural network to obtain restored data;

[0063] Anomalies in the image to be detected are detected based on the restored data.

[0064] Furthermore, the processor 1001 may call the abnormality detection program stored in the memory 1005 and perform the following operations:

[0065] The obtaining an average value of the data in the weakened convolution layer and using the average value as a processing result of the weakened convolution layer includes:

[0066] Determine the L1 norm of the convolution kernels in other convolution layers, where other convolution kernels are convolution layers other than the weakened convolution layer;

[0067] Sort the L1 norm values ​​of the convolution kernels in other convolutional layers;

[0068] Determine the average value of the channels corresponding to the sorted convolution kernels.

[0069] Furthermore, the processor 1001 may call the abnormality detection program stored in the memory 1005 and perform the following operations:

[0070] Receive images to be detected sent by multiple cameras through a double buffer mechanism;

[0071] The step of determining the required operator type during the processing of the image to be detected, calling the corresponding operator for processing, and obtaining the processing result includes:

[0072] The images to be detected in the double buffer areas are processed alternately to obtain the processing results.

[0073] Furthermore, the processor 1001 may call a detection program for abnormal conditions in a waste incineration power plant pit stored in the memory 1005 and perform the following operations:

[0074] The image to be detected is subjected to convolution and normalization processing simultaneously to obtain the processing result.

[0075] Furthermore, the operators include convolution operators, activation operators, pooling operators, upsampling operators and splicing operators of multiple scales.

[0076] Furthermore, the weights and / or activation functions in the detection method are of fixed-point type.

[0077] Based on the hardware structure of the above-mentioned terminal, various embodiments of the method for detecting abnormal conditions in the waste incineration power plant pit of the present application are proposed.

[0078] This application provides a method for detecting abnormal conditions in a waste incineration power plant pit. The neural network algorithm used in this application is described using a YOLO network as an example. Other convolutional neural networks can be used in specific embodiments. The application scenario is a garbage station, but other scenarios can be used in specific implementations, which are not limited here.

[0079] See also Figure 2 In the first embodiment of the present application, a method for detecting abnormal conditions in a waste incineration power plant pit includes the following steps:

[0080] S100, receiving an image to be detected sent by a camera;

[0081] Specifically, in one embodiment, the detection image is an image captured in real time by a data acquisition device. To meet the real-time data requirements in scenarios such as garbage stations, the camera transmits the data to a processing terminal (computer) at the edge of the garbage station via a wired connection. During the abnormality detection process, the image of the garbage station captured in real time by the camera serves as the image to be detected. The camera can also be other data acquisition devices, such as scanners, cameras, and other devices with imaging capabilities.

[0082] It is understandable that before conducting anomaly detection, a survey was conducted on the conditions of waste incineration power plants, and it was found that the types of waste that affect production mainly include wood, mattresses, gas cylinders, and iron sheets. For the above four types of waste, a data set was prepared based on the conditions of the waste incineration power plant. The data set was then used to train the detection algorithm, and the trained convolutional neural network algorithm was used as the preset neural network algorithm. Furthermore, if the data in the data set is insufficient, it can be expanded based on the existing data set. For example, the original image collected in the first step can be expanded by using methods such as mirroring, rotation, scale transformation, random extraction, and color dithering. The method of expanding the data set can refer to the existing technology and will not be repeated here. Furthermore, by adjusting indicators such as brightness, contrast, saturation, and hue, the sensitivity of the neural network algorithm to color is reduced.

[0083] For example, the YOLOv4tiny network algorithm consists of a backbone network, a core network, and a detection head. Compared to standard versions like YOLOv4, YOLOv4-tiny reduces the number of network layers and parameters, thereby reducing computational complexity and memory usage. This enables it to run quickly on resource-constrained devices, such as embedded and mobile devices, while maintaining high accuracy.

[0084] In this application, we first count the types of operators involved in the traditional YOLOv4tiny network algorithm. There are 7 types of operators involved in YOLOv4tiny, including 3 configurations of Conv2D (convolution kernel operator), LeakyReLU (activation function operator), Concat (splicing operator), Maxpool2d (pooling operator), and Upsample (upsampling operator). The types of operators can vary depending on the actual neural network algorithm used. As an embodiment, all 7 operators can be deployed to the programmable logic end. As another embodiment, Concat can be performed using pointers on the processing system end, and only the other 6 operators are deployed to the programmable logic end. Then, the code script (defined as the upper layer) other than the operators in the traditional YOLOv4tiny network algorithm is deployed to the processing system end. After the deployment is completed, the image of the garbage station captured in real time by the data acquisition device is received as the image to be detected.

[0085] S200, determining the required operator type during the processing of the image to be detected, and calling the corresponding operator to perform processing to obtain a processing result;

[0086] Specifically, the backbone network extracts basic feature information from the input image, and the neck network further fuses and processes the features extracted by the backbone network to generate a multi-scale feature representation and detect the target through the detection head based on the feature map output by the neck network. In these processing processes, before each operator is needed, the required operator type is determined, and the operator corresponding to the required operator type is called for processing to obtain the corresponding processing result. For example: in the process of extracting basic feature information from the input image through the backbone network, convolution operators, normalization operators and pooling operators are needed. Specifically, the first convolution operator is used to extract the features of the image, and then the normalization operator is used to make the data distribution more stable, accelerate the convergence speed, and improve the stability; finally, the pooling operator is used to reduce the size of the normalized feature data to reduce the amount of calculation. The features extracted by the backbone network are further integrated and processed through the neck network to generate multi-scale feature representations. Convolution operators and upsampling operators are required. Specifically, the feature map output by the backbone network can be further extracted and adjusted through the second convolution operator. Through different convolution kernel parameters, the features are further refined to provide more representative features for the subsequent detection head. The feature maps of different scales or branches are then spliced ​​in the channel dimension through the splicing operator to integrate multi-scale feature information. Finally, the feature map output by the splicing operator is amplified through the upsampling operator to match its size with the high-resolution feature map for splicing. In the process of detecting targets based on the feature map output by the neck network through the detection head, convolution operators and activation function operators are required. First, the feature map output by the neck network is finally extracted and processed through the third convolution operator to generate features for target detection and classification. Then, the output result is mapped to the [0,1] interval through the activation function operator. It should be noted that this application divides operators into seven types. When an operator is needed, it calls the operator in the programmable logic module for processing, which greatly reduces the number of operators. At the same time, there is no need to design and adjust each operator of the same type separately, which greatly reduces the complexity and workload of model design, reduces design costs and reduces the probability of error.

[0087] It is understandable that after obtaining the image to be detected in step S100, the image to be detected may be preprocessed through the input layer, such as data denoising and standardization.

[0088] S300: Detect abnormalities in the image to be detected based on the processing result and the upper layer of the preset convolutional neural network.

[0089] Specifically, according to the processing results and the upper layer of the preset convolutional neural network, it is determined whether the image to be detected belongs to an abnormal situation and the category of the abnormal situation to which the abnormal situation belongs.

[0090] In the technical solution of the present application, an image to be detected is received from a camera; the required operator type is determined during the processing of the image to be detected, and the corresponding operator is called for processing to obtain a processing result; abnormal conditions in the image to be detected are detected based on the processing result and the upper layer of the preset convolutional neural network. This solution classifies the operator types, and then determines the operator type for processing the image during the image processing process, and performs corresponding image processing by calling the operator. This can make the neural network algorithm lightweight, lay the foundation for the edge deployment of the algorithm, and meet the needs of some special scenarios; at the same time, there is no need to design and adjust each similar operator separately, which greatly reduces the complexity and workload of model design, and reduces the design cost and error probability.

[0091] Furthermore, step S200: determining the required operator type during the processing of the image to be detected, calling the corresponding operator for processing, and obtaining the processing result includes:

[0092] Step S210, determining the convolution layer in the convolution operator that has the smallest contribution to the detection result, and defining it as the weakened convolution layer;

[0093] Step S220, obtaining an average value of the data in the weakened convolution layer, and using the average value as a processing result of the weakened convolution layer;

[0094] Specifically, since the neural network algorithm has multiple layers of convolution, and each layer of convolution has multiple convolution kernels, in order to further improve the computing speed to meet the rapid recognition requirements required by application scenarios such as waste incineration power plants, the present application further obtains the trained preset convolutional neural network algorithm during training, and then deletes only the i-th convolution layer in the preset convolutional neural network algorithm each time, that is, the preset convolutional neural network algorithm is pruned (one convolution layer is pruned). For the convenience of description, it is defined as a pruned convolutional neural network algorithm, and then the pruned convolutional neural network algorithm is used for recognition. Finally, the recognition accuracy of the pruned convolutional neural network algorithm is determined. This is repeated N times, and the recognition accuracy of each pruned convolutional neural network algorithm is defined as ACC1, ACC2...ACC N , using the accuracy of the complete trained preset convolutional neural network algorithm, the contribution of each cropped convolutional neural network algorithm can be obtained. For example, the contribution W i =(ACC / ACC i ) 2 , where ACC is the accuracy of the fully trained preset convolutional neural network algorithm.

[0095] After obtaining the contribution of each convolutional layer, the convolutional layer with the smallest contribution to the detection result is determined and defined as the weakened convolutional layer. The average value of the data in the weakened convolutional layer is obtained and used as the processing result of the weakened convolutional layer.

[0096] Correspondingly, step S300: detecting abnormalities in the image to be detected based on the processing result and the upper layer of the preset convolutional neural network, includes:

[0097] Step S310, restoring the average value through the upper layer of the preset convolutional neural network to obtain restored data;

[0098] Step S320: detecting abnormalities in the image to be detected based on the restored data.

[0099] Specifically, after obtaining the average value of the data in the weakened convolutional layer, the upper layer of the preset convolutional neural network (specifically, before deconvolution or feature extraction) performs restoration based on this average value to obtain restored data. Anomalies in the image to be detected are then detected based on this restored data. In this way, during the convolution process, only the average value of the convolutional layer with the lowest shared accuracy is calculated before restoration. This not only does not affect the convolution layers with higher recognition accuracy, but also performs data restoration before recognition, ensuring accurate recognition while reducing the computational complexity (the computational complexity of calculating the average value is less than that of multiple convolution kernel deconvolutions).

[0100] Furthermore, step S220, obtaining an average value of the data in the weakened convolution layer and using the average value as a processing result of the weakened convolution layer, includes:

[0101] Step S221, determining the L1 norm value of the convolution kernel in other convolution layers, where the other convolution kernels are convolution layers other than the weakened convolution layer;

[0102] Step S222, sorting the L1 norm values ​​of the convolution kernels in other convolutional layers;

[0103] Step S223: Determine the average value of the channels corresponding to the sorted convolution kernels.

[0104] Specifically, independently calculate the L1 norm of each convolution kernel in all other convolutional layers (unpruned convolutional layers, that is, convolutional layers excluding the weakened convolutional layers) and sort the L1 norms of the convolution kernels in each convolutional layer from largest to smallest. Determine the m channels corresponding to the m convolution kernels and calculate the average value of the channel values.

[0105] In order to better understand this application, this application is compared with the existing direction (structured branch reduction):

[0106] At present, structured branch reduction is achieved by cutting off a channel while the calculation of other channels is still in progress (multiple channels are operated in parallel, and the cut channel is not calculated, but is just idle waiting), which makes it impossible to reduce the convolution operation time and causes resource waste. Moreover, since the data of the channel is deleted, the accuracy will be affected. Figure 3 As shown, the white data is deleted. This application is equivalent to retaining the mean of the pruned channel on the basis of structured pruning, such as Figure 4 As shown, this application is equivalent to filling the average value with Figure 3 The white portion in the middle shows a single convolution operation, which results in fewer channel parameters after structured pruning. However, compared to the parameters before pruning, the advantage of reduced storage is still retained (the convolution kernel parameters of one channel are deleted). This application retains the average value of the channel data calculated after structured pruning in the prior art and then restores it to ensure accuracy.

[0107] Furthermore, step S100, the step of receiving the image to be detected sent by the camera, includes:

[0108] Receive images to be detected sent by multiple cameras through a double buffer mechanism;

[0109] Step S200, determining the required operator type during the processing of the image to be detected, calling the corresponding operator for processing, and obtaining the processing result, includes:

[0110] The images to be detected in the double buffer areas are processed alternately to obtain the processing results.

[0111] Specifically, this application also implements a double-buffering mechanism, using a "ping-pong" strategy for interface data, allowing loading and calculation to proceed simultaneously. If there are multiple cameras, and a double-buffering mechanism is implemented at the input port, while one buffer is calculating, the other can be used to store data. The two buffers alternately receive data transmitted by different cameras, achieving a doubling of throughput.

[0112] Furthermore, step S200: determining the required operator type during the processing of the image to be detected, and calling the corresponding operator for processing to obtain the processing result, including:

[0113] The image to be detected is subjected to convolution and normalization processing simultaneously to obtain the processing result.

[0114] Specifically, in order to significantly reduce data exchange and computational complexity, this embodiment also fuses the convolutional layer and the normalization layer when they both exist in the neural network algorithm. After fusion, the computational complexity of the two layers becomes that of one layer, reducing data exchange and computational complexity and further improving the lightweightness of the algorithm.

[0115] Furthermore, the parameters in the detection method described in this application are fixed-point numbers. Specifically, the weights or activations involved in each operator and upper-level code are represented by fixed-point numbers. This can further improve the calculation speed and ensure the real-time performance of signal processing.

[0116] In addition, please refer to Figure 5 The present application also provides an edge deployment method for detecting abnormal conditions in a waste incineration power plant pit, the edge deployment method comprising:

[0117] Step S1, extracting operators involved in a preset convolutional neural network algorithm and deploying the operators to a programmable logic terminal;

[0118] Step S2: extract the upper-layer code script of the preset convolutional neural network algorithm and burn it to the processing system end. The upper-layer code script is used to call the operator in the programmable logic end to perform corresponding processing.

[0119] Specifically, due to electromagnetic shielding during normal operation of a waste incineration power plant, it is impossible to upload captured images wirelessly. Furthermore, to prevent gas cylinders, iron sheets, and other waste from damaging power plant equipment and affecting the efficiency and effectiveness of waste combustion, and to quickly detect and promptly address anomalies, edge deployment is required near the waste incineration power plant. This application first counts the operator types involved in the traditional YOLOv4tiny network algorithm. There are seven types of operators involved in YOLOv4tiny, including three configurations of Conv2D (convolution kernel operator), LeakyReLU (activation function operator), Concat (concatenation operator), Maxpool2d (pooling operator), and Upsample (upsampling operator). The operator types can vary depending on the actual neural network algorithm used. As one embodiment, all seven operators can be deployed on the programmable logic side. As another embodiment, Concat can be performed using pointers on the processing system side, with only the other six operators deployed on the programmable logic side. Then, the code script (defined as the upper layer) of the traditional YOLOv4tiny network algorithm except the operators is deployed to the processing system end to achieve edge deployment.

[0120] As an example, the entire edge deployment of computer equipment, such as Figure 6As shown, the computer device includes a clock module, a programmable logic end, a communication module and a processing system end. Various operators of the neural network algorithm are deployed in the programmable logic end. There are multiple communication modules. The communication module and the clock module are the same as those in the prior art. This application mainly adds a programmable logic end and deploys the upper-level code script on the processing system end. The other modules are the same as the prior art. The communication module is modified as needed to determine the connection relationship with other modules.

[0121] The edge deployment method further includes:

[0122] Two data buffer areas are set up, and the two data buffer areas alternately receive the image to be detected sent by the camera. When one of the two data buffer areas receives the image to be detected sent by the camera, the image to be detected in the other data buffer area is processed by the upper-level code script calling the corresponding operator.

[0123] Specifically, this application also implements a double-buffering mechanism, using a "ping-pong" strategy for interface data, allowing loading and computation to proceed simultaneously. When a double-buffering mechanism is implemented on the input port, while one buffer is computing, the other can be used to store data, alternating between the two buffers to achieve a doubling of throughput.

[0124] Furthermore, it is understood that before the step of extracting the operators involved in the preset convolutional neural network algorithm and deploying the operators to the programmable logic end, the following steps are included:

[0125] An initial convolutional neural network algorithm is trained according to a preset image set to obtain the preset convolutional neural network algorithm.

[0126] Specifically, prior to deployment, we conducted research on waste-to-energy plants and found that the four main types of waste affecting production were wood, mattresses, gas cylinders, and sheet metal. We created a dataset for these four types of waste, drawing on the data from waste-to-energy plants. This dataset was then used to train the detection algorithm, and the trained convolutional neural network became the default neural network algorithm.

[0127] Furthermore, before the step of training the initial convolutional neural network algorithm according to the preset image set to obtain the preset convolutional neural network algorithm, the method further includes:

[0128] The convolution layer and the normalization layer in the initial convolutional neural network algorithm are fused.

[0129] Specifically, to significantly reduce data exchange and computational complexity, this embodiment also fuses the convolutional layer and the normalization layer when both exist in the neural network algorithm. After fusion, the computational complexity of the two layers is reduced to one, reducing data exchange and computational complexity, and further improving the lightweight nature of the algorithm (that is, the convolution operator has both convolution and normalization functions). Furthermore, the parameters in the detection method described in this application are fixed-point numbers.

[0130] Furthermore, before the step of training an initial convolutional neural network algorithm according to a preset image set and obtaining the preset convolutional neural network algorithm, the method further includes:

[0131] Deleting the i-th convolutional layer in the preset convolutional neural network algorithm each time to obtain a pruned convolutional neural network algorithm;

[0132] Performing recognition using the pruned convolutional neural network algorithm to obtain a recognition accuracy rate of the pruned convolutional neural network algorithm;

[0133] The recognition contribution of the i-th convolutional layer is determined according to the recognition accuracy of the pruned convolutional neural network algorithm.

[0134] Specifically, during the training process, after obtaining the trained preset convolutional neural network algorithm, only the i-th convolution layer in the preset convolutional neural network algorithm is deleted each time, that is, the preset convolutional neural network algorithm is pruned (one convolution layer is pruned). For the convenience of description, it is defined as a pruned convolutional neural network algorithm. Then, the pruned convolutional neural network algorithm is used for recognition. Finally, the recognition accuracy of the pruned convolutional neural network algorithm is determined. This is repeated N times, and the recognition accuracy of each pruned convolutional neural network algorithm is defined as ACC1, ACC2...ACC N , using the accuracy of the complete trained preset convolutional neural network algorithm, the contribution of each cropped convolutional neural network algorithm can be obtained. For example, the contribution W i =(ACC / ACC i ) 2 , where ACC is the accuracy of the fully trained preset convolutional neural network algorithm. This allows for optimization of the computational speed during subsequent recognition by utilizing the contribution of each convolution kernel layer to the accuracy. For details, please refer to step S200 and the detailed steps in the above embodiment, which will not be detailed here.

[0135] Testing using the technical solution in this application has achieved both abnormal garbage identification and low-latency, low-power edge deployment. This reduces the number of scheduling and switching between operators. The process that originally required multiple scheduling of different operators can now complete the calculation of multiple operators at once, reducing additional time overhead and lowering the latency of the entire computational process. For example, in image classification tasks, integrating the three operators of convolution, batch normalization, and activation functions allows for faster final classification results, with an inference latency of approximately 427ms, achieving 2.43 FPS, and a throughput of 3.43 GOP / 0.428s ≈ 8.01 GOPS. When not accelerated, based on the current CPU time of 496s, the acceleration ratio is 496*1000 ms / 427 ms = 1161, achieving good acceleration. The on-chip power consumption is only 1.883W, much lower than the PC end. The terminal temperature is 28.5℃, the thermal resistance is 7.1℃ / W (6.5W), the effective junction temperature is 1.9℃ / W, the power consumption of the external device is 0W, and the power margin level is medium. The specific power consumption is as follows: Figure 7 As shown, it meets the requirements of lightweight deployment.

[0136] In this application, operator types are classified, some operators are deployed on the programmable logic side, and others are deployed on the processing system side. Then, the operator type for image processing is determined during image processing, and the corresponding image processing is performed by calling the operator. This makes the neural network algorithm lightweight, lays the foundation for edge deployment of the algorithm, and can meet the needs of some special scenarios. At the same time, there is no need to design and adjust each operator of the same type separately, which greatly reduces the complexity and workload of model design, and reduces design costs and error probability.

[0137] In addition, the present application also provides a computer-readable storage medium.

[0138] The computer-readable storage medium of the present application stores a program for detecting abnormal conditions of a waste incineration power plant pit. When the program for detecting abnormal conditions of a waste incineration power plant pit is executed by a processor, the steps of the method for detecting abnormal conditions of a waste incineration power plant pit as described above are implemented.

[0139] Among them, the method implemented when the detection program for abnormal conditions of the waste incineration power plant pit is executed can refer to the various embodiments of the detection method for abnormal conditions of the waste incineration power plant pit of the present application, and will not be repeated here.

[0140] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0142] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0143] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0144] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present application may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0145] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0146] The above description is only a preferred embodiment of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application description and drawings under the inventive concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for detecting abnormal conditions in a waste incineration power plant pit, characterized in that: The method comprises: receiving an image to be detected sent by a camera, where the image to be detected is obtained by photographing a waste incineration power plant pit by the camera; Determining the required operator type during the processing of the image to be detected, and calling the operator corresponding to the operator type for processing to obtain a processing result, wherein the operator is deployed on the programmable logic end; Detecting abnormalities in the image to be detected based on the processing results and an upper layer of a preset convolutional neural network, the upper layer of the preset convolutional neural network being burned into a processing system end; The step of determining the required operator type during processing of the image to be detected, and calling an operator corresponding to the operator type for processing to obtain a processing result includes: Determine the convolution layer in the convolution operator that contributes the least to the detection result, which is defined as the weakened convolution layer; Obtaining an average value of the data in the weakened convolution layer, and using the average value as a processing result of the weakened convolution layer; The detecting of abnormalities in the image to be detected based on the processing result and the upper layer of the preset convolutional neural network includes: Restoring the average value through the upper layer of the preset convolutional neural network to obtain restored data; Anomalies in the image to be detected are detected based on the restored data.

2. The abnormal situation detection method according to claim 1, characterized in that: The obtaining an average value of the data in the weakened convolution layer and using the average value as a processing result of the weakened convolution layer includes: Determine the L1 norm of the convolution kernels in other convolution layers, where other convolution kernels are convolution layers other than the weakened convolution layer; Sort the L1 norm values ​​of the convolution kernels in other convolutional layers; Determine the average value of the channels corresponding to the sorted convolution kernels.

3. The abnormal situation detection method according to claim 1, characterized in that: The step of receiving the image to be detected sent by the camera includes: Receive images to be detected sent by multiple cameras through a double buffer mechanism; The step of determining the required operator type during the processing of the image to be detected, calling the operator corresponding to the operator type for processing, and obtaining the processing result includes: The images to be detected in the double buffer areas are processed alternately to obtain the processing results.

4. The abnormal situation detection method according to claim 1, characterized in that: The step of determining the required operator type during the processing of the image to be detected, calling the operator corresponding to the operator type for processing, and obtaining the processing result includes: The image to be detected is subjected to convolution and normalization processing simultaneously to obtain the processing result.

5. An edge deployment method for detecting abnormal conditions in a waste incineration power plant pit, characterized in that: The edge deployment method includes: Extracting operators involved in a preset convolutional neural network algorithm and deploying the operators to the programmable logic end; The upper-layer code script of the preset convolutional neural network algorithm is extracted and burned into the processing system end, and the upper-layer code script is used to call the operator in the programmable logic end to process the image to be detected as described in any one of claims 1-4.

6. The edge deployment method according to claim 5, wherein: The edge deployment method further includes: Two data buffer areas are set up, and the two data buffer areas alternately receive the image to be detected sent by the camera. When one of the data buffer areas receives the image to be detected sent by the camera, the image to be detected in the other data buffer area is processed by the upper-level code script by calling the corresponding operator.

7. The edge deployment method according to claim 5, wherein: Before the step of extracting operators involved in the preset convolutional neural network algorithm and deploying the operators to the programmable logic end, the method includes: An initial convolutional neural network algorithm is trained according to a preset image set to obtain the preset convolutional neural network algorithm.

8. The edge deployment method according to claim 7, wherein: Before the step of training the initial convolutional neural network algorithm according to the preset image set to obtain the preset convolutional neural network algorithm, the method further includes: Deleting the i-th convolutional layer in the preset convolutional neural network algorithm each time to obtain a pruned convolutional neural network algorithm; Performing recognition using the pruned convolutional neural network algorithm to obtain a recognition accuracy rate of the pruned convolutional neural network algorithm; The recognition contribution of the i-th convolutional layer is determined according to the recognition accuracy of the pruned convolutional neural network algorithm.

9. A computer device, characterized in that: The computer device includes a programmable logic end and a processing system end. The programmable logic end includes: a convolution kernel composed of multiple operators, the convolution kernel is communicatively connected to the processing system end, and the processing system end is burned with an upper-level code script of a preset convolutional neural network algorithm. The upper-level code script is used to call the operators in the programmable logic end to process the image to be detected as described in any one of claims 1-4.

Images