Detection method for abnormal condition of reactor pit of waste incineration power plant, edge deployment method of detection method and equipment

Abstract

The invention discloses a waste incineration power plant pile pit abnormal condition detection method, an edge deployment method of the detection method and equipment, the abnormal condition detection method comprises the following steps: receiving a to-be-detected image sent by a camera, the to-be-detected image being obtained by the camera shooting the waste incineration power plant pile pit; determining a required operator type in the processing process of the to-be-detected image, and calling a corresponding operator for processing to obtain a processing result; and detecting an abnormal condition in the to-be-detected image according to the processing result and an upper layer of a preset convolutional neural network. According to the method, the operator types are classified, then the operator types of the processed images are determined in the image processing process, and the corresponding image processing is performed by calling the operators, so that the neural network algorithm can be lightened, and a foundation is laid for edge deployment of the algorithm.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular, to a method for detecting abnormal conditions in the stacking pit of a waste incineration power plant, an edge deployment method and device for the detection method. Background Art

[0002] With the development of the global economy and the advancement of urbanization, according to relevant reports, urban waste has shown an explosive growth. Under the current situation, waste incineration power generation has become the best way to achieve the 'decrease in quantity, harmlessness, and resource utilization' of domestic waste. On the one hand, compared with landfilling waste, the incineration method can not only avoid the pollution of groundwater by landfill leachate, but also avoid potential safety hazards caused by gases such as methane generated by fermentation. On the other hand, compared with thermal power generation, incineration power generation uses domestic waste instead of fossil fuels, which can reduce greenhouse gas emissions.

[0003] With the popularization of waste incineration power plants, problems in industrial production have also emerged. For example, abnormal conditions such as the mixing of large waste and fires in the waste yard often seriously affect the production efficiency and safe operation of the power plant. Currently, foreign objects in waste incineration power plants are often manually identified by the human eye. However, due to the relatively complex environment of the waste yard itself, manual identification cannot well distinguish waste foreign objects, resulting in serious safety hazards and low efficiency. Summary of the Invention

[0004] The main purpose of this application is to provide a method for detecting abnormal conditions in the stacking pit of a waste incineration power plant, an edge deployment method and device for the detection method, aiming to solve the technical problem of low efficiency of the existing manual detection method.

[0005] In a first aspect, to achieve the above object, this application provides a method for detecting abnormal conditions in the stacking pit of a waste incineration power plant, the method comprising: Receiving a to-be-detected image sent by a camera; Determining the required operator type during the processing of the to-be-detected image, and calling the corresponding operator for processing to obtain a processing result; Detecting abnormal conditions in the to-be-detected image according to the processing result and the upper layer of a preset convolutional neural network.

[0006] Optionally, the determining the required operator type during the processing of the to-be-detected image, and calling the corresponding operator for processing to obtain a processing result includes: Determining the convolutional layer with the least contribution to the detection result in the convolutional operator, and defining it as the weakening convolutional layer; Obtaining the average value of the data in the weakening convolutional layer, and using the average value as the processing result of the weakening convolutional layer; Detecting abnormal conditions in the image to be detected according to the processing result and the upper layer of the preset convolutional neural network includes: Restoring the average value through the upper layer of the preset convolutional neural network to obtain restored data; Detecting abnormal conditions in the image to be detected according to the restored data.

[0007] Optionally, obtaining the average value of the data in the weakening convolutional layer and using the average value as the processing result of the weakening convolutional layer includes: Determining the L1 norm values of the convolutional kernels in other convolutional layers, where the other convolutional kernels are convolutional layers other than the weakening convolutional layer; Sorting the L1 norm values of the convolutional kernels in other convolutional layers; Determining the average value corresponding to the channel corresponding to the sorted convolutional kernel.

[0008] Optionally, the step of receiving the image to be detected sent by the camera includes: Receiving the image to be detected sent by multiple cameras through a double-buffer mechanism; The step of determining the required operator type during the processing of the image to be detected, calling the corresponding operator for processing, and obtaining the processing result includes: Alternately processing the image to be detected in the double-buffer area to obtain the processing result.

[0009] Optionally, the step of determining the required operator type during the processing of the image to be detected, calling the corresponding operator for processing, and obtaining the processing result includes: Performing convolution and normalization processing on the image to be detected simultaneously to obtain the processing result.

[0010] Optionally, the operator includes convolutional operators, activation operators, pooling operators, upsampling operators, and splicing operators of multiple scales.

[0011] Optionally, the weights and / or activation functions in the detection method are of fixed-point type.

[0012] In a second aspect, the present application also provides an edge deployment method for a method for detecting abnormal conditions in a garbage incineration power plant stack pit. The edge deployment method includes: Extracting the operators involved in the preset convolutional neural network algorithm and deploying the operators to the programmable logic end; Extracting the upper-layer code script of the preset convolutional neural network algorithm and burning it to the processing system end. The upper-layer code script is used to call the operators in the programmable logic end to perform corresponding processing on the image to be detected.

[0013] Optionally, the edge deployment method further includes: Set up two data buffers. The two data buffers alternately receive the images to be detected sent by the camera. When one of them is receiving the images to be detected sent by the camera, the images to be detected in the other buffer are processed by the corresponding operators called by the upper-layer code script for the images to be detected.

[0014] Optionally, before the step of extracting the operators involved in the preset convolutional neural network algorithm and deploying the operators to the programmable logic end, it includes: Train the initial convolutional neural network algorithm according to a preset image set to obtain the preset convolutional neural network algorithm.

[0015] Optionally, before the step of training the initial convolutional neural network algorithm according to a preset image set to obtain the preset convolutional neural network algorithm, it further includes: Delete the i-th convolutional layer in the preset convolutional neural network algorithm each time to obtain a cropped convolutional neural network algorithm; Perform recognition through the cropped convolutional neural network algorithm to obtain the recognition accuracy of the cropped convolutional neural network algorithm; Determine the recognition contribution degree of the i-th convolutional layer according to the recognition accuracy of the cropped convolutional neural network algorithm.

[0016] In a third aspect, the present application also provides a computer device. The computer device includes a programmable logic end and a processing system end. The programmable logic end includes: a convolution kernel composed of multiple operators. The convolution kernel is communicatively connected to the processing system end. The upper-layer code script of the preset convolutional neural network algorithm is burned in the processing system end. The upper-layer code script is used to call the operators in the programmable logic end to perform corresponding processing on the images to be detected.

[0017] In the technical solution of the present application, by receiving the images to be detected sent by the camera, the images to be detected are obtained by the camera photographing the stacking pit of the waste incineration power plant; during the processing of the images to be detected, determine the required operator types, and call the corresponding operators for processing to obtain the processing results; detect the abnormal conditions in the images to be detected according to the processing results and the upper layer of the preset convolutional neural network. In this solution, the operator types are classified, and then the operator types for processing the images are determined during the image processing process. By calling the operators for corresponding image processing, the neural network algorithm can be made lightweight, laying a foundation for the edge deployment of the algorithm, and meeting the needs of some special scenarios; at the same time, there is no need to design and adjust each similar operator separately, greatly reducing the complexity and workload of model design, and reducing the design cost and error probability. Description of the Drawings

[0018] Figure 1 Schematic diagram of the hardware structure of the terminal involved in the solution of the embodiment of the present application; Figure 2 Schematic flow chart of the first embodiment of the method for detecting abnormal conditions in the stacking pit of a waste incineration power plant according to the present application; Figure 3 Schematic diagram of pruning operation in the convolutional layer in the prior art of the present application; Figure 4 Schematic diagram of operations in the convolutional layer of the present application; Figure 5 Schematic flow chart of the edge deployment method for the method of detecting abnormal conditions in the stacking pit of a waste incineration power plant according to the present application; Figure 6 Schematic diagram of the internal circuit of the computer for edge deployment according to the present application; Figure 7 Schematic diagram of the power consumption result after deployment in the embodiment of the present application.

[0019] The realization of the purpose, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0020] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0021] The method for detecting abnormal conditions in the stacking pit of a waste incineration power plant involved in the embodiment of the present application is mainly applied to a terminal, and the terminal can be a device with processing functions such as a PC, a portable computer, a mobile terminal, etc.

[0022] Referring to Figure 1 , Figure 1 is a schematic diagram of the terminal structure involved in the solution of the embodiment of the present application. The terminal in the embodiment of the present application is a computer device, and this computer device is generally located in special scenarios such as waste incineration power plants. This scenario generally meets the requirements of lightweight or fast processing; in the embodiment of the present application, the computer device may include a processor 1001 (such as a CPU), a communication bus 1002, a programmable logic terminal (module) 1003, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components; the memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. Optionally, the memory 1005 can also be a storage device independent of the aforementioned processor 1001, the programmable logic terminal (module) 1003. It can be understood that the terminal in the embodiment of the present application may further include a camera, and the camera is used to acquire images in the application scenario and transmit the acquired images to the processor 1001 through a wired connection.

[0023] Those skilled in the art can understand that Figure 1 the hardware structure shown in [[ ]] does not constitute a limitation on the device, and may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0024] Continuing to refer to [[ ]] Figure 1 , Figure 1 the memory 1005 as a computer-readable storage medium in [[ ]] may include an operating system, a programmable logic unit (module), and a detection program for abnormal situations.

[0025] In [[ ]] Figure 1 the operators required for the detection program of abnormal situations are deployed in the programmable logic unit (module), and the processor 1001 can call the detection program of abnormal situations in the garbage incineration power plant pit stored in the memory 1005 and perform the following operations: Receive the image to be detected sent by the camera, where the image to be detected is obtained by the camera photographing the garbage incineration power plant pit; Determine the type of operator required during the processing of the image to be detected, and call the corresponding operator for processing to obtain a processing result; Detect abnormal situations in the image to be detected according to the processing result and the upper layer of the preset convolutional neural network.

[0026] Furthermore, the processor 1001 can call the detection program of abnormal situations stored in the memory 1005 and perform the following operations: The determining the type of operator required during the processing of the image to be detected, and calling the corresponding operator for processing to obtain a processing result includes: Determine the convolutional layer with the least contribution to the detection result in the convolutional operator, and define it as the weakening convolutional layer; Obtain the average value of the data in the weakening convolutional layer, and use the average value as the processing result of the weakening convolutional layer; The detecting abnormal situations in the image to be detected according to the processing result and the upper layer of the preset convolutional neural network includes: Restore the average value through the upper layer of the preset convolutional neural network to obtain restored data; Detect abnormal situations in the image to be detected according to the restored data.

[0027] Furthermore, the processor 1001 can call the detection program of abnormal situations stored in the memory 1005 and perform the following operations: The obtaining the average value of the data in the weakening convolutional layer, and using the average value as the processing result of the weakening convolutional layer includes: Determine the L1 norm values of the convolution kernels in other convolutional layers, where the other convolutional kernels are convolutional layers other than the weakening convolutional layer; Sort the L1 norm values of the convolution kernels in other convolutional layers; Determine the average value corresponding to the channel corresponding to the sorted convolution kernel.

[0028] Further, the processor 1001 can call the detection program for abnormal situations stored in the memory 1005 and perform the following operations: Receive the images to be detected sent by multiple cameras through a double-buffer mechanism; The step of determining the required operator type during the processing of the images to be detected and calling the corresponding operator for processing to obtain a processing result includes: Alternately process the images to be detected in the double-buffer area to obtain the processing result.

[0029] Further, the processor 1001 can call the detection program for abnormal situations in the waste incineration power plant pit stored in the memory 1005 and perform the following operations: Perform convolution and normalization processing on the images to be detected simultaneously to obtain the processing result.

[0030] Further, the operators include convolution operators, activation operators, pooling operators, upsampling operators, and splicing operators of multiple scales.

[0031] Further, the weights and / or activation functions in the detection method are of fixed-point type.

[0032] Based on the above hardware structure of the terminal, various embodiments of the detection method for abnormal situations in the waste incineration power plant pit of the present application are proposed.

[0033] The present application provides a detection method for abnormal situations in the waste incineration power plant pit. In the present application, the neural network algorithm is described by taking the YOLO network as an example. Other convolutional neural networks can be selected in specific embodiments. The application scenario is a waste station, and other scenarios can be used in specific implementations, which are not limited here.

[0034] Please refer to Figure 2 , in the first embodiment of the present application, the detection method for abnormal situations in the waste incineration power plant pit includes the following steps: S100, receive the images to be detected sent by the camera; Specifically, in one embodiment, the detected image is an image captured in real time by a data acquisition device. To meet the requirement of data real-time in scenarios such as garbage stations, the camera is transmitted to a processing terminal (computer) at the edge of the garbage station through a wired connection. During the abnormal situation detection process, the image of the garbage station captured by the camera in real time is used as the image to be detected. Among them, the camera can also be other data acquisition devices, such as scanners, cameras and other devices with shooting functions.

[0035] It can be understood that before the abnormal situation detection, a survey is conducted on the situation of the waste incineration power plant, and it is found that the types of garbage affecting production mainly include wood, mattresses, gas cylinders and iron sheets. For the above four types of garbage, a data set is prepared according to the situation of the waste incineration power plant. Then, the detection algorithm is trained using the data set, and the trained convolutional neural network algorithm is used as the preset neural network algorithm. Further, when the data in the data set is insufficient, the existing data set can be expanded. For example, the original images collected in the first step can be data-expanded by using methods such as mirroring, rotation, scale transformation, random cropping, and color jittering. The way to expand the data set can refer to the existing technology and will not be elaborated here. Further, by adjusting indicators such as brightness, contrast, saturation and hue, the sensitivity of the neural network algorithm to colors is reduced.

[0036] Taking the YOLOv4tiny network algorithm as an example for illustration, the traditional YOLOv4tiny network algorithm includes a backbone network, a neck network and a detection head. Compared with standard versions such as YOLOv4, YOLOv4-tiny reduces the number of network layers and parameters, thus reducing the computational complexity and memory occupancy. This enables it to run quickly on devices with limited resources, such as embedded devices and mobile devices, while ensuring a relatively high accuracy.

[0037] In this application, first, the types of operators involved in the traditional YOLOv4tiny network algorithm are counted. There are 7 types of operators involved in YOLOv4tiny, including 3 configurations of Conv2D (convolution operator), LeakyReLU (activation function operator), Concat (concatenation operator), Maxpool2d (pooling operator), Upsample (upsampling operator). The types of operators can vary according to the different neural network algorithms actually used. As one embodiment, all these 7 operators can be deployed to the programmable logic side. As another embodiment, Concat can be performed with a pointer at the processing system side, and only the other 6 operators are deployed to the programmable logic side. Then, the code script (defined as the upper layer) of the traditional YOLOv4tiny network algorithm except the operators is deployed to the processing system side. After the deployment is completed, the image of the garbage station captured in real time by the data acquisition device is received as the image to be detected.

[0038] S200. During the process of processing the image to be detected, determine the required operator type and call the corresponding operator for processing to obtain a processing result. Specifically, extract basic feature information from the input image through a backbone network, further fuse and process the features extracted by the backbone network through a neck network to generate multi-scale feature representations, and detect targets based on the feature maps output by the neck network through a detection head. During these processes, before using an operator each time, determine the required operator type and call the operator corresponding to the required operator type for processing to obtain the corresponding processing result. For example, during the process of extracting basic feature information from the input image through the backbone network, convolution operators, normalization operators, and pooling operators are required. Specifically, extract the features of the image through the first type of convolution operator, then make the data distribution more stable through the normalization operator to accelerate the convergence speed and improve stability; finally, reduce the size of the feature data after normalization through the pooling operator to reduce the amount of calculation. During the process of further fusing and processing the features extracted by the backbone network through the neck network to generate multi-scale feature representations, convolution operators, concatenation operators, and upsampling operators are required. Specifically, continue to extract and adjust the features of the feature map output by the backbone network through the second type of convolution operator, and further refine the features through different convolution kernel parameters to provide more representative features for the subsequent detection head. Then, concatenate the feature maps of different scales or different branches in the channel dimension through the concatenation operator to integrate multi-scale feature information. Finally, magnify the feature map output by the concatenation operator through the upsampling operator to match its size with that of the high-resolution feature map for the concatenation operation. During the process of detecting targets based on the feature map output by the neck network through the detection head, convolution operators and activation function operators are required. First, perform final feature extraction and processing on the feature map output by the neck network through the third type of convolution operator to generate features for target detection and classification, and then map the output result to the interval [0, 1] through the activation function operator. It should be noted that in this application, the operators are divided into 7 types, and then when an operator needs to be used, call the operator in the programmable logic module for processing, which greatly reduces the number of operators. At the same time, there is no need to design and adjust each similar operator separately, which greatly reduces the complexity and workload of model design, and reduces the design cost and error probability.

[0039] It can be understood that after obtaining the image to be detected in step S100, the image to be detected can also be preprocessed through an input layer first, such as data denoising, normalization, etc.

[0040] S300. Detect abnormal situations in the image to be detected based on the processing result and the upper layer of the preset convolutional neural network.

[0041] Specifically, based on the processing result and the upper layer of the preset convolutional neural network, it is determined whether the image to be detected belongs to an abnormal situation and the category of the abnormality to which the abnormal situation belongs.

[0042] In the technical solution of this application, an image to be detected sent by a camera is received; during the processing of the image to be detected, the type of operator required is determined, and the corresponding operator is called for processing to obtain a processing result; based on the processing result and the upper layer of the preset convolutional neural network, the abnormal situation in the image to be detected is detected. In this solution, the types of operators are classified, and then the type of operator for processing the image is determined during the image processing process. By calling the operator for corresponding image processing, the neural network algorithm can be made lightweight, laying a foundation for the edge deployment of the algorithm and meeting the needs of some special scenarios; at the same time, there is no need to design and adjust each similar operator separately, greatly reducing the complexity and workload of model design, and reducing the design cost and error probability.

[0043] Further, step S200: The process of determining the type of operator required during the processing of the image to be detected, calling the corresponding operator for processing, and obtaining a processing result includes: Step S210, determining the convolutional layer with the least contribution to the detection result in the convolutional operator, which is defined as the weakened convolutional layer; Step S220, obtaining the average value of the data in the weakened convolutional layer, and using the average value as the processing result of the weakened convolutional layer; Specifically, since there are multiple convolutional layers in the neural network algorithm and there are multiple convolutional kernels in each convolutional layer, in order to further improve the calculation speed to meet the fast recognition requirements of application scenarios such as waste incineration power plants, this application further obtains a trained preset convolutional neural network algorithm during training, and then only deletes the i-th convolutional layer in the preset convolutional neural network algorithm each time, that is, the preset convolutional neural network algorithm is cropped (cropping one convolutional layer). For the convenience of description, it is defined as the cropped convolutional neural network algorithm, and then the cropped convolutional neural network algorithm is used for recognition, and finally the recognition accuracy of the cropped convolutional neural network algorithm is determined. This is repeated N times, and the recognition accuracy of each cropped convolutional neural network algorithm is defined as ACC1, ACC2...ACC N , using the accuracy of the complete trained preset convolutional neural network algorithm, the contribution of each cropped convolutional neural network algorithm can be obtained. Exemplarily, the contribution W i =(ACC / ACC i ) 2 , where ACC is the accuracy of the complete trained preset convolutional neural network algorithm.

[0044] After obtaining the contribution degrees of each convolutional layer, the convolutional layer with the smallest contribution degree to the detection result is determined and defined as the weakened convolutional layer. The average value of the data in the weakened convolutional layer is obtained, and the average value is used as the processing result of the weakened convolutional layer.

[0045] Correspondingly, step S300: The detecting the abnormal situation in the image to be detected according to the processing result and the upper layer of the preset convolutional neural network includes: Step S310, restoring the average value through the upper layer of the preset convolutional neural network to obtain restored data; Step S320, detecting the abnormal situation in the image to be detected according to the restored data.

[0046] Specifically, after obtaining the average value of the data in the weakened convolutional layer, the upper layer of the preset convolutional neural network (specifically before deconvolution or feature extraction) restores according to the average value to obtain restored data, and then detects the abnormal situation in the image to be detected according to the restored data. In this way, during the convolution process, only the average value of the convolutional layer with the lowest contribution to the accuracy is calculated and then restored. On the one hand, it does not affect the convolution with high recognition accuracy, and on the other hand, data restoration is performed before recognition, which not only ensures the accuracy of recognition but also reduces the computational complexity (the process of calculating the average value has less computational complexity than multiple convolutions and deconvolutions).

[0047] Further, step S220, the obtaining the average value of the data in the weakened convolutional layer and using the average value as the processing result of the weakened convolutional layer includes: Step S221, determining the L1 norm values of the convolutional kernels in other convolutional layers, where the other convolutional kernels are convolutional layers other than the weakened convolutional layer; Step S222, sorting the L1 norm values of the convolutional kernels in other convolutional layers; Step S223, determining the average value corresponding to the channel corresponding to the sorted convolutional kernel.

[0048] Specifically, independently calculate the independent L1 norm values of each convolutional kernel in the convolutional layers of other convolutional layers (the convolutional layers that have not been pruned, that is, the convolutional layers other than the weakened convolutional layer) in turn, and sort the L1 norm values of the convolutional kernels from largest to smallest in each convolutional layer. Determine the m channels corresponding to the m convolutional kernels, and obtain the average value of the channel.

[0049] To better understand this application, this application is compared and described with the existing direction (structured pruning): Currently, in structured pruning, when a certain channel is pruned, the operations of other channels are still in progress (multi-channel operations are parallel. Although the pruned channel is no longer being calculated, it is just idle and waiting), which results in the inability to reduce the convolution operation time and causes resource waste. Moreover, since the data of this channel is deleted, it will affect the accuracy rate. As Figure 3 shown, the white part is the deleted data. This application is equivalent to retaining the mean value of the pruned channel on the basis of structured pruning. As Figure 4 shown, this application is equivalent to filling the average value into the Figure 3 white part in. From the perspective of a single convolution operation, this results in a reduction in the channel parameters after structured pruning, but compared with the parameters before pruning, the advantage of reducing storage is still retained (the convolution kernel parameters of one channel are deleted). This application retains the calculated average value of the data in the channel after structured pruning in the prior art and restores it later, which can ensure the accuracy rate.

[0050] Further, in step S100, the step of receiving the image to be detected sent by the camera includes: Receiving the images to be detected sent by multiple cameras through a double-buffer mechanism; In step S200, the step of determining the required operator type during the processing of the image to be detected and calling the corresponding operator for processing to obtain a processing result includes: Alternately processing the images to be detected in the double-buffer area to obtain the processing result.

[0051] Specifically, this application also sets up a double-buffer mechanism, and the interface data uses the "ping-pong" strategy, enabling loading and calculation to be carried out simultaneously. There are multiple cameras. When there is a double-buffer mechanism at the input port, when one buffer (buffer area) is calculating, the other buffer can be used to store data, and the two buffers alternate to receive data transmitted from different cameras, achieving a two-fold increase in throughput.

[0052] Further, in step S200: The step of determining the required operator type during the processing of the image to be detected and calling the corresponding operator for processing to obtain a processing result includes: Performing convolution and normalization processing on the image to be detected simultaneously to obtain the processing result.

[0053] Specifically, in order to greatly reduce data exchange and the amount of computation, in this embodiment, when there are both a convolution layer and a normalization layer in the neural network algorithm, the convolution layer and the normalization layer are fused. After fusion, the computational amount of the two layers becomes one layer, reducing data exchange and the amount of computation, and further improving the lightweight of the algorithm.

[0054] Further, in the detection method described in this application, the parameters are of fixed-point type. Specifically, the weights or activations involved in each operator and the upper-layer code are represented by fixed-point numbers, which can further improve the calculation speed and ensure the real-time performance of signal processing.

[0055] In addition, please refer to Figure 5 , this application also provides an edge deployment method for the abnormal situation detection method of the waste incineration power plant pit. The edge deployment method includes: Step S1: Extract the operators involved in the preset convolutional neural network algorithm and deploy the operators to the programmable logic end; Step S2: Extract the upper-layer code script of the preset convolutional neural network algorithm and burn it to the processing system end. The upper-layer code script is used to call the operators in the programmable logic end for corresponding processing.

[0056] Specifically, due to electromagnetic shielding during the normal operation of the waste incineration power plant, it is impossible to upload the captured images wirelessly. At the same time, in order to avoid problems such as damage to the power plant equipment caused by gas cylinders and iron sheets and other garbage, and to affect the combustion efficiency and effect of the garbage, in order to meet the need to quickly detect abnormalities and facilitate the timely cleaning of abnormalities, edge deployment needs to be carried out at a location near the waste incineration power plant. In this application, the types of operators involved in the traditional YOLOv4tiny network algorithm are first counted. There are 7 types of operators involved in YOLOv4tiny, including 3 configured Conv2D (convolution calculation operator), LeakyReLU (activation function operator), Concat (concatenation operator), Maxpool2d (pooling operator), Upsample (upsampling operator). The types of operators can vary according to the different neural network algorithms actually used. As an example, these 7 operators can all be deployed to the programmable logic end. As another example, Concat can be implemented with a pointer at the processing system end, and only the other 6 operators are deployed to the programmable logic end. Then, the code script (defined as the upper layer) other than the operators in the traditional YOLOv4tiny network algorithm is deployed to the processing system end, thus realizing edge deployment.

[0057] As an example, the entire computer device after edge deployment, as Figure 6 shown, the computer device includes a clock module, a programmable logic end, a communication module, and a processing system end. Various operators of the neural network algorithm are deployed inside the programmable logic end. There are multiple communication modules. The communication module and the clock module are the same as those in the prior art. In this application, mainly a programmable logic end is added, and the upper-layer code script is deployed to the processing system end. Other modules are the same as those in the prior art. The communication module is modified according to needs for the connection relationship with other modules.

[0058] The edge deployment method further includes: Two data buffers are set. The two data buffers alternately receive the images to be detected sent by the camera, and when one of them is receiving the images to be detected sent by the camera, the images to be detected in the other are processed by the corresponding operators called by the upper-layer code script.

[0059] Specifically, the present application also sets a double-buffer mechanism, and the interface data uses the "ping-pong" strategy, so that loading and calculation can be carried out simultaneously. When there is a double-buffer mechanism at the input port, when one buffer is calculating, the other buffer can be used to store data, and the two buffers alternate to achieve a two-fold increase in throughput.

[0060] Furthermore, it can be understood that before the step of extracting the operators involved in the preset convolutional neural network algorithm and deploying the operators to the programmable logic end, it includes: Training the initial convolutional neural network algorithm according to a preset image set to obtain the preset convolutional neural network algorithm.

[0061] Specifically, before deployment, a survey is conducted on the situation of waste incineration power plants, and it is found that the types of waste affecting production mainly include wood, mattresses, gas cylinders, and iron sheets. For the above four types of waste, a data set is prepared according to the situation of waste incineration power plants. Then, the detection algorithm is trained using the data set, and the trained convolutional neural network algorithm is used as the preset neural network algorithm.

[0062] Furthermore, before the step of training the initial convolutional neural network algorithm according to a preset image set to obtain the preset convolutional neural network algorithm, it further includes: Fusing the convolutional layer and the normalization layer in the initial convolutional neural network algorithm.

[0063] Specifically, in order to greatly reduce data exchange and the amount of computation, in this embodiment, when there are both a convolutional layer and a normalization layer in the neural network algorithm, the convolutional layer and the normalization layer are fused. After fusion, the amount of computation of the two layers becomes one layer, reducing data exchange and the amount of computation, and further improving the lightweight of the algorithm (that is, the convolutional operator has the functions of convolution and normalization). Further, the parameters in the detection method described in the present application are of fixed-point number type.

[0064] Furthermore, before the step of training the initial convolutional neural network algorithm according to a preset image set to obtain the preset convolutional neural network algorithm, it further includes: Each time, the i-th convolutional layer in the preset convolutional neural network algorithm is deleted to obtain a cropped convolutional neural network algorithm; Perform recognition through the described pruned convolutional neural network algorithm to obtain the recognition accuracy of the pruned convolutional neural network algorithm; Determine the recognition contribution degree of the i-th convolutional layer according to the recognition accuracy of the pruned convolutional neural network algorithm.

[0065] Specifically, during training, after obtaining the trained preset convolutional neural network algorithm, only delete the i-th convolutional layer in the preset convolutional neural network algorithm each time, that is, perform pruning on the preset convolutional neural network algorithm (prune one convolutional layer). For the convenience of description, it is defined as the pruned convolutional neural network algorithm. Then use the pruned convolutional neural network algorithm for recognition, and finally determine the recognition accuracy of this pruned convolutional neural network algorithm. Repeat this N times. The recognition accuracy of each pruned convolutional neural network algorithm is defined as ACC1, ACC2...ACC N , using the accuracy of the complete trained preset convolutional neural network algorithm, the contribution degree of each pruned convolutional neural network algorithm can be obtained. Exemplarily, the contribution degree W i =(ACC / ACC i ) 2 , where ACC is the accuracy of the complete trained preset convolutional neural network algorithm. So that in the subsequent recognition process, the contribution degree of each convolutional kernel to the accuracy can be used to optimize the calculation speed. Specifically, reference can be made to step S200 and the refined steps in the above embodiment, which will not be elaborated here.

[0066] Testing with the technical solution in this application realizes abnormal garbage recognition while also realizing low-latency and low-power edge deployment. It reduces the scheduling and switching times between operators. The process that originally required multiple scheduling of different operators can now complete the calculation of multiple operators at one time, reducing the additional time overhead and lowering the latency of the entire calculation process. For example, in an image classification task, after integrating the three operators of convolution, batch normalization, and activation function, the final classification result can be obtained faster. The inference latency is about 427 ms, reaching 2.43 FPS, and the throughput is 3.43 GOP / 0.428 s≈8.01 GOPS. Without acceleration, it is 496 s based on the current CPU. The speedup ratio is 496 * 1000 ms / 427 ms = 1161, achieving good acceleration. The on-chip power consumption is only 1.883 W, far lower than that of the PC side. The end point temperature is 28.5 °C, the thermal resistance is 7.1 °C / watt (6.5 watts), the effective junction temperature is 1.9 °C / watt, the off-chip device power consumption is 0 watts, and the power consumption margin level is medium. The specific power consumption is as Figure 7 shown, meeting the requirements of lightweight deployment.

[0067] In this application, the operator types are classified, and the operators are deployed on the programmable logic side, while the others are deployed on the processing system side. Then, during the process of processing the image, the operator type of the processed image is determined, and the corresponding image processing is performed by calling the operator. In this way, the neural network algorithm can be made lightweight, laying a foundation for the edge deployment of the algorithm and meeting the needs of some special scenarios. At the same time, there is no need to design and adjust each operator of the same type separately, greatly reducing the complexity and workload of model design, and lowering the design cost and error probability.

[0068] In addition, this application also provides a computer-readable storage medium.

[0069] A detection program for abnormal situations in the pit of a waste incineration power plant is stored on the computer-readable storage medium of this application. When the detection program for abnormal situations in the pit of a waste incineration power plant is executed by a processor, the steps of the detection method for abnormal situations in the pit of a waste incineration power plant as described above are implemented.

[0070] Among them, the method implemented when the detection program for abnormal situations in the pit of a waste incineration power plant is executed can refer to each embodiment of the detection method for abnormal situations in the pit of a waste incineration power plant of this application, and will not be elaborated here.

[0071] Those skilled in the art should understand that the embodiments of this application can be provided as a method, a system, or a computer program product. Therefore, this application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] This application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0073] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means that implement the function specified in one or more of the flow Figure 1 acts or a plurality of acts and / or boxes Figure 1 boxes or a plurality of boxes.

[0074] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the function specified in one or more of the flow Figure 1 acts or a plurality of acts and / or boxes Figure 1 boxes or a plurality of boxes.

[0075] It should be noted that, in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of other elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application may be implemented by means of hardware comprising several distinct elements and by means of a suitably programmed computer. In the unit claims listing several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. do not denote any order. These words may be interpreted as names.

[0076] Although the preferred embodiments of the present application have been described, additional changes and modifications can be made by those skilled in the art once they learn of the basic inventive concept. Therefore, the appended claims are intended to be construed to cover the preferred embodiments as well as all changes and modifications falling within the scope of the present application.

[0077] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural transformation made by using the content of the specification and drawings of the present application under the inventive concept of the present application, or any direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A method for detecting abnormal conditions in a waste incineration power plant pit, characterized in that: The method comprises: Receiving an image to be detected sent by a camera, wherein the image to be detected is obtained by photographing a waste incineration power plant pit by the camera; Determining the required operator type during the processing of the image to be detected, and calling the corresponding operator for processing to obtain the processing result; The abnormal situation in the image to be detected is detected according to the processing result and the upper layer of the preset convolutional neural network.

2. The abnormal situation detection method according to claim 1, characterized in that: The method of determining the required operator type during the processing of the image to be detected, and calling the corresponding operator for processing to obtain the processing result includes: Determine the convolution layer in the convolution operator that contributes the least to the detection result, which is defined as the weakened convolution layer; Obtaining an average value of the data in the weakened convolution layer, and using the average value as a processing result of the weakened convolution layer; The detecting of abnormal conditions in the image to be detected according to the processing result and the upper layer of the preset convolutional neural network includes: Restoring the average value through the upper layer of the preset convolutional neural network to obtain restored data; The abnormality in the image to be detected is detected according to the restored data.

3. The abnormal situation detection method according to claim 2, characterized in that: The obtaining an average value of the data in the weakened convolution layer and taking the average value as a processing result of the weakened convolution layer includes: Determine the L1 norm value of the convolution kernel in other convolution layers, where other convolution kernels are convolution layers other than the weakened convolution layer; Sort the L1 norm values ​​of the convolution kernels in other convolutional layers; Determine the average value of the channels corresponding to the sorted convolution kernels.

4. The abnormal situation detection method according to claim 1, characterized in that: The step of receiving the image to be detected sent by the camera includes: Receive the images to be detected sent by multiple cameras through a double buffer mechanism; The step of determining the required operator type during the processing of the image to be detected, calling the corresponding operator for processing, and obtaining the processing result includes: The images to be detected in the double buffer areas are processed alternately to obtain the processing results.

5. The abnormal situation detection method according to claim 1, characterized in that: The step of determining the required operator type during the processing of the image to be detected, calling the corresponding operator for processing, and obtaining the processing result includes: The image to be detected is subjected to convolution and normalization processing simultaneously to obtain the processing result.

6. An edge deployment method for detecting abnormal conditions in a waste incineration power plant pit, characterized in that: The edge deployment method comprises: Extracting operators involved in a preset convolutional neural network algorithm and deploying the operators to a programmable logic end; The upper-layer code script of the preset convolutional neural network algorithm is extracted and burned into the processing system end, and the upper-layer code script is used to call the operator in the programmable logic end to perform corresponding processing on the image to be detected.

7. The edge deployment method according to claim 6, characterized in that: The edge deployment method further includes: Two data buffer areas are set, and the two data buffer areas alternately receive the image to be detected sent by the camera. When one of them receives the image to be detected sent by the camera, the image to be detected in the other one is processed by the upper-level code script by calling the corresponding operator.

8. The edge deployment method according to claim 6, characterized in that: Before the step of extracting operators involved in the preset convolutional neural network algorithm and deploying the operators to the programmable logic end, the method includes: An initial convolutional neural network algorithm is trained according to a preset image set to obtain the preset convolutional neural network algorithm.

9. The edge deployment method according to claim 8, characterized in that: Before the step of training the initial convolutional neural network algorithm according to the preset image set to obtain the preset convolutional neural network algorithm, the method further includes: Deleting the i-th convolutional layer in the preset convolutional neural network algorithm each time to obtain a pruned convolutional neural network algorithm; Performing recognition by using the pruned convolutional neural network algorithm to obtain a recognition accuracy rate of the pruned convolutional neural network algorithm; The recognition contribution of the i-th convolutional layer is determined according to the recognition accuracy of the pruned convolutional neural network algorithm.

10. A computer device, characterized in that: The computer device includes a programmable logic end and a processing system end. The programmable logic end includes: a convolution kernel composed of multiple operators, the convolution kernel is communicatively connected to the processing system end, and the processing system end is burned with an upper-level code script of a preset convolutional neural network algorithm. The upper-level code script is used to call the operators in the programmable logic end to perform corresponding processing on the image to be detected.

Images