A method, device, equipment and storage medium for detecting dangerous behaviors in intelligent warehouses
By introducing deep learning models into the pixel aggregation network for multiple rounds of feature extraction and feature fusion, and using detection boxes set with preset vector ranges for classification and regression prediction, the problems of low accuracy and insufficient detection efficiency of small and medium-sized objects in the prior art are solved, and efficient and accurate detection of dangerous behaviors is achieved.
Patent Information
- Application Number
- CN202411913640.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-24
- Publication Date
- 2025-04-04
AI Technical Summary
The existing image-based hazardous behavior detection methods have insufficient accuracy and efficiency, especially when detecting small targets, the accuracy rate is low, and the detection efficiency needs to be improved.
By introducing deep learning models into the pixel aggregation network, multiple rounds of feature extraction and feature fusion are performed, and classification and regression prediction are used to use detection boxes set with preset vector ranges to improve the accuracy and detection efficiency of small object detection.
The accuracy of small target detection is improved, and by accelerating detection efficiency, the overall performance of intelligent warehouse hazardous behavior detection is significantly improved.
Smart Images
Figure CN119763191A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of machine vision, and in particular to a method, device, equipment and storage medium for detecting dangerous behaviors in an intelligent warehouse. Background Art
[0002] Smart warehouses generally have only a small number of staff, and improper behavior of staff can cause major safety hazards. Therefore, it is necessary to detect dangerous behavior of staff in key areas to avoid safety risks.
[0003] The existing methods for image-based behavior detection are mainly based on image classification and image recognition. The image classification method generally performs pedestrian detection first, cuts out the pedestrian area and performs image classification, and directly classifies the pedestrian's action. This solution is simple, but can only be used in simple scenes with obvious action characteristics, and the detection accuracy is low; the image recognition method mainly detects objects closely related to the action to identify pedestrian actions. The general process is to first detect people, then detect the pedestrian area to see if there are behavior-related objects to determine whether the pedestrian is performing a specific behavior. However, the behavior recognition accuracy of this method depends on the detection accuracy of behavior-related objects, and the detection efficiency needs to be improved. Summary of the invention
[0004] In view of this, the purpose of the present invention is to provide a method, device, equipment and storage medium for detecting dangerous behaviors in smart warehouses, which can improve the accuracy of small target detection by introducing a deep learning model in a pixel aggregation network, and can accelerate the detection efficiency by setting a detection frame based on a preset vector range. The specific scheme is as follows:
[0005] In a first aspect, the present application discloses a method for detecting dangerous behaviors in an intelligent warehouse, which is applied to a dangerous behavior recognition network, comprising:
[0006] Performing several rounds of feature extraction on the received image to be detected to obtain several feature images corresponding to the several rounds of feature extraction, and extracting target feature images corresponding to the first round of feature extraction and the second round of feature extraction from the several feature images;
[0007] Performing feature extraction processing on the target feature image through a deep learning model in a local pixel aggregation network to obtain a processed target feature image, and performing feature fusion on the processed target feature image and non-target feature images in the plurality of feature images through the pixel aggregation network to obtain a plurality of feature fusion images;
[0008] Performing classification prediction on the plurality of feature fusion images to obtain classification feature information, and performing regression prediction on the plurality of feature fusion images through a preset vectorized deep learning network to obtain regression feature information;
[0009] Based on the regression feature information and the classification feature information, it is determined whether there is a dangerous behavior in the image to be detected.
[0010] Optionally, performing several rounds of feature extraction on the received image to be detected to obtain several feature images corresponding to the several rounds of feature extraction includes:
[0011] If an image to be detected is received, the image to be detected is input as an input image into a local feature extraction network;
[0012] Based on the feature extraction network, several rounds of feature extraction are performed on the input to obtain several feature images; wherein the feature image obtained in each round of feature extraction is the input image for the next round of feature extraction.
[0013] Optionally, the target feature image is subjected to feature extraction processing by a deep learning model in a local pixel aggregation network to obtain a processed target feature image, and the processed target feature image and non-target feature images in the plurality of feature images are subjected to feature fusion by the pixel aggregation network to obtain a plurality of feature fusion images, including:
[0014] Inputting the target feature image and non-target feature images among the plurality of feature images into a local pixel aggregation network;
[0015] Performing feature extraction on the target feature image through the deep learning model in the pixel aggregation network to obtain a processed target feature image corresponding to the target feature image;
[0016] The pixel aggregation network is used to perform feature fusion on the processed target feature image and non-target feature images in the plurality of feature images to obtain a plurality of feature fused images.
[0017] Optionally, performing classification prediction on the plurality of feature fusion images to obtain classification feature information includes:
[0018] The plurality of feature fusion images are classified and predicted through a classification deep learning layer in a dangerous behavior recognition network to determine classification feature information of a plurality of prediction objects existing in the plurality of feature fusion images.
[0019] Optionally, performing regression prediction on the plurality of feature fusion images by using a preset vectorized deep learning network to obtain regression feature information includes:
[0020] Setting parameters of a target frame, expressing the parameters of the target frame as a discretized vector distribution, and setting an integral range of the discretized vector distribution;
[0021] Determine a vectorized distributed focal loss based on the integral range, and fit the vectorized distributed focal loss through a preset deep learning network to obtain a preset vectorized deep learning network;
[0022] Regression prediction is performed on the plurality of feature fusion images through the preset vectorized deep learning network to obtain corresponding regression information.
[0023] Optionally, the determining whether there is a dangerous behavior in the image to be detected based on the regression feature information and the classification feature information includes:
[0024] Determining whether there is target classification feature information corresponding to preset dangerous object feature information in the classification feature information;
[0025] If the target classification feature information exists, determining the target regression feature information corresponding to the target classification feature information, and outputting a first detection result indicating the presence of a dangerous behavior based on the target classification feature information and the target regression feature information;
[0026] If the target classification feature information does not exist, a second detection result indicating that no dangerous behavior exists is output.
[0027] Optionally, the pixel aggregation network is a PAN network, and the deep learning model is a Transformer model.
[0028] In a second aspect, the present application discloses a dangerous behavior detection device for an intelligent warehouse, which is applied to a dangerous behavior identification network, comprising:
[0029] A feature extraction module is used to perform several rounds of feature extraction on the received image to be detected to obtain several feature images corresponding to the several rounds of feature extraction, and to extract target feature images corresponding to the first round of feature extraction and the second round of feature extraction from the several feature images;
[0030] A feature fusion module, used to perform feature extraction processing on the target feature image through a deep learning model in a local pixel aggregation network to obtain a processed target feature image, and perform feature fusion on the processed target feature image and non-target feature images in the plurality of feature images through the pixel aggregation network to obtain a plurality of feature fusion images;
[0031] An information prediction module, used to perform classification prediction on the plurality of feature fusion images to obtain classification feature information, and perform regression prediction on the plurality of feature fusion images through a preset vectorized deep learning network to obtain regression feature information;
[0032] The dangerous behavior detection module is used to determine whether there is a dangerous behavior in the image to be detected based on the regression feature information and the classification feature information.
[0033] In a third aspect, the present application discloses an electronic device, comprising:
[0034] Memory, used to store computer programs;
[0035] A processor is used to execute the computer program to implement the aforementioned intelligent warehouse dangerous behavior detection method.
[0036] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned method for detecting dangerous behaviors in an intelligent warehouse.
[0037] In the present application, firstly, the method of the present application can perform several rounds of feature extraction on the received image to be detected to obtain several feature images corresponding to the several rounds of feature extraction, and extract the target feature images corresponding to the first round of feature extraction and the second round of feature extraction from the several feature images; perform feature extraction processing on the target feature image through the deep learning model in the local pixel aggregation network to obtain the processed target feature image, and perform feature fusion on the processed target feature image and the non-target feature images in the several feature images through the pixel aggregation network to obtain several feature fusion images; perform classification prediction on the several feature fusion images to obtain classification feature information, and perform regression prediction on the several feature fusion images through a preset vectorized deep learning network to obtain regression feature information; and determine whether there is a dangerous behavior in the image to be detected based on the regression feature information and the classification feature information. It can be seen that through the method of the present application, it is necessary to perform several rounds of feature extraction on the received image to be detected, and determine the target feature images corresponding to the first two rounds of feature extraction from the obtained several feature extraction images, and then the target feature image can be processed by the deep learning model in the local pixel aggregation network for feature extraction, and the processed target feature image and the non-target feature images in the several feature images can be feature fused through the pixel aggregation network, and then the obtained several feature fusion images can be classified and predicted to obtain classification feature information, and finally, the several feature fusion images can be regressed and predicted through the preset vectorized deep learning network, and it can be determined whether there is dangerous behavior in the image to be detected based on the obtained regression feature information and classification feature information. In this way, the accuracy of small target detection can be improved by introducing a deep learning model in the pixel aggregation network, and the detection efficiency can be accelerated by setting a detection frame based on a preset vector range. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0039] Figure 1 A flow chart of a dangerous behavior detection method for an intelligent warehouse disclosed in this application;
[0040] Figure 2 A schematic diagram of a dangerous behavior identification network disclosed in this application;
[0041] Figure 3 A flowchart of a specific intelligent warehouse dangerous behavior detection method disclosed in this application;
[0042] Figure 4 This is a schematic diagram of the structure of a dangerous behavior detection device for an intelligent warehouse disclosed in this application;
[0043] Figure 5 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION
[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0045] In the prior art, dangerous behavior detection is usually performed through image classification-based or image recognition-based methods, but the former can only be used in simple scenarios with obvious action characteristics and has low detection accuracy; the latter's behavior recognition accuracy depends on the detection accuracy of behavior-related objects, and the detection efficiency needs to be improved.
[0046] In order to overcome the above-mentioned technical problems, the present application discloses a method, device, equipment and storage medium for detecting dangerous behaviors in intelligent warehouses, which can improve the accuracy of small target detection by introducing a deep learning model in the pixel aggregation network, and can accelerate the detection efficiency by setting a detection frame based on a preset vector range.
[0047] See also Figure 1 As shown, an embodiment of the present invention discloses a method for detecting dangerous behaviors in an intelligent warehouse, which is applied to a dangerous behavior identification network, and includes:
[0048] Step S11, performing several rounds of feature extraction on the received image to be detected to obtain several feature images corresponding to the several rounds of feature extraction, and extracting target feature images corresponding to the first round of feature extraction and the second round of feature extraction from the several feature images.
[0049] In this embodiment, the dangerous behavior detection of people in the warehouse is mainly divided into two parts. The first part is pedestrian detection. It is necessary to collect images or videos of the warehouse in real time to perform screen detection on the images or videos in the warehouse to determine whether there are pedestrians in the current screen. If there are pedestrians, the second part is to detect small targets. After identifying the pedestrians in the warehouse, it is necessary to detect related dangerous items such as cigarettes, telephones, etc. If related small targets are detected, it indicates that the pedestrian has dangerous behavior. If only pedestrians are detected but no related small targets are detected, it indicates that the behavior does not exist.
[0050] Specifically, after the warehouse monitoring equipment has collected images that need to be detected for dangerous behaviors, the images that need to be detected for dangerous behaviors can be input into the dangerous behavior recognition network. After the dangerous behavior recognition network receives the images to be detected, it is necessary to perform several rounds of feature extraction on the received images to be detected to obtain several feature images corresponding to the several rounds of feature extraction. Specifically, if an image to be detected is received, the image to be detected is input as an input image into the local feature extraction network; based on the feature extraction network, several rounds of feature extraction are performed on the input to obtain several feature images. That is, if Figure 2 As shown, after the danger recognition network receives the image to be detected, it performs four rounds of feature extraction on the image to be detected through the local feature extraction network, and the feature image obtained after each round of feature extraction is used as the input image for the next round of feature extraction. The method for dangerous behavior recognition network to detect dangerous behaviors is an improved algorithm based on the PP-YOLOE detection algorithm. The feature extraction network is CSPRepResNet, and CSPRepResNet consists of four modules with similar structures. Figure 2 C2 to C5 in the figure are the feature images output by the four modules, respectively. C2 and C3 are the target feature images corresponding to the first two rounds of feature extraction. After the convolution and other operations in the module, the resolution of the output feature image is reduced, but it contains higher-level semantic features, and the perception field of the points in the output feature image will also increase accordingly according to the resolution reduction ratio. In this way, the features in the image to be detected can be magnified to obtain several feature images.
[0051] Furthermore, since the area of the small target in the image is relatively small, the feature images corresponding to the first round of feature extraction and the second round of feature extraction in several feature images can be used as target feature images, and further feature extraction can be performed on the target feature images to magnify the features of the small target in the image.
[0052] Step S12: performing feature extraction processing on the target feature image through a deep learning model in a local pixel aggregation network to obtain a processed target feature image, and performing feature fusion on the processed target feature image and non-target feature images in the plurality of feature images through the pixel aggregation network to obtain a plurality of feature fusion images.
[0053] In this embodiment, it is necessary to perform feature extraction processing on the target feature image through a local pixel aggregation network, that is, a deep learning model in the Trans-PAN network, to obtain a processed target feature image. Specifically, Figure 2As shown, it is necessary to extract features from the target feature image through the deep learning model in the pixel aggregation network to obtain the processed target feature image corresponding to the image semantics of the target feature image, that is, after inputting the obtained several feature images into the pixel aggregation network, it is first necessary to further extract features from the target feature images corresponding to the first round of feature extraction and the second round of feature extraction through the deep learning model in the pixel aggregation network, that is, the transformer module. It should be noted that the transformer module is used to extract the spatial correlation features of small related objects and pedestrians. Since the pixel point perception field of the target feature image corresponding to the first two rounds of feature extraction is small, the complete semantic information of the small target can be extracted, and the perception field of the feature image corresponding to the subsequent feature extraction is large, so only the transformer module is used to extract features from the target feature images corresponding to the first two rounds of feature extraction again to amplify the features of the small target in the image.
[0054] Furthermore, it is necessary to perform feature fusion on the processed target feature image and non-target feature images in several feature images through a pixel aggregation network to obtain several feature fusion images. Specifically, Figure 2 As shown in , the pixel aggregation network needs to perform feature fusion on the processed feature image obtained by the transformer module and the non-target feature image that has not been processed by the transformer module, so that each image contains some features of other images after processing, such as Figure 2 As shown, after feature fusion, C2 to C5 are obtained to obtain P2 to P5 respectively. It should be noted that the P2 layer is derived by returning the feature map parameters after the first feature extraction process of CSPRepResNet, and then inputting them into the transformer module to obtain the P2 layer. In this way, after the deep learning model, that is, the feature extraction of the transformer module again, the features of small targets can be effectively amplified, thereby improving the recognition rate of small-sized related targets.
[0055] Step S13: performing classification prediction on the plurality of feature fusion images to obtain classification feature information, and performing regression prediction on the plurality of feature fusion images through a preset vectorized deep learning network to obtain regression feature information.
[0056] In this embodiment, it is necessary to perform classification prediction and regression prediction on the several feature fusion images obtained, and then obtain corresponding classification feature information and regression feature information. Specifically, it is necessary to set a target detection frame based on a preset vector range, and then perform motion detection on the several feature fusion images through the target detection frame to identify objects existing in the several feature fusion images, and use the identified several objects as prediction objects. It should be noted that since the objects existing in the image are of different sizes, several detection frames of different sizes can be set for detection to ensure the accuracy of object recognition. Furthermore, after the detection is completed and all the predicted objects in the image are obtained, it is necessary to determine the object categories of the several predicted objects, and then use the obtained object categories as classification feature information.
[0057] Furthermore, it is necessary to perform regression prediction on the feature fusion image. Specifically, it is necessary to perform motion detection on the several feature fusion images based on the target detection frame through a preset vectorized deep learning network to determine the detection areas currently existing in the several feature fusion images and corresponding to the several predicted objects detected. Then, it is necessary to record the detection area information of the detection area. It should be noted that it is also necessary to adjust the size of the target detection frame according to the preset vector range, and re-perform motion detection until the detection area information corresponding to the several predicted objects is recorded, so as to use all the recorded detection area information as regression feature information. In this way, by adjusting the detection frame, the complete detection of the feature image can be guaranteed, thereby improving the accuracy of dangerous behavior detection.
[0058] Step S14: determining whether there is any dangerous behavior in the image to be detected based on the regression feature information and the classification feature information.
[0059] In the present embodiment, it is necessary to determine whether there is any dangerous behavior in the image to be detected based on the regression feature information and the classification feature information. Specifically, it is necessary to determine whether there is target classification feature information corresponding to the preset dangerous object feature information in the classification feature information. For example, if there is feature information representing cigarettes in the classification feature information, and there is feature information representing cigarettes in the preset dangerous object feature information, then the feature information representing cigarettes is used as the target classification feature information, and then the target regression feature information corresponding to the target classification feature information is determined, and a first detection result representing the presence of dangerous behavior is output based on the target classification feature information and the target regression feature information. If there is no target classification feature information, then a second detection result representing the absence of dangerous behavior is output.
[0060] It can be seen that in this embodiment, it is necessary to perform several rounds of feature extraction on the received image to be detected, and determine the target feature images corresponding to the first two rounds of feature extraction from the obtained several feature extraction images, and then the target feature image can be feature extracted by the deep learning model in the local pixel aggregation network, and the processed target feature image and the non-target feature images in the several feature images can be feature fused by the pixel aggregation network, and then the obtained several feature fusion images can be classified and predicted to obtain classification feature information, and finally the several feature fusion images can be regressed and predicted by the preset vectorized deep learning network, and whether there is a dangerous behavior in the image to be detected can be determined based on the obtained regression feature information and classification feature information. In this way, on the one hand, due to the small size of the relevant target object, the receptive field of the point in the high-level feature map may be much larger than the target size, and the detection model will lose the small target when using the high-level feature map for detection. Therefore, the image can be further feature extracted by introducing a deep learning model in the pixel aggregation network, thereby improving the accuracy of small target detection; on the other hand, the detection efficiency can be accelerated by setting a detection frame based on a preset vector range.
[0061] Based on the above embodiments, it can be known that the method of the present application needs to perform classification prediction and regression prediction on the feature fusion image. For this reason, this embodiment describes in detail how to perform classification prediction and regression prediction on the feature fusion image. Figure 3 As shown, an embodiment of the present invention discloses a method for detecting dangerous behaviors in an intelligent warehouse, which is applied to a dangerous behavior identification network, and includes:
[0062] Step S21: classify and predict the multiple feature fusion images through the classification deep learning layer in the dangerous behavior recognition network to determine the classification feature information of the multiple prediction objects in the multiple feature fusion images.
[0063] In this embodiment, it is necessary to set a target detection frame by a preset vector range, and then classify and predict a number of feature fusion images by the target detection frame to determine a number of prediction objects existing in the number of feature fusion images. Specifically, Figure 2As shown, it is necessary to perform classification prediction through the Classification branch in the ET-head module in the PP-YOLOE detection algorithm, and it is necessary to set a target detection frame through a preset vector range, and classify and predict the several feature fusion images through the target detection frame to identify the objects in the several feature fusion images. For example, there may be pedestrians and beverages, telephones, cigarettes and other items held by pedestrians in the feature fusion images, and the size of each object is different. Therefore, it is necessary to set detection frames of various sizes to identify the objects in the feature fusion images to avoid the situation of missing objects, thereby improving the accuracy of dangerous behavior detection. In addition, it is also necessary to determine the object categories of several predicted objects. Specifically, after detecting several predicted objects, it is necessary to compare the features corresponding to the predicted objects with the features in the preset feature set to determine the object category of the predicted object, and then use the identified object category as classification feature information.
[0064] Step S22: setting parameters of the target frame, and expressing the parameters of the target frame as a discretized vector distribution, and setting an integration range of the discretized vector distribution.
[0065] In this embodiment, it is necessary to perform motion detection on several feature fusion images through the target detection frame to determine the detection areas corresponding to several predicted objects currently existing in the several feature fusion images. Specifically, the PP-YOLOE detection algorithm will regress, that is, Figure 2 The regression branch in the ET-head module is regarded as a distribution prediction task and DFL (Distribution Focal Loss loss function) is used in the regression branch to solve the problem of bounding box uncertainty, so that the network can quickly focus on the distribution of the neighboring area of the target position. In this embodiment, the DFL regression range is vectorized and the regression negative semi-axis is introduced. It should be noted that the DFL regression range vectorization means that the value x of the regression range reg_ma is not set to a scalar with a fixed value, but is set to a vector of a series of values. In this way, the regression is performed within multiple regression ranges, thereby minimizing the impact of large and small targets that cannot be taken into account. Further, the introduction of the regression negative semi-axis means that the regression range [0, reg_max] is extended to [-reg_max, reg_max]. This can increase the search range of the small target box, that is, the integral range of the discretized vector distribution, so that the algorithm can effectively and accurately regress the box of the small target. Therefore, the set target detection frame can be used to perform mobile detection on several feature fusion images to determine the detection areas currently existing in the several feature fusion images corresponding to the several predicted objects, and record the detection area information corresponding to the detection area.
[0066] Step S23, determining a vectorized distributed focal loss based on the integration range, and fitting the vectorized distributed focal loss through a preset deep learning network to obtain a preset vectorized deep learning network.
[0067] In this embodiment, it is necessary to determine the vectorized distributed focus loss according to the integration range, that is, to integrate according to the integration range to obtain the vectorized distributed focus loss, that is, the vectorized DFL, and then fit the vectorized distributed focus loss through the preset deep learning network to obtain the preset vectorized deep learning network.
[0068] Step S24: performing regression prediction on the plurality of feature fusion images through the preset vectorized deep learning network to obtain corresponding regression information.
[0069] In this embodiment, it is necessary to adjust the target detection frame according to the preset vector range, that is, adjust the target detection frame according to the regression range [-reg_max, reg_max] to re-perform motion detection until the detection area information corresponding to the several predicted objects obtained by classification prediction is recorded, and then all the detection area information is used as regression feature information. In this way, the detection accuracy of the detection frame can be improved by adjusting the preset vector range, thereby improving the accuracy of dangerous behavior detection.
[0070] It can be seen that in this embodiment, the classification prediction of the several feature fusion images is first performed through the classification deep learning layer in the dangerous behavior recognition network to determine the classification feature information of several predicted objects in the several feature fusion images. Then, the parameters of the target frame are set, and the parameters of the target frame are expressed as a discretized vector distribution, and the integral range of the discretized vector distribution is set, and the vectorized distribution focus loss is determined based on the integral range, and the vectorized distribution focus loss is fit through a preset deep learning network to obtain a preset vectorized deep learning network, and finally, the several feature fusion images are regressed and predicted through the preset vectorized deep learning network to obtain the corresponding regression information. In this way, the regression negative semi-axis can be introduced by vectorizing the regression range, and the regression upper limit can be adjusted according to different data sets, reducing the noise when regressing small targets, and improving the prediction effect of small targets.
[0071] See also Figure 4 As shown, an embodiment of the present invention discloses a dangerous behavior detection device for an intelligent warehouse, which is applied to a dangerous behavior identification network, including:
[0072] The feature extraction module 11 is used to perform several rounds of feature extraction on the received image to be detected to obtain several feature images corresponding to the several rounds of feature extraction, and extract the target feature images corresponding to the first round of feature extraction and the second round of feature extraction from the several feature images;
[0073] A feature fusion module 12 is used to perform feature extraction processing on the target feature image through a deep learning model in a local pixel aggregation network to obtain a processed target feature image, and perform feature fusion on the processed target feature image and non-target feature images in the plurality of feature images through the pixel aggregation network to obtain a plurality of feature fusion images;
[0074] The information prediction module 13 is used to perform classification prediction on the plurality of feature fusion images to obtain classification feature information, and perform regression prediction on the plurality of feature fusion images through a preset vectorized deep learning network to obtain regression feature information;
[0075] The dangerous behavior detection module 14 is used to determine whether there is a dangerous behavior in the image to be detected based on the regression feature information and the classification feature information.
[0076] In this embodiment, first, the method of the present application can be used to perform several rounds of feature extraction on the received image to be detected to obtain several feature images corresponding to the several rounds of feature extraction, and extract the target feature images corresponding to the first round of feature extraction and the second round of feature extraction from the several feature images; perform feature extraction processing on the target feature image through the deep learning model in the local pixel aggregation network to obtain a processed target feature image, and perform feature fusion on the processed target feature image and the non-target feature images in the several feature images through the pixel aggregation network to obtain several feature fusion images; perform classification prediction on the several feature fusion images to obtain classification feature information, and perform regression prediction on the several feature fusion images through a preset vectorized deep learning network to obtain regression feature information; and determine whether there is a dangerous behavior in the image to be detected based on the regression feature information and the classification feature information. It can be seen that through the method of the present application, it is necessary to perform several rounds of feature extraction on the received image to be detected, and determine the target feature images corresponding to the first two rounds of feature extraction from the obtained several feature extraction images, and then the target feature image can be processed by the deep learning model in the local pixel aggregation network for feature extraction, and the processed target feature image and the non-target feature images in the several feature images can be feature fused through the pixel aggregation network, and then the obtained several feature fusion images can be classified and predicted to obtain classification feature information, and finally, the several feature fusion images can be regressed and predicted through the preset vectorized deep learning network, and it can be determined whether there is dangerous behavior in the image to be detected based on the obtained regression feature information and classification feature information. In this way, the accuracy of small target detection can be improved by introducing a deep learning model in the pixel aggregation network, and the detection efficiency can be accelerated by setting a detection frame based on a preset vector range.
[0077] In some embodiments, the feature extraction module 11 may specifically include:
[0078] A first image input unit, configured to input the image to be detected as an input image to a local feature extraction network if the image to be detected is received;
[0079] The first feature extraction unit is used to perform several rounds of feature extraction on the input based on the feature extraction network to obtain several feature images; wherein the feature image obtained in each round of feature extraction is the input image for the next round of feature extraction.
[0080] In some embodiments, the feature fusion module 12 may specifically include:
[0081] A second image input unit, used for inputting the target feature image and non-target feature images among the plurality of feature images into a local pixel aggregation network;
[0082] A second feature extraction unit is used to extract features from the target feature image through a deep learning model in the pixel aggregation network to obtain a processed target feature image corresponding to the target feature image;
[0083] A feature fusion unit is used to perform feature fusion on the processed target feature image and non-target feature images in the plurality of feature images through the pixel aggregation network to obtain a plurality of feature fusion images.
[0084] In some embodiments, the information prediction module 13 may specifically include:
[0085] The classification prediction unit is used to perform classification prediction on the plurality of feature fusion images through the classification deep learning layer in the dangerous behavior recognition network to determine the classification feature information of the plurality of prediction objects existing in the plurality of feature fusion images.
[0086] In some embodiments, the information prediction module 13 may specifically include:
[0087] An integration range setting unit, used to set parameters of a target frame, express the parameters of the target frame as a discretized vector distribution, and set an integration range of the discretized vector distribution;
[0088] A deep learning network generating unit, used for determining a vectorized distributed focal loss based on the integral range, and fitting the vectorized distributed focal loss through a preset deep learning network to obtain a preset vectorized deep learning network;
[0089] A regression prediction unit is used to perform regression prediction on the plurality of feature fusion images through the preset vectorized deep learning network to obtain corresponding regression information.
[0090] In some embodiments, the dangerous behavior detection module 14 may specifically include:
[0091] An information matching unit, used to determine whether there is target classification feature information corresponding to preset dangerous object feature information in the classification feature information;
[0092] A first detection result generating unit is used to determine target regression feature information corresponding to the target classification feature information if the target classification feature information exists, and output a first detection result indicating the existence of a dangerous behavior based on the target classification feature information and the target regression feature information;
[0093] The second detection result generating unit is configured to output a second detection result indicating that no dangerous behavior exists if the target classification feature information does not exist.
[0094] In some embodiments, the pixel aggregation network is a PAN network, and the deep learning model is a Transformer model.
[0095] Furthermore, the present application also discloses an electronic device. Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.
[0096] Figure 5 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the intelligent warehouse dangerous behavior detection method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.
[0097] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.
[0098] In addition, the memory 22 as a carrier for resource storage may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.
[0099] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the intelligent warehouse dangerous behavior detection method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs that can be used to complete other specific tasks.
[0100] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned intelligent warehouse dangerous behavior detection method is implemented. The specific steps of the method can refer to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.
[0101] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0102] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0103] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0104] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0105] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A method for detecting dangerous behaviors in an intelligent warehouse, characterized in that: Applied to dangerous behavior identification network, including: Performing several rounds of feature extraction on the received image to be detected to obtain several feature images corresponding to the several rounds of feature extraction, and extracting target feature images corresponding to the first round of feature extraction and the second round of feature extraction from the several feature images; Performing feature extraction processing on the target feature image through a deep learning model in a local pixel aggregation network to obtain a processed target feature image, and performing feature fusion on the processed target feature image and non-target feature images in the plurality of feature images through the pixel aggregation network to obtain a plurality of feature fusion images; Performing classification prediction on the plurality of feature fusion images to obtain classification feature information, and performing regression prediction on the plurality of feature fusion images through a preset vectorized deep learning network to obtain regression feature information; Based on the regression feature information and the classification feature information, it is determined whether there is a dangerous behavior in the image to be detected.
2. The method for detecting dangerous behaviors in an intelligent warehouse according to claim 1, characterized in that: The step of performing several rounds of feature extraction on the received image to be detected to obtain several feature images corresponding to the several rounds of feature extraction includes: If an image to be detected is received, the image to be detected is input as an input image into a local feature extraction network; Based on the feature extraction network, several rounds of feature extraction are performed on the input to obtain several feature images; wherein the feature image obtained in each round of feature extraction is the input image for the next round of feature extraction.
3. The intelligent warehouse dangerous behavior detection method according to claim 1 is characterized in that: The target feature image is subjected to feature extraction processing by a deep learning model in a local pixel aggregation network to obtain a processed target feature image, and the processed target feature image and non-target feature images in the plurality of feature images are subjected to feature fusion by the pixel aggregation network to obtain a plurality of feature fusion images, including: Inputting the target feature image and non-target feature images among the plurality of feature images into a local pixel aggregation network; Performing feature extraction on the target feature image through the deep learning model in the pixel aggregation network to obtain a processed target feature image corresponding to the target feature image; The pixel aggregation network is used to perform feature fusion on the processed target feature image and non-target feature images in the plurality of feature images to obtain a plurality of feature fused images.
4. The intelligent warehouse dangerous behavior detection method according to claim 1 is characterized in that: The performing classification prediction on the plurality of feature fusion images to obtain classification feature information includes: The plurality of feature fusion images are classified and predicted through a classification deep learning layer in a dangerous behavior recognition network to determine classification feature information of a plurality of prediction objects existing in the plurality of feature fusion images.
5. The intelligent warehouse dangerous behavior detection method according to claim 4 is characterized in that: The performing regression prediction on the plurality of feature fusion images by using a preset vectorized deep learning network to obtain regression feature information includes: Setting parameters of a target frame, expressing the parameters of the target frame as a discretized vector distribution, and setting an integral range of the discretized vector distribution; Determine a vectorized distributed focal loss based on the integral range, and fit the vectorized distributed focal loss through a preset deep learning network to obtain a preset vectorized deep learning network; Regression prediction is performed on the plurality of feature fusion images through the preset vectorized deep learning network to obtain corresponding regression information.
6. The intelligent warehouse dangerous behavior detection method according to claim 1 is characterized in that: The determining whether there is a dangerous behavior in the image to be detected based on the regression feature information and the classification feature information includes: Determining whether there is target classification feature information corresponding to preset dangerous object feature information in the classification feature information; If the target classification feature information exists, determining the target regression feature information corresponding to the target classification feature information, and outputting a first detection result indicating the presence of a dangerous behavior based on the target classification feature information and the target regression feature information; If the target classification feature information does not exist, a second detection result indicating that no dangerous behavior exists is output.
7. The method for detecting dangerous behaviors in an intelligent warehouse according to any one of claims 1 to 6, characterized in that: The pixel aggregation network is a PAN network, and the deep learning model is a Transformer model.
8. An intelligent warehouse dangerous behavior detection device, characterized in that: Applied to dangerous behavior identification network, including: A feature extraction module is used to perform several rounds of feature extraction on the received image to be detected to obtain several feature images corresponding to the several rounds of feature extraction, and to extract target feature images corresponding to the first round of feature extraction and the second round of feature extraction from the several feature images; A feature fusion module, used to perform feature extraction processing on the target feature image through a deep learning model in a local pixel aggregation network to obtain a processed target feature image, and perform feature fusion on the processed target feature image and non-target feature images in the plurality of feature images through the pixel aggregation network to obtain a plurality of feature fusion images; An information prediction module, used to perform classification prediction on the plurality of feature fusion images to obtain classification feature information, and perform regression prediction on the plurality of feature fusion images through a preset vectorized deep learning network to obtain regression feature information; The dangerous behavior detection module is used to determine whether there is a dangerous behavior in the image to be detected based on the regression feature information and the classification feature information.
9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor is used to execute the computer program to implement the dangerous behavior detection method for an intelligent warehouse as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the method for detecting dangerous behaviors in an intelligent warehouse as described in any one of claims 1 to 7 is implemented.