Intelligent supervision method and system, edge end and storage medium

Abstract

The invention discloses an intelligent supervision method and system, an edge end and a storage medium. The method comprises the steps that the edge end detects whether a target object in a real-time image has a dangerous behavior or not based on a step-by-step recognition model and detects whether the target object is in a safe area or not; if the target object is in the safe area and dangerous behaviors occur, an anomaly detection result is sent to a server, and the server pushes the anomaly detection result to a client of a related manager; according to the method provided by the invention, target identification detection and front-end and rear-end interaction are separated and executed by different main bodies, so that the computing power requirement on each main body can be reduced, the system cost is reduced, and the system compatibility is improved.

Description

Technical Field

[0001] The present invention belongs to the field of logistics safety technology, and specifically relates to an intelligent supervision method and system, an edge terminal, and a storage medium. Background Art

[0002] The safety management and safety accident prevention of logistics warehouses are very important parts of logistics enterprise operations. In the safety management of enterprises, managers need to constantly explore and update the safety management methods and safety accident prevention models of enterprises, so as to improve their overall safety management level. However, in the actual implementation process, safety accidents often occur due to the influence of imperfect management technology, management methods and management concepts.

[0003] With the rapid development of information technology, the 21st century has entered an era of informatization and intelligence. The efficient application of information technology has played a huge role in promoting the development of productivity. The use of artificial intelligence machine vision intelligent management technology in the basic management of safe production in logistics warehouses is of great significance to improving the safety of logistics warehouses and personnel, safe production and operation of warehouses.

[0004] Artificial intelligence technology takes equipment, materials, personnel, vehicles and environment as research objects, and realizes automatic analysis and extraction of effective information in video sources by identifying and extracting important objects in video sources, replacing human eyes with cameras, and learning and thinking in a more "intelligent" way. The application of this management system can achieve 24-hour uninterrupted, unmanned intelligent duty, achieving the goal of greatly saving resources and manpower. This method can quickly detect various types of safety hazards and warn them, thereby reducing the workload of manual inspections.

[0005] However, the current supervision method mainly uses high-performance computing servers to deploy artificial intelligence detection algorithms to identify and supervise targets. Although high-performance computing servers have the ability to process large-scale data sets, the procurement cost and operation and maintenance cost of high-performance computing servers, especially those equipped with high-end graphics processors or other dedicated acceleration hardware, are very high. At the same time, the current method integrates the front-end, back-end and algorithms on the server, resulting in heavy computing tasks on the server, and the method cannot be executed on servers with lower computing power. Once the business depends on a specific high-performance hardware configuration, if it needs to be migrated to other platforms or technologies in the future, it may encounter compatibility issues, increasing the complexity and cost of migration. Especially in the case of rapid algorithm upgrades or technology iterations, this dependence may limit flexibility.

[0006] Therefore, current supervision methods are costly and less compatible. Summary of the invention

[0007] The embodiments of the present invention provide an intelligent supervision method and system, an edge terminal, and a storage medium, which can solve the problems of high cost and low compatibility of current supervision methods.

[0008] In a first aspect, an embodiment of the present invention provides an intelligent supervision method, the method is applied to an intelligent supervision system, the system includes an edge terminal, a server and a client, and the method includes:

[0009] The edge determines whether the target object in the real-time image has any dangerous behavior based on the trained step-by-step recognition model, and determines whether the target object is in a safe area based on the trained human tracking model;

[0010] The step-by-step recognition model includes a target object recognition model, a super-resolution enhancement network and at least one classification recognition model; the target object recognition model is used to recognize the target object in the real-time image to obtain the location information of the target object area, and the super-resolution enhancement network is used to enhance the target object image to obtain a super-resolution target object image; the classification recognition model is used to recognize whether the target object has a dangerous behavior based on the super-resolution target object image; the target object image is a low-resolution image obtained by cutting the real-time image based on the location information;

[0011] If the target object is in a safe area and the target object has a dangerous behavior, the edge terminal selects the target object in the real-time image and sends the abnormal detection result to the server, where the abnormal detection result includes the real-time image after the selection and the category of the dangerous behavior;

[0012] The server pushes the anomaly detection results to the client of the manager responsible for the target object and / or dangerous behavior.

[0013] In a second aspect, an embodiment of the present invention provides an intelligent monitoring system, the system comprising an edge terminal, a server, and a client;

[0014] The edge is used to determine whether the target object in the real-time image has any dangerous behavior based on the trained step-by-step recognition model, and to determine whether the target object is in a safe area based on the trained human tracking model;

[0015] The step-by-step recognition model includes a target object recognition model, a super-resolution enhancement network and at least one classification recognition model; the target object recognition model is used to recognize the target object in the real-time image to obtain the location information of the target object area, and the super-resolution enhancement network is used to enhance the target object image to obtain a super-resolution target object image; the classification recognition model is used to recognize whether the target object has a dangerous behavior based on the super-resolution target object image; the target object image is a low-resolution image obtained by cutting the real-time image based on the location information;

[0016] If the target object is in a safe area and the target object has a dangerous behavior, the edge end is further used to select the target object in the real-time image and send an abnormal detection result to the server, wherein the abnormal detection result includes the real-time image after the selection and the category of the dangerous behavior;

[0017] The server is used to push the anomaly detection results to the client of the manager responsible for the target object and / or dangerous behavior.

[0018] In a third aspect, an embodiment of the present invention provides an edge terminal, wherein the edge terminal includes:

[0019] A target recognition module is used to determine whether a target object in a real-time image has engaged in dangerous behavior based on a trained step-by-step recognition model, and to determine whether the target object is located in a safe area based on a trained human tracking model;

[0020] The step-by-step recognition model includes a target object recognition model, a super-resolution enhancement network and at least one classification recognition model; the target object recognition model is used to recognize the target object in the real-time image to obtain the location information of the target object area, and the super-resolution enhancement network is used to enhance the target object image to obtain a super-resolution target object image; the classification recognition model is used to recognize whether the target object has a dangerous behavior based on the super-resolution target object image; the target object image is a low-resolution image obtained by cutting the real-time image based on the location information;

[0021] An image processing module, which is used to select the target object in the real-time image if the target object is in a safe area and the target object performs dangerous behavior;

[0022] An interactive module is used to send anomaly detection results to a server so that the server can push the anomaly detection results to the client of the manager responsible for the target object and / or abnormal behavior, wherein the anomaly detection results include a real-time image after the selection and the category of dangerous behavior.

[0023] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed, the method of the first aspect described above can be implemented.

[0024] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: according to the intelligent supervision method provided by the present invention, by separating the target recognition task and the front-end and back-end interaction tasks on different subjects, the computing cost and computing power requirements for each subject can be reduced, thereby reducing the system cost and increasing the system stability; by performing target detection using a step-by-step recognition model that can detect both details and the whole, the detection effect at the edge can be enhanced; by superimposing the target detection results and the edge detection results, the detection error problem in the edge and boundary areas of the image, and the problem of false alarms of dangerous behaviors in non-safe areas can be avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of the structure of an intelligent monitoring system provided by an embodiment of the present invention;

[0026] Figure 2 A schematic diagram of the implementation flow of a step-by-step recognition model training method provided by an embodiment of the present invention;

[0027] Figure 3 A schematic diagram of a scenario for training a step-by-step recognition model provided by an embodiment of the present invention;

[0028] Figure 4 A schematic diagram of an implementation flow of an intelligent supervision method provided by an embodiment of the present invention;

[0029] Figure 5 A schematic diagram of the structure of an edge terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0031] Figure 1 The structure diagram of an intelligent monitoring system provided by an embodiment of the present invention is shown. As an example but not a limitation, the system 100 may include an edge terminal 110 , a server 120 and a client 130 .

[0032] In one possible implementation, the edge terminal 110 can be connected to a camera in a logistics warehouse and receive a video stream from the camera. Each frame of real-time image in the video stream is detected in real time, and based on the built-in trained distribution recognition model, it is determined whether the target object in the real-time image has committed a dangerous behavior, and based on the trained human body tracking model, it is determined whether the target object is in a safe area. If the edge terminal 110 detects through a frame of real-time image that the target object has committed a dangerous behavior and the target object that has committed the dangerous behavior is located in a safe area, the edge terminal can select the target object that has committed the dangerous behavior on the frame of image, and then send the abnormal detection result to the server 120. The server 120 can push the abnormal detection result to the client 130 of the administrator who is responsible for the target object and / or such dangerous behavior.

[0033] Exemplarily, the step-by-step recognition model may include a target object recognition model, a super-resolution enhancement network and at least one classification recognition model; the target object recognition model is used to identify the target object in the real-time image to obtain the location information of the area where the target object is located, and the super-resolution enhancement network is used to enhance the target object image to obtain a super-resolution target object image; the classification recognition model is used to identify whether the target object has committed dangerous behavior based on the super-resolution target object image.

[0034] Exemplarily, the target object image is a low-resolution image obtained by cropping a real-time image according to position information of the target object.

[0035] Exemplarily, the abnormal detection result may include a real-time image after the frame is selected and a category of dangerous behavior.

[0036] Exemplarily, dangerous behaviors may include not wearing a helmet, smoking, sudden illness, dropping of cargo, etc.

[0037] For example, if a worker who is not wearing a safety helmet enters a safety area pre-designated by the management, the edge terminal 110 responsible for the area can trigger a safety alarm and send a real-time image of the detected behavior to the server 120; then the server 120 pushes a warning message including the abnormal detection result to the client 130 of the worker who is not wearing a safety helmet, reminding him to wear a safety helmet.

[0038] For example, if goods accidentally fall from a transport vehicle during circulation, the edge end 110 pushes the abnormality detection result to the client 130 of the manager responsible for the area through the server 120, reminding the manager to clean up and return the fallen goods.

[0039] In one example, the server 120 may integrate the detection anomaly processing module and the interaction information integration module. The detection anomaly processing module may determine the client ID held by the corresponding responsible person through the received anomaly detection result, and then the interaction information integration module pushes the anomaly detection result to the client 130 corresponding to the ID.

[0040] Exemplarily, the backend of the client 130 may be deployed on the server 120 , and the server 120 may provide interactive data and information to users and administrators.

[0041] Optionally, the server 120 can also manage and maintain the edge terminal 110 and the camera.

[0042] Optionally, the server 120 may also perform statistical analysis and other data processing on the data regularly, for example, reporting the monthly security report to a manager with relevant authority.

[0043] In one example, the client 130 may integrate a multi-target visual detection human-computer interaction module, and users (including managers, staff, etc.) may log in to the module through a web page or application to view abnormal detection results and receive message reminders of abnormal detection results. Depending on the user's authority, the manager's client may define a safe area and forcefully remind the target object to stop its dangerous behavior.

[0044] According to the system provided by the present invention, by allowing different entities to perform target detection and recognition, front-end and back-end interaction processes, the computing power requirements for each device can be reduced, so that the method provided by the present invention can be executed on the edge terminal and server with lower configuration, thereby reducing system costs and improving system compatibility.

[0045] Figure 2 The flowchart shown is a method for implementing a step-by-step recognition model training method provided by an embodiment of the present invention. As an example but not a limitation, the training method can be used to train the step-by-step recognition model used in the above system 100. The training method can include steps S201-S204, and each step is described below.

[0046] S201, inputting a sample image into a target object recognition model to obtain location information of an area where a sample target object is located.

[0047] In some embodiments, the sample image may be obtained by performing data enhancement processing on the original sample.

[0048] Exemplarily, the original sample may be a labeled sample.

[0049] Artificial intelligence visual inspection technology requires a large number of labeled image samples as training sets. In deep learning, the size of the training set affects the accuracy and generalization ability of the entire model. In enterprise production, data sampling, cleaning and labeling often require a lot of manpower, money and time. By expanding the original samples through data enhancement processing, the demand for labeled original samples can be reduced, thereby reducing production costs.

[0050] In a possible implementation, the data enhancement process may include one or more of the following: image cropping, image rotation, adding noise, and adding mosaic.

[0051] In one example, the basic principle of image cropping is to remove uninteresting and redundant parts of the original sample image or adjust the focus area of ​​the image to achieve the purpose of highlighting the subject, changing the image ratio, removing interference elements, etc.

[0052] For example, irrelevant information at the edge of the image may be removed to make the main object or region of interest in the image more prominent, thereby improving the accuracy and efficiency of the model in recognizing the image.

[0053] For example, the image composition can be changed by adjusting the visual center of the image to make it more in line with aesthetic or analytical requirements, thereby improving the overall visual effect of the original sample.

[0054] For example, the aspect ratio of the original sample can be adjusted or the image size can be reduced according to the requirements of the application in the client, thereby improving compatibility with special display devices, saving storage space and network transmission efficiency.

[0055] In one example, the basic idea of ​​image rotation is to transform the coordinates of each pixel on the image according to the same center point and rotation angle.

[0056] Exemplarily, the center point may be the center of the image or may be specified by the user.

[0057] Image rotation can improve the model's ability to recognize and match features, thereby improving the stability and accuracy of the system.

[0058] In one example, the image acquisition process in the real world is often interfered by various noises, such as camera sensor noise, changes in lighting conditions, signal attenuation during transmission, etc. Therefore, by randomly adding artificial noise to the original sample, the model can effectively recognize and classify images under noisy conditions, thereby improving its performance in complex environments. At the same time, exposing the model to diverse inputs during training can enable the model to learn to ignore unimportant details (i.e., noise) and focus on key features; thereby improving the model's tolerance to noise, enhancing its recognition performance for noisy images, and improving the robustness of the model. Adding noise can also increase the diversity and capacity of sample images, forcing the model to learn more general features, reduce dependence on specific details of sample images, avoid overfitting of the model, and enhance the generalization ability of the model.

[0059] In one example, adding mosaics is to merge pixels in certain local areas of an image into larger areas with uniform colors to simulate privacy protection processing, reduce image details, or perform specific abstract processing.

[0060] Generally, when publishing or sharing images, in order to protect personal privacy or sensitive information (such as employee faces), these sensitive areas are covered by adding mosaics to make the privacy / sensitive information unrecognizable, thereby protecting the privacy of users. At the same time, adding mosaics to the original samples can also simulate situations where image details are missing, such as distant targets or low image quality, so that the model can still work effectively in this case. As a special form of data enhancement, adding mosaics can introduce new visual changes, increase the diversity of sample images, help the model learn invariant features under different visual representations, and improve the generalization ability of the model.

[0061] Alternatively, see Figure 3 , the original sample can be expanded by 4 times, 6 times, 4 times, and 6 times respectively through image cropping, image rotation, adding noise, and adding mosaic.

[0062] In some embodiments, see Figure 3 , input the sample image into the target object recognition model to obtain the location information of the area where the sample target object is located.

[0063] Exemplarily, the sample target object is of the same type as the target object, which may be a complete human body, a specific part of a human body, or an object. The specific type needs to be determined according to the category of the dangerous behavior.

[0064] For example, if the dangerous behavior is smoking, the sample target object / target object may include the human head and human hands; if the dangerous behavior includes not wearing a helmet, the sample target object / target object may include the human head; if the dangerous behavior includes not wearing safety clothing, the sample target object / target object may include the human torso.

[0065] In one possible implementation, see Figure 3 , the target object recognition model can include a convolutional neural network, a fully connected layer, and a RoI pooling layer.

[0066] In one example, after the convolutional neural network extracts the features of the sample image, Gaussian noise can be added to the features to obtain a Gaussian noise map to improve the generalization and learning capabilities of the model, thereby providing the model with better detection performance for targets in different scenarios. Afterwards, the fully connected layer obtains the confidence region of the sample target object on the feature map based on the Gaussian noise map, and the RoI pooling layer maps the confidence region to a fixed size, and then outputs the probability distribution of the classification of the sample target object through the subsequent fully connected layer, such as the probability that the target object is a human head, the probability that it is a certain object, etc., and the location information of the area where the sample target object is located, such as the coordinates of the predicted classification box.

[0067] For example, Gaussian noise can be added to the feature map. In this case, the detail feature map / overall feature map with noise can be expressed as:

[0068]

[0069] in, is a detail feature map / overall feature map with noise, X is a detail feature map / overall feature map, N(0,σ2) represents Gaussian noise, σ2 is the noise variance, and 0 is the noise mean.

[0070] S202, determining a loss function of a target object recognition model according to a predicted classification box and a true classification box of the sample target object, and training the target object recognition model.

[0071] S203: training a super-resolution enhancement network according to the sample target object image and the super-resolution sample target object image.

[0072] Exemplarily, the sample target object image may be a low-resolution image cropped from the sample image according to the actual position of the sample target object.

[0073] In one example, the super-resolution enhancement network can be a generative adversarial network, using a generator and a discriminator for adversarial training. The sample target object image can be used as the input of the generator, and the low-noise super-resolution sample target object image can be used as the edge pinching discrimination input. In adversarial training, the generator will gradually improve the quality of the sample target object image and input it into the discriminator for discrimination until the image generated by the generator cannot be distinguished by the discriminator, and a trained super-resolution enhancement network can be obtained.

[0074] Exemplarily, the ImageNet dataset may be used as sample target object images and super-resolution sample target object images.

[0075] S204: training a classification recognition model based on the super-resolution sample target object image.

[0076] In a possible implementation, a dangerous behavior can correspond to a classification recognition model. The classification recognition model is used to identify whether the sample target object has performed the corresponding dangerous behavior. In this way, the detection problem can be transformed into a 0-1 classification problem, reducing the interference and influence of the background layer on the image detection.

[0077] For example, the classification recognition model can be trained under the constraint of cross entropy loss.

[0078] Exemplarily, the target object recognition model, the super-resolution enhancement network and the classification recognition model can be trained under different loss function constraints.

[0079] According to the training method provided by the present invention, by performing data enhancement on the original samples to obtain sample images, and performing training based on the sample images, the generalization ability and robustness of the trained model can be improved, and the difficulty of image acquisition can be reduced.

[0080] Figure 4 The flowchart of an implementation of an intelligent supervision method provided by an embodiment of the present invention is shown. As an example but not a limitation, the intelligent supervision method can be used in the above system 100, and the intelligent supervision method can include steps S401-S404, and each step is described below.

[0081] S401, the edge determines whether the target object in the real-time image has performed dangerous behavior based on the trained step-by-step recognition model, and determines whether the target object is located in a safe area based on the trained human body tracking model.

[0082] In some embodiments, if the target object performs dangerous behavior and is located in a safe area, step S402 may be performed.

[0083] For example, the edge end can interact with the camera in the logistics warehouse through the TCP / IP protocol to obtain the video stream, and simultaneously input each frame of real-time image in the video stream into the step-by-step recognition model and the human body tracking model.

[0084] In a possible implementation, the step-by-step recognition model may be trained by the above training method. After the step-by-step recognition model is trained, it may be migrated to the edge end.

[0085] Exemplarily, the step-by-step recognition model may include a target object recognition model, a super-resolution enhancement network, and at least one classification recognition model. The target object recognition model may be used to recognize the target object in the real-time image to obtain the location information of the target object area, the super-resolution enhancement network is used to enhance the target object image to obtain a target object image with a higher granularity and super-resolution; the classification recognition model is used to recognize whether the target object has engaged in dangerous behavior based on the super-resolution target object image.

[0086] Exemplarily, the target object image is a low-resolution image obtained by cropping the real-time image according to the position information.

[0087] Dividing the real-time image into small areas, enhancing it, and then performing behavior recognition can improve the detection rate of small dangerous behaviors of the target object, such as smoking, and avoid the situation where small target objects are missed.

[0088] In one example, the tensorRT acceleration function on the edge and the model conversion function in the NVIDIA and pytorch architectures can be used to convert the step-by-step recognition model into an engine model that can be accelerated using CUDA (Compute Unified Device Architecture). This can improve the reasoning speed of edge computing while maintaining model performance, save GPU memory, and enable more real-time monitoring tasks for cameras on a single edge.

[0089] In one possible implementation, the human body tracking model can use the Mediapipe framework to perform skeleton tracking on the human body and obtain the coordinates of key skeleton points of the human body; such as the head, chest, legs, knees, arms, feet, etc.

[0090] For example, after obtaining the human coordinates output by the human tracking model, the edge end can also perform intrusion detection based on these coordinates and the boundary of the defined safe area, and detect whether the coordinates of the feet of the human body as the target object are within the safe area.

[0091] Optionally, if the target object is an object, the edge end may directly confirm whether the object is within the safe area according to the area size of the target object.

[0092] In other embodiments, if the target object does not engage in dangerous behavior or is not in a safe area, the subsequent process is not performed, and the edge continues to detect and analyze the next frame of real-time image.

[0093] S402: The edge terminal selects a target object in the real-time image.

[0094] In one example, the edge can save the real-time image, then select the target object in the real-time image based on the image drawing algorithm and the location information output by the step-by-step recognition model, and finally save the anomaly detection result, that is, the real-time image after the selection and the category of the dangerous behavior.

[0095] S403: The edge sends anomaly detection results to the server.

[0096] Accordingly, the server receives anomaly detection results.

[0097] S404, the server pushes the anomaly detection result to the corresponding client.

[0098] Exemplarily, the corresponding client may include a client of a manager responsible for the target object and / or the dangerous behavior, such as an employee who performs the dangerous behavior or a manager responsible for safety maintenance.

[0099] Accordingly, users can obtain anomaly detection results from the client and obtain location information where dangerous behaviors occur.

[0100] According to the intelligent supervision method provided by the present invention, by setting the target recognition task and the front-end and back-end interaction tasks separately on different subjects, the computational cost and computing power requirements for each subject can be reduced, thereby reducing the system cost and increasing the system stability; by performing target detection with a step-by-step recognition model that can detect both details and the whole, the detection effect at the edge can be enhanced; by superimposing the target detection results and the edge detection results, the detection error problem of the edge and boundary area of ​​the image, and the problem of false alarms of dangerous behaviors in non-safe areas can be avoided. Furthermore, the TCP / IP protocol is used for communication between the various subjects of the system, so that as long as each subject and the camera are under the same router, the camera can be automatically addressed and located, thereby enhancing the automation of the system.

[0101] Figure 5 The diagram is a schematic diagram of the structure of an edge terminal provided by an embodiment of the present invention. As an example but not a limitation, the edge terminal 110 may include a target recognition module 111 , an image processing module 112 and an interaction module 113 .

[0102] The target recognition module 111 may include a step-by-step recognition model and a human tracking model. The target recognition module 111 is used to determine whether a target object in a real-time image has a dangerous behavior based on the trained step-by-step recognition model, and to determine whether the target object is located in a safe area based on the trained human tracking model.

[0103] The step-by-step recognition model is used to determine whether the target object in the real-time image has committed a dangerous behavior based on the trained step-by-step recognition model, and to determine whether the target object is located in a safe area based on the trained human body tracking model; the step-by-step recognition model includes a target object recognition model, a super-resolution enhancement network, and at least one classification recognition model; the target object recognition model is used to recognize the target object in the real-time image to obtain the location information of the area where the target object is located, and the super-resolution enhancement network is used to enhance the target object image to obtain a super-resolution target object image; the classification recognition model is used to recognize whether the target object has committed a dangerous behavior based on the super-resolution target object image; the target object image is a low-resolution image obtained by cropping the real-time image based on the location information;

[0104] The image processing module 112 is used to select the target object in the real-time image if the target object is in a safe area and the target object has a dangerous behavior;

[0105] Interaction module 113, interaction module 113 is used to send anomaly detection results to the server so that the server pushes the anomaly detection results to the client of the manager responsible for the target object and / or abnormal behavior, wherein the anomaly detection results include the real-time image after the selection and the category of dangerous behavior.

[0106] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0107] In the above embodiments, the description of each embodiment has its own emphasis. For the part that is not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

Claims

1. An intelligent supervision method, characterized in that: Applied to an intelligent monitoring system, the system includes a client, an edge terminal and a server, and the method includes: The edge end determines whether a target object in a real-time image has a dangerous behavior based on a trained step-by-step recognition model, and determines whether the target object is located in a safe area based on a trained human tracking model; Wherein, the step-by-step recognition model includes a target object recognition model, a super-resolution enhancement network and at least one classification recognition model; the target object recognition model is used to recognize the target object in the real-time image to obtain the location information of the area where the target object is located, and the super-resolution enhancement network is used to enhance the target object image to obtain a super-resolution target object image; the classification recognition model is used to recognize whether the target object has committed a dangerous behavior based on the super-resolution target object image; the target object image is a low-resolution image obtained by cutting the real-time image based on the location information; If the target object is in a safe area and the target object has a dangerous behavior, the edge terminal selects the target object in the real-time image and sends an abnormality detection result to the server, wherein the abnormality detection result includes the real-time image after the selection and the category of the dangerous behavior; The server pushes the abnormality detection result to the client of the manager responsible for the target object and / or the dangerous behavior.

2. The method according to claim 1, characterized in that A category of dangerous behavior corresponds to a classification recognition model, and the classification recognition model is specifically used to identify whether the target object has committed the dangerous behavior corresponding to the classification recognition model.

3. The method according to claim 1, characterized in that The target object recognition model, the super-resolution enhancement network and the classification recognition model are independently trained according to sample images under different loss function constraints.

4. The method according to claim 3, characterized in that The sample image is obtained by performing data enhancement processing and expansion on the original sample.

5. The method according to claim 4, characterized in that The data enhancement processing includes one or more of the following: image cropping, image rotation, adding noise, and adding mosaic.

6. The method according to claim 1, characterized in that The classification recognition model is an Inception-ResNet network.

7. An intelligent monitoring system, characterized in that: The system includes an edge terminal, a server, and a client terminal; The edge end is used to determine whether a target object in a real-time image has performed a dangerous behavior based on a trained step-by-step recognition model, and to determine whether the target object is located in a safe area based on a trained human tracking model; Wherein, the step-by-step recognition model includes a target object recognition model, a super-resolution enhancement network and at least one classification recognition model; the target object recognition model is used to identify the target object in the real-time image to obtain the location information of the area where the target object is located, and the super-resolution enhancement network is used to enhance the target object image to obtain a super-resolution target object image; the classification recognition model is used to identify whether the target object has committed a dangerous behavior based on the super-resolution target object image; the target object image is a low-resolution image obtained by cutting the real-time image according to the location information; if the target object is in a safe area and the target object has committed a dangerous behavior, the edge end is also used to select the target object in the real-time image and send an anomaly detection result to the server, wherein the anomaly detection result includes the real-time image after the selection and the category of the dangerous behavior; The server is used to push the abnormality detection result to the client of the manager responsible for the target object and / or the dangerous behavior.

8. An edge end, characterized in that: include: A target recognition module, which is used to determine whether a target object in a real-time image has engaged in dangerous behavior based on a trained step-by-step recognition model, and to determine whether the target object is located in a safe area based on a trained human tracking model; Wherein, the step-by-step recognition model includes a target object recognition model, a super-resolution enhancement network and at least one classification recognition model; the target object recognition model is used to recognize the target object in the real-time image to obtain the location information of the area where the target object is located, and the super-resolution enhancement network is used to enhance the target object image to obtain a super-resolution target object image; the classification recognition model is used to recognize whether the target object has committed a dangerous behavior based on the super-resolution target object image; the target object image is a low-resolution image obtained by cutting the real-time image based on the location information; An image processing module, wherein if the target object is in a safe area and the target object performs dangerous behavior, the image processing module is used to select the target object in the real-time image; An interactive module is used to send an anomaly detection result to a server so that the server pushes the anomaly detection result to a client of a manager responsible for the target object and / or the dangerous behavior, wherein the anomaly detection result includes a real-time image after the selection and the category of the dangerous behavior.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the edge terminal, the method according to any one of claims 1 to 7 is implemented.

Images