A smart production safety supervision system based on ARM platform AI application deployment

Abstract

The present invention relates to a smart safety production supervision system based on ARM platform AI application deployment, which belongs to the field of AI application deployment technology. Among them, the system includes an AI application module, a model transplantation module, a model call module, a front-end device acquisition module, and a terminal supervision module; the AI ​​application module trains and exports the constructed edge AI application model through a deep learning framework to obtain a model file; the model transplantation module converts the model file into a data format instruction file of an embedded system platform, and quantifies the model; the model call module manages the AI ​​application model throughout its life cycle, deploys the model as an online service, and performs model call verification; the front-end device acquisition module collects images of the production site, and outputs the collected images to the edge computing server in the form of signals; the terminal supervision module calls and deploys the edge AI application model, performs model reasoning calculation based on the edge computing server, and sends the calculation results to the terminal supervision platform.

Description

Technical Field

[0001] The present invention belongs to the technical field of AI application deployment, and specifically relates to an intelligent production safety supervision system based on ARM platform AI application deployment. Background Art

[0002] The factory production environment is often unable to carry out necessary safety management and control in a timely manner due to the imperfect safety infrastructure. At present, the main control measures during the safety management period are for the safety manager to conduct inspections based on safety regulations and systems. When facing the safety supervision of the work site, such measures are affected by objective factors such as production scale, factory construction environment, and limited working space. Inspections are difficult, consume a lot of manpower and material resources, and have low safety management efficiency. Some factories fail to operate according to the standardized system, lack the operation and management of safety production factors, separate standardization creation from operation, and the factory operators have high mobility and weak safety awareness, resulting in many hidden dangers at the work site and great difficulty in safety management.

[0003] At present, illegal area intrusion detection is extremely important in safety management and accident prevention. However, safety inspections of safety behaviors and factory environments are mostly implemented by people in practice. It is difficult to meet management needs in terms of real-time, accuracy, and reliability. More real-time, accurate, and intelligent means are needed to solve them. Traditional computer vision and sensor technology have strong limitations in actual field applications. The recognition efficiency is low in complex scenes with multiple people. The safety supervision system based on this generally lacks judgment and analysis of triggering events and lacks intelligence. Therefore, the efficiency and quality of safety production management need to be further improved. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a smart production safety supervision system based on ARM platform AI application deployment.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] A safe production intelligent supervision system based on ARM platform AI application deployment, including: AI application module, model transplantation module, model calling module, front-end equipment acquisition module, terminal supervision module;

[0007] The AI ​​application module is used to train the constructed edge AI application model through a deep learning framework to export a model file;

[0008] The model transplantation module is used to convert the model file into a data format instruction file of the embedded system platform through a neural network exchange format, and quantize the edge AI application model;

[0009] The model calling module is used to manage the edge AI application model throughout its life cycle, deploy the edge AI application model as an online service, and perform model calling verification;

[0010] The front-end device acquisition module is used to collect images of the production site through a monitoring camera. Under the control of the video timing, the collected images are filtered and amplified and then output to the edge computing server in the form of signals;

[0011] The terminal supervision module is used to call and deploy the edge AI application model according to the security supervision category of the front-end device acquisition module, perform model inference calculation based on the edge computing server and send the calculation results to the terminal supervision platform.

[0012] Specifically, the edge AI application models include regional intelligent intrusion detection model, personnel fall recognition model, and safety rope target detection model;

[0013] The regional intelligent intrusion detection model obtains an intrusion detection image after performing image preprocessing on the production site detection image through the monitoring camera production site detection image;

[0014] A safety protection zone is set according to the location area of ​​the production site, and an edge feature map is obtained by detecting the edge based on the approximate gradient amplitude of the difference between two adjacent pixels in the diagonal direction of the gray matrix of the initial image of the background of the detection area. The edge feature map is morphologically processed to remove residual noise to obtain an anti-interference detection image, and the edge feature points in the anti-interference detection image are mapped to a polar coordinate system through parameter space transformation. A polar coordinate pixel point accumulation threshold is preset. If the accumulated value of the coordinate point in the polar coordinate system exceeds the polar coordinate pixel point accumulation threshold, the straight line direction of the coordinate point in the polar coordinate system corresponding to the rectangular coordinate system is extracted, and the edge feature points in the straight line direction are connected to obtain a regional detection map of the safety protection zone;

[0015] According to the initial background image of the detection area, the intrusion detection image is adaptively threshold segmented to obtain a target image and a background image. After target detection is performed on the target image, a detection frame target is obtained, and the background image is used to update the initial background image of the detection area. The detected detection frame targets are synthesized into moving targets, and intrusion judgment is performed based on the moving target and the regional detection map to obtain a regional intrusion result.

[0016] Specifically, the personnel fall recognition model performs high-precision fall detection on the real-time video stream collected by the surveillance camera, and the safety rope target detection model is used to identify the specific position information of the human body and the safety rope vest in the real-time video stream.

[0017] Specifically, the full life cycle management is an end-to-end model service process, including project launch management, feature engineering pipeline, model application experiment, and automated machine learning workflow pipeline; the project launch management is used to locate the data source of the initial data analysis of the model; the feature engineering pipeline is used to define the application code and orchestration components for continuous deployment of integrated components, perform data cleaning tasks through data conversion rules and cleaning rules, execute feature calculation tasks, improve feature engineering rules based on calculation result feedback and load them into the feature storage system; the model application experiment is used to create training data sets and test data sets, trigger model training through the training data sets, interactively test and verify model parameters according to the test set, export the model after the iterative training stops, and submit the code to the repository; the automated machine learning workflow pipeline is used to extract model versions, perform automatic model evaluation and iterative hyperparameter updates, and push the trained model to the model registry.

[0018] Specifically, it is characterized in that the quantization processing method is: converting the trained single-precision floating-point model into a fixed-point model with a low bit width according to the quantization range and the bit width.

[0019] Specifically, the edge computing server uses dual storage units for data backup, controls data reception and storage based on the FPGA core controller, receives data signals through the blocking chip and transmits them to the corresponding internal control unit, caches data through the internal FIFO, waits for the FPGA to control the FIFP read operation, and sends the cached data to the dual storage units.

[0020] Specifically, the safety supervision categories include factory environment safety, factory environment space site safety, and employee operation behavior; the factory environment safety category remotely monitors the factory work site and main entrances and exits through the regional intelligent intrusion detection model; the factory environment space site safety category automatically identifies safety hazards through the personnel fall recognition model; the employee operation behavior category monitors whether workers are wearing safety ropes correctly through the safety rope target detection model, promptly detects violations and issues alarms.

[0021] Specifically, the edge AI application model packages the application and its dependencies into a portable container for deployment through the Docker containerization platform.

[0022] The beneficial effects of the present invention are:

[0023] Through sensors, monitoring equipment and AI models, various safety risk factors in the production environment are monitored in real time, and a closed-loop management mechanism from hidden danger discovery, early warning, processing to review is established to ensure that each hidden danger can be thoroughly resolved and recorded in the system to provide a basis for subsequent accident tracing and experience summary; remote monitoring of factory operation sites and main entrances and exits, automatic identification of safety hazards and personnel violations through intelligent regional intrusion models; based on the ARM platform, intelligent transformation of existing monitoring equipment is carried out, and AI application algorithms are optimized in real time according to actual application scenarios. A multi-level safe and intelligent production supervision system based on the Internet of Things is constructed, providing a modular platform, promoting function expansion, deployment and personalized customization, realizing data integration and sharing, and improving the generalization ability of AI application models. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0025] Figure 1 This is a structural block diagram of an intelligent production safety supervision system based on ARM platform AI application deployment of the present invention. DETAILED DESCRIPTION

[0026] The example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein: on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art. In addition, the described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to give a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid blurring various aspects of the present disclosure. The blocks shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices. The flowcharts shown in the accompanying drawings are only exemplary illustrations and do not necessarily include all the content and operations / steps, nor do they necessarily have to be executed in the order described. For example, some operations / steps may be decomposed, while some operations / steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0027] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0028] See also Figure 1 , a smart safety production supervision system based on ARM platform AI application deployment, including: AI application module, model transplantation module, model calling module, front-end equipment acquisition module, terminal supervision module;

[0029] The AI ​​application module is used to train the constructed edge AI application model through a deep learning framework to export a model file;

[0030] The model transplantation module is used to convert the model file into a data format instruction file of the embedded system platform through a neural network exchange format, and quantize the edge AI application model;

[0031] The model calling module is used to manage the edge AI application model throughout its life cycle, deploy the edge AI application model as an online service, and perform model calling verification;

[0032] The front-end device acquisition module is used to collect images of the production site through a monitoring camera. Under the control of the video timing, the collected images are filtered and amplified and then output to the edge computing server in the form of signals;

[0033] The terminal supervision module is used to call and deploy the edge AI application model according to the security supervision category of the front-end device acquisition module, perform model inference calculation based on the edge computing server and send the calculation results to the terminal supervision platform.

[0034] Specifically, the edge AI application models include regional intelligent intrusion detection model, personnel fall recognition model, and safety rope target detection model;

[0035] The regional intelligent intrusion detection model obtains an intrusion detection image after performing image preprocessing on the production site detection image through the monitoring camera production site detection image;

[0036] A safety protection zone is set according to the location area of ​​the production site, and an edge feature map is obtained by detecting the edge based on the approximate gradient amplitude of the difference between two adjacent pixels in the diagonal direction of the gray matrix of the initial image of the background of the detection area. The edge feature map is morphologically processed to remove residual noise to obtain an anti-interference detection image, and the edge feature points in the anti-interference detection image are mapped to a polar coordinate system through parameter space transformation. A polar coordinate pixel point accumulation threshold is preset. If the accumulated value of the coordinate point in the polar coordinate system exceeds the polar coordinate pixel point accumulation threshold, the straight line direction of the coordinate point in the polar coordinate system corresponding to the rectangular coordinate system is extracted, and the edge feature points in the straight line direction are connected to obtain a regional detection map of the safety protection zone;

[0037] According to the initial background image of the detection area, the intrusion detection image is adaptively threshold segmented to obtain a target image and a background image. After target detection is performed on the target image, a detection frame target is obtained, and the background image is used to update the initial background image of the detection area. The detected detection frame targets are synthesized into moving targets, and intrusion judgment is performed based on the moving target and the regional detection map to obtain a regional intrusion result.

[0038] In this embodiment, the input image is subjected to image preprocessing such as grayscale transformation, smoothing filtering, and image enhancement; the background modeling is completed using the video input image after mean filtering of the first ten frames; the edge of the input image is detected using the Prewitt edge detection operator, and morphological processing and cumulative probability Hough transform are performed on it, and finally the detected straight line is linearly fitted to delineate the safety protection area; the image to be detected and the background image are differentially operated, the maximum inter-class variance method is used to achieve threshold segmentation, and multi-frame decision fusion is used to complete the adaptive segmentation of the safety protection area; the color features of the HSV color space and the LBP texture features are collaboratively used to achieve shadow removal, and the detection results are morphologically processed to further eliminate the influence of noise; the background image is randomly updated according to the processed detection results and the detection results are output.

[0039] Specifically, the personnel fall recognition model performs high-precision fall detection through the real-time video stream collected by the monitoring camera, and the safety rope target detection model is used to identify the specific position information of the human body and the safety rope vest in the real-time video stream.

[0040] In this embodiment, the AI ​​application collects the uploaded video files through the front-end device acquisition module for fall detection. The uploaded video will be analyzed frame by frame by the system to identify and mark the fall in each frame of the video. The AI ​​application model uses PyTorch as the main framework for deep learning, and uses the Streamlit framework to quickly develop interactive web applications, so that users of the terminal supervision platform can easily upload pictures and videos and perform fall detection of real-time cameras. At the same time, it supports switching different model files to meet different detection needs.

[0041] Specifically, the full life cycle management is an end-to-end model service process, including project launch management, feature engineering pipeline, model application experiment, and automated machine learning workflow pipeline; the project launch management is used to locate the data source of the initial data analysis of the model; the feature engineering pipeline is used to define the application code and orchestration components for continuous deployment of integrated components, perform data cleaning tasks through data conversion rules and cleaning rules, execute feature calculation tasks, improve feature engineering rules based on calculation result feedback and load them into the feature storage system; the model application experiment is used to create training data sets and test data sets, trigger model training through the training data sets, interactively test and verify model parameters according to the test set, export the model after the iterative training stops, and submit the code to the repository; the automated machine learning workflow pipeline is used to extract model versions, perform automatic model evaluation and iterative hyperparameter updates, and push the trained model to the model registry.

[0042] Specifically, it is characterized in that the quantization processing method is: converting the trained single-precision floating-point model into a fixed-point model with a low bit width according to the quantization range and the bit width.

[0043] Specifically, the edge computing server uses dual storage units for data backup, controls data reception and storage based on the FPGA core controller, receives data signals through the blocking chip and transmits them to the corresponding internal control unit, caches data through the internal FIFO, waits for the FPGA to control the FIFP read operation, and sends the cached data to the dual storage units.

[0044] Specifically, the safety supervision categories include factory environment safety, factory environment space site safety, and employee operation behavior; the factory environment safety category remotely monitors the factory work site and main entrances and exits through the regional intelligent intrusion detection model; the factory environment space site safety category automatically identifies safety hazards through the personnel fall recognition model; the employee operation behavior category monitors whether workers are wearing safety ropes correctly through the safety rope target detection model, promptly detects violations and issues alarms.

[0045] Specifically, the edge AI application model packages the application and its dependencies into a portable container for deployment through the Docker containerization platform.

[0046] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0047] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0048] The program code included on the computer readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or their combinations, and the programming language includes object-oriented programming languages-such as Java, Smalltalk, C++, and also includes conventional procedural programming languages-such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0049] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A smart production safety supervision system based on ARM platform AI application deployment, characterized in that: include: AI application module, model transplantation module, model calling module, front-end equipment acquisition module, terminal supervision module; The AI ​​application module is used to train the constructed edge AI application model through a deep learning framework to export a model file; The edge AI application model includes a regional intelligent intrusion detection model, a personnel fall recognition model, and a safety rope target detection model; the regional intelligent intrusion detection model uses a surveillance camera to detect images on the production site, and obtains an intrusion detection image after image preprocessing of the production site detection image; a safety protection zone is set according to the location area of ​​the production site, and the edge is detected by approximating the gradient amplitude of the difference between two adjacent pixels in the diagonal direction of the grayscale matrix of the initial image of the background of the detection area to obtain an edge feature map, and the edge feature map is morphologically processed to remove residual noise to obtain an anti-interference detection image, and the edge feature points in the anti-interference detection image are mapped to the polar coordinate system through parameter space transformation, and the polar coordinates are preset. Pixel point accumulation threshold, if the accumulation value of the coordinate point in the polar coordinate system exceeds the polar coordinate pixel point accumulation threshold, then extract the straight line direction of the coordinate point in the polar coordinate system corresponding to the rectangular coordinate system, connect the edge feature points in the straight line direction to obtain the regional detection map of the safety protection zone; perform adaptive threshold segmentation on the intrusion detection image according to the initial background image of the detection area to obtain the target image and the background image, perform target detection on the target image to obtain the detection frame target, and use the background image to update the initial background image of the detection area, synthesize the detected detection frame target into a moving target, and perform intrusion judgment based on the moving target and the regional detection map to obtain the regional intrusion result; The model transplantation module is used to convert the model file into a data format instruction file of the embedded system platform through a neural network exchange format, and quantize the edge AI application model; The model calling module is used to manage the edge AI application model throughout its life cycle, deploy the edge AI application model as an online service, and perform model calling verification; The full lifecycle management is an end-to-end model service process, including project launch management, feature engineering pipeline, model application experiment, and automated machine learning workflow pipeline; the project launch management is used to locate the data source of the initial data analysis of the model; the feature engineering pipeline is used to define the application code and orchestration components for continuous deployment of integrated components, perform data cleaning tasks through data conversion rules and cleaning rules, execute feature calculation tasks, improve feature engineering rules based on calculation result feedback and load them into the feature storage system; The model application experiment is used to create a training data set and a test data set, trigger model training through the training data set, test and verify the model parameters in an interactive manner according to the test set, export the model after the iterative training stops, and submit the code to the repository; the automated machine learning workflow pipeline is used to extract the model version, perform automatic model evaluation and iterative hyperparameter update, and push the trained model to the model registry; The front-end device acquisition module is used to collect images of the production site through a monitoring camera. Under the control of the video timing, the collected images are filtered and amplified and then output to the edge computing server in the form of signals; The terminal supervision module is used to call and deploy the edge AI application model according to the security supervision category of the front-end device acquisition module, perform model inference calculation based on the edge computing server and send the calculation results to the terminal supervision platform.

2. The system according to claim 1, characterized in that The personnel fall recognition model performs high-precision fall detection through the real-time video stream collected by the monitoring camera, and the safety rope target detection model is used to identify the specific position information of the human body and the safety rope vest in the real-time video stream.

3. The system according to claim 1, characterized in that The quantization processing method is: converting the trained single-precision floating-point model into a fixed-point model with low bit width according to the quantization range and bit width.

4. The system according to claim 1, characterized in that The edge computing server uses dual storage units for data backup, controls data reception and storage based on the FPGA core controller, receives data signals through the blocking chip and transmits them to the corresponding internal control unit, caches data through the internal FIFO, waits for the FPGA to control the FIFP read operation, and sends the cached data to the dual storage units.

5. The system according to claim 1, characterized in that The safety supervision categories include factory environment safety, factory environment space site safety, and employee operation behavior; the factory environment safety category remotely monitors the factory operation site and main entrances and exits through a regional intelligent intrusion detection model; the factory environment space site safety category automatically identifies safety hazards through a personnel fall recognition model; the employee operation behavior category monitors whether workers are wearing safety ropes correctly through a safety rope target detection model, promptly detects violations and issues alarms.

6. The system according to claim 1, characterized in that The edge AI application model packages the application and its dependencies into a portable container for deployment through the Docker containerization platform.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the intelligent safety production supervision system based on ARM platform AI application deployment as described in any one of claims 1-6.

8. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions, when executed by a computer processor, are used to execute the intelligent production safety supervision system based on ARM platform AI application deployment as described in any one of claims 1-6.

Images