A safety analysis method and device for chemical direct operation

Abstract

The application provides a safety analysis method and device for chemical direct operation, comprising: receiving a service parameter called by a user; obtaining safety analysis initialization object information of the chemical direct operation based on the service parameter; calling video data based on the initialization object information, the video data including recorded videos of high-altitude, hot work and confined space work scenes in the chemical direct operation, and preprocessing the video data; and processing the video data based on a convolutional neural network, and respectively performing object detection, pose estimation and behavior recognition to perform safety analysis work.

Description

Technical Field

[0001] This invention relates to the field of chemical safety analysis, and in particular to a method and apparatus for safety analysis of direct chemical operations. Background Technology

[0002] Intelligent video recognition technology evolved from image recognition technology. As a key technology in computer vision, it has seen numerous mature applications in fields such as road traffic, military, and industry. With the advent of convolutional neural networks, the algorithm accuracy of current intelligent video recognition technology far surpasses that of traditional algorithms. It can maintain video recognition accuracy while providing real-time video monitoring and analysis functions, providing a technological foundation for some application scenarios with high real-time requirements.

[0003] Meanwhile, in the field of chemical safety assurance, traditional video surveillance systems that rely mainly on manpower often suffer from problems such as low supervision efficiency, difficulty in performance evaluation, and insufficient objectivity due to the mechanization of duty personnel's work, long work cycles, and large individual differences.

[0004] In order to overcome the above-mentioned defects in the existing technology, there is an urgent need in the field for a safety analysis method and device for direct chemical operations, which can be used for safety analysis of direct chemical operations and improve the efficiency of on-site safety supervision of chemical enterprises. Summary of the Invention

[0005] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form to prepare for the more detailed descriptions that follow.

[0006] To overcome the aforementioned deficiencies in existing technologies, this invention provides a safety analysis method for direct chemical operations, which may include: receiving business parameters invoked by a user; obtaining safety analysis initialization object information for the direct chemical operation based on the business parameters; retrieving video data based on the initialization object information, the video data including recorded videos of high-altitude, hot work, and confined space operation scenarios in the direct chemical operation, and preprocessing the video data; and processing the video data based on a convolutional neural network to perform object detection, attitude estimation, and behavior recognition, respectively, for safety analysis.

[0007] In one embodiment, preferably, the object detection may include: extracting multiple sub-images of different sizes from the images in the video data as image features based on a convolutional neural network, deriving large-size image features from the small-size image features, feeding them back to the small-size image features, predicting the target object category in the video data through learning the image features, reducing the loss function through multiple prior boxes to fit the actual location of the target object, and calculating the probability and confidence of an object at the location.

[0008] In one embodiment, preferably, the loss function Loss is expressed by the following formula:

[0009]

[0010] Where, x i , The predicted and actual x-coordinates of the i-th grid are y and y, respectively. i , These are the predicted and actual x-coordinates of the i-th grid, respectively, ω. i , h represents the predicted width and actual horizontal width of the i-th grid, respectively. i , These are the predicted height and actual height of the i-th grid, respectively, where s is the number of grids, B is the number of preselected boxes, and λ is the height of the grid. coord and λ noobj These are all coefficients used to balance the imbalance between the proportions of negative and positive samples. This indicates that the j-th preselected box in the i-th grid does not contain the target. This indicates that the j-th preselected box in the i-th grid contains the target, c i and These represent the predicted foreground probability and the actual foreground probability, respectively. The grid consists of equal-sized blocks segmented from the original image. The confidence level is expressed as: confidence = P(obj) × IOU; where P(obj) is the probability that an object exists in the corresponding grid. To predict the intersection-union ratio (IoU) between the object's position and the actual object's position, S (T∩P) S is the area where the predicted object position intersects the actual object position; (T∪P) Let T be the area formed by the predicted object position and the actual object position, and P be the predicted object position.

[0011] In one embodiment, preferably, the pose estimation may include: extracting features from the video data based on a convolutional network to obtain a set of feature maps; using a CNN network to extract a partial confidence map (PCM) and a partial correlation field (PAF) from the feature maps, wherein the PCM is a keypoint heatmap used to characterize the location of keypoints, and the PAF is a two-dimensional vector of each limb that preserves the position and orientation information between human limb regions; and adopting a bottom-up approach to first detect each keypoint of multiple people in the video data, and then associate each keypoint with multiple individuals to achieve real-time pose detection for multiple people.

[0012] In one embodiment, preferably, the behavior recognition may include: based on a 3D convolutional neural network, extracting features from the foreground and background of the video data using different spans through a fast channel and a slow channel to obtain feature images, wherein the convolution width used in the fast channel is smaller than that in the slow channel; then concatenating the feature images obtained by convolution or segmentation through the fast channel with the feature images obtained through the slow channel to obtain a complete feature image; and classifying the feature images through a classifier to achieve real-time behavior recognition of multiple people in the video data.

[0013] In one embodiment, preferably, the step of extracting features from the foreground and background of the video data using different spans through fast and slow channels to obtain feature images may include: selecting a temporal span τ for the slow channel, wherein the value of the temporal span τ ranges from 14 to 18; the temporal span of the fast channel is τ / α, wherein the value of α ranges from 6 to 10; and configuring the convolution width ratio of the fast channel to the slow channel to be β, wherein the value of β ranges from 1 / 6 to 1 / 10.

[0014] In one embodiment, preferably, the preprocessing of the video data may include: processing the video data in a multi-threaded asynchronous manner according to a preset data processing logic; and marking old data after retrieving the video data; the security analysis method may further include: after completing the security analysis, storing the analysis result data in a database to update the video data, and periodically cleaning up the marked old data.

[0015] In one embodiment, preferably, it may further include: receiving user instructions to upload, query, and preview video data of the direct chemical operation online; and providing visual feedback of the analysis results of the safety analysis work to the user.

[0016] Another aspect of the present invention provides a safety analysis apparatus for direct chemical operations, which may include: a memory; and a processor coupled to the memory, the processor being configured to perform the safety analysis method for direct chemical operations as described in any one of the above descriptions.

[0017] The present invention also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the safety analysis method for direct chemical operations as described above. Attached Figure Description

[0018] The above-described features and advantages of the present invention will be better understood after reading the following detailed description of embodiments of the present disclosure in conjunction with the accompanying drawings. In the drawings, components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.

[0019] Figure 1 A schematic flowchart illustrating a safety analysis method for direct chemical operations according to one aspect of the present invention;

[0020] Figure 2 This is a schematic diagram illustrating the output image of an object detection model according to an embodiment of the present invention;

[0021] Figure 3 A flowchart illustrating a multi-threaded asynchronous data processing procedure according to an embodiment of the present invention;

[0022] Figure 4 A flowchart illustrating violation detection according to an embodiment of the present invention; and

[0023] Figure 5 This is a schematic diagram of the device structure of a safety analysis apparatus for direct chemical operations according to another aspect of the present invention. Detailed Implementation

[0024] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Although the description of the present invention is presented in conjunction with preferred embodiments, this does not mean that the features of the invention are limited to these embodiments. On the contrary, the purpose of describing the invention in conjunction with embodiments is to cover other options or modifications that may be derived based on the claims of the present invention. To provide a thorough understanding of the invention, many specific details will be included in the following description. The invention may also be implemented without using these details. Furthermore, to avoid confusion or obscuring the focus of the invention, some specific details will be omitted in the description.

[0025] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0026] Furthermore, the terms "upper," "lower," "left," "right," "top," "bottom," "horizontal," and "vertical" used in the following description should be understood as the orientations shown in the relevant paragraphs and accompanying drawings. These relative terms are for illustrative purposes only and do not imply that the described apparatus must be manufactured or operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0027] It is understood that although terms such as "first," "second," and "third" may be used herein to describe various components, regions, layers, and / or parts, these components, regions, layers, and / or parts should not be limited by these terms, and these terms are only used to distinguish different components, regions, layers, and / or parts. Therefore, the first components, regions, layers, and / or parts discussed below may be referred to as second components, regions, layers, and / or parts without departing from some embodiments of the present invention.

[0028] To overcome the aforementioned deficiencies in existing technologies, this invention provides a safety analysis method for direct chemical operations, which combines intelligent video recognition technology to perform safety analysis of direct chemical operations, thereby improving the efficiency of on-site safety supervision in chemical enterprises.

[0029] Figure 1 A schematic flowchart illustrating a safety analysis method for direct chemical operations according to one aspect of the present invention.

[0030] Please refer to Figure 1 The safety analysis method 100 for direct chemical operations provided by this invention may include:

[0031] Step 101: Receive the business parameters invoked by the user.

[0032] In one embodiment, preferably, the safety analysis method for direct chemical operations provided by the present invention may further include: receiving user instructions to upload, query, and preview video data of the direct chemical operations online; and providing visual feedback of the analysis results of the safety analysis to the user.

[0033] A front-end module can be built to provide users with browser page interaction and data visualization functions, supporting users in browsing system data. For example, users can access the system display page through the front-end module, obtain video information and analysis results, and also call corresponding business functions and transmit relevant parameters as needed.

[0034] Furthermore, preferably, the interaction between the user and the method for safety analysis of direct chemical operations provided by the present invention in step 101 may also include: receiving user-uploaded videos, allowing users to browse a local video file list via an interface and select specific videos to upload; responding to user commands to query videos, allowing users to query the server's existing video list in a specified area, with the video list displaying information including: video location, video name, and analysis status of each model; when there is a need for video analysis, multiple videos can be selected, parameters filled in, and models selected, and then sent in batches to the backend module for processing; providing users with online video previews, allowing users to preview existing videos in real time through a preview window; assisting users in managing image algorithm models, such as image model status monitoring and model restart; users can understand the model's operating status through model health indicator signals, and can manually restart models that are offline or unresponsive; receiving user-set parameters, such as model selection, frame interval, and job type; users can manually select the target model and set the frame interval, or use the default configuration if the frame interval is not set; and providing users with result analysis and previews, including previews of image model output results, violation alarms, and visualization of statistical data.

[0035] For example, in the video upload process, the front-end module is mainly used to define the page display area, display keywords statically, use placeholder text in text boxes, call the local system function to query the local video file list via the file selection button, and call the back-end function via the upload video button. Meanwhile, the back-end provides the front-end with video upload server functionality, parsing the user's local files into binary streams and transmitting them to the server for decoding and storage.

[0036] In one embodiment, for a video that has already been analyzed, a result query interface can be called to obtain information about the analysis results. For example, different dimensions of result information can be obtained by sequentially selecting parameters such as video, tags, and images. The lowest-dimensional image information may include: the original image, localization diagram, analysis result labels, confidence level, key nodes, etc. Users can manually judge and mark the correctness of the analysis results to improve the accuracy of subsequent analyses. In addition to these hashed analysis result information, users can also call a statistical information query interface to obtain statistical charts of the results, using the model or labels as parameters.

[0037] Please continue to refer to this. Figure 1 The safety analysis method 100 for direct chemical operations provided by this invention may further include:

[0038] Step 102: Obtain the safety analysis initialization object information for the direct chemical operation based on the business parameters. Step 103: Retrieve video data based on the initialization object information. The video data includes recorded videos of high-altitude, hot work, and confined space operation scenarios in the direct chemical operation, and preprocess the video data.

[0039] In one embodiment, steps 102 and 103 can be executed by constructing a backend module. For example, the backend module can be configured to receive the frontend business parameters, combine them with video information, construct a video analysis initialization object, process the data in a multi-threaded asynchronous manner according to a preset data processing logic, update the processed data to the database, and then transmit it to the processor to execute the processing algorithm. At the same time, a timed task module executes a loop task.

[0040] Figure 2 A flowchart illustrating multi-threaded asynchronous data processing according to an embodiment of the present invention is shown.

[0041] Can be combined Figure 2 In one embodiment, the safety analysis method for direct chemical operations provided by the present invention, which processes data in a multi-threaded asynchronous manner according to a preset data processing logic, may further include:

[0042] Step 201: Based on the number of models, construct multiple data processing queues and multiple data processing threads. The specific number is determined by the number of model categories and the workload of real-time analysis, and can be dynamically adjusted.

[0043] Step 202: Sort the video analysis initialization objects into different data processing queues according to their categories;

[0044] Step 203: The data processing thread sequentially retrieves the video analysis initialization object from the corresponding data processing queue, adjusts the state information of the video analysis initialization object, retrieves the corresponding video data from the database, and marks the old data at the same time;

[0045] Step 204: Based on the selected image processing model, package it into a network transmission object and send it to the processor to execute the model algorithm;

[0046] Step 205: Perform security analysis, and store the obtained image analysis results in the database; and

[0047] Step 206: Update the status information and video information of the video analysis initialization object, and periodically clean up the marked old data.

[0048] The process of obtaining the analysis results data can be completed with the assistance of a message middleware. The data is first stored in the message middleware, and then retrieved asynchronously by the data consumer. The image data is uploaded to the image hosting service, and the remaining data and the URL address generated by the image hosting service are stored in the database.

[0049] In one embodiment, the database mentioned in step 205 may include, for example, a video information table, a model analysis label table, an object detection analysis result table, a pose estimation analysis result table, a behavior recognition analysis result table, and a file path table.

[0050] In one embodiment, the timed cleanup of the marked old data can be performed cyclically by a timed task module, which may include cleaning up the marked old data in the database module, checking the model status, and caching statistics.

[0051] Please continue to refer to this. Figure 1 The safety analysis method 100 for direct chemical operations provided by this invention may further include:

[0052] Step 104: Process the video data based on a convolutional neural network, and perform object detection, pose estimation and behavior recognition respectively to conduct security analysis.

[0053] Further, in a preferred embodiment, the object detection may include: based on a convolutional neural network, extracting multiple sub-images of different sizes from the images in the video data as image features, deriving large-size image features from the small-size image features, and then feeding these large-size image features back to the small-size image features. For example, taking an image of size 640×640 as an example, the feature sizes of the large, medium, and small dimensions can be 80×80, 40×40, and 20×20, respectively.

[0054] Subsequently, the target object category in the video data is predicted by learning the image features, and the loss function is reduced by multiple prior boxes to fit the actual location of the target object. The probability and confidence of an object being located at the location are then calculated.

[0055] Furthermore, the loss function Loss can be expressed by the following formula:

[0056]

[0057] Where, x i , The predicted and actual x-coordinates of the i-th grid are y and y, respectively. i , These are the predicted and actual x-coordinates of the i-th grid, respectively, ω. i , h represents the predicted width and actual horizontal width of the i-th grid, respectively.i , These are the predicted height and actual height of the i-th grid, respectively, where s is the number of grids, B is the number of preselected boxes, and λ is the height of the grid. coord and λ noobj These are all coefficients used to balance the imbalance between the proportions of negative and positive samples. This indicates that the j-th preselected box in the i-th grid does not contain the target. This indicates that the j-th preselected box in the i-th grid contains the target, c i and These represent the predicted foreground probability and the actual foreground probability, respectively. The grid consists of equal-sized blocks segmented from the original image. The confidence level is expressed as: confidence = P(obj) × IOU; where P(obj) is the probability that an object exists in the corresponding grid. To predict the intersection-union ratio (IoU) between the object's position and the actual object's position, S (T∩P) S is the area where the predicted object position intersects the actual object position; (T∪P) Let T be the area formed by the predicted object position and the actual object position, and P be the predicted object position.

[0058] More specifically, the specific data involved in this object detection step may include: object category, object detection confidence, object detection frame image URL address, object detection alarm time, object detection progress percentage, horizontal and vertical coordinates of the first point of object detection, horizontal and vertical coordinates of the second point of object detection, object detection bounding box diagram, etc.

[0059] The object categories can include: people, safety helmets, gloves, masks, ordinary goggles, welding goggles, red clothing signs, green clothing signs, text blocks on vests, sparks, gas cutting / welding guns, gas cylinders, fire extinguishers, scaffolding, lifelines, work signs, warning lines, protective coverings for hot work operations, safety buckles, and manholes, etc.

[0060] In one embodiment, the object detection step may further include: receiving data processed and transmitted by the backend by the object detection unit; selecting a pre-loaded training model on the hardware GPU; analyzing and processing the video file by combining the video path and frame skipping parameters; using a Python video processing extension package to obtain information such as video size and frame rate, and calculating the total video length; adjusting the training model to half-precision mode, sacrificing a small amount of accuracy for a considerable speed improvement; then processing the video file frame by frame in a loop, skipping frames at specified positions; during the loop, converting the frame image to half-precision mode before copying it to the GPU for forward inference, and performing non-maximum suppression on the inference results. Finally, adjusting the output image size to the input image size.

[0061] Figure 3 This is a schematic diagram illustrating the output image of an object detection model according to an embodiment of the present invention. The output image of this embodiment is as follows... Figure 3 As shown.

[0062] Finally, the data can be extracted from the processing results and packaged together with the status response, description information, video name, video path, video description, video length, image name, category ID list, confidence list, coordinate list, etc., into JSON and transmitted back to the backend. Preferably, to avoid some characters being invisible during transmission, the image data format can be converted to base64.

[0063] In a preferred embodiment, the pose estimation in step 104 may include: extracting features from the video data based on a convolutional network to obtain a set of feature maps; using a CNN network to extract a Part Confidence Map (PCM) and Part Affinity Fields (PAF) from the feature maps, wherein the PCM is a keypoint heatmap used to characterize the location of keypoints, and the PAF is a two-dimensional vector of each limb that preserves the positional and orientation information between human limb regions; and using a bottom-up approach to first detect each keypoint of multiple people in the video data, and then associate each keypoint with multiple individuals to achieve real-time pose detection for multiple people.

[0064] The data involved in this pose estimation step includes, for example, pose estimation confidence, pose estimation frame image URL, pose recognition frame image corresponding time coordinates, pose recognition progress percentage, nose coordinates, neck coordinates, right shoulder coordinates, right elbow coordinates, right wrist coordinates, left shoulder coordinates, left elbow coordinates, left wrist coordinates, mid-hip coordinates, right hip coordinates, right knee coordinates, right ankle coordinates, left hip coordinates, left knee coordinates, left ankle coordinates, right eye coordinates, left eye coordinates, right ear coordinates, left ear coordinates, left big toe coordinates, left little toe coordinates, left heel coordinates, right big toe coordinates, right little toe coordinates, and right heel coordinates, etc.

[0065] More specifically, in one embodiment, the pose estimation unit can receive data from the backend, preload the trained model after selecting the hardware GPU, and then call and start the OpenPose wrapper. The wrapper, along with video parameters and frame skipping parameters, is passed to the prediction function for analysis and inference. After inference is complete, the inference result data for each frame can be encapsulated into JSON and transmitted back to the backend, with the image portion also converted to base64 format.

[0066] In a preferred embodiment, the behavior recognition in step 104 can be implemented by a behavior recognition unit, which can read the data transmitted between the default configuration and the backend, run the device specified in the configuration file, preload the model, and then load the video file into memory before calling the SlowFast inference method.

[0067] Furthermore, the reasoning process for behavior recognition in step 104 may include: based on a 3D convolutional neural network, extracting features from the foreground and background of the video data using different spans through fast and slow channels respectively to obtain feature images, wherein the convolution width used in the fast channel is smaller than that in the slow channel. Using a smaller convolution width in the fast channel reduces computational load, thus achieving real-time computation and obtaining more feature images.

[0068] Then, the feature image obtained by convolution or segmentation through the fast channel is stitched together with the feature image obtained through the slow channel to obtain a complete feature image, which is then classified by a classifier to achieve real-time behavior recognition of multiple people in the video data.

[0069] Further preferably, the step of extracting features from the foreground and background of the video data using different spans through fast and slow channels to obtain feature images may include: selecting a temporal span τ for the slow channel, wherein the value of the temporal span τ ranges from 14 to 18, for example, it can typically be set to 16. The temporal span of the fast channel is τ / α, where the value of α ranges from 6 to 10, for example, it can typically be set to 8. The convolution width ratio between the fast channel and the slow channel is configured as β, where the value of β ranges from 1 / 6 to 1 / 10, and this value is typically set to β1 / 8.

[0070] In the above process, the inference result data of each frame of image can first be processed and filtered, then converted into JSON format for standardization, then encapsulated with parameters such as response code, response description, and video information, and finally sent back to the backend for further processing.

[0071] More specifically, the data involved in this behavior recognition step may include: behavior recognition confidence score, behavior recognition frame image URL, behavior recognition progress percentage, horizontal and vertical coordinates of the first behavior recognition point, and horizontal and vertical coordinates of the second behavior recognition point. A behavior recognition rectangular frame can be constructed based on the horizontal and vertical coordinates of the first and second behavior recognition points to determine the position of the identified object in the frame image. The behavior recognition confidence score can be used to determine the behavior of the object within the behavior recognition rectangular frame.

[0072] It is readily understood that in the safety analysis method for direct chemical operations provided by this invention, many method steps require the coordinated cooperation of front-end modules, back-end modules, and processors.

[0073] For example, in one embodiment, during the image processing model setting stage, the front-end can primarily display the running status of the three algorithm models using three status flags, and use three buttons to call backend functions to restart faulty models. Simultaneously, the front-end can provide frame interval input functions for object detection and pose estimation models, a dropdown menu-style job type selection function, and the function to combine various parameters with the selected video and transmit them to the back-end. The back-end can primarily act as a medium for communication between the front-end and the processor algorithm, organizing and packaging the data from the front-end to the processor algorithm, then receiving the results returned by the algorithm and handing them over to other components for further processing. It can also call custom scripts to provide the front-end with model status information and model restart functions.

[0074] For example, in one embodiment of the present invention, the processing of the file list, the video list to be analyzed, and the video list components may also require the coordinated cooperation of the front-end module, the back-end module, and the processor. For instance, this can be achieved by setting up file list components, video list components to be analyzed, and video list components respectively. The composition of these components can be divided into a front-end part, a back-end part, and a database part.

[0075] In one embodiment, the file list component's front-end primarily uses multiple nested dropdown lists to simulate a graphical file system commonly used by users. Each video file has a rename button and three status indicators on its right. Each status indicator indicates whether the video has been analyzed by a specified model and whether analysis data is stored. A button in the upper right corner of the component refreshes the video status. Furthermore, the front-end may include a video preview area, allowing users to view the video content in real-time after selecting a video in the file list. The back-end primarily provides logical support for the interactive operations of the front-end. For query and update tasks, the back-end converts them into database language using a preset method and proxies database operations. This database may include a video information table and a file path table. The nested logic of the dropdown lists mainly relies on data in the file path table, and the rename operation also modifies the file path table. The video format and status indicators are taken from the video information table.

[0076] For the video list component to be analyzed, in one embodiment, its front-end part is mainly used to provide the function of displaying video objects in the analysis process, as well as the control of the start and technology of the analysis process; while the back-end part is mainly responsible for the overall process management of video processing, following the preset data processing logic to process data in a multi-threaded asynchronous manner, updating the unanalyzed video information, and feeding it back to the front-end part for real-time display on the page.

[0077] For the analyzed video list component, in one embodiment, the upper part of its front-end can summarize and display the analyzed videos, with the right side of the video displaying the status indicators of the model analysis, for example, green indicating analysis completed and no alarms, and yellow indicating analysis completed but alarms present. The lower part of the front-end can display the result data of the selected analyzed videos, distinguished by multiple tag types. The right side of each type displays the total number of occurrences, the total number of frames, and the maximum number of occurrences per frame. Clicking the small arrow to the left of a tag allows viewing the specific details of each tag type; for example, colored frames represent alarm frames, and colorless frames represent non-alarm frames. Administrators can manually view alarm details and add manual judgment to the analysis results. The back-end mainly analyzes whether the end of the video result triggers an alarm, which can be referenced in conjunction with [reference / reference / etc.]. Figure 4 .

[0078] Figure 4 A flowchart illustrating violation detection according to an embodiment of the present invention.

[0079] like Figure 4 As shown, the main data sources for the analysis are the results obtained from three major algorithm models: object detection, pose estimation, and behavior recognition. Combined with the introduced standard object size information, the relative positions and distances between different objects in the frame image can be calculated. By combining human behavior and joint data, it can be determined whether someone is using an object to perform a certain action. Figure 4 In the embodiment shown, the rule pool can formulate rules for the behavior of target objects and people based on the safety production management specifications for direct chemical operations and combined with scenario information. These rules can mainly include, for example: (1) whether the safety items specified in the specifications are missing in a certain scenario; (2) whether there are any dangerous factors that pose a hazard to a certain scenario; (3) whether a person wears safety equipment in compliance with regulations in a certain scenario; (4) whether a person uses equipment in compliance with regulations, etc.

[0080] The database mentioned above can include object detection results data, pose estimation results data, and behavior recognition results data. Taking object detection results data as an example, the table structure in the database can be as shown in Table 1.

[0081] Table 1. Structure of Object Detection Results

[0082]

[0083]

[0084] In one embodiment, the data statistics function can be implemented by setting up a statistical data component, which can also be divided into a front-end part and a back-end part. The front-end part can provide users with the ability to view statistical data with different labels through drop-down boxes, provide users with the ability to query data through text boxes, and provide users with the ability to update cached data through different buttons. The back-end part can mainly cache data from the hard drive to memory for the front-end part to call at any time, thereby improving operating efficiency. In addition, a scheduled task unit is also run, whose functions include: cleaning up old data that has been marked and periodically caching statistical data.

[0085] In summary, the chemical direct operation safety analysis method provided by this invention offers users browser page interaction and data visualization capabilities, supporting users in browsing system data. Through data interaction, transmission, and processing, combined with three major algorithm models—object detection, attitude estimation, and behavior recognition—the system is provided with the ability to intelligently process video files, outputting diverse information for safety analysis. This allows for multi-dimensional analysis of chemical operation site video files, while also incorporating chemical direct operation safety production management standards, thereby improving the accuracy and efficiency of chemical direct operation safety analysis.

[0086] Although the methods described above are illustrated and depicted as a series of actions for the sake of simplicity, it should be understood and appreciated that these methods are not limited by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but which may be understood by those skilled in the art, according to one or more embodiments.

[0087] Figure 5 This is a schematic diagram of the device structure of a safety analysis apparatus for direct chemical operations according to another aspect of the present invention.

[0088] According to another aspect of the invention, an embodiment of a safety analysis device 500 for direct chemical operations is also provided herein.

[0089] like Figure 5 As shown, the safety analysis device 500 for direct chemical operations provided in this embodiment may include a memory 501 and a processor 502 coupled to the memory 501. The processor 502 may be configured to implement any of the above-described safety analysis methods for direct chemical operations.

[0090] According to another aspect of the invention, an embodiment of a computer storage medium is also provided herein.

[0091] The computer storage medium contains a computer program. When executed by a processor, this computer program can implement the steps of any of the aforementioned safety analysis methods for direct chemical operations.

[0092] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0093] The processors described herein can be implemented using electronic hardware, computer software, or any combination thereof. Whether such processors are implemented as hardware or software will depend on the specific application and the overall design constraints imposed on the system. As an example, the processors, any portion thereof, or any combination thereof presented in this disclosure can be implemented using microprocessors, microcontrollers, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuitry, and other suitable processing components configured to perform the various functions described throughout this disclosure. The functionality of the processors, any portion thereof, or any combination thereof presented in this disclosure can be implemented using software executed by a microprocessor, microcontroller, DSP, or other suitable platform.

[0094] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read and write information to / from the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0095] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A safety analysis method for chemical direct operation, comprising: receiving a service parameter invoked by a user; obtaining safety analysis initialization object information of the chemical direct operation based on the service parameter; calling video data based on the initialization object information, the video data including recorded videos of high-altitude, hot work and confined space operation scenes in the chemical direct operation, and preprocessing the video data; and processing the video data based on a convolutional neural network, and respectively performing object detection, pose estimation and behavior recognition to perform safety analysis work; wherein the object detection comprises: based on a convolutional neural network, extracting multiple sizes of sub-images in images in the video data as image features, deriving large size image features from small size image features, feeding back to the small size image features, predicting target object categories in the video data through learning of the image features, fitting the actual positions of the target objects through a plurality of prior boxes to reduce a loss function, and calculating the probability and confidence of having objects in the positions; the pose estimation comprises: based on a convolutional network, extracting features of the video data to obtain a group of feature maps, using a CNN network to extract a part confidence map (PCM) and a part association field (PAF) of the feature maps, the part confidence map (PCM) being a key point heat map for representing key point positions, and the part association field (PAF) being a two-dimensional vector of each limb that maintains the position and direction information between the limb regions; and in a bottom-up manner, detecting each key point of multiple people in the video data, and associating the key points with multiple individuals to realize real-time pose detection of the multiple people; the behavior recognition comprises: based on a 3D convolutional neural network, using different spans to extract features for the foreground and background of the video data through a fast channel and a slow channel to obtain feature images, wherein the convolution width used by the fast channel is smaller than that of the slow channel, then splicing the feature images after convolution or cutting through the fast channel with the feature images through the slow channel to obtain complete feature images, and classifying through a classifier to realize real-time behavior recognition of multiple people in the video data. The loss function Loss is represented by the following formula:

2. The security analysis method of claim 1, wherein, using different spans to extract features for the foreground and background of the video data through a fast channel and a slow channel to obtain feature images, comprising: wherein, , are the predicted and actual horizontal coordinates of the i-th grid, respectively, , are the predicted and actual horizontal coordinates of the i-th grid, respectively, , are the predicted and actual horizontal width of the i-th grid, respectively, , are the predicted and actual height of the i-th grid, respectively, s is the number of grids, and B is the number of bounding boxes, and are coefficients for balancing the imbalance between the negative and positive samples, = 1 indicates that the j-th bounding box of the i-th grid does not contain the target, = 1 indicates that the j-th bounding box of the i-th grid contains the target, and represent the predicted and actual foreground probabilities, respectively, and the grid is a block of equal size divided from the original picture, The confidence is expressed as: ; wherein P(obj) is a probability of the corresponding grid having an object, IOU is an intersection over union of a predicted object position and a real object position, is an intersection area of the predicted object position and the real object position; is an union area of the predicted object position and the real object position, T is the real object position, and P is the predicted object position.

3. The security analysis method of claim 1, wherein, selecting a time span τ for the slow channel, the value range of the time span τ being 14-18, and the time span of the fast channel being τ / α, wherein the value range of α is 6-10; and configuring the convolution width ratio of the fast channel and the slow channel as β, and the value range of β being 1 / 6-1 / 10. The preprocessing of the video data comprises:

4. The security analysis method of claim 1, wherein, processing the video data in a multi-thread asynchronous manner according to a preset data processing logic; and after calling the video data, marking old data; the safety analysis method further comprises: after completing the safety analysis work, storing analysis result data into a database to update the video data, and cleaning the marked old data at regular intervals. Further comprising:

5. The security analysis method of claim 1, wherein, ​ receiving user instruction to upload, query and online preview the video data of the chemical direct operation; and visualizing feedback of analysis result of the safety analysis work to the user. 6.A safety analysis device for chemical direct operation, comprising: a memory; and a processor coupled with the memory, configured to execute the safety analysis method for chemical direct operation according to any one of claims 1-5.

7. A computer readable medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the safety analysis method for chemical direct operation according to any one of claims 1-5. The computer program is executed by the processor to implement the safety analysis method for chemical direct operation according to any one of claims 1-5.

Images