Method and device for identifying hidden violations in explosion-related videos

Abstract

This invention discloses a method and apparatus for identifying potential hazards and violations in explosive-related videos, relating to the field of blasting construction. The method includes: S1 constructing a video authenticity detection model, a video target recognition model, and an identity recognition model; S2 constructing a labeled database and a facial information database; S3 training and optimizing the video target recognition model and the identity recognition model; S4 acquiring explosive-related videos; S5 performing authenticity analysis on the explosive-related videos, identifying relevant information about blasting operations, and identifying information about blasting operators; S6 analyzing potential hazards and violations in the blasting videos. The explosive-related videos are analyzed using the video authenticity detection model, the optimized video target recognition model, and the optimized identity recognition model to determine the authenticity of the videos, relevant operational information during the blasting process, and information about the operators. Based on these analysis results, it is determined whether there are safety hazards in the operational process, thus standardizing the operations of the personnel.

Description

Technical Field

[0001] This invention relates to the field of blasting construction, and in particular to a method and device for identifying potential hazards and violations in videos involving explosives. Background Technology

[0002] In the field of safety supervision in petroleum exploration technology, ensuring the work quality, procedural standardization, and personal safety of blasters and construction workers during civil blasting operations is a technical challenge in this field.

[0003] Current technologies mostly involve construction workers wearing information collection terminals, performing standardized operations, and recording the entire work process. The video data is then submitted to an information collection center daily for human evaluation to determine if the construction process is up to standard. For example, checks might be made on whether safety helmets or anti-static clothing were worn, or whether blasters' certifications were verified. During this evaluation process, staff need to review a large number of videos daily, and visual fatigue can lead to missed checks and underreporting. Human judgment is subjective, and data verification cannot strictly adhere to established standards. Furthermore, traditional technologies waste significant human, material, and financial resources. Summary of the Invention

[0004] The purpose of this invention is to design a method and device for identifying potential hazards and violations in explosive-related videos in order to solve the above-mentioned problems.

[0005] The present invention achieves the above objectives through the following technical solutions:

[0006] Methods for identifying potential hazards and violations in videos involving explosives include:

[0007] S1. Construct a video authenticity detection model, a video target recognition model, and an identity recognition model. The video target recognition model is a YOLOv5 detection model, and the identity recognition model is a face recognition model.

[0008] S2. Construct a label database and a face information database. Label the training dataset and export the label files to form the label database. The face information database stores relevant information of certified civil explosives operators.

[0009] S3. The video target recognition model is trained and optimized using a labeled database, and the identity recognition model is trained and optimized using a face information database.

[0010] S4. Obtain the explosive-related video to be analyzed;

[0011] S5. Import the explosive-related videos to be analyzed into the video authenticity detection model, the optimized video target recognition model, and the optimized identity recognition model to perform authenticity analysis on the explosive-related videos, identify relevant information about blasting operations, and identify information about blasting operators.

[0012] S6. Analyze the authenticity of explosive videos, identify relevant information about blasting operations and the information of blasting operators, and analyze the potential hazards and violations in the blasting videos.

[0013] The device for identifying potential hazards and violations in videos involving explosives includes an underlying server, which comprises:

[0014] Storage; storage is used to store computer programs;

[0015] Processor; The processor is used to execute computer programs, and when the processor executes the computer programs, it implements the steps of the above-described method for identifying potential hazards and violations in explosive videos.

[0016] The beneficial effects of this invention are as follows: by analyzing explosive-related videos through a video authenticity detection model, an optimized video target recognition model, and an optimized identity recognition model, the authenticity of the explosive-related videos, relevant operational information during the blasting process, and information on the operators performing the blasting are analyzed and judged. Based on these analysis results, it is determined whether there are any safety hazards in the process operation, standardizes the operation of staff, and achieves efficient operation, accurate judgment, and intelligent analysis, making the management and control of safety issues more efficient and saving a lot of manpower, material resources, and financial resources. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the video target recognition model in this invention;

[0018] Figure 2 This is a flowchart illustrating the process of analyzing the cleaning degree of explosive-related videos in this invention;

[0019] Figure 3 This is a flowchart of the identity recognition model in this invention;

[0020] Figure 4 This is a flowchart of the video authenticity detection model in this invention;

[0021] Figure 5 This invention is an RNN text recognition framework;

[0022] Figure 6 This invention relates to a single-layer bidirectional RNN network. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0024] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0025] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0026] In the description of this invention, it should be understood that the terms "upper," "lower," "inner," "outer," "left," "right," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used to facilitate the description of this invention and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0027] Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0028] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, terms such as "set" and "connection" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0029] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0030] Methods for identifying potential hazards and violations in videos involving explosives include:

[0031] S1. Construct a video authenticity detection model, a video target recognition model, and an identity recognition model. The video target recognition model is a YOLOv5 detection model, and the identity recognition model is a face recognition model.

[0032] S2. Construct a labeled database and a facial information database. Label the training dataset and export the labeled files to form the labeled database. The facial information database stores relevant information of certified civil explosives operators.

[0033] S3. The video target recognition model is trained and optimized using a labeled database, and the identity recognition model is trained and optimized using a face information database.

[0034] S4. Obtain the explosive-related video to be analyzed.

[0035] S0. Analyze and evaluate the image clarity in the explosive-related videos to be analyzed, and filter and eliminate those that are not analyzed; for example... Figure 2 As shown, it specifically includes:

[0036] S01. Perform a Fourier transform on the image of the explosive-related video to be analyzed, converting it to the frequency domain. N is the length of the image sequence, f(i,j) represents an N×N matrix, where i = 0, 1, 2, ..., N-1 and j = 0, 1, 2, ..., N-1, and F(k,l) denotes the Fourier transform of f(i,j).

[0037] S02, Remove low-frequency signals below the preset frequency;

[0038] S03. Use Fast Fourier Transform to convert the image from the frequency domain to the spatial domain.

[0039] In the above formula, f(a,b) represents a matrix of size N×N, where a = 0, 1, 2, ..., N-1 and b = 0, 1, 2, ..., N-1, and f(a,b) represents the inverse Fourier transform of F(k,l).

[0040] Where F(k,l) is the average pixel value in the frequency domain, P(k,b) is the frequency domain form, N is the sequence length, and b = 0, 1, 2, ..., N-1.

[0041] S04. Calculate the mean amplitude in the spatial domain. Images with a mean amplitude greater than a preset threshold are clear images, while those with a mean amplitude less than a preset threshold are blurry images. Blurry images are then discarded.

[0042] S5. Import the explosive-related videos to be analyzed into the video authenticity detection model, the optimized video target recognition model, and the optimized identity recognition model to perform authenticity analysis on the explosive-related videos, identify relevant information about blasting operations, and identify information about blasting operators.

[0043] S6. Analyze the authenticity of explosive videos, identify relevant information about blasting operations and the information of blasting operators, and analyze the potential hazards and violations in the blasting videos.

[0044] By analyzing explosive-related videos using video authenticity detection models, optimized video target recognition models, and optimized identity recognition models, the system can determine the authenticity of the videos, relevant operational information during the blasting process, and information about the personnel performing the blasting. Based on these analysis results, it can determine whether there are any safety hazards in the process, standardize the operation of staff, and achieve efficient operation, accurate judgment, and intelligent analysis, making the management of safety issues more efficient and saving a lot of manpower, material resources, and financial resources.

[0045] The complex conditions of field video recording result in inconsistent video quality. Therefore, before analyzing explosive-related videos, it is essential to first analyze and evaluate the image clarity to quickly filter out blurry videos from a large number of videos and improve the efficiency of re-inspection. Blur reduces image clarity, seriously affecting image quality and leading to difficulties or even failures in image analysis, processing, and reception. Therefore, it is necessary to use effective fuzz evaluation methods to control the use of blurry images, thereby improving the overall performance of the system.

[0046] The device for identifying potential hazards and violations in videos involving explosives includes an underlying server, which comprises:

[0047] Storage; storage is used to store computer programs;

[0048] Processor; The processor is used to execute computer programs, and when the processor executes the computer programs, it implements the steps of the above-described method for identifying potential hazards and violations in explosive videos.

[0049] The hazard and violation identification device also includes a data acquisition device, which is used to collect blasting videos and related information about blasting personnel. The data acquisition device communicates with the underlying server.

[0050] The hidden danger and violation identification device also includes a cloud server and a remote terminal. The signal terminal of the cloud server is connected to the signal terminal of the remote terminal and the signal terminal of the underlying server.

[0051] The video object recognition model is the YOLOv5 detection model, such as... Figure 1 As shown, the object detection uses the first-order object detection algorithm—YOLOv5. This method has good compatibility with the PyTorch framework, making it easy to train on your own datasets. The PyTorch framework is also easier to deploy in production. Not only is the environment easy to configure, but model training is also very fast, and batch inference produces real-time results. It can perform effective inference directly on single images, batches of images, videos, and even webcam input. Finally, YOLOv5's detection speed is extremely fast, enabling the acquisition of evaluation results for a large number of assignment videos in a short time.

[0052] YOLOv5 is a single-stage object detection algorithm. This algorithm incorporates several new improvements, resulting in significant performance enhancements in both speed and accuracy. The main improvements are as follows:

[0053] Input end: During the model training phase, some improvement ideas were proposed, mainly including Mosaic data augmentation, adaptive anchor box calculation, and adaptive image scaling;

[0054] Baseline network: It integrates some new ideas from other detection algorithms, mainly including: Focus structure and CSP structure;

[0055] Neck network: Object detection networks often insert some layers between the Backbone and the final Head output layer. YOLOv5 added the FPN+PAN structure.

[0056] Head Output Layer: The anchor box mechanism of the output layer is the same as that of YOLOv4. The main improvements are the loss function GIOU_Loss during training and the DIOU_nms for selecting predicted boxes.

[0057] The identity recognition model is a face recognition model, and the process of face recognition is as follows: Figure 3 As shown, face detection: using a face detection model (object detection model or other detection model) to find the location of all faces in the image (the result is a detection box), and then cutting out the face portion of the image;

[0058] Face alignment: Correcting faces using a face alignment model, based on facial key points, and aligning with the key points of a standard face;

[0059] Face encoding: Encoding face images using deep learning models to extract facial features;

[0060] Identity verification: Comparing facial features with data in a facial database to determine the identity of the person.

[0061] FaceNet is a general-purpose face recognition system: it uses deep convolutional neural networks (CNNs) to learn how to map images to Euclidean space. Spatial distance is directly related to image similarity: different images of the same person have small spatial distances, while images of different people have larger spatial distances, which can be used for face verification, recognition, and clustering.

[0062] FaceNet uses deep neural networks to extract features and triplet loss to measure the distance error between samples during training. Before training or during online learning, it continuously presents "difficulties" to the neural network, constantly searching for the "most unlike itself" among the samples while simultaneously searching for the "most like others." Through stochastic gradient descent, it continuously reduces the differences between itself and all its own samples while maximizing the differences with others, ultimately reaching an optimal point. This embedding learning further trains the output layer of the original feature extraction network, thereby improving the representation of features.

[0063] For the entire FaceNet architecture, an extremely deep network, Inception ResNet-v2, is used for feature extraction. The model structure consists of three Inception modules with residual connections and one Inception v4 module.

[0064] The overall framework of the model is basically the same as other classic deep learning methods. The feature extraction part is also based on CNN, except that it uses the Inception-v4 deep network followed by a feature normalization layer, which makes the features in a second normal form.

[0065] This approach maps all image features onto a hypersphere, thus mitigating differences caused by the imaging environment of the samples. Finally, triplet loss is used as the loss function, along with stochastic gradient descent (SDG) for backpropagation. The Inception model also connects to the residuals, which is one of the key features of this method, improving training convergence speed.

[0066] The video authenticity detection model is an OCR text recognition model, such as... Figure 4 As shown, the video authenticity detection model in this invention mainly consists of two parts:

[0067] Video authenticity verification: OCR text recognition technology is used to check whether the watermark time in the lower right corner of the video shooting area is consistent with the working time of the day, to prevent the appearance of spliced ​​videos.

[0068] Nighttime construction identification: By using videos taken on the day of the operation, the date of the watermark on the video can be extracted to determine whether workers are carrying out construction work at night, thus preventing this phenomenon and strengthening safety management.

[0069] The video authenticity detection model is based on the main framework of the RNN text recognition algorithm, such as... Figure 5 As shown,

[0070] (1) The window size in Max pooling is 1*2, which ensures that the proposed features have horizontal length, which is beneficial for recognizing longer texts;

[0071] (2) Training CNN+RNN is relatively difficult, so BatchNorm was added to help the model converge;

[0072] Advantages:

[0073] (1) End-to-end training is possible;

[0074] (2) No character segmentation and horizontal scaling are required; only vertical scaling to a fixed length is needed. At the same time, it can recognize sequences of any length.

[0075] (3) It can train dictionary-based models and arbitrary models that are not dictionary-based;

[0076] (4) The training speed is fast and the model is very small.

[0077] The entire CRNN network can be divided into three parts: Convolutional Layers ----- Here, the convolutional layer is a regular CNN network used to extract the convolutional feature maps of the input image, that is, to convert the image into a convolutional feature matrix;

[0078] Recurrent Layers – This recurrent network layer is a deep bidirectional LSTM network that further extracts text sequence features based on convolutional features. A deep RNN network refers to an RNN network with more than two layers. For a single-layer bidirectional RNN network, the structure is as follows: Figure 6 As shown,

[0079] The second type of stack-shaped deep bidirectional structure was used in CRNN.

[0080] Transcription Layers – This layer converts the RNN output into a softmax function to produce a character output.

[0081] This invention addresses safety management issues in geophysical exploration processes, such as: blasting personnel working without proper certification, not wearing safety helmets and anti-static clothing, and nighttime detonation and blasting. Through model detection, it determines whether safety hazards exist in the process and standardizes the operations of personnel.

[0082] By integrating the underlying algorithms and the front end, it achieves efficient operation, accurate judgment, and intelligent analysis, making the management of security issues more efficient and saving a lot of manpower, material resources, and financial resources.

[0083] It integrates target detection technology, OCR text recognition technology, face recognition technology, video blur determination technology, and distributed information processing technology. It can assess the standardization of explosive video operations, process the video through underlying algorithms, and generate analysis results through the front-end functional platform, which are then saved to the back-end and transmitted to regulatory authorities or data collection centers.

[0084] The technical solutions of the present invention are not limited to the specific embodiments described above. Any technical modifications made in accordance with the technical solutions of the present invention fall within the protection scope of the present invention.

Claims

1. A method for hazard violation identification for explosive video, characterized in that, include: S1. Construct a video authenticity detection model, a video target recognition model, and an identity recognition model. The video target recognition model is a YOLOv5 detection model, and the identity recognition model is a face recognition model. The YOLOv5 detection model includes an input end, a baseline network, a Neck network, and a Head output layer from input to output. The baseline network includes a Focus structure and a CSP structure. S2. Construct a label database and a face information database. Label the training dataset and export the label files to form the label database. The face information database stores relevant information of certified civil explosives operators. S3. The video target recognition model is trained and optimized using a labeled database, and the identity recognition model is trained and optimized using a face information database. S4. Obtain the explosive-related video to be analyzed; S0. Analyze and evaluate the image clarity in the explosive-related videos to be analyzed, and filter and eliminate those that are not. Specifically, this includes: S01, Fourier transform the image of the blast video to be analyzed into the frequency domain; ; N is the length of the image sequence, f(i,j) represents a matrix of size N x N, where i = 0, 1, 2, ···, N-1 and j = 0, 1, 2, ···, N-1, F(k,l) represents the Fourier transform of f(i,j); S02, Remove low-frequency signals below the preset frequency; S03. converting the image from the frequency domain to the spatial domain using a fast Fourier transform; In the above equation, f(a,b) represents a matrix of size N x N, where a = 0, 1, 2, ···, N-1 and b = 0, 1, 2, ···, N-1, and f(a,b) denotes the inverse Fourier transform of F(k,l). where F(k, l) is the frequency domain pixel mean, P(k, b) is the frequency domain form, and N is the sequence length, b = 0, 1, 2, ···, N - 1. S04. Calculate the mean amplitude in the spatial domain, and determine whether the image is a blurry image based on the mean amplitude, and remove blurry images; specifically: images with a mean amplitude greater than a preset threshold are clear images, and those with a mean amplitude less than a preset threshold are blurry images. S5. The explosive-related videos to be analyzed are imported into the video authenticity detection model, the optimized video target recognition model, and the optimized identity recognition model. This process analyzes the authenticity of the explosive-related videos, identifies information related to the blasting operation, and identifies the information of the blasting personnel. The identity recognition model performs face recognition as follows: It uses a face detection model to locate all faces in the image and cuts out the face portion of the image; it uses a face alignment model to correct the faces, aligning them with the key points of a standard face; it uses a deep learning model to encode the face images and extract facial features; and it compares the facial features with data in the face database to determine the identity of the face, thus achieving face recognition. S6. Analyze the authenticity of explosive videos, identify relevant information about blasting operations and the information of blasting operators, and analyze the potential hazards and violations in the blasting videos.

2. A device for identifying hidden violations in explosive video, characterized in that, Including the underlying servers, which include: Storage; storage is used to store computer programs; Processor; the processor is used to execute a computer program, and when the processor executes the computer program, it implements the steps of the method for identifying potential violations in explosive video as described in claim 1.

3. The device for identifying hidden violations in explosive video according to claim 2, characterized in that, The hazard and violation identification device also includes a data acquisition device, which is used to collect blasting videos and related information about blasting personnel. The data acquisition device communicates with the underlying server.

4. The device for identifying hidden violations in explosive video according to claim 2, characterized in that, The hidden danger and violation identification device also includes a cloud server and a remote terminal. The signal terminal of the cloud server is connected to the signal terminal of the remote terminal and the signal terminal of the underlying server.

Images