Safety helmet identification method and system based on video streaming technology

Abstract

The invention relates to the technical field of computer vision, and discloses a safety helmet identification method and system based on a video streaming technology, and the method comprises the steps: receiving real-time video streaming data, and transmitting and controlling the video streaming data through a streaming media transmission protocol; identifying the safety helmet body and the operator, and fusing and analyzing an identification result to judge whether the operator wears the safety helmet or not; and performing real-time alarm on a person who does not wear a safety helmet, recording and storing images and video evidences of the person who does not wear the safety helmet, and realizing intelligent management and establishing an operation and maintenance system according to an identification result. According to the method, negligence and errors possibly caused by manual monitoring are reduced through automatic identification and alarm processes, and the supervision efficiency is improved. Image / video evidences of people who do not wear safety helmets are recorded and stored, and valuable data support is provided for accident investigation, safety audit and compliance inspection.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a helmet recognition method and system based on video stream technology. Background Art

[0003] Although the current intelligent video analysis technology has certain limitations in environmental adaptability, with the development of disciplines such as image processing, image analysis, and computer vision, the introduction of many excellent algorithms will make video analysis more intelligent. Intelligent video surveillance will develop towards adapting to more complex and changeable scenarios, towards identifying and analyzing more behaviors and abnormal events, towards lower costs, towards true "scene-based content analysis", and towards early warning and prevention.

[0004] In order to solve the problem of wearing safety helmets in power stations, the traditional method of using dedicated personnel to check is that on the one hand, there are a large number of videos that are not checked by anyone, and there is a great deal of missed detection; on the other hand, due to human factors, it is impossible to concentrate on staring at the monitoring screen for a long time. This project proposes a solution based on intelligent video analysis technology, which is automatically identified by the computer. Once an abnormality is found, it will automatically alarm, which greatly reduces the workload of people and improves work efficiency. Summary of the invention

[0005] In view of the above existing problems, the present invention is proposed.

[0006] Therefore, the present invention provides a helmet recognition method based on video streaming technology, which can

[0007] By real-time monitoring and identifying whether workers are wearing helmets, it can effectively remind workers to comply with safety regulations, enhance their safety awareness, and promptly detect the behavior of not wearing helmets, thereby preventing potential safety accidents.

[0008] In order to solve the above technical problems, the present invention provides the following technical solutions: a helmet recognition method based on video streaming technology, comprising: receiving real-time video streaming data, transmitting and controlling the video streaming data through a streaming media transmission protocol; identifying the helmet body and the operator, and fusing and analyzing the recognition results to determine whether the operator is wearing a helmet; issuing real-time alarms to personnel who are not wearing helmets, recording and saving images and video evidence of personnel who are not wearing helmets, and realizing intelligent management and establishing an operation and maintenance system based on the recognition results.

[0009] As a preferred solution of the helmet recognition method based on video stream technology described in the present invention, the receiving of real-time video stream data includes establishing an RTSP session and obtaining the video stream through the RTSP protocol.

[0010] As a preferred solution of the helmet recognition method based on video stream technology described in the present invention, the transmission and control of video stream data includes using the RTP protocol to perform real-time transmission of media data and the RTCP protocol to provide network congestion information.

[0011] As a preferred solution of the helmet recognition method based on video stream technology described in the present invention, the step of identifying the helmet body and the operator includes: the step of using a convolutional neural network to recognize the helmet body includes training a convolutional neural network model to recognize the helmet from the preprocessed image. The step of using a color recognition algorithm to recognize the color of the helmet includes using a SLIC algorithm to generate superpixels and analyze color information in the Lab color space.

[0012] As a preferred solution of the helmet recognition method based on video stream technology described in the present invention, the recognition of the helmet body and the operator also includes calculating the coordinates of human joint points through a human posture recognition algorithm.

[0013] As a preferred solution of the helmet recognition method based on video stream technology described in the present invention, the fusion analysis recognition result includes logically combining the helmet body recognition result, the human body recognition result and the color recognition result.

[0014] As a preferred solution of the helmet recognition method based on video streaming technology described in the present invention, the real-time alarm includes sending an alarm signal when a person not wearing a helmet is detected, and storing the alarm event and related image data in a database for subsequent query and analysis.

[0015] Another object of the present invention is to provide a safety helmet recognition system based on video streaming technology, which can meet the safety supervision requirements in environments such as construction sites.

[0016] As a preferred solution of the helmet recognition system based on video stream technology described in the present invention, it includes: a video stream receiving module, an image preprocessing module, a helmet recognition and analysis module, an alarm and recording module, and an intelligent management and operation and maintenance module;

[0017] The video stream receiving module is responsible for establishing an RTSP session, receiving real-time video stream data through the RTSP protocol, and using the RTP protocol to perform real-time transmission of media data, while using the RTCP protocol to monitor network congestion to ensure stable reception of video streams;

[0018] The image preprocessing module preprocesses the received video frames to provide higher quality image data for subsequent image recognition;

[0019] The helmet recognition and analysis module uses a convolutional neural network model to recognize the helmet body of the preprocessed image; uses a color recognition algorithm to analyze the color of the helmet; calculates the coordinates of the human joint points through a human posture recognition algorithm to determine whether the operator is wearing a helmet, and integrates these recognition results for analysis;

[0020] The alarm and recording module sends an alarm signal in real time when a person not wearing a helmet is detected, and stores the alarm event, related images and video evidence in a database for subsequent query and analysis;

[0021] The intelligent management and operation and maintenance module implements intelligent management based on the identification results, including but not limited to statistical analysis, report generation, and abnormal event tracking, and establishes a complete operation and maintenance system to ensure the long-term stable operation and continuous optimization of the system.

[0022] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of a helmet recognition method based on video stream technology are implemented.

[0023] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of a helmet recognition method based on video stream technology.

[0024] Beneficial effects of the present invention: This method can monitor in real time and immediately warn workers who are not wearing helmets, ensuring that potential safety hazards are dealt with in a timely manner. The system records images and video evidence of workers not wearing helmets, which facilitates the subsequent analysis of the cause of the accident and the tracing of responsibility. Automated helmet recognition reduces the need for manual monitoring, reduces management costs, and improves management efficiency. By analyzing the collected data and generating statistical reports, managers can understand the overall situation of helmet wearing and formulate more effective safety measures. By transmitting video streams over the network, managers can remotely monitor multiple work sites and achieve centralized management. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0026] Figure 1 A schematic flow chart of a method for identifying a helmet based on video streaming technology is provided in accordance with an embodiment of the present invention.

[0027] Figure 2 A SLIC algorithm convergence diagram of a helmet recognition method based on video stream technology provided by an embodiment of the present invention.

[0028] Figure 3 A schematic diagram of working modules of a helmet recognition system based on video streaming technology provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0030] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0031] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0032] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0033] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0034] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0035] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a helmet recognition method based on video stream technology, comprising:

[0036] S1: Receive real-time video stream data, transmit and control the video stream data through the streaming media transmission protocol.

[0037] Furthermore, an RTSP session is established and the video stream is obtained through the RTSP protocol. The client connects to the streaming server and sends an RTSP description command (DESCRIBE). The streaming server provides feedback through an SDP description, which includes information such as the number of streams and media types. The client then analyzes the SDP description and sends an RTSP setup command (SETUP) for each stream in the session. The RTSP setup command tells the server the port that the client uses to receive media data. After the streaming connection is established, the client sends a play command (PLAY), and the server starts to transmit the media stream (RTP packet) to the client on UDP. During the playback process, the client can also send commands to the server to control fast forward, fast rewind, and pause. Finally, the client can send a termination command (TERADOWN) to end the streaming session.

[0038] S2: Identify the helmet and the operator, and integrate and analyze the identification results to determine whether the operator is wearing a helmet.

[0039] Furthermore, in order to more accurately identify the behavior and color of people wearing helmets, the recognition logic is mainly divided into three directions: helmet body recognition, human body recognition, and color recognition. Only by integrating the three recognition algorithms can we accurately determine whether a person is wearing a helmet and the person's identity.

[0040] The step of using convolutional neural network to identify the helmet entity includes training a convolutional neural network model to identify the helmet from the preprocessed image.

[0041] An improved convolutional neural network (CNN) structure is adopted, named "EHDN". In EHDN, a new convolutional layer is introduced, called "multi-scale feature fusion layer (MSFF)", which is used to extract features of different scales, initialize the weights of EHDN, and set hyperparameters such as learning rate, batch size, and number of iterations. The loss function is used for training, and the multi-scale features of the image are extracted through the MSFF layer:

[0042]

[0043] Among them, F MSFF is the fused feature, is the feature map of the i-th scale, ω i is the corresponding weight, and N is the number of scales.

[0044] Use an improved activation function to improve the discrimination of features:

[0045] f(x)=tanh(α·x)+max(0,β·x)

[0046] Among them, α and β are learnable parameters used to adjust the shape of the activation function.

[0047] It should be noted that it can identify workers wearing helmets, detect helmets in real time, issue real-time warnings for unqualified personnel, realize intelligent management, effectively identify people wearing helmets and those not wearing helmets in real time, and can capture people not wearing helmets in real time, record and store them, and save on-site picture / video evidence. This algorithm can recognize in real time and has strong adaptability to different environments such as light and cloudy days. It is not affected by the occlusion of people's glasses, beards, hairstyles, expressions, etc., and is not affected by different postures of people such as front, back, side, running, and bowing. The helmet recognition algorithm mainly performs intelligent recognition of red, yellow, blue, white, and orange helmets, and conducts all-weather intelligent analysis without the need for personnel on duty or manual operation.

[0048] Furthermore, human posture recognition involves algorithms such as human joint recognition, human tracking, and action recognition, and action recognition is performed based on the human posture of video frames within a certain period of time.

[0049] Human joint recognition is based on the openpose algorithm, which calculates the coordinates of 25 human joints by predicting the heatmap and paf of the human body. The algorithm principle is:

[0050] (1) Human body tracking

[0051] Since action recognition is based on human posture information in a time series, it is necessary to track each detected pedestrian and perform feature extraction. The human tracking algorithm selected is the deepsort algorithm, which can quickly track multiple people with high accuracy.

[0052] (2) Action Recognition

[0053] Action recognition is based on the coordinates of human joints within a time period. The steps are:

[0054] A. Set the time interval (e.g. T=5);

[0055] B. Extract the coordinates of the joint points of the same person in T frame images, with a size of T*25*2, and normalize the coordinates based on the neck coordinates;

[0056] C. Calculate the motion state of all nodes in two adjacent frames and obtain a feature vector with a size of (T-1)*25*2;

[0057] D. The dimension of the merged feature vector is (2T-1)*25*2, and the classification model is trained based on the feature vector.

[0058] During detection, in order to avoid interference caused by the environment such as light, people walking, obstructions, temporary cable ducts, temporary foreign objects, etc., edge detection can be used to eliminate interference. The principle is to use edge detection to first identify the positioning device, and then identify it within the device area.

[0059] Furthermore, the color recognition algorithm is implemented through SLIC. The specific implementation steps of SLIC are as follows:

[0060] a Initialize seed points (cluster centers): Distribute seed points evenly in the image according to the set number of superpixels. Assuming that the image has a total of N pixels, which are pre-divided into K superpixels of the same size, then the size of each superpixel is N / K, and the distance (step length) between adjacent seed points is approximately S = sqrt(N / K).

[0061] b. Reselect the seed point in the n*n neighborhood of the seed point (generally n=3). The specific method is: calculate the gradient values ​​of all pixels in the neighborhood and move the seed point to the place with the smallest gradient in the neighborhood. The purpose of this is to avoid the seed point falling on the contour boundary with a large gradient, so as not to affect the subsequent clustering effect.

[0062] c assigns a class label to each pixel in the neighborhood around each seed point (i.e., which cluster center it belongs to). Unlike the standard k-means search in the entire image, SLIC limits the search range to 2S*2S, which can accelerate the convergence of the algorithm, such as Figure 2 As shown: the expected superpixel size is S*S, but the search range is 2S*2S.

[0063] d distance metric. For each pixel found, the distance between it and the seed point is calculated. The distance calculation method is as follows:

[0064]

[0065] Where dlab represents the color distance, dxy represents the spatial distance, S = sqrt(N / K), and m is a fixed constant used to adjust the superpixel density. Its value range is [1,40], and it is generally 10. Ds is the final distance.

[0066] The above steps are iterated until the error converges (the cluster center of each pixel no longer changes).

[0067] After training and testing on multiple mainstream classification models such as VGGNet, GoogLeNet, and Inception series, combined with the speed and accuracy of the model, we finally adopted the optimized InceptionV1 network model for classification. Compared with the multiple Inception modules of the standard InceptionV1 network, we optimized it to 3 Inception modules, called InceptionV1-OnFire, which greatly reduced the model complexity and improved the recognition speed while ensuring the accuracy of flame classification and recognition.

[0068] S3: Issue real-time alerts to personnel who are not wearing helmets, record and save image and video evidence of personnel who are not wearing helmets, implement intelligent management based on the recognition results, and establish an operation and maintenance system.

[0069] Furthermore, the preliminary recognition results from the helmet body recognition, human body recognition and color recognition modules are received. The preprocessed data is input into the logic combination function. The helmet body recognition result (H), human body recognition result (P) and color recognition result (C) are three independent outputs, and the three results are integrated through the logic combination function:

[0070] LCF(H,P,C)=f(H)·g(P)·h(C)

[0071] The system calls sub-functions to process the helmet, human body and color recognition results respectively. The sub-functions are defined as follows:

[0072]

[0073] Among them, α', β', γ' are weight coefficients, θ H ,θ P ,θ C is the preset threshold

[0074] The system calculates the output of each sub-function, adjusts the confidence level according to the weight coefficient and threshold, and multiplies the outputs of the three sub-functions to obtain the final logical combination result.

[0075] When the output of LCF(H,P,C) is lower than a certain alarm threshold θ alert When the alarm is triggered, the real-time alarm is triggered. After the alarm is triggered, the alarm signal (AS) is generated and sent through the function:

[0076]

[0077] Among them, SEND(M) is a function for sending a message, and M is a message containing alarm information.

[0078] Embodiment 2, the second embodiment of the present invention, is different from the previous embodiment in that:

[0079] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0080] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0081] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0082] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0083] Example 3, reference Figure 3 , which is an embodiment of the present invention, provides a helmet recognition system based on video stream technology, characterized in that it includes a video stream receiving module 1, an image preprocessing module 2, a helmet recognition and analysis module 3, an alarm and recording module 4, and an intelligent management and operation and maintenance module 5;

[0084] The video stream receiving module 1 is responsible for establishing an RTSP session, receiving real-time video stream data through the RTSP protocol, and using the RTP protocol for real-time transmission of media data. At the same time, it uses the RTCP protocol to monitor network congestion to ensure stable reception of video streams.

[0085] Image preprocessing module 2, preprocesses the received video frames to provide higher quality image data for subsequent image recognition;

[0086] Safety helmet recognition and analysis module 3 uses a convolutional neural network model to recognize the safety helmet body of the preprocessed image; uses a color recognition algorithm to analyze the color of the safety helmet; calculates the coordinates of the human joint points through a human posture recognition algorithm to determine whether the operator is wearing a safety helmet, and integrates these recognition results for analysis;

[0087] The alarm and recording module 4 sends an alarm signal in real time when a person not wearing a helmet is detected, and stores the alarm event, related images and video evidence in the database for subsequent query and analysis;

[0088] Intelligent management and operation and maintenance module 5 implements intelligent management based on the identification results, including but not limited to statistical analysis, report generation, abnormal event tracking, and establishes a complete operation and maintenance system to ensure the long-term stable operation and continuous optimization of the system.

[0089] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A helmet recognition method based on video streaming technology, characterized in that: include, Receive real-time video stream data, and transmit and control the video stream data through streaming media transmission protocol; Identify the helmet and the operator, and integrate and analyze the identification results to determine whether the operator is wearing a helmet; Provide real-time alerts to people who are not wearing helmets, record and save image and video evidence of people who are not wearing helmets, implement intelligent management and establish an operation and maintenance system based on the recognition results.

2. A method for identifying helmets based on video streaming technology as claimed in claim 1, characterized in that: The receiving of real-time video stream data includes establishing an RTSP session and acquiring the video stream through the RTSP protocol.

3. A method for identifying a helmet based on video streaming technology as claimed in claim 2, characterized in that: The transmission and control of video stream data includes using the RTP protocol to perform real-time transmission of media data and the RTCP protocol to provide network congestion information.

4. A method for identifying a helmet based on video streaming technology as claimed in claim 3, characterized in that: The step of identifying the helmet body and the operator includes: using a convolutional neural network to identify the helmet body, including training a convolutional neural network model to identify the helmet from the preprocessed image. The step of identifying the color of the helmet using a color recognition algorithm includes using a SLIC algorithm to generate superpixels and analyze color information in the Lab color space.

5. A method for identifying a helmet based on video streaming technology as claimed in claim 4, characterized in that: The identification of the helmet body and the operator also includes calculating the coordinates of human joint points through a human posture recognition algorithm.

6. A method for identifying a helmet based on video streaming technology as claimed in claim 5, characterized in that: The fusion analysis recognition result includes logically combining the helmet body recognition result, the human body recognition result and the color recognition result.

7. A method for identifying a helmet based on video streaming technology as claimed in claim 6, characterized in that: The real-time alarm includes sending an alarm signal when a person not wearing a helmet is detected, and storing the alarm event and related image data in a database for subsequent query and analysis.

8. A system using the method for identifying helmets based on video stream technology as claimed in any one of claims 1 to 7, characterized in that: It includes video stream receiving module, image preprocessing module, helmet recognition and analysis module, alarm and recording module, intelligent management and operation and maintenance module; The video stream receiving module is responsible for establishing an RTSP session, receiving real-time video stream data through the RTSP protocol, and using the RTP protocol to perform real-time transmission of media data, while using the RTCP protocol to monitor network congestion to ensure stable reception of video streams; The image preprocessing module preprocesses the received video frames to provide higher quality image data for subsequent image recognition; The helmet recognition and analysis module uses a convolutional neural network model to recognize the helmet body of the preprocessed image; uses a color recognition algorithm to analyze the color of the helmet; calculates the coordinates of the human joint points through a human posture recognition algorithm to determine whether the operator is wearing a helmet, and integrates these recognition results for analysis; The alarm and recording module sends an alarm signal in real time when a person not wearing a helmet is detected, and stores the alarm event, related images and video evidence in a database for subsequent query and analysis; The intelligent management and operation and maintenance module implements intelligent management based on the identification results, including but not limited to statistical analysis, report generation, and abnormal event tracking, and establishes a complete operation and maintenance system to ensure the long-term stable operation and continuous optimization of the system.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images