A coal mine intelligent video recognition method

Abstract

The present invention relates to the technical field of coal mine video recognition, and in particular to a method for intelligent video recognition in coal mines. The method comprises: step S1, reading video frames frame by frame in a video stream; step S2, performing a first-class mark and a second-class mark on the currently read video frames; step S3, performing posture recognition on the portraits in the video frames that have completed the first-class mark; and step S4, performing behavior recognition on the portraits in the video frames that have completed the second-class mark. The present invention performs real-time intelligent analysis on video data to identify unsafe behaviors of underground coal mine workers, formulates different recognition methods for different area types, and analyzes whether there are people falling down and distinguishes abnormal behaviors of walking, standing, and walking in areas without hurdles through posture recognition. For areas with hurdles, analyzes whether there are unsafe behaviors of people climbing over railings through behavior recognition, and issues warning prompts in time to improve the quality and intelligence of video recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of coal mine video recognition, and in particular to a coal mine intelligent video recognition method. Background Art

[0002] The traditional coal mine industrial television monitoring system mainly completes the functions of on-site real-time image acquisition, transmission, storage, and display. With the continuous advancement of coal mine intelligent construction, intelligent video has been increasingly used in coal mine safety supervision and scheduling. In addition to the functions of traditional video monitoring systems, intelligent video monitoring systems also have image processing and analysis, alarm triggering, and alarm linkage control. It relies on AI artificial intelligence real-time analysis to achieve full intelligent operation. Through multi-dimensional and multi-angle video recognition, analysis, and statistics, it can timely discover hidden dangers and remind relevant personnel to ensure the implementation of coal mine safety production systems and reduce coal mine safety accidents. Since employees' unsafe behavior is the direct cause of accidents, there is an urgent need for an intelligent recognition and early warning system for miners' unsafe behavior.

[0003] Chinese Patent Publication No.: CN109272496B discloses a fire image recognition method for coal mine fire video monitoring, comprising the following steps: 1. constructing flame pixel comparison sample data and non-flame pixel comparison sample data and storing them in a monitoring computer; 2. image acquisition and transmission: a camera arranged at a coal mine fire monitoring point captures images of its surroundings and sends the captured coal mine fire monitoring images to the monitoring computer; 3. image enhancement: the monitoring computer calls an image enhancement processing module and sequentially uses an improved unsharp masking method and a histogram equalization method to perform image enhancement processing on the coal mine fire monitoring image; 4. fire image recognition: the monitoring computer calls a fire image recognition module and uses a KNN algorithm to analyze and process the coal mine fire monitoring image after image enhancement processing to determine whether it is a fire image; it can be seen that the existing intelligent video safety recognition technology lacks unsafe behaviors of miners and is difficult to capture and distinguish different behavioral characteristics. Summary of the invention

[0004] To this end, the present invention provides a coal mine intelligent video recognition method to overcome the problem in the prior art that due to the diversity of miners' unsafe behaviors, it is difficult to capture and distinguish different behavioral characteristics, resulting in low video recognition quality.

[0005] To achieve the above object, the present invention provides a coal mine intelligent video recognition method, comprising:

[0006] Obtain the video stream uploaded by each camera unit, and perform first-class marking and second-class marking on the currently read video frame according to the position target in the video stream;

[0007] Marking joint points of the portrait in the video stream with numbers;

[0008] Performing posture recognition on the portrait in the video frame that has completed the first type of marking, calculating the real-time angle according to the joint point number, and when it is determined that the real-time angle is within the second standard angle range, determining the real-time tilt angle according to the standard tilt angle, and comparing the real-time distance with the standard distance, so as to perform the first posture marking or the second posture marking on the video frame;

[0009] When it is determined that the real-time angle is not within the second standard angle range, and the real-time angle is less than the standard angle, a first warning prompt is issued; when it is determined that the real-time angle is not within the second standard angle range, the real-time angle is greater than or equal to the standard angle, and the real-time center of gravity movement distance is greater than the standard center of gravity movement distance, a real-time trajectory route is generated, the real-time similarity is calculated, and the real-time similarity is compared with the standard similarity to perform a third posture mark or a second posture mark on the video frame;

[0010] Behavior recognition is performed on the human figures in the video frames that have completed the second category of marking, the real-time longitudinal distance is calculated according to the joint point numbers, and the real-time longitudinal distance is compared with the standard longitudinal distance to analyze whether there are any personnel violations and issue a second warning prompt.

[0011] Further, a video frame with a class of tags is obtained, and a posture recognition is performed on a portrait in any of the video frames, a real-time angle formed by a real-time posture line and a horizontal frame line of the video frame is obtained, and the real-time angle is compared with a second standard angle range.

[0012] If the real-time angle is within the second standard angle range, the real-time tilt angle will be determined according to the standard tilt angle to analyze the posture;

[0013] If the real-time angle is not within the second standard angle range, the real-time angle is compared with the first standard angle to analyze whether to issue a first warning prompt.

[0014] Further, when it is determined that the real-time angle is not within the second standard angle range, the real-time angle is compared with the first standard angle.

[0015] If the real-time angle is smaller than the standard angle, it is determined that a person has fallen to the ground, and a first warning prompt is issued;

[0016] If the real-time angle is greater than or equal to the standard angle, obtain the currently read video frame and the next video frame to mark the portrait contour to calculate the real-time center of gravity movement distance, and judge the real-time center of gravity movement distance according to the standard center of gravity movement distance to determine whether to check and process the camera mechanism.

[0017] Further, when it is determined that the real-time angle is not within the second standard angle range and the real-time angle is greater than or equal to the standard angle, the currently read video frame and the next video frame are obtained to mark the portrait contour, and the real-time distance between the two center of gravity positions is calculated according to the center of gravity positions of the portrait contours in the two video frames, recorded as the real-time center of gravity moving distance, and the real-time center of gravity moving distance is determined according to the standard center of gravity moving distance.

[0018] If the real-time center of gravity moving distance is less than or equal to the standard center of gravity moving distance, continue to obtain the currently read video frame and the next video frame to mark the portrait contour, calculate the real-time center of gravity moving distance, and judge the real-time center of gravity moving distance according to the standard center of gravity moving distance. If it is determined again that the real-time center of gravity moving distance is less than or equal to the standard center of gravity moving distance, then check the camera mechanism;

[0019] If the real-time center of gravity moving distance is greater than the standard center of gravity moving distance, a real-time trajectory route will be generated, the real-time similarity will be calculated, and the real-time similarity will be compared with the standard similarity to perform a third posture mark or a second posture mark on the video frame.

[0020] Furthermore, when it is determined that the real-time center of gravity movement distance is greater than the standard center of gravity movement distance, continuous video frames are continuously acquired with a preset acquisition time to mark the portrait contour, and the center of gravity position of the portrait contour in each video frame is recorded as a waypoint. The center of gravity position of the portrait contour in the currently read video frame is used as the starting point, and the straight lines passing through the waypoints are connected frame by frame to generate a real-time trajectory route.

[0021] Further, the real-time trajectory route is obtained, the similarity between the real-time trajectory route and the standard trajectory route is calculated to obtain the real-time similarity, and the real-time similarity is compared with the standard similarity.

[0022] If the real-time similarity is less than the standard similarity, marking the video frame with a third posture;

[0023] If the real-time similarity is greater than or equal to the standard similarity, a second posture mark is performed on the video frame.

[0024] Further, when the real-time angle is determined to be within the second standard angle range, a real-time ray formed by the joint points corresponding to the first preset number and the second preset number is obtained, a real-time tilt angle formed by the real-time ray and the horizontal frame line of the video frame is calculated, and the real-time tilt angle is determined according to the standard tilt angle.

[0025] If the real-time tilt angle is less than the standard tilt angle, marking the video frame with a first posture;

[0026] If the real-time tilt angle is greater than or equal to the standard tilt angle, the real-time distance is compared with the standard distance to perform a first posture mark or a second posture mark on the video frame.

[0027] Furthermore, when it is determined that the real-time tilt angle is greater than or equal to the standard tilt angle, the real-time distance between the joint points corresponding to the first preset number and the second preset number is obtained, and the real-time distance is compared with the standard distance. If it is determined that the real-time distance is less than the standard distance, the video frame is marked with a first posture.

[0028] Furthermore, when it is determined that the real-time tilt angle is greater than or equal to the standard tilt angle, the real-time distance between the joint points corresponding to the first preset number and the second preset number is obtained, and the real-time distance is compared with the standard distance. If it is determined that the real-time distance is greater than or equal to the standard distance, the video frame is marked with a second posture.

[0029] Further, the video frames with the second category of markings are obtained, and the behavior recognition is performed on the portrait in any of the video frames, a rectangular coordinate system is established with the lower horizontal frame line of the video frame as the horizontal axis and any vertex of the lower horizontal frame line as the origin, the vertical coordinates of the joint points corresponding to the first preset number and the fourth preset number are obtained, and the difference between the two vertical coordinates is calculated to obtain the real-time longitudinal distance, and the real-time longitudinal distance is compared with the standard longitudinal distance.

[0030] If the real-time longitudinal distance is less than the standard longitudinal distance, no second warning prompt is given, and behavior recognition continues for the next video frame;

[0031] If the real-time longitudinal distance is greater than or equal to the standard longitudinal distance, it is determined that a person has violated the rules and a second warning prompt is issued.

[0032] Compared with the prior art, the beneficial effect of the present invention lies in that the video files of the camera devices set in various areas of the coal mine are acquired through the control assembly, and the unsafe behaviors of the workers in the coal mine are identified by real-time intelligent analysis of the intelligent data. According to the position information of the camera device, the corresponding video files are divided into two categories, and the areas without hurdles in the coal mine are recorded as one category, and the areas with hurdles in the coal mine are recorded as the second category, so as to formulate different identification methods for different area types. For the areas without hurdles, the postures of the people who appear are recognized to analyze whether there are people falling to the ground, and to distinguish between the abnormal behaviors of people walking, standing, and walking, so as to improve the quality and intelligence of video recognition. For the areas with hurdles, the behavior of the people who appear is recognized to analyze whether there is unsafe behavior of people climbing over the railings, and timely issue warning prompts, so as to improve the quality and intelligence of video recognition.

[0033] Furthermore, since the real-time posture line is the spine line of the human body, which is generally perpendicular to the ground, it is used as a posture feature to perform a specific analysis of the actual posture of the person. The real-time angle is the actual angle between the identified real-time posture line of the person and the horizontal direction. By selecting human features, the posture of the person can be analyzed simply and effectively. If the real-time angle is determined to be within the second standard angle range, indicating that the human spine line is perpendicular to the horizontal ground, the posture of the person is further distinguished to increase the intelligence of video recognition. If the real-time angle is determined not to be within the second standard angle range, further intelligent analysis is required to determine whether a person has fallen to the ground.

[0034] In particular, by setting a threshold, it is possible to simply and effectively analyze whether a person has fallen to the ground. If the real-time angle is determined to be smaller than the standard angle, it is determined that a person has fallen to the ground, and a first warning prompt is issued. If the real-time angle is determined to be greater than or equal to the standard angle, an intelligent analysis of the center of gravity of the person is performed to perform posture recognition after eliminating the fault of the camera mechanism.

[0035] Furthermore, if it is determined that the real-time center of gravity moving distance is less than or equal to the standard center of gravity moving distance, it means that the center of gravity has basically not changed. It may be due to video freeze caused by camera mechanism failure, etc., and it is determined again by obtaining the next video frame to determine whether the camera mechanism needs to be checked and processed. If it is determined that the real-time center of gravity moving distance is greater than the standard center of gravity moving distance, it means that the center of gravity has shifted, and it is necessary to further determine the center of gravity trajectory and intelligently analyze the personnel posture.

[0036] Furthermore, by analyzing the angle formed by the ray between the two ankles of the human body and the horizontal ground when the real-time angle is determined to be within the second standard angle range, the behavioral characteristics of walking and standing of the person can be distinguished. If the real-time inclination angle is determined to be less than the standard inclination angle, a first posture mark is performed, indicating normal standing behavior. If the real-time inclination angle is determined to be greater than or equal to the standard inclination angle, an intelligent analysis is performed on the distance between the two ankles to distinguish the behavioral characteristics of walking and standing. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the process of the coal mine intelligent video recognition method according to an embodiment of the present invention;

[0038] Figure 2 A logic decision diagram for performing a first early warning prompt in an embodiment of the present invention;

[0039] Figure 3 A logic decision diagram for marking a video frame with a third posture or a second posture in an embodiment of the present invention;

[0040] Figure 4 This is a logic decision diagram for performing the second early warning prompt in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0043] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0044] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0045] See also Figure 1 As shown, it is a schematic diagram of the process of the coal mine intelligent video recognition method according to an embodiment of the present invention. The present invention provides a coal mine intelligent video recognition method, comprising:

[0046] Step S1, obtaining the video stream uploaded by each camera unit, reading the video frames in the video stream frame by frame, and performing target recognition on any currently read video frame to obtain target information, wherein the target information includes a portrait target and a position target;

[0047] Step S2, if it is determined that there is a portrait target in the target information, the portrait in the video frame is marked with a joint number, and a position target corresponding to the video frame is obtained, so as to perform a first-class mark and a second-class mark on the video frame;

[0048] Step S3, performing posture recognition on the portrait in the video frame that has completed the first type of marking, and when it is determined that the real-time angle is within the second standard angle range, determining the real-time tilt angle according to the standard tilt angle, and comparing the real-time distance with the standard distance, so as to perform the first posture marking or the second posture marking on the video frame;

[0049] When it is determined that the real-time angle is not within the second standard angle range, and the real-time angle is less than the standard angle, a first warning prompt is issued; when it is determined that the real-time angle is not within the second standard angle range, the real-time angle is greater than or equal to the standard angle, and the real-time center of gravity movement distance is greater than the standard center of gravity movement distance, a real-time trajectory route is generated, the real-time similarity is calculated, and the real-time similarity is compared with the standard similarity to perform a third posture mark or a second posture mark on the video frame;

[0050] Step S4, performing behavior recognition on the human figures in the video frames that have completed the second category labeling to analyze whether there are any violations by personnel and issue a second warning prompt.

[0051] In this embodiment, the control assembly obtains the video files of the camera devices installed in various areas of the coal mine, and the intelligent data is analyzed in real time to identify the unsafe behaviors of the workers in the coal mine. According to the position information in the target information, the corresponding video files are divided into two categories. The areas without hurdles in the coal mine are recorded as one category, and the areas with hurdles in the coal mine are recorded as the second category, so as to formulate different recognition methods for different area types. For the areas without hurdles, the posture of the people who appear is recognized to analyze whether there are people falling to the ground, and to distinguish between the abnormal behaviors of people walking, standing, and walking, so as to improve the quality and intelligence of video recognition. For the areas with hurdles, the behavior of the people who appear is recognized to analyze whether there is unsafe behavior of people climbing over the railings, and timely issue warning prompts to improve the quality and intelligence of video recognition.

[0052] The joint point numbering process in this embodiment is to scan the portrait area, extract the real-time joint point coordinates of the human body, number the real-time joint point coordinates according to the standard joint point names and corresponding numbers in the joint point database, and obtain the joint point number markings corresponding to each joint point. For example, if the left ankle is numbered 10, the left knee joint is numbered 9, the right ankle is numbered 13, and the right knee joint is numbered 12, then the first preset number is 10, the second preset number is 13, the third preset number is 9, and the fourth preset number is 12. The real-time ray is the ray where the line between the left ankle and the right ankle is located, and the real-time posture line is the line between the midpoint of the line connecting the left waist joint point and the right waist joint point in the portrait and the neck joint point.

[0053] See also Figure 2 As shown, it is a logic decision diagram for performing the first early warning prompt in an embodiment of the present invention;

[0054] Specifically, in step S3, a second standard angle range is set, a video frame with a class of tags is obtained, and a posture recognition is performed on a portrait in any of the video frames, and a real-time angle formed by a real-time posture line and a horizontal frame line of the video frame is obtained, and the real-time angle is compared with the second standard angle range.

[0055] If the real-time angle is within the second standard angle range, the real-time tilt angle will be determined according to the standard tilt angle to analyze the posture;

[0056] If the real-time angle is not within the second standard angle range, the real-time angle is compared with the first standard angle to analyze whether to issue a first warning prompt.

[0057] The second standard angle range set in this embodiment is between 88 degrees and 92 degrees, which represents the standard angle range between the real-time posture line of a person in a standing posture and the horizontal direction. Since the real-time posture line is the spine line of the human body, which is generally perpendicular to the ground, it is used as a posture feature to perform a specific analysis of the actual posture of the person. The real-time angle is the actual angle between the identified real-time posture line of the person and the horizontal direction. By selecting human features, the posture of the person can be analyzed simply and effectively. If the real-time angle is determined to be within the second standard angle range, indicating that the human spine line is perpendicular to the horizontal ground, the posture of the person is further distinguished to increase the intelligence of video recognition. If the real-time angle is determined not to be within the second standard angle range, further intelligent analysis is required to determine whether there is a person falling to the ground.

[0058] Specifically, a first standard angle is set, and when it is determined that the real-time angle is not within the range of a second standard angle, the real-time angle is compared with the first standard angle.

[0059] If the real-time angle is smaller than the standard angle, it is determined that a person has fallen to the ground, and a first warning prompt is issued;

[0060] If the real-time angle is greater than or equal to the standard angle, obtain the currently read video frame and the next video frame to mark the portrait contour to calculate the real-time center of gravity movement distance, and judge the real-time center of gravity movement distance according to the standard center of gravity movement distance to determine whether to check and process the camera mechanism.

[0061] The first standard angle set in this embodiment is 30°. If a person falls to the ground, the spine line is generally 30 degrees to the ground. By setting a threshold, it is possible to simply and effectively analyze whether the person has fallen to the ground. If the real-time angle is determined to be less than the standard angle, it is determined that a person has fallen to the ground, and a first warning prompt is issued. If the real-time angle is determined to be greater than or equal to the standard angle, an intelligent analysis of the center of gravity of the person is performed to perform posture recognition and distinction after eliminating the fault of the camera mechanism.

[0062] See also Figure 3 As shown, it is a logic decision diagram for marking a video frame with a third posture or a second posture according to an embodiment of the present invention;

[0063] Specifically, a standard center of gravity moving distance is set. When it is determined that the real-time angle is not within the second standard angle range and the real-time angle is greater than or equal to the standard angle, the currently read video frame and the next video frame are obtained to mark the portrait contour. According to the center of gravity position of the portrait contour in the two video frames, the real-time distance between the two center of gravity positions is calculated and recorded as the real-time center of gravity moving distance. The real-time center of gravity moving distance is determined according to the standard center of gravity moving distance.

[0064] If the real-time center of gravity moving distance is less than or equal to the standard center of gravity moving distance, continue to obtain the currently read video frame and the next video frame to mark the portrait contour, calculate the real-time center of gravity moving distance, and judge the real-time center of gravity moving distance according to the standard center of gravity moving distance. If it is determined again that the real-time center of gravity moving distance is less than or equal to the standard center of gravity moving distance, then check the camera mechanism;

[0065] If the real-time center of gravity moving distance is greater than the standard center of gravity moving distance, a real-time trajectory route will be generated, the real-time similarity will be calculated, and the real-time similarity will be compared with the standard similarity to perform a third posture mark or a second posture mark on the video frame.

[0066] The standard center of gravity moving distance set in this embodiment is 0.01 meters, which means that the center of gravity change approaches zero. If the real-time center of gravity moving distance is determined to be less than or equal to the standard center of gravity moving distance, it means that the center of gravity has basically not changed. It may be due to video freeze caused by camera mechanism failure, etc. The next video frame is obtained and judged again to determine whether the camera mechanism needs to be checked and processed. If the real-time center of gravity moving distance is determined to be greater than the standard center of gravity moving distance, it means that the center of gravity has shifted, and it is necessary to further judge the center of gravity trajectory and intelligently analyze the personnel posture.

[0067] Specifically, a standard trajectory route and a standard similarity are set. When it is determined that the real-time center of gravity movement distance is greater than the standard center of gravity movement distance, continuous video frames are continuously acquired for a preset acquisition time to mark the portrait contour, and the center of gravity position of the portrait contour in each video frame is recorded as a waypoint. The center of gravity position of the portrait contour in the currently read video frame is taken as the starting point, and the straight lines passing through the waypoints are connected frame by frame to generate a real-time trajectory route.

[0068] The standard trajectory route set in this embodiment is a walking route, which is related to the actual area characteristics corresponding to each video frame, such as whether obstacles need to be avoided when passing through this area; the standard similarity is set between 60% and 70%, indicating the similarity between the two curves.

[0069] Specifically, the real-time trajectory route is obtained, the similarity between the real-time trajectory route and the standard trajectory route is calculated to obtain the real-time similarity, and the real-time similarity is compared with the standard similarity.

[0070] If the real-time similarity is less than the standard similarity, marking the video frame with a third posture;

[0071] If the real-time similarity is greater than or equal to the standard similarity, a second posture mark is performed on the video frame to determine that the walking posture is normal.

[0072] In this embodiment, the method for calculating the real-time similarity adopts the Jaccard similarity coefficient method, and the similarity between two sets is calculated by using the Jaccard similarity coefficient, that is, the ratio of the intersection and the union of the two sets is calculated.

[0073] The third posture mark indicates that the person's walking posture is irregular and abnormal.

[0074] Specifically, a standard tilt angle is set, and when the real-time angle is determined to be within the second standard angle range, a real-time ray formed by the joint points corresponding to the first preset number and the second preset number is obtained, and a real-time tilt angle formed by the real-time ray and the horizontal frame line of the video frame is calculated, and the real-time tilt angle is determined according to the standard tilt angle.

[0075] If the real-time tilt angle is less than the standard tilt angle, marking the video frame with a first posture;

[0076] If the real-time tilt angle is greater than or equal to the standard tilt angle, the real-time distance is compared with the standard distance to perform a first posture mark or a second posture mark on the video frame.

[0077] The standard tilt angle set in this embodiment is the angle between the ray between the two ankles of the human body and the horizontal ground when the person is walking or standing normally, which is set between 30 degrees and 40 degrees;

[0078] When the real-time angle is determined to be within the second standard angle range, the angle formed by the ray between the two ankles of the human body and the horizontal ground is analyzed to distinguish the behavioral characteristics of a person walking and standing. If the real-time inclination angle is determined to be less than the standard inclination angle, a first posture mark is performed to indicate normal standing behavior. If the real-time inclination angle is determined to be greater than or equal to the standard inclination angle, an intelligent analysis is performed on the distance between the two ankles to distinguish the behavioral characteristics of walking and standing.

[0079] Specifically, a standard distance is set, and when it is determined that the real-time tilt angle is greater than or equal to the standard tilt angle, the real-time distance between the joint points corresponding to the first preset number and the second preset number is obtained, and the real-time distance is compared with the standard distance.

[0080] If the real-time distance is less than the standard distance, a first posture mark is performed on the video frame.

[0081] If the real-time distance is greater than or equal to the standard distance, a second posture mark is performed on the video frame.

[0082] The standard distance set in this embodiment is 3 centimeters; the first posture mark represents normal standing behavior, and the second posture mark represents normal walking behavior.

[0083] See also Figure 4 As shown, it is a logic decision diagram for performing the second early warning prompt in an embodiment of the present invention;

[0084] Specifically, a standard longitudinal distance is set, and video frames with two types of markings are obtained, and behavior recognition is performed on the portrait in any of the video frames. A rectangular coordinate system is established with the lower horizontal frame line of the video frame as the horizontal axis and any vertex of the lower horizontal frame line as the origin. The longitudinal coordinates of the joint points corresponding to the first preset number and the fourth preset number are obtained, and the difference between the two longitudinal coordinates is calculated to obtain the real-time longitudinal distance, and the real-time longitudinal distance is compared with the standard longitudinal distance.

[0085] If the real-time vertical distance is less than the standard vertical distance, no second warning will be given and the next video will be processed.

[0086] Frames for behavior recognition;

[0087] If the real-time longitudinal distance is greater than or equal to the standard longitudinal distance, it is determined that a person has violated the rules and a second warning prompt is issued.

[0088] The standard longitudinal distance set in this embodiment is 30 centimeters, which represents the longitudinal vertical distance between the ankle on one side of the human body and the knee joint on the other side. It can be selected according to the average height characteristics of actual coal mine workers.

[0089] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A coal mine intelligent video recognition method, characterized in that: include, Obtain the video stream uploaded by each camera unit, and perform first-class marking and second-class marking on the currently read video frame according to the position target in the video stream; Among them, the first-class mark means that the area without fences in the coal mine is recorded as the first class, and the second-class mark means that the area with fences in the coal mine is recorded as the second class; Marking joint points of the portrait in the video stream with numbers; Performing posture recognition on the portrait in the video frame that has completed the first type of marking, calculating the real-time angle according to the joint point number, and when it is determined that the real-time angle is within the second standard angle range, determining the real-time tilt angle according to the standard tilt angle, and comparing the real-time distance with the standard distance, so as to perform the first posture marking or the second posture marking on the video frame; When it is determined that the real-time angle is not within the second standard angle range, and the real-time angle is less than the standard angle, a first warning prompt is issued; when it is determined that the real-time angle is not within the second standard angle range, the real-time angle is greater than or equal to the standard angle, and the real-time center of gravity movement distance is greater than the standard center of gravity movement distance, a real-time trajectory route is generated, the real-time similarity is calculated, and the real-time similarity is compared with the standard similarity to perform a third posture mark or a second posture mark on the video frame; wherein, a video frame with a class of tags is obtained, and a posture recognition is performed on a portrait in any of the video frames, a real-time angle formed by a real-time posture line and a horizontal frame line of the video frame is obtained, and the real-time angle is compared with a second standard angle range, If the real-time angle is within the second standard angle range, the real-time tilt angle will be determined according to the standard tilt angle to analyze the posture; If the real-time angle is not within the second standard angle range, the real-time angle is compared with the first standard angle to analyze whether to issue a first warning prompt; When it is determined that the real-time angle is within the second standard angle range, a real-time ray formed by the joint points corresponding to the first preset number and the second preset number is obtained, a real-time tilt angle formed by the real-time ray and the horizontal frame line of the video frame is calculated, and the real-time tilt angle is determined according to the standard tilt angle. If the real-time tilt angle is less than the standard tilt angle, marking the video frame with a first posture; If the real-time tilt angle is greater than or equal to the standard tilt angle, obtaining the real-time distance between the joint points corresponding to the first preset number and the second preset number, comparing the real-time distance with the standard distance, and if it is determined that the real-time distance is less than the standard distance, marking the video frame with a first posture; Among them, the first preset number is the left ankle number, the second preset number is the right ankle number, and the fourth preset number is the right knee joint number; the real-time posture line is the line between the midpoint of the line connecting the left waist joint point and the right waist joint point in the portrait and the neck joint point; the currently read video frame and the next video frame are obtained to mark the portrait contour, and the real-time distance between the two center of gravity positions is calculated according to the center of gravity position of the portrait contour in the two video frames, and recorded as the real-time center of gravity movement distance; the lower horizontal frame line of the video frame is used as the horizontal axis, and any vertex of the lower horizontal frame line is used as the origin to establish a rectangular coordinate system, obtain the vertical coordinates of the joint points corresponding to the first preset number and the fourth preset number, and calculate the difference between the two vertical coordinates to obtain the real-time longitudinal distance; the first posture mark indicates normal standing behavior, the second posture mark indicates normal walking behavior, and the third posture mark indicates that the walking posture of the person is irregular and abnormal; Behavior recognition is performed on the human figures in the video frames that have completed the second category of marking, the real-time longitudinal distance is calculated according to the joint point numbers, and the real-time longitudinal distance is compared with the standard longitudinal distance to analyze whether there are any personnel violations and issue a second warning prompt.

2. The coal mine intelligent video recognition method according to claim 1, characterized in that: When it is determined that the real-time angle is not within the second standard angle range, the real-time angle is compared with the first standard angle. If the real-time angle is smaller than the first standard angle, it is determined that a person has fallen to the ground, and a first warning prompt is issued; If the real-time angle is greater than or equal to the first standard angle, obtain the currently read video frame and the next video frame to mark the portrait contour to calculate the real-time center of gravity movement distance, and judge the real-time center of gravity movement distance according to the standard center of gravity movement distance to determine whether to check and process the camera mechanism.

3. The coal mine intelligent video recognition method according to claim 2, characterized in that: When it is determined that the real-time angle is not within the second standard angle range and the real-time angle is greater than or equal to the first standard angle, the real-time center of gravity moving distance is determined according to the standard center of gravity moving distance, If the real-time center of gravity moving distance is less than or equal to the standard center of gravity moving distance, continue to obtain the currently read video frame and the next video frame to mark the portrait contour, calculate the real-time center of gravity moving distance, and judge the real-time center of gravity moving distance according to the standard center of gravity moving distance. If it is determined again that the real-time center of gravity moving distance is less than or equal to the standard center of gravity moving distance, then check the camera mechanism; If the real-time center of gravity moving distance is greater than the standard center of gravity moving distance, a real-time trajectory route will be generated, the real-time similarity will be calculated, and the real-time similarity will be compared with the standard similarity to perform a third posture mark or a second posture mark on the video frame.

4. The coal mine intelligent video recognition method according to claim 3 is characterized in that: When it is determined that the real-time center of gravity movement distance is greater than the standard center of gravity movement distance, continuous video frames are continued to be acquired for the preset acquisition time to mark the portrait contour, and the center of gravity position of the portrait contour in each video frame is recorded as the waypoint. The center of gravity position of the portrait contour in the currently read video frame is taken as the starting point, and the straight lines passing through the waypoints are connected frame by frame to generate a real-time trajectory route.

5. The coal mine intelligent video recognition method according to claim 4, characterized in that: The real-time trajectory route is obtained, the similarity between the real-time trajectory route and the standard trajectory route is calculated to obtain the real-time similarity, and the real-time similarity is compared with the standard similarity. If the real-time similarity is less than the standard similarity, marking the video frame with a third posture; If the real-time similarity is greater than or equal to the standard similarity, a second posture mark is performed on the video frame.

6. The coal mine intelligent video recognition method according to claim 5, characterized in that: When it is determined that the real-time tilt angle is greater than or equal to the standard tilt angle, the real-time distance between the joint points corresponding to the first preset number and the second preset number is obtained, and the real-time distance is compared with the standard distance. If it is determined that the real-time distance is greater than or equal to the standard distance, the video frame is marked with a second posture.

7. The coal mine intelligent video recognition method according to claim 1, characterized in that: Obtain video frames that have completed the second category of labeling, and perform behavior recognition on the portrait in any of the video frames, and compare the real-time longitudinal distance with the standard longitudinal distance. If the real-time longitudinal distance is less than the standard longitudinal distance, no second warning prompt is given, and behavior recognition continues for the next video frame; If the real-time longitudinal distance is greater than or equal to the standard longitudinal distance, it is determined that a person has violated the rules and a second warning prompt is issued.

Images