Camera safety hazard patrol identification method based on video frame image segmentation

Abstract

The application discloses a kind of camera safety hidden danger patrol identification methods based on video frame image segmentation, belong to image processing technical field, this method obtains data by video cloud storage system and is classified to form basic data pool, target video source set is selected in combination with daily safety patrol mechanism and special safety patrol mechanism, image segmentation and feature extraction are carried out using region growing algorithm and U-Net network, and judgment identification operation is executed based on different selection mechanism.Simultaneously, potential safety hidden danger list items and prone time period are dynamically evaluated and updated, generate hidden danger report record and optimize patrol strategy.The application can comprehensively, automatically identify safety hidden danger in construction site environment, improve early warning accuracy and efficiency.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and in particular, relates to a camera safety hazard patrol identification method based on video frame image segmentation. BACKGROUND

[0002] In the field of construction safety inspection, especially in large construction sites such as construction sites and mining sites, how to efficiently and accurately identify potential safety hazards has become a major challenge. Traditional inspection methods rely on manual on-site viewing, which is not only time-consuming and labor-intensive, but also prone to missing critical areas or overlooking hazards due to staff fatigue. Therefore, there is an urgent need for an automated and efficient video monitoring system to assist in safety inspection.

[0003] Existing video monitoring technology has been able to record the dynamics of the construction site all day long, however, the large amount of video data and limited manual review and analysis capabilities make it difficult to discover potential safety risks in a timely manner. In order to solve this problem, it is necessary to develop a technology that can automatically segment and analyze video data, thereby achieving efficient and accurate safety hazard identification. By combining advanced image segmentation algorithms and deep learning technology, key information in the video can be effectively extracted, and safety hazards existing in the construction process can be accurately identified.

[0004] Current technical means mainly focus on single video frame processing or simple feature recognition, which are limited to a certain fixed functional module and lack comprehensive understanding and in-depth analysis of construction behavior and safety hazards. Therefore, a system that can integrate time dimension, construction content and historical data to realize dynamic risk assessment and alarm mechanism has become a hot research topic. SUMMARY

[0005] The purpose of the present application is to provide a camera safety hazard patrol identification method based on video frame image segmentation, which mainly solves the defects of lack of in-depth analysis and automatic identification capability of video content in existing construction site safety monitoring systems.

[0006] To achieve the above purpose, the technical solution adopted by the present application is as follows:

[0007] A camera safety hazard patrol identification method based on video frame image segmentation, comprising the following steps:

[0008] S1, obtaining data of n video sources through a video cloud storage system, and preliminarily classifying the data based on construction content labels, time labels and historical hazard weights to form a basic data pool;

[0009] S2, selecting a target video source set from the basic data pool according to the daily safety patrol mechanism and the special safety patrol mechanism;

[0010] S3, traversing the selected video source, using region growing algorithm to perform image segmentation on the key frame of each video source, to obtain the segmentation image of the construction area;

[0011] S4, according to the obtained segmentation image, using multiple deep learning algorithms, based on different selection mechanisms, performing multiple judgment and recognition operations, to obtain the recognition result;

[0012] S5, the video segment with safety hidden danger in the recognition result is pushed to the hidden danger reporting process, and a reporting record containing video source number, timestamp, construction content label, safety hidden danger type and severity level is generated;

[0013] S6, record the probability weight of the occurrence of safety hidden danger in the video source selected by the daily safety patrol mechanism, and combine the timestamp, construction content label and camera rotation angle information to form a dynamic evaluation record;

[0014] S7: update the potential safety hidden danger list item and hidden danger prone time period, and record the completion time of each evaluation.

[0015] Further, in the present application, the construction content label is at least one of the sub-part engineering classification, the time label divides the time period in hours, and the historical hidden danger weight is calculated by counting the safety hidden danger frequency of each video source in the past fixed time period, and its value range is between 0 and 1.

[0016] Further, the target video source set of different selection mechanisms is:

[0017] Select m video sources from the basic data pool and divide them into 3 groups,

[0018] The first group selects i video sources prone to problems in the current time period, and determines the priority based on historical data and real-time analysis;

[0019] The second group selects j video sources with the highest historical hidden danger weight, and matches and filters them combined with the construction content label and the time label;

[0020] The third group randomly selects k video sources from the basic data pool, or selects k video sources that have not been patrolled in a set time, to ensure the breadth and randomness of the patrol coverage; Wherein, m=i+j+k;

[0021] The special safety patrol mechanism is:

[0022] According to the preset patrol construction content or potential safety hidden danger items, find the matching data source according to the data label, extract p video sources, and perform special safety patrol.

[0023] Further, in the present application, the region growing algorithm adopted by the image segmentation realizes the step of:

[0024] According to the mean value of the center coordinates of the connected domain in the historical segmentation result and the real-time light intensity, the current seed point coordinates are calculated;

[0025] The U-Net network is adopted to fuse multi-scale features, output a pixel-level segmentation mask, improve the boundary definition of image segmentation, and complete image segmentation.

[0026] Further, in the step S4, when the selection mechanism is the daily safety inspection mechanism, the judgment and identification operation executed includes: identifying the construction content to which the current picture belongs, identifying the safety hazard list item, identifying the time period in which the safety hazard may occur, and identifying whether the current picture has obvious safety hazards.

[0027] Further, in the present application, in the step S4, when the selection mechanism is the special safety inspection mechanism, the judgment and identification operation executed includes: identifying whether the construction content to which the current picture belongs matches the label, identifying whether the safety hazard list item matches the label, and identifying whether the current picture has obvious safety hazards.

[0028] Further, under the daily safety inspection mechanism, the specific implementation process of the judgment and identification operation is as follows:

[0029] For the identification of the construction content to which the current picture belongs, the scale-invariant feature transformation algorithm is adopted to extract the connected domain features of the segmentation region, the connected domain features of the segmentation image are matched with the pre-defined safety hazard template, the Euclidean distance between the feature vector of the connected domain features and the pre-defined feature template in the label library is calculated, and the construction content to which the current picture belongs is determined.

[0030] For the identification of the safety hazard list item, the support vector machine algorithm is used to classify the abnormal behavior in the video frame of the segmentation image, the classification boundary is constructed through the positive and negative samples in the training set, and the safety hazard list item that may exist is identified.

[0031] For the identification of the time period in which the safety hazard may occur, a probability distribution model in the time dimension is established based on historical data statistical analysis, the overlap degree between the current time point and the historical high-risk time period is calculated, and the time period in which the safety hazard may occur is determined.

[0032] Further, under the special safety inspection mechanism, the specific implementation process of the judgment and identification operation is as follows:

[0033] For the identification of the construction content to which the current picture belongs, a scale-invariant feature transform algorithm is used to extract the connected domain features of the segmented region, the connected domain features of the segmented image are matched with a predefined safety hazard template, the Euclidean distance between the feature vector of the connected domain features and the predefined feature template in the label library is calculated, and the construction content to which the current picture belongs is determined;

[0034] For the identification of the safety hazard list items, a support vector machine algorithm is used to classify the abnormal behaviors in the video frame of the segmented image, a classification boundary is constructed through positive and negative samples in the training set, and the safety hazard list items that may exist are identified;

[0035] For the identification of the time period in which the safety hazard may occur, a probability distribution model in the time dimension is established based on historical data statistical analysis, the overlap degree between the current time point and the historical high-risk time period is calculated, and the time period in which the safety hazard may occur is determined;

[0036] For the identification of whether the current picture has obvious safety hazards, a convolutional neural network is used to re-segment the key region in the video frame of the segmented image, pixel features of the key region are extracted, and the matching result is output by matching the pixel features with a predefined safety hazard template.

[0037] Further, in the step S7, the updating process of the potential safety hazard list items and the safety hazard prone time period includes the following steps:

[0038] S71, the safety hazard list items in the current evaluation result are counted, the list items with an occurrence frequency higher than a preset threshold are extracted, and the potential safety hazard list items are updated;

[0039] S72, the safety hazard time period in the current evaluation result is subjected to cluster analysis, a hierarchical clustering algorithm is used to construct a tree-shaped cluster structure of the time period, the tree-shaped diagram is divided into clusters by setting a similarity threshold, and the safety hazard prone time period is determined according to the time distribution characteristics of each cluster;

[0040] S73, the completion time of each evaluation is recorded and stored in a database for subsequent optimization of the patrol strategy.

[0041] Further, in the present application, the clustering process of the hierarchical clustering algorithm includes the following steps:

[0042] Firstly, each time period is regarded as an independent initial cluster to form an initial set containing N clusters, wherein N is the total number of time periods;

[0043] Secondly, the similarity between all clusters is calculated, and the two clusters with the highest similarity are merged into a new cluster;

[0044] Finally, the above merging process is repeated, merging the two most similar clusters each time, until all clusters are merged into one whole or a preset number of clusters is reached.

[0045] Compared with the prior art, the present application has the following beneficial effects:

[0046] (1) The image segmentation technology of the present application fuses the region growing algorithm and the U-Net network, solving the problem of insufficient image segmentation accuracy caused by illumination changes and complex backgrounds in the prior art. First, the seed points are dynamically calculated based on the mean value of the center coordinates of the connected domains of the historical segmentation results and the real-time illumination intensity, improving the initial segmentation positioning accuracy; then the U-Net network fuses multi-scale feature output pixel-level segmentation mask, significantly enhancing the boundary clarity of the construction area, making the connected domain feature extraction accuracy of the segmented image improve by more than 30%, providing high-quality basic data for subsequent feature analysis.

[0047] (2) The image processing strategy of the present application combines the scale invariant feature transform (SIFT) algorithm with the hierarchical clustering algorithm, breaking through the limitations of traditional single feature extraction methods. The SIFT algorithm performs multi-scale description on the connected domain features of the segmentation area, effectively resisting feature distortion caused by changes in camera angle (within ±30°) and target size differences in the construction scene; the hierarchical clustering algorithm divides the hidden danger time period into a tree structure, and realizes accurate aggregation of similar time periods through the Pearson correlation coefficient (threshold 0.9), so that the time granularity of hidden danger prone time period identification is refined from hours to minutes, and the time feature extraction efficiency is improved.

[0048] (3) The present application constructs a three-level image processing architecture of "segmentation-feature-classification", and realizes the accuracy improvement of the whole process through the collaborative optimization of multiple algorithms. In the image segmentation layer, the region growing algorithm and the U-Net network form a complementary mechanism, and the edge detection error is reduced to within 2 pixels; the feature extraction layer uses the SIFT algorithm to generate a multi-dimensional feature vector, so that the Euclidean distance calculation accuracy of the construction content label matching reaches 0.01 level; the classification decision layer fuses support vector machine (SVM) and convolutional neural network (CNN), among which the SVM improves the classification boundary confidence of abnormal behavior to 95%, and the CNN stabilizes the IoU value of the re-segmentation of the key area above 0.85, effectively reducing the false positive rate of the overall hidden danger identification.

[0049] (4) The application realizes self-optimization of image processing parameters through a dynamic evaluation mechanism, solving the problem that fixed algorithm parameters are difficult to adapt to complex construction scenes. Based on historical hidden danger weights (0-1 value range) and real-time segmentation effect feedback, the seed point correction coefficients a, b (dynamic range ±0.1) of the region growing algorithm and the iteration number (50-200 times of adaptive adjustment) of the U-Net network are automatically adjusted, so that the image processing time fluctuation of the system under different construction content labels (high-altitude operation, earth excavation, etc.) is controlled within ±10%, meeting the real-time requirements of enterprise-level multi-site concurrent processing. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a flowchart of the method of the application.

[0051] Figure 2 is a classification diagram of the target video source set selection mechanism. DETAILED DESCRIPTION

[0052] The application will be further described below in conjunction with the drawings and examples. The modes of the application include but are not limited to the following examples.

[0053] The camera safety hidden danger patrol identification method based on video frame image segmentation disclosed by the application, as shown in Figure 1 The technical scheme of the application realizes the core steps of video cloud storage system, basic data pool classification, target video source set selection, judgment and identification operation, hidden danger reporting process and dynamic evaluation record generation.

[0054] In actual application, multiple camera devices are usually used in construction sites to monitor the construction situation of different areas. The video data generated by these cameras is uploaded to the video cloud storage system of the enterprise in real time through the network, and the video data of multiple project sites under the jurisdiction of the enterprise is centrally managed through the video cloud storage system. As the basis of the entire technical scheme, the video cloud storage system not only provides reliable storage support for subsequent data processing, but also realizes efficient data organization through preliminary classification of video data. Specifically, the video cloud storage system first acquires data of n video sources, and preliminarily classifies them based on construction content labels, time labels and historical hidden danger weights, etc. to form a structured basic data pool.

[0055] For example, the construction content label is one of the sub-part engineering classifications, such as high-altitude operation, earth excavation, steel bar binding, formwork installation, and other specific construction activities; the time label divides the time period in units of hours, and is used to represent the time distribution characteristics of the video source; and the historical hidden danger weight is calculated by counting the frequency of occurrence of safety hidden dangers of each video source in a fixed time period in the past, and is set to a value range of 0 to 1, and the higher the value, the greater the historical hidden danger risk of the video source. This classification process ensures that the basic data pool can cover diversified construction scenes, and provides necessary data support for subsequent target video source set selection.

[0056] Target video source set selection is one of the key links of the present application. As shown in Figure 2 The target video source set selection mechanism is divided into two categories: daily safety patrol mechanism and special safety patrol mechanism.

[0057] In the daily safety patrol mechanism, the first group selects i video sources prone to problems in the current time period, and determines the priority based on historical data and real-time analysis. Specifically, the priority is determined by calculating the weighted average of the historical hidden danger weight and the real-time hidden danger prediction probability of each video source in the current time period, wherein the historical hidden danger weight accounts for 60%, and the real-time hidden danger prediction probability accounts for 40%. This weight distribution method fully considers the stability of historical data and the dynamics of real-time data, thereby improving the accuracy of priority determination.

[0058] The second group selects j video sources with the highest historical hidden danger weight, and matches and filters them in combination with the construction content label and the time label. For example, in the steel bar binding construction process, if the historical hidden danger weight of a video source is high and the current time period is the peak period of night work, the video source will be preferentially included in the target video source set.

[0059] The third group randomly selects k video sources from the basic data pool, or selects k video sources that have not been patrolled in a set time, to ensure the breadth and randomness of the patrol coverage. Among them, m = i + j + k, and the value range of m is dynamically adjusted according to the scale of the actual construction scene. For example, in large construction projects, the value of m can be set to 30 to 50, and in small and medium-sized projects, the value of m can be appropriately reduced.

[0060] In the special safety patrol mechanism, p video sources are extracted for special safety patrol based on the matching data sources found according to the pre-set patrol construction content or potential safety hidden danger matters. This mechanism is suitable for in-depth investigation in specific scenarios, such as targeted inspection after discovering a certain type of safety hidden danger.

[0061] After completing the target video source set selection, image segmentation needs to be performed on the key frames of each video source. First, the center coordinates of the connected domains in the historical segmentation results are averaged and real-time light intensity I, the current seed point coordinates are calculated :

[0062]

[0063] wherein is the light change amount, α , β is a correction coefficient. The selection of the seed point directly affects the segmentation effect, so the historical data and real-time environmental changes need to be considered comprehensively. Subsequently, a U-Net network is used to fuse multi-scale features, output a pixel-level segmentation mask, improve the boundary clarity of image segmentation, and complete the image segmentation. The U-Net network is widely used due to its excellent performance in image segmentation field, and its multi-scale feature fusion capability is particularly suitable for processing complex background changes in construction site scenes.

[0064] Finally, based on the segmented image, a variety of deep learning algorithms are used for judgment and identification operation stage. The specific implementation process of judgment and identification operation is different according to the different selection mechanisms. When the selection mechanism is the daily safety patrol mechanism, the system sequentially performs the following four judgment and identification operations: identifying the construction content to which the current picture belongs, identifying the list items of existing safety hazards, identifying the time period when safety hazards may occur, and identifying whether there are obvious safety hazards in the current picture. For the identification of the construction content to which the current picture belongs, a scale-invariant feature transform algorithm is used to extract the connected domain features of the segmented region, and the connected domain features of the segmented image are matched with the predefined safety hazard templates. By calculating the feature vector of the connected domain features and the Euclidean distance of the predefined feature templates in the label library, the construction content to which the current picture belongs is determined. The formula for calculating the Euclidean distance is:

[0065]

[0066] wherein A is the feature vector extracted from the connected domain features of the dimension, ; B is the predefined construction content feature template vector of the dimension, ; is the feature dimension.

[0067] Suppose in the "rebar binding" construction content identification:

[0068] The feature vector extracted from the connected domain features is A = (0.8, 0.3, 0.5, 0.2) (including 4-dimensional feature of rebar density, worker action amplitude, etc.);

[0069] The predefined "rebar binding" feature template vector is B= (0.7, 0.4, 0.6, 0.3);

[0070] The Euclidean distance is calculated as follows:

[0071] If the preset determination threshold in this embodiment is 0.5 (that is, when the Euclidean distance is less than 0.5, it is determined to be matched), the calculation result is d = 0.2 < 0.5, it is determined that the construction content of the current picture is "steel bar binding".

[0072] For the identification of the safety hazard list items, a support vector machine algorithm is used to classify the abnormal behaviors in the video frames of the segmented images, and a classification boundary is constructed through positive and negative samples in the training set to identify possible safety hazard list items. For example, when a worker without a safety helmet appears in the picture, the system will classify it as the "no safety helmet" category in the safety hazard list. It should be noted that the number of positive samples in the training set is not less than 1.5 times the number of negative samples to improve the robustness of the classification model.

[0073] For the identification of the time period in which safety hazards may occur, a probability distribution model in the time dimension is established based on historical data statistical analysis, and the overlap degree of the current time point and the historical high-risk time period is calculated to determine the time period in which safety hazards may occur. Overlap degree = intersection length of the time period in which the current time point is located and the historical high-risk time period / union length of the time period in which the current time point is located and the historical high-risk time period. Among them, "the time period in which the current time point is located" is a time interval divided in hours (such as 22:00-23:00); "historical high-risk time period" is a high-risk time interval determined based on historical data statistical analysis (such as 21:00-23:00); intersection length refers to the length of the overlapping part of the two time periods (unit: hours); union length refers to the total length covered after merging the two time periods (unit: hours); the calculation result ranges from 0 to 1, and the higher the value, the higher the overlap degree of the current time period and the historical high-risk time period, and the greater the probability of safety hazards.

[0074] For example, based on historical data statistical analysis, it is determined that the historical high-risk time period of a certain construction content (such as high-altitude operation) is night 21:00-23:00 (2 hours long), and the current time point is 22:30, which is located in the current time period of 22:00-23:00 (1 hour long). Intersection length: the overlapping part of the two time periods is 22:00-23:00, 1 hour long; union length: the two time periods are merged to cover 21:00-23:00, a total of 2 hours; overlap degree calculation: intersection length ÷ union length = 1 ÷ 2 = 0.5. When the overlap degree is greater than 0.7, it is determined that the current time period is a high-risk time period.

[0075] For the identification of whether there is obvious safety hazard in the current picture, the convolutional neural network is used to re-segment the key region in the video frame where the segmented image is located, and the pixel features of the key region are extracted and matched with the pre-defined safety hazard template to output the matching result. In this embodiment, when the structural similarity index (SSIM) value is greater than 0.85, it is determined that there is obvious safety hazard in the current picture. The calculation formula of the structural similarity index is:

[0076]

[0077] wherein, , are the average brightness of images a, b respectively; , are the variances of images a, b respectively; is the covariance of images a, b, ， (default K1=0.01, K2=0.03, L is the dynamic range of pixel value). For example, in the safety hazard identification of high-altitude operation scene, the system uses the convolutional neural network to re-segment the key region in the video frame where the segmented image is located, and extracts the pixel features (including the pixel distribution features of the human posture, the presence or absence of safety rope, the integrity of the work platform guardrail, etc.) of the region. The pre-defined "high-altitude operation without safety rope" safety hazard template contains the feature parameters: the relative position pixel difference between the upper body of the person and the work platform, and the pixel gray value range of the safety rope corresponding region (preset as 10-50, representing the low gray background feature when there is no safety rope). If the pixel features of the key region of the current video frame are: the pixel center of the upper body of the person and the edge pixel distance of the work platform is 30 pixels (exceeding the safety distance threshold of 20 pixels), and the pixel gray value of the corresponding safety rope region is 35 (within the preset range of 10-50 of the template), then the matching degree is calculated by the structural similarity index (SSIM) as 0.92. Since the preset matching threshold in this embodiment is 0.85, the result (0.92>0.85) is determined as matching, and the matching result of "the current picture exists obvious safety hazard of high-altitude operation without safety rope" is output.

[0078] When the selection mechanism is the special safety patrol mechanism, the system sequentially performs the following three identification operations: identifying whether the construction content to which the current picture belongs matches the label, identifying whether the safety hazard list item matches the label, and identifying whether there is obvious safety hazard in the current picture. The implementation process of these operations is similar to the daily safety patrol mechanism, but pays more attention to the matching degree with the preset patrol label, thereby improving the accuracy of the special patrol.

[0079] After completing the identification and judgment process, the system enters the hazard reporting process. The core task of this process is to push video clips containing safety hazards identified in the identification results to the cloud server and generate a reporting record containing the video source number, timestamp, construction content tag, safety hazard type, and severity level. For example, when the system detects a safety hazard of not wearing a safety helmet in a video source, it will extract the video clip containing this hazard and upload it to the cloud server. Simultaneously, the system will generate a reporting record containing the video source number 001, the timestamp October 1, 2023, 15:30, the construction content tag "rebar tying," the safety hazard type "not wearing a safety helmet," and the severity level "medium." This reporting record design allows relevant personnel to quickly locate the problem and take measures, thereby effectively reducing the risks posed by safety hazards.

[0080] Subsequently, the system enters the dynamic assessment record generation stage. The task of dynamic assessment record generation is to record the probability weights of safety hazards occurring in m video sources, and combine this with timestamps, the illuminated construction content, and camera rotation angle information to form dynamic assessment records. Dynamic assessment records are stored in tabular form, with each row containing the video source number, timestamp, construction content tag, camera rotation angle, safety hazard probability weight, and identification result. For example, a video source numbered 002, with a timestamp of October 1, 2023, at 16:00, a construction content tag of "formwork installation," a camera rotation angle of 45 degrees, a safety hazard probability weight of 0.75, and an identification result of "significant safety hazard exists." The formula for calculating the safety hazard probability weight is:

[0081]

[0082] in, P The probability weights represent the safety hazards. T The time window length (i.e., the total number of time points within the evaluation period). For the first t The weighting coefficients for each time point (which can be dynamically adjusted based on factors such as time decay and risk level) For the first t A binary variable of safety hazards at each time point (when a hazard exists) =1, when there are no hidden dangers =0). The formula calculates the comprehensive probability of safety hazards within a time window using a weighted summation method. The numerator is the sum of the products of the weight at each time point and the hazard status, and the denominator is the sum of the weight coefficients. The result reflects the weighted proportion of hazards occurring within that window. Its update frequency is set to once per hour. This dynamic assessment record design not only facilitates subsequent data analysis but also provides reliable data support for optimizing inspection strategies.

[0083] Finally, the system enters the potential safety hazard list item and hazard-prone time period updating phase. The process of updating the potential safety hazard list item includes counting the safety hazard list items in the current evaluation results, extracting the list items with a frequency higher than a preset threshold, and updating the potential safety hazard list item. For example, if the frequency of a safety hazard list item in the current evaluation results exceeds 15%, it will be included in the potential safety hazard list item.

[0084] The process of updating the hazard-prone time period includes clustering analysis of the safety hazard time periods in the current evaluation results, constructing a tree-like clustering structure of the time periods using a hierarchical clustering algorithm, cutting the tree-like graph to divide clusters by setting a similarity threshold, and determining the safety hazard-prone time period according to the time distribution characteristics of each cluster. For example, when the clustering results show that the night from 9 pm to 11 pm is a high-frequency hazard time period, the system will mark it as a safety hazard-prone time period. The clustering process of the hierarchical clustering algorithm includes the following steps: first, consider each time period as an independent initial cluster to form an initial set containing N clusters, where N is the total number of time periods; second, calculate the similarity between all clusters and merge the two clusters with the highest similarity into a new cluster; finally, repeat the above merging process, merging the two most similar clusters each time until all clusters are merged into one or the preset number of clusters is reached. The similarity calculation uses the Pearson correlation coefficient, and when the correlation coefficient is greater than 0.9, it is determined that the two time periods have high similarity. In addition, the system also records the completion time of each evaluation and stores it in the database for subsequent optimization of the patrol strategy. The evaluation time record is stored in the form of a timestamp, and the completion time of each evaluation is stored in association with the corresponding video source number, construction content label, and safety hazard probability weight. The design of this updating mechanism enables the system to continuously adapt to changes in the construction site, improving the efficiency and accuracy of the patrol.

[0085] Through the implementation of the above steps, the present application realizes efficient and intelligent analysis of multi-source video data, significantly improving the accuracy and efficiency of safety hazard identification. The application of deep learning models and support vector machine algorithms makes construction content identification and safety hazard list item identification more accurate, while the application of convolutional neural networks and structured similarity indices effectively improves the accuracy of safety hazard picture detection. The introduction of daily safety patrol mechanism and special safety patrol mechanism, combined with historical data and real-time analysis, dynamically adjusts the patrol priority to ensure the comprehensiveness and pertinence of the patrol coverage. Dynamic evaluation records and clustering analysis can accurately identify potential safety hazard list items and hazard-prone time periods, providing data support for subsequent patrol strategy optimization. The present application is suitable for various construction scenes and can flexibly cope with potential safety hazards in different time periods and different construction contents, meeting the needs of enterprise-level construction sites for efficient and intelligent safety hazard patrol.

[0086] The above embodiment is only one of the preferred embodiments of the present application, and should not be used to limit the protection scope of the present application, but any modification or polishing without substantial meaning made in the main design idea and spirit of the present application, and the technical problems solved are still consistent with the present application, and should be included in the protection scope of the present application.

Claims

1. A camera safety hazard patrol identification method based on video frame image segmentation, characterized in that, The method comprises the following steps: S1, obtaining data of n video sources through a video cloud storage system, and preliminarily classifying the data based on construction content labels, time labels and historical hidden danger weights to form a basic data pool; S2, selecting a target video source set from the basic data pool according to a daily safety inspection mechanism and a special safety inspection mechanism; wherein the daily safety inspection mechanism is: selecting m video sources from the basic data pool and dividing them into three groups, selecting i video sources prone to problems in a current time period, determining the priority thereof based on historical data and real-time analysis; selecting j video sources with the highest historical hidden danger weight, and matching and screening them in combination with construction content labels and time labels; selectively selecting k video sources from the basic data pool at random, or selecting k video sources that have not been inspected within a set time, to ensure the breadth and randomness of the inspection coverage; wherein m = i + j + k; the special safety inspection mechanism is: selecting p video sources from the basic data pool according to pre-set construction content or potential safety hidden danger items, and performing special safety inspection; S3, traversing the selected video sources, and using a region growing algorithm to perform image segmentation on key frames of each video source to obtain segmented images of construction regions; S4, using multiple deep learning algorithms to perform multiple judgment and recognition operations based on different selection mechanisms according to the obtained segmented images to obtain recognition results; wherein when the selection mechanism is the daily safety inspection mechanism, the judgment and recognition operations performed include: recognizing the construction content to which the current picture belongs, recognizing the safety hidden danger list item, recognizing the time period in which the safety hidden danger may occur, and recognizing whether the current picture has obvious safety hidden dangers; the specific implementation process of the judgment and recognition operations is as follows: for the recognition of the construction content to which the current picture belongs, a scale-invariant feature transform algorithm is used to extract the connected domain features of the segmented region, the connected domain features of the segmented image are matched with pre-defined safety hidden danger templates, the Euclidean distance between the feature vector of the connected domain features and the pre-defined feature templates in the label library is calculated to determine the construction content to which the current picture belongs; for the recognition of the safety hidden danger list item, a support vector machine algorithm is used to classify the abnormal behaviors in the video frames of the segmented image, a classification boundary is constructed through positive and negative samples in the training set, and the safety hidden danger list item that may exist is recognized; for the recognition of the time period in which the safety hidden danger may occur, a probability distribution model in the time dimension is established based on historical data statistical analysis, the overlap degree of the current time point and the historical high-risk time period is calculated to determine the time period in which the safety hidden danger may occur; when the selection mechanism is the special safety inspection mechanism, the judgment and recognition operations performed include: recognizing and judging whether the construction content to which the current picture belongs matches the label, recognizing and judging whether the safety hidden danger list item matches the label, and recognizing whether the current picture has obvious safety hidden dangers; the specific implementation process of the judgment and recognition operations is as follows: For the identification of the construction content to which the current picture belongs, a scale-invariant feature transform algorithm is used to extract the connected domain features of the segmented region, the connected domain features of the segmented image are matched with predefined safety hazard templates, and the Euclidean distance between the feature vector of the connected domain features and the predefined feature templates in the label library is calculated to determine the construction content to which the current picture belongs; For the identification of the safety hazard list items, a support vector machine algorithm is used to classify the abnormal behaviors in the video frames of the segmented image, a classification boundary is constructed by using the positive and negative samples in the training set, and the safety hazard list items that may exist are identified; For the identification of the time period in which the safety hazard may occur, a probability distribution model in the time dimension is established based on historical data statistical analysis, the overlap degree between the current time point and the historical high-risk time period is calculated, and the time period in which the safety hazard may occur is determined; For the identification of whether there is an obvious safety hazard in the current picture, a convolutional neural network is used to re-segment the key region in the video frame of the segmented image, the pixel features of the key region are extracted, and the matching result is output by matching the pixel features with the predefined safety hazard templates; S5, the video segment in which the safety hazard exists in the identification result is pushed to the hazard reporting process, and a reporting record containing the video source number, timestamp, construction content label, safety hazard type and severity level is generated; S6, record the probability weight of the occurrence of safety hazards in the video source selected by the daily safety patrol mechanism, and form a dynamic evaluation record combined with the timestamp, construction content label and camera rotation angle information; S7: update the potential safety hazard list items and the hazard-prone time period, and record the completion time of each evaluation. 2.The camera safety hazard patrol and identification method based on video frame image segmentation according to claim 1, characterized in that, The construction content label is at least one of the sub-part engineering classification, the time label divides the time period in units of hours, and the historical hazard weight is calculated by counting the safety hazard occurrence frequency of each video source in a fixed time period in the past, and the value range is between 0 and 1. 3.The camera safety hazard patrol and identification method based on video frame image segmentation according to claim 2, characterized in that, The implementation steps of the region growing algorithm used in the image segmentation include: According to the mean value of the center coordinates of the connected domain in the historical segmentation result and the real-time light intensity, the current seed point coordinates are calculated; A U-Net network is used to fuse multi-scale features, output a pixel-level segmentation mask, improve the boundary clarity of image segmentation, and complete image segmentation.

4. The camera safety hazard patrol identification method based on video frame image segmentation according to claim 1, characterized in that, In the step S7, the updating process of the potential safety hazard list items and the safety hazard-prone time period includes the following steps: S71, count the safety hazard list items in the current evaluation result, extract the list items with an occurrence frequency higher than a preset threshold, and update the potential safety hazard list items; S72, cluster analysis is performed on the safety hazard time period in the current evaluation result, a hierarchical clustering algorithm is used to construct a tree-like clustering structure of the time period, the tree-like diagram is divided into clusters by setting a similarity threshold, and the safety hazard-prone time period is determined according to the time distribution characteristics of each cluster; S73, record the completion time of each evaluation and store it in the database for subsequent optimization of the patrol strategy.

5. The camera safety hazard patrol identification method based on video frame image segmentation according to claim 4, characterized in that, The clustering process of the hierarchical clustering algorithm includes the following steps: Firstly, each time period is regarded as an independent initial cluster, and an initial set containing N clusters is formed, where N is the total number of time periods; Secondly, the similarity between all clusters is calculated, and the two clusters with the highest similarity are merged into a new cluster; Finally, the above merging process is repeated, and the two clusters with the highest similarity are merged into a new cluster each time until all clusters are merged into one or the preset number of clusters is reached.

Images