A lifeline warning method for high-altitude operations

Abstract

The present application discloses a lifeline warning method for high-altitude operations, in the field of image data processing, including: obtaining image data containing images of operators and images of construction areas; extracting lifeline areas using image segmentation algorithms; extracting timestamps and location information of image data as metadata, and constructing spatiotemporal indexes; constructing a seat belt wearing recognition model based on a convolutional neural network, setting up a spatial attention mechanism and a channel attention mechanism; constructing a small sample data set; combining the small sample data set with the corresponding metadata; training the model with Few‑shot Learning and meta-learning using a small sample data set with spatiotemporal labels to identify the seat belt wearing status; obtaining the location coordinates of the image data according to the spatiotemporal index, and combining the identified seat belt wearing status to perform risk assessment through a decision tree. In view of the complex and changeable high-altitude operation environment in the prior art, and the low efficiency of the supervision of operators wearing seat belts, the present application improves the supervision efficiency.

Description

Technical Field

[0001] The present application relates to the field of image data processing, and in particular to a lifeline warning method for high-altitude operations. Background Art

[0002] With the continuous expansion of the scale of construction projects and the increasing complexity of construction technology, aerial work has become the norm in construction. The aerial work environment is complex and changeable, and there are many safety hazards in the construction process. The traditional manual inspection and supervision methods are inefficient and difficult to meet the increasing safety management needs. In order to effectively prevent high-altitude falling accidents and improve the level of construction safety management and control, it is urgent to introduce advanced technical means to realize intelligent supervision of aerial work safety.

[0003] At present, the safety management of aerial work mainly relies on traditional methods such as manual inspection and video monitoring. Manual inspection is time-consuming and labor-intensive, and it is difficult to achieve all-weather and full-coverage supervision, which is prone to omissions and blind spots. Although the traditional video monitoring system can record the situation of the work site in real time, it lacks intelligent analysis and early warning functions, and requires safety management personnel to manually view the video screen, which is inefficient and difficult to guarantee accuracy.

[0004] In recent years, with the development of computer vision and artificial intelligence technology, intelligent safety supervision solutions have received more and more attention. Behavior recognition technology based on video analysis can automatically detect the behavior of operators and determine whether there are any illegal operations or safety hazards. However, due to the complexity and variability of the high-altitude working environment, the existing behavior recognition algorithms face many challenges in practical applications. First, the video image quality in high-altitude working scenes is often poor. Affected by factors such as lighting, haze, and occlusion, it is difficult to accurately locate and extract the human body and safety belt area in the image. Secondly, the risks faced by different construction stages and types of work are different, and the behavior patterns and safety belt wearing requirements of operators are also different. The existing recognition models are difficult to adapt to the changing scene requirements. Summary of the invention

[0005] In response to the problems in the prior art of complex and changeable high-altitude working environment and low efficiency in supervising workers' wearing of safety belts, this application provides a lifeline warning method for high-altitude working, which improves the efficiency of safety supervision by constructing a spatiotemporal index and a recognition model that integrates scene prior information.

[0006] The purpose of this application is achieved through the following technical solutions.

[0007] The present application provides a lifeline warning method for high-altitude operations, comprising: S1, acquiring image data including images of operators and images of construction areas; S2, segmenting the image data using an image segmentation algorithm to extract a human body area and a lifeline area including a safety belt from the image data;

[0008] S3, extracting the timestamp and location information of the image data as metadata, and constructing a spatiotemporal index of the image data according to the metadata; specifically, extracting the shooting time information from the image data obtained in step S1, and formatting the shooting time information into timestamp data, where the timestamp data includes time dimensions such as year, month, day, hour, minute, and second; extracting the GPS positioning information from the EXIF ​​metadata of the image data, and converting the GPS positioning information into spatial coordinate data, where the spatial coordinate data includes spatial dimensions such as longitude, latitude, and altitude; associating the timestamp data with the spatial coordinate data to generate spatiotemporal metadata of the image data, where the spatiotemporal metadata includes the time tag and spatial tag of the image data; Use spatiotemporal metadata to build a spatiotemporal index of image data, use the Geohash algorithm to encode spatial coordinate data, generate a unique Geohash code, and the Geohash code represents the spatial position of the image data; use the time tree algorithm to encode timestamp data and generate a unique time tree code, which represents the time position of the image data; combine the Geohash code and the time tree code to generate a spatiotemporal code for the image data, which uniquely identifies the spatiotemporal position of the image data; build a spatiotemporal index of image data based on the spatiotemporal code, use an inverted index structure to organize the mapping relationship between image data and spatiotemporal codes, and support fast retrieval based on time periods and spatial regions.

[0009] S4, construct a seat belt wearing recognition model based on convolutional neural network to identify the seat belt wearing status of workers; in the constructed seat belt wearing recognition model, set up a spatial attention mechanism and a channel attention mechanism; wherein, the spatial attention mechanism embeds the prior information of the building edge and scaffolding area to guide the model to pay attention to the corresponding area; set the weights of the seat belt color and reflective strip in the channel attention mechanism; S5, obtain small sample data of seat belts of workers in different construction stages and different types of work, and divide the small sample data into scenes according to the construction stage and type of work to construct a small sample data set; combine the small sample data set with the corresponding metadata to obtain a small sample data set with spatiotemporal labels; S6, use the small sample data set with spatiotemporal labels to perform Few-shot Learning and meta-learning training on the constructed seat belt wearing recognition model; input the extracted human body area and lifeline area into the trained seat belt wearing recognition model for prediction to identify the seat belt wearing status.

[0010] S7, obtains the position coordinates of the image data according to the constructed spatiotemporal index, combines it with the identified seat belt wearing status, performs risk assessment through a decision tree, and triggers the corresponding warning level. Specifically, according to the timestamp information of the image data, a time range query is performed on the B+ tree spatiotemporal index to obtain the leaf nodes adjacent to the timestamp, and the spatial position coordinate data corresponding to the input image stored in the leaf nodes is extracted; the pixel coordinates of the human body area in the input image are mapped to the spatial coordinate system of the construction site to obtain the actual position coordinates of the operator at the construction site; the Euclidean distance is calculated between the position coordinates of the operator and the image position coordinates extracted in S71, and if the Euclidean distance is less than the set threshold, it is determined that the positions of the two match, and the spatial position of the operator is confirmed; according to the prediction result of the seat belt wearing recognition model in S6, the confidence score of the operator's seat belt wearing status is obtained, and the confidence score is divided into three levels: high, medium and low, which respectively represent the three states of qualified wearing, irregular wearing and not wearing; the spatial position coordinates of the operator and the confidence score of the seat belt wearing status are input into a preset decision tree model, and the decision tree model evaluates the safety risk level of the operator at the current position by analyzing the association rules between the position and the wearing status; wherein, the decision tree model is constructed based on a large amount of historical accident data and expert experience, and is constructed through information entropy and Gini index, etc. The indicator selects the optimal partition attribute and generates a tree-structured risk determination rule; the non-leaf nodes of the decision tree represent the partition attributes, such as location height, horizontal distance, wearing status, etc., and the leaf nodes represent the safety risk level, which is divided into four levels: safe, prompt, warning, and dangerous; for the input location coordinates and wearing status data, the decision tree model starts from the root node and recursively divides the data instance into different child nodes until it reaches the leaf node and divides the instance into the corresponding safety risk level; according to the risk level evaluated by the decision tree model, combined with the preset warning rules, the corresponding level of warning prompt is adaptively triggered; among them, the warning rules can be dynamically set based on factors such as the safety management requirements of the construction site and the emergency response capabilities of the personnel, and different warning methods can be set for different risk levels, such as voice reminders, flash warnings, SMS notifications, on-site alarms, etc.; when the risk level evaluated by the decision tree is safe, no warning is triggered; when the risk level is prompt, the wearing status is reminded by voice; when the risk level is warning, the wearing status and location danger are reminded by flash warnings and SMS notifications; when the risk level is dangerous, the on-site highlight alarm is triggered to notify the on-site safety officer to take emergency measures.

[0011] Furthermore, S2, an image segmentation algorithm is used to segment the image data to extract the human body area and the lifeline area including the seat belt in the image data, including: using a median filtering algorithm to perform denoising on the salt and pepper noise and Gaussian noise in the image data; median filtering calculates the median of the pixels in the window by sliding the window to replace the central pixel, which can effectively eliminate the influence of salt and pepper noise and Gaussian noise; a histogram equalization algorithm is used to enhance the denoised image; histogram equalization increases the contrast and clarity of the image by stretching the distribution range of the pixel grayscale value, making the image features more obvious, and providing better input data for subsequent semantic segmentation. The pre-trained fully convolutional network (FCN) model is used to perform semantic segmentation on the pre-processed image data; FCN extracts multi-scale features of the image through convolution and pooling operations, and upsamples the feature map to the original resolution through deconvolution operations to achieve end-to-end pixel-level classification; each pixel in the image data is divided into two categories: foreground or background through the FCN model, where the foreground includes the human body and seat belt area, and the background includes other areas such as buildings; semantic segmentation obtains the pixel-level mask of the foreground area, providing candidate areas for subsequent regional segmentation. The watershed algorithm is used for regional segmentation on the foreground image data obtained by semantic segmentation; the watershed algorithm aggregates pixels into different water areas by treating the image as a terrain surface and flooding it from the local minimum; the watershed algorithm calculates the aggregation metric based on the similarity between pixels, and classifies pixels with similarity greater than the set threshold into the same area; the region is continuously expanded through the region growing strategy until the foreground is segmented into multiple connected human body areas and seat belt areas, achieving regional level segmentation. For the segmented human region image, the open pose human posture estimation algorithm is used to extract the human skeleton data; open pose uses convolutional neural networks to learn the spatial position and connection relationship of the key points of the human body, and obtains the skeleton topology including the head, torso, and limbs; for the segmented seat belt region image, the Canny edge detection algorithm is used to extract the edge contour of the seat belt; the Canny algorithm smoothes the image through Gaussian filtering, calculates the gradient amplitude and direction, performs non-maximum suppression and double threshold detection, and obtains the pixel-level edge of the seat belt; by performing differential calculation of the gradient direction of the edge pixels of the seat belt, the shape characteristics and length information of the seat belt are obtained, and the contour data including the position and size of the seat belt are extracted. The extracted human skeleton data and the seat belt contour data are combined and associated in spatial position; by calculating the Euclidean distance between the skeleton key points and the seat belt contour line, the spatial position relationship between each part of the human body and the seat belt is determined, and the position of the seat belt worn on the human torso is determined; according to the combination of the human skeleton and the seat belt contour, the lifeline region data representing the human posture and the seat belt wearing position is generated.

[0012] Furthermore, a seat belt wearing recognition model based on a convolutional neural network is constructed, including: using the ResNet convolutional neural network as the backbone network of the seat belt wearing recognition model; ResNet solves the degradation problem of the difficulty of deep network training by introducing residual connections, and can build a deeper feature extractor; the backbone network consists of multiple convolutional layers, pooling layers and fully connected layers, and extracts the feature representation of the input image data at different scales and abstract levels through layer-by-layer convolution and downsampling operations, providing basic features for the subsequent attention mechanism. After the convolutional layer of the backbone network, the spatial attention mechanism is introduced to construct a spatial attention layer; the spatial attention adaptively enhances the regional features that are helpful for wearing status recognition by learning the importance weights of different regions of the image; the spatial attention layer fuses the building edge features and the scaffolding area mask, and calculates the spatial attention weight based on the fused feature map; by introducing the prior information of the building structure and the high-altitude operation scene, the spatial attention layer can focus on the key areas of seat belt wearing in a targeted manner. In the spatial attention layer, the Canny algorithm is used to perform edge detection on the input image and extract the contour edge features of the building. The Canny algorithm obtains clear and continuous building edges through Gaussian filtering, gradient calculation, non-maximum suppression and other steps. The extracted building edge features are multiplied element by element with the feature map output by the backbone network, so that the backbone network features are integrated with the building structure information to obtain a feature map containing the building edge prior, highlighting the indicative role of the building boundary on the wearing of seat belts.

[0013] In the spatial attention layer, the semantic segmentation algorithm SegNet is used to locate the scaffolding area; SegNet realizes pixel-level scaffolding area segmentation through the encoder-decoder architecture to obtain a binary mask map representing the scaffolding position; the scaffolding area mask map is added element by element to the feature map that integrates the building edge information, and the spatial position information of the scaffolding area is introduced into the feature representation; by integrating the prior information of the building structure and the high-altitude work scene, the spatial attention layer can better focus on the key area of ​​the seat belt wearing. After the convolution layer of the backbone network and before the spatial attention layer, the channel attention layer is set; the channel attention layer adaptively enhances the feature dimensions related to the wearing state recognition by learning the importance weights of different feature channels; the channel attention layer integrates the color features of the seat belt and the texture features of the reflective strip, and calculates the channel attention weight based on the fused feature map; by introducing the prior information of the key attributes of the seat belt, the channel attention layer can specifically enhance the distinctive seat belt feature representation.

[0014] In the channel attention layer, the HSV color space is used to convert the RGB image data into the HSV space; the HSV space describes the color through three channels: hue H, saturation S, and brightness V, which is more in line with the visual characteristics of the human eye; the color histograms of the two channels H and S are counted to obtain the color distribution characteristics of the seat belt area; the color histogram features are fused with the feature map output by the backbone network in the channel dimension to obtain a feature map that integrates the color prior information of the seat belt, highlighting the significant color attributes of the seat belt. In the channel attention layer, the gradient directional histogram HOG algorithm is used to extract the gradient information of the input image; HOG obtains the feature descriptor that describes the texture and edge structure of the object by counting the directional histogram of the pixel gradient in the local area; the gradient descriptor of the reflective strip area is obtained by the HOG algorithm, and is mapped to the same channel dimension as the feature map output by the backbone network to generate the reflective strip texture feature map; the reflective strip texture feature map is multiplied channel by channel with the feature map of the fused color information to obtain a feature map that integrates the color and reflective performance prior information of the seat belt, enhancing the expression of the key attributes of the seat belt.

[0015] Furthermore, S44 obtains a feature map that integrates prior information of building structures and high-altitude work scenes, and uses the SegNet algorithm to perform semantic segmentation on the input image data to achieve pixel-level scene understanding; SegNet adopts an encoder-decoder architecture, and maps the input image to a low-dimensional feature space through the encoder network to extract high-level semantic information of the image; the encoder consists of multiple convolutional layers and pooling layers, and gradually reduces the size of the feature map through downsampling operations, increases the receptive field, and captures global semantics; the decoder network is symmetrical with the encoder, and upsamples the low-dimensional feature map to the original image size through deconvolution operations, and combines the pooling index saved in the encoder stage to restore the object boundaries and spatial details; the decoder outputs a pixel-level segmentation result map, and each pixel is assigned a specific category label. The semantic segmentation result map generated by SegNet is binarized to extract the scaffolding area in the segmentation result; the pixel values ​​belonging to the scaffolding category in the segmentation result map are set to 1, and the remaining background pixel values ​​are set to 0 to generate a binary mask map with the same size as the input image; the scaffolding area mask map represents the spatial position and shape information of the scaffolding in the aerial work scene, providing scene prior for subsequent feature fusion.

[0016] The scaffolding area mask map is added element by element to the feature map that integrates the building edge prior information to achieve the feature level fusion of the scene prior. For each pixel position in the feature map, its feature value is added to the scaffolding mask pixel value at the corresponding position, and the spatial information of the scaffolding area is superimposed on the feature map. Through the element-by-element addition operation, on the basis of integrating the building edge information, the scaffolding position prior in the aerial work scene is further introduced to obtain a feature map representation that integrates the building structure and scene semantics. The feature map that integrates the scene prior information is input into the spatial attention layer, and the scaffolding area mask is used to guide the calculation of the spatial attention weight. The spatial attention layer adaptively adjusts the feature expression by learning the importance weights of different regions of the feature map. The scaffolding area mask is used as the prior guidance of the spatial attention, so that the attention mechanism gives a higher weight value to the key area of ​​the aerial work according to the scaffolding position information. By introducing the scene prior into the attention mechanism, the spatial attention layer can enhance the feature expression of the safety belt wearing area in a targeted manner and improve the recognition accuracy. Among them, in deep learning, low-dimensional feature space refers to a representation space with a lower dimension than the original input data. It extracts the essential characteristics and internal structure of the data by transforming and compressing the original data. High-level semantic information refers to the content understanding and conceptual expression in the image with a higher level of abstraction and closely related to the task objectives. Compared with low-level visual features such as texture and edges, high-level semantic information pays more attention to the overall meaning of the image, scene content, and object categories.

[0017] Further, S46, a feature map integrating the seat belt color prior information is obtained, including: using the HSV color space to convert the RGB image data into the HSV color space, the HSV space consists of three channels: hue, saturation, and value; extracting the image data of the H channel and the S channel, the H channel represents the type of color, and the S channel represents the purity of the color. These two channels contain the main information of the color, and the V channel mainly represents the brightness information; by converting the RGB image into the HSV space, different colors can be better described and distinguished, especially for targets with specific color attributes such as seat belts.

[0018] The color histogram statistics are performed on the H channel and the S channel respectively to obtain the color distribution of the seat belt area on these two channels; the color histogram obtains a vector representing the color distribution by counting the frequency of occurrence of each color value in the image; the abscissa of the histogram represents the color value, and the ordinate represents the number of pixels where the color value appears; for the seat belt area, the histogram of the H channel reflects the main color types and distribution of the seat belts, and the histogram of the S channel reflects the saturation distribution of the seat belt colors. These two histograms together describe the color attributes of the seat belts.

[0019] The color histograms of the H channel and the S channel are concatenated into a color histogram feature vector, which contains the color distribution information of the seat belt area; the color histogram feature vector is mapped to the same channel dimension as the backbone network output feature map through the fully connected layer to generate a seat belt color prior feature map; the function of the fully connected layer is to convert the color histogram feature vector into a dimension and scale compatible with the backbone network feature map for subsequent feature fusion; by learning the mapping relationship from the color histogram feature to the feature map channel dimension, the network can embed the seat belt color prior information into the feature representation. The generated seat belt color prior feature map is added element by element with the feature map output by the backbone network in the channel dimension to achieve the embedding and fusion of color features; by adding the color prior feature map to the backbone network feature map channel by channel, the seat belt color information can be combined with other visual features to obtain an enhanced feature representation containing color attributes; the fused feature map introduces the prior knowledge of seat belt color on the basis of retaining the original backbone network features, so that the network can pay more attention to and utilize the feature patterns related to the seat belt color.

[0020] Further, S47, a feature map that integrates the prior information of the seat belt color and reflective performance is obtained, and the gradient information of the image is extracted using the Histogram of Directed Gradients (HOG) algorithm on the input image data. The HOG algorithm generates a HOG descriptor that reflects the texture characteristics of the image by calculating the gradient amplitude and direction of pixels in different directions in the local area of ​​the image; the HOG descriptor divides the image into small cells, calculates the direction and amplitude of the pixel gradient in each cell, and then normalizes and combines the gradient information to obtain a feature vector representing the local texture characteristics; the HOG feature is sensitive to the edges and texture changes in the image, can effectively capture the structure and shape information of the object, and is widely used in target detection and recognition tasks.

[0021] The reflective strip area detection algorithm is used to locate the position of the seat belt reflective strip in the image. The reflective strip area detection algorithm can identify the area with reflective strip features in the image through edge detection, shape analysis and other methods; the HOG descriptor corresponding to the reflective strip area is extracted to obtain the feature representation of the reflective strip texture. The HOG features of the reflective strip area can describe the texture and structural properties of the reflective strip, providing prior information for subsequent feature fusion; by extracting the HOG features of the reflective strip area in a targeted manner, the characteristic pattern of the reflective performance of the seat belt can be more effectively captured and utilized. The extracted reflective strip HOG features are mapped to the same channel dimension as the output feature map of the backbone network through the fully connected layer to generate a reflective strip texture feature map with the same size as the feature map; the role of the fully connected layer is to convert the reflective strip HOG features into dimensions and scales compatible with the backbone network feature map for subsequent feature fusion; by learning the mapping relationship between the reflective strip HOG features and the channel dimension of the feature map, the network can embed the reflective strip texture prior information into the feature representation.

[0022] The reflective strip texture feature map is multiplied channel by channel with the feature map that integrates the seat belt color prior information. By multiplying each channel in the feature map with the corresponding reflective strip texture feature, the texture prior information of the reflective strip area is fused in the channel dimension of the feature map. Through the channel-by-channel multiplication operation of the feature map, the color information of the seat belt and the reflective strip texture information can be combined and interacted to obtain a comprehensive feature representation that includes both color and reflective performance attributes. The fused feature map, while retaining the original color features, further introduces the prior knowledge of the reflective strip texture, so that the network can more comprehensively utilize the various attribute information of the seat belt for identification. The feature map integrated with the prior information of seat belt attributes is input into the channel attention layer, and the calculation of channel attention weights is guided by the texture information of reflective strips. The channel attention layer adaptively adjusts the feature expression by learning the importance weights of different feature channels, and enhances the focus on key task-related features. Using the texture features of reflective strips as prior guidance, the channel attention layer can assign higher weights to corresponding channels according to the characteristic patterns of reflective performance, and highlight the characteristic expression of the reflective strip area. By combining the texture information of reflective strips with the channel attention mechanism, the network can more specifically enhance the feature channels that are sensitive to the reflective performance of seat belts and improve the recognition ability of reflective strips.

[0023] Furthermore, a small sample data set is constructed, including: collecting image data of seat belt wearing covering different construction stages, including civil construction stage and facade construction stage; the civil construction stage usually involves construction contents such as foundation and main structure, while the facade construction stage focuses on the appearance and decoration of the building; by obtaining image data of different construction stages, it is possible to cover all stages of the construction process and improve the diversity and representativeness of the data set. Collecting image data of seat belt wearing covering different types of work, including rebar workers, formwork workers and masonry workers; the working environment, operating characteristics and seat belt wearing requirements of workers in different types of work may be different; by obtaining image data of different types of work, it is possible to cover various types of work scenes at the construction site and improve the diversity and comprehensiveness of the types of work in the data set.

[0024] The acquired image data of workers wearing safety belts in different construction stages and different types of work are combined according to the two dimensions of construction stage category and type of work category; the image data is divided into scenes to obtain a small sample data set of safety belts corresponding to the cross-combination of each construction stage and each type of work; through the combination and division of the two dimensions, a more fine-grained scene-specific small sample data set can be obtained, which helps the model learn the characteristics of safety belt wearing in different scenes. The shooting timestamp information of the image in each small sample data set is extracted as the time metadata of the corresponding small sample data; the pixel coordinates of the worker's safety belt wearing area in the image are used to convert the pixel coordinates into the spatial position coordinates of the construction site through the coordinate mapping method; the converted position coordinates are used as the position metadata of the corresponding small sample data, indicating the spatial position information of the safety belt wearing behavior.

[0025] Each small sample data is associated and annotated with the corresponding time metadata (shooting timestamp) and location metadata (spatial location coordinates); a small sample data with a triple structure consisting of a small sample image, a shooting timestamp and a spatial location coordinate is generated with time-space labels; through the association of time-space labels, each small sample data contains multi-dimensional information such as image content, shooting time and spatial location, providing support for subsequent time-space analysis and modeling. Based on the small sample data with time-space labels, time-space clustering is performed according to the two index keys of construction stage category and work type category; through time-space clustering, small sample data with similar construction stage and work type attributes are organized together to construct a small sample data set with time-space index structure; through time-space indexing, each small sample data is mapped and associated with the corresponding construction stage and work type, which facilitates the rapid retrieval and access of related small sample data according to the construction stage and work type.

[0026] Further, S56, performing spatiotemporal clustering on small sample data, including: clustering small sample data using the ST-DBSCAN spatiotemporal clustering model: inputting the small sample data set with spatiotemporal labels into the ST-DBSCAN (Spatial-Temporal Density-Based Spatial Clustering of Applications with Noise) spatiotemporal clustering model; by setting density thresholds of the time dimension and the space dimension, dividing the spatiotemporally adjacent and density-connected small sample data into the same spatiotemporal cluster; ST-DBSCAN can discover sample groups that are closely related in time and space by considering the density connectivity of samples in both time and space dimensions to form spatiotemporal clusters. Among them, small sample data refers to the small number of labeled samples in each category in tasks such as target detection and image classification. Compared with traditional large sample learning, small sample learning aims to use limited labeled samples to train a model with strong generalization ability through meta-learning, data enhancement, transfer learning and other technologies. In the construction site safety supervision scenario, due to the low frequency of accidents and the complex and changeable construction environment, it is often difficult to obtain a large number of accident case images. Therefore, it is necessary to adopt a small sample learning method to mine effective feature representations from a small amount of historical accident data to achieve intelligent identification of safety hazards.

[0027] The construction stage factor and the work type factor are introduced into the density calculation formula of ST-DBSCAN, and different weights are set for small sample data of different construction stages and work types. The construction stage factor is set to α, which indicates the influence of different construction stages on sample similarity, and the value range is [0, 1]; the work type factor is set to β, which indicates the influence of different work types on sample similarity, and the value range is [0, 1]; the construction stage factor α and the work type factor β are introduced into the density calculation formula of ST-DBSCAN, and different weights are given to small sample data of different construction stages and work types; the original ST-DBSCAN density calculation formula is: density = count / (π*Eps^2*Eps_t), where count is the sample point The number of samples in the neighborhood of the sample point, Eps is the spatial neighborhood radius, and Eps_t is the temporal neighborhood radius; after introducing the construction stage factor and the type of work factor, the modified density calculation formula is: density = (α*count_stage+β*count_trade) / (π*Eps^2*Eps_t), where count_stage is the number of samples belonging to the same construction stage in the neighborhood of the sample point, and count_trade is the number of samples belonging to the same type of work in the neighborhood of the sample point; by introducing the construction stage factor α and the type of work factor β in the density calculation, different weights are given to samples of different construction stages and types of work, so that the clustering model can adjust the similarity measure between samples according to the characteristics of the scene and type of work. According to the actual situation of the construction site, different α and β weight values ​​are set for small sample data of different construction stages and types of work; for different construction stages, such as the civil construction stage and the facade construction stage, different α values ​​can be set according to their influence on the safety belt wearing detection; for different types of work, such as reinforcement workers, formwork workers and masonry workers, different β values ​​can be set according to their working characteristics and safety belt wearing requirements; by dynamically adjusting the values ​​of α and β, the clustering model can adapt to the characteristics of different construction stages and types of work, thereby improving the scenario adaptability of the clustering results.

[0028] The spatiotemporal clusters generated by clustering are indexed to generate cluster-level spatiotemporal index items, and the spatiotemporal clusters are indexed using a B+ tree. For each spatiotemporal cluster generated by clustering, the common time label and location label of all small sample data in the cluster are obtained; the time label represents the time range of the spatiotemporal cluster, which can be determined by obtaining the minimum and maximum timestamps of the samples in the cluster; the location label represents the spatial range of the spatiotemporal cluster, which can be determined by obtaining the minimum and maximum spatial coordinates of the samples in the cluster; by obtaining the common time label and location label, the boundaries and range of each spatiotemporal cluster in time and space can be determined.

[0029] The construction phase category and work type category corresponding to each spatiotemporal cluster are associated to generate cluster-level spatiotemporal index items; the construction phase category and work type category of the spatiotemporal cluster are determined by analyzing the construction phase and work type attributes of the samples within the cluster; if the construction phase and work type attributes of the samples within the cluster are consistent, they are directly used as the construction phase and work type labels of the spatiotemporal cluster; if there are differences in the construction phase and work type attributes of the samples within the cluster, the main construction phase and work type labels of the spatiotemporal cluster can be determined by voting or weighted average. The cluster-level spatiotemporal index item contains the time range, spatial range, and corresponding construction stage and type of work information of the spatiotemporal cluster; the time range can be represented by the start and end values ​​of the timestamp, such as [start_time, end_time]; the spatial range can be represented by the minimum and maximum values ​​of the spatial coordinates, such as [min_x, min_y, max_x, max_y]; the construction stage and type of work information can be represented by the corresponding category labels, such as "civil construction stage" and "rebar worker"; the time range, spatial range, construction stage and type of work information are combined into a concise index item to facilitate subsequent indexing and query.

[0030] Based on the generated cluster-level spatiotemporal index items, the spatiotemporal clusters are indexed using a B+ tree. The B+ tree is a multi-way balanced search tree suitable for efficient indexing and querying of large-scale data. The combination of construction phases and types of work is used as the key value of the B+ tree, such as "Civil Construction Phase_Rebar Worker" and "Facade Construction Phase_Formwork Worker". The leaf nodes of the B+ tree store pointers or references to all small sample data in the corresponding spatiotemporal cluster for fast access to the data in the cluster. The internal nodes of the B+ tree store key values ​​and subtree pointers for navigation and positioning to the corresponding leaf nodes. Given the combined key value of the construction phase and type of work, the corresponding spatiotemporal cluster can be quickly located through the B+ tree index. Starting from the root node of the B+ tree, comparison and navigation are performed based on the key value, and the corresponding leaf nodes are recursively traversed downward. In the leaf node, the pointers or references of all small sample data in the corresponding spatiotemporal cluster are obtained, and the data in the cluster is directly accessed through the pointers or references. The balance and multi-way characteristics of the B+ tree ensure that the query time complexity is O(logn), where n is the number of spatiotemporal clusters, and efficient data retrieval and access are achieved. When new small sample data is added, it is necessary to divide it into the corresponding spatiotemporal cluster according to its spatiotemporal attributes, and update the cluster-level spatiotemporal index items; if the new sample causes the time range or spatial range of the spatiotemporal cluster to change, the time range and spatial range in the index item need to be updated accordingly; when the number or distribution of samples in the spatiotemporal cluster changes significantly, it may be necessary to re-cluster and re-index to maintain the effectiveness and efficiency of the index; regularly balance and optimize the B+ tree index, such as node splitting and merging, to maintain the balance of the tree and query performance.

[0031] Preferably, the spatiotemporal data of the small sample data set is preprocessed to extract spatiotemporal metadata and construct spatiotemporal feature vectors; the construction stage weight and the type of work weight are introduced to improve the ST-DBSCAN algorithm and realize the clustering of spatiotemporal feature vectors; the key features of each spatiotemporal cluster are extracted to generate cluster-level spatiotemporal index items; the spatiotemporal cluster index items are organized using B+ trees to support fast retrieval based on spatiotemporal semantics. The shooting time, location coordinates, construction stage and type of work information of the image are extracted to form spatiotemporal metadata; the time window is divided to extract the one-hot coding of the construction stage and the one-hot coding of the type of work. The construction stage weight and the type of work weight are introduced to give different importance to the data of different stages and types of work; in the density calculation formula, the neighborhood density of the time dimension, space dimension, construction stage dimension and type of work dimension are considered at the same time; by setting a multidimensional density threshold, the data points with adjacent spatiotemporal and similar characteristics are divided into the same spatiotemporal cluster. The time window, location coordinate range, main construction phase and main work type of the spatiotemporal cluster are extracted to form cluster-level index items. The cluster-level index items are organized using a B+ tree, with the combination of construction phase and work type as the key value.

[0032] Furthermore, S6 uses a small sample data set with spatiotemporal labels to perform few-shot learning and meta-learning training on the constructed seat belt wearing recognition model, extracts scene information from the input image and retrieves target small sample data: uses image processing and computer vision techniques, such as convolutional neural network (CNN) or scene parsing algorithm, to extract scene information from the input image data, including construction stage category and work type category; based on the extracted scene information, searches through the B+ tree index structure in the small sample data set with spatiotemporal index structure to find the target small sample data that matches the input image scene; the B+ tree index structure uses the combination of construction stage and work type as the index key, and locates the storage location of the target small sample data that matches the input scene by traversing the internal nodes and leaf nodes of the B+ tree; from the retrieved target small sample data, extracts multiple labeled images containing positive and negative samples of seat belt wearing to form a support sample set for subsequent model fine-tuning.

[0033] Few-shot Learning using Prototypical Networks: Input the support samples into the pre-trained seat belt wearing recognition model, and use the model's feature extractor to obtain the feature vectors of the support samples; cluster the feature vectors according to the seat belt wearing status labels of the support samples (positive samples or negative samples) to obtain the prototype vectors of each category; the prototype vector is the average of the feature vectors of samples in the same category, representing the center point or representative vector of the category; the prototype vector of the positive sample represents the feature representation of seat belt wearing, and the prototype vector of the negative sample represents the feature representation of not wearing seat belts; select the remaining samples from the small sample training set as query samples, input them into the seat belt wearing recognition model, and obtain the feature vectors of the query samples; calculate the correlation between the query sample feature vector and each The Euclidean distance between the prototype vectors of the categories measures the similarity between the query sample and each category; the Euclidean distance is converted into a probability distribution through the softmax function to obtain the probability that the query sample belongs to each category; the softmax function maps the distance value to the range of (0, 1) and ensures that the sum of the probabilities of each category is 1; the larger the probability value, the closer the query sample is to the prototype vector of the category, and the greater the possibility of belonging to the category; the category with the largest probability value is selected as the predicted category of the query sample, that is, the predicted result of the seat belt wearing status; the classification loss of the query sample on the support sample is calculated, that is, the difference between the predicted category and the true category; the commonly used loss functions include cross-entropy loss (Cross-entropy loss Loss) or squared error loss (Mean Squared ErrorLoss); the loss function measures the classification performance of the model on small sample data. The smaller the loss value, the better the model's fitting ability on the supported samples; use the gradient descent algorithm, such as stochastic gradient descent (SGD) or Adam optimizer, to fine-tune the model parameters according to the calculated classification loss; use the back propagation algorithm to calculate the gradient of the loss function to the model parameters, and update the model parameters according to the direction and size of the gradient; through multiple iterative optimizations, the classification performance of the model on small sample data is continuously improved, and rapid adaptation to new scenarios is achieved.

[0034] Preferably, the small sample seat belt wearing recognition method based on the prototype network improves the classification performance and generalization ability of the model under small sample conditions by introducing the attention mechanism and the dynamic prototype adjustment strategy. The pre-trained recognition model is used to extract the supporting sample features, and the initial prototype vector is generated by clustering; the channel attention and spatial attention mechanisms are introduced in the feature extraction network to enhance the feature representation ability; according to the distance between the query sample and the prototype vector, the position of the prototype vector is dynamically adjusted to adapt to the query sample distribution; the classification loss of the query sample is calculated, and the model parameters are fine-tuned using the gradient descent algorithm. The channel attention module adaptively adjusts the feature responses of different channels by learning the importance weights between feature channels. The spatial attention module highlights the significant areas related to the task by learning the importance weights of different positions on the feature map. The dynamic prototype adjustment strategy includes: for the prototype vector closest to the query sample, it is moved toward the query sample by weighted averaging; for the prototype vector far from the query sample, it is moved away from the query sample by weighted averaging. The classification loss of the query sample is calculated using the cross entropy loss function, and the model parameters are fine-tuned using the gradient descent algorithm.

[0035] Furthermore, S63 uses Prototypical Network and Model-Agnostic Meta-Learning (MAML) algorithm for meta-learning training, and uses small sample data sets under different construction stages and types of work as different meta-learning tasks to form a meta-learning task set; each meta-learning task corresponds to a small sample data set under a specific construction stage and type of work, representing different scenarios and conditions; the meta-learning task set covers seat belt wearing recognition scenarios under different construction environments, reflecting the diversity and complexity of the tasks; for each task in the meta-learning task set, the corresponding small sample data set is divided into a support set and a query set; the support set is used for inner-loop training of Prototypical Network as training data for the model to adapt to the task; the query set is used to evaluate the generalization performance of the model on the task, as test data for the model on new samples; the division ratio of the support set and the query set can be adjusted according to the specific task and data volume, and the common ratio is 5:1 or 8:2, etc.; Prototypical Network is used as the base learner of the MAML meta-learning method, and inner-loop training is performed on each meta-learning task; the support set of the input task is used, and the Prototypical The feature extractor of the network obtains the feature vector of the support sample; calculates the prototype vector of each category (seat belt wearing and not wearing), that is, the average value of the feature vectors of samples in the same category; inputs the query set, obtains the feature vector of the query sample, and calculates the Euclidean distance between the query sample and the prototype vector of each category.

[0036] The Euclidean distance is converted into a probability distribution through the softmax function to obtain the predicted category probability of the query sample; the classification loss of the query sample is calculated, that is, the cross entropy loss between the predicted probability and the true label; the classification loss is used to classify the Prototypical The parameters of the PrototypicalNetwork are updated by gradient descent to obtain the updated model parameters for the task; the updated model parameters on all meta-learning tasks are summarized, the sum of the query losses for each task is calculated, and the gradient is calculated for them; the updated model parameters on all tasks are summarized to obtain a set of model parameters optimized for different tasks; on the query set of each task, the prediction probability and classification loss of the query samples are calculated using the updated model parameters; the query losses of all tasks are summed to obtain the total query loss on the entire meta-learning task set; the gradient of the total query loss is calculated to obtain the meta-gradient, which represents the model performance gradient on the entire task set; the initial parameters of the PrototypicalNetwork are updated by outer-loop using the MAML meta-gradient descent algorithm; the initial parameters of the PrototypicalNetwork are updated by gradient descent according to the direction and size of the meta-gradient; the updated initial parameters represent a universal model parameter applicable to all tasks, which has the ability to quickly adapt across tasks; the outer-loop update process aims to optimize the initial parameters of the model so that it can quickly adapt and generalize on new tasks; the above inner-loop training and outer-loop update process are repeated until the PrototypicalNetwork is trained. The initial parameters of the network converge; the inner-loop training and outer-loop update are performed iteratively multiple times, so that the initial parameters of the model are continuously optimized and converged. The converged initial parameters represent a Few-shot Learning meta-model with fast cross-task learning capabilities; the meta-model can quickly adapt to new seat belt wearing recognition tasks under new construction stages and types of work through a small number of supporting samples; the optimized Few-shot Learning meta-model is saved for fast adaptation and generalization of small sample data under new construction stages and types of work; the trained meta-model parameters are saved as a general model for seat belt wearing recognition tasks; the meta-model and a small number of supporting samples are used for fast adaptation and fine-tuning under new construction stages and types of work; through the cross-task learning capabilities of the meta-model, fast generalization and efficient recognition of seat belt wearing recognition tasks in new scenarios are achieved.

[0037] Preferably, the small sample safety accident image recognition method based on prototype network and meta-learning improves the generalization performance and rapid adaptability of the model under different construction stages and types of work by introducing task attention mechanism and adaptive step size adjustment strategy. Small sample data sets under different construction stages and types of work are used as meta-learning tasks to construct meta-learning task sets; prototype network is used as base learner, prototype vector is calculated on support set, and classification loss is calculated on query set; initial parameters of prototype network are optimized through inner loop training and outer loop update to obtain meta-model suitable for all tasks; meta-model is fine-tuned with a small number of support samples to quickly adapt to new safety accident recognition tasks. Task attention mechanism is introduced to adaptively adjust prototype vectors between different tasks through attention weights. In the inner loop training, the parameters of prototype network are updated task-specifically using gradient descent algorithm. Adaptive step size adjustment strategy is introduced to dynamically adjust the step size of outer loop update according to the query loss of each task. The initial parameters of prototype network are optimized on the query loss of all tasks using MAML meta-gradient descent algorithm.

[0038] Compared with the prior art, the advantages of this application are:

[0039] This application uses an image segmentation algorithm to process image data, and accurately locates the human body area and seat belt area in the image through steps such as semantic segmentation, region generation, and skeleton extraction. At the same time, the timestamp and location information of the image are extracted to construct a spatiotemporal index. It can accurately identify the location of the human body and seat belts, and realize real-time association and tracking of people and seat belts through spatiotemporal indexes.

[0040] This application constructs a seat belt wearing recognition model based on convolutional neural network, and introduces spatial attention mechanism and channel attention mechanism. Among them, spatial attention integrates the prior information of building edges and scaffolding areas, so that the model can focus on the key areas of high-altitude operations; channel attention integrates the feature expression of key attributes such as seat belt color and reflective strips, enhancing the model's sensitivity to seat belt features. The recognition model can make full use of scene prior knowledge and seat belt attribute information to achieve a high recognition accuracy in complex and changeable high-altitude working environments.

[0041] This application obtains small sample data sets for different construction stages and types of work, and constructs a spatiotemporal index through spatiotemporal clustering, so that the small sample data is structured in a form indexed by construction stage and type of work. When training the model, a Few-shot Learning method based on spatiotemporal index is adopted. The model is fine-tuned by extracting small sample data that matches the scene to be identified, so that the model can quickly adapt to the new construction environment with a small number of samples. At the same time, a meta-learning method is used, and different construction stages and types of work are used as different meta-learning tasks. By optimizing the initial parameters of the model, it has the ability to learn quickly across tasks.

[0042] This application not only identifies the seat belt wearing status, but also obtains the corresponding location coordinates based on the spatiotemporal index, and associates the seat belt wearing status with the location information. Risk assessment is performed through a decision tree algorithm, and a graded warning mechanism is established based on the seat belt usage and location. The location of the safety hazard is located in real time, and the corresponding warning is triggered according to the danger level. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is an exemplary flow chart of a lifeline warning method for aerial work according to some embodiments of the present application;

[0044] Figure 2 is an exemplary flow chart of obtaining lifeline area data according to some embodiments of the present application;

[0045] Figure 3 is an exemplary flow chart of obtaining a feature map using a seat belt wearing recognition model according to some embodiments of the present application;

[0046] Figure 4 is an exemplary flow chart of constructing a small sample data set with a spatiotemporal index structure according to some embodiments of the present application;

[0047] Figure 5 This is an exemplary flow chart of fine-tuning seat belt wearing recognition model parameters according to some embodiments of the present application. DETAILED DESCRIPTION

[0048] The method and system provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0049] Figure 1This is an exemplary flow chart of a high-altitude work lifeline warning method shown in some embodiments of the present application, a high-altitude work lifeline warning method, including: acquiring image data containing images of workers and images of construction areas; using an image segmentation algorithm to segment the image data, extracting human body areas and lifeline areas containing safety belts in the image data; extracting the timestamp and location information of the image data as metadata, and constructing a spatiotemporal index of the image data based on the metadata; constructing a seat belt wearing recognition model based on a convolutional neural network to identify the seat belt wearing status of workers; in the constructed seat belt wearing recognition model, setting a spatial attention mechanism and a channel attention mechanism; wherein the spatial attention mechanism embeds prior information of building edges and scaffolding areas to guide the model to pay attention to the corresponding areas; in The weights of the seat belt color and reflective strip are set in the channel attention mechanism; small sample data of seat belts of workers in different construction stages and different types of work are obtained, and the small sample data are divided into scenes according to the construction stage and type of work to construct a small sample data set; the small sample data set is combined with the corresponding metadata to obtain a small sample data set with spatiotemporal labels; the constructed seat belt wearing recognition model is trained by Few-shot Learning and meta-learning using the small sample data set with spatiotemporal labels; the extracted human body area and lifeline area are input into the trained seat belt wearing recognition model for prediction to identify the seat belt wearing status; the position coordinates of the image data are obtained according to the constructed spatiotemporal index, and combined with the identified seat belt wearing status, risk assessment is performed through a decision tree to trigger the corresponding warning level.

[0050] According to the layout and monitoring requirements of the construction site, choose a suitable location to install high-definition cameras, such as tower cranes, scaffolding, temporary buildings, etc.; the high-definition camera should have sufficient resolution and image quality to ensure that clear images of workers and construction areas are captured; the camera's field of view should cover the main work area and personnel activity area to reduce blind spots and dead angles; the camera should be installed at an appropriate height and angle to capture both the full-body image of the workers and the overall environment of the construction area. According to the lighting conditions and shooting requirements of the construction site, set the camera's exposure, white balance, focus and other parameters to ensure image clarity and color reproduction; set the camera's shooting frame rate and resolution, and select the appropriate frame rate and resolution to optimize data transmission and storage while meeting the image quality requirements; set the camera's trigger conditions, such as motion detection, area intrusion, etc., to automatically trigger shooting when workers enter the designated area or move; you can set timed shooting or manual shooting mode to obtain image data in a specific time period or specified scene.

[0051] The median filtering algorithm is used to denoise the image data: the size of the original image data is 1024x768, and a 5x5 filter window is selected to perform median filtering on the image; for each pixel in the image, 25 pixel values ​​within its 5x5 neighborhood are extracted, sorted by size, and the 13th value is selected as the new value of the pixel; the filter window slides pixel by pixel on the image until all pixels are processed to obtain the denoised image data.

[0052] The histogram equalization algorithm is used to enhance the denoised image: for the denoised image data, its grayscale histogram is calculated, and the frequency of each grayscale level in the range of 0-255 is counted; the grayscale histogram is normalized to obtain the probability density function of the grayscale level, such as [0.05, 0.1, 0.15,......., 0.02]; the cumulative distribution function of the grayscale level is calculated, and the original grayscale value is mapped to the equalized grayscale value, such as the grayscale value 100 is mapped to 150; based on the equalized grayscale value, the enhanced image data is generated to improve the contrast and brightness of the image.

[0053] Figure 2 This is an exemplary flowchart for obtaining lifeline area data according to some embodiments of the present application, and a pre-trained fully convolutional network FCN semantic segmentation model is used to perform semantic segmentation on the pre-processed image data: a pre-trained FCN-8s model is selected, which is trained on the Pascal VOC dataset and can recognize 20 common object categories; the pre-processed image data is adjusted to the input size of 512x512 of the FCN-8s model and normalized; the image data is input into the FCN-8s model, and after 5 convolutional layers and 3 upsampling layers, a pixel-level classification result with a size of 512x512 is obtained; the classification result is thresholded, the pixels of the human body and the seat belt are marked as foreground, and the pixels of the building are marked as background, to obtain a binary foreground mask image.

[0054] The watershed algorithm is used to perform regional segmentation on the foreground image data after semantic segmentation: the gradient amplitude image is calculated for the foreground image data, and the gradient of the image is calculated using the Sobel operator to obtain a gradient image with a size of 512x512; the gradient image is thresholded, and pixels with a gradient amplitude greater than 20 are marked as region boundaries, and the remaining pixels are marked as inside the region; regional growth is performed on the pixels inside the region, and adjacent pixels with a gradient amplitude less than 10 are aggregated into the same region to obtain mask images of the human body region and the seat belt region; morphological processing is performed on the regional mask image to remove small regions with an area less than 100 pixels and merge adjacent regions with an area greater than 1000 pixels.

[0055] The open pose human posture estimation algorithm is used to extract human skeleton data from human region image data: the human region image data is adjusted to the input size of 368x368 of the open pose model and normalized; the image data is input into the open pose model, and features are extracted through the VGG-19 convolutional network to obtain a feature map with a size of 46x46; the feature map is decoded and post-processed to obtain the coordinates of 18 key points of the human body, such as the head (100, 50), left shoulder (120, 80), right shoulder (140, 80), etc.; according to the coordinates of the key points, the human skeleton topology structure is generated, such as the head is connected to the torso, the left and right shoulders are connected to the torso, etc., to obtain the human skeleton data. The Canny edge detection algorithm is used to extract the seat belt edge on the seat belt area image data: the seat belt area image data is grayed and smoothed with a 3x3 Gaussian filter to reduce noise interference; the Sobel operator is used to calculate the gradient of the image to obtain a gradient amplitude image and a gradient direction image with a size of 200x200; the gradient amplitude image is non-maximum suppressed, and the pixels with the largest local gradient are marked as edge candidate points, and the remaining pixels are set to 0; the edge candidate points are double-thresholded, and pixels with gradient amplitudes greater than 80 are marked as strong edges, pixels with gradient amplitudes between 40 and 80 are marked as weak edges, and pixels with gradient amplitudes less than 40 are eliminated; the strong edges and weak edges are connected. Analysis is performed on the connection, and the weak edges with strong edges in the 8-neighborhood are retained, and the remaining weak edges are eliminated to obtain the final seat belt edge contour data.

[0056] The human skeleton data and the seat belt contour data are combined to obtain the lifeline area data: the human skeleton data and the seat belt contour data are superimposed on the original image, and their relative positions in the image are determined according to the coordinate positions of the skeleton key points and contour pixels; the spatial position relationship between the human torso skeleton and the seat belt contour is analyzed, and the average distance and angle between the torso skeleton line and the seat belt contour line are calculated, such as the average distance is 10 pixels and the angle is 5 degrees; according to the thresholds of the distance and the angle, it is judged whether the seat belt is worn correctly, such as the average distance is less than 20 pixels and the angle is less than 10 degrees, it is considered that the seat belt is worn correctly; a lifeline area mask image representing the human posture and the seat belt position is generated, the torso skeleton line and the seat belt contour line are marked as white, and the remaining areas are marked as black, and morphological processing is performed to obtain the final lifeline area data.

[0057] Extract the timestamp and location information of the image data as metadata: A set of image data was collected at the construction site, with a total of 1,000 images, each with a size of 1920x1080 pixels; the file name of each image contains the timestamp information in the format of "YYYYMMDD_HHMMSS", such as "20220101_120000" means 12:00:00 on January 1, 2022; the EXIF ​​information of each image contains the GPS location information in the format of "longitude, latitude", such as "120.123456, 30.123456" means 120.123456 degrees east longitude and 30.123456 degrees north latitude; extract the timestamp and location information of each image, and construct a metadata list, such as [(20220101_120000, 120.123456, 30.123456),.......].

[0058] Construct a spatiotemporal index of image data based on metadata: sort the metadata list by timestamp to construct a time index, such as [20220101_120000, 20220101_120005, ...]; cluster the metadata list according to location information to construct a spatial index, such as [area A (120.1234, 30.1234), area B (120.5678, 30.5678), ....]; combine the time index and the spatial index to construct a spatiotemporal index structure, such as [(20220101_120000, area A), (20220101_120005, area B), ...].

[0059] Figure 3This is an exemplary flow chart of obtaining feature maps using a safety belt wearing recognition model as shown in some embodiments of the present application. A small sample data set is constructed. During the civil construction phase, a mobile phone and a surveillance camera are used to collect image data of the construction site. A total of 1,000 images are collected, of which 500 are images of rebar work, 300 are images of formwork work, and 200 are images of masonry work; during the facade construction phase, drones and high-altitude cameras are used to collect image data of the construction site. A total of 800 images are collected, of which 400 are images of rebar work, 250 are images of formwork work, and 150 are images of masonry work. The collected image data is manually screened and classified, and the work images of rebar workers, formwork workers, and masonry workers are identified based on the features of the workers' tools, clothing, and operating areas; and finally 900 images of rebar work, 550 images of formwork work, and 350 images of masonry work are obtained. The 500 steel bar work images, 300 formwork work images, and 200 masonry work images in the civil construction stage are divided into three small sample data sets: civil construction-steel, civil construction-formwork, and civil construction-masonry; the 400 steel bar work images, 250 formwork work images, and 150 masonry work images in the facade construction stage are divided into three small sample data sets: facade-steel, facade-formwork, and facade-masonry; finally, 6 safety belt small sample data sets are obtained, each of which corresponds to a combination of construction stage and work type.

[0060] For an image in the civil engineering-rebar small sample data set, its shooting timestamp "20220101_120000" is extracted and used as the time metadata of the image; target detection is performed on the image, and the pixel coordinates of the seat belt wearing area in the image are located as (100, 200, 300, 400), that is, the upper left corner coordinates are (100, 200), and the lower right corner coordinates are (300, 400); according to the CAD plan of the construction site, the pixel coordinates (100, 200, 300, 400) are mapped to spatial position coordinates (10.5m, 20.8m, 15.6m, 25.9m), which are used as the position metadata of the image; for the images in the other 5 small sample data sets, the above steps are repeated to extract the time metadata and position metadata. For an image in the civil engineering-rebar small sample data set, its image data, time metadata "20220101_120000" and location metadata (10.5m, 20.8m, 15.6m, 25.9m) are associated to generate a triple structure (image_data, "20220101_120000", (10.5m, 20.8m, 15.6m, 25.9m)); add spatiotemporal labels to the triple, mark it as belonging to the civil engineering construction stage and the rebar worker type, and generate labeled small sample data (image_data, "20220101_120000", (10.5m, 20.8m, 15.6m, 25.9m), "civil engineering", "rebar worker"); repeat the above steps for the images in the other 5 small sample data sets to generate small sample data with triple structure and spatiotemporal labels.

[0061] Based on the small sample data with spatiotemporal labels, a small sample data set with spatiotemporal index structure is constructed: all the data in the six small sample data sets are sorted according to the time label and location label to obtain a small sample data sequence arranged in time and space order; the small sample data sequence is input into the ST-DBSCAN model, and the neighborhood radius of the time dimension is set to 1 day, the neighborhood radius of the space dimension is set to 10 meters, the minimum number of cluster samples is set to 20, the time weight is set to 0.6, and the space weight is set to 0.4; for the small sample data in the civil construction stage, the density calculation formula is multiplied by a coefficient of 1.2, and for the small sample data in the facade construction stage, the density calculation formula is multiplied by a coefficient of 0.8; for the small sample data of the reinforcement worker type, the density calculation formula is multiplied by a coefficient of 1.5, and for the small sample data of the formwork worker type, the density calculation formula is multiplied by a coefficient of 0. The density calculation formula is multiplied by a coefficient of 1.0. For the small sample data of masonry workers, the density calculation formula is multiplied by a coefficient of 0.5. After the ST-DBSCAN algorithm is run, 8 spatiotemporal clusters are obtained, each cluster contains 20-150 images, and the images in the cluster are clustered in time and space. For cluster 1, its time label range is "20220101-20220105", the position label range is (0m-50m, 0m-80m), the associated construction stage is civil engineering, and the associated type of work is reinforcement worker. The spatiotemporal index items of cluster 1 are generated ("20220101-20220105", (0m-50m, 0m-80m), "civil engineering", "rebar worker"); for the other 7 clusters, the above steps are repeated to generate the corresponding spatiotemporal index items and construct a B+ tree index. For a newly collected image, extract its time label "20220103_090500" and location label (15.2m, 30.6m), perform spatiotemporal neighborhood search on the B+ tree, and find the spatiotemporal neighboring clusters 1, 2, and 3; calculate the spatiotemporal density of the new image and clusters 1, 2, and 3, and obtain density values ​​of 0.8, 0.6, and 0.4, respectively. Divide the new image into cluster 1 with the highest density, and update the spatiotemporal index items of cluster 1 to ("20220101-20220105", (0m-50m, 0m-80m), "civil engineering", "rebar worker"); according to the construction schedule, the civil construction phase ends after "20220105" and the facade construction phase begins. Accordingly, update the construction phase labels of each spatiotemporal cluster and reconstruct the B+ tree index.

[0062] Figure 4It is an exemplary flow chart of obtaining feature maps using a seat belt wearing recognition model as shown in some embodiments of the present application, and constructing a seat belt wearing recognition model based on a convolutional neural network, including: Input data: 1,000 color images of aerial workers wearing seat belts collected from construction sites, and the image size is uniformly scaled to 224x224. Selection and composition of the backbone network: The pre-trained ResNet-50 is used as the backbone network, which consists of 49 convolutional layers, 1 global average pooling layer and 1 fully connected layer. The number of channels of the convolutional layer ranges from 64 to 2048, and the receptive field size ranges from 3x3 to 7x7. The multi-scale features of the image are extracted by layer-by-layer superposition. A batch normalization (BN) layer and a ReLU activation function are added after each convolutional layer to improve the nonlinear expression ability of the network. A maximum pooling layer is added after some convolutional layers to downsample the feature map and increase the receptive field. The last convolutional block (conv5_x) of ResNet-50 outputs a 14x14 feature map with 2048 channels, which integrates high-level semantic information of the input image.

[0063] Construction of spatial attention layer: After the conv5_x convolution layer of ResNet-50, a spatial attention layer is inserted to fuse the prior information of the building structure and the scaffolding area. The spatial attention layer consists of three parts: building edge feature extraction, scaffolding area mask generation, and attention weight calculation. The Canny algorithm is used to extract building edge features, the SegNet semantic segmentation model is used to generate the scaffolding area mask, and the convolution layer and Sigmoid activation function are used to calculate the attention weight. The output of the spatial attention layer is an attention feature map of 14x14x2048 with the same size as the conv5_x feature map. Extraction and fusion of building edge features: The Canny algorithm is used to perform edge detection on the 224x224 input image to generate a binary edge map E of the building outline. The high and low thresholds of the Canny algorithm are set to 100 and 200 respectively, and an 8-neighborhood connected domain analysis is performed on the detected edges to extract connected edge curves. The building edge binary map E is upsampled to 14x14 and aligned with the output feature map F_c of conv5_x in the spatial dimension to obtain E_u. The upsampled edge map E_u is multiplied element by element with the feature map F_c to obtain a 14x14x2048 feature map F_e that integrates the building edge information. The process of element-by-element multiplication of the building edge and the feature map can be expressed as: F_e(i, j, c) = F_c(i, j, c)*E_u(i, j), where i, j are spatial coordinates and c is the channel index. The F_e feature map that integrates the building edge highlights the activation response of the building contour area in the image, suppresses the background interference of the non-building area, and makes the model pay more attention to the structural characteristics of the building.

[0064] Generation of scaffolding region mask and fusion of prior information: The pre-trained SegNet semantic segmentation model is used to segment the scaffolding region of the input image. The SegNet model adopts an encoder-decoder architecture. The encoder uses 13 convolutional layers of VGG16 to extract features, and the decoder uses 13 deconvolutional layers to restore spatial details. The encoder downsamples the input image by 8 times to 28x28 and extracts high-level semantic feature maps with 512 channels. The decoder upsamples the 28x28 feature map layer by layer to 224x224, and uses 1x1 convolution to reduce the number of channels to 2, corresponding to the scaffolding and background categories. The 224x224x2 tensor output by the decoder is activated by Softmax to obtain the probability of each pixel belonging to the scaffolding and background. The probability map is thresholded, and pixels with scaffolding probabilities greater than 0.5 are set to 1, and the remaining pixels are set to 0 to generate the scaffolding binary mask map M. Fusion of scaffolding mask map and building edge feature map: Downsample the 224x224 scaffolding mask map M to 14x14 and align it with the building edge feature map F_e in the spatial attention layer in the spatial dimension. Add the downsampled scaffolding mask map M_d to F_e element by element to obtain the feature map F_a that integrates the scene prior information. The process of adding the scaffolding mask and the feature map element by element can be expressed as: F_a(i, j, c) = F_e(i, j, c) + M_d(i, j). By adding the mask and the feature map, the prior information of the scaffolding area is embedded into the building edge feature map, highlighting the key area of ​​high-altitude operations. The scene prior feature map guides the calculation of attention weights: F_a integrated with the scene prior is input into the spatial attention layer, and the spatial attention weight is calculated through convolution and sigmoid function. A 3x3 convolutional layer is used to reduce the number of channels of F_a from 2048 to 49, and then a 7x7 attention weight matrix W_s is obtained by Sigmoid activation. The attention weight W_s corresponds to F_a one-to-one in the spatial dimension, characterizing the importance of each area in the feature map. Adaptive average pooling is performed on F_a to obtain a 2048-dimensional feature vector, which is weighted by multiplying it by W_s to obtain a weighted feature vector. The weighted feature vector is multiplied element-by-element with F_a to obtain the scene prior feature map F_{sa} with spatial attention added. Under the guidance of the attention weight, the model adaptively enhances the feature activations of key areas such as the scaffolding and weakens the interference of the background.

[0065] After the convolution layer of the backbone network and before the spatial attention layer, a channel attention layer is set. The channel attention layer fuses the color features of the seat belts and the texture features of the reflective strips, and calculates the channel attention weights based on the fused feature map to enhance the expression of the key attributes of the seat belts; Setting of the channel attention layer: insert a channel attention layer after the conv5_x convolution layer of ResNet-50 and before the spatial attention layer. The channel attention layer is used to fuse the color features of the seat belts and the texture features of the reflective strips, and calculate the channel weights based on the fused feature map. The channel attention layer includes three submodules: color feature extraction, texture feature extraction, and channel weight calculation. The output of the channel attention layer is a weight vector with the same number of channels as the input feature map, which is used to weight the importance of different channels.

[0066] Extraction and fusion of seat belt color features: Convert the 224x224 RGB input image to the HSV color space, and extract the image data of the H channel and the S channel. Perform 16-bin color histogram statistics on the H channel and the S channel respectively to obtain 16-dimensional color distribution histograms H_h and S_h. Concatenate H_h and S_h into a 32-dimensional color histogram feature vector C_h to characterize the color distribution of the seat belt area. Input C_h into a 32x2048 fully connected layer to map the color features to the same 2048 dimensions as the conv5_x feature map. Restore the 2048-dimensional color feature vector to a 14x14x2048 seat belt color prior feature map F_c. Add F_c to the output feature map F_x of conv5_x element by element in the channel dimension to obtain the feature map F_{xc} that fused the color information. The process of adding color features and convolution feature maps channel by channel can be expressed as: F_{xc}(i, j, c) = F_x(i, j, c) + F_c(i, j, c).

[0067] Extraction and fusion of texture features of seat belt reflective strips: Use the HOG algorithm to extract the gradient information of the input image, calculate the gradient histogram of 9 directions in the 8x8 pixel block, and generate the HOG feature map. Use the reflective strip area detection algorithm to locate the position of the seat belt reflective strips, and extract the HOG feature H_r corresponding to the reflective strip area. Input H_r into a 5184x2048 fully connected layer (5184=14x14x64, where 64 is the HOG feature dimension) to obtain a 2048-dimensional reflective strip texture feature vector. Restore the 2048-dimensional reflective strip texture feature vector to a 14x14x2048 reflective strip texture prior feature map F_r. Multiply F_r and F_{xc}, which fused the color information, element by element in the channel dimension to obtain the feature map F_{xcr} that fused the color and texture. The process of channel-by-channel multiplication of texture features and color feature maps can be expressed as: F_{xcr}(i, j, c) = F_{xc}(i, j, c) * F_r(i, j, c). F_{xcr} fused with seat belt attribute prior is input into the channel attention layer: the channel attention layer uses adaptive average pooling to compress F_{xcr} into a 2048-dimensional feature descriptor in the spatial dimension. The 2048-dimensional feature is input into two 2048x512 and 512x2048 fully connected layers, with ReLU activation in the middle to obtain the channel attention weight. The 2048-dimensional channel weight W_c is multiplied by F_{xcr} to obtain the weighted adjusted seat belt attribute enhanced feature map F_{ca}. The channel attention that integrates color and texture priors helps to enhance the discriminative features of seat belts and improve detection accuracy.

[0068] Figure 5This is an exemplary flow chart for fine-tuning the parameters of the seat belt wearing recognition model shown in some embodiments of the present application, S6, using a small sample data set with spatiotemporal labels to perform Few-shot Learning and meta-learning training on the constructed seat belt wearing recognition model, including: Scene information extraction: Use a pre-trained scene classification model (such as Places365-CNN) to perform scene recognition on the input image, and obtain the construction stage to which the image belongs as "subject". Use a human posture estimation algorithm based on key point detection (such as open pose) to detect and analyze the human body in the image, and judge the job type as "scaffolder" based on the human posture and surrounding environment. The extracted scene information is "subject-scaffolder". In the B+ tree index, use "subject-scaffolder" as the index key to query and locate the target small sample data location stored in the leaf node. An example of the index structure of a B+ tree: root node: ["Foundation", "Main body", "Decoration", "Maintenance"]; first-level child nodes: "Main body" -> ["Rebar worker", "Concrete worker", "Scaffolder", "Electrician"]; leaf nodes: "Scaffolder" -> [Image_1, Image_2,......., Image_10000]; based on the index key "Main body - Scaffolder", the query locates the target small sample data set containing 10,000 images.

[0069] Support sample extraction and training set construction: From the retrieved 10,000 "subject-scaffolder" small sample images, 2,000 images are randomly selected as support samples. Among the 2,000 support samples, 1,000 are positive samples (wearing seat belts) and 1,000 are negative samples (not wearing seat belts), with a positive-negative sample ratio of 1:1. Each support sample image is 224x224 in size and is accompanied by a corresponding seat belt wearing status label (0 or 1). The 2,000 support sample images and labels are organized into a small sample training set D_s: D_s = {(x_1, y_1), (x_2, y_2), ......., (x_{2000}, y_{2000})}, where x_i is the i-th support sample image and y_i is the corresponding seat belt wearing status label.

[0070] Support sample set: 2000 labeled images extracted from the target small sample data, including 1000 positive samples (wearing seat belts) and 1000 negative samples (not wearing seat belts), with an image size of 224x224. Query sample set: 1000 images are randomly selected from the remaining 8000 images in the target small sample data as query samples, which are used to evaluate the performance of the model after fine-tuning, with an image size of 224x224. Feature extraction: Use the pre-trained ResNet-50 as the feature extractor of the seat belt wearing recognition model to extract the 2048-dimensional feature vectors of the support samples and query samples. Input the 2000 support samples into ResNet-50 to obtain 2000 2048-dimensional feature vectors F_s = {f_1, f_2,......., f_{2000}}. Input 1000 query samples into ResNet-50 and obtain 1000 2048-dimensional feature vectors F_q = {q_1, q_2, ...., q_{1000}}.

[0071] Prototype vector calculation: According to the seat belt wearing status label of the support sample, its feature vector F_s is divided into positive sample feature vector F_{s0} and negative sample feature vector F_{s1}, with 1000 feature vectors for each category. K-means clustering is performed on F_{s0} and F_{s1} respectively, and the number of cluster centers is set to 1 to obtain the positive sample prototype vector P_0 and the negative sample prototype vector P_1, both of which are 2048-dimensional vectors. The prototype vectors P_0 and P_1 represent the center points of the two categories of wearing seat belts and not wearing seat belts in the feature space, respectively. Query sample classification and loss calculation: For each feature vector q_i in the query sample set F_q, the Euclidean distance d_0 and d_1 between it and the positive sample prototype vector P_0 and the negative sample prototype vector P_1 are calculated. d_0=||q_i-P_0||_2, d_1=||q_i-P_1||_2, input the Euclidean distances d_0 and d_1 into the softmax function to obtain the probabilities p_0 and p_1 of the query sample belonging to the positive sample and the negative sample.

[0072] p_0=frac{e^{-d_0}}{e^{-d_0}+e^{-d_1}},p_1=1-p_0,take the category with larger probability value as the predicted category hat{y}_i of the query sample, and calculate the cross entropy loss with the true label y_i of the query sample. L_i=-[y_ilog(hat{y}_i)+(1-y_i)log(1-hat{y}_i)]. Average the cross entropy losses of 1000 query samples to get the loss L on the query set. L=frac{1}{1000}sum_{i=1}^{1000}L_i. Model fine-tuning: Use the Adam optimizer to fine-tune the parameters of ResNet-50 with the query set loss L as the optimization target. Set the learning rate to 1e-4 and the number of fine-tuning iterations to 10. In each iteration, use a batch of query samples (such as 32) to calculate the loss and gradient, and update the model parameters. After 10 iterations, the fine-tuned seat belt wearing recognition model M_t is obtained. The Few-shot model is trained based on the MAML meta-learning method, and the meta-learning task set is constructed: 5,000 small sample images of aerial workers wearing seat belts are collected from different construction stages and types of work, and the size of each image is 224x224. The 5,000 images are divided into 100 meta-learning tasks according to the construction stage and type of work information. Each task corresponds to a specific scene, such as "main body-rebar worker" and "decoration-electrician". Each meta-learning task contains 50 images from the same scene, of which 25 are positive samples (wearing seat belts) and 25 are negative samples (not wearing seat belts).

[0073] Base learner selection: The same Prototypical Network as in the fine-tuning stage is used as the base learner for MAML. The Prototypical Network consists of a 4-layer convolutional neural network (each layer contains 64 3x3 convolution kernels, ReLU activation function and 2x2 maximum pooling) and a fully connected layer, with an output dimension of 64. MAML training process: Outer-loop initialization: Randomly initialize the parameters theta of the Prototypical Network as the starting point for MAML meta-learning. For each meta-learning task T_i: Randomly select 20 images from the 50 images of task T_i as the support set S_i, and 30 images as the query set Q_i. Inner-loop update: Calculate the prototype vectors P_0 and P_1 of positive and negative samples on the support set S_i, each prototype vector is 64 dimensions. Calculate the classification loss L_i of the query sample on the query set Q_i: L_i = frac{1}{|Q_i|}sum_{x_j inQ_i}-log frac{exp(-d(f_theta(x_j), P_{y_j}))}{sum_{k=0}^1exp(-d(f_theta(x_j), P_k))}, where f_theta is the Prototypical Network, d is the Euclidean distance metric function, and y_j is the true label of the query sample.

[0074] Use gradient descent to update the base learner parameters: theta_i' = theta-alpha nabla_theta L_i, where alpha is the inner-loop learning rate, set to 0.01. Outer-loop update: Sum up the updated parameters theta_i' of all meta-learning tasks, calculate the total loss of the query set: L_q = sum_{i = 1}^{100}L(f_{theta_i'}; Q_i), calculate the gradient of the total loss L_q with respect to the initial parameters theta of the base learner: g = nabla_theta L_q, and use the MAML meta-gradient descent algorithm to update the initial parameters theta: theta left arrow theta-betag, where beta is the outer-loop learning rate, set to 0.001. Repeat steps 2 and 3 for a total of 10,000 iterations until the initial parameters theta of the base learner converge, and obtain the optimized Few-shot meta-model M_f.

[0075] The trained seat belt wearing recognition model is used for early warning. The high-altitude working scene images collected by the video surveillance system have a resolution of 1920x1080 and a frame rate of 25 frames per second. The shooting time and location coordinates of each frame of the image have been associated with the image through the spatiotemporal index. The extracted human body area image blocks are input into the trained few-shot seat belt wearing recognition model for prediction. The few-shot model uses the Prototypical Network as the infrastructure and is fine-tuned with 20 support samples to achieve rapid adaptation in the current scenario. The model outputs the seat belt wearing status (0 for not wearing, 1 for wearing) and confidence score for each human body area. Five human body areas are detected in the current frame, and the recognition results are as follows: human body 1: wearing a seat belt, confidence 0.95; human body 2: wearing a seat belt, confidence 0.92; human body 3: not wearing a seat belt, confidence 0.88; human body 4: wearing a seat belt, confidence 0.97; human body 5: not wearing a seat belt, confidence 0.85. According to the timestamp and device ID of the input image, the location coordinates of the image are queried from the spatiotemporal index database. The shooting location of the current image is (120.234, 30.567), that is, the longitude is 120.234 degrees and the latitude is 30.567 degrees. The features such as seat belt wearing status, lifeline area analysis results, and location coordinates are input into the pre-built decision tree model for risk assessment.

[0076] The decision tree model gives the warning level according to the following rules: Rule 1: If the proportion of people not wearing safety belts exceeds 20% and they are in a high-risk area (such as the top of the tower crane), a red warning is triggered. Rule 2: If the proportion of people not wearing safety belts is between 10% and 20% and they are in a medium-risk area (such as the edge of the floor), an orange warning is triggered. Rule 3: If the proportion of people not wearing safety belts is less than 10% and they are in a low-risk area (such as indoors), a yellow warning is triggered. Rule 4: If everyone wears a safety belt and the lifeline status is normal, no warning is triggered. According to the current recognition results and location coordinates, the decision tree model gives an orange warning level. The system automatically generates a warning message: "Warning: There are 2 people not wearing safety belts in the construction area of ​​the 10th floor of the No. 2 tower crane. Please deal with it in time!" The warning information is sent to the on-site safety management personnel in a timely manner through SMS, voice calls, App push, etc., and recorded in the safety management log.

Claims

1. A lifeline warning method for high-altitude operations, comprising: S1, acquiring image data including images of workers and images of construction areas; S2, using an image segmentation algorithm to segment the image data, extracting the human body area and the lifeline area including the seat belt in the image data; S3, extracting the timestamp and location information of the image data as metadata, and constructing a spatiotemporal index of the image data based on the metadata; S4, constructing a seat belt wearing recognition model based on a convolutional neural network to identify the seat belt wearing status of the operator; in the constructed seat belt wearing recognition model, setting a channel attention layer; S5, build a small sample data set; S6, using a small sample data set with spatiotemporal labels to perform Few-shot Learning and meta-learning training on the constructed seat belt wearing recognition model; inputting the extracted human body area and lifeline area into the trained seat belt wearing recognition model for prediction, and identifying the seat belt wearing status; S7, obtaining the position coordinates of the image data according to the constructed spatiotemporal index, combining the identified seat belt wearing status, performing risk assessment through a decision tree, and triggering a corresponding warning level; In the channel attention layer, the HSV color space is used to convert the RGB image data into the HSV color space, the color histograms of the H and S channels are counted, and the color histogram features are fused with the feature map output by the backbone network in the channel dimension to obtain a feature map that integrates the seat belt color prior information; In the channel attention layer, the gradient directional histogram algorithm HOG is used to extract the gradient information of the input image, and the gradient descriptor HOG feature of the reflective strip area is obtained. The HOG feature is mapped to the same channel dimension as the feature map output by the backbone network to generate the reflective strip texture feature map. The reflective strip texture feature map is multiplied channel by channel with the feature map of the fused color information to obtain the feature map of the fused seat belt color and reflective performance prior information. The feature map of the fused seat belt attribute prior information is input into the channel attention layer to calculate the attention weight; Among them, construct a small sample data set, including: The shooting timestamp information of the images in each small sample data set is extracted as the time metadata of the corresponding small sample data, the pixel coordinates of the area where the worker wears the safety belt in the image are obtained, and the pixel coordinates are converted into the spatial position coordinates of the construction site through the coordinate mapping method as the position metadata of the corresponding small sample data; Associating and labeling each small sample data with the corresponding time metadata and location metadata, generating small sample data with time and space labels in a triple structure consisting of a small sample image, a shooting timestamp, and a spatial location coordinate; Based on small sample data with spatiotemporal labels, spatiotemporal clustering is performed according to the two index keys of construction stage category and work type category to construct a small sample data set with spatiotemporal index structure.

2. The high-altitude operation lifeline warning method according to claim 1 is characterized by: S2, extracting the human body area and the lifeline area including the seat belt in the image data, including: S21, using a median filtering algorithm to perform denoising on the image data; using a histogram equalization algorithm to perform enhancement processing on the denoised image to obtain pre-processed image data; S22, using a pre-trained fully convolutional network (FCN) semantic segmentation model to perform semantic segmentation on the preprocessed image data, and dividing the image data into a foreground area and a background area through pixel-level classification, wherein the foreground area includes a human body and a seat belt, and the background area includes a building, to obtain foreground image data after semantic segmentation; S23, using a watershed algorithm to perform region segmentation on the foreground image data after semantic segmentation, and aggregating pixels with pixel similarity greater than a threshold into the same region through a region generation strategy to obtain a human body region and a seat belt region in the foreground image data; S24, using the openpose human posture estimation algorithm to extract human skeleton data from the human body region image data, the skeleton data includes the head, torso and limbs; using the Canny edge detection algorithm to extract the edge of the seat belt from the seat belt region image data, and obtaining contour data including the shape and length of the seat belt through gradient difference calculation; S25, combining the human skeleton data and the seat belt contour data, and obtaining lifeline area data including human body posture information and seat belt position information by calculating the spatial position relationship between the skeleton data and the contour data.

3. The high-altitude operation lifeline warning method according to claim 2 is characterized by: S4, builds a seat belt wearing recognition model based on convolutional neural network, including: S41, uses ResNet convolutional neural network as the backbone network of the seat belt wearing recognition model. The backbone network consists of multiple convolutional layers, pooling layers and fully connected layers to extract multi-scale features of input image data; S42, after the convolutional layer of the backbone network, introduces the spatial attention mechanism to construct a spatial attention layer. The spatial attention layer fuses the building edge features and the scaffolding area mask, and calculates the spatial attention weight based on the fused feature map; S43, in the spatial attention layer, the Canny algorithm is used to perform edge detection on the input image data, and the edge features of the building outline are extracted. The extracted building edge features are multiplied element by element with the feature map output by the backbone network to obtain a feature map that integrates the building edge prior information; S44, in the spatial attention layer, the semantic segmentation algorithm SegNet is used to locate the scaffolding area, and the scaffolding area mask map and the feature map fused with the building edge prior information are added element by element to obtain a feature map fused with the building structure and the aerial work scene prior information. The feature map fused with the scene prior information is input into the spatial attention layer to calculate the attention weight; S45, after the convolution layer of the backbone network and before the spatial attention layer, a channel attention layer is set. The channel attention layer fuses the color features of the seat belts and the texture features of the reflective strips, and calculates the channel attention weight based on the fused feature map.

4. The high-altitude operation lifeline warning method according to claim 3 is characterized by: S44, obtains a feature map integrating the prior information of the building structure and the aerial work scene, including: The SegNet algorithm is used to perform semantic segmentation on the input image data. The input image is mapped to a low-dimensional feature space through the encoder network, and the high-level semantic information of the image is extracted. The feature map is deconvolved to the original image size through the decoder network to generate a pixel-level segmentation result map. Binarize the semantic segmentation result image, extract the scaffolding area in the segmentation result, and generate a scaffolding area mask image with the same size as the input image. The scaffolding pixel value in the mask image is 1, and the background pixel value is 0; The scaffolding area mask map and the feature map that integrates the building edge prior information are added element by element. By adding each feature value in the feature map to the scaffolding mask pixel value at the corresponding position, the prior information of the scaffolding area is superimposed on the feature map to obtain a feature map that integrates the building structure and the aerial work scene prior information. The feature map fused with scene prior information is input into the spatial attention layer, and the calculation of spatial attention weights is guided by the scaffolding region mask.

5. The high-altitude operation lifeline warning method according to claim 3 is characterized by: The feature map integrating the seat belt color prior information is obtained, including: Use HSV color space to convert RGB image data into HSV color space, and extract image data of H channel and S channel; Perform color histogram statistics on the H channel and the S channel respectively to obtain the color distribution histogram of the seat belt area on the H and S channels; The color histograms of the H channel and the S channel are concatenated into a color histogram feature vector, and the color histogram feature vector is mapped to the same channel dimension as the backbone network output feature map through a fully connected layer to generate a seat belt color prior feature map; The seat belt color prior feature map and the feature map output by the backbone network are added element by element in the channel dimension, and the color feature is embedded into the feature map of the backbone network to obtain a feature map that integrates the seat belt color prior information.

6. The high-altitude operation lifeline warning method according to claim 3 is characterized by: To the feature map that integrates the prior information of seat belt color and reflective performance, including: The HOG algorithm is used to extract the gradient information of the image on the input image data. The HOG descriptor reflecting the texture characteristics of the image is generated by calculating the gradient amplitude and direction of the pixel points in the local area of ​​the image in different directions. The reflective strip area detection algorithm is used to locate the position of the seat belt reflective strip in the image, and the HOG descriptor corresponding to the reflective strip area is extracted to obtain the feature representation of the reflective strip texture; The extracted reflective strip HOG features are mapped to the same channel dimension as the backbone network output feature map through a fully connected layer to generate a reflective strip texture feature map with the same size as the feature map; The reflective strip texture feature map is multiplied channel by channel with the feature map that integrates the seat belt color prior information. By multiplying each channel in the feature map with the corresponding reflective strip texture feature, the texture prior information of the reflective strip area is integrated in the channel dimension of the feature map to obtain a feature map that integrates the seat belt color and reflective performance prior information. The feature map fused with the prior information of seat belt attributes is input into the channel attention layer, and the calculation of channel attention weights is guided by the texture information of the reflective strip.

7. The high-altitude operation lifeline warning method according to any one of claims 2 to 6 is characterized in that: S5, build a small sample data set, including: S51, obtaining image data of workers wearing safety belts at different construction stages, where the construction stages include a civil construction stage and a facade construction stage; S52, obtaining safety belt wearing image data of workers of different types of work, including steel bar workers, formwork workers, and masonry workers; S53, the acquired image data of workers wearing safety belts in different construction stages and different types of work are combined according to the two dimensions of construction stage category and type of work category, and the image data is divided into scenes to obtain a small sample data set of safety belts corresponding to the cross-combination of each construction stage and each type of work.

8. The high-altitude operation lifeline warning method according to claim 7 is characterized by: Perform spatiotemporal clustering of small sample data, including: The small sample data set with spatiotemporal labels is input into the ST-DBSCAN spatiotemporal clustering model. By setting the density thresholds of the time dimension and the space dimension, the small sample data with spatiotemporal proximity and density connection are divided into the same spatiotemporal cluster. In the density calculation formula of ST-DBSCAN, the construction stage factor and the work type factor are introduced, and different weights are set for small sample data of different construction stages and work types; For each spatiotemporal cluster generated by clustering, obtain the common time label and location label of all small sample data in the cluster, associate the construction stage category and work type category corresponding to the spatiotemporal cluster, and generate cluster-level spatiotemporal index items; Based on the generated cluster-level spatiotemporal index items, the B+ tree is used to index the spatiotemporal clusters. The combination of construction stage and work type is used as the key value, and the pointers of all small sample data in the corresponding spatiotemporal cluster are stored in the tree child nodes of the B+ tree.

9. The high-altitude operation lifeline warning method according to claim 8 is characterized by: S6, using a small sample dataset with spatiotemporal labels to perform Few-shot Learning and meta-learning training on the constructed seat belt wearing recognition model, including: S61, extracting scene information from the input image data, where the scene information includes a construction stage category and a work type category; based on the extracted scene information, in a small sample data set with a spatiotemporal index structure, retrieving target small sample data matching the input image scene through a B+ tree index structure; wherein, from the retrieved target small sample data, obtaining a plurality of labeled images including positive samples and negative samples of wearing a safety belt as support samples to form a small sample training set for model fine-tuning; S62, using the seat belt wearing recognition model to obtain a feature vector of the support sample, clustering the feature vector according to the seat belt wearing state label of the sample to obtain a prototype vector of each category; The remaining samples in the small sample training set are used as query samples to obtain the query sample feature vector; Calculate the Euclidean distance between the query sample feature vector and each category prototype vector, convert the Euclidean distance into a probability distribution through the softmax function, and take the category with the largest probability distribution as the predicted category of the query sample; Calculate the classification loss of the query sample on the support sample and use the gradient descent algorithm to fine-tune the model parameters.

10. The high-altitude operation lifeline warning method according to claim 9 is characterized in that: S6, using a small sample dataset with spatiotemporal labels to perform Few-shot Learning and meta-learning training on the constructed seat belt wearing recognition model, also includes: S63, using PrototypicalNetwork as the base learner of the Model-AgnosticMeta-Learning method, and taking small sample data sets under different construction stages and types of work as different meta-learning tasks to form a meta-learning task set; For each task in the meta-learning task set, the corresponding small sample dataset is divided into a support set and a query set. The support set is used for inner-loop training of PrototypicalNetwork, and the query set is used to evaluate the generalization performance of the model on the corresponding task. For each meta-learning task, input the support set of the task, calculate the prototype vector of each category, input the query set, and calculate the classification loss of the query sample; use the classification loss to perform inner-loop gradient descent update on the parameters of PrototypicalNetwork to obtain the updated model parameters for the corresponding task; Summarize the updated model parameters on all meta-learning tasks, calculate the query loss of each task, sum the query losses and calculate the gradient, use the MAML meta-gradient descent algorithm to perform outer-loop updates on the initial parameters of PrototypicalNetwork, and obtain the optimized initial parameters for all tasks; Repeat the above inner-loop training and outer-loop update process until the initial parameters of PrototypicalNetwork converge and obtain the Few-shotLearning meta-model.

Images