High-altitude work protection warning method, device and computer-readable medium

Abstract

The embodiments of the present disclosure disclose a method, device and computer-readable medium for early warning of protection of high-altitude work. A specific implementation of the method includes: collecting real-time monitoring video streams as video streams to be detected; performing target detection on the video streams to be detected; for each construction worker description information in the construction worker description information set, executing the following processing steps: performing image segmentation on the video image; determining the type of working surface; determining the risk area description information; determining the edge distance of the construction worker description information corresponding to the construction worker based on the risk area location information; performing early warning operations on the construction worker description information corresponding to the construction worker based on the risk area protection facility information, edge distance and the construction worker clothing information included in the construction worker description information. This implementation method monitors the safety of high-altitude work in real time, realizes automated early warning for high-altitude work safety monitoring, and avoids the occurrence of direct threats to the life safety of construction workers caused by low detection efficiency of high-altitude work safety monitoring and low early warning efficiency of high-altitude work.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and more particularly to a method, device, and computer-readable medium for early warning of protection during high-altitude operations. Background Art

[0002] During construction, due to a lack of risk awareness, high-altitude work often presents numerous safety hazards. These hazards can lead to falls, posing a direct threat to the lives of construction workers. Currently, the most common method for monitoring high-altitude work safety is for construction site managers to monitor workers through surveillance video.

[0003] However, the inventors have found that when the above method is adopted, the following technical problems often occur:

[0004] The high-altitude work environment is complex, and manual monitoring methods are highly subjective and have blind spots, making it difficult to track all potential dangers in real time. When problems arise, manual intervention may not respond quickly enough, and the detection efficiency of high-altitude work safety monitoring and the early warning efficiency of high-altitude work cannot be guaranteed, which may pose a direct threat to the lives of construction workers.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the Invention

[0006] The content of this disclosure is used to briefly introduce concepts that will be described in detail in the detailed description section below. The content of this disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0007] Some embodiments of the present disclosure provide a method, device, and computer-readable medium for early warning of protection during aerial work to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a method for early warning of work-at-height protection, the method comprising: collecting a real-time surveillance video stream as a video stream to be detected, wherein the video stream to be detected is a video stream of a work surface captured in real time by monitoring equipment within the work area. Target detection is performed on the video stream to generate worker description information, thereby obtaining a set of worker description information. The worker description information in the set of worker description information represents workers working within the work area, and the worker description information in the set of worker description information includes worker location information and worker attire information, wherein the worker attire information represents the wearing status of the safety equipment of the worker corresponding to the worker description information. For each worker description in the set of worker description information, the following processing steps are performed: based on the worker location information included in the worker description information, image segmentation is performed on the video image contained in the video stream to be detected to generate cropped video images, thereby obtaining a cropped video image sequence. Based on the cropped video image sequence, the type of the work surface is determined. Based on the cropped video image sequence and the work surface type, risk area description information is determined, where the risk area description information includes: risk area location information and risk area protective facility information, where the risk area protective facility information represents the status of the protective facilities in the risk area corresponding to the risk area description information. Based on the risk area location information, the edge distance of the construction personnel corresponding to the construction personnel description information is determined. Based on the risk area protective facility information, the edge distance, and the construction personnel attire included in the construction personnel description information, an early warning operation is performed on the construction personnel corresponding to the construction personnel description information.

[0009] In a second aspect, some embodiments of the present disclosure provide a high-altitude work protection and warning device, the device comprising: an acquisition unit, configured to acquire a real-time monitoring video stream as a video stream to be detected, wherein the video stream to be detected is a video stream for the work surface shot in real time by monitoring equipment in the high-altitude work area. A target detection unit, configured to perform target detection on the video stream to be detected to generate construction worker description information and obtain a construction worker description information set, wherein the construction worker description information in the construction worker description information set represents the construction workers working in the high-altitude work area, and the construction worker description information in the construction worker description information set includes: construction worker location information and construction worker clothing information, wherein the construction worker clothing information represents the safety equipment wearing status of the construction worker corresponding to the construction worker description information. The processing unit is configured to execute the following processing steps for each construction worker description information in the above-mentioned construction worker description information set: according to the construction worker position information included in the above-mentioned construction worker description information, perform image segmentation on the video image contained in the above-mentioned video stream to be detected to generate a cropped video image and obtain a cropped video image sequence; according to the above-mentioned cropped video image sequence, determine the working surface type; according to the above-mentioned cropped video image sequence and the above-mentioned working surface type, determine the risk area description information, wherein the above-mentioned risk area description information includes: risk area position information and risk area protection facility information, wherein the above-mentioned risk area protection facility information represents the protection facility status of the risk area corresponding to the risk area description information; according to the above-mentioned risk area position information, determine the edge distance of the construction worker corresponding to the above-mentioned construction worker description information; according to the above-mentioned risk area protection facility information, the above-mentioned edge distance and the above-mentioned construction worker clothing information, perform early warning operations on the construction worker corresponding to the above-mentioned construction worker description information.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation of the first aspect is implemented.

[0012] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: Through the high-altitude work protection and early warning methods of some embodiments of the present disclosure, high-altitude work safety is monitored in real time, and automated early warning for high-altitude work safety monitoring is achieved, avoiding the occurrence of direct threats to the life safety of construction workers caused by low detection efficiency and low early warning efficiency of high-altitude work safety monitoring. Specifically, the reasons for the low detection efficiency and early warning efficiency of high-altitude work safety monitoring are: the high-altitude environment is complex, and manual monitoring methods have great subjectivity and blind spots in vision, making it difficult to track all potential hazards in real time. When problems arise, manual intervention may not respond quickly enough, and the detection efficiency and early warning efficiency of high-altitude work safety monitoring cannot be guaranteed, which may pose a direct threat to the life safety of construction workers. Based on this, the high-altitude work protection and early warning methods of some embodiments of the present disclosure first collect real-time monitoring video streams as the video stream to be detected, wherein the video stream to be detected is a video stream of the work surface captured in real time by monitoring equipment within the high-altitude work area. By collecting real-time monitoring video streams, real-time monitoring of high-altitude work safety can be achieved. Next, target detection is performed on the video stream to be detected to generate worker description information, resulting in a set of worker description information. The worker description information in the set represents workers working within the high-altitude work area and includes worker location information and worker attire information. The attire information represents the safety equipment worn by the worker corresponding to the worker description information. By identifying the worker's location and safety equipment, a preliminary assessment of the worker's safety is performed. Finally, for each worker description in the set, the following processing steps are performed: First, based on the worker location information included in the worker description information, the video image contained in the video stream to be detected is segmented to generate cropped video images, resulting in a cropped video image sequence. Cropping the building portion of the video image for subsequent risk area identification reduces interference from image components other than the building during the identification process, enabling more efficient risk area identification, while also reducing computational complexity and improving processing speed. Second, based on the cropped video image sequence, the work surface type is determined. Different work face risk areas have different detection targets, so the work face type needs to be determined. The third step is to determine risk area description information based on the cropped video image sequence and the work face type. This risk area description information includes: risk area location information and risk area protective facility information. The risk area protective facility information represents the status of protective facilities in the risk area corresponding to the risk area description information. This risk area location information and risk area protective facility information are used to further assess the safety of construction workers.The fourth step is to determine the edge distance of the construction personnel corresponding to the construction personnel description information based on the above-mentioned risk area location information. The fifth step is to perform early warning operations on the construction personnel corresponding to the above-mentioned construction personnel description information based on the above-mentioned risk area protective facility information, the above-mentioned edge distance and the construction personnel clothing information included in the above-mentioned construction personnel description information. The safety of the construction personnel is judged from three aspects: protective facilities, edge distance and construction personnel clothing information, and early warning is issued for situations where safety hazards exist. In this way, all potential dangers can be tracked in real time, and early warnings can be issued immediately when problems arise, thereby improving the detection efficiency of high-altitude work safety monitoring and the early warning efficiency of high-altitude work, and avoiding the occurrence of direct threats to the life safety of construction personnel due to low detection efficiency of high-altitude work safety monitoring and low early warning efficiency of high-altitude work. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 is a flow chart of some embodiments of the high-altitude work protection and early warning method according to the present disclosure;

[0015] Figure 2 Schematic diagram of the structure of some embodiments of the high-altitude work protection and warning device according to the present disclosure;

[0016] Figure 3 It is a structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

[0019] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] refer to Figure 1 , shows a process 100 of some embodiments of the high-altitude work protection and early warning method according to the present disclosure. The high-altitude work protection and early warning method includes the following steps:

[0024] Step 101: collect real-time monitoring video stream as the video stream to be detected.

[0025] In some embodiments, the execution subject (eg, a computing device) of the high-altitude work protection warning method may collect a real-time monitoring video stream as a video stream to be detected through a wired connection or a wireless connection.

[0026] The video stream to be detected is a video stream of the work surface captured in real time by monitoring equipment within the high-altitude work area. The high-altitude work area is an area where construction workers are working at a height of 2 meters or more above the ground. The monitoring equipment represents a surveillance camera located within the high-altitude work area and facing the work surface.

[0027] It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0028] Step 102 : performing target detection on the video stream to be detected to generate description information of construction personnel and obtain a set of description information of construction personnel.

[0029] In some embodiments, the execution entity may perform target detection on the video stream to be detected to generate description information of the construction personnel and obtain a set of description information of the construction personnel.

[0030] The construction worker description information in the above-mentioned construction worker description information set represents construction workers working in the above-mentioned high-altitude work area. The construction worker description information in the above-mentioned construction worker description information set includes: construction worker location information and construction worker attire information. The construction worker attire information represents the wearing status of the safety equipment of the construction worker corresponding to the construction worker description information. As an example, the above-mentioned safety equipment may be a hard hat, protective clothing, and a safety rope. The above-mentioned construction worker location information represents the current position of the corresponding construction worker in the work surface.

[0031] In practice, the execution entity may perform target detection on the video stream to be detected by using an SSD (Single Shot MultiBox Detector) algorithm to generate description information of the construction workers and obtain a set of description information of the construction workers.

[0032] In some optional implementations of some embodiments, the execution subject performs target detection on the video stream to be detected to generate construction worker description information, and obtains a construction worker description information set, which may include the following steps:

[0033] The first step is to perform denoising on the video stream to be detected to obtain a denoised video stream.

[0034] In practice, the execution subject may perform denoising on the video stream to be detected by Gaussian filtering to obtain a denoised video stream.

[0035] The second step is to normalize the denoised video stream to obtain the preprocessed video stream.

[0036] In practice, the execution subject may perform normalization processing on the denoised video stream by a linear normalization method to obtain a preprocessed video stream.

[0037] The third step is to generate the above-mentioned construction worker description information set through the pre-trained construction worker detection model and the above-mentioned preprocessed video stream.

[0038] The construction worker detection model takes the preprocessed video stream as input and outputs the set of construction worker description information. In practice, the construction worker detection model can be an object detection model. For example, the YOLO V5 (You Only Look Once Version 5) model can be used.

[0039] In practice, the execution entity may input the pre-processed video stream into the pre-trained construction worker detection model to generate the construction worker description information set.

[0040] Step 103: For each construction worker description information in the construction worker description information set, perform the following processing steps:

[0041] Step 1031 : performing image segmentation on the video image contained in the video stream to be detected according to the construction worker position information included in the construction worker description information, so as to generate cropped video images and obtain a cropped video image sequence.

[0042] In some embodiments, the execution entity may perform image segmentation on the video image included in the video stream to be detected based on the construction worker location information included in the construction worker description information to generate cropped video images and obtain a cropped video image sequence.

[0043] In practice, the execution entity may use a deep learning model to segment the video image contained in the video stream to be detected based on the construction worker location information included in the construction worker description information, thereby generating cropped video images and obtaining a cropped video image sequence. As an example, the deep learning model may be a U-Net model.

[0044] In some optional implementations of some embodiments, the execution entity performs image segmentation on the video image included in the video stream to be detected based on the construction worker location information included in the construction worker description information to generate a cropped video image, and obtaining the cropped video image sequence may include the following steps:

[0045] For each video image of a construction worker in the video stream to be detected and containing the construction worker description information, the following segmentation steps are performed:

[0046] The first step is to convert the above video image into a grayscale image to obtain a video grayscale image.

[0047] In practice, the execution entity may perform grayscale conversion on the video image by using a weighted average method to obtain a video grayscale image.

[0048] In the second step, edge detection is performed on the grayscale image of the video according to the construction worker position information included in the construction worker description information to obtain an image after edge detection.

[0049] In practice, the execution entity may perform edge detection on the grayscale video image using the Canny edge detection algorithm based on the construction worker position information included in the construction worker description information to obtain an edge-detected image. The edge-detected image represents an image after the edge of the construction worker corresponding to the construction worker position information is framed.

[0050] The third step is to perform contour extraction on the image after edge detection to generate a set of building contour points.

[0051] The building contour points in the above building contour point set include: contour point abscissas and contour point ordinates.

[0052] In practice, the execution entity may perform contour extraction on the edge-detected image along the edges defined in the edge-detected image to generate a set of building contour points.

[0053] The fourth step is to classify the building contour point set based on the horizontal and vertical coordinates of the contour points to obtain a building contour point horizontal coordinate set and a building contour point vertical coordinate set.

[0054] In practice, the above-mentioned execution entity can determine the contour point horizontal coordinates of each building contour point in the above-mentioned building contour point set as a building contour point horizontal coordinate set, and determine the contour point vertical coordinates of each building contour point in the above-mentioned building contour point set as a building contour point vertical coordinate set.

[0055] As an example, the above-mentioned building contour point set can be {(2, 10), (1, 6), (6, 7), (4, 5), (5, 9), (3, 8), (100, 100)}, then the building contour point horizontal coordinate set can be {2, 1, 6, 4, 5, 3, 100}, and the building contour point vertical coordinate set can be {10, 6, 7, 5, 9, 8, 100}.

[0056] In the fifth step, the abscissa set of the building outline points and the ordinate set of the building outline points are sorted respectively to obtain a abscissa sequence of the building outline points and a ordinate sequence of the building outline points.

[0057] In practice, the execution entity may arrange the building outline point abscissa set and the building outline point ordinate set in descending order to obtain a building outline point abscissa sequence and a building outline point ordinate sequence.

[0058] As an example, the above-mentioned building contour point horizontal coordinate set can be {2, 1, 6, 4, 5, 3, 100}, and the above-mentioned building contour point vertical coordinate set can be {10, 6, 7, 5, 9, 8, 100}. Therefore, the building contour point horizontal coordinate sequence can be {1, 2, 3, 4, 5, 6, 100}, and the building contour point vertical coordinate sequence can be {5, 6, 7, 8, 9, 10, 100}.

[0059] In the sixth step, a set of building contour points after elimination is generated according to the above-mentioned building contour point abscissa sequence and building contour point ordinate sequence.

[0060] In practice, first, the execution entity may determine the first quartile of the horizontal coordinates and the third quartile of the horizontal coordinates of the building outline points according to the horizontal coordinate sequence of the building outline points by using the first quartile calculation formula and the third quartile calculation formula. Then, the execution entity may determine the first quartile of the vertical coordinates and the third quartile of the vertical coordinates of the building outline points according to the vertical coordinate sequence of the building outline points by using the first quartile calculation formula and the third quartile calculation formula. Next, the execution entity may determine the upper limit of the horizontal coordinates of the building outline points and the lower limit of the horizontal coordinates of the building outline points according to the first quartile of the horizontal coordinates and the third quartile of the horizontal coordinates. Further, the execution entity may determine the upper limit of the vertical coordinates and the lower limit of the vertical coordinates of the building outline points according to the first quartile of the vertical coordinates and the third quartile of the vertical coordinates by using the target formula. Finally, the execution entity may remove at least one building contour point that does not meet the contour point screening condition from the building contour point set to obtain the removed building contour point set, wherein the contour point screening condition is the lower limit of the horizontal coordinate of the building contour point < the horizontal coordinate < the upper limit of the horizontal coordinate of the building contour point or the lower limit of the vertical coordinate of the building contour point < the vertical coordinate < the upper limit of the vertical coordinate of the building contour point.

[0061] Among them, the calculation formula for the first quartile is

[0062]

[0063] if Rather than being an integer, Q1 is the interpolated value between the data points at that location and the next:

[0064] Q1=Data lower +[fraction×(Data upper -Data lower )]

[0065] Among them, Q1 is the first quartile, n is the number of data points, For the data set Data points at each location, lower is less than or equal to Data points upper is greater than The data points, fraction is The decimal part of .

[0066] Among them, the calculation formula for the third quartile is

[0067]

[0068] if Rather than being an integer, Q3 is the interpolated value between the data points at that location and the next:

[0069] Q3=Data lower +[fraction×(Data upper -Data lower )]

[0070] Among them, Q3 is the third quartile, n is the number of data points, For the data set Data points at each location, lower is less than or equal to Data points upper is greater than The data points, fraction is The decimal part of .

[0071] Among them, the above target formula is upper limit = Q3 + 1.5 × (Q3-Q1), lower limit = Q1-1.5 × (Q3-Q1), Q1 is the first quartile, and Q3 is the third quartile.

[0072] As an example, the horizontal coordinate sequence of the above-mentioned building contour points can be {1, 2, 3, 4, 5, 6, 100}, and the vertical coordinate sequence of the above-mentioned building contour points can be {5, 6, 7, 8, 9, 10, 100}, n=7, (n+1)÷4=2, 3(n+1)÷4=6, then the first quartile of the horizontal coordinate of the above-mentioned building contour point horizontal coordinate sequence can be Q1=Data2=2, and the third quartile of the horizontal coordinate can be Q3=Data6=6. The upper limit of the horizontal coordinate of the building contour point is Q3+1.5×(Q3-Q1)=6+1.5×(6-2)=12, and the lower limit of the horizontal coordinate of the building contour point is Q1-1.5×(Q3-Q1)=2-1.5×(6-2)=-4. The first quartile of the vertical coordinate sequence of the building outline points can be Q1=Data2=6, the third quartile of the vertical coordinate can be Q3=Data6=10, the upper limit of the vertical coordinate of the building outline points is Q3+1.5×(Q3-Q1)=10+1.5×(10-6)=16, and the lower limit of the vertical coordinate of the building outline points is Q1-1.5×(Q3-Q1)=6-1.5×(10-6)=0. Since the above building outline point set can be {(2,10), (1,6), (6,7), (4,5), (5,9), (3,8), (100,100)}, the building outline point set after elimination is {(2,10), (1,6), (6,7), (4,5), (5,9), (3,8)}.

[0073] The seventh step is to perform segmented fitting based on the above-mentioned building contour point set after elimination to obtain a building contour line segment set.

[0074] In practice, the execution entity may first perform linear clustering on the set of eliminated building contour points to obtain a set of eliminated building contour point subsets. Within each set of eliminated building contour point subsets, the eliminated building contour points within each set of eliminated building contour point subsets are located around the same line segment. Finally, for each set of eliminated building contour point subsets within the set of eliminated building contour point subsets, the execution entity may perform linear regression fitting on the set of eliminated building contour point subsets to obtain building contour line segments within the set of building contour line segments.

[0075] In the eighth step, the building contour lines in the above building contour line segment set are smoothed to obtain building contour information.

[0076] In practice, the execution entity smoothly connects each building outline in the building outline segment set to obtain building outline information.

[0077] In the ninth step, the video image is cropped according to the building outline information to obtain a cropped video image in the cropped video image sequence.

[0078] In practice, the execution entity may crop the video image according to the building outline information to obtain the cropped video image in the cropped video image sequence.

[0079] Step 1032: Determine the type of the work surface based on the cropped video image sequence.

[0080] In some embodiments, the execution entity may determine the type of the work surface based on the cropped video image sequence.

[0081] In practice, the execution entity may input the cropped video image sequence into a recurrent neural network to determine the type of the work surface, wherein the recurrent neural network may be an LSTM (Long Short-Term Memory) network.

[0082] In some optional implementations of some embodiments, the execution entity may determine the type of the work surface according to the cropped video image sequence, which may include the following steps:

[0083] In the first step, a pre-trained working surface feature extraction model is used to extract working surface features from the cropped video image sequence to obtain a working surface feature set.

[0084] The working surface feature extraction model is a model that takes the cropped video image sequence as input and outputs a working surface feature set. In practice, the working surface feature extraction model can be a convolutional neural network model.

[0085] In practice, the execution entity may input the cropped video image sequence into the pre-trained work surface feature extraction model to generate a work surface feature set.

[0086] In the second step, based on the above working surface feature set, the working surface type is identified for the working surface corresponding to the above cropped video image sequence to obtain the above working surface type.

[0087] In practice, the execution entity may use the naive Bayes algorithm to identify the type of the working surface by using the working surface feature set to obtain the type of the working surface corresponding to the cropped video image sequence.

[0088] Step 1033: Determine risk area description information based on the cropped video image sequence and the work surface type.

[0089] In some embodiments, the execution entity may determine the risk area description information based on the cropped video image sequence and the work surface type.

[0090] The risk area description information includes: risk area location information and risk area protective facility information. The risk area protective facility information indicates the status of protective facilities in the risk area corresponding to the risk area description information. As an example, the protective facility may be a protective fence. The risk area location information indicates the coordinates of the risk area corresponding to the risk area description information in the image pixel coordinate system.

[0091] In practice, the above-mentioned execution entity can perform risk area detection on the cropped video image sequence according to the type of the working surface to obtain risk area description information.

[0092] Optionally, the above-mentioned working surface type includes: a roof working surface type and a facade working surface type. The above-mentioned roof working surface type represents the working surface corresponding to the roof, and the above-mentioned facade working surface represents the working surface corresponding to the facade.

[0093] In some optional implementations of some embodiments, the execution entity determines the risk area description information based on the cropped video image sequence and the work surface type, which may include the following steps:

[0094] In the first step, in response to determining that the above-mentioned working surface type is a roof working surface type, the following first processing step is performed:

[0095] In the first sub-step, edge detection is performed on the cropped video image sequence to obtain position information of edge risk areas.

[0096] In practice, the execution entity may perform edge detection on the cropped video image sequence using a Sobel operator to obtain position information of edge risk areas.

[0097] In the second sub-step, hole detection is performed on the cropped video image sequence to obtain the location information of the hole risk area.

[0098] In practice, the execution entity may perform hole detection on the cropped video image sequence using an object detection algorithm to obtain location information of the hole risk area. As an example, the object detection algorithm may be the Faster R-CNN (Faster Region-based Convolutional Neural Network) algorithm.

[0099] The third sub-step is to determine the risk area location information included in the risk area description information based on the edge risk area location information and the hole risk area location information.

[0100] In practice, the execution entity may combine the edge risk area location information and the hole risk area location information into the risk area location information included in the risk area description information.

[0101] In the second step, in response to determining that the above-mentioned working surface type is a facade working surface type, hole detection is performed on the above-mentioned cropped video image sequence to obtain the risk area position information included in the above-mentioned risk area description information.

[0102] In practice, the execution entity may perform hole detection on the cropped video image sequence using an object detection algorithm to obtain the risk area location information included in the risk area description information. As an example, the object detection algorithm may be a Faster R-CNN (Faster Region-based Convolutional Neural Network) algorithm.

[0103] In the third step, the risk area feature extraction model trained in advance is used to extract features of the risk area corresponding to the above-mentioned risk area location information to obtain the feature information of the protective facilities in the risk area.

[0104] The risk area feature extraction model is a model that uses the risk area location information as input and outputs the feature information of the risk area protection facilities. In practice, the risk area feature extraction model can be a convolutional neural network model.

[0105] In practice, the above-mentioned execution entity can input the above-mentioned risk area location information into the above-mentioned pre-trained risk area feature extraction model to extract features of the risk area corresponding to the above-mentioned risk area location information to obtain feature information of risk area protection facilities.

[0106] The fourth step is to generate the risk area protection facility information included in the risk area description information based on the risk area protection facility feature information and the pre-trained protection facility identification model.

[0107] The protective facility identification model takes the risk area protective facility feature information as input and outputs the risk area protective facility information included in the risk area description information. In practice, the protective facility identification model can be a deep learning-based target detection model. As an example, the protective facility identification model can be an SSD (Single Shot MultiBox Detector) model.

[0108] In practice, the above-mentioned execution entity can input the above-mentioned risk area protection facility characteristic information into the above-mentioned pre-trained protection facility identification model to generate the risk area protection facility information included in the above-mentioned risk area description information.

[0109] Step 1034: Determine the edge distance of the construction worker corresponding to the construction worker description information based on the risk area location information.

[0110] In some embodiments, the execution entity may determine the edge distance of the construction personnel corresponding to the construction personnel description information based on the risk area location information.

[0111] The edge distance is the shortest distance between the construction personnel description information and the location information of the risk area.

[0112] In practice, the execution entity uses the shortest distance from the construction worker location information included in the construction worker description information to the risk area location information as the edge distance of the construction worker corresponding to the construction worker description information.

[0113] In some optional implementations of some embodiments, the execution entity may determine the edge distance of the construction worker corresponding to the construction worker description information based on the risk area location information, which may include the following steps:

[0114] The first step is to map the boundary coordinate points of the risk area corresponding to the above risk area location information to obtain a set of edge coordinates of the risk area.

[0115] The risk area edge coordinate set represents coordinate points on the corresponding edge of the risk area in the geodetic coordinate system.

[0116] In practice, the execution entity converts the risk area corresponding to the risk area position information from an image pixel coordinate system to coordinate point mapping in a geodetic coordinate system to obtain a set of edge coordinates of the risk area.

[0117] In the second step, based on the risk area edge coordinate set, a region boundary information set corresponding to the risk area position information is generated.

[0118] Each area boundary information in the area boundary information set includes: coordinates of a left endpoint of the boundary and coordinates of a right endpoint of the boundary.

[0119] In practice, the execution entity may fit the risk area edge coordinate set to obtain a set of area boundary information corresponding to the risk area position information.

[0120] In the third step, for each region boundary information in the region boundary information set, the following sub-edge distance determination steps are performed:

[0121] The first sub-step is to determine the vector product of the construction worker position information and the left endpoint coordinates and the right endpoint coordinates included in the risk area edge segment information.

[0122] In the second sub-step, in response to determining that the vector product is less than or equal to zero, the vertical distance from the construction worker position information to the risk area edge line segment information is determined as the sub-edge distance of the construction worker corresponding to the construction worker description information.

[0123] The third sub-step, in response to determining that the above-mentioned vector product is greater than zero, determines the minimum distance from the above-mentioned construction personnel position information to the left endpoint coordinates and the right endpoint coordinates included in the above-mentioned risk area edge segment information as the sub-edge distance of the construction personnel corresponding to the above-mentioned construction personnel description information.

[0124] In the fourth step, the sub-adjacent edge distances that meet the screening conditions in the obtained sub-adjacent edge distance set are determined as the adjacent edge distances of the construction personnel corresponding to the construction personnel description information.

[0125] Among them, the above screening condition is that the distance between the sub-adjacent edges is the smallest.

[0126] The above-mentioned content of "in some optional implementations of some embodiments" is an inventive point of the present disclosure, which solves the following technical problem, namely, "when the construction workers are closer to the edge of the risk area, the probability of an accident increases. However, the edge distance cannot be accurately determined, and thus the construction workers at the edge cannot be effectively warned, making it impossible to effectively ensure the safety of the construction workers' lives." Based on this, first, the present disclosure maps the boundary coordinate points of the risk area corresponding to the above-mentioned risk area location information to obtain a risk area edge coordinate set, wherein the above-mentioned risk area edge coordinate set represents the coordinate points on the corresponding edge of the risk area. The image coordinate system is usually a two-dimensional pixel coordinate. When determining the edge distance, it is necessary to consider the curvature of the earth and the actual geographical location. Therefore, it is converted to a geodetic coordinate system to ensure that the measurement results are consistent with the geographical location, reflect the actual spatial relationship, and thus provide more reliable data. Then, based on the above-mentioned risk area edge coordinate set, a region boundary information set is generated for the risk area corresponding to the above-mentioned risk area location information, wherein each region boundary information in the region boundary information set includes: the coordinates of the left endpoint of the boundary and the coordinates of the right endpoint of the boundary. Then, for each area boundary information in the above-mentioned area boundary information set, the following sub-adjacent edge distance determination steps are performed: the first step is to determine the vector product of the left endpoint coordinates and the right endpoint coordinates included in the above-mentioned construction worker position information and the above-mentioned risk area edge segment information. The second step is to determine, in response to determining that the above-mentioned vector product is less than or equal to zero, the vertical distance from the above-mentioned construction worker position information to the above-mentioned risk area edge segment information is determined as the sub-adjacent edge distance of the construction worker corresponding to the above-mentioned construction worker description information. The third step is to determine, in response to determining that the above-mentioned vector product is greater than zero, the minimum distance from the above-mentioned construction worker position information to the left endpoint coordinates and the right endpoint coordinates included in the above-mentioned risk area edge segment information is determined as the sub-adjacent edge distance of the construction worker corresponding to the above-mentioned construction worker description information. Selecting different distance determination methods according to the position of the construction worker relative to the risk area edge can more accurately evaluate the adjacent edge distance of the construction worker. Finally, the sub-adjacent edge distance that meets the screening conditions in the obtained sub-adjacent edge distance set is determined as the adjacent edge distance of the construction worker corresponding to the above-mentioned construction worker description information. In this way, the distance between construction workers and the edge can be determined more accurately, thereby achieving effective early warning for construction workers near the edge and avoiding situations where the safety of construction workers cannot be effectively guaranteed.

[0127] Step 1035 : Based on the risk area protection facility information, the edge distance, and the construction worker clothing information included in the construction worker description information, an early warning operation is performed on the construction worker corresponding to the construction worker description information.

[0128] In some embodiments, the execution entity may perform early warning operations on the construction personnel corresponding to the construction personnel description information based on the risk area protection facility information, the edge distance, and the construction personnel clothing information included in the construction personnel description information.

[0129] In practice, the above-mentioned execution entities can perform sound warning operations on the construction workers corresponding to the construction worker description information based on the risk area protection facility information, edge distance and construction worker description information including the construction worker clothing information.

[0130] Optionally, the above-mentioned construction personnel clothing information includes: protective clothing wearing status, protective clothing inflation device status and five-point protective belt wearing status, and the above-mentioned five-point protective belt wearing status includes: protective rope retraction device status, safety buckle device status and inertial measurement device status, wherein the above-mentioned protective clothing wearing status indicates whether the construction personnel are wearing inflatable protective clothing, the above-mentioned protective clothing inflation device status indicates whether the protective clothing inflation device included in the above-mentioned inflatable protective clothing inflates the above-mentioned inflatable protective clothing, the above-mentioned five-point protective belt wearing status indicates whether the construction personnel are wearing five-point protective belts, and the above-mentioned protective rope retraction device status indicates whether the protective rope retraction device included in the above-mentioned five-point protective belt retracts the above-mentioned five-point protective belt. The above-mentioned safety buckle device status includes: locked state and unlocked state. The above-mentioned inertial measurement device status includes: stationary state and non-stationary state.

[0131] In some optional implementations of some embodiments, the execution entity performs an early warning operation on the construction workers corresponding to the construction worker description information based on the risk area protection facility information, the edge distance, and the construction worker clothing information included in the construction worker description information, which may include the following steps:

[0132] In the first step, in response to determining that the above-mentioned working surface type is a facade working surface type, the following first warning operation is performed:

[0133] In the first sub-step, in response to determining that the protective clothing wearing status indicates that the construction worker is wearing inflatable protective clothing, the following protective clothing warning operations are performed:

[0134] In response to determining that the safety buckle device is in a locked state and the inertial measurement device is in a non-stationary state, the protective suit inflation device is controlled to inflate the inflatable protective suit.

[0135] In practice, in response to determining that the safety buckle device is in a locked state and the inertial measurement device is in a non-stationary state, the execution entity may control the protective suit inflation device to inflate the inflatable protective suit through a wireless connection.

[0136] In the second sub-step, in response to determining that the wearing state of the inflatable protective suit indicates that the construction worker is not wearing the inflatable protective suit, the following protective belt warning operation is performed:

[0137] In response to determining that the state of the safety buckle device is a locked state and the state of the inertial measurement device is a non-stationary state, the protection rope retraction device is controlled to adjust the five-point protection belt.

[0138] In practice, in response to determining that the safety buckle device is in a locked state and the inertial measurement device is in a non-stationary state, the execution entity may control the protective rope retraction device to tighten the five-point protective belt through a wireless connection.

[0139] In the second step, in response to determining that the above-mentioned working surface type is a roof working surface type, the following second warning operation is performed:

[0140] In response to determining that the safety buckle device is in the locked state and the risk area protective facility information indicates that no protective facilities exist, the following third warning operation is performed on the construction worker corresponding to the construction worker description information based on the edge distance:

[0141] In response to determining that the edge distance satisfies a first screening condition, the protective rope retraction device is controlled to adjust the five-point protective belt, wherein the first screening condition is that the edge distance is less than or equal to a target value.

[0142] As an example, the target value may be 1 meter.

[0143] In practice, in response to determining that the edge distance satisfies the first screening condition, the execution entity may control the protective rope retraction device to tighten the five-point protective belt through a wireless connection.

[0144] The above-mentioned content of "In some optional implementations of some embodiments" is an inventive point of the present disclosure, which solves the following technical problem, namely, "different working surfaces may have unique hazards and environmental conditions. If early warnings are not performed in combination with the characteristics of the working surface, it may not be possible to effectively identify all specific risks, resulting in certain situations being ignored or false alarms, which may pose a direct threat to the life safety of construction workers." Based on this, the present disclosure first performs the following first early warning operation in response to determining that the above-mentioned working surface type is a vertical working surface type: Step 1, in response to determining that the protective clothing wearing state indicates that the construction worker is wearing an inflatable protective suit, the following protective clothing early warning operation is performed: Step 2, in response to determining that the safety buckle device state is locked and the inertial measurement device state is non-stationary, the protective clothing inflation device is controlled to inflate the inflatable protective suit. Step 3, in response to determining that the inflatable protective suit wearing state indicates that the construction worker is not wearing the inflatable protective suit, the following protective belt early warning operation is performed: Step 4, in response to determining that the safety buckle device state is locked and the inertial measurement device state is non-stationary, the protective rope retraction device is controlled to adjust the five-point protective belt. Two warning operations are taken depending on whether or not inflatable protective clothing is worn. If inflatable protective clothing is worn, a protective clothing warning operation is taken, and if inflatable protective clothing is not worn, a protective belt warning operation is taken, thereby enhancing the safety of construction workers. Finally, in response to determining that the above-mentioned working surface type is a rooftop working surface type, the following second warning operation is performed: in response to determining that the state of the above-mentioned safety buckle device is locked and the above-mentioned risk area protection facility information indicates that the protective facility does not exist, according to the above-mentioned edge distance, the following third warning operation is performed on the construction worker corresponding to the above-mentioned construction worker description information: in response to determining that the above-mentioned edge distance meets the first screening condition, the above-mentioned protective rope retraction device is controlled to adjust the above-mentioned five-point protective belt, wherein the above-mentioned first screening condition is that the edge distance is less than or equal to the target value. According to the characteristics of the working surface, that is, the type of working surface, different warning methods are adopted for different working surfaces, thereby improving the safety of construction workers.

[0145] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: Through the high-altitude work protection and early warning methods of some embodiments of the present disclosure, high-altitude work safety is monitored in real time, and automated early warning for high-altitude work safety monitoring is achieved, avoiding the occurrence of direct threats to the life safety of construction workers caused by low detection efficiency and low early warning efficiency of high-altitude work safety monitoring. Specifically, the reasons for the low detection efficiency and early warning efficiency of high-altitude work safety monitoring are: the high-altitude environment is complex, and manual monitoring methods have great subjectivity and blind spots in vision, making it difficult to track all potential hazards in real time. When problems arise, manual intervention may not respond quickly enough, and the detection efficiency and early warning efficiency of high-altitude work safety monitoring cannot be guaranteed, which may pose a direct threat to the life safety of construction workers. Based on this, the high-altitude work protection and early warning methods of some embodiments of the present disclosure first collect real-time monitoring video streams as the video stream to be detected, wherein the video stream to be detected is a video stream of the work surface captured in real time by monitoring equipment within the high-altitude work area. By collecting real-time monitoring video streams, real-time monitoring of high-altitude work safety can be achieved. Next, target detection is performed on the video stream to be detected to generate worker description information, resulting in a set of worker description information. The worker description information in the set represents workers working within the high-altitude work area and includes worker location information and worker attire information. The attire information represents the safety equipment worn by the worker corresponding to the worker description information. By identifying the worker's location and safety equipment, a preliminary assessment of the worker's safety is performed. Finally, for each worker description in the set, the following processing steps are performed: First, based on the worker location information included in the worker description information, the video image contained in the video stream to be detected is segmented to generate cropped video images, resulting in a cropped video image sequence. Cropping the building portion of the video image for subsequent risk area identification reduces interference from image components other than the building during the identification process, enabling more efficient risk area identification, while also reducing computational complexity and improving processing speed. Second, based on the cropped video image sequence, the work surface type is determined. Different work face risk areas have different detection targets, so the work face type needs to be determined. The third step is to determine risk area description information based on the cropped video image sequence and the work face type. This risk area description information includes: risk area location information and risk area protective facility information. The risk area protective facility information represents the status of protective facilities in the risk area corresponding to the risk area description information. This risk area location information and risk area protective facility information are used to further assess the safety of construction workers.The fourth step is to determine the edge distance of the construction personnel corresponding to the construction personnel description information based on the above-mentioned risk area location information. The fifth step is to perform early warning operations on the construction personnel corresponding to the above-mentioned construction personnel description information based on the above-mentioned risk area protective facility information, the above-mentioned edge distance and the construction personnel clothing information included in the above-mentioned construction personnel description information. The safety of the construction personnel is judged from three aspects: protective facilities, edge distance and construction personnel clothing information, and early warning is issued for situations where safety hazards exist. In this way, all potential dangers can be tracked in real time, and early warnings can be issued immediately when problems arise, thereby improving the detection efficiency of high-altitude work safety monitoring and the early warning efficiency of high-altitude work, and avoiding the occurrence of direct threats to the life safety of construction personnel due to low detection efficiency of high-altitude work safety monitoring and low early warning efficiency of high-altitude work.

[0146] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a high-altitude work protection warning device. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the high-altitude work protection and warning device can be specifically applied to various electronic devices.

[0147] like Figure 2As shown, some embodiments of the high-altitude work protection and warning device 200 include: an acquisition unit 201, a target detection unit 202, and a processing unit 203. The acquisition unit 201 is configured to acquire a real-time monitoring video stream as a video stream to be detected, wherein the video stream to be detected is a video stream of the work surface shot in real time by a monitoring device in the high-altitude work area. The target detection unit 202 is configured to perform target detection on the video stream to be detected to generate construction worker description information and obtain a construction worker description information set, wherein the construction worker description information in the construction worker description information set represents the construction workers working in the high-altitude work area, and the construction worker description information in the construction worker description information set includes: construction worker location information and construction worker clothing information, wherein the construction worker clothing information represents the wearing status of the safety equipment of the construction worker corresponding to the construction worker description information. The processing unit 203 is configured to perform the following processing steps for each construction worker description information in the above-mentioned construction worker description information set: based on the construction worker position information included in the above-mentioned construction worker description information, perform image segmentation on the video image contained in the above-mentioned video stream to be detected to generate a cropped video image and obtain a cropped video image sequence. Based on the above-mentioned cropped video image sequence, determine the type of working surface. Based on the above-mentioned cropped video image sequence and the above-mentioned working surface type, determine the risk area description information, wherein the above-mentioned risk area description information includes: risk area location information and risk area protective facility information, wherein the above-mentioned risk area protective facility information represents the protective facility status of the risk area corresponding to the risk area description information. Based on the above-mentioned risk area location information, determine the edge distance of the construction worker corresponding to the above-mentioned construction worker description information. Based on the above-mentioned risk area protective facility information, the above-mentioned edge distance and the above-mentioned construction worker clothing information, perform early warning operations on the construction worker corresponding to the above-mentioned construction worker description information.

[0148] It is understandable that the units recorded in the high-altitude work protection warning device 200 are the same as those in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the high-altitude work protection and warning device 200 and the units included therein, and will not be repeated here.

[0149] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, a computing device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0150] like Figure 3As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory 302 or a program loaded from a storage device 308 into a random access memory 303. Various programs and data required for the operation of the electronic device 300 are also stored in the random access memory 303. The processing device 301, the read-only memory 302, and the random access memory 303 are connected to each other via a bus 304. An input / output interface 305 is also connected to the bus 304.

[0151] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0152] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 309, or installed from the storage device 308, or installed from the read-only memory 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are performed.

[0153] It should be noted that the computer-readable medium described in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0154] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0155] The computer-readable medium may be included in the electronic device or may exist separately and not incorporated into the electronic device. The computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: collect a real-time surveillance video stream as a video stream to be detected, wherein the video stream to be detected is a video stream of a work surface captured in real time by monitoring equipment within the high-altitude work area; perform object detection on the video stream to be detected to generate worker description information, thereby obtaining a set of worker description information. The worker description information in the set of worker description information represents workers working within the high-altitude work area, and the worker description information in the set of worker description information includes worker location information and worker attire information, wherein the worker attire information represents the safety equipment worn by the worker corresponding to the worker description information. For each worker description in the set of worker description information, perform the following processing steps: perform image segmentation on the video image contained in the video stream to be detected based on the worker location information included in the worker description information to generate a cropped video image, thereby obtaining a cropped video image sequence. Based on the cropped video image sequence, the type of working surface is determined. Based on the cropped video image sequence and the type of working surface, risk area description information is determined, wherein the risk area description information includes: risk area location information and risk area protective facility information, wherein the risk area protective facility information represents the status of the protective facilities in the risk area corresponding to the risk area description information. Based on the risk area location information, the edge distance of the construction personnel corresponding to the construction personnel description information is determined. Based on the risk area protective facility information, the edge distance, and the construction personnel clothing information included in the construction personnel description information, an early warning operation is performed on the construction personnel corresponding to the construction personnel description information.

[0156] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0157] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0158] The units described in some embodiments of the present disclosure may be implemented in software or hardware. The units described may also be provided in a processor. For example, they may be described as: a processor including an acquisition unit, a target detection unit, and a processing unit. The names of these units do not, in some cases, constitute limitations on the units themselves. For example, the acquisition unit may also be described as a "unit for acquiring a real-time monitoring video stream as a video stream to be detected."

[0159] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0160] The above description is only an illustration of some preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A high-altitude work protection and early warning method, comprising: Collecting a real-time monitoring video stream as a video stream to be detected, wherein the video stream to be detected is a video stream of the working surface shot in real time by a monitoring device in the high-altitude working area; Performing target detection on the video stream to be detected to generate construction worker description information, obtaining a construction worker description information set, wherein the construction worker description information in the construction worker description information set represents construction workers working in the high-altitude work area, and the construction worker description information in the construction worker description information set includes: construction worker location information and construction worker clothing information, wherein the construction worker clothing information represents the wearing status of safety equipment of the construction worker corresponding to the construction worker description information; For each construction worker description information in the construction worker description information set, perform the following processing steps: Performing image segmentation on the video image included in the video stream to be detected according to the construction worker position information included in the construction worker description information to generate cropped video images, thereby obtaining a cropped video image sequence; determining a work surface type according to the cropped video image sequence; Determining risk area description information based on the cropped video image sequence and the type of the working surface, wherein the risk area description information includes: risk area location information and risk area protection facility information, wherein the risk area protection facility information represents the status of the protection facilities in the risk area corresponding to the risk area description information; According to the risk area location information, the edge distance of the construction personnel description information corresponding to the construction personnel is determined, including: mapping the boundary coordinate points of the risk area corresponding to the risk area location information to obtain a risk area edge coordinate set; generating a region boundary information set for the risk area corresponding to the risk area location information according to the risk area edge coordinate set, wherein each region boundary information in the region boundary information set includes: the coordinates of the left endpoint of the boundary and the coordinates of the right endpoint of the boundary; for each region boundary information in the region boundary information set, the following sub-edge distance determination steps are performed: determining the distance between the construction personnel location information and the risk area edge line The vector product of the left endpoint coordinates and the right endpoint coordinates included in the segment information; in response to determining that the vector product is less than or equal to zero, determining the vertical distance from the construction worker position information to the risk area edge segment information as the sub-adjacent edge distance of the construction worker corresponding to the construction worker description information; in response to determining that the vector product is greater than zero, determining the minimum distance from the construction worker position information to the left endpoint coordinates and the right endpoint coordinates included in the risk area edge segment information as the sub-adjacent edge distance of the construction worker corresponding to the construction worker description information; and determining the sub-adjacent edge distance that meets the screening condition in the obtained sub-adjacent edge distance set as the adjacent edge distance of the construction worker corresponding to the construction worker description information; Performing an early warning operation on the construction workers corresponding to the construction worker description information based on the risk area protection facility information, the edge distance, and the construction worker clothing information included in the construction worker description information; The step of performing image segmentation on the video image contained in the video stream to be detected based on the construction worker position information included in the construction worker description information to generate a cropped video image and obtain a cropped video image sequence includes: For each video image of a construction worker in the video stream to be detected and containing the construction worker description information, the following segmentation steps are performed: Converting the video image into a grayscale image to obtain a video grayscale image; Performing edge detection on the video grayscale image according to the construction worker position information included in the construction worker description information to obtain an edge-detected image; Performing contour extraction on the edge-detected image to generate a building contour point set, wherein the building contour points in the building contour point set include: contour point abscissas and contour point ordinates; Classifying the building contour point set by the horizontal and vertical coordinates to obtain a building contour point horizontal coordinate set and a building contour point vertical coordinate set; sorting the building outline point abscissa set and the building outline point ordinate set respectively to obtain a building outline point abscissa sequence and a building outline point ordinate sequence; Generate a set of building contour points after elimination according to the building contour point abscissa sequence and the building contour point ordinate sequence; Performing segmented fitting based on the eliminated building contour point set to obtain a building contour line segment set; Smoothing the building contour lines in the building contour line segment set to obtain building contour information; Cropping the video image according to the building outline information to obtain a cropped video image in the cropped video image sequence; Among them, according to the horizontal coordinate sequence of the building outline points, the first quartile calculation formula and the third quartile calculation formula are used to determine the first quartile and the third quartile of the horizontal coordinate sequence of the building outline points; according to the vertical coordinate sequence of the building outline points, the first quartile and the third quartile of the vertical coordinate sequence of the building outline points are determined by the first quartile calculation formula and the third quartile calculation formula; according to the first quartile and the third quartile of the horizontal coordinate, the upper limit of the horizontal coordinate of the building outline points and the upper limit of the building outline points are determined by the target formula. a lower limit of the horizontal coordinate of the building contour point; determining the upper limit of the vertical coordinate of the building contour point and the lower limit of the vertical coordinate of the building contour point by the target formula according to the first quartile of the vertical coordinate and the third quartile of the vertical coordinate; removing at least one building contour point in the building contour point set that does not meet the contour point screening condition from the building contour point set to obtain a removed building contour point set, wherein the contour point screening condition is the lower limit of the horizontal coordinate of the building contour point < the horizontal coordinate < the upper limit of the horizontal coordinate of the building contour point or the lower limit of the vertical coordinate of the building contour point < the vertical coordinate < the upper limit of the vertical coordinate of the building contour point; Among them, the calculation formula for the first quartile is: if Not an integer, is the interpolated value between the data points at this location and the next location: in, is the first quartile, is the number of data points, For the data set Data points at locations, is less than or equal to data points, is greater than data points, yes the decimal part of ; Among them, the calculation formula for the third quartile is: if Not an integer, is the interpolated value between the data points at this location and the next location: in, is the third quartile, is the number of data points, For the data set Data points at locations, is less than or equal to data points, is greater than data points, yes The decimal part of Among them, the target formula is upper limit = , lower limit = , is the first quartile, The third quartile.

2. The method according to claim 1, wherein The target detection is performed on the video stream to be detected to generate construction personnel description information, and a construction personnel description information set is obtained, including: Performing denoising on the video stream to be detected to obtain a denoised video stream; Normalizing the denoised video stream to obtain a preprocessed video stream; The construction worker description information set is generated by using a pre-trained construction worker detection model and the pre-processed video stream.

3. The method according to claim 2, wherein: The determining the type of the work surface according to the cropped video image sequence includes: Performing work surface feature extraction on the cropped video image sequence using a pre-trained work surface feature extraction model to obtain a work surface feature set; According to the working surface feature set, working surface type identification is performed on the working surface corresponding to the cropped video image sequence to obtain the working surface type.

4. The method according to claim 3, wherein: The working surface types include: roof working surface type and facade working surface type; and The determining of risk area description information according to the cropped video image sequence and the work surface type includes: In response to determining that the working surface type is a rooftop working surface type, the following first processing step is performed: Performing edge detection on the cropped video image sequence to obtain edge risk area position information; Performing hole detection on the cropped video image sequence to obtain hole hole risk area location information; Determining the risk area location information included in the risk area description information according to the edge risk area location information and the hole risk area location information; In response to determining that the work surface type is a facade work surface type, performing hole detection on the cropped video image sequence to obtain risk area position information included in the risk area description information; Using a pre-trained risk area feature extraction model, feature extraction is performed on the risk area corresponding to the risk area location information to obtain feature information of protective facilities in the risk area; The risk area protection facility information included in the risk area description information is generated according to the risk area protection facility feature information and a pre-trained protection facility identification model.

5. A high-altitude work protection and warning device, comprising: The acquisition unit is configured to acquire a real-time monitoring video stream as a video stream to be detected, wherein the video stream to be detected is a video stream of the working surface shot in real time by a monitoring device in the high-altitude working area; The target detection unit is configured to perform target detection on the video stream to be detected to generate construction worker description information and obtain a construction worker description information set, wherein the construction worker description information in the construction worker description information set represents construction workers working in the high-altitude working area, and the construction worker description information in the construction worker description information set includes: construction worker location information and construction worker clothing information, wherein the construction worker clothing information represents the wearing status of safety equipment of the construction worker corresponding to the construction worker description information; The processing unit is configured to perform the following processing steps for each construction worker description information in the construction worker description information set: according to the construction worker position information included in the construction worker description information, perform image segmentation on the video image contained in the video stream to be detected to generate a cropped video image and obtain a cropped video image sequence; according to the cropped video image sequence, determine the type of working surface; according to the cropped video image sequence and the working surface type, determine the risk area description information, wherein the risk area description information includes: risk area position information and risk area protection facility information, wherein the risk area protection facility information represents the protection facility status of the risk area corresponding to the risk area description information; according to the risk area position information, determine the edge distance of the construction worker corresponding to the construction worker description information, including: mapping the boundary coordinate points of the risk area corresponding to the risk area position information to obtain a risk area edge coordinate set; according to the risk area edge coordinate set, generate a region boundary information set for the risk area corresponding to the risk area position information, wherein the Each area boundary information in the area boundary information set includes: the coordinates of the left endpoint of the boundary and the coordinates of the right endpoint of the boundary; for each area boundary information in the area boundary information set, the following sub-adjacent edge distance determination step is performed: determining the vector product of the left endpoint coordinates and the right endpoint coordinates included in the construction worker position information and the risk area edge segment information; in response to determining that the vector product is less than or equal to zero, determining the vertical distance from the construction worker position information to the risk area edge segment information as the sub-adjacent edge distance of the construction worker corresponding to the construction worker description information; in response to determining that the vector product is greater than zero, determining the minimum distance from the construction worker position information to the left endpoint coordinates and the right endpoint coordinates included in the risk area edge segment information as the sub-adjacent edge distance of the construction worker corresponding to the construction worker description information; determining the sub-adjacent edge distance that meets the screening condition in the obtained sub-adjacent edge distance set as the adjacent edge distance of the construction worker corresponding to the construction worker description information; performing an early warning operation on the construction worker corresponding to the construction worker description information based on the risk area protection facility information, the adjacent edge distance and the construction worker clothing information; The step of performing image segmentation on the video image contained in the video stream to be detected based on the construction worker position information included in the construction worker description information to generate a cropped video image and obtain a cropped video image sequence includes: For each video image of a construction worker in the video stream to be detected and containing the construction worker description information, the following segmentation steps are performed: Converting the video image into a grayscale image to obtain a video grayscale image; Performing edge detection on the video grayscale image according to the construction worker position information included in the construction worker description information to obtain an edge-detected image; Performing contour extraction on the edge-detected image to generate a building contour point set, wherein the building contour points in the building contour point set include: contour point abscissas and contour point ordinates; Classifying the building contour point set by the horizontal and vertical coordinates to obtain a building contour point horizontal coordinate set and a building contour point vertical coordinate set; sorting the building outline point abscissa set and the building outline point ordinate set respectively to obtain a building outline point abscissa sequence and a building outline point ordinate sequence; Generate a set of building contour points after elimination according to the building contour point abscissa sequence and the building contour point ordinate sequence; Performing segmented fitting based on the eliminated building contour point set to obtain a building contour line segment set; Smoothing the building contour lines in the building contour line segment set to obtain building contour information; Cropping the video image according to the building outline information to obtain a cropped video image in the cropped video image sequence; Among them, according to the horizontal coordinate sequence of the building outline points, the first quartile calculation formula and the third quartile calculation formula are used to determine the first quartile and the third quartile of the horizontal coordinate sequence of the building outline points; according to the vertical coordinate sequence of the building outline points, the first quartile and the third quartile of the vertical coordinate sequence of the building outline points are determined by the first quartile calculation formula and the third quartile calculation formula; according to the first quartile and the third quartile of the horizontal coordinate, the upper limit of the horizontal coordinate of the building outline points and the upper limit of the building outline points are determined by the target formula. a lower limit of the horizontal coordinate of the building contour point; determining the upper limit of the vertical coordinate of the building contour point and the lower limit of the vertical coordinate of the building contour point by the target formula according to the first quartile of the vertical coordinate and the third quartile of the vertical coordinate; removing at least one building contour point in the building contour point set that does not meet the contour point screening condition from the building contour point set to obtain a removed building contour point set, wherein the contour point screening condition is the lower limit of the horizontal coordinate of the building contour point < the horizontal coordinate < the upper limit of the horizontal coordinate of the building contour point or the lower limit of the vertical coordinate of the building contour point < the vertical coordinate < the upper limit of the vertical coordinate of the building contour point; Among them, the calculation formula for the first quartile is: if Not an integer, is the interpolated value between the data points at this location and the next location: in, is the first quartile, is the number of data points, For the data set Data points at locations, is less than or equal to data points, is greater than data points, yes the decimal part of ; Among them, the calculation formula for the third quartile is: if Not an integer, is the interpolated value between the data points at this location and the next location: in, is the third quartile, is the number of data points, For the data set Data points at locations, is less than or equal to data points, is greater than data points, yes The decimal part of Among them, the target formula is upper limit = , lower limit = , is the first quartile, The third quartile.

6. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer-readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images