High-altitude operation protection early warning method and device and computer readable medium

Abstract

The embodiment of the invention discloses a high-altitude operation protection early warning method and device and a computer readable medium. A specific embodiment of the method comprises the following steps: collecting a real-time monitoring video stream as a to-be-detected video stream; performing target detection on the to-be-detected video stream; for each piece of construction personnel description information in the construction personnel description information set, the following processing steps are executed: carrying out image segmentation on a video image; determining a working face type; determining risk area description information; according to the risk area position information, determining the limb distance of the constructor corresponding to the constructor description information; and according to the risk area protection facility information, the edge distance and the construction personnel dressing information included in the construction personnel description information, performing early warning operation on the construction personnel corresponding to the construction personnel description information. According to the embodiment, the high-altitude operation safety is monitored in real time, automatic early warning for high-altitude operation safety monitoring is achieved, and the situation that the life safety of constructors is directly threatened due to the fact that the detection efficiency of high-altitude operation safety monitoring and the early warning efficiency of high-altitude operation are low is avoided.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method, device, and computer-readable medium for early warning of protection against aerial work. Background Art

[0002] During the construction process, due to insufficient risk awareness, there are often many safety hazards in high-altitude operations, and the safety hazards that induce construction workers to fall often pose a direct threat to the lives of construction workers. At present, when conducting safety monitoring of high-altitude operations, the usual method is: construction site managers monitor the safety of construction workers through surveillance videos.

[0003] However, the inventors have found that when the above method is adopted, the following technical problems often occur:

[0004] The high-altitude working environment is complex, and manual monitoring methods have great subjectivity and blind spots, making it difficult to track all potential dangers in real time. When problems arise, manual intervention may not respond quickly enough to ensure the detection efficiency of high-altitude working safety monitoring and the early warning efficiency of high-altitude working, which may pose a direct threat to the lives of construction workers.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose a high-altitude work protection warning method, device and computer-readable medium to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In the first aspect, some embodiments of the present disclosure provide a method for high-altitude work protection and early warning, the method comprising: collecting a real-time monitoring video stream as a video stream to be detected, wherein the video stream to be detected is a video stream for a work surface shot in real time by a monitoring device in a high-altitude work area. Performing target detection on the video stream to be detected to generate construction personnel description information, and obtaining a set of construction personnel description information, wherein the construction personnel description information in the construction personnel description information set represents the construction personnel working in the high-altitude work area, and the construction personnel description information in the construction personnel description information set includes: construction personnel position information and construction personnel clothing information, wherein the construction personnel clothing information represents the wearing status of the safety equipment of the construction personnel corresponding to the construction personnel description information. For each construction personnel description information in the construction personnel description information set, performing the following processing steps: according to the construction personnel position information included in the construction personnel description information, performing image segmentation on the video image contained in the video stream to be detected to generate a cropped video image, and obtaining a cropped video image sequence. According to the cropped video image sequence, determining the type of the work surface. Based on the above cropped video image sequence and the above work surface type, the risk area description information is determined, wherein the above risk area description information includes: risk area location information and risk area protection facility information, wherein the above risk area protection facility information represents the protection facility status of the risk area corresponding to the risk area description information. Based on the above risk area location information, the edge distance of the construction personnel corresponding to the above construction personnel description information is determined. Based on the above risk area protection facility information, the above edge distance and the construction personnel clothing information included in the above construction personnel description information, an early warning operation is performed on the construction personnel corresponding to the above construction personnel description information.

[0009] In a second aspect, some embodiments of the present disclosure provide a high-altitude work protection warning device, the device comprising: a collection unit, configured to collect a real-time monitoring video stream as a video stream to be detected, wherein the video stream to be detected is a video stream for a work surface shot in real time by a monitoring device in the high-altitude work area. A target detection unit, configured to perform target detection on the video stream to be detected to generate construction personnel description information and obtain a construction personnel description information set, wherein the construction personnel description information in the construction personnel description information set represents the construction personnel working in the high-altitude work area, and the construction personnel description information in the construction personnel description information set includes: construction personnel location information and construction personnel clothing information, wherein the construction personnel clothing information represents the wearing status of the safety equipment of the construction personnel corresponding to the construction personnel description information. The processing unit is configured to execute the following processing steps for each construction worker description information in the above-mentioned construction worker description information set: according to the construction worker position information included in the above-mentioned construction worker description information, perform image segmentation on the video image contained in the above-mentioned video stream to be detected to generate a cropped video image and obtain a cropped video image sequence; according to the above-mentioned cropped video image sequence, determine the type of working surface; according to the above-mentioned cropped video image sequence and the above-mentioned working surface type, determine the risk area description information, wherein the above-mentioned risk area description information includes: risk area position information and risk area protection facility information, wherein the above-mentioned risk area protection facility information represents the protection facility status of the risk area corresponding to the risk area description information; according to the above-mentioned risk area position information, determine the edge distance of the construction worker corresponding to the above-mentioned construction worker description information; according to the above-mentioned risk area protection facility information, the above-mentioned edge distance and the above-mentioned construction worker clothing information, perform early warning operations on the construction worker corresponding to the above-mentioned construction worker description information.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0012] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the high-altitude operation protection and early warning method of some embodiments of the present disclosure, the safety of high-altitude operation is monitored in real time, and the automatic early warning for the safety monitoring of high-altitude operation is realized, avoiding the occurrence of direct threats to the life safety of construction personnel caused by the low detection efficiency of the safety monitoring of high-altitude operation and the low early warning efficiency of high-altitude operation. Specifically, the reasons for the low detection efficiency of the safety monitoring of high-altitude operation and the low early warning efficiency of high-altitude operation are: the high-altitude environment is complex, the manual monitoring method has great subjectivity and blind spots in vision, and it is difficult to track all potential dangers in real time. When problems occur, the response of manual intervention may not be fast enough, and the detection efficiency of the safety monitoring of high-altitude operation and the early warning efficiency of high-altitude operation cannot be guaranteed, which may cause a direct threat to the life safety of construction personnel. Based on this, the high-altitude operation protection and early warning method of some embodiments of the present disclosure, first, collects real-time monitoring video streams as video streams to be detected, wherein the above-mentioned video streams to be detected are video streams for the working surface shot in real time by monitoring equipment in the high-altitude operation area. Real-time monitoring of the safety of high-altitude operation can be achieved by collecting real-time monitoring video streams. Next, target detection is performed on the above-mentioned video stream to be detected to generate construction personnel description information and obtain a set of construction personnel description information, wherein the construction personnel description information in the above-mentioned construction personnel description information set represents the construction personnel working in the above-mentioned high-altitude working area, and the construction personnel description information in the above-mentioned construction personnel description information set includes: construction personnel position information and construction personnel clothing information, wherein the construction personnel clothing information represents the wearing status of the safety equipment of the construction personnel corresponding to the construction personnel description information. By identifying the position of the construction personnel and the wearing status of the safety equipment, the safety of the construction personnel is preliminarily judged. Finally, for each construction personnel description information in the above-mentioned construction personnel description information set, the following processing steps are performed: the first step is to perform image segmentation on the video image contained in the above-mentioned video stream to be detected according to the construction personnel position information included in the above-mentioned construction personnel description information to generate a cropped video image and obtain a cropped video image sequence. Cropping the building part in the video image for subsequent risk area identification can reduce the interference of other parts of the image except the building during the identification process, can more effectively identify the risk area, and can also reduce the computational complexity and improve the processing speed. The second step is to determine the type of the working surface according to the above-mentioned cropped video image sequence. Different working face risk areas have different detection targets, so it is necessary to determine the working face type. The third step is to determine the risk area description information based on the above-mentioned cropped video image sequence and the above-mentioned working face type, wherein the above-mentioned risk area description information includes: risk area location information and risk area protection facility information, wherein the above-mentioned risk area protection facility information represents the protection facility status of the risk area corresponding to the risk area description information. The safety of the construction personnel is further judged through the risk area location information and risk area protection facility information.The fourth step is to determine the edge distance of the construction personnel corresponding to the construction personnel description information according to the above risk area location information. The fifth step is to perform early warning operations on the construction personnel corresponding to the above construction personnel description information according to the above risk area protection facility information, the above edge distance and the construction personnel clothing information included in the above construction personnel description information. The safety of construction personnel is judged from three aspects: protective facilities, edge distance and construction personnel clothing information, and early warning is issued for situations where safety hazards exist. In this way, all potential dangers can be tracked in real time, and early warnings can be issued immediately when problems occur, thereby improving the detection efficiency of high-altitude work safety monitoring and the early warning efficiency of high-altitude work, and avoiding the occurrence of direct threats to the life safety of construction personnel due to low detection efficiency of high-altitude work safety monitoring and low early warning efficiency of high-altitude work. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 is a flow chart of some embodiments of the high-altitude operation protection and early warning method disclosed in the present invention;

[0015] Figure 2 It is a schematic diagram of the structure of some embodiments of the high-altitude operation protection and warning device disclosed in the present invention;

[0016] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0019] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] refer to Figure 1 , shows a process 100 of some embodiments of the high-altitude operation protection and early warning method according to the present disclosure. The high-altitude operation protection and early warning method comprises the following steps:

[0024] Step 101: collect a real-time monitoring video stream as a video stream to be detected.

[0025] In some embodiments, the executor of the high-altitude operation protection warning method (for example, a computing device) can collect real-time monitoring video streams as video streams to be detected through wired or wireless connections.

[0026] The video stream to be detected is a video stream of the working surface shot in real time by the monitoring equipment in the high-altitude working area. The high-altitude working area is an area where construction workers work at a height of 2 meters or more from the ground. The monitoring equipment represents a monitoring camera located in the high-altitude working area and facing the working surface.

[0027] It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or developed in the future.

[0028] Step 102: perform target detection on the video stream to be detected to generate construction worker description information and obtain a construction worker description information set.

[0029] In some embodiments, the execution entity may perform target detection on the video stream to be detected to generate construction personnel description information and obtain a construction personnel description information set.

[0030] The construction worker description information in the above-mentioned construction worker description information set represents the construction workers working in the above-mentioned high-altitude working area. The construction worker description information in the above-mentioned construction worker description information set includes: construction worker location information and construction worker clothing information. The construction worker clothing information represents the wearing status of the safety equipment of the construction worker corresponding to the construction worker description information. As an example, the above-mentioned safety equipment can be a safety helmet, protective clothing and a protective rope. The above-mentioned construction worker location information represents the current position of the corresponding construction worker in the working surface.

[0031] In practice, the execution subject may perform target detection on the video stream to be detected by using an SSD (Single Shot MultiBox Detector) algorithm to generate construction worker description information and obtain a construction worker description information set.

[0032] In some optional implementations of some embodiments, the execution subject performs target detection on the video stream to be detected to generate construction personnel description information, and obtains a construction personnel description information set, which may include the following steps:

[0033] The first step is to perform denoising on the above video stream to be detected to obtain a denoised video stream.

[0034] In practice, the execution subject may perform denoising on the video stream to be detected by Gaussian filtering to obtain a denoised video stream.

[0035] The second step is to normalize the denoised video stream to obtain a preprocessed video stream.

[0036] In practice, the execution subject may perform normalization processing on the denoised video stream by a linear normalization method to obtain a preprocessed video stream.

[0037] The third step is to generate the above-mentioned construction personnel description information set through the pre-trained construction personnel detection model and the above-mentioned preprocessed video stream.

[0038] The construction worker detection model is a model that takes the preprocessed video stream as input and the construction worker description information set as output. In practice, the construction worker detection model may be a target detection model. As an example, the construction worker detection model may be a YOLO V5 (You Only Look Once Version 5) model.

[0039] In practice, the execution entity may generate the construction worker description information set by inputting the preprocessed video stream into the pre-trained construction worker detection model.

[0040] Step 103: for each construction worker description information in the construction worker description information set, perform the following processing steps:

[0041] Step 1031 , according to the construction worker position information included in the construction worker description information, the video image included in the video stream to be detected is segmented to generate a cropped video image, thereby obtaining a cropped video image sequence.

[0042] In some embodiments, the execution entity may perform image segmentation on the video image included in the video stream to be detected according to the construction worker location information included in the construction worker description information to generate a cropped video image and obtain a cropped video image sequence.

[0043] In practice, the execution subject may perform image segmentation on the video image contained in the video stream to be detected by using a deep learning model according to the construction worker location information included in the construction worker description information, so as to generate a cropped video image and obtain a cropped video image sequence. As an example, the deep learning model may be a U-Net model.

[0044] In some optional implementations of some embodiments, the execution subject performs image segmentation on the video image included in the video stream to be detected according to the construction worker position information included in the construction worker description information to generate a cropped video image, and obtaining a cropped video image sequence may include the following steps:

[0045] For each video image of a construction worker corresponding to the construction worker description information in the above-mentioned video stream to be detected, the following segmentation steps are performed:

[0046] The first step is to convert the above video image into a grayscale image to obtain a video grayscale image.

[0047] In practice, the execution subject may convert the video image into a grayscale image by weighted averaging to obtain a video grayscale image.

[0048] In the second step, according to the construction worker position information included in the construction worker description information, edge detection is performed on the video grayscale image to obtain an image after edge detection.

[0049] In practice, the execution subject may perform edge detection on the grayscale image of the video according to the construction worker position information included in the construction worker description information by using the Canny edge detection algorithm to obtain an image after edge detection. The image after edge detection represents an image after edge framing of the construction worker corresponding to the construction worker position information.

[0050] The third step is to perform contour extraction on the above edge detected image to generate a set of building contour points.

[0051] The building contour points in the above building contour point set include: contour point abscissas and contour point ordinates.

[0052] In practice, the execution entity may perform contour extraction on the image after edge detection along the edges framed in the image after edge detection to generate a set of building contour points.

[0053] The fourth step is to classify the building contour point set based on the horizontal and vertical coordinates of the contour points to obtain a building contour point horizontal coordinate set and a building contour point vertical coordinate set.

[0054] In practice, the above-mentioned execution entity can determine the contour point horizontal coordinates of each building contour point in the above-mentioned building contour point set as a building contour point horizontal coordinate set, and determine the contour point vertical coordinates of each building contour point in the above-mentioned building contour point set as a building contour point vertical coordinate set.

[0055] As an example, the above-mentioned building contour point set can be {(2, 10), (1, 6), (6, 7), (4, 5), (5, 9), (3, 8), (100, 100)}, then the building contour point horizontal coordinate set can be {2, 1, 6, 4, 5, 3, 100}, and the building contour point vertical coordinate set can be {10, 6, 7, 5, 9, 8, 100}.

[0056] In the fifth step, the abscissa set of the building contour points and the ordinate set of the building contour points are sorted respectively to obtain a abscissa sequence of the building contour points and a ordinate sequence of the building contour points.

[0057] In practice, the execution entity may arrange the abscissa set of the building outline points and the ordinate set of the building outline points in descending order from large to small to obtain a abscissa sequence of the building outline points and a ordinate sequence of the building outline points.

[0058] As an example, the horizontal coordinate set of the building contour points can be {2, 1, 6, 4, 5, 3, 100}, and the vertical coordinate set of the building contour points can be {10, 6, 7, 5, 9, 8, 100}. Therefore, the horizontal coordinate sequence of the building contour points can be {1, 2, 3, 4, 5, 6, 100}, and the vertical coordinate sequence of the building contour points can be {5, 6, 7, 8, 9, 10, 100}.

[0059] Step 6: Generate a set of building contour points after elimination based on the above building contour point abscissa sequence and building contour point ordinate sequence.

[0060] In practice, first, the execution subject may determine the first quartile of the horizontal coordinate and the third quartile of the horizontal coordinate of the building outline point horizontal coordinate sequence according to the horizontal coordinate sequence of the building outline point through the first quartile calculation formula and the third quartile calculation formula. Then, the execution subject may determine the first quartile of the vertical coordinate and the third quartile of the vertical coordinate sequence of the building outline point according to the vertical coordinate sequence of the building outline point through the first quartile calculation formula and the third quartile calculation formula. Next, the execution subject may determine the upper limit of the horizontal coordinate of the building outline point and the lower limit of the horizontal coordinate of the building outline point through the target formula according to the first quartile of the horizontal coordinate and the third quartile of the horizontal coordinate. Further, the execution subject may determine the upper limit of the vertical coordinate and the lower limit of the vertical coordinate of the building outline point through the target formula according to the first quartile of the vertical coordinate and the third quartile of the vertical coordinate. Finally, the above-mentioned execution entity can eliminate at least one building contour point in the above-mentioned building contour point set that does not meet the contour point screening condition from the above-mentioned building contour point set to obtain the eliminated building contour point set, wherein the above-mentioned contour point screening condition is the lower limit of the horizontal coordinate of the above-mentioned building contour point < the horizontal coordinate < the upper limit of the horizontal coordinate of the above-mentioned building contour point or the lower limit of the vertical coordinate of the above-mentioned building contour point < the vertical coordinate < the upper limit of the vertical coordinate of the above-mentioned building contour point.

[0061] Among them, the calculation formula for the first quartile is:

[0062]

[0063] if Rather than being an integer, Q1 is the interpolated value between the data points at that location and the next location:

[0064] Q1=Data lower +[fraction×(Data upper -Data lower )]

[0065] Where Q1 is the first quartile, n is the number of data points, For the data set Data points at each location, Data lower is less than or equal to Data points upper is greater than The data point, fraction is The decimal part of .

[0066] Among them, the calculation formula for the third quartile is

[0067]

[0068] if Rather than being an integer, Q3 is the interpolated value between the data points at this location and the next location:

[0069] Q3=Data lower +[fraction×(Data upper -Data lower )]

[0070] Among them, Q3 is the third quartile, n is the number of data points, For the data set Data points at each location, Data lower is less than or equal to Data points upper is greater than The data point, fraction is The decimal part of .

[0071] Among them, the above target formula is upper limit = Q3 + 1.5 × (Q3-Q1), lower limit = Q1-1.5 × (Q3-Q1), Q1 is the first quartile, and Q3 is the third quartile.

[0072] As an example, the horizontal coordinate sequence of the above-mentioned building contour points can be {1, 2, 3, 4, 5, 6, 100}, and the vertical coordinate sequence of the above-mentioned building contour points can be {5, 6, 7, 8, 9, 10, 100}, n=7, (n+1)÷4=2, 3(n+1)÷4=6, then the first quartile of the horizontal coordinate of the above-mentioned building contour point horizontal coordinate sequence can be Q1=Data2=2, and the third quartile of the horizontal coordinate can be Q3=Data6=6, the upper limit of the horizontal coordinate of the building contour point is Q3+1.5×(Q3-Q1)=6+1.5×(6-2)=12, and the lower limit of the horizontal coordinate of the building contour point is Q1-1.5×(Q3-Q1)=2-1.5×(6-2)=-4. The first quartile of the ordinate of the ordinate sequence of the building contour points can be Q1=Data2=6, the third quartile of the ordinate can be Q3=Data6=10, the upper limit of the ordinate of the building contour points is Q3+1.5×(Q3-Q1)=10+1.5×(10-6)=16, and the lower limit of the ordinate of the building contour points is Q1-1.5×(Q3-Q1)=6-1.5×(10-6)=0. Since the above building contour point set can be {(2,10), (1,6), (6,7), (4,5), (5,9), (3,8), (100,100)}, the building contour point set after elimination is {(2,10), (1,6), (6,7), (4,5), (5,9), (3,8)}.

[0073] The seventh step is to perform piecewise fitting based on the above-mentioned building contour point set after elimination to obtain a building contour line segment set.

[0074] In practice, first, the execution subject may linearly cluster the above-mentioned building contour point set after elimination to obtain a group of subsets of building contour points after elimination. Among them, in the above-mentioned group of subsets of building contour points after elimination, each of the building contour points after elimination in each subset of building contour points after elimination is located around the same line segment. Finally, for each subset of building contour points after elimination in the above-mentioned group of subsets of building contour points after elimination, the execution subject may fit the above-mentioned subset of building contour points after elimination by a linear regression method to obtain a building contour line segment in the building contour line segment set.

[0075] In the eighth step, the building contour lines in the above building contour line segment set are smoothed to obtain building contour information.

[0076] In practice, the execution entity smoothly connects each building outline in the building outline segment set to obtain building outline information.

[0077] In the ninth step, the video image is cropped according to the building outline information to obtain a cropped video image in the cropped video image sequence.

[0078] In practice, the execution entity may crop the video image according to the building outline information to obtain the cropped video image in the cropped video image sequence.

[0079] Step 1032: Determine the type of the work surface according to the cropped video image sequence.

[0080] In some embodiments, the execution entity may determine the type of the work surface according to the cropped video image sequence.

[0081] In practice, the execution subject may input the cropped video image sequence into a recurrent neural network to determine the type of the work surface, wherein the recurrent neural network may be an LSTM (Long Short-Term Memory) network.

[0082] In some optional implementations of some embodiments, the execution subject determines the type of the work surface according to the cropped video image sequence, which may include the following steps:

[0083] In the first step, the working surface features are extracted from the above cropped video image sequence through a pre-trained working surface feature extraction model to obtain a working surface feature set.

[0084] The working surface feature extraction model is a model that takes the cropped video image sequence as input and takes the working surface feature set as output. In practice, the working surface feature extraction model can be a convolutional neural network model.

[0085] In practice, the execution entity may input the cropped video image sequence into the pre-trained work surface feature extraction model to generate a work surface feature set.

[0086] In the second step, based on the above working surface feature set, the working surface type is identified for the working surface corresponding to the above cropped video image sequence to obtain the above working surface type.

[0087] In practice, the execution entity may use the naive Bayes algorithm to identify the type of the working surface by using the working surface feature set to obtain the type of the working surface corresponding to the cropped video image sequence.

[0088] Step 1033, determining risk area description information according to the cropped video image sequence and the work surface type.

[0089] In some embodiments, the execution entity may determine the risk area description information according to the cropped video image sequence and the work surface type.

[0090] The risk area description information includes: risk area location information and risk area protection facility information, wherein the risk area protection facility information represents the state of the protection facility of the risk area corresponding to the risk area description information. As an example, the protection facility may be a protection fence. The risk area location information represents the coordinates of the risk area corresponding to the risk area description information in the image pixel coordinate system.

[0091] In practice, the above-mentioned execution entity can perform risk area detection on the cropped video image sequence according to the type of the working surface to obtain risk area description information.

[0092] Optionally, the above-mentioned work surface type includes: a roof work surface type and a facade work surface type. Among them, the above-mentioned roof work surface type represents the work surface corresponding to the roof. The above-mentioned facade work surface represents the work surface corresponding to the facade.

[0093] In some optional implementations of some embodiments, the execution subject determines the risk area description information according to the cropped video image sequence and the work surface type, which may include the following steps:

[0094] In the first step, in response to determining that the above-mentioned working surface type is a roof working surface type, the following first processing step is performed:

[0095] The first sub-step is to perform edge detection on the cropped video image sequence to obtain edge risk area position information.

[0096] In practice, the execution entity may perform edge detection on the cropped video image sequence using a Sobel operator to obtain edge risk area position information.

[0097] In the second sub-step, hole detection is performed on the cropped video image sequence to obtain the location information of the hole risk area.

[0098] In practice, the execution subject may perform hole detection on the cropped video image sequence through a target detection algorithm to obtain the location information of the hole risk area. As an example, the target detection algorithm may be a Faster R-CNN (Faster Region-based Convolutional Neural Network) algorithm.

[0099] The third sub-step is to determine the risk area location information included in the risk area description information based on the edge risk area location information and the hole risk area location information.

[0100] In practice, the above-mentioned execution entity may combine the above-mentioned edge risk area location information and the above-mentioned hole risk area location information into the risk area location information included in the above-mentioned risk area description information.

[0101] In the second step, in response to determining that the above-mentioned working surface type is a facade working surface type, hole detection is performed on the above-mentioned cropped video image sequence to obtain the risk area position information included in the above-mentioned risk area description information.

[0102] In practice, the execution subject may perform hole detection on the cropped video image sequence through a target detection algorithm to obtain the risk area location information included in the risk area description information. As an example, the target detection algorithm may be a Faster R-CNN (Faster Region-based Convolutional Neural Network) algorithm.

[0103] The third step is to extract features of the risk area corresponding to the above risk area location information through the pre-trained risk area feature extraction model to obtain the feature information of the risk area protection facilities.

[0104] The risk area feature extraction model is a model that takes the risk area location information as input and takes the risk area protection facility feature information as output. In practice, the risk area feature extraction model can be a convolutional neural network model.

[0105] In practice, the above-mentioned execution entity may input the above-mentioned risk area location information into the above-mentioned pre-trained risk area feature extraction model to extract features of the risk area corresponding to the above-mentioned risk area location information, and obtain feature information of risk area protection facilities.

[0106] The fourth step is to generate the risk area protection facility information included in the risk area description information based on the risk area protection facility feature information and the pre-trained protection facility identification model.

[0107] The protective facility identification model is a model that takes the risk area protective facility feature information as input and the risk area protective facility information included in the risk area description information as output. In practice, the protective facility identification model can be a target detection model based on deep learning. As an example, the protective facility identification model can be an SSD (Single Shot MultiBox Detector) model.

[0108] In practice, the above-mentioned execution entity can input the above-mentioned risk area protection facility characteristic information into the above-mentioned pre-trained protection facility identification model to generate the risk area protection facility information included in the above-mentioned risk area description information.

[0109] Step 1034, determining the edge distance of the construction personnel corresponding to the construction personnel description information according to the risk area location information.

[0110] In some embodiments, the execution entity may determine the edge distance of the construction personnel corresponding to the construction personnel description information based on the risk area location information.

[0111] Among them, the above-mentioned edge distance is the shortest distance from the construction personnel description information to the above-mentioned risk area location information.

[0112] In practice, the execution entity uses the shortest distance from the construction worker location information included in the construction worker description information to the risk area location information as the edge distance of the construction worker corresponding to the construction worker description information.

[0113] In some optional implementations of some embodiments, the execution entity determines the edge distance of the construction personnel corresponding to the construction personnel description information according to the risk area location information, which may include the following steps:

[0114] The first step is to map the boundary coordinate points of the risk area corresponding to the above risk area location information to obtain a set of edge coordinates of the risk area.

[0115] The risk area edge coordinate set represents coordinate points on the corresponding edge of the risk area in the geodetic coordinate system.

[0116] In practice, the execution entity converts the risk area corresponding to the risk area position information from an image pixel coordinate system to a coordinate point mapping in a geodetic coordinate system to obtain a set of edge coordinates of the risk area.

[0117] In the second step, based on the risk area edge coordinate set, a set of area boundary information of the risk area corresponding to the risk area position information is generated.

[0118] Each area boundary information in the area boundary information set includes: the coordinates of the left endpoint of the boundary and the coordinates of the right endpoint of the boundary.

[0119] In practice, the execution entity may fit the risk area edge coordinate set to obtain a set of area boundary information of the risk area corresponding to the risk area position information.

[0120] Step 3: For each area boundary information in the above area boundary information set, perform the following sub-edge distance determination steps:

[0121] The first sub-step is to determine the vector product of the left endpoint coordinates and the right endpoint coordinates included in the construction personnel position information and the risk area edge line segment information.

[0122] In a second sub-step, in response to determining that the vector product is less than or equal to zero, a vertical distance from the construction worker position information to the risk area edge line segment information is determined as a sub-edge distance of the construction worker corresponding to the construction worker description information.

[0123] The third sub-step, in response to determining that the above-mentioned vector product is greater than zero, determines the minimum distance from the above-mentioned construction personnel position information to the left endpoint coordinates and the right endpoint coordinates included in the above-mentioned risk area edge segment information as the sub-edge distance of the construction personnel corresponding to the above-mentioned construction personnel description information.

[0124] In the fourth step, the sub-adjacent edge distances satisfying the screening conditions in the obtained sub-adjacent edge distance set are determined as the adjacent edge distances of the construction personnel corresponding to the construction personnel description information.

[0125] Among them, the above screening condition is that the distance between the sub-adjacent edges is the smallest.

[0126] The above-mentioned content of "in some optional implementation methods of some embodiments" as an inventive point of the present disclosure solves the following technical problem, namely, "when the construction workers are closer to the edge of the risk area, the probability of an accident increases, but the edge distance cannot be accurately determined, and thus the construction workers at the edge cannot be effectively warned, making it impossible to effectively ensure the life safety of the construction workers". Based on this, firstly, the present disclosure maps the boundary coordinate points of the risk area corresponding to the above-mentioned risk area location information to obtain a risk area edge coordinate set, wherein the above-mentioned risk area edge coordinate set represents the coordinate points on the corresponding edge of the risk area. The image coordinate system is usually a two-dimensional pixel coordinate. When determining the edge distance, the curvature of the earth and the actual geographical location need to be considered. Therefore, it is converted into a geodetic coordinate system to ensure that the measurement results are consistent with the geographical location, reflect the actual spatial relationship, and thus provide more reliable data. Then, according to the above-mentioned risk area edge coordinate set, a regional boundary information set for the risk area corresponding to the above-mentioned risk area location information is generated, wherein each regional boundary information in the regional boundary information set includes: the coordinates of the left endpoint of the boundary and the coordinates of the right endpoint of the boundary. Then, for each area boundary information in the above-mentioned area boundary information set, the following sub-adjacent edge distance determination steps are performed: the first step is to determine the vector product of the left endpoint coordinates and the right endpoint coordinates included in the above-mentioned construction personnel position information and the above-mentioned risk area edge segment information. The second step is, in response to determining that the above-mentioned vector product is less than or equal to zero, the vertical distance from the above-mentioned construction personnel position information to the above-mentioned risk area edge segment information is determined as the sub-adjacent edge distance of the construction personnel corresponding to the above-mentioned construction personnel description information. The third step is, in response to determining that the above-mentioned vector product is greater than zero, the minimum distance from the above-mentioned construction personnel position information to the left endpoint coordinates and the right endpoint coordinates included in the above-mentioned risk area edge segment information is determined as the sub-adjacent edge distance of the construction personnel corresponding to the above-mentioned construction personnel description information. Selecting different distance determination methods according to the position of the construction personnel relative to the edge of the risk area can more accurately evaluate the adjacent edge distance of the construction personnel. Finally, the sub-adjacent edge distance that meets the screening conditions in the obtained sub-adjacent edge distance set is determined as the adjacent edge distance of the construction personnel corresponding to the above-mentioned construction personnel description information. In this way, the distance of construction workers from the edge can be determined more accurately, thereby achieving effective early warning for construction workers near the edge and avoiding the situation where the life safety of construction workers cannot be effectively guaranteed.

[0127] Step 1035, based on the risk area protection facility information, the edge distance and the construction worker clothing information included in the construction worker description information, an early warning operation is performed on the construction worker corresponding to the construction worker description information.

[0128] In some embodiments, the execution entity may perform early warning operations on the construction personnel corresponding to the construction personnel description information according to the risk area protection facility information, the edge distance, and the construction personnel clothing information included in the construction personnel description information.

[0129] In practice, the above-mentioned executing entities can perform sound warning operations on the construction workers corresponding to the construction worker description information based on the risk area protection facility information, edge distance and construction worker description information including the construction worker clothing information.

[0130] Optionally, the above-mentioned construction personnel clothing information includes: protective clothing wearing status, protective clothing inflation device status and five-point protection belt wearing status, and the above-mentioned five-point protection belt wearing status includes: protection rope retraction device status, safety buckle device status and inertial measurement device status, wherein the above-mentioned protective clothing wearing status indicates whether the construction personnel wear inflatable protective clothing, the above-mentioned protective clothing inflation device status indicates whether the protective clothing inflation device included in the above-mentioned inflatable protective clothing inflates the above-mentioned inflatable protective clothing, the above-mentioned five-point protection belt wearing status indicates whether the construction personnel wear the five-point protection belt, and the above-mentioned protection rope retraction device status indicates whether the protection rope retraction device included in the above-mentioned five-point protection belt retracts the above-mentioned five-point protection belt. The above-mentioned safety buckle device status includes: locked state and unlocked state. The above-mentioned inertial measurement device status includes: stationary state and non-stationary state.

[0131] In some optional implementations of some embodiments, the execution entity performs an early warning operation on the construction personnel corresponding to the construction personnel description information according to the risk area protection facility information, the edge distance, and the construction personnel clothing information included in the construction personnel description information, which may include the following steps:

[0132] In the first step, in response to determining that the above-mentioned working surface type is a facade working surface type, the following first warning operation is performed:

[0133] In the first sub-step, in response to determining that the protective clothing wearing state indicates that the construction personnel are wearing inflatable protective clothing, the following protective clothing warning operations are performed:

[0134] In response to determining that the state of the safety buckle device is a locked state and the state of the inertial measurement device is a non-stationary state, the protective clothing inflation device is controlled to inflate the inflatable protective clothing.

[0135] In practice, in response to determining that the safety buckle device is in a locked state and the inertial measurement device is in a non-stationary state, the execution entity may control the protective suit inflation device to inflate the inflatable protective suit via a wireless connection.

[0136] In the second sub-step, in response to determining that the wearing state of the inflatable protective clothing indicates that the construction personnel are not wearing the inflatable protective clothing, the following protective belt warning operation is performed:

[0137] In response to determining that the state of the safety buckle device is a locked state and the state of the inertial measurement device is a non-stationary state, the protective rope retraction device is controlled to adjust the five-point protective belt.

[0138] In practice, in response to determining that the safety buckle device is in a locked state and the inertial measurement device is in a non-stationary state, the execution entity may control the protective rope retraction device to tighten the five-point protective belt through a wireless connection.

[0139] In the second step, in response to determining that the above-mentioned working surface type is a roof working surface type, the following second warning operation is performed:

[0140] In response to determining that the state of the safety buckle device is locked and the risk area protection facility information indicates that the protection facility does not exist, the following third warning operation is performed on the construction personnel corresponding to the construction personnel description information according to the edge distance:

[0141] In response to determining that the edge distance satisfies a first screening condition, the protective rope retraction device is controlled to adjust the five-point protective belt, wherein the first screening condition is that the edge distance is less than or equal to a target value.

[0142] As an example, the above target value may be 1 meter.

[0143] In practice, in response to determining that the edge distance satisfies the first screening condition, the execution entity may control the protective rope retracting device to tighten the five-point protective belt through a wireless connection.

[0144] The above-mentioned content of "in some optional implementations of some embodiments" as an inventive point of the present disclosure solves the following technical problem, namely, "different working surfaces may have unique dangers and environmental conditions. If early warning is not combined with the characteristics of the working surface, all specific risks may not be effectively identified, resulting in certain situations being ignored or false alarms, which may pose a direct threat to the life safety of construction personnel." Based on this, the present disclosure first responds to determining that the above-mentioned working surface type is a vertical working surface type, and performs the following first early warning operation: the first step, in response to determining that the wearing state of the above-mentioned protective clothing represents that the construction personnel wear inflatable protective clothing, the following protective clothing early warning operation is performed: in response to determining that the state of the above-mentioned safety buckle device is a locked state and the state of the above-mentioned inertial measurement device is a non-stationary state, the above-mentioned protective clothing inflation device is controlled to inflate the above-mentioned inflatable protective clothing. The second step, in response to determining that the wearing state of the above-mentioned inflatable protective clothing represents that the construction personnel do not wear inflatable protective clothing, the following protective belt early warning operation is performed: in response to determining that the state of the above-mentioned safety buckle device is a locked state and the state of the above-mentioned inertial measurement device is a non-stationary state, the above-mentioned protective rope retraction device is controlled to adjust the above-mentioned five-point protective belt. Two types of warning operations are taken according to whether or not inflatable protective clothing is worn. If inflatable protective clothing is worn, a protective clothing warning operation is taken, and if inflatable protective clothing is not worn, a protective belt warning operation is taken, thereby enhancing the safety of construction personnel. Finally, in response to determining that the above-mentioned working surface type is a rooftop working surface type, the following second warning operation is performed: In response to determining that the state of the above-mentioned safety buckle device is a locked state and the above-mentioned risk area protection facility information indicates that the protective facility does not exist, according to the above-mentioned edge distance, the following third warning operation is performed on the construction personnel corresponding to the above-mentioned construction personnel description information: In response to determining that the above-mentioned edge distance meets the first screening condition, the above-mentioned protective rope retraction device is controlled to adjust the above-mentioned five-point protective belt, wherein the above-mentioned first screening condition is that the edge distance is less than or equal to the target value. According to the characteristics of the working surface, that is, the type of working surface, different warning methods are taken for different working surfaces, thereby improving the safety of construction personnel.

[0145] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the high-altitude operation protection and early warning method of some embodiments of the present disclosure, the safety of high-altitude operation is monitored in real time, and the automatic early warning for the safety monitoring of high-altitude operation is realized, avoiding the occurrence of direct threats to the life safety of construction personnel caused by the low detection efficiency of the safety monitoring of high-altitude operation and the low early warning efficiency of high-altitude operation. Specifically, the reasons for the low detection efficiency of the safety monitoring of high-altitude operation and the low early warning efficiency of high-altitude operation are: the high-altitude environment is complex, the manual monitoring method has great subjectivity and blind spots in vision, and it is difficult to track all potential dangers in real time. When problems occur, the response of manual intervention may not be fast enough, and the detection efficiency of the safety monitoring of high-altitude operation and the early warning efficiency of high-altitude operation cannot be guaranteed, which may cause a direct threat to the life safety of construction personnel. Based on this, the high-altitude operation protection and early warning method of some embodiments of the present disclosure, first, collects real-time monitoring video streams as video streams to be detected, wherein the above-mentioned video streams to be detected are video streams for the working surface shot in real time by monitoring equipment in the high-altitude operation area. Real-time monitoring of the safety of high-altitude operation can be achieved by collecting real-time monitoring video streams. Next, target detection is performed on the above-mentioned video stream to be detected to generate construction personnel description information and obtain a set of construction personnel description information, wherein the construction personnel description information in the above-mentioned construction personnel description information set represents the construction personnel working in the above-mentioned high-altitude working area, and the construction personnel description information in the above-mentioned construction personnel description information set includes: construction personnel position information and construction personnel clothing information, wherein the construction personnel clothing information represents the wearing status of the safety equipment of the construction personnel corresponding to the construction personnel description information. By identifying the position of the construction personnel and the wearing status of the safety equipment, the safety of the construction personnel is preliminarily judged. Finally, for each construction personnel description information in the above-mentioned construction personnel description information set, the following processing steps are performed: the first step is to perform image segmentation on the video image contained in the above-mentioned video stream to be detected according to the construction personnel position information included in the above-mentioned construction personnel description information to generate a cropped video image and obtain a cropped video image sequence. Cropping the building part in the video image for subsequent risk area identification can reduce the interference of other parts of the image except the building during the identification process, can more effectively identify the risk area, and can also reduce the computational complexity and improve the processing speed. The second step is to determine the type of the working surface according to the above-mentioned cropped video image sequence. Different working face risk areas have different detection targets, so it is necessary to determine the working face type. The third step is to determine the risk area description information based on the above-mentioned cropped video image sequence and the above-mentioned working face type, wherein the above-mentioned risk area description information includes: risk area location information and risk area protection facility information, wherein the above-mentioned risk area protection facility information represents the protection facility status of the risk area corresponding to the risk area description information. The safety of the construction personnel is further judged through the risk area location information and risk area protection facility information.The fourth step is to determine the edge distance of the construction personnel corresponding to the construction personnel description information according to the above risk area location information. The fifth step is to perform early warning operations on the construction personnel corresponding to the above construction personnel description information according to the above risk area protection facility information, the above edge distance and the construction personnel clothing information included in the above construction personnel description information. The safety of construction personnel is judged from three aspects: protective facilities, edge distance and construction personnel clothing information, and early warning is issued for situations where safety hazards exist. In this way, all potential dangers can be tracked in real time, and early warnings can be issued immediately when problems occur, thereby improving the detection efficiency of high-altitude work safety monitoring and the early warning efficiency of high-altitude work, and avoiding the occurrence of direct threats to the life safety of construction personnel due to low detection efficiency of high-altitude work safety monitoring and low early warning efficiency of high-altitude work.

[0146] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a high-altitude operation protection warning device. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the high-altitude operation protection and warning device can be specifically applied to various electronic devices.

[0147] like Figure 2As shown, the high-altitude work protection warning device 200 of some embodiments includes: a collection unit 201, a target detection unit 202 and a processing unit 203. The collection unit 201 is configured to collect real-time monitoring video streams as video streams to be detected, wherein the video streams to be detected are video streams for the working surface shot in real time by monitoring equipment in the high-altitude work area. The target detection unit 202 is configured to perform target detection on the video streams to be detected to generate construction personnel description information and obtain a construction personnel description information set, wherein the construction personnel description information in the construction personnel description information set represents the construction personnel working in the high-altitude work area, and the construction personnel description information in the construction personnel description information set includes: construction personnel location information and construction personnel clothing information, wherein the construction personnel clothing information represents the wearing status of the safety equipment of the construction personnel corresponding to the construction personnel description information. The processing unit 203 is configured to perform the following processing steps for each construction worker description information in the above-mentioned construction worker description information set: according to the construction worker position information included in the above-mentioned construction worker description information, the video image contained in the above-mentioned video stream to be detected is segmented to generate a cropped video image and obtain a cropped video image sequence. According to the above-mentioned cropped video image sequence, the type of the working surface is determined. According to the above-mentioned cropped video image sequence and the above-mentioned working surface type, the risk area description information is determined, wherein the above-mentioned risk area description information includes: risk area location information and risk area protection facility information, wherein the above-mentioned risk area protection facility information characterizes the protection facility status of the risk area corresponding to the risk area description information. According to the above-mentioned risk area location information, the edge distance of the construction worker corresponding to the above-mentioned construction worker description information is determined. According to the above-mentioned risk area protection facility information, the above-mentioned edge distance and the above-mentioned construction worker clothing information, an early warning operation is performed on the construction worker corresponding to the above-mentioned construction worker description information.

[0148] It is understandable that the units recorded in the high-altitude work protection warning device 200 are similar to those in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the high-altitude operation protection warning device 200 and the units contained therein, and will not be repeated here.

[0149] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, a computing device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0150] like Figure 3As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory 302 or a program loaded from a storage device 308 into a random access memory 303. Various programs and data required for the operation of the electronic device 300 are also stored in the random access memory 303. The processing device 301, the read-only memory 302, and the random access memory 303 are connected to each other via a bus 304. An input / output interface 305 is also connected to the bus 304.

[0151] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0152] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the read-only memory 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0153] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0154] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0155] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: collects a real-time monitoring video stream as a video stream to be detected, wherein the video stream to be detected is a video stream for the working surface shot in real time by the monitoring equipment in the high-altitude working area. Perform target detection on the video stream to be detected to generate construction personnel description information, and obtain a construction personnel description information set, wherein the construction personnel description information in the construction personnel description information set represents the construction personnel working in the high-altitude working area, and the construction personnel description information in the construction personnel description information set includes: construction personnel position information and construction personnel clothing information, wherein the construction personnel clothing information represents the wearing status of the safety equipment of the construction personnel corresponding to the construction personnel description information. For each construction personnel description information in the construction personnel description information set, perform the following processing steps: according to the construction personnel position information included in the construction personnel description information, perform image segmentation on the video image contained in the video stream to be detected to generate a cropped video image, and obtain a cropped video image sequence. Determine the type of working surface based on the above-mentioned cropped video image sequence. Determine the risk area description information based on the above-mentioned cropped video image sequence and the above-mentioned working surface type, wherein the above-mentioned risk area description information includes: risk area location information and risk area protection facility information, wherein the above-mentioned risk area protection facility information represents the status of the protection facilities in the risk area corresponding to the risk area description information. Determine the edge distance of the construction personnel corresponding to the above-mentioned construction personnel description information based on the above-mentioned risk area location information. Perform early warning operations on the construction personnel corresponding to the above-mentioned construction personnel description information based on the above-mentioned risk area protection facility information, the above-mentioned edge distance and the construction personnel clothing information included in the above-mentioned construction personnel description information.

[0156] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0157] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0158] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be provided in a processor, for example, may be described as: a processor including an acquisition unit, a target detection unit, and a processing unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the acquisition unit may also be described as "a unit for acquiring a real-time monitoring video stream as a video stream to be detected".

[0159] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0160] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A high-altitude operation protection early warning method, comprising: Collecting a real-time monitoring video stream as a video stream to be detected, wherein the video stream to be detected is a video stream for the working surface shot in real time by a monitoring device in the high-altitude working area; Performing target detection on the video stream to be detected to generate construction personnel description information, and obtaining a construction personnel description information set, wherein the construction personnel description information in the construction personnel description information set represents construction personnel working in the high-altitude working area, and the construction personnel description information in the construction personnel description information set includes: construction personnel location information and construction personnel clothing information, wherein the construction personnel clothing information represents the wearing status of safety equipment of the construction personnel corresponding to the construction personnel description information; For each construction worker description information in the construction worker description information set, the following processing steps are performed: According to the construction worker position information included in the construction worker description information, the video image included in the video stream to be detected is segmented to generate a cropped video image, thereby obtaining a cropped video image sequence; Determining the type of the work surface according to the cropped video image sequence; Determine risk area description information according to the cropped video image sequence and the type of the working surface, wherein the risk area description information includes: risk area location information and risk area protection facility information, wherein the risk area protection facility information represents the state of the protection facilities of the risk area corresponding to the risk area description information; Determine the distance from the edge of the construction personnel corresponding to the construction personnel description information according to the risk area location information; According to the risk area protection facility information, the edge distance and the construction worker clothing information included in the construction worker description information, an early warning operation is performed on the construction workers corresponding to the construction worker description information.

2. The method according to claim 1, wherein: The target detection is performed on the video stream to be detected to generate construction personnel description information, and a construction personnel description information set is obtained, including: Performing denoising on the video stream to be detected to obtain a denoised video stream; Performing normalization processing on the denoised video stream to obtain a preprocessed video stream; The construction worker description information set is generated by using a pre-trained construction worker detection model and the pre-processed video stream.

3. The method according to claim 2, wherein: The step of performing image segmentation on the video image included in the video stream to be detected according to the construction worker position information included in the construction worker description information to generate a cropped video image and obtain a cropped video image sequence includes: For each video image of a construction worker in the video stream to be detected and containing the description information of the construction worker, the following segmentation steps are performed: Convert the video image into a grayscale image to obtain a video grayscale image; According to the construction worker position information included in the construction worker description information, edge detection is performed on the video grayscale image to obtain an image after edge detection; Performing contour extraction on the edge-detected image to generate a building contour point set, wherein the building contour points in the building contour point set include: contour point abscissas and contour point ordinates; Classifying the building contour point set by the horizontal and vertical coordinates to obtain a building contour point horizontal coordinate set and a building contour point vertical coordinate set; sorting the set of building outline points' horizontal coordinates and the set of building outline points' vertical coordinates to obtain a sequence of building outline points' horizontal coordinates and a sequence of building outline points' vertical coordinates; Generate a set of building contour points after elimination according to the building contour point abscissa sequence and the building contour point ordinate sequence; Performing piecewise fitting according to the eliminated building contour point set to obtain a building contour line segment set; Smoothing the building contour lines in the building contour line segment set to obtain building contour information; The video image is cropped according to the building outline information to obtain a cropped video image in the cropped video image sequence.

4. The method according to claim 3, wherein: The step of determining the type of the work surface according to the cropped video image sequence includes: Using a pre-trained working surface feature extraction model, extracting working surface features from the cropped video image sequence to obtain a working surface feature set; According to the work surface feature set, the work surface type is identified for the work surface corresponding to the cropped video image sequence to obtain the work surface type.

5. The method according to claim 4, wherein: The working surface types include: roof working surface type and facade working surface type; and The step of determining risk area description information according to the cropped video image sequence and the work surface type includes: In response to determining that the work surface type is a roof work surface type, the following first processing step is performed: Performing edge detection on the cropped video image sequence to obtain edge risk area position information; Performing hole detection on the cropped video image sequence to obtain location information of the hole risk area; Determine the risk area location information included in the risk area description information according to the edge risk area location information and the hole risk area location information; In response to determining that the work surface type is a facade work surface type, performing hole detection on the cropped video image sequence to obtain risk area position information included in the risk area description information; Through a pre-trained risk area feature extraction model, feature extraction is performed on the risk area corresponding to the risk area location information to obtain feature information of risk area protection facilities; based on the risk area protection facility feature information and a pre-trained protection facility identification model, the risk area protection facility information included in the risk area description information is generated.

6. A high-altitude operation protection warning device, comprising: The acquisition unit is configured to acquire a real-time monitoring video stream as a video stream to be detected, wherein the video stream to be detected is a video stream for a working surface shot in real time by a monitoring device in a high-altitude working area; The target detection unit is configured to perform target detection on the video stream to be detected to generate construction personnel description information and obtain a construction personnel description information set, wherein the construction personnel description information in the construction personnel description information set represents the construction personnel working in the high-altitude working area, and the construction personnel description information in the construction personnel description information set includes: construction personnel location information and construction personnel clothing information, wherein the construction personnel clothing information represents the wearing status of the safety equipment of the construction personnel corresponding to the construction personnel description information; The processing unit is configured to execute the following processing steps for each construction worker description information in the construction worker description information set: according to the construction worker position information included in the construction worker description information, perform image segmentation on the video image contained in the video stream to be detected to generate a cropped video image and obtain a cropped video image sequence; according to the cropped video image sequence, determine the type of working surface; according to the cropped video image sequence and the working surface type, determine the risk area description information, wherein the risk area description information includes: risk area position information and risk area protection facility information, wherein the risk area protection facility information represents the protection facility status of the risk area corresponding to the risk area description information; according to the risk area position information, determine the edge distance of the construction worker corresponding to the construction worker description information; according to the risk area protection facility information, the edge distance and the construction worker's clothing information, perform early warning operations on the construction worker corresponding to the construction worker description information.

7. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images