A method and apparatus for safety monitoring of an engineering site

Abstract

The application is suitable for the technical field of computers, and provides a safety monitoring method and device for an engineering site, the method comprising: identifying a target overhead image to obtain a region type to which each pixel in the target overhead image belongs; identifying a target ground image to obtain a target object and position information of the target object in the target ground image; calculating a motion risk index between the target objects; determining a region risk index; calculating a risk coefficient of the target object according to the motion risk index and the region risk index; and performing safety monitoring on the engineering site according to the risk coefficient. The above scheme obtains multi-dimensional information of the engineering site from the perspective of dynamic operation of the entire engineering site, calculates the risk coefficient of the target object according to the motion risk index and the region risk index, and can perform safety early warning more comprehensively and monitor the safety of the engineering site more comprehensively.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computers, and particularly relates to a safety monitoring method and device for an engineering site. BACKGROUND

[0002] An engineering site is a place where safety accidents occur frequently, so the safety of the engineering site needs to be monitored. At present, when the safety of an engineering site is monitored, a computer vision or deep neural network method is used to detect whether workers wear safety helmets and to issue a warning to those who do not wear safety helmets. However, the detection rate of whether a safety helmet is worn or not is not high, and sufficient safety cannot be ensured, and the method can only play a preliminary reminding role, and the safety warning role that can be provided is very limited, and the safety of the engineering site cannot be comprehensively monitored. SUMMARY

[0003] The application provides a safety monitoring method and device for an engineering site, which can solve the above problems.

[0004] In a first aspect, the application provides a safety monitoring method for an engineering site, comprising:

[0005] obtaining a target overhead image and a target ground image of an engineering site;

[0006] identifying the target overhead image to obtain a region type to which each pixel in the target overhead image belongs;

[0007] identifying the target ground image to obtain a target object and position information thereof in the target ground image; the number of the target objects is at least two;

[0008] calculating a motion risk index between the target objects according to the target objects and the position information thereof;

[0009] determining a target region to which the target objects belong according to the region type to which each pixel belongs, and determining a region risk index according to the target region;

[0010] calculating a risk coefficient of the target objects according to the motion risk index and the region risk index;

[0011] monitoring the safety of the engineering site according to the risk coefficient.

[0012] Further, the target objects include target workers and target vehicles; the position information of the target workers includes a first center coordinate and a first size of a first bounding box; and the position information of the target vehicles includes a second center coordinate and a second size of a second bounding box.

[0013] The motion risk index between the target objects is calculated according to the target objects and the position information of the target objects, and the motion risk index between the target worker and the target vehicle is calculated according to the first center coordinate, the first size, the second center coordinate and the second size.

[0014] The predicted collision time of the target collision event between the target worker and the target vehicle is calculated according to the first center coordinate, the first size, the second center coordinate and the second size.

[0015] If the target collision event is a real collision event, the motion risk index between the target worker and the target vehicle is calculated according to a preset motion risk index calculation rule.

[0016] Further, the target ground image includes a plurality of image frame groups collected by the same image collection device.

[0017] The predicted collision time of the target collision event between the target worker and the target vehicle is calculated according to the first center coordinate, the first size, the second center coordinate and the second size.

[0018] The initial collision time of the target collision event between the target worker and the target vehicle is calculated according to the first center coordinate, the first size, the second center coordinate and the second size of each of the image frame groups.

[0019] The average value of all the initial collision times is calculated to obtain the predicted collision time of the target collision event between the target worker and the target vehicle.

[0020] Further, each of the image frame groups includes at least three consecutive images.

[0021] The initial collision time of the target collision event between the target worker and the target vehicle is calculated according to the first center coordinate, the first size, the second center coordinate and the second size of each of the image frame groups.

[0022] The first acceleration and the first speed of the target worker are calculated according to the first center coordinate and a first preset calculation rule of the three consecutive images, and the second speed is calculated according to the first acceleration and the first speed; wherein the first speed is the speed of the target worker in the first image of the three consecutive images; the second speed is the speed of the target worker in the third image of the three consecutive images.

[0023] calculate a second acceleration and a third speed of the target vehicle according to the second center coordinates of the continuous three frames of images and a second preset calculation rule, and calculate a fourth speed according to the second acceleration and the third speed; wherein the third speed is a speed of the target vehicle in a first frame of image in the continuous three frames of images; and the fourth speed is a speed of the target vehicle in a third frame of image in the continuous three frames of images;

[0024] calculate a target distance between the target worker and the target vehicle in the third frame of image in the continuous three frames of images according to the first size, the second size, the second speed and the fourth speed;

[0025] if it is determined that there is a collision risk between the target worker and the target vehicle according to the target distance, calculate an initial collision time of a target collision event according to the first acceleration, the second acceleration, the first size, the second size, the second speed and the fourth speed.

[0026] Further, after the target distance between the target worker and the target vehicle in the third frame of image in the continuous three frames of images is calculated according to the first size, the second size, the second speed and the fourth speed, the method further comprises:

[0027] if it is determined that there is no collision risk between the target worker and the target vehicle according to the target distance, an expected collision time of the target collision event is infinite.

[0028] Further, if the target collision event is a real collision event, calculate a motion risk index between the target worker and the target vehicle according to a preset motion risk index calculation rule, comprising:

[0029] if the expected collision time is less than a first preset warning time threshold, obtain a backup collision time corresponding to the expected collision time;

[0030] if the backup collision time is greater than a second preset warning time threshold and less than the first preset warning time threshold, calculate a motion risk index between the target worker and the target vehicle according to a preset coefficient and the expected collision time.

[0031] Further, the identifying the target overhead image to obtain a region type to which each pixel in the target overhead image belongs comprises:

[0032] inputting the target overhead image into a trained region identification model for identification to obtain a region type to which each pixel in the target overhead image belongs.

[0033] Further, before the inputting the target overhead image into the trained region identification model to identify the region type to which each pixel in the target overhead image belongs, the method further comprises:

[0034] obtaining a sample training set; the sample training set comprises a sample overhead image and a sample region type to which each pixel in the sample overhead image belongs;

[0035] training an initial identification model using the sample training set to obtain the trained region identification model.

[0036] Further, the obtaining the sample training set comprises:

[0037] obtaining an initial overhead image, and processing the initial overhead image according to a preset image processing strategy to obtain the sample overhead image; the image processing strategy comprises one or more of a brightness adjustment strategy, a hue adjustment strategy, a saturation adjustment strategy, a contrast adjustment strategy, a noise adjustment strategy, an edge enhancement strategy, an image mirroring strategy, an image scaling strategy, an image removal strategy, and an image mixing strategy.

[0038] obtaining a sample region type to which each pixel in the sample overhead image belongs, and determining a sample training set according to the sample overhead image and the sample region type to which each pixel in the sample overhead image belongs.

[0039] In a second aspect, an embodiment of the present application provides a safety monitoring device for an engineering site, comprising:

[0040] a first obtaining unit configured to obtain a target overhead image and a target ground image of the engineering site;

[0041] a first identification unit configured to identify the target overhead image to obtain a region type to which each pixel in the target overhead image belongs;

[0042] a second identification unit configured to identify the target ground image to obtain a target object and position information of the target object in the target ground image; the number of the target objects is at least two;

[0043] a first calculation unit configured to calculate a motion risk index between the target objects according to the target objects and the position information of the target objects;

[0044] a first processing unit configured to determine a target region to which the target objects belong according to the region type to which each pixel belongs, and determine a region risk index according to the target region;

[0045] a second calculation unit configured to calculate a risk coefficient of the target objects according to the motion risk index and the region risk index.

[0046] a second processing unit, configured to perform safety monitoring on the construction site according to the risk coefficient.

[0047] Further, the target object includes a target worker and a target vehicle; the position information of the target worker includes a first center coordinate and a first size of a first bounding box; and the position information of the target vehicle includes a second center coordinate and a second size of a second bounding box.

[0048] The first calculation unit is specifically configured to:

[0049] calculate a predicted collision time of a target collision event between the target worker and the target vehicle according to the first center coordinate, the first size, the second center coordinate and the second size;

[0050] if the target collision event is a real collision event, calculate a motion risk index between the target worker and the target vehicle according to a preset motion risk index calculation rule.

[0051] Further, the target ground image includes a plurality of image frame groups collected by a same image collection device.

[0052] The first calculation unit is specifically configured to:

[0053] calculate an initial collision time of a target collision event between the target worker and the target vehicle according to the first center coordinate, the first size, the second center coordinate and the second size of each of the image frame groups;

[0054] calculate an average value of all the initial collision times to obtain a predicted collision time of a target collision event between the target worker and the target vehicle.

[0055] Further, each of the image frame groups includes at least three continuous images.

[0056] The first calculation unit is specifically configured to:

[0057] calculate a first acceleration and a first speed of the target worker according to the first center coordinate and a first preset calculation rule of the three continuous images, and calculate a second speed according to the first acceleration and the first speed; wherein the first speed is a speed of the target worker in a first image of the three continuous images; and the second speed is a speed of the target worker in a third image of the three continuous images.

[0058] calculate a second acceleration and a third speed of the target vehicle according to the second center coordinates of the continuous three frames of images and a second preset calculation rule, and calculate a fourth speed according to the second acceleration and the third speed; wherein the third speed is a speed of the target vehicle in a first frame of image in the continuous three frames of images; and the fourth speed is a speed of the target vehicle in a third frame of image in the continuous three frames of images;

[0059] calculate a target distance between the target worker and the target vehicle in the third frame of image in the continuous three frames of images according to the first size, the second size, the second speed and the fourth speed;

[0060] if it is determined that there is a collision risk between the target worker and the target vehicle according to the target distance, calculate an initial collision time of a target collision event according to the first acceleration, the second acceleration, the first size, the second size, the second speed and the fourth speed.

[0061] Further, the first calculation unit is specifically further configured to:

[0062] if it is determined that there is no collision risk between the target worker and the target vehicle according to the target distance, an expected collision time of the target collision event is infinite.

[0063] Further, the first calculation unit is specifically configured to:

[0064] if the expected collision time is less than a first preset warning time threshold, obtain a backup collision time corresponding to the expected collision time;

[0065] if the backup collision time is greater than a second preset warning time threshold and less than the first preset warning time threshold, calculate a motion risk index between the target worker and the target vehicle according to a preset coefficient and the expected collision time.

[0066] Further, the first identification unit is specifically configured to:

[0067] input the target overhead image into a trained region identification model for identification to obtain a region type to which each pixel in the target overhead image belongs.

[0068] Further, the first identification unit is specifically further configured to:

[0069] obtain a sample training set; the sample training set includes a sample overhead image and a sample region type to which each pixel in the sample overhead image belongs;

[0070] train an initial identification model using the sample training set to obtain the trained region identification model.

[0071] Further, the first identification unit is specifically further configured to:

[0072] An initial overhead image is acquired, and the initial overhead image is processed according to a preset image processing strategy to obtain a sample overhead image; the image processing strategy includes one or more of a brightness adjustment strategy, a hue adjustment strategy, a saturation adjustment strategy, a contrast adjustment strategy, a noise adjustment strategy, an edge enhancement strategy, an image mirroring strategy, an image scaling strategy, an image removal strategy, and an image mixing strategy.

[0073] A sample region type to which each pixel of the sample overhead image belongs is acquired, and a sample training set is determined according to the sample overhead image and the sample region type to which each pixel of the sample overhead image belongs.

[0074] In a third aspect, an embodiment of the present application provides a safety monitoring device for an engineering site, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the safety monitoring method for the engineering site according to the first aspect.

[0075] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the safety monitoring method for the engineering site according to the first aspect.

[0076] In the embodiment of the present application, a target overhead image and a target ground image of the engineering site are acquired, the target overhead image is identified to obtain a region type to which each pixel of the target overhead image belongs, the target ground image is identified to obtain a target object and position information of the target object in the target ground image, a motion risk index between the target objects is calculated according to the target object and the position information of the target object, a target region to which the target object belongs is determined according to the region type to which each pixel belongs, and a region risk index is determined according to the target region, a danger coefficient of the target object is calculated according to the motion risk index and the region risk index, and the engineering site is monitored according to the danger coefficient. The above scheme comprehensively acquires multi-dimensional information of the engineering site from the perspective of dynamic operation of the entire engineering site, calculates the danger coefficient of the target object according to the motion risk index and the region risk index, can comprehensively perform safety early warning through the danger coefficient, and comprehensively monitors the safety of the engineering site. BRIEF DESCRIPTION OF DRAWINGS

[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0078] Figure 1 is a schematic flow chart of a safety monitoring method of an engineering site provided by the first embodiment of the present application;

[0079] Figure 2 is a schematic diagram of a target overhead view in a safety monitoring method of an engineering site provided by the first embodiment of the present application;

[0080] Figure 3 is a schematic diagram of a region type to which each pixel in a target overhead view image belongs in a safety monitoring method of an engineering site provided by the first embodiment of the present application;

[0081] Figure 4 is an effect schematic diagram of a random scaling method in an image processing strategy in a safety monitoring method of an engineering site provided by the first embodiment of the present application;

[0082] Figure 5 is an effect schematic diagram of a plurality of image processing strategies in a safety monitoring method of an engineering site provided by the first embodiment of the present application;

[0083] Figure 6 is an effect schematic diagram of a plurality of image processing strategies in a safety monitoring method of an engineering site provided by the first embodiment of the present application;

[0084] Figure 7 is a schematic diagram of a target object and its position information in a target ground image in a safety monitoring method of an engineering site provided by the first embodiment of the present application;

[0085] Figure 8 is a schematic diagram of ground camera arrangement in a safety monitoring method of an engineering site provided by the first embodiment of the present application;

[0086] Figure 9 is a schematic diagram of a safety monitoring device of an engineering site provided by the second embodiment of the present application;

[0087] Figure 10 is a schematic diagram of a safety monitoring device of an engineering site provided by the third embodiment of the present application. DETAILED DESCRIPTION

[0088] In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular architectures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and

[0089] It will be understood that the terms "comprises" and / or "comprising," when used in this specification, include the presence of one or more features, integers, steps, operations, elements, and / or components but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0090] It will be understood that the term "and / or," when used in this specification, includes the possibility of both there being a combination of features, integers, steps, operations, elements, and / or components and that there being one or the other alone (i.e., with the exclusion of the other) without

[0091] As used in this specification and claims, the terms "if" and "when" can be interpreted to mean "upon determination" or "in response to a determination" or "upon detection" or "in response to a detection," depending on the context. Similarly, the phrase "if determined" or "if detected [a described condition or event]" can be interpreted to mean "upon determination" or "in response to a determination" or "upon detection" or "in response to a detection" of [a described condition or event], depending on the context.

[0092] In addition, the terms "first," "second," "third," etc. as used in the description of embodiments herein and in the claims, if any, are used for differentiation only and are not meant to or used to indicate or imply relative importance.

[0093] Reference throughout this specification to "one embodiment" or "an embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. Thus, the appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in additional embodiments," and so on, in various places throughout this specification are not necessarily all referring to the same embodiment, unless otherwise specified. The terms "comprise," "comprising," "has," "having," "include," "including," and "contains," "containing," or variants thereof, mean "including but not limited to," unless otherwise specified.

[0094] See Figure 1 , Figure 1is a schematic flowchart of a safety monitoring method of an engineering site provided by the first embodiment of the present application. The execution subject of the safety monitoring method of an engineering site in this embodiment is a device with the safety monitoring function of an engineering site, such as a server, a desktop computer, and the like. As shown in Figure 1 the safety monitoring method of an engineering site can include:

[0095] S101: Obtain a target overhead image and a target ground image of an engineering site.

[0096] In this embodiment, two image acquisition devices are arranged at the engineering site to acquire images, one is an image acquisition device with an overhead view, and the other is an image acquisition device placed on the ground. The target overhead image is an image of the engineering site acquired by the overhead view, which can be acquired by a drone or a tower camera, etc. As shown in Figure 2 Figure 2 is a schematic diagram of the target overhead view, which should include all areas in the entire engineering site.

[0097] The target ground image is acquired by the image acquisition device placed on the ground, specifically, the image acquisition device can be placed around the engineering site, and when placed, the image acquisition device can be a camera placed 2 meters high from the ground. Here, 2 meters high is only an example and is not limited.

[0098] In this embodiment, the number of target overhead images and target ground images acquired by the engineering site is not limited, which can be one or more. After the image acquisition device of the engineering site acquires the target overhead image and the target ground image, the target overhead image and the target ground image are sent to the local device. The local device obtains the target overhead image and the target ground image of the engineering site.

[0099] S102: Identify the target overhead image to obtain the area type to which each pixel in the target overhead image belongs.

[0100] After the device obtains the target overhead image, the target overhead image is identified by a preset image recognition method, and the area type to which each pixel in the target overhead image belongs is identified. In this embodiment, the preset image recognition method is not limited, as long as the area type to which each pixel in the target overhead image belongs can be identified.

[0101] As shown in Figure 3 Figure 3 is a schematic diagram of the area type to which each pixel in the target overhead image belongs. Generally, the area type of the engineering site can include but is not limited to building area, facility area, road area, water area, office area, construction area. ​​

[0102] In an implementation, to accurately identify the region type to which each pixel in the target overhead image belongs, the target overhead image can be identified by a neural network. The neural network mode can quickly and accurately process the target overhead image and output the region type to which each pixel belongs.

[0103] The device inputs the target overhead image into the trained region identification model for identification to obtain the region type to which each pixel in the target overhead image belongs. The trained region identification model can be pre-set in the device or called from another device. The trained region identification model can include an input layer, a hidden layer, and an output layer (loss function layer). The input layer includes an input layer node for receiving an input target overhead image from the outside. The hidden layer is used to process the target overhead image and extract the region type to which each pixel in the target overhead image belongs. The output layer is used to output the region type to which each pixel in the target overhead image belongs.

[0104] In a possible implementation, the region identification model is pre-trained by the local device. The training method of the region identification model can be as follows:

[0105] The device obtains a sample training set, where the sample training set includes sample overhead images and the sample region type to which each pixel in the sample overhead images belongs. The initial identification model is trained using the sample training set to obtain the trained region identification model. In the training process, the sample overhead images and the sample region type to which each pixel in the sample overhead images belongs are used as training data and input into the initial identification model. The loss function of the initial identification model is adjusted to constantly improve the model, thereby obtaining the final region identification model.

[0106] During the model training process, a large amount of sample data is needed for training to obtain a more accurate model. If the number of sample data sets is insufficient, the model accuracy may not be high enough. Obtaining a large amount of rich sample data also consumes a large amount of resources. In the present embodiment, to avoid this problem, the limited samples can be expanded to improve the recognition ability of the model to objects such as construction machinery, workers, and materials under complex and variable construction site environmental conditions, and the anti-interference ability and robustness. The basic idea of the filling is to fully consider the possible situations on the construction site, so that the training set contains the possible situations on the construction site, thereby improving the accuracy of identification.

[0107] The device obtains an initial overhead image and processes the initial overhead image according to a preset image processing strategy to obtain a sample overhead image. The initial overhead image is a limited sample, and the device performs data enhancement on the limited sample through the preset image processing strategy. Then, the device obtains a sample region type to which each pixel of the sample overhead image belongs, and determines a sample training set according to the sample overhead image and the sample region type to which each pixel of the sample overhead image belongs.

[0108] The image processing strategy includes one or more of a brightness adjustment strategy, a tone adjustment strategy, a saturation adjustment strategy, a contrast adjustment strategy, a noise adjustment strategy, an edge enhancement strategy, an image mirroring strategy, an image scaling strategy, an image removal strategy, and an image mixing strategy. The image processing strategy will be described in detail below.

[0109] From the perspective of enhancing the anti-interference ability of the model, under different time and weather conditions at the construction site, the images obtained by the camera will have significant differences in brightness, saturation, and contrast; images obtained by different types of cameras will also differ in tone; and some noise points may be generated in the process of image generation and transmission. In order to avoid the interference of these factors on the identification ability of the model, the following methods can be used to expand the limited number of samples in this embodiment:

[0110] 1. Random brightness method: in the HSV color space, add a random value within a certain threshold range to the brightness component of all pixels in the image, thereby randomly adjusting the brightness of the image to simulate the different illumination differences at the construction site.

[0111]

[0112] v i ′=v i +δ, δ [-Δ, Δ], Δ ∈ [0, 0.5]

[0113] where P is a set of all pixels in the initial overhead image, v i is the brightness component of a certain pixel, v i ′ is the processed pixel brightness component value, δ is the random increase value of brightness, and Δ is the threshold value of the brightness increase.

[0114] 2. Random saturation method: in the HSV color space, add a random value within a certain threshold range to the saturation component of all pixels in the image, thereby randomly adjusting the saturation of the image to simulate different light environments at the construction site

[0115]

[0116] s i ′=s i+ γ, γ ∈ [-Γ, Γ], Γ ∈ [0, 0.5]

[0117] where P is the set of all pixels in the initial top-view image, s i is the saturation component of a certain pixel, s i ' is the processed pixel saturation component value, γ is the random increase value of saturation, and Γ is the threshold value of the saturation increase amount.

[0118] 3. Random hue method: In the HSV color space, the hue component of all pixels in the image is added with a random value within a certain threshold range, thereby randomly adjusting the hue of the image to simulate the hue difference on the image under different camera and light conditions.

[0119]

[0120] h i ' = h i + η, η ∈ [-H, H], H ∈ [0°, 180°]

[0121] where P is the set of all pixels in the image, h i is the hue component of a certain pixel, h i ' is the processed pixel hue component value, η is the random increase value of hue, and H is the threshold value of the hue increase amount.

[0122] 4. Random contrast method: In the RGB color space, the red, green, and blue components are multiplied by a factor within a certain threshold range, thereby randomly enhancing or weakening the contrast of the image, which can also simulate the contrast difference on the image under different camera and light conditions.

[0123]

[0124] r i ' = r i × α, g i ' = g i × α, b i ' = b i × α, α > 0

[0125] where r, g, and b are the red, green, and blue color components of a certain pixel, and α is the contrast factor.

[0126] 5. Gaussian noise method: Adding noise values of a two-dimensional Gaussian distribution with mean 0 as shown in the following formula to the original image can effectively enhance the anti-interference ability of the model. In application, by selecting a suitable variance, a corresponding Gaussian distribution noise matrix can be obtained, which is used as the operator of convolution operation to perform convolution operation on the original image, and the image with added Gaussian noise can be obtained.

[0127]

[0128] where, σ W and σ H are the variance of the horizontal and vertical axis, and p is the correlation coefficient between W and H.

[0129] 6. Salt and pepper noise method: also known as impulse noise, it randomly sets some pixels in the image to pure black or pure white. This noise can be used to simulate the errors generated by the camera and transmission device under sudden strong interference, such as a failed sensor causing the pixel value to be the minimum value, i.e. black point, and a saturated sensor causing the pixel value to be the maximum value, i.e. white point.

[0130] 7. Edge enhancement method: since convolutional neural networks are good at learning texture features of objects, this method enhances the edge features in the image through the Sobel operator, so as to help the model learn the effective features of the object faster and better, thereby improving the recognition ability of the model.

[0131] From the perspective of enhancing the generalization ability of the model, in the case of limited sample quantity, in order to enable the model to recognize completely new samples, that is, to enable the model to truly abstract the essential features of the object, the following data enhancement methods can be used in this embodiment to improve the generalization ability of the model:

[0132] 1. Random scaling method: in order to enable the model to have strong recognition ability for objects of different sizes, the original image can be randomly scaled within a certain range, such as [0.5, 1.5] times, to simulate objects of different distances and sizes, thereby improving the generalization ability of the model to the effective features of the object. The specific effect is as shown in Figure 4 .

[0133] 2. Mirror method: by horizontally mirroring the original image, the number of effective samples can be doubled without affecting the quality, which is a very effective data enhancement method. The specific effect is as shown in Figure 5 .

[0134] 3. Random erasing method: randomly generate multiple rectangular regions of a certain size in the image, and fill the region with the average pixel value of the entire image. This method can simulate the occlusion effect of the object, thereby greatly improving the generalization ability of the model to the object. The effect is as shown in Figure 5 .

[0135] 4. Random cutting method: randomly generate multiple rectangular regions of a certain size in the image, and fill the region with zero value, i.e. black. This method is similar to the above-mentioned random erasing method, derived from the idea of the commonly used regularization method Cutout in neural network learning, which can effectively avoid the problem of the model only recognizing the local area of the object during the learning process, thereby improving the recognition ability of the model. The effect is as shown in Figure 5 .

[0136] 5. Random Erasing: This method divides the image into SxS regions and randomly sets the pixel value of the region to 0. Similar to random erasing and random cropping, it simulates the effect of occlusion and avoids the problem of the model focusing only on the local part of the object. It is an effective data augmentation method. The effect is shown in Figure 5 .

[0137] 6. Array Occlusion: This method uses a rectangular array with a pixel value of 0 to occlude the original image, forcing the model to randomly learn each part of the object and improving the model's recognition ability of the object. The effect is shown in Figure 6 .

[0138] 7. Mixed method: This method mixes two objects in the image together, and the label is evenly distributed. For example, in the output of the neural network, the label of object A, i.e. the ideal output, is [1 0], the label of object B, i.e. the ideal output, is

[01] , and the label of the mixed image after mixing is [0.5 0.5]. This method draws on the idea of label smoothing of neural networks and is also a regularization strategy that can effectively prevent overfitting of the neural network during training. The effect is shown in Figure 6 .

[0139] 8. Cut and mix method: This method synthesizes a new image by cutting part from other images and pasting it to the target image. Thus, the model's learning must be based on multiple features of the object rather than a local feature that is easy to learn. At the same time, the label of the corresponding position of the synthesized new image is set according to the size ratio of the cut part and the remaining part of the object in the target image, such as 0.6:0.4. The effect is shown in Figure 6 .

[0140] 9. Mosaic mixing method: This method is to synthesize one image from four training images according to a certain proportion, helping the model learn how to recognize smaller objects. This method is similar to the random scaling method, but has higher learning efficiency. The effect is shown in Figure 6 .

[0141] It can be understood that the above-mentioned image processing strategies are only illustrative and do not constitute a limitation on the image processing strategy.

[0142] S103: identifying the target ground image to obtain target objects and position information of the target objects in the target ground image; the number of the target objects is at least two.

[0143] The device identifies the target ground image to obtain target objects and position information of the target objects in the target ground image. In this embodiment, the identification method of the target ground image is not limited as long as the target objects and the position information of the target objects in the target ground image can be identified.

[0144] Specifically, the target object can include workers and other construction machinery and vehicles; the location information can be the center coordinates (x, y) of the bounding box of the target object, and the size (w, h) of the bounding box.

[0145] like Figure 7 As shown, Figure 7 This is a schematic diagram of target objects and their location information in a ground image. At least two target objects are required for a collision hazard to occur.

[0146] In one implementation, to accurately identify target objects and their location information in a target ground image, a neural network can be used to identify the target ground image. The neural network model can quickly and accurately process target objects and their location information in the target ground image.

[0147] The device inputs a target ground image into a trained object recognition model for identification, obtaining the target objects and their location information within the target ground image. The device can have a pre-set trained object recognition model or can retrieve a trained object recognition model from another device. The trained object recognition model can include an input layer, a hidden layer, and an output layer (loss function layer). The input layer includes an input layer node for receiving the input target ground image from an external source. The hidden layer processes the target ground image to extract the target objects and their location information. The output layer outputs the target objects and their location information from the target ground image.

[0148] In one possible implementation, the object recognition model is pre-trained by the local device. The training method for the object recognition model can refer to the training process of the region recognition model described in S102, and will not be repeated here.

[0149] The target objects and their location information in the target ground image obtained here can be added to the training set of the neural network to optimize the model. The optimized model can better adapt to the current engineering site, which helps to improve the model's anti-interference ability and improve the accuracy of the site.

[0150] S104: Calculate the motion hazard index between the target objects based on the target objects and their location information.

[0151] According to the motion information of the dynamic entities such as the target objects in the construction site and the position information of the target objects obtained by the target ground image of the construction site, we can analyze various factors affecting the safety of the target objects and the degree of danger, so as to provide reliable warning information for the target objects. The device calculates the motion danger index between the target objects according to the target objects and the position information of the target objects. Then, if the target collision event is a real collision event, the motion danger index between the target worker and the target vehicle is calculated according to the preset motion danger index calculation rule.

[0152] The larger the motion danger index value is, the higher the danger is, and the collision is likely to occur. The smaller the value is, the smaller the danger is, and the collision possibility is very low or does not exist.

[0153] Specifically, the target objects can include target workers and target vehicles, wherein the position information of the target workers includes a first center coordinate and a first size of a first bounding box; and the position information of the target vehicles includes a second center coordinate and a second size of a second bounding box. The first center coordinate (x, y) of the first bounding box of the target worker, the first size (w, h), the second center coordinate (x', y') of the second bounding box of the target vehicle, and the second size (w', h').

[0154] The device calculates the predicted collision time of the target collision event between the target worker and the target vehicle according to the first center coordinate, the first size, the second center coordinate and the second size. We can obtain the position of any target object at any time, so as to calculate the relative motion and the predicted collision time between any two target objects.

[0155] Since the error of calculating the predicted collision time only once may be large, it is not reliable enough, therefore, in the embodiment, multiple initial collision times can be calculated to obtain a more accurate predicted collision time. Specifically, the target ground image includes multiple groups of image frames collected by the same image collection device. The initial collision time of the target collision event between the target worker and the target vehicle is calculated according to the first center coordinate, the first size, the second center coordinate and the second size of each group of image frames. The average value of all initial collision times is calculated to obtain the predicted collision time of the target collision event between the target worker and the target vehicle. For example, the device can calculate the initial collision time T every n (0 < n < 30) frames, and every m (2 < m < 20) times as a group to calculate the average value, and take the value as the predicted collision time T collision .

[0156] In an embodiment, in order to more accurately calculate the predicted collision time, each group of image frames includes at least three consecutive images when calculating the initial collision time.

[0157] The device calculates the initial collision time of the target collision event between the target worker and the target vehicle according to the first center coordinates, the first size, the second center coordinates and the second size of each group of image frames, where it is assumed that the worker and the engineering machinery are uniformly accelerated motion. The device can first calculate the first acceleration and the first speed of the target worker according to the first center coordinates of the continuous three frames of images and the first preset calculation rule, and calculate the second speed according to the first acceleration and the first speed. Wherein, the first speed is the speed of the target worker in the first frame of the continuous three frames of images; the second speed is the speed of the target worker in the third frame of the continuous three frames of images.

[0158] Specifically, the frame rate of the continuous three frames of images is f, then the two frame time difference is Δt = 1 / f; it is assumed that the positions of the target worker and the target vehicle in the image within the continuous three frames are and Wherein, the subscripts 1, 2, 3 here represent three frames, the speeds are v1, v2, v3 and v1', v2', v3', and the accelerations are a and a'; the average width and height of the bounding box of the target worker and the target vehicle within the continuous three frames are w, h and w', h' respectively. The two-frame motion of the target worker can be described by the following formula:

[0159]

[0160]

[0161] v2 = v1 + aΔt

[0162] Wherein,

[0163]

[0164]

[0165]

[0166] From the above formula, the first acceleration of the target worker is:

[0167]

[0168] The first acceleration of the target worker is:

[0169]

[0170] The second speed of the target worker is:

[0171] v3 = v1 + 2aΔt

[0172] Then, the device calculates a second acceleration and a third speed of the target vehicle according to the second center coordinates of the three continuous images and a second preset calculation rule, and calculates a fourth speed according to the second acceleration and the third speed; the third speed is the speed of the target vehicle in the first image of the three continuous images; and the fourth speed is the speed of the target vehicle in the third image of the three continuous images. The specific calculation method can refer to the calculation method of the first acceleration, the first speed and the second speed, which will not be described here.

[0173] The second acceleration, the third speed and the fourth speed calculated according to the calculation method of the first acceleration, the first speed and the second speed are as follows:

[0174]

[0175]

[0176] v′3=v′1+2a′Δt

[0177] Then, the device calculates a target distance between the target worker and the target vehicle in the third image of the three continuous images according to the first size, the second size, the second speed and the fourth speed. Specifically, the target distance between the target worker and the target vehicle in the third image can be calculated by the following formula:

[0178]

[0179]

[0180] The target distance X3 between the target worker and the target vehicle in the third image is calculated in order to determine whether the target worker and the target vehicle have a collision danger, and the target distance between the target worker and the target vehicle in the first image is X1. Before calculating the initial collision time, it is necessary to determine the relative motion direction of the two, if X1>X3, the two move towards each other or in the same direction, which has a collision danger, and the expected collision time is calculated. That is, the device determines that the target worker and the target vehicle have a collision danger according to the target distance, and then calculates the initial collision time of the target collision event according to the first acceleration, the second acceleration, the first size, the second size, the second speed and the fourth speed.

[0181] Specifically, the initial collision time of the target collision event can be calculated by the following formula:

[0182]

[0183] When determining whether there is a collision risk between a target worker and a target vehicle, if it is determined that there is no collision risk between the target worker and the target vehicle based on the target distance, that is, X1 < X3, then they are moving in opposite or the same direction and there is no collision risk, and the predicted collision time of the target collision event is infinite.

[0184] After calculating the predicted collision time, within the perspective of a specific camera, a construction machinery or vehicle may collide with a worker, along with the corresponding collision time. However, the two-dimensional planar perspective of the camera determines that this collision prediction may be invalid. For example, if the distances of the two from the camera are different, one far and one near, they will only pass by each other. Therefore, the device also needs to detect whether the target collision event is a real collision event. If the target collision event is a real collision event, then calculate the motion risk index between the target worker and the target vehicle according to the preset motion risk index calculation rule.

[0185] In this embodiment, multiple ground cameras can be set to detect whether it is a real collision. As Figure 8 shown, Figure 8 is a schematic diagram of the arrangement of ground cameras. When the ground cameras are arranged axially symmetrically and the lenses face each other parallelly, they have exactly equivalent functions in the analysis of object motion. Therefore, the ground cameras at the construction site should avoid axial symmetric arrangement, and when the number is small, an odd number of cameras should be used to avoid wasting hardware resources.

[0186] Specifically, a first preset warning time threshold and a second warning time threshold are preset in the device. If the predicted collision time is less than the first preset warning time threshold, then obtain the alternative collision time corresponding to the predicted collision time. Here, the alternative collision time is the collision time obtained from the target ground images collected by other cameras. If the alternative collision time is greater than the second preset warning time threshold and less than the first preset warning time threshold, then calculate the motion risk index between the target worker and the target vehicle according to the preset coefficient and the predicted collision time. At this time, it indicates that the risk of this collision event is relatively small and the situation is not urgent, and the motion risk index between the target worker and the target vehicle can continue to be calculated. The specific calculation method of the motion risk index between the target worker and the target vehicle is as follows:

[0187] α j = c1exp(-c2T collision ) + c3

[0188] where c1, c2, and c3 are constants, and T collision is the predicted collision time.

[0189] If the standby collision time is less than the second preset warning time threshold, the target worker and the target vehicle will have a real collision soon, which is relatively urgent, the motion danger index between the target worker and the target vehicle is not continued to be calculated, a collision warning can be directly given to the target worker, and the safety level is further improved.

[0190] S105: Determine the target region to which the target object belongs according to the region type to which each pixel belongs, and determine the region danger index according to the target region.

[0191] The device determines the target region to which the target object belongs according to the region type to which each pixel belongs. In normal work, the engineering machinery often appears in a certain region, that is, the corresponding work region. According to the analysis of construction site safety accident cases, the areas near the working engineering machinery, the construction site road, the parking lot exit, the edge of the deep pit or the concave area, etc. are the high-accident areas. When the worker appears in these areas, there is usually a high safety risk. The device determines the region danger index according to the target region. When the target object appears in the region with a higher safety risk, the region danger index is also higher. For example, if it is in the work region of the engineering machinery, the road range, etc., the region danger index value is larger, and in the office area, the idle area, etc., the region danger index value is smaller.

[0192] S106: Calculate the danger coefficient of the target object according to the motion danger index and the region danger index.

[0193] The device calculates the motion danger index and the region danger index to calculate the danger coefficient of the target object. Specifically, the motion danger index is α and the region danger index is β, and a reasonable weight coefficient k α and k β are set, and the danger coefficient of the target object can be calculated as:

[0194] Γ j = k α × α j + k β × β w,h

[0195] Wherein, Γ j is the danger coefficient of the target object j; α j is the motion danger index of the target object j, indicating the collision risk faced by the target object j at the current moment, the larger the value is, the higher the danger is, and the collision is likely to occur, the smaller the value is, the smaller the danger is, and the collision possibility is very low or does not exist; β w,his a region danger index faced by the target object j at the image position (w, h), and the index value is larger if the target object is in the working area, road, or the like of the construction machine, and the index value is smaller if the target object is in the office area, idle area, or the like; k α and k β are weight coefficients of the motion danger index and the region danger index, respectively.

[0196] S107: Perform safety monitoring on the construction site according to the danger coefficient.

[0197] The device performs safety monitoring on the construction site according to the danger coefficient, that is, a corresponding level of warning signal can be output according to the calculated danger coefficient. Different danger coefficients can be pre-set in the device to correspond to different warning signal levels, for example, when the danger coefficient is 0-30, the warning signal level is low risk; when the danger coefficient is 30-60, the warning signal level is medium risk; when the danger coefficient is 60-80, the warning signal level is high risk; and when the danger coefficient is 80-100, the warning signal level is fatal risk. Through different levels of warning signals, safety detection of the construction site is realized.

[0198] In the embodiments of the present application, a target overhead image and a target ground image of the construction site are obtained; the target overhead image is recognized to obtain the region type to which each pixel in the target overhead image belongs; the target ground image is recognized to obtain target objects and their position information in the target ground image; a motion danger index between the target objects is calculated according to the target objects and their position information; a target region to which the target objects belong is determined according to the region type to which each pixel belongs, and a region danger index is determined according to the target region; a danger coefficient of the target objects is calculated according to the motion danger index and the region danger index; and safety monitoring on the construction site is performed according to the danger coefficient. The above-mentioned scheme comprehensively obtains multi-dimensional information of the construction site from the dynamic operation point of view of the entire construction site, calculates the danger coefficient of the target objects through the motion danger index and the region danger index, and can more comprehensively perform safety warning and more comprehensively monitor the safety of the construction site through the danger coefficient.

[0199] It should be understood that the size of the serial number of each step in the above-mentioned embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0200] Please refer to Figure 9 , Figure 9 is a schematic diagram of a safety monitoring device of a construction site provided by the second embodiment of the present application. Each unit included is used to execute Figure 1 each step in the corresponding embodiment. For details, please refer to Figure 1The related description in the corresponding embodiment. For ease of illustration, only the part related to the present embodiment is shown. For details, refer to Figure 9 The safety monitoring device 9 of the engineering site includes:

[0201] The first acquisition unit 910 is configured to acquire a target overhead image and a target ground image of the engineering site.

[0202] The first identification unit 920 is configured to identify the target overhead image to obtain a region type to which each pixel in the target overhead image belongs.

[0203] The second identification unit 930 is configured to identify the target ground image to obtain a target object and position information of the target object in the target ground image; the number of the target objects is at least two.

[0204] The first calculation unit 940 is configured to calculate a motion risk index between the target objects according to the target objects and the position information of the target objects.

[0205] The first processing unit 950 is configured to determine a target region to which the target objects belong according to the region type to which each pixel belongs, and determine a region risk index according to the target region.

[0206] The second calculation unit 960 is configured to calculate a risk coefficient of the target objects according to the motion risk index and the region risk index.

[0207] The second processing unit 970 is configured to perform safety monitoring on the engineering site according to the risk coefficient.

[0208] Further, the target objects include target workers and target vehicles; the position information of the target workers includes a first center coordinate and a first size of a first bounding box; and the position information of the target vehicles includes a second center coordinate and a second size of a second bounding box.

[0209] The first calculation unit 940 is specifically configured to:

[0210] calculate an expected collision time of a target collision event between the target workers and the target vehicles according to the first center coordinate, the first size, the second center coordinate, and the second size.

[0211] If the target collision event is a real collision event, calculate a motion risk index between the target workers and the target vehicles according to a preset motion risk index calculation rule.

[0212] Further, the target ground image includes a plurality of groups of image frames collected by a same image collection device.

[0213] The first calculation unit 940 is specifically configured to:

[0214] calculate an initial collision time of a target collision event between the target worker and the target vehicle according to the first center coordinates, the first size, the second center coordinates and the second size of each group of image frames;

[0215] calculate an average value of all the initial collision times to obtain a predicted collision time of the target collision event between the target worker and the target vehicle.

[0216] Further, each group of image frames includes at least three continuous images;

[0217] The first calculation unit 940 is specifically configured to:

[0218] calculate a first acceleration and a first speed of the target worker according to the first center coordinates of the three continuous images and a first preset calculation rule, and calculate a second speed according to the first acceleration and the first speed; wherein the first speed is a speed of the target worker in a first image of the three continuous images; the second speed is a speed of the target worker in a third image of the three continuous images;

[0219] calculate a second acceleration and a third speed of the target vehicle according to the second center coordinates of the three continuous images and a second preset calculation rule, and calculate a fourth speed according to the second acceleration and the third speed; wherein the third speed is a speed of the target vehicle in the first image of the three continuous images; the fourth speed is a speed of the target vehicle in the third image of the three continuous images;

[0220] calculate a target distance between the target worker and the target vehicle in the third image of the three continuous images according to the first size, the second size, the second speed and the fourth speed;

[0221] If it is determined that there is a collision risk between the target worker and the target vehicle according to the target distance, calculate an initial collision time of a target collision event according to the first acceleration, the second acceleration, the first size, the second size, the second speed and the fourth speed.

[0222] Further, the first calculation unit 940 is specifically configured to:

[0223] If it is determined that there is no collision risk between the target worker and the target vehicle according to the target distance, the predicted collision time of the target collision event is infinite.

[0224] Further, the first calculation unit 940 is specifically configured to:

[0225] If the predicted collision time is less than a first preset warning time threshold, a backup collision time corresponding to the predicted collision time is obtained.

[0226] If the backup collision time is greater than a second preset warning time threshold and less than the first preset warning time threshold, a motion danger index between the target worker and the target vehicle is calculated according to a preset coefficient and the predicted collision time.

[0227] Further, the first identification unit 920 is specifically configured to:

[0228] The target overhead image is input into the trained region identification model for identification to obtain a region type to which each pixel in the target overhead image belongs.

[0229] Further, the first identification unit 920 is specifically configured to:

[0230] A sample training set is obtained; the sample training set includes a sample overhead image and a sample region type to which each pixel in the sample overhead image belongs.

[0231] The initial identification model is trained using the sample training set to obtain the trained region identification model.

[0232] Further, the first identification unit 920 is specifically configured to:

[0233] An initial overhead image is obtained, and the initial overhead image is processed according to a preset image processing strategy to obtain a sample overhead image; the image processing strategy includes one or more of a brightness adjustment strategy, a hue adjustment strategy, a saturation adjustment strategy, a contrast adjustment strategy, a noise adjustment strategy, an edge enhancement strategy, an image mirroring strategy, an image scaling strategy, an image removal strategy, and an image mixing strategy.

[0234] A sample region type to which each pixel in the sample overhead image belongs is obtained, and a sample training set is determined according to the sample overhead image and the sample region type to which each pixel in the sample overhead image belongs.

[0235] Figure 10 is a schematic diagram of a safety monitoring device for an engineering site provided by the third embodiment of the present application. As shown in Figure 10As shown, the safety monitoring device 10 of the construction site of this embodiment comprises a processor 100, a memory 101, and a computer program 102 stored in the memory 101 and capable of running on the processor 100, such as a safety monitoring program of a construction site. The processor 100 implements the steps in each of the above-mentioned safety monitoring method embodiments of a construction site when executing the computer program 102, such as steps 101-107 shown above. Figure 1 Alternatively, the processor 100 implements the functions of each module / unit in each of the above-mentioned device embodiments when executing the computer program 102, such as the functions of modules 910-970 shown above. Figure 9

[0236] For example, the computer program 102 can be divided into one or more modules / units, which are stored in the memory 101 and executed by the processor 100 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 102 in the safety monitoring device 10 of the construction site. For example, the computer program 102 can be divided into a first acquisition unit, a first identification unit, a second identification unit, a first calculation unit, a first processing unit, a second calculation unit, and a second processing unit, each of which has the following specific functions:

[0237] The first acquisition unit is configured to acquire a target overhead image and a target ground image of a construction site.

[0238] The first identification unit is configured to identify the target overhead image to obtain a region type to which each pixel in the target overhead image belongs.

[0239] The second identification unit is configured to identify the target ground image to obtain a target object and its position information in the target ground image; the number of target objects is at least two.

[0240] The first calculation unit is configured to calculate a motion risk index between the target objects according to the target objects and their position information.

[0241] The first processing unit is configured to determine a target region to which the target objects belong according to the region type to which each pixel belongs, and determine a region risk index according to the target region.

[0242] The second calculation unit is configured to calculate a risk coefficient of the target objects according to the motion risk index and the region risk index.

[0243] The second processing unit is configured to perform safety monitoring on the construction site according to the risk coefficient.​

[0244] The safety monitoring device of the construction site can include, but is not limited to, a processor 100, a memory 101. Those skilled in the art can understand that, Figure 10 The safety monitoring device of the construction site 10 is only an example and does not constitute a limitation on the safety monitoring device of the construction site 10, and can include more or fewer components than the diagram, or combine certain components, or different components, for example, the safety monitoring device of the construction site can also include an input / output device, a network access device, a bus, etc.

[0245] The processor 100 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0246] The memory 101 can be an internal storage unit of the safety monitoring device of the construction site 10, such as a hard disk or memory of the safety monitoring device of the construction site 10. The memory 101 can also be an external storage device of the safety monitoring device of the construction site 10, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the safety monitoring device of the construction site 10 can also include both the internal storage unit and the external storage device of the safety monitoring device of the construction site 10. The memory 101 is used to store the computer program and other programs and data required by the safety monitoring device of the construction site. The memory 101 can also be used to temporarily store data that has been output or will be output.

[0247] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units, since based on the same concept as the method embodiments of the present application, the specific functions and the technical effects brought by them can be referred to the method embodiments part, and will not be repeated here.

[0248] The embodiments of the present application further provide a network device, comprising at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in any of the above method embodiments when executing the computer program.

[0249] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the steps in any of the above method embodiments.

[0250] The embodiments of the present application provide a computer program product, which, when executed on a mobile terminal, enables the mobile terminal to implement the steps in any of the above method embodiments.

[0251] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application implements all or part of the processes in the above embodiments, which can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer readable storage medium, and the computer program, when executed by a processor, can implement the steps in each of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0252] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0253] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0254] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely schematic. The division of the modules or units is merely a logical function division. There can be another division during actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0255] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0256] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A safety monitoring method for engineering sites, characterized in that, include: Acquire top-view images and ground images of the target at the engineering site; The target top-view image is identified to obtain the region type to which each pixel in the target top-view image belongs; The target ground image is identified to obtain the target object and its location information in the target ground image; The number of target objects is at least two; Calculate the motion hazard index between the target objects based on the target objects and their location information; The target region to which the target object belongs is determined based on the region type to which each pixel belongs, and the region danger index is determined based on the target region; Calculate the risk coefficient of the target object based on the motion risk index and the area risk index; Safety monitoring is conducted at the construction site based on the aforementioned risk factor. The target objects include target workers and target vehicles; the location information of the target workers includes the first center coordinates and first dimensions of a first bounding box; the location information of the target vehicles includes the second center coordinates and second dimensions of a second bounding box. The step of calculating the motion hazard index between the target objects based on the target objects and their location information includes: The estimated collision time of the target collision event between the target worker and the target vehicle is calculated based on the first center coordinates, the first dimension, the second center coordinates, and the second dimension. If the target collision event is a real collision event, the motion hazard index between the target worker and the target vehicle is calculated according to the preset motion hazard index calculation rules, wherein multiple ground cameras are set up to detect whether it is a real collision. The target ground image includes multiple sets of image frames acquired by the same image acquisition device; The step of calculating the estimated collision time of the target collision event between the target worker and the target vehicle based on the first center coordinates, the first dimension, the second center coordinates, and the second dimension includes: The initial collision time of the target collision event between the target worker and the target vehicle is calculated based on the first center coordinates, the first size, the second center coordinates, and the second size of each group of image frames. Calculate the average of all the initial collision times to obtain the estimated collision time of the target collision event between the target worker and the target vehicle; Each group of image frames includes at least three consecutive image frames; The calculation of the initial collision time of the target collision event between the target worker and the target vehicle based on the first center coordinates, the first size, the second center coordinates, and the second size of each group of image frames includes: Based on the first center coordinates and the first preset calculation rule of the three consecutive frames of images, the first acceleration and the first velocity of the target worker are calculated, and the second velocity is calculated based on the first acceleration and the first velocity; wherein, the first velocity is the velocity of the target worker in the first frame of the three consecutive frames of images; and the second velocity is the velocity of the target worker in the third frame of the three consecutive frames of images. Based on the second center coordinates and the second preset calculation rule of the three consecutive frames of images, the second acceleration and the third velocity of the target vehicle are calculated, and the fourth velocity is calculated based on the second acceleration and the third velocity; wherein, the third velocity is the velocity of the target vehicle in the first frame of the three consecutive frames of images; and the fourth velocity is the velocity of the target vehicle in the third frame of the three consecutive frames of images. Calculate the target distance between the target worker and the target vehicle in the third frame of the three consecutive frames based on the first size, the second size, the second speed, and the fourth speed; If a collision risk is determined between the target worker and the target vehicle based on the target distance, the initial collision time of the target collision event is calculated based on the first acceleration, the second acceleration, the first size, the second size, the second speed, and the fourth speed. If the target collision event is a real collision event, then the motion hazard index between the target worker and the target vehicle is calculated according to a preset motion hazard index calculation rule, including: If the estimated collision time is less than the first preset warning time threshold, then the backup collision time corresponding to the estimated collision time is obtained, and the backup collision time is the collision time obtained based on the target ground image collected by other cameras; If the backup collision time is greater than the second preset warning time threshold and less than the first preset warning time threshold, then the motion hazard index between the target worker and the target vehicle is calculated based on the preset coefficient and the expected collision time. The motion hazard index between the target worker and the target vehicle is calculated as follows: α j =c1exp(-c2T collision )+c3; Where c1, c2, and c3 are constants, and T collision This is the estimated collision time.

2. The safety monitoring method for engineering sites as described in claim 1, characterized in that, After calculating the target distance between the target worker and the target vehicle in the third frame of the three consecutive frames based on the first size, the second size, the second speed, and the fourth speed, the method further includes: If it is determined based on the target distance that there is no risk of collision between the target worker and the target vehicle, then the expected collision time of the target collision event is infinite.

3. The safety monitoring method for engineering sites as described in claim 1, characterized in that, The step of identifying the target top-view image to obtain the region type to which each pixel in the target top-view image belongs includes: The target top-view image is input into a trained region recognition model for recognition, and the region type to which each pixel in the target top-view image belongs is obtained.

4. The safety monitoring method for engineering sites as described in claim 3, characterized in that, Before inputting the target top-view image into the trained region recognition model for recognition to obtain the region type to which each pixel in the target top-view image belongs, the method further includes: Obtain a sample training set; the sample training set includes a sample top-view image and the sample region type to which each pixel belongs; The initial recognition model is trained using the sample training set to obtain a trained region recognition model.

5. The safety monitoring method for engineering sites as described in claim 4, characterized in that, The acquisition of the sample training set includes: An initial top-view image is acquired, and the initial top-view image is processed according to a preset image processing strategy to obtain a sample top-view image; the image processing strategy includes one or more of the following: brightness adjustment strategy, hue adjustment strategy, saturation adjustment strategy, contrast adjustment strategy, noise adjustment strategy, edge enhancement strategy, image mirroring strategy, image scaling strategy, image removal strategy, and image blending strategy. Obtain the sample region type to which each pixel belongs in the sample top view image, and determine the sample training set based on the sample top view image and the sample region type to which each pixel belongs.

6. A safety monitoring device for an engineering site, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 5.

Images