Video monitoring method and system for safety monitoring of construction site

Abstract

The invention relates to the technical field of image processing, in particular to a video monitoring method and system for safety monitoring of a construction site, and the method comprises the steps: obtaining a first target image of a road construction region of the construction site; splicing the first target images to obtain a second target image; determining a moving speed index of the moving object at the current moment and a moving direction of the moving object by using the second target images at different moments; removing pixel points corresponding to the moving object from the second target image to obtain a third target image; determining the structural complexity of the road construction area in the third target image based on the key pixel points and the fixed pixel points; determining a first possibility according to the moving speed index, the moving direction, the structural complexity and the spatial distance between the moving object and other objects; and determining a target object possibly colliding with the moving object based on the first possibility and giving an alarm. Therefore, the accuracy and reliability of construction safety monitoring are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a video monitoring method and system for construction site safety monitoring. Background Art

[0002] A construction site is a place where construction activities such as building, civil engineering and equipment installation are carried out. At a construction site, there are usually various mechanical equipment, building materials and construction personnel working together to complete the project. Safety management of the construction site is an important cornerstone of engineering construction, and accurate safety monitoring of the construction site is an important means to protect construction personnel.

[0003] In some scenarios, safety accidents such as mechanical collisions are prone to occur during road construction. The road construction range is usually long, making it difficult to monitor the entire road surface. Therefore, drones are often used to obtain two-dimensional images at fixed positions, obtain the moving speed of moving objects from the two-dimensional images, set thresholds for the moving speed and alarm, and avoid collision accidents caused by excessive moving speeds of moving objects, thereby achieving safe monitoring of the construction area. However, in the actual construction process, since there is no clear road planning at the construction site, the structure is complex, and the moving paths of workers and moving objects are relatively random, it is impossible to accurately obtain the possibility of collision between moving objects based only on the speed of the moving objects, and it is difficult to accurately avoid collision accidents, resulting in low accuracy and reliability of safety monitoring at the construction site. Summary of the invention

[0004] In order to solve the technical problem of low accuracy and reliability of safety monitoring at construction sites, the purpose of the present invention is to provide a video monitoring method and system for safety monitoring at construction sites. The technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present invention provides a video surveillance method for construction site safety monitoring, comprising: acquiring a first target image of a road construction area of ​​a construction site from a surveillance video; splicing the first target image according to key pixel points in the first target image to obtain a second target image within the road construction range of the road construction area; using the second target images at different times to determine a moving speed index of a moving object in the second target image at the current time and a moving direction of the moving object; removing pixel points corresponding to the moving object from the second target image to obtain a third target image, the third target image including fixed pixel points; determining a structural complexity of the road construction area in the third target image based on key pixel points and fixed pixel points; determining a first possibility of a collision between a moving object and other objects based on a moving speed index, a moving direction, a structural complexity, and a spatial distance between the moving object and other objects; determining a target object that may collide with the moving object based on the first possibility and issuing an alarm.

[0006] Optionally, obtaining a first target image of a road construction area at a construction site from a surveillance video includes: obtaining images of the construction site at various times from the surveillance video; clustering each image to obtain multiple image clusters; for each image cluster, determining a maximum grayscale value and a maximum depth value of pixels in the image cluster, a grayscale mean value, a depth mean value, and a sum of Euclidean distances of all pixels in the image cluster, the sum of Euclidean distances being the sum of Euclidean distances between all pixels in the image cluster and a center point of the image cluster; determining a matching index between the image cluster and the road construction area based on the grayscale maximum value, the depth maximum value, the grayscale mean value, the depth mean value, and the sum of Euclidean distances; and selecting a target image cluster based on the matching index, and determining that the image in the target image cluster is the first target image of the road construction area.

[0007] Optionally, the first target image is spliced ​​according to key pixel points in the first target image to obtain a second target image within the road construction range of the road construction area, including: determining the depth of the pixel points in the first target image and the first Gaussian curvature of the pixel points; determining the local feature contribution of the pixel points according to the depth of the pixel points, the first Gaussian curvature and the matching index between the image cluster where the pixel points are located and the road construction area; determining the key pixel points in the first target image based on the local feature contribution; and using the key pixel points to splice the first target images at adjacent moments to obtain the second target image within the road construction range of the road construction area.

[0008] Optionally, using the second target images at different moments to determine the moving speed index of the moving object in the second target image at the current moment and the moving direction of the moving object includes: binarizing the second target images at different moments; subtracting the binarized second target images at adjacent moments, marking the position of the target pixel point where the first difference is not 0, and setting the grayscale value of the target pixel point to the target value; obtaining the second Gaussian curvature and gradient amplitude of the target pixel point in the area where the moving object is located in the second target image at different moments; determining the second possibility that the target pixel point is a feature point of the area where the moving object is located based on the second Gaussian curvature and gradient amplitude, and normalizing the second possibility to obtain a normalized possibility; determining the target pixel point whose normalized possibility is greater than the first threshold as a feature point; based on the feature point Determine that the imaging of the same moving object in the second target image matches, and obtain multiple second target images containing the same moving object; determine the first moving speed of the same moving object at the initial moment and the second moving speed at the latest moment according to the Euclidean distance and time interval of adjacent second target images in the multiple second target images, and the latest moment is the moment closest to the current moment; use the first moving speed, the second moving speed and the second difference between the latest moment and the initial moment to determine the moving speed index of the moving object at the current moment; determine that the direction in which the center point coordinates of the area where the moving object of the second target image at the previous moment points to the center point coordinates of the area where the moving object of the second target image at the current moment is the moving direction of the moving object at the current moment, and the previous moment is the moment before the current moment.

[0009] Optionally, the second possibility of determining that the target pixel point is a feature point of the area where the moving object is located based on the second Gaussian curvature and the gradient amplitude includes: calculating the first product of the second Gaussian curvature and the gradient amplitude of the target pixel point; and determining a first ratio of the first product to the average value of the gradient amplitudes of all target pixel points in the area where the moving object is located in the second target image as the second possibility.

[0010] Optionally, determining the moving speed index of the moving object at the current moment using the first moving speed, the second moving speed and the second difference between the most recent moment and the initial moment includes: calculating the absolute value of the third difference between the second moving speed and the first moving speed, and the second ratio of the absolute value of the third difference to the second difference; determining the sum of the second ratio and the first moving speed as the moving speed index of the moving object at the current moment.

[0011] Optionally, determining the structural complexity of the road construction area in the third target image based on key pixels and fixed pixels includes: grouping the fixed pixels with the key pixels as the center points to obtain multiple local similarity groups; determining the mean of the spatial distances between the key pixels as the search radius; determining a first number of local similarity groups within the search radius of the current local similarity group, a second number of all pixels in the current local similarity group, the intra-group local contribution of the current local similarity group, and the inter-group local contribution of all local similarity groups, the intra-group local contribution being the mean of the local feature contributions of all pixels in the local similarity group, and the inter-group local contribution being the mean of the intra-group local contributions of all local similarity groups; determining the structural complexity of the location of the road construction area where the local similarity group is located based on the first number, the second number, the intra-group local contribution and the inter-group local contribution.

[0012] Optionally, determining the first possibility of a collision between a moving object and other objects based on a moving speed index, a moving direction, a structural complexity, and a spatial distance between the moving object and other objects includes: determining an average speed index of the moving speed index of the moving object and the moving speed indexes of the other objects; calculating an angle between the moving direction of the moving object at a current moment and the overall moving direction of the moving object, the overall moving direction being the sum vector of the moving directions of the moving object at each moment; determining a second product of the average speed index, the angle, and the structural complexity; and determining a third ratio of the second product to the spatial distance as the first possibility.

[0013] Optionally, determining a target object that may collide with the moving object based on the first possibility and issuing an alarm includes: normalizing the first possibility to obtain a normalized value; determining other objects whose normalized values ​​are greater than a second threshold as target objects that may collide with the moving object and issuing an alarm.

[0014] In a second aspect, an embodiment of the present invention provides a video surveillance system for construction site safety monitoring, comprising: a processor and a memory; wherein the memory is used to store computer programs that can be run on the processor; the processor is used to execute the program stored in the memory to implement the steps of the video surveillance method for construction site safety monitoring as mentioned in the first aspect.

[0015] The present invention has the following beneficial effects: first, a first target image of a road construction area of ​​a construction site is obtained from a monitoring video, and the first target image is spliced ​​according to key pixel points in the first target image to obtain a second target image within the road construction range of the road construction area. Then, the second target images at different times are used to determine the moving speed index of the moving object in the second target image at the current time and the moving direction of the moving object. Then, the pixel points corresponding to the moving object are removed from the second target image to obtain a third target image, and the third target image includes fixed pixel points. Secondly, the structural complexity of the road construction area in the third target image is determined based on the key pixel points and the fixed pixel points, and the first possibility of the collision between the moving object and the other object is determined according to the moving speed index, the moving direction, the structural complexity and the spatial distance between the moving object and the other object, and finally, the target object that may collide with the moving object is determined based on the first possibility and an alarm is issued.

[0016] In this way, the embodiment of the present invention can obtain images at different times in the road construction area, and obtain the moving speed index and moving direction of the moving object through the images at different times. When determining the possibility of a collision of a moving object, not only the moving speed index and moving direction of the moving object are considered, but also the structural complexity of the position of the moving object in the road construction area is combined, thereby improving the accuracy of determining the possibility of a collision of the moving object and effectively avoiding collision accidents. The accuracy and reliability of construction safety monitoring are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flowchart of a video monitoring method for construction site safety monitoring provided by one embodiment of the present invention;

[0019] Figure 2 A schematic diagram of a clustering result of an image provided by an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a road construction area with key pixel points marked provided by an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of position changes of a moving object at different times provided by an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of an image after a mask is set on a second target image provided by an embodiment of the present invention;

[0023] Figure 6 A schematic structural diagram of a video surveillance system for construction site safety monitoring provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the video monitoring method and system for construction site safety monitoring proposed by the present invention, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0025] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0026] The following is a detailed description of a video monitoring method and system for construction site safety monitoring provided by the present invention in conjunction with the accompanying drawings.

[0027] Embodiment 1:

[0028] See also Figure 1 , which shows a flow chart of a video monitoring method for construction site safety monitoring provided by an embodiment of the present invention, including:

[0029] S101, acquiring a first target image of a road construction area of ​​a construction site from a monitoring video.

[0030] Specifically, the construction site in the embodiment of the present invention includes a road construction area and other areas, and the monitoring video can be obtained by a drone equipped with a depth camera. The specific process is: install a depth camera on the drone, and make the drone rise to a fixed height and move at a constant speed, obtain the monitoring video of the construction site at an angle perpendicular to the ground, and store the obtained monitoring video in the drone's own storage device. Due to the long span of the road construction area, in order to obtain a complete image of the road construction area, it is necessary to splice the images obtained multiple times in the monitoring video. In order to facilitate image splicing, the embodiment of the present invention allows the images obtained by the depth camera at adjacent moments to have a certain overlap, which is convenient for subsequent image splicing, that is, the drone traverses the entire construction site and moves slowly and at a constant speed, and obtains an image of the construction site once per second until the image of the entire construction site is completely obtained. Further, the image of the construction site obtained by the depth camera can be an RGBD image, which is equivalent to an image type that combines RGB images and depth information. In order to improve the efficiency of subsequent image processing, the RGBD image can be grayed and then the subsequent steps of the embodiment of the present invention are performed.

[0031] Furthermore, when obtaining the first target image of the road construction area of ​​the construction site from the monitoring video, as an optional embodiment of the present invention, firstly, images of the construction site at various times are obtained from the monitoring video, and then each image is clustered to obtain multiple image clusters. Secondly, for each image cluster, the maximum grayscale value and the maximum depth value of the pixel points of the image in the image cluster, the grayscale mean value, the depth mean value and the sum of the Euclidean distances of all the pixel points of the image in the image cluster are determined. The sum of the Euclidean distances is the sum of the Euclidean distances between all the pixel points in the image cluster and the center point of the image cluster. The matching index between the image cluster and the road construction area is determined based on the grayscale maximum value, the depth maximum value, the grayscale mean value, the depth mean value and the sum of the Euclidean distances. Finally, the target image cluster is selected based on the matching index, and the image in the target image cluster is determined as the first target image of the road construction area.

[0032] Specifically, the density-based spatial clustering algorithm (DBSCAN) can be used to cluster each image. The neighborhood radius in the DBSCAN algorithm is set to 5, and the minimum number of samples is also set to 5, so that n is output. j image clusters, each of which includes multiple similar images, and each image includes multiple pixels. Figure 2 As shown, Figure 2 A schematic diagram of a clustering result of an image provided by an embodiment of the present invention, Figure 2 In the figure, including the road construction area 201 and other areas 202, it can be seen that Figure 2 The grayscale and depth of the road construction area 201 and other areas 202 are quite different. Therefore, the first target image of the road construction area of ​​the construction site can be determined based on the grayscale and depth of the pixel points of the image in the image cluster.

[0033] Furthermore, the matching index between the image cluster and the road construction area refers to the matching degree between the features of the image in the image cluster and the features of the road construction area. The larger the matching index, the more the features of the image in the image cluster conform to the image features of the road construction area. When determining the matching index, as an optional embodiment of the present invention, the matching index between the image cluster and the road construction area is calculated using the following formula:

[0034]

[0035] In the above formula, δ j h represents the matching index between the jth image cluster and the road construction area. j,max Represents the maximum grayscale value of the pixel points in the j-th image cluster. dep represents the grayscale mean of all pixels in the jth image cluster. j,max Indicates the maximum depth of the pixels in the j-th image cluster. Represents the depth mean of all pixels in the jth image cluster. j,s Represents the sum of the Euclidean distances between all pixels in the j-th image cluster and the center point of the j-th image cluster. Represents the grayscale distribution difference of each pixel in the jth image cluster. Represents the depth distribution difference of each pixel in the j-th image cluster.

[0036] Furthermore, according to the above method described in the embodiment of the present invention, all image clusters are traversed to obtain the matching index between each image cluster and the road construction area. When selecting the target image cluster based on the matching index, the matching indexes of all image clusters are normalized to the range of (0,1), and the location of the image in the image cluster whose normalization result is greater than the third threshold is marked as the road construction area. The third threshold can be determined according to actual conditions. In the embodiment of the present invention, the third threshold is set to 0.5.

[0037] S102: splicing the first target image according to key pixel points in the first target image to obtain a second target image within a road construction range of the road construction area.

[0038] Specifically, due to the long span of the road construction area, it is necessary to take multiple shots by the drone to obtain a complete image of the construction road area. In order to splice the multiple shooting results of the drone, it is necessary to screen the pixels in the road construction area to retain the key pixels. The key pixels can retain the detailed information of the road construction area and reflect the local characteristics of the road construction area. Among them, if the Gaussian curvature of a pixel in the road construction area is large, the structure of the location of the pixel is different from that of other locations. If the depth difference between a pixel and other pixels is large, the spatial distribution difference between the pixel and other pixels is also greater. Therefore, when selecting key pixels, the pixels can be screened based on their Gaussian curvature and depth. Based on the screened key pixels, the first target images at adjacent moments are spliced ​​to obtain the second target image of the road construction area.

[0039] Furthermore, when determining the second target image of the road construction area, as an optional embodiment of the present invention, the depth of the pixel points in the first target image and the first Gaussian curvature of the pixel points are first determined; then the local feature contribution of the pixel points is determined based on the depth of the pixel points, the first Gaussian curvature and the matching index between the image cluster where the pixel points are located and the road construction area; then the key pixel points in the first target image are determined based on the local feature contribution; finally, the key pixel points are used to splice the first target images at adjacent moments to obtain the second target image within the road construction range of the road construction area.

[0040] Specifically, the local feature contribution of a pixel point in the embodiment of the present invention can be calculated using the following formula:

[0041]

[0042] In the above formula, ε i Indicates the local feature contribution of the i-th pixel in the first target image of the road construction area. i Represents the depth of the i-th pixel in the first target image of the road construction area. K represents the mean depth of all pixels in the first target image of the road construction area. i Represents the first Gaussian curvature of the i-th pixel in the first target image of the road construction area. Represents the mean value of the first Gaussian curvature of all pixels in the first target image of the road construction area. j It represents the matching index between the j-th image cluster where the i-th pixel point is located and the road construction area. The larger the value is, the more effective the information contained in the i-th pixel point is. Represents the depth difference between the i-th pixel and other pixels in the road construction area. Represents the curvature difference between the i-th pixel and other pixels in the road construction area.

[0043] Furthermore, all pixels in the first target image are traversed to obtain the local feature contribution of all pixels. When determining the key pixel in the first target image based on the local feature contribution, the local feature contribution is first normalized to the range of (0, 1), and the pixel whose normalization result is greater than the fourth threshold is marked as the key pixel. Figure 3 As shown, Figure 3 A schematic diagram of a road construction area with key pixel points marked provided by an embodiment of the present invention, Figure 3 The image of the road construction area includes a plurality of key pixel points 301. The fourth threshold value can be set according to actual conditions, and in the embodiment of the present invention, the value is 0.7.

[0044] Furthermore, after marking the key pixels, the first target images at adjacent moments are spliced ​​using the corner point detection principle to obtain a complete second target image of the road construction area. The corner point detection principle can refer to the prior art, and the embodiments of the present invention will not be described in detail here.

[0045] S103, using the second target images at different times, determining a moving speed index of the moving object in the second target image at the current time and a moving direction of the moving object.

[0046] Specifically, since the position of a fixed object in the images acquired at different times is the same, the position of a moving object in the images acquired at different times will change. Figure 4 As shown, Figure 4 A schematic diagram of the position change of a moving object at different times provided by an embodiment of the present invention, from Figure 4 It can be seen that at different times, the position of the moving object 401 changes significantly. Therefore, it is necessary to analyze the movement trend of the moving object, that is, to analyze the moving speed and moving direction of the moving object, so as to predict the possible collision of the moving object.

[0047] Furthermore, when determining the moving speed index and moving direction of the moving object, as an optional embodiment of the present invention, firstly, the second target image at different moments is binarized; then, the second target images at adjacent moments after the binarization are subtracted, the position of the target pixel point where the first difference value is not 0 is marked, and the grayscale value of the target pixel point is set to the target value; then, the second Gaussian curvature and gradient amplitude of the target pixel point in the area where the moving object is located in the second target image at different moments are obtained; secondly, the second possibility that the target pixel point is a feature point of the area where the moving object is located is determined based on the second Gaussian curvature and gradient amplitude, and the second possibility is normalized to obtain a normalized possibility; and the target pixel point whose normalized possibility is greater than the first threshold is determined as a feature point; then, based on the feature point, the same The imaging of the moving object in the second target image is matched to obtain multiple second target images containing the same moving object; and according to the Euclidean distance and time interval of adjacent second target images in the multiple second target images, the first moving speed at the initial moment and the second moving speed at the latest moment of the same moving object are determined, and the latest moment is the moment closest to the current moment; then the moving speed index of the moving object at the current moment is determined by using the first moving speed, the second moving speed and the second difference between the latest moment and the initial moment; finally, the direction in which the center point coordinates of the area where the moving object of the second target image at the previous moment points to the center point coordinates of the area where the moving object of the second target image at the current moment is determined as the moving direction of the moving object at the current moment, and the previous moment is the moment before the current moment.

[0048] Specifically, the second target image of the road construction area at different times is binarized, and the second target images at adjacent times are subtracted after binarization. The positions of the pixels where the difference is not 0 are marked, and the connected domain where the pixels where the difference is not 0 are located is the area where the moving object is located. Then, a mask is set for the second target image based on the marking result, that is, the grayscale value of the marked pixels is set to the target value, and the grayscale value of the remaining pixels is set to 0. The target value can be 1. Figure 5 As shown, Figure 5 A schematic diagram of an image after setting a mask on a second target image is provided in one embodiment of the present invention, Figure 5 It can be seen that the grayscale value marked as 1 is the moving object 501, and the grayscale value marked as 0 is the background.

[0049] Furthermore, since the positions of moving objects in images obtained at different times are different, and there may be multiple moving objects in the same image, it is necessary to match the images of the same moving object in images obtained at different times. When matching the images of the same moving object in images obtained at different times, matching can be performed based on the feature points of the moving object, which are pixels that can retain obvious structural features, that is, pixels with large Gaussian curvature and large gradient amplitude with surrounding pixels.

[0050] Furthermore, when determining the second possibility, the present invention, as an optional embodiment of the present invention, first calculates the first product of the second Gaussian curvature of the target pixel and the gradient amplitude; then determines the first ratio of the first product to the average value of the gradient amplitude of all target pixel points in the area where the moving object is located in the second target image as the second possibility.

[0051] Specifically, the second probability can be calculated using the following formula:

[0052]

[0053] In the above formula, γ i,z Indicates the second possibility that the i-th target pixel point in the area where the z-th moving object is located in the second target image is the feature point of the area. K i,z represents the second Gaussian curvature of the i-th target pixel in the area where the z-th moving object is located in the second target image. i,z Represents the gradient amplitude of the i-th target pixel in the area where the z-th moving object is located in the second target image. represents the average value of the gradient amplitude of all target pixels in the area where the zth moving object is located in the second target image, It represents the difference between the overall grayscale distribution of the i-th target pixel in the area where the z-th moving object in the second target image is located and the area where the z-th moving object in the second target image is located.

[0054] Furthermore, all marked target pixels are traversed, and the second probability of the target pixels is normalized to the range of (0, 1), and the target pixels whose normalized probability is greater than the first threshold are marked as feature points of the corresponding area. The first threshold can be determined according to actual conditions, and the value is 0.7 in the embodiment of the present invention. Based on the determined feature points, the imaging of the same moving object in multiple second target images is matched using the corner point matching principle to obtain multiple second target images containing the same moving object.

[0055] Furthermore, when determining the moving speed of the moving object at each moment, the calculation can be performed based on the ratio of the Euclidean distance between the center coordinates of the same moving object in two adjacent second target images to the time interval. Specifically, the calculation can be performed using the following formula:

[0056]

[0057] In the above formula, v z,k is the moving speed of the zth moving object at the kth moment. z,k Represents the Euclidean distance between the center coordinates of the same z-th moving object between adjacent second target images. z,k Indicates the time interval between two adjacent second target images.

[0058] Furthermore, when determining the moving speed index of the moving object at the current moment, as an optional embodiment of the present invention, the absolute value of the third difference between the second moving speed and the first moving speed, and the second ratio of the absolute value of the third difference to the second difference are first calculated; then the sum of the second ratio and the first moving speed is determined as the moving speed index of the moving object at the current moment.

[0059] Specifically, the moving speed index of the moving object at the current moment can be calculated using the following formula:

[0060]

[0061] In the above formula, v z is the moving speed index of the zth moving object at the current moment. z,1 Indicates the first moving speed of the zth moving object at the initial moment. z,m It represents the second moving speed of the zth moving object at the most recent time m. z,k Indicates the time interval between the initial time and the latest time (second difference). Indicates the degree to which the speed of the zth moving object changes over time.

[0062] Further, in the embodiment of the present invention, the direction from the center point coordinates of the moving object in the second target image obtained at the previous moment to the center point coordinates of the moving object in the second target image at the next moment is the moving direction of the moving object at the current moment, which is recorded as represents the moving direction of the zth moving object at the kth moment. Traverse the second target image at all moments, and record the sum vector of the moving direction obtained by the moving object at each moment as is the overall moving direction of the zth moving object. In the embodiment of the present invention, the angle between the moving direction of the zth moving object at the current moment and the overall moving direction of the zth moving object is used as the variation range of the moving direction of the zth moving object.

[0063] S104, removing pixel points corresponding to the moving object from the second target image to obtain a third target image, where the third target image includes fixed pixel points.

[0064] Specifically, in order to analyze the structural complexity of the road construction area, it is necessary to delete the pixel points corresponding to the moving object obtained in the above embodiment of the present invention, and retain the fixed pixel points of the road surface part in the road construction area. The third target image only contains the fixed pixel points of the road surface area of ​​the road construction area.

[0065] S105: Determine the structural complexity of the road construction area in the third target image based on the key pixels and the fixed pixels.

[0066] Specifically, after filtering out the pixels of moving objects, the third target image contains key pixels of the road surface area and other fixed pixels. Structural complexity refers to the complexity of the road structure in the road construction area. The larger the value, the more complex the road structure in the road construction area.

[0067] Furthermore, when determining the structural complexity of the road construction area in the third target image, as an optional embodiment of the present invention, firstly, the fixed pixels are grouped with the key pixels as the center points to obtain multiple groups of local similarity groups; then the mean of the spatial distance between the key pixels is determined as the search radius; then the first number of local similarity groups within the search radius of the current local similarity group, the second number of all pixels in the current local similarity group, the local contribution within the current local similarity group and the local contribution between all local similarity groups are determined, the local contribution within the group being the mean of the local feature contribution of all pixels in the local similarity group, and the local contribution between groups being the mean of the local contribution within the group of all local similarity groups; finally, the structural complexity of the position of the road construction area where the local similarity group is located is determined according to the first number, the second number, the local contribution within the group and the local contribution between groups.

[0068] Specifically, based on the key pixels in the third target image, fixed pixels with similar spatial distribution are grouped, that is, all pixels except the key pixels in the third target image are grouped, and the pixels closest to the key pixels are grouped into one group, that is, a local similarity group. If a certain pixel is at the same distance from multiple key pixels, the pixel and the key pixel with the largest contribution to the local feature are grouped as a local similarity group, thereby obtaining n a Locally similar groups.

[0069] Furthermore, the structural complexity of the area where the pixels in each local similarity group are located is analyzed. If the number of pixels in the local similarity group is small and the local feature contribution of each pixel is high, the degree of structural variation of the position where the pixels in the local similarity group are located is large. Taking the mean of the spatial distances between the key pixels as the search radius, if the local similarity groups within the search radius of the key pixel are relatively dense, the structure of the position where the key pixel corresponds to the local similarity group is relatively complex. Therefore, a local similarity group can be taken as an example to analyze its corresponding structural complexity. In the embodiment of the present invention, the following formula is used to calculate the structural complexity of the position of the road construction area where the local similarity group is located:

[0070]

[0071] In the above formula, u a Represents the structural complexity of the location of the road construction area where the a-th local similarity group is located. a,i Represents the second number of pixels in the a-th local similarity group. a′ Represents the first number of local similarity groups within the search radius of the a-th local similarity group. It represents the mean value of the local feature contribution of all pixels in the current a-th local similarity group, that is, the local contribution within the group. It represents the mean of the intra-group local contribution of all locally similar groups, that is, the inter-group local contribution. The larger the value is, the greater the degree of structural change of the location of the road construction area where the local similarity group is located.

[0072] Traverse all local similarity groups and obtain the structural complexity of the location of the road construction area where each local similarity is located.

[0073] S106, determining a first possibility of collision between the moving object and other objects according to the moving speed index, the moving direction, the structural complexity, and the spatial distance between the moving object and other objects.

[0074] Specifically, the first possibility refers to the possibility of the moving object colliding with other objects. The larger the value of the first possibility, the higher the possibility of the moving object colliding with other objects. The other objects can be moving objects or stationary objects, which is not limited in the embodiments of the present invention.

[0075] Further, when determining the first possibility, as an optional embodiment of the present invention, first determine the moving speed index of the moving object and the average speed index of the moving speed indexes of other objects; then calculate the angle between the moving direction of the moving object at the current moment and the overall moving direction of the moving object, and the overall moving direction is the sum vector of the moving directions of the moving object at each moment; then determine the second product of the average speed index, the angle, and the structural complexity; finally, determine the third ratio of the second product to the spatial distance as the first possibility.

[0076] Specifically, in the embodiment of the present invention, the first possibility can be calculated using the following formula:

[0077]

[0078] In the above formula, q z,z′ Represents the zth moving object and the zth ′ The first possibility of collision with other objects. Indicates the zth moving object and the zth ′ The average speed index is the average of the moving speed indexes of other objects. z,a Represents the structural complexity of the position a of the zth moving object. Represents the moving direction of the z-th moving object at the k-th moment. is the overall moving direction of the zth moving object. is the angle between the moving direction of the zth moving object at the kth moment and the overall moving direction, that is, the range of change of the moving direction of the zth moving object. z,z′ Represents the zth moving object and the zth ′ The spatial distance between other objects.

[0079] S107 , based on the first possibility, determine a target object that may collide with the moving object and generate an alarm.

[0080] Specifically, when determining the target object, other objects whose first probability is greater than a preset value can be selected as the target object, or other objects whose normalized values ​​are greater than a second threshold can be selected as the target object after normalizing the first probability. When determining the target object and issuing an alarm, as an optional embodiment of the present invention, the first probability is first normalized to obtain a normalized value; then other objects whose normalized values ​​are greater than the second threshold are determined as target objects that may collide with the moving object and an alarm is issued.

[0081] When the first possibility is normalized, it can be normalized to a range of (0, 1). If the normalized value is greater than a second threshold, it is considered that the moving object may collide with the other object. The second threshold can be determined according to actual conditions, and in the embodiment of the present invention, the value is 0.5.

[0082] The embodiment of the present invention can obtain images at different times in the road construction area, and obtain the moving speed index and moving direction of the moving object through the images at different times. When determining the possibility of a collision of a moving object, not only the moving speed index and moving direction of the moving object are considered, but also the structural complexity of the position of the moving object in the road construction area is combined, thereby improving the accuracy of determining the possibility of a collision of the moving object and effectively avoiding collision accidents. The accuracy and reliability of construction safety monitoring are improved.

[0083] Embodiment 2:

[0084] Corresponding to the video monitoring method for construction site safety monitoring provided in the above embodiment, based on the same technical concept, an embodiment of the present invention further provides a video monitoring system for construction site safety monitoring, the video monitoring system for construction site safety monitoring is used to execute the above video monitoring method for construction site safety monitoring, Figure 6 A schematic diagram of the structure of a video monitoring system for construction site safety monitoring is provided to implement various embodiments of the present invention. Figure 6 The video surveillance system for construction site safety monitoring may have relatively large differences due to different configurations or performances, and may include one or more processors 601 and memory 602, the memory 602 is used to store computer programs that can be run on the processor 601, and the processor 601 is used to execute the program stored in the memory 602 to achieve the above Figure 1 The memory 602 may be a temporary storage or a permanent storage. The application stored in the memory 602 may include one or more modules (not shown in the figure), each of which may include a series of computer executable instructions in the video monitoring system for construction site safety monitoring.

[0085] Furthermore, the processor 601 can be configured to communicate with the memory 602 to execute a series of computer executable instructions in the memory 602 on the video monitoring system for construction site safety monitoring. The video monitoring system for construction site safety monitoring can also include one or more power supplies 603, one or more wired or wireless network interfaces 604, one or more input and output interfaces 605, and one or more keyboards 606.

[0086] Specifically in this embodiment, the video surveillance system for construction site safety monitoring includes a processor, a communication interface, a memory and a communication bus; wherein the processor, the communication interface and the memory communicate with each other through the bus; the memory is used to store computer programs; the processor is used to execute the programs stored in the memory to achieve the above Figure 1 The various steps in the method embodiment are similar to those in the method embodiment, and have the beneficial effects of the above method embodiments. To avoid repetition, the embodiments of the present invention will not be described in detail here.

[0087] It should be noted that the video surveillance system for construction site safety monitoring provided in the embodiment of the present invention and the video surveillance method for construction site safety monitoring provided in the embodiment of the present invention are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned video surveillance method for construction site safety monitoring, and has the same or similar beneficial effects, and the repetitions will not be repeated.

[0088] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0089] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A video monitoring method for construction site safety monitoring, characterized in that: The video monitoring method for construction site safety monitoring comprises: Acquire a first target image of a road construction area of ​​the construction site from the surveillance video; splicing the first target image according to the key pixel points in the first target image to obtain a second target image within the road construction range of the road construction area; Determine a moving speed index of a moving object in the second target image at a current moment and a moving direction of the moving object by using the second target image at different moments; Eliminate the pixel points corresponding to the moving object from the second target image to obtain a third target image, wherein the third target image includes fixed pixel points; Determining the structural complexity of the road construction area in the third target image based on the key pixel points and the fixed pixel points; Determining a first possibility of a collision between the moving object and the other object according to the moving speed index, the moving direction, the structural complexity, and a spatial distance between the moving object and the other object; Based on the first possibility, a target object that may collide with the moving object is determined and an alarm is issued.

2. The video monitoring method for construction site safety monitoring according to claim 1, characterized in that: The step of acquiring a first target image of a road construction area at a construction site from a monitoring video comprises: Acquire images of the construction site at various times from the surveillance video; Clustering each of the images to obtain a plurality of image clusters; For each image cluster, determine the maximum grayscale value and the maximum depth value of the pixel points of the image in the image cluster, the grayscale mean value, the depth mean value and the sum of the Euclidean distances of all the pixel points of the image in the image cluster, where the sum of the Euclidean distances is the sum of the Euclidean distances between all the pixel points in the image cluster and the center point of the image cluster; Determine a matching index between the image cluster and the road construction area according to the grayscale maximum value, the depth maximum value, the grayscale mean value, the depth mean value, and the sum of the Euclidean distances; A target image cluster is selected based on the matching index, and an image in the target image cluster is determined to be a first target image in the road construction area.

3. The video monitoring method for construction site safety monitoring according to claim 1, characterized in that: The step of stitching the first target image according to the key pixel points in the first target image to obtain a second target image within the road construction area comprises: Determine the depth of a pixel in the first target image and a first Gaussian curvature of the pixel; Determine the local feature contribution of the pixel point according to the depth of the pixel point, the first Gaussian curvature, and a matching index between the image cluster where the pixel point is located and the road construction area; Determine key pixel points in the first target image based on the local feature contribution; The key pixel points are used to stitch the first target images at adjacent moments to obtain a second target image within the road construction range of the road construction area.

4. The video monitoring method for construction site safety monitoring according to claim 1, characterized in that: The method of determining the moving speed index of the moving object in the second target image at the current moment and the moving direction of the moving object by using the second target image at different moments includes: Binarizing the second target image at different times; Subtract the binarized second target images at adjacent moments, mark the position of the target pixel where the first difference value is not 0, and set the gray value of the target pixel to the target value; Acquire a second Gaussian curvature and a gradient amplitude of a target pixel point in the area where the moving object is located in the second target image at different times; Determining a second possibility that the target pixel is a feature point of the region where the moving object is located according to the second Gaussian curvature and the gradient amplitude, and normalizing the second possibility to obtain a normalized possibility; Determine the target pixel point whose normalized probability is greater than a first threshold as the feature point; Determine, based on the feature points, that the images of the same moving object in the second target image match, and obtain a plurality of second target images containing the same moving object; Determine, according to the Euclidean distance and the time interval between adjacent second target images in the plurality of second target images, a first moving speed at an initial moment and a second moving speed at a latest moment of the same moving object, wherein the latest moment is a moment closest to the current moment; Determine a moving speed index of the moving object at a current moment by using the first moving speed, the second moving speed, and a second difference between the most recent moment and an initial moment; Determine the direction in which the center point coordinates of the area where the moving object of the second target image at the previous moment is located point to the center point coordinates of the area where the moving object of the second target image at the current moment as the moving direction of the moving object at the current moment, and the previous moment is the moment before the current moment.

5. The video monitoring method for construction site safety monitoring according to claim 4, characterized in that: The second possibility of determining, according to the second Gaussian curvature and the gradient amplitude, that the target pixel point is a feature point of the area where the moving object is located includes: Calculating a first product of a second Gaussian curvature of the target pixel and the gradient amplitude; A first ratio of the first product to an average value of the gradient amplitudes of all target pixel points in the area where the moving object is located in the second target image is determined as the second possibility.

6. The video monitoring method for construction site safety monitoring according to claim 4, characterized in that: Determining the moving speed index of the moving object at the current moment by using the first moving speed, the second moving speed, and the second difference between the most recent moment and the initial moment includes: calculating an absolute value of a third difference between the second moving speed and the first moving speed, and a second ratio of the absolute value of the third difference to the second difference; The sum of the second ratio and the first moving speed is determined as the moving speed index of the moving object at the current moment.

7. The video monitoring method for construction site safety monitoring according to claim 3, characterized in that: Determining the structural complexity of the road construction area in the third target image based on the key pixel points and the fixed pixel points includes: The fixed pixels are grouped using the key pixels as center points to obtain multiple groups of local similarity groups; Determine the mean of the spatial distances between the key pixels as the search radius; Determine a first number of local similarity groups within a search radius of a current local similarity group, a second number of all pixels in the current local similarity group, an intra-group local contribution of the current local similarity group, and an inter-group local contribution of all local similarity groups, wherein the intra-group local contribution is the average of the local feature contributions of all pixels in the local similarity group, and the inter-group local contribution is the average of the intra-group local contributions of all local similarity groups; The structural complexity of the location of the road construction area where the local similarity group is located is determined according to the first number, the second number, the local contribution within the group, and the local contribution between the groups.

8. The video monitoring method for construction site safety monitoring according to any one of claims 1 to 7, characterized in that: The determining, according to the moving speed index, the moving direction, the structural complexity, and the spatial distance between the moving object and the other object, of the first possibility of the collision between the moving object and the other object comprises: Determine an average speed index of the moving speed index of the moving object and the moving speed indexes of the other objects; Calculate the angle between the moving direction of the moving object at the current moment and the overall moving direction of the moving object, where the overall moving direction is the sum vector of the moving directions of the moving object at each moment; Determine a second product of the average speed index, the angle, and the structural complexity; A third ratio of the second product to the spatial distance is determined as the first possibility.

9. The video monitoring method for construction site safety monitoring according to any one of claims 1 to 7, characterized in that: The step of determining the target object that may collide with the moving object based on the first possibility and issuing an alarm includes: Normalizing the first possibility to obtain a normalized value; Other objects whose normalized values ​​are greater than a second threshold are determined to be target objects that may collide with the moving object and an alarm is issued.

10. A video surveillance system for construction site safety monitoring, characterized in that: The video surveillance system for construction site safety monitoring includes: a processor and a memory; wherein the memory is used to store computer programs that can be run on the processor; the processor is used to execute the program stored in the memory to implement the steps of the video surveillance method for construction site safety monitoring as described in any one of claims 1-9.

Images