A construction site material intelligent monitoring method, device, equipment and storage medium

Abstract

The application discloses a kind of construction site material intelligent monitoring method, device, equipment and storage medium, applied to construction site material monitoring management field, the monitoring video data of image acquisition equipment is acquired, target frame image is determined from monitoring video data every interval preset time period;Target frame image is input material image segmentation model, and material segmentation result is obtained;When material segmentation result does not exist target material, then, authorization judgment processing is carried out;When material segmentation result exists target material, the center point position of target material segmentation result in material segmentation result is determined;When the offset of center point position is greater than or equal to first preset threshold, then, authorization judgment processing is carried out;When the offset of center point position is less than first preset threshold, then, the mask of target material segmentation result is determined;When the offset of mask is greater than or equal to second preset threshold, then, authorization judgment processing is carried out.The application improves the accuracy and efficiency of construction site material monitoring, reduces labor cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of construction site material monitoring management, and in particular to a construction site material intelligent monitoring method, a construction site material intelligent monitoring device, an electronic device and a computer readable storage medium. BACKGROUND

[0002] In a complex construction site environment, materials and equipment of different companies are often stacked in different intervals, and companies need to monitor the materials to ensure that the materials are not moved or damaged by unauthorized personnel during transportation and use, so as to prevent other company personnel or outsiders from mistakenly taking or stealing, resulting in economic losses and safety problems. The current material monitoring method is generally through manual monitoring of the monitoring video, and the video monitoring needs to be watched by a special person in real time, which not only requires a large number of human resources, but also increases the labor cost, and the reaction speed of the person is slow, which may miss some places, and is not conducive to material monitoring. SUMMARY

[0003] The present application aims to provide a construction site material intelligent monitoring method, device, electronic device and computer readable storage medium, which is applied to the field of construction site material monitoring management, and the method divides the materials in the monitoring video by image segmentation through artificial intelligence, and monitors the materials according to the set process based on the segmented material images, thereby improving the accuracy and efficiency of the construction site material monitoring, and avoiding the problems of high labor cost and poor monitoring effect caused by manual monitoring.

[0004] To solve the above technical problems, the present application provides a construction site material intelligent monitoring method, comprising:

[0005] An image acquisition device for monitoring a target area of a construction site is set, monitoring video data of the image acquisition device is acquired, and a target frame image is determined from the monitoring video data every predetermined time period;

[0006] A target material that needs to be monitored is set, the target frame image is input into a material image segmentation model, and a material segmentation result is obtained;

[0007] When the target material does not exist in the material segmentation result, an authorization judgment process is performed; when the target material exists in the material segmentation result, a center point position of the target material segmentation result in the material segmentation result is determined;

[0008] When the offset of the center point position is greater than or equal to a first predetermined threshold, the authorization judgment process is performed; when the offset of the center point position is less than the first predetermined threshold, a mask of the target material segmentation result is determined;

[0009] When the offset of the mask is greater than or equal to a second preset threshold, the authorization judgment processing is performed.

[0010] Optionally, a target frame image is determined from the monitoring video data every preset time period, including:

[0011] Converting all frame images of the monitoring video data in the preset time period into grayscale images;

[0012] Determining a Laplacian value of each pixel point in the grayscale image, and determining a Laplacian mean value of the grayscale image based on the Laplacian value;

[0013] Determining a Laplacian variance of the grayscale image based on the Laplacian mean value and the Laplacian value;

[0014] Determining the grayscale image with the largest Laplacian variance as a target grayscale image, and determining the frame image converted into the target grayscale image as the target frame image.

[0015] Optionally, the offset of the center point position is determined by the center point position of the target material segmentation result in a current preset time period, the center point positions of the target material segmentation results in a first preset number of previous preset time periods, and the center point positions of the target material segmentation results in a second preset number of previous preset time periods.

[0016] The offset of the mask is determined by the set of mask center points of the target material segmentation result in a current preset time period and the set of mask center points of the target material segmentation result in a third preset number of previous preset time periods.

[0017] Optionally, the method further includes:

[0018] When receiving a new material sample set, performing data augmentation on the new material sample set based on light intensity simulation and light angle simulation to obtain an augmented sample set;

[0019] Performing incremental learning training on the material image segmentation model based on the augmented sample set to obtain a trained material image segmentation model.

[0020] Optionally, the authorization judgment processing includes:

[0021] Determining whether a current variation time is within a preset authorization time, and / or whether a preset identifier exists in the target frame image;

[0022] If yes, determining that the variation of the target material is an authorized variation, saving variation video data of the target material from the monitoring video data, and canceling monitoring of the target material;

[0023] If no, an alarm is generated, the change video data of the target material is saved from the monitoring video data, and the change video data is sent to a supervisor.

[0024] Optionally, the method further comprises:

[0025] When it is determined based on the material segmentation result that the position of the target material has an occlusion, it is determined whether the target material is a material that can be occluded.

[0026] If yes, an occlusion time of the occlusion is determined, and if the occlusion time exceeds a preset occlusion time, an alarm is generated.

[0027] If no, the alarm is generated.

[0028] Optionally, determining whether the target material is a material that can be occluded comprises:

[0029] Based on the material segmentation result, a type of the occlusion at the position of the target material is determined, and it is determined whether the type of the occlusion is an authorized occlusion type of the target material.

[0030] To solve the above technical problem, the present application provides a construction site material intelligent monitoring device, comprising:

[0031] A first module is configured to set an image acquisition device for monitoring a target area of a construction site, acquire monitoring video data of the image acquisition device, and determine a target frame image from the monitoring video data every preset time period.

[0032] A second module is configured to set a target material that needs to be monitored, input the target frame image into a material image segmentation model, and obtain a material segmentation result.

[0033] A third module is configured to perform authorized judgment processing when the target material does not exist in the material segmentation result, and determine a center point position of a target material segmentation result in the material segmentation result when the target material exists in the material segmentation result.

[0034] A fourth module is configured to perform the authorized judgment processing when an offset of the center point position is greater than or equal to a first preset threshold, and determine a mask of the target material segmentation result when the offset of the center point position is less than the first preset threshold.

[0035] A fifth module is configured to perform the authorized judgment processing when an offset of the mask is greater than or equal to a second preset threshold.

[0036] To solve the above technical problem, the present application provides an electronic device, comprising:

[0037] a memory for storing a computer program;

[0038] a processor for implementing the above-mentioned construction site material intelligent monitoring method when executing the computer program.

[0039] To solve the above technical problems, the present application provides a computer readable storage medium, the computer readable storage medium has computer executable instructions stored therein, the computer executable instructions are executed by a processor to implement the above-mentioned construction site material intelligent monitoring method.

[0040] It can be seen that the method of the present application sets up an image acquisition device for monitoring the target area of the construction site, acquires the monitoring video data of the image acquisition device, determines the target frame image from the monitoring video data every preset time period, sets the target material that needs to be monitored, inputs the target frame image into the material image segmentation model, and obtains the material segmentation result. When the target material does not exist in the material segmentation result, authorization judgment processing is performed. When the target material exists in the material segmentation result, the center point position of the target material segmentation result in the material segmentation result is determined. When the offset of the center point position is greater than or equal to the first preset threshold, authorization judgment processing is performed. When the offset of the center point position is less than the first preset threshold, the mask of the target material segmentation result is determined. When the offset of the mask is greater than or equal to the second preset threshold, authorization judgment processing is performed.

[0041] The present application performs image segmentation on the materials in the monitoring video in an artificial intelligence manner, monitors the materials according to the set process based on the segmented material images, improves the accuracy and efficiency of the construction site material monitoring, and avoids the problems of high labor cost and poor monitoring effect caused by manual monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0043] Figure 1 A flow chart of a construction site material intelligent monitoring method provided by an embodiment of the present application;

[0044] Figure 2 A structural block diagram of a construction site material intelligent monitoring device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0045] Clearly, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present application.

[0046] In a complex construction site environment, materials and equipment of different companies are often stacked in different intervals, and companies need to monitor the materials to ensure that the materials are not moved or damaged by unauthorized personnel during transportation and use, to prevent misplacement or theft by other company personnel or outsiders. Currently, there are some problems in the method of related construction site material monitoring and management.

[0047] Real-time monitoring of materials and equipment: In a complex construction site, real-time monitoring of materials and equipment is required. If traditional video monitoring is used, the corresponding labor cost is high, and video monitoring requires a dedicated person to watch the monitoring video in real time, which not only requires a large number of human resources, but also increases labor costs. Human reaction speed is slow, and there may be omissions, and video monitoring may be blocked, making it difficult to trace in the later stage.

[0048] Identity verification of related equipment and resources: When the protected equipment needs to be moved and used reasonably, the user and the use time should be reasonably regulated. In general, a person needs to be stationed on site to confirm that the person is authorized, or a camera needs to monitor the person in the picture to identify whether the person is authorized, which consumes human and material resources.

[0049] Material and equipment flow information recording: The construction site material information is recorded, stored, and used to integrate the entire system information and save the relevant change information video to facilitate future reverse tracking and tracking of materials. Currently, the information is recorded by manual recording, which is inefficient and may contain errors.

[0050] The present application proposes a new material and equipment management method through computer vision technology in the monitoring of materials and equipment, verification of worker identity, and recording of material and equipment information, and records the change information of materials and equipment.

[0051] The following will be combined Figure 1 , Figure 1 A flowchart of a construction site material intelligent monitoring method provided by an embodiment of the present application, the method can include:

[0052] S101: Set an image acquisition device for monitoring a target area of a construction site, acquire monitoring video data of the image acquisition device, and determine a target frame image from the monitoring video data every preset time period.

[0053] The embodiment can set an image acquisition device for monitoring a target area of a construction site. The embodiment does not limit the specific type of the image acquisition device or the format of the monitoring video data acquired thereby, and can be set based on actual application.

[0054] Every preset time period, the embodiment can determine a target frame image from the monitoring video data, for construction material identification. This can not only reduce the overall model calculation amount, but also effectively speed up the judgment speed of the overall model. The embodiment does not limit the specific length of the preset time period, which can generally be 30 seconds.

[0055] The target frame image in the embodiment can be the clearest frame image in the monitoring video data within the preset time period. The embodiment does not limit the specific manner of determining the target frame image. Generally, all frame images of the monitoring video data within the preset time period can be converted into grayscale images. The Laplacian value of each pixel point in the grayscale image is determined, and the Laplacian mean value of the grayscale image is determined based on the Laplacian value. The Laplacian variance of the grayscale image is determined based on the Laplacian mean value and the Laplacian value. The grayscale image with the largest Laplacian variance is determined as the target grayscale image, and the frame image converted into the target grayscale image is determined as the target frame image.

[0056] Specifically, if the time length of the preset time period is T and the frame image of the monitoring video data per second is k frames, the total number of frame images within the preset time period is T*k.

[0057] For any frame image, it can be converted into a grayscale image. The conversion formula of the grayscale image can be:

[0058] N(x, y) = 0.299 * R(x, y) + 0.587 * G(x, y) + 0.114 * B(x, y);

[0059] In the formula, (x, y) is the coordinate of the pixel point, R(x, y) is the pixel value of the corresponding coordinate point in the red channel of the frame image, G(x, y) is the pixel value of the corresponding coordinate point in the green channel of the frame image, R(x, y) is the pixel value of the corresponding coordinate point in the blue channel of the frame image, and N(x, y) is the pixel value of the corresponding coordinate point in the grayscale image.

[0060] Further, the embodiment can determine the Laplacian value of each pixel point in the grayscale image as follows:

[0061]

[0062] ​wherein (x, y) is the coordinate of a pixel point, is a partial differential symbol, and L(x, y) is the Laplacian value of the corresponding coordinate point in the gray image.

[0063] Further, the Laplacian mean of the gray image can be determined based on the Laplacian value:

[0064] ;

[0065] wherein u is the Laplacian mean of the gray image, M is the width of the frame image, and N is the length of the frame image.

[0066] Finally, the Laplacian variance of the gray image can be determined based on the Laplacian mean and the Laplacian value:

[0067] ;

[0068] wherein, is the Laplacian variance of the gray image.

[0069] The embodiment can convert T*k frame images in total in a preset time period into gray images and perform calculation of the Laplacian variance, and after the calculation is completed, the gray image with the maximum Laplacian variance can be determined from the T*k gray images as the target gray image, and the frame image corresponding thereto is the target frame image.

[0070] S102: Set a target material to be monitored, input the target frame image into a material image segmentation model, and obtain a material segmentation result.

[0071] The target frame image is input into the material image segmentation model to obtain the material segmentation result. The material image segmentation model in the embodiment can simultaneously segment multiple materials in the target frame image.

[0072] The embodiment does not limit the training manner of the material image segmentation model, and generally a data set containing multiple construction site materials can be obtained, such as a data set already labeled on the network. Further, the data set can be labeled according to a preset classification, and generally the construction site materials can be labeled in a size class manner, and the model is trained based on the labeled data set.

[0073] The embodiment does not limit the specific types of the construction site materials, and the general material types can include: building materials, construction equipment, auxiliary tools and equipment, construction machinery, and safety protection equipment.

[0074] The building materials can include: cement: the main binder for concrete and mortar; sandstone: including sand and gravel, which is an important component of concrete and mortar; reinforcing steel: to enhance the tensile strength of concrete structures; bricks: basic materials for building walls, such as red bricks, hollow bricks, etc.; wood: used for formwork, scaffolding and decorative structures; glass: used for windows, doors and curtain walls; thermal insulation materials: such as rock wool, glass wool, used for thermal insulation of buildings; waterproof materials: such as waterproof paint, waterproof coiled material, used for waterproof treatment.

[0075] The construction equipment can include: formwork: a forming mold for concrete pouring; scaffolding: a platform for providing support for high-altitude work; surveying instruments: total station, level, theodolite, etc., used for construction surveying and setting out; temporary facilities; generators: used when power is insufficient or power is off; temporary lighting equipment: to ensure lighting during night construction or insufficient light; water pumps: for drainage or water supply.

[0076] The construction machinery can include: hoisting machinery such as tower cranes and truck cranes; earthmoving machinery such as excavators and bulldozers; concrete machinery such as concrete mixers and concrete pump trucks; compaction machinery such as road rollers and rammer machines; pile driving machinery such as pile drivers; road construction machinery such as pavers.

[0077] The auxiliary tools and equipment can include: power tools such as drills, cutters and welders; hand tools such as hammers, wrenches, shovels and brushes; surveying instruments such as total stations, levels and theodolites.

[0078] The safety protection equipment can include: safety helmets, safety belts, goggles, masks, protective clothing and gloves.

[0079] Since the types of materials in the training samples are various, and not every material needs to be monitored, the embodiment can set target materials that need to be monitored from various materials, and the embodiment can monitor the changes of the target materials. The target materials in the embodiment can be changed, and specifically can be set based on actual engineering needs.

[0080] And the embodiment does not limit the specific way of how to set the target materials, and generally the materials can be marked as target materials that need to be monitored in the form of issuing protection commands. If the monitoring of the target materials is to be cancelled, the protection command of the target materials can be cancelled.

[0081] Further, the embodiment can perform incremental updating on the material image segmentation model. When a new material sample set is received, the new material sample set is data augmented based on light intensity simulation and light angle simulation to obtain an augmented sample set; the material image segmentation model is incrementally learned and trained based on the augmented sample set to obtain a trained material image segmentation model.

[0082] In a specific embodiment, when new materials need to be monitored, the newly added materials in the monitoring video data are simply labeled under good lighting conditions to form a small sample training data set, and then the small sample data set is expanded under different lighting intensity and angle changes to simulate the material display changes under different lighting conditions.

[0083] When simulating different intensity lightings, the mathematical formula for adjusting the lighting intensity is as follows:

[0084] Iq=α*I+β;

[0085] In the formula, I is the original brightness value of a pixel point in the image, Iq is the brightness value of the pixel point after adjusting the lighting intensity, a is the brightness coefficient, and β is the brightness offset value. When a>1, the brightness increases; if a<1, the brightness decreases; when β>0, the brightness increases; when β<0, the brightness decreases.

[0086] In this embodiment, for each image in the newly added material sample set, the brightness value of the pixel point in the image can be changed by simulating different intensity lightings to obtain the expanded sample.

[0087] When random sunlight is used, the direction vector of the light source can be set as D=(Dx, Dy), which represents the direction of the light source relative to the image. In order to standardize the calculation, the vector is usually unitized:

[0088] ;

[0089] In the formula, Du is the unit vector of the light source with a length of 1, and Dx and Dy are the components of the direction vector D.

[0090] Further, the position vector of each pixel can be calculated. For a pixel point with coordinates (x, y), the normalized position vector of the pixel point is calculated as follows:

[0091] ;

[0092] In the formula, P is the normalized position vector of the pixel point, M is the width of the frame image, N is the length of the frame image, and (x, y) is the coordinates of the pixel point.

[0093] Further, the dot product calculation can be used to simulate the sunlight. The dot product of the position vector of the pixel point and the unit vector of the light source reflects the cosine value between the two:

[0094] cos(θ)=Du*P;

[0095] Wherein, θ is the included angle between the position vector of the pixel point and the unit vector of the light source, and cos(θ) is the cosine value of the included angle.

[0096] The embodiment can limit the cosine value of the included angle:

[0097]

[0098] Wherein, I(θ) is the limited cosine value, and max is the maximum value function.

[0099] Finally, the change of the simulated light with the point light source is realized by limiting the cosine value:

[0100] Ip = I(θ) * I;

[0101] Wherein, I is the original brightness value of a pixel point in the image, and Ip is the brightness value of the pixel point after adjusting the light angle.

[0102] The embodiment can change the brightness value of the pixel point in the image by simulating different angle light, so as to obtain the expanded sample. In the embodiment, the light intensity simulation and the light angle simulation can act on the image in the newly added material sample set at the same time.

[0103] The embodiment can perform incremental learning training on the material image segmentation model based on the expanded sample set, and obtain the trained material image segmentation model. Specifically, the material image segmentation model can be trained by using the transfer learning strategy, so that the material image segmentation model is adapted to the new image segmentation task, and further, the model is consolidated and the learning rate is adjusted by using the incremental learning method, so as to improve the performance of the model in the new image segmentation task.

[0104] S103: When the target material does not exist in the material segmentation result, the authorization judgment processing is performed; when the target material exists in the material segmentation result, the center point position of the target material segmentation result in the material segmentation result is determined.

[0105] The embodiment can determine whether the target material changes and whether the change of the target material is authorized.

[0106] In the embodiment, when the target material does not exist in the material segmentation result, that is, the target material cannot be recognized in the target frame image, it can be determined that the target material changes, and further, the authorization judgment processing is needed to determine whether the change of the target material is authorized.

[0107] ​The embodiment is not limited to the specific manner of authorization judgment processing. Generally, it can be judged whether the current change time is within the preset authorization time and / or whether the preset identifier exists in the target frame image. If yes, it is determined that the change of the target material is an authorized change, the change video data of the target material is saved from the monitoring video data, and the monitoring of the target material is canceled. If no, an alarm is generated, the change video data of the target material is saved from the monitoring video data, and the change video data is sent to the supervisor. In the embodiment, the change time of the target material can also be saved synchronously to facilitate the traceability of the administrator.

[0108] The authorization judgment can be mainly divided into time authorization judgment and authorization judgment of a specific identifier person. The authorization time judgment only needs to check whether the current change time is within the authorization time period, and the authorization of the specific identifier person needs to set a specific identifier that can appear in the video image, such as a specific clothes, a specific safety helmet, and the like.

[0109] Further, when the target material exists in the material segmentation result, the embodiment can determine the center point position of the target material segmentation result in the material segmentation result. That is, when the target material can be recognized in the target frame image, it is determined that the target material is still within the monitorable range, and at this time, it is needed to judge whether the target material is moved within the monitorable range.

[0110] Specifically, the center point position of the target material segmentation result can be determined, and whether the target material is changed is judged by comparing the center point positions.

[0111] S104: When the offset of the center point position is greater than or equal to the first preset threshold, the authorization judgment processing is performed; and when the offset of the center point position is less than the first preset threshold, the mask of the target material segmentation result is determined.

[0112] The embodiment can determine the offset of the center point position. When the offset of the center point position is greater than or equal to the first preset threshold, the authorization judgment processing is performed.

[0113] The embodiment is not limited to the specific manner of determining the offset of the center point position. Generally, it can be determined by the center point position of the target material segmentation result in the current preset time period, the center point positions of the target material segmentation results in the first preset number of preset time periods, and the center point positions of the target material segmentation results in the second preset number of preset time periods.

[0114] Specifically, the coordinates (x j ,y j ) of the center point position of the target material segmentation result in the current preset time period, the coordinates (x j-m ,yj-m The coordinates (x, y) of the center point of the target material segmentation result within the first two preset time periods. j-n ,y j-n Generally, the first preset quantity can be 1, i.e., m=1, and the second preset quantity can be 4, i.e., n=4.

[0115] First, this embodiment can determine the position of the center point (x). j ,y j ) and the position of the center point (x) j-m ,y j-m The first offset between:

[0116] ;

[0117] Furthermore, determine the location of the center point (x) j ,y j ) and the position of the center point (x) j-n ,y j-n The second offset between:

[0118] ;

[0119] In the formula, d1 is the first offset, d2 is the second offset, d1+d2 is the offset of the center point position, (x j ,y j (x) represents the coordinates of the center point of the target material segmentation result within the current preset time period. j-m ,y j-m (x) represents the coordinates of the center point of the target material segmentation result within the first preset number of preset time periods, where (x) j-n ,y j-n ) represents the coordinates of the center point of the target material segmentation result within the first second preset number of preset time periods.

[0120] In this embodiment, a first preset threshold can be determined. When the offset of the center point is greater than or equal to the first preset threshold, it can be determined that the target material has shifted. At this time, authorization judgment processing is required to determine whether the change of the target material is an authorized change.

[0121] If the offset of the center point is less than a first preset threshold, it can be considered that the target material has not changed, or that the change in the target material is within an acceptable range. Therefore, this embodiment can use a mask of the target material segmentation result to make a more accurate determination of the change.

[0122] S105: When the mask offset is greater than or equal to the second preset threshold, authorization judgment processing is performed.

[0123] In this embodiment, when the offset of the mask is greater than or equal to the second preset threshold, the authorization judgment processing is performed.

[0124] The embodiment is not limited to a specific way of determining the mask offset. Generally, the mask midpoint set of the target material segmentation result in the current preset time period and the mask midpoint set of the target material segmentation result in the previous third preset number of preset time periods can be determined.

[0125] Specifically, the mask midpoint set A of the target material segmentation result in the current preset time period can be determined, and the mask midpoint set B of the target material segmentation result in the previous third preset number of preset time periods can be determined. Generally, the third preset number can be 1.

[0126] The calculation formula of the mask offset can be:

[0127] ;

[0128] In the formula, Y is the mask offset, A is the mask midpoint set of the target material segmentation result in the current preset time period, and B is the mask midpoint set of the target material segmentation result in the previous third preset number of preset time periods.

[0129] The embodiment can determine the second preset threshold, which represents the maximum threshold of mask change. When the mask offset is greater than or equal to the second preset threshold, it can be determined that the target material has changed, and the authorization judgment processing needs to be performed to determine whether the change of the target material is authorized.

[0130] If the mask offset is less than the second preset threshold, it can be considered that the target material has not changed, or the change of the target material is within an allowable range, that is, in the current target frame image, the target material has not changed or the change of the target material is within an allowable range.

[0131] The embodiment adopts a dual judgment mechanism of center point position offset and mask change to accurately identify abnormal movement of materials and equipment, and combines spatial position and shape change algorithm to improve the accuracy and reliability of change detection.

[0132] Further, due to the frequent coming and going of vehicles and personnel in the factory, it is easy to occur that the target material is blocked during the target frame image recognition process. Therefore, the embodiment can perform a blocking judgment of the target material.

[0133] The embodiment is not limited to the specific manner of determining whether the position of the target material is blocked, and generally, when it is determined based on the material segmentation result that the position of the target material is blocked, it is determined whether the target material is a blockable material; if yes, the blocking time of the blocking material is determined, and if the blocking time of the blocking material exceeds the preset blocking time, an alarm is generated; if no, an alarm is generated.

[0134] The embodiment is not limited to the specific manner of determining whether the position of the target material is blocked, and generally, when it is determined based on the material segmentation result that the position of the target material is blocked, it is determined whether the target material is a blockable material; if yes, the blocking time of the blocking material is determined, and if the blocking time of the blocking material exceeds the preset blocking time, an alarm is generated; if no, an alarm is generated.

[0135] Further, the embodiment can also determine whether the blocking material is an authorized blocking type, for example, in some cases, the material is allowed to be blocked by a specific type of blocking material, and at this time, no alarm is needed. Therefore, in the embodiment, the type of the blocking material at the position of the target material is determined based on the material segmentation result, and it is determined whether the type of the blocking material is an authorized blocking type of the target material.

[0136] In an embodiment, generally, the blocking material can have two types, one is a vehicle type, and one is a human type. When performing the blocking determination, it is first determined whether the blocking material exists, and if yes, the type of the blocking material is determined.

[0137] If the type of the blocking material is the vehicle type, and the target material does not have an authorized blocking type, or the authorized blocking type is the human type, an alarm is generated.

[0138] If the type of the blocking material is the human type, and the target material does not have an authorized blocking type, or the authorized blocking type is the vehicle type, an alarm is generated.

[0139] Based on the above embodiment, the present application performs image segmentation on the material in the monitoring video in an artificial intelligence manner, and monitors the material according to the set process based on the segmented material image, thereby improving the accuracy and efficiency of the construction site material monitoring, and avoiding the problems of high labor cost and poor monitoring effect caused by manual monitoring.

[0140] The following will be described in combination with Figure 2 , Figure 2 The structure block diagram of the construction site material intelligent monitoring device provided by the embodiment of the present application can include:

[0141] The first module 100 is configured to set an image acquisition device for monitoring a target area of a construction site, acquire monitoring video data of the image acquisition device, and determine a target frame image from the monitoring video data every preset time period.

[0142] The second module 200 is configured to set a target material to be monitored, input the target frame image into a material image segmentation model, and obtain a material segmentation result.

[0143] The third module 300 is configured to perform authorization judgment processing when the target material does not exist in the material segmentation result, and determine a center point position of a target material segmentation result in the material segmentation result when the target material exists in the material segmentation result.

[0144] The fourth module 400 is configured to perform the authorization judgment processing when the offset of the center point position is greater than or equal to a first preset threshold, and determine a mask of the target material segmentation result when the offset of the center point position is less than the first preset threshold.

[0145] The fifth module 500 is configured to perform the authorization judgment processing when the offset of the mask is greater than or equal to a second preset threshold.

[0146] Based on the above embodiments, the material in the monitoring video is segmented by using artificial intelligence, and the material is monitored according to a set process based on the segmented material image, thereby improving the accuracy and efficiency of the construction site material monitoring, and avoiding the problems of high labor cost and poor monitoring effect caused by manual monitoring.

[0147] Based on the above embodiments, the first module 100 can include:

[0148] The first unit is configured to convert all frame images of the monitoring video data in the preset time period into grayscale images.

[0149] The second unit is configured to determine a Laplacian value of each pixel point in the grayscale image, and determine a Laplacian mean value of the grayscale image based on the Laplacian value.

[0150] The third unit is configured to determine a Laplacian variance of the grayscale image based on the Laplacian mean value and the Laplacian value.

[0151] The fourth unit is configured to determine the grayscale image with the maximum Laplacian variance as a target grayscale image, and determine the frame image converted into the target grayscale image as the target frame image.

[0152] Based on the above embodiments, the offset of the center point position is determined by the center point position of the target material segmentation result in the current preset time period, the center point positions of the target material segmentation results in the first preset number of preset time periods, and the center point positions of the target material segmentation results in the second preset number of preset time periods.

[0153] The offset of the mask is determined by a set of mask center points of the target material segmentation result in a current preset time period and a set of mask center points of the target material segmentation result in a previous third preset number of preset time periods.

[0154] Based on the above embodiments, the device can further include:

[0155] The sixth module is configured to, when a new material sample set is received, perform data augmentation on the new material sample set based on light intensity simulation and light angle simulation to obtain an augmented sample set.

[0156] The seventh module is configured to perform incremental learning training on the material image segmentation model based on the augmented sample set to obtain a trained material image segmentation model.

[0157] Based on the above embodiments, the authorization judgment process includes:

[0158] The fifth unit is configured to determine whether the current change time is within a preset authorization time and / or whether a preset identifier exists in the target frame image; if yes, the sixth unit is executed; if no, the seventh unit is executed.

[0159] The sixth unit is configured to determine that the change of the target material is an authorized change, save change video data of the target material from the monitoring video data, and cancel monitoring of the target material.

[0160] The seventh unit is configured to generate an alarm alert, save the change video data of the target material from the monitoring video data, and send the change video data to a supervisor.

[0161] Based on the above embodiments, the device can further include:

[0162] The eighth module is configured to, when it is determined based on the material segmentation result that a position of the target material exists an occlusion, determine whether the target material is a occludable material; if yes, the ninth module is executed; if no, the tenth module is executed.

[0163] The ninth module is configured to determine an occlusion time of the occlusion; if the occlusion time exceeds a preset occlusion time, an alarm alert is generated.

[0164] The tenth module is configured to generate the alarm alert.

[0165] Based on the above embodiments, the eighth module can include:

[0166] An eighth unit is configured to determine a type of the occlusion object on the target material position based on the material segmentation result, and determine whether the type of the occlusion object is an authorized occlusion type of the target material.

[0167] Based on the above embodiments, the present application further provides an electronic device, which can include a memory and a processor, wherein the memory has a computer program stored therein, and the processor can implement the steps provided by the above embodiments when invoking the computer program in the memory. Of course, the device can further include various necessary network interfaces, power supplies and other components.

[0168] The present application further provides a computer readable storage medium having a computer program stored thereon, and the computer program can implement the method provided by the embodiments of the present application when executed by a terminal or a processor. The storage medium can include a U disk, a mobile hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various storage medium capable of storing program codes.

[0169] In this document, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

Claims

1. A method for intelligent monitoring of construction site materials, characterized in that, include: Set up an image acquisition device for monitoring the target area of ​​the construction site, acquire the monitoring video data of the image acquisition device, and determine the target frame image from the monitoring video data at preset time intervals; Set the target material to be monitored, input the target frame image into the material image segmentation model, and obtain the material segmentation result; If the target material is not present in the material segmentation result, an authorization judgment is performed; if the target material is present in the material segmentation result, the center point position of the target material segmentation result is determined. When the offset of the center point is greater than or equal to the first preset threshold, the authorization judgment process is performed. When the offset of the center point is less than the first preset threshold, the mask of the target material segmentation result is determined. When the offset of the mask is greater than or equal to the second preset threshold, the authorization judgment process is performed.

2. The intelligent monitoring method for construction site materials according to claim 1, characterized in that, Determining target frame images from the monitoring video data at preset time intervals includes: Convert all frames of the surveillance video data within the preset time period into grayscale images; Determine the Laplacian value of each pixel in the grayscale image, and determine the Laplacian mean of the grayscale image based on the Laplacian value; The Laplacian variance of the grayscale image is determined based on the Laplacian mean and the Laplacian value. The grayscale image with the largest Laplacian variance is determined as the target grayscale image, and the frame image converted to the target grayscale image is determined as the target frame image.

3. The intelligent monitoring method for construction site materials according to claim 1, characterized in that, The offset of the center point position is determined by the center point position of the target material segmentation result in the current preset time period, the center point position of the target material segmentation result in the first preset number of preset time periods, and the center point position of the target material segmentation result in the second preset number of preset time periods. The offset of the mask is determined by the set of mask midpoints of the target material segmentation results within the current preset time period and the set of mask midpoints of the target material segmentation results within the previous third preset number of preset time periods.

4. The intelligent monitoring method for construction site materials according to claim 1, characterized in that, Also includes: When a new material sample set is received, the data of the new material sample set is amplified based on the simulation of light intensity and light angle to obtain an amplified sample set. The material image segmentation model is incrementally trained based on the amplified sample set to obtain the trained material image segmentation model.

5. The intelligent monitoring method for construction site materials according to claim 1, characterized in that, The authorization determination process includes: Determine whether the current change time is within a preset authorized time period, and / or whether a preset identifier exists in the target frame image; If so, determine that the change in the target material is an authorized change, save the change video data of the target material from the monitoring video data, and cancel the monitoring of the target material; If not, generate an alarm, save the change video data of the target material from the monitoring video data, and send the change video data to the supervisor.

6. The intelligent monitoring method for construction site materials according to claim 1, characterized in that, Also includes: When it is determined from the material segmentation results that there is an obstruction at the location of the target material, it is determined whether the target material is an obstructable material; If so, then determine the duration of the obstruction. If the occlusion time exceeds the preset occlusion time, an alarm will be generated; If not, then generate the aforementioned alarm.

7. The intelligent monitoring method for construction site materials according to claim 6, characterized in that, Determining whether the target material is an obstructable material includes: Based on the material segmentation results, determine the type of the obstruction at the target material location, and determine whether the type of the obstruction is an authorized obstruction type for the target material.

8. A smart monitoring device for construction site materials, characterized in that, include: The first module is used to set up an image acquisition device for monitoring the target area of ​​the construction site, acquire the monitoring video data of the image acquisition device, and determine the target frame image from the monitoring video data at preset time intervals. The second module is used to set the target material to be monitored, input the target frame image into the material image segmentation model, and obtain the material segmentation result. The third module is used to perform authorization judgment processing when the target material is not found in the material segmentation result; When the target material exists in the material segmentation result, determine the center point position of the target material segmentation result in the material segmentation result; The fourth module is used to perform the authorization judgment process when the offset of the center point position is greater than or equal to the first preset threshold. When the offset of the center point is less than the first preset threshold, the mask of the target material segmentation result is determined. The fifth module is used to perform the authorization judgment process when the offset of the mask is greater than or equal to the second preset threshold.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor is configured to implement the intelligent monitoring method for construction site materials as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the intelligent monitoring method for construction site materials as described in any one of claims 1 to 7.

Images