A Construction Scene Monitoring Method, Device, Equipment and Storage Medium

Abstract

A construction scenario monitoring method, device, equipment and storage medium provided by an embodiment of the present application belong to the field of computer technology. The method includes: collecting a first construction image and a second construction image in real time; inputting the first construction image and the second construction image into a pre-trained convolutional neural network model to output a target image; determining the construction actions and / or motion postures of the construction objects in the target image; determining construction quality according to the construction actions; and / or predicting the construction safety of the construction objects according to the motion postures. The present application can enable no fixed personnel to be arranged for supervision during building construction, effectively reducing labor costs; in addition, real-time monitoring can be achieved through image processing, reducing the probability of construction safety accidents and low construction quality caused by human factors, effectively improving construction quality and ensuring construction safety.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a construction scene monitoring method, device, equipment, and storage medium. Background Art

[0002] Currently, during building construction, an artificial supervision mechanism or a personal self-awareness mechanism is often used to control construction safety and quality.

[0003] However, whether through an artificial supervision mechanism or a personal self-awareness mechanism, accidents often occur during construction, and it is relatively common for construction quality problems to exist due to human factors.

[0004] Therefore, how to solve the above problems is an urgent problem to be solved currently. Summary of the Invention

[0005] This application provides a construction scene monitoring method, device, equipment, and storage medium, aiming to improve the above problems.

[0006] In a first aspect, a construction scene monitoring method provided by this application includes:

[0007] Real-time collect a first construction image and a second construction image;

[0008] Input the first construction image and the second construction image into a pre-trained convolutional neural network model to output a target image;

[0009] Determine the construction actions and / or movement postures of the construction objects in the target image;

[0010] Determine the construction quality according to the construction actions; and / or,

[0011] Predict the construction safety of the construction objects according to the movement postures.

[0012] In a possible embodiment, the size specification of the first construction image is smaller than that of the second construction image.

[0013] In a possible embodiment, inputting the first construction image and the second construction image into a pre-trained convolutional neural network model to output a target image includes:

[0014] Respectively determine the target image positions in the first construction image and the second construction image;

[0015] According to a preset cropping size, with the target image positions as the centers, crop out a first target image corresponding to the first construction image and a second target image corresponding to the second construction image;

[0016] Input the first target image and the second target image into a pre-trained convolutional neural network model to output a target image.

[0017] In a possible embodiment, the pre-trained convolutional neural network model includes 5 convolutional layers and 3 fully connected layers.

[0018] In a possible embodiment, inputting the first target image and the second target image into a pre-trained convolutional neural network model to output a target image includes:

[0019] Use 5 of the convolutional layers to extract the first convolutional feature and the second convolutional feature corresponding to the first target image and the second target image respectively;

[0020] Based on cross-correlation operation, correlate the first convolutional feature and the second convolutional feature to generate a third feature;

[0021] Input the third feature into 3 of the fully connected layers to output a target image.

[0022] In a possible embodiment, determining construction quality according to the construction action includes:

[0023] Obtain construction force data matching the construction action;

[0024] Obtain the construction quality corresponding to the construction force data.

[0025] In a possible embodiment, the method further includes:

[0026] When it is determined that the construction quality and / or the construction safety of the construction object do not meet the preset conditions, send an alarm message to the construction object and the monitoring center.

[0027] In a second aspect, the present application further provides a construction scenario monitoring device, and the device includes:

[0028] A real-time acquisition module, configured to acquire a first construction image and a second construction image in real time;

[0029] A first processing module, configured to input the first construction image and the second construction image into a pre-trained convolutional neural network model to output a target image;

[0030] A second processing module, configured to determine the construction action and / or motion posture of the construction object in the target image;

[0031] A third processing module, configured to determine construction quality according to the construction action; and / or,

[0032] A fourth processing module, configured to predict the construction safety of the construction object according to the motion posture.

[0033] In a third aspect, the present application further provides an electronic device, including:

[0034] a memory for storing executable instructions;

[0035] a processor for implementing the construction scene monitoring method according to any one of the first aspects when executing the executable instructions stored in the memory.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processing device, it executes the steps of the construction scene monitoring method according to any one of the first aspects.

[0037] For the construction scene monitoring method, device, equipment and storage medium provided by the present application, the present application collects the first construction image and the second construction image in real time, and uses a pre-trained convolutional neural network model to process the first construction image and the second construction image, and outputs a target image, so as to determine the construction actions and / or motion postures of the construction objects based on the target image; determine the construction quality according to the construction actions; and / or predict the construction safety of the construction objects according to the motion postures. Therefore, when carrying out building construction, there is no need to arrange fixed personnel for supervision, which can effectively reduce labor costs; in addition, real-time monitoring can be achieved through image processing, reducing the probability of construction safety accidents and low construction quality caused by human factors, effectively improving construction quality and ensuring construction safety. The present application can better improve the efficiency of supervision. Staff can quickly detect potential safety hazards, strengthen the quality and progress control of the construction process, and greatly save manpower and material resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0039] Figure 1 It is a schematic structural diagram of an electronic device provided by the first embodiment of the present application;

[0040] Figure 2 It is a flowchart of a construction scene monitoring method provided by the second embodiment of the present application;

[0041] Figure 3 For Figure 2 a schematic diagram of a convolutional neural network model in a construction scene monitoring method shown.

[0042] Figure 4 This is a schematic diagram of the functional modules of a construction site monitoring device provided in the third embodiment of the present application. Detailed implementation manners

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings. Apparently, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without making creative efforts shall fall within the protection scope of the present application.

[0044] The first embodiment

[0045] Figure 1 This is a schematic diagram of the structure of an electronic device provided in the embodiments of the present application. In the present application, an Figure 1 illustrated schematic diagram is used to describe an electronic device 100 for implementing an example of the construction site monitoring method and device of the embodiments of the present application.

[0046] As Figure 1 shown in the schematic diagram of the structure of an electronic device, the electronic device 100 includes one or more processors 102, one or more storage devices 104, and an image acquisition device 106. These components are interconnected through a bus system and / or other forms of connection mechanisms (not shown). It should be noted that Figure 1 the components and structure of the illustrated electronic device 100 are exemplary, not restrictive. According to needs, the electronic device may have Figure 1 some of the components shown, or Figure 1 other components and structures not shown.

[0047] The processor 102 may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.

[0048] It should be understood that the processor 102 in the embodiments of the present application may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0049] The storage device 104 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media.

[0050] It should be understood that the storage device 104 in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM), and direct rambus random access memory (DR RAM).

[0051] Among them, one or more computer program instructions can be stored on the computer-readable storage medium, and the processor 102 can run the program instructions to implement the client functions (implemented by the processor) in the embodiments of the present application described below and / or other desired functions.

[0052] The image acquisition device 106 is used to acquire construction images in real time. For example, the image acquisition device 106 can be a camera, such as a monocular camera or a binocular camera.

[0053] Second Embodiment:

[0054] Referring to Figure 2 the flowchart of a construction site monitoring method shown, the method specifically includes the following steps:

[0055] Step S201, acquire the first construction image and the second construction image in real time.

[0056] Optionally, the acquisition of the first construction image and the second construction image can be performed by a camera installed at the construction site, or can be acquired by a drone. Here, no specific limitation is made.

[0057] In one embodiment, the size specification of the first construction image is smaller than that of the second construction image.

[0058] For example, assume that the size specification of the first construction image is: 127*127*3. The size specification of the second construction image is: 227*227*3.

[0059] Step S202, input the first construction image and the second construction image into a pre-trained convolutional neural network model to output a target image.

[0060] Optionally, the pre-trained convolutional neural network model includes 5 convolutional layers and 3 fully connected layers.

[0061] As an implementation manner, step S202 includes: respectively determining the target image positions in the first construction image and the second construction image; cropping out a first target image corresponding to the first construction image and a second target image corresponding to the second construction image with the target image positions as the centers according to a preset cropping size; inputting the first target image and the second target image into the pre-trained convolutional neural network model to output a target image.

[0062] It should be understood that the target image position refers to the area in the construction image that includes the construction object and / or construction equipment.

[0063] Optionally, the preset cropping size can be set according to actual needs. Here, no specific limitation is made.

[0064] Continuing with the above example, assuming the preset cropping size is w*h, then centered on the target image position according to the preset cropping size, a first target image corresponding to the first construction image and a second target image corresponding to the second construction image are cropped: namely, an image with a size of 2w*2h is cropped centered on the target image position in the first construction image. Similarly, an image with a size of 2w*2h is cropped centered on the target image position in the second construction image. Finally, the two 2w*2h images are input into a pre-trained convolutional neural network model for convolution to extract target features, so as to output the target image.

[0065] In the above implementation process, by processing the two construction images through a pre-trained convolutional neural network model, the accuracy of image processing can be effectively improved, so as to improve the monitoring efficiency of the construction quality and / or construction safety at the construction site, to ensure the construction safety of construction personnel and improve the construction quality.

[0066] Optionally, inputting the first target image and the second target image into a pre-trained convolutional neural network model to output the target image includes: using 5 convolutional layers to extract first convolutional features and second convolutional features corresponding to the first target image and the second target image respectively; correlating the first convolutional features and the second convolutional features based on cross-correlation operation to generate a third feature; inputting the third feature into 3 fully connected layers to output the target image.

[0067] For example, as Figure 3 shown, Layer1 (Convolutional Layer 1): The convolutional layer sequentially performs convolutional feature extraction on two inputs. First, a convolutional kernel of 11*11*96 is used for convolution, the convolution stride is 4, the ReLU function is used as the activation function, and max pooling operation is performed. Then, a kernel of 3*3 with a stride of 2 is used, and after regularization, it is input into Layer2.

[0068] Layer2 (Convolutional Layer 2): A convolutional kernel of 5*5*256 is used for convolution, 2 rows and 2 columns are padded at the edges, the convolution stride is 2, the ReLU activation function is used, max pooling is performed, a kernel of 3*3 is used, the stride is 2, and after regularization, it is input into Layer3.

[0069] Layer3 (Convolutional Layer 3): The convolutional kernel is 3*3*384, the activation function is ReLU, and it is output to Convolutional Layer 4.

[0070] Layer4 (Convolutional Layer 4): The convolutional kernel is 3*3*384, the activation function is ReLU, and it is sequentially output to the next layer.

[0071] Layer 5 (Convolutional Layer 5): The convolutional kernel is 3*3*256. One row and one column are added to supplement the edges for convolution operation. The activation function is ReLU, and max pooling is performed with a 3*3 kernel and a stride of 2.

[0072] Then, the data obtained by convolving the two inputs are subjected to cross-correlation operation together. Next, the data after the cross-correlation operation are input into the fully connected layers: Fc-layer6 outputs 4D image data, the activation function is ReLU, and the drop_out_ratio is 0.5. Fc-layer7 outputs 4D image data, the activation function is ReLU, and the drop_out_ratio is 0.5. Fc-layer8 outputs 4D image data (the coordinate positions of the upper left and lower right corners of the target box).

[0073] It should be understood that the above examples are only illustrative and not restrictive.

[0074] Step S203, determine the construction actions and / or motion postures of the construction objects in the target image.

[0075] Step S204, determine the construction quality according to the construction actions.

[0076] As an implementation manner, step S204 includes: obtaining the construction force data matching the construction actions; obtaining the construction quality corresponding to the construction force data.

[0077] Optionally, the construction force data can be pre-stored.

[0078] It should be understood that if the construction force data is pre-stored, it must be historical construction force data obtained based on different construction actions.

[0079] Similarly, the construction quality corresponding to the construction force data can also be pre-stored. That is, a relationship table is established in advance for the different construction qualities obtained from different construction force data, so as to quickly confirm the construction quality according to the construction force data and improve the supervision of the construction quality.

[0080] And / or, step S205, predict the construction safety of the construction objects according to the motion postures.

[0081] In a possible embodiment, the construction scene monitoring method further includes: when it is determined that the construction quality and / or the construction safety of the construction objects do not meet the preset conditions, sending an alarm message to the construction objects and the monitoring center.

[0082] It should be understood that each construction object (i.e., construction personnel) carries an alarm terminal. After receiving the alarm information, the alarm terminal will give an alarm to warn the construction personnel, reducing the probability of safety accidents. Or when there are construction quality problems, it warns the construction personnel to increase or decrease the construction intensity to achieve better construction quality.

[0083] Among them, the preset conditions can be set according to actual needs. For example, the construction quality meets the acceptance standards. Or the construction quality is set to multiple levels, and the preset condition is a certain level, such as level 5, that is, the condition is met only when the construction quality reaches level 5.

[0084] It can be understood that this application will send different forms of alarm information according to construction quality and construction safety respectively. When there are construction quality problems, a signal flashing indication is sent to make the terminal device bound to the construction object flash the corresponding color, such as flashing yellow. When there are construction safety problems, a voice alarm is sent to make the terminal device bound to the construction object emit a voice alarm, such as: Pay attention to construction safety / There is danger ahead, etc. For another example, if the identity can be correctly recognized at this time, the name will be broadcast and the behavior will be recorded, which is convenient for later assessment and strengthening the awareness of safety risk prevention for individual personnel.

[0085] Third Embodiment:

[0086] Refer to Figure 4 A construction scene monitoring device shown. The construction scene monitoring device 500 includes: a real-time acquisition module 510, a first processing module 520, a second processing module 530, a third processing module 540, and / or a fourth processing module 550.

[0087] Among them, the real-time acquisition module 510 is used to acquire the first construction image and the second construction image in real time;

[0088] The first processing module 520 is used to input the first construction image and the second construction image into a pre-trained convolutional neural network model and output a target image;

[0089] The second processing module 530 is used to determine the construction actions and / or movement postures of the construction objects in the target image;

[0090] The third processing module 540 is used to determine the construction quality according to the construction actions; and / or,

[0091] The fourth processing module 550 is used to predict the construction safety of the construction object according to the movement posture.

[0092] It should be noted that for the specific functions of the construction scene monitoring device 500 provided in this embodiment, please refer to the description of the method embodiment, and details are not described herein again.

[0093] Further, this embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processing device, it executes the steps of any one of the construction scenario monitoring methods provided in the above embodiments.

[0094] A computer program product of a construction scenario monitoring method and device provided in an embodiment of the present application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, reference can be made to the method embodiments and will not be elaborated herein.

[0095] It should be noted that the above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware, or any arbitrary combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0096] It should be understood that the term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship. Specifically, it can be understood by referring to the context before and after.

[0097] In this application, "at least one" means one or more, and "a plurality of" means two or more. "At least one of the following" or a similar expression means any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0098] It should be understood that in various embodiments of this application, the sequence numbers of the above processes do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this application.

[0099] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0100] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0101] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in an electrical, mechanical, or other form.

[0102] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0103] In addition, in each embodiment of the present application, each functional unit may be integrated into a processing unit, may exist separately as individual physical units, or two or more units may be integrated into one unit.

[0104] The above are only the preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application. It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

Claims

1. A method for monitoring a construction scenario, characterized in that, the method includes: Real-time collecting a first construction image and a second construction image; Inputting the first construction image and the second construction image into a pre-trained convolutional neural network model to output a target image, and the pre-trained convolutional neural network model realizes the recognition and positioning marking of target features by outputting the coordinates (x, y) of feature points on the output image; Determining the construction actions and movement postures of the construction objects in the target image; Determining the construction quality according to the construction actions; and, Predicting the construction safety of the construction objects according to the movement postures; Wherein, inputting the first construction image and the second construction image into a pre-trained convolutional neural network model to output a target image includes: Respectively determining the target image positions in the first construction image and the second construction image; Cutting out a first target image corresponding to the first construction image and a second target image corresponding to the second construction image centered on the target image position according to a preset cropping size; Inputting the first target image and the second target image into a pre-trained convolutional neural network model to output a target image; the pre-trained convolutional neural network model includes 5 convolutional layers and 3 fully connected layers; Inputting the first target image and the second target image into a pre-trained convolutional neural network model to output a target image includes: Using the 5 convolutional layers to extract first convolutional features and second convolutional features corresponding to the first target image and the second target image respectively; Associating the first convolutional feature and the second convolutional feature based on cross-correlation operation to generate a third feature; Inputting the third feature into the 3 fully connected layers to output a target image.

2. The method according to claim 1, characterized in that, To ensure the continuity of its actions, the first construction image and the second construction image are continuous images, and the sizes of the selected images are different according to the distance from the acquisition point.

3. The method according to claim 1, characterized in that, Determining the construction quality according to the construction actions includes: Obtaining construction force data matching the construction actions; Obtaining the construction quality corresponding to the construction force data.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: When it is determined that the construction quality and the construction safety of the construction objects do not meet the preset conditions, sending an alarm message to the construction objects and the monitoring center.

5. A construction scenario monitoring device, characterized in that, the device includes: A real-time collection module for real-time collecting a first construction image and a second construction image; A first processing module for inputting the first construction image and the second construction image into a pre-trained convolutional neural network model to output a target image; A second processing module for determining the construction actions and movement postures of the construction objects in the target image; A third processing module for determining the construction quality according to the construction actions; and, A fourth processing module for predicting the construction safety of the construction objects according to the movement postures; The first processing module is specifically used for: Determine the target image positions in the first construction image and the second construction image respectively; Centered on the target image positions, crop out a first target image corresponding to the first construction image and a second target image corresponding to the second construction image according to a preset cropping size; Input the first target image and the second target image into a pre-trained convolutional neural network model to output a target image; the pre-trained convolutional neural network model includes 5 convolutional layers and 3 fully connected layers; Inputting the first target image and the second target image into the pre-trained convolutional neural network model to output a target image includes: Use 5 of the convolutional layers to extract first convolutional features and second convolutional features corresponding to the first target image and the second target image respectively; Based on the cross-correlation operation, correlate the first convolutional feature and the second convolutional feature to generate a third feature; Input the third feature into 3 of the fully connected layers to output a target image.

6. An electronic device, Characterized in that, Comprising: A memory for storing executable instructions; A processor, when executing the executable instructions stored in the memory, implements the construction scene monitoring method according to any one of claims 1 to 4.

7. A computer-readable storage medium, Characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is run by a processing device, it executes the construction scene monitoring method according to any one of claims 1-4.

Images