An online construction site robot monitoring management method and system

Abstract

The application relates to the field of construction monitoring, and discloses an online construction site robot monitoring management method and system, which measures and models the construction site and the construction process at any time, compares the construction site and the construction process with construction drawings, realizes monitoring of the construction and rapid prompting of abnormal conditions, discovers the abnormal conditions in the construction process in time, rapidly prompts rectification, greatly avoids huge losses caused by construction failures and untimely detection, can work in an environment with people and equipment (can automatically remove noises such as people and facilities), constructs a visual 3D scene graph and extracts key information such as planes and straight lines, realizes automatic measurement, and is more convenient and efficient to use.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of construction monitoring, and particularly relates to an online construction site robot monitoring management method and system. BACKGROUND

[0002] Building construction quality supervision is a necessary link in the construction process, which affects and determines the construction progress and quality to a certain extent. Comparing the consistency of the construction situation and the drawings is one of the important ways of building construction quality monitoring.

[0003] At present, the consistency of the construction situation and the drawings is mainly realized by the on-site operation of functional personnel such as on-site supervisors and quality management personnel, for example, the distance, angle and other data of the key position are measured on site by measuring tape, range finder and other measuring tools, and then the results are obtained by manually comparing the building construction drawings and construction requirements. Obviously, this on-site monitoring method is affected by the on-site construction conditions and the quality of the functional personnel themselves, and the efficiency is not high, and multiple on-site measurements will also affect the construction progress. In the building construction process, if the construction drawings and the actual construction situation can be compared and monitored in real time, it will help to improve the efficiency and reliability of construction supervision.

[0004] In the prior art, a model matching method for building drawings is provided in application No. CN2021104819089, the implementation principle of which is as follows: obtaining the building drawings; calibrating the building drawings to obtain first picture data; using a preset drawing recognition model to recognize and calibrate the building drawings to obtain second picture data; comparing the first picture data with the second picture data to determine the recognition degree of the building drawings by the preset drawing recognition model; which can be used to guide the correction of the preset drawing recognition model, thereby helping to improve the efficiency of building drawing recognition using the preset drawing recognition model, but it only matches the templates of drawings and drawings, so the intelligent technology still needs to be developed. SUMMARY

[0005] The present application aims to provide an online construction site robot monitoring management method and system to solve the problems raised in the background art.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0007] An online construction site robot monitoring management method, comprising the following steps:

[0008] The collection of field data is based on the preset sensing measurement device to collect real-time image data Ii and laser point cloud data Li of the construction site, and to establish live data {Pi, Li, Ii} based on the observation pose data Pi of the sensing measurement device, wherein the sensing measurement device performs data acquisition along a preset collection operation route;

[0009] The field model reconstruction based on live data is to analyze the multi-frame noise filtering of the laser point cloud data Li according to the image data Ii, to obtain denoised laser point cloud Li', and to perform splicing reconstruction on the denoised laser point cloud Li' based on the observation pose data Pi, thereby generating a construction site model M;

[0010] The extraction of key feature data is to perform point cloud plane fitting on the construction site model M, to extract key planes, and to calculate key feature data based on a plurality of the key planes, wherein the feature data includes: building surface data {(pij, sj)}, angle data {theta ij} between building surfaces, and straight line equations {l ij};

[0011] The mapping comparison based on the drawing engineering mapping is to map and convert corresponding point positions of the construction site model M and the corresponding CAD drawing model to obtain a corresponding mapping relationship, and to compare the construction site model M based on the corresponding mapping relationship to realize building construction supervision.

[0012] As a further scheme of the present application, the step of obtaining denoised laser point cloud by analyzing the multi-frame noise filtering of the laser point cloud data according to the image data specifically includes:

[0013] Live data {Pi, Li, Ii} at time i is obtained, a preset deep learning human and object monitoring algorithm is used to perform object recognition on the image data Ii in the live data, and an image region ROI is obtained;

[0014] Based on the corresponding matching relationship between the image data Ii and the laser point cloud data Li, the corresponding part of the point cloud data in the laser point cloud data Li is removed;

[0015] The remaining laser point cloud data Li is processed based on a distance-based clustering algorithm to filter out independent discrete points, thereby obtaining denoised laser point cloud Li'.

[0016] As a further scheme of the present application, the step of performing point cloud plane fitting on the construction site model M, extracting key planes, and calculating key feature data based on a plurality of the key planes specifically includes:

[0017] Based on the construction site model M, the point cloud plane is fitted by a random sampling consistent fitting plane algorithm, key planes in the model are extracted, each plane equation of the building and the corresponding point set are obtained, and the building surface data is represented as:

[0018] {(pij, Sj)}

[0019] Wherein, pij represents the plane equation of the building surface with serial number j, and Sj represents the point set of the building surface with serial number j;

[0020] Based on the building surface data of non-intersecting building surfaces, the intersecting straight line equation {li ij} and the angle data {theta ij} between the building surfaces are calculated, and the angle data {theta ij} is represented as:

[0021] theta ij = arccos (ni, nj)

[0022] Wherein, ni is the normal vector of pi, nj is the normal vector of pj, and ij represents the serial number of the building surface, and the intersecting straight line equation {li ij} is obtained by simultaneously solving the pi and pj equations.

[0023] As a further scheme of the application: the step of mapping and converting the construction site model M and the corresponding CAD drawing model to obtain the corresponding mapping relationship Y includes:

[0024] A group of feature points on three non-parallel lines are selected from the construction site model M, and corresponding feature matching points are selected on the CAD drawing model;

[0025] Based on the rigid rotation principle, the rotation matrix R and the translation matrix T between the CAD drawing model and the construction site model M are established, and the corresponding mapping relationship is obtained by matrix calculation, which can be represented as:

[0026] Y = X * R + T

[0027] Wherein, X represents any design point in the CAD drawing model, Y represents the corresponding point coordinates to be solved in the construction operation surface based on the corresponding relationship of the site construction model, and the Y can be used for verifying the actual collected data in the site construction model M.

[0028] As a further scheme of the application: in the step of comparing the construction site model M based on the corresponding mapping relationship to realize building construction supervision, specifically including:

[0029] Based on the angle data {theta ij} between the building surfaces measured in the construction site model} through the mapping relationship and the angle between the respective corresponding building surfaces in the CAD drawing model, when the deviation value is too large, real-time output warning;

[0030] Based on the measured straight line equation{l ij} between the intersecting lines in the construction site model

[0031] Based on the measured straight line equation{l ij} between the intersecting lines in the construction site model ij} through the mapping relationship and the angle between the respective corresponding building surfaces in the CAD drawing model, when the deviation value is too large, real-time output warning;

[0032] Based on the measured straight line equation{l ij} between the intersecting lines in the construction site model

[0033] As a further scheme of the present application: the sensing and measuring device executes the data acquisition step in a preset acquisition operation route;

[0034] When the sensing and measuring device moves a distance di along the x direction at time i, the observation pose data Pi of the sensing and measuring device at time i can be represented as:

[0035] Pi={xi,yi,zi,ai,bi,gi}={di,0,0,0,0,0}

[0036] Where xi, yi, zi, ai, bi, gi correspond to the x, y, z coordinates of the sensing and measuring device at time i, respectively, as well as the pitch, roll and roll angles, and the coordinate system takes the starting point of the initial collection as the origin and the initial orientation of the initial collection to determine the coordinate axes.

[0037] The embodiment of the present application aims to provide an online construction site robot monitoring and management system, comprising:

[0038] The data acquisition module is used to collect the field data, based on the preset sensing and measuring device to collect the real-time image data Ii and the laser point cloud data Li of the construction site, and based on the observation pose data Pi of the sensing and measuring device to establish the live data {Pi, Li, Ii}, the sensing and measuring device executes the data acquisition in a preset acquisition operation route;

[0039] The model establishing module is configured to perform live model reconstruction based on live data, analyze multiple frames of noise filtering of the laser point cloud data Li according to the image data Ii, obtain denoised laser point cloud Li', and perform splicing reconstruction on the denoised laser point cloud Li' based on the observation pose data Pi, thereby generating a construction site model M.

[0040] The feature extraction module is configured to perform extraction of key feature data, perform point cloud plane fitting on the construction site model M, extract key planes, and calculate key feature data based on a plurality of the key planes, wherein the feature data includes building surface data {(pij, sj)}, angle data {theta ij} between building surfaces, and straight line equations {l ij}.

[0041] The online comparison module is configured to perform mapping comparison based on engineering drawings, map corresponding points of the construction site model M and a corresponding CAD drawing model to obtain a corresponding mapping relationship, and compare the construction site model M based on the corresponding mapping relationship to achieve construction supervision.

[0042] As a further scheme of the present application, the model establishing module includes a noise processing unit and a model establishing unit, and the noise processing unit includes:

[0043] A noise marking subunit is configured to obtain live data {Pi, Li, Ii} at time i, perform object recognition of workers and equipment based on a preset deep learning human and object monitoring algorithm on image data Ii in the live data, and perform image region ROI.

[0044] A noise removal subunit is configured to remove corresponding part of point cloud data in the laser point cloud data Li corresponding to the image region ROI based on a corresponding matching relationship between the image data Ii and the laser point cloud data Li.

[0045] A discrete screening subunit is configured to process the remaining laser point cloud data Li based on a distance-based clustering algorithm to screen and remove independent discrete points, thereby obtaining denoised laser point cloud Li'.

[0046] Compared with the prior art, the beneficial effects of the present application are: by measuring and modeling the construction site and the construction process online at any time, and comparing with the construction drawings, the construction situation is monitored and the abnormal situation is quickly prompted, the abnormality in the construction process is found in time, and the rectification is quickly prompted, so that the huge loss caused by construction failure but not timely detection is greatly avoided; and it can work in an environment with people and equipment (can automatically exclude noise such as people and facilities), and a visual 3D scene graph is constructed and key information such as planes and straight lines is extracted, automatic measurement is realized, and it is more convenient and efficient to use. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 It is a flowchart of an online construction site robot monitoring management method.

[0048] Figure 2 It is a cooperation schematic view of a sensing and measuring device and a moving guide rail in an online construction site robot monitoring management system.

[0049] Figure 3 It is a composition block diagram of an online construction site robot monitoring management system. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0051] The specific implementation mode of the present application is described in detail below in combination with specific examples.

[0052] The present application discloses an online construction site robot monitoring management method and system, the system is a robot system, which is composed of a sensing and measuring device, a moving guide rail, a background operation processing device and the like. The moving guide rail is installed above the construction site, the sensing and measuring device can move on the moving guide rail to measure the image and laser data of the construction site, and the data is processed by the operation processing background. The operation processing background is also responsible for controlling the movement of the sensing and measuring device on the moving guide rail and determining its real-time position. In the operation processing of the background, the modeling of the construction site is first carried out through data acquisition, then the key data features (that is, the main data features required for construction monitoring and detection) are extracted, and finally the measured data is compared with the construction drawings online to find the abnormality in the construction process in real time. Specifically:

[0053] As Figure 1 The online construction site robot monitoring management method provided by one embodiment of the present application comprises the following steps:

[0054] S10, collection of field data, based on a preset sensing measurement device, real-time image data Ii and laser point cloud data Li of the construction site are collected, and live data {Pi, Li, Ii} is established based on observation pose data Pi of the sensing measurement device, the sensing measurement device performs data acquisition in a preset collection operation route, and the observation pose data Pi represents the position and pitch angle state when the sensing device collects data.

[0055] S20, reconstruction of the field model based on the live data, noise multi-frame filtering of the laser point cloud data Li is performed according to the image data Ii, denoising laser point cloud Li' is obtained, and the denoising laser point cloud Li' is spliced and reconstructed based on the observation pose data Pi, and a construction site model M is generated.

[0056] S30, extraction of key feature data, point cloud plane fitting is performed on the construction site model M, key planes are extracted, and key feature data is calculated based on a plurality of key planes, the feature data includes: building surface data {(pij, sj)}, angle data {theta ij} between building surfaces, and straight line equation {l ij}.

[0057] S40, based on the mapping comparison of the drawing engineering, the construction site model M and the corresponding CAD drawing model are mapped and converted to obtain a corresponding mapping relationship, and the construction site model M is compared based on the corresponding mapping relationship to realize building construction supervision.

[0058] In the embodiment, through data collection of the related robot system and the sensing measurement device, a real-time online detection and supervision method is realized, the construction site and the construction process can be measured and modeled at any time online, and compared with the construction drawings to realize monitoring of the construction situation and rapid prompt of abnormal situations, through timely discovery of abnormalities in the construction process and rapid prompt of rectification, great losses caused by construction errors but not timely detection are greatly avoided; and the device can work in an environment with people and equipment (noise such as people and facilities can be automatically removed), and a visual 3D scene graph and key information such as planes and straight lines are constructed to realize automatic measurement, which is more convenient and efficient.

[0059] As another preferred embodiment of the present application, the step of noise multi-frame filtering of the laser point cloud data based on image data analysis to obtain denoising laser point cloud specifically includes:

[0060] Obtain the real-time data {Pi,Li,Ii} at time i. Based on a preset deep learning human and object detection algorithm, perform object recognition of workers and implementing equipment on the image data Ii in the real-time data, and perform image region ROI.

[0061] Based on the correspondence between the image data Ii and the laser point cloud data Li, the point cloud data in the laser point cloud data Li that corresponds to the image region ROI is removed.

[0062] Distance-based clustering algorithms are used to process the remaining laser point cloud data Li to filter out independent discrete points and obtain denoised laser point cloud Li′.

[0063] In this embodiment, considering that the monitoring and detection are conducted online, there will be situations such as construction workers entering and leaving, and construction in progress during the monitoring and detection process. It is necessary to remove noise points, workers, construction equipment and other objects from the data. Therefore, it is necessary to use deep learning human body detection and object algorithm detection to identify and remove workers, pedestrians and related equipment and facilities. The image region ROI here can be understood as the labeling of objects such as workers.

[0064] In another preferred embodiment of the present invention, the steps of performing point cloud plane fitting on the construction site model M, extracting key planes, and calculating key feature data based on several key planes specifically include:

[0065] Based on the construction site model M, the point cloud plane is fitted using a random sampling consistent fitting plane algorithm to extract key planes from the model, obtain the equations of each building plane and the corresponding point set, and represent the building surface data as follows:

[0066] {(πj,Sj)}

[0067] Where πj represents the plane equation of the building surface with index j, and Sj represents the set of points of the building surface with index j.

[0068] Calculate the equations of intersecting straight lines based on the building surface data of non-intersecting building surfaces {l ij} and the angle data between building surfaces {θ ij}, included angle data {θ ij} is represented as:

[0069] θ ij =arccos(ni,nj)

[0070] Where ni is the normal vector of πi, nj is the normal vector of πj, and ij represents the building surface index. By solving the equations of πi and πj simultaneously, the equation of the intersecting line {l} is obtained. ij}

[0071] In this embodiment, the method of automatic feature extraction is given. First, based on the construction site model M, the key planes in the model are automatically extracted by point cloud plane fitting. For a conventional construction site, the ground / floor, main wall (generally vertical plane), large column plane, etc. in the model are automatically extracted in priority. The extraction method can adopt the random sampling consistent fitting plane algorithm. Next, based on different two-by-two intersecting building planes, the intersection straight line equation between them and the angle between the planes are calculated, which are important data features for monitoring construction judgment. The fitting extraction process can also be manually extracted.

[0072] Two key points can be directly selected in the 3D dense point cloud model to obtain the distance between them, that is, the first measurement point A (A x ,A y ,A z ) and the second measurement point B (B x ,B y ,B z ) are selected on the target to be measured, the coordinates of A and B in the three-dimensional point cloud scene are obtained, and the distance formula is used.

[0073]

[0074] The distance of the target to be measured can be obtained, and the related data can be compared with the length, width and other measurement values in the drawing.

[0075] As another preferred embodiment of the present application, the step of mapping and converting the construction site model M and the corresponding CAD drawing model to obtain the corresponding mapping relationship Y includes:

[0076] A group of feature points on three non-parallel lines are selected from the construction site model M, and corresponding feature matching points are selected on the CAD drawing model.

[0077] Based on the rigid body rotation principle, the rotation matrix R and the translation matrix T between the CAD drawing model and the construction site model M are established, and the corresponding mapping relationship is obtained by matrix calculation, which can be represented as:

[0078] Y=X*R+T

[0079] Where X represents any design point in the CAD drawing model, Y represents the corresponding point coordinates to be solved in the construction operation surface based on the corresponding relationship of the site construction model, and Y can be used to verify the actual data collected in the site construction model M.

[0080] In this embodiment, each construction operation surface has a corresponding CAD drawing model, and the comparison between the two needs to establish the coordinate mapping between the construction operation surface and the CAD drawing model. According to the three-dimensional point cloud scene model (i.e., the construction scene model M), three coordinate points not on the same straight line in the scene and the corresponding design points on the model drawing are obtained, and the mapping relationship between the three coordinate points and the corresponding points is established. First, the coordinate conversion matrix of the three coordinate points of the construction operation surface (here, the construction operation surface refers to the actual state under the ideal state, which corresponds to M, so the final deviation from M is the error generated in the construction) to the corresponding design points on the CAD drawing is obtained. According to the calculated coordinate conversion matrix, the points in the construction operation surface can be mapped to the corresponding CAD drawing model, thereby completing the corresponding relationship between the CAD drawing model and the construction operation surface.

[0081] The coordinate conversion matrix is obtained according to the rigid body rotation principle, which includes a rotation matrix and a translation matrix. First, select three points not on the same straight line in the construction operation surface, and set them as R1(x r1 ,y r1 ,z r1 ), R2(x r2 ,y r2 ,z r2 ), and R3(x r3 ,y r3 ,z r3 ), respectively. The corresponding points of the three points R1, R2, and R3 in the CAD drawing model are set as C1(x1, y1, 0), C1(x2, y2, 0), and C1(x3, y3, 0), respectively. The mapping relationship can be represented as:

[0082]

[0083] R represents the rotation matrix, and T represents the translation matrix. Through matrix calculation, the following can be obtained:

[0084]

[0085] If

[0086] then the rotation matrix can be represented as:

[0087]

[0088] The rotation matrix R and the translation matrix T between the CAD drawing model and the construction operation surface are established by the known three design point coordinates in the CAD drawing model and the coordinates of the corresponding three points in the construction operation surface. The mapping relationship between the CAD drawing model and the construction operation surface can be obtained through R and T, and the mapping relationship can be represented as:

[0089] Y = X * R + T

[0090] Wherein, X represents any design point in the CAD drawing model, Y represents the corresponding point coordinate to be solved in the construction operation surface, through the mapping relationship, any point in the construction operation surface can be mapped to the corresponding CAD drawing model, so as to complete the rapid comparison of the CAD drawing model and the construction operation surface, and the comparison mode can greatly reduce the workload of the construction site measurement, and quickly complete the building construction quality supervision.

[0091] As another preferred embodiment of the application, in the step of comparing the construction site model M based on the corresponding mapping relationship to realize building construction supervision, specifically comprising:

[0092] Based on the measured angle data {θ ij} between the building surfaces in the construction site model, the angle between the respective corresponding building surfaces in the CAD drawing model is compared through the mapping relationship, and when the deviation value is too large, the warning is output in real time.

[0093] Based on the measured straight line equation {l ij} intersecting each other between the building surfaces in the construction site model, the straight line equation between the respective corresponding building surfaces in the CAD drawing model is compared through the mapping relationship, and when the deviation value is too large, the warning is output in real time.

[0094] Based on the measured angle data {θ ij} between the straight line equations {l ij} intersecting each other in the construction site model, the angle data between the respective corresponding building surfaces in the CAD drawing model is compared through the mapping relationship, and when the deviation value is too large, the warning is output in real time.

[0095] Based on the line-surface distance between the measured straight line equation {l ij} and the building surface data {(πj, Sj)} in the construction site model, the angle data between the respective corresponding building surfaces in the CAD drawing model is compared through the mapping relationship, and when the deviation value is too large, the warning is output in real time.

[0096] As another preferred embodiment of the application, in the step of acquiring data by the sensing measurement device along the preset acquisition operation route;

[0097] When the sensing measurement device moves a distance di in the x direction at time i, the observation pose data Pi of the sensing measurement device at time i can be represented as:

[0098] Pi={xi, yi, zi, αi, βi, γi}={di, 0, 0, 0, 0, 0}

[0099] Wherein, xi, yi, zi, ai, bi, yi respectively correspond to the x, y, z coordinates of the sensing and measuring device at the i-th moment, and the pitch, roll and roll angles, and the coordinate system takes the starting point of the starting collection as the origin and the initial orientation of the starting collection to determine the coordinate axes.

[0100] In this embodiment, the setting of the related robot trolley for carrying the sensing and measuring device is involved, and the sensing and measuring device is composed of a camera, a 3D laser radar, and necessary data acquisition fusion and transmission modules, power supply and fixing devices, etc. The sensing and measuring device is fixed on the motor and the trolley, the latter is installed on the moving guide rail and can be controlled to slide accurately forward and backward on the moving guide rail. In specific implementation, the camera can be a high-precision RGB camera. Generally, the camera angle can be about 90 degrees, and the resolution is >4k, considering the detection angle range and detection distance. A plurality of cameras can also be combined to achieve the same. The 3D laser can use a multi-line laser radar with a detection range of 180-degree spherical surface (i.e., the distance point cloud data in the range of about 180 degrees in front can be obtained), and the effective measurement distance is generally 80 meters. The camera and the 3D laser need to be calibrated in advance to remove distortion and achieve basic registration of image pixels and laser points. In summary, the effective measurement range of the sensing and measuring device is the image + distance data in the range of about 90-degree angle in front, and the effective detection distance is 3-80 meters.

[0101] The moving guide rail is composed of a guide rail body, a motor and a trolley that can move accurately on the guide rail, and the sensing and measuring device is fixed on the motor and the trolley. The guide rail body, the motor and the trolley can use a stepper motor combined with a gear guide rail to achieve accurate displacement control and feedback. The positioning accuracy of the trolley on the guide rail can be higher than 0.1 millimeter, and the angle accuracy can be higher than 0.01 degree. The 3D laser radar and the camera in the sensing and measuring device have a basically overlapping observation range. The internal and external parameter calibration and registration between them can be achieved through a prior calibration program. Based on the registration result, the position / region of each laser point corresponding to the image point and the position / region of the image point corresponding to the laser point can be obtained.

[0102] After installation is completed, data collection on the construction site can be performed at any time. During data collection, the sensing and measuring device obtains image data and laser point cloud data in the current detection range in real time. At the same time, the trolley carrying the sensing and measuring device is controlled to move forward on the guide rail to collect data at different positions in sequence. The current collection position is determined by the guide rail and the motor. Specifically, taking a linear guide rail as an example: Figure 2

[0103] ​Taking the starting point of starting collection as the origin and the direction along the guide rail as the x direction; assuming that the moving distance of the trolley carrying the sensing and measuring device along the guide rail from the starting point position is di, then the position and attitude at this time are:

[0104] Pi={xi, yi, zi, ai, bi, yi}={di, 0, 0, 0, 0, 0}

[0105] Wherein, xi, yi, zi, ai, bi, yi respectively correspond to the x, y, z coordinates of the sensing and measuring device at the i time, and the pitch, roll and roll angles, the coordinate system takes the starting point of starting collection as the origin and the initial orientation of starting collection to determine the coordinate axis; by controlling the trolley carrying the sensing and measuring device to move on the guide rail, data collection is sequentially performed at different positions, multiple frames of data can be obtained, which can cover the entire construction area to be monitored.

[0106] As shown in Figure 2 , the application also provides an online construction site robot monitoring management system, which comprises:

[0107] A data acquisition module 100 is used for performing on-site data acquisition, collecting image data Ii and laser point cloud data Li of the construction site in real time based on a preset sensing and measuring device, and establishing live data {Pi, Li, Ii} based on the observation pose data Pi of the sensing and measuring device, wherein the sensing and measuring device performs data acquisition along a preset acquisition route.

[0108] A model establishment module 200 is used for on-site model reconstruction based on live data, analyzes the multi-frame noise filtering of the laser point cloud data Li based on the image data Ii, obtains denoising laser point cloud Li', and performs splicing reconstruction on the denoising laser point cloud Li' based on the observation pose data Pi to generate a construction site model M.

[0109] A feature extraction module 300 is used for extracting key feature data, performing point cloud plane fitting on the construction site model M, extracting key planes, and calculating key feature data based on a plurality of key planes, wherein the feature data includes: building surface data {(pij, Sj)}, angle data {theta ij} between building surfaces, and straight line equations {l ij}.

[0110] An online comparison module 400 is used for performing mapping comparison based on drawings engineering, mapping and converting corresponding point positions of the construction site model M and the corresponding CAD drawing model to obtain a corresponding mapping relationship, and comparing the construction site model M based on the corresponding mapping relationship to realize building construction supervision.

[0111] As another preferred embodiment of the present application, the model establishing module comprises a noise processing unit and a model establishing unit, the noise processing unit comprises:

[0112] A noise marking subunit is configured to acquire live data {Pi, Li, Ii} at time i, perform object recognition of workers and equipment based on preset deep learning human and object monitoring algorithms on image data Ii in the live data, and perform image region ROI.

[0113] A noise removing subunit is configured to remove corresponding part of point cloud data in the laser point cloud data Li corresponding to the image region ROI based on the corresponding matching relationship between the image data Ii and the laser point cloud data Li.

[0114] A discrete screening subunit is configured to process the remaining laser point cloud data Li based on a distance-based clustering algorithm to screen and remove independent discrete points, and acquire denoised laser point cloud Li'.

[0115] In general, the present application discloses an online construction site robot monitoring management method and system. The system is erected at a specific position of a construction site, can measure and model the construction site and construction process at any time, and can compare with construction drawings to realize monitoring of the construction and rapid prompting of abnormal conditions.

[0116] The system comprises a sensing and measuring device, a moving guide rail, a background operation processing device and the like. The sensing and measuring device comprises a camera, a 3D laser radar, necessary data acquisition fusion and transmission modules, power supply and fixing devices and the like. The sensing and measuring device is fixed on a motor and a trolley. The latter is installed on the moving guide rail and can be controlled to slide accurately forward and backward on the moving guide rail.

[0117] The present application discloses a method for online construction site measurement and building construction drawing monitoring comparison using the above system. The method comprises three main parts of data acquisition and construction site modeling, key data feature extraction, data measurement and online comparison with drawings. The process is as follows:

[0118] Firstly, the robot system is installed and erected at a suitable position of the construction site.

[0119] During construction monitoring, the sensing and measuring device is controlled to move on the guide rail by the trolley, data acquisition is sequentially performed at different positions, multiple frames of data (each frame of data comprises laser point cloud, image data and the position and pose thereof) can be acquired, and the entire construction area to be monitored can be covered. Noise regions of workers and tools are detected based on image data, corresponding laser point cloud regions are removed, and then a model of the construction site is constructed by matching and combining multiple frames of data.

[0120] Based on the construction site model M, the key planes in the model are extracted by point cloud plane fitting, and then the key data features are extracted, including but not limited to the distance between key points, plane equation and point cloud, straight line equation, plane angle, etc.

[0121] Finally, through the comparison of the mapping relationship between the construction operation surface and the CAD drawing, the angle data between the planes, the projection of the straight line data in the CAD drawing, the angle relationship between the two straight lines, the angle relationship between the straight line and the plane, etc., the construction abnormal situation is detected in real time, and the warning is given quickly, which is convenient for rectification.

[0122] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The program can be stored in a non-volatile computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0123] Other embodiments of the present disclosure will be apparent to those skilled in the art with the disclosure in the specification and the examples. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure that follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and examples are only considered as exemplary, and the true scope and spirit of the present disclosure are indicated by the claims.

[0124] It should be understood that the present disclosure is not limited to the precise construction that has been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is only limited by the appended claims.

Claims

1. An online construction site robot monitoring management method, characterized by, Comprise the following steps: The collection of field data is based on a preset sensor measurement device to collect real-time image data and laser point cloud data of the construction site, and based on observation pose data of the sensor measurement device to establish live data , the sensor measurement device performs data acquisition along a preset collection operation route, and the observation pose data characterizes the position and pitch angle state of the sensor measurement device when collecting data. Live model reconstruction based on live data, from image data Analyzing the laser point cloud data Multi-frame filtering of noise, obtaining denoised laser point cloud And based on the observation pose data Splicing reconstruction on the denoised laser point cloud Generating a construction site model M; The extraction of the key feature data, the point cloud plane fitting of the construction site model M, the extraction of the key plane, and the calculation of the key feature data based on a plurality of the key planes, the feature data including: building surface data , angle data between the building surfaces , and straight line equation . Based on the mapping comparison of the drawing engineering, the construction site model M is mapped and transformed with the corresponding CAD drawing model to obtain a corresponding mapping relationship, and the construction site model M is compared based on the corresponding mapping relationship to realize building construction supervision; The step of filtering noise from the laser point cloud data based on image data analysis to obtain denoised laser point cloud specifically includes: Obtaining live data at time i , based on a preset deep learning human body and object monitoring algorithm, image data in the live data , performing object recognition of workers and equipment, and performing image region ROI; based on the image data corresponding matching relationship with the laser point cloud data , the laser point cloud data corresponding to the image region ROI is removed. Distance-based clustering algorithm for the remaining laser point cloud data is processed to filter out independent discrete points to obtain a denoised laser point cloud ; The step of performing point cloud plane fitting on the construction site model M, extracting key planes, and calculating key feature data based on a plurality of key planes specifically includes: Based on the construction site model M, the point cloud plane is fitted by the random sample consensus fitting plane algorithm, the key planes in the model are extracted, the equations of various planes of the building and the corresponding point sets are obtained, and the building surface data is represented as: wherein represents a plane equation of a building face with the sequence number j, represents a point set of a building face with the sequence number j; Calculating straight line equations that intersect each other based on the building surface data of non-intersecting building surfaces } and the angle data between the building surfaces } The angle data } is expressed as: in, for The normal vector, for The normal vector, ij, represents the building surface index, and is obtained by combining the following... and Equations, to obtain the equations of intersecting lines { } 2. The online construction site robot monitoring management method of claim 1, wherein The step of mapping and transforming the corresponding point positions of the construction site model M and the corresponding CAD drawing model to obtain a corresponding mapping relationship Y includes: A group of feature points on three non-parallel lines are selected from the construction site model M, and corresponding feature matching points are selected on the CAD drawing model; Based on the rigid rotation principle, the rotation matrix R and the translation matrix T between the CAD drawing model and the construction site model M are established, and the corresponding mapping relationship is obtained by matrix calculation, which can be represented as: Where X represents any design point in the CAD drawing model, Y represents the corresponding point coordinates in the construction operation surface to be solved based on the corresponding relationship of the site construction model, and Y can be used to verify the actual collected data in the site construction model M.

3. The method of claim 2, wherein the method further comprises: In the step of comparing the construction site model M based on the corresponding mapping relationship to realize building construction supervision, specifically includes: Based on the measured angle data between the building surfaces in the construction site model }, by mapping relationship and the angle between the respective corresponding building surfaces in the CAD drawing model are compared, when the deviation value is too large, real-time output warning; Based on the measured straight line equations of the intersecting building surfaces in the construction site model, } and the straight line equations of the corresponding building surfaces in the CAD drawing model through mapping relationship, and when the deviation value is too large, real-time output warning; Based on the measured straight line equation in the construction site model { The angle data between the intersecting points { The system compares the angle data between the corresponding building surfaces in the CAD drawing model with the mapping relationship, and outputs an alarm in real time when the deviation value is too large. Based on the linear equation of the measured construction site model } and the line-surface distance between the building surface data The angle data between the respective corresponding building surfaces in the CAD drawing model is compared through the mapping relationship, and when the deviation value is too large, a warning is output in real time.

4. The method of claim 1, wherein the method further comprises: In the step of the sensing measurement device performing data acquisition with a preset acquisition operation route; when the sensor measurement device moves a distance di in the x direction at time i, the observed pose data of the sensor measurement device at time i may be represented as: wherein, corresponding respectively to the x, y, z coordinates and the pitch, roll and yaw angles of the sensor and measurement device at the i-th time instant, the coordinate system having its origin at the starting point of the collection and its axes determined by the initial orientation of the collection.

5. An online construction site robot monitoring management system characterized by, Comprise: The data acquisition module is configured to perform acquisition of field data, and based on a preset sensing measurement device, to acquire real-time image data of a construction site and laser point cloud data , and based on observation pose data of the sensing measurement device to establish live data , wherein the sensing measurement device acquires data along a preset acquisition operation route, and the observation pose data characterizes a position and a pitch angle state of the sensing measurement device when acquiring data. A model establishing module is configured to reconstruct a live model based on live data according to image data analyze the laser point cloud data perform multi-frame filtering of noise to obtain denoised laser point cloud and based on the observation pose data stitch and reconstruct the denoised laser point cloud to generate a construction site model M; a feature extraction module for performing extraction of key feature data, performing point cloud plane fitting on the construction site model M, extracting key planes, and calculating key feature data based on a plurality of the key planes, the feature data including: building surface data , angle data between building surfaces , and straight line equations ; An online comparison module for performing mapping comparison based on drawing engineering, mapping and transforming the corresponding point positions of the construction site model M and the corresponding CAD drawing model to obtain a corresponding mapping relationship, comparing the construction site model M based on the corresponding mapping relationship to realize building construction supervision; The model establishment module includes a noise processing unit and a model establishment unit, and the noise processing unit includes: Noise marking subunit, for obtaining live data at time i , based on the preset deep learning human body, object monitoring algorithm to the image data in the live data , the object recognition of workers, implement equipment, and image region ROI; a noise removing sub-unit, configured to remove noise from the image data based on the image data corresponding matching relationship with the laser point cloud data corresponding to the image region ROI; and remove the part of the laser point cloud data corresponding to the image region ROI. A discrete screening subunit is used to screen and remove independent discrete points from the remaining laser point cloud data based on a distance-based clustering algorithm The processing is performed to screen and remove independent discrete points to obtain a denoised laser point cloud ; The step of performing point cloud plane fitting on the construction site model M, extracting key planes, and calculating key feature data based on a plurality of key planes specifically includes: Based on the construction site model M, the point cloud plane is fitted by the random sample consensus fitting plane algorithm, the key planes in the model are extracted, the equations of various planes of the building and the corresponding point sets are obtained, and the building surface data is represented as: wherein, represents a plane equation of a building face with the serial number j, represents a point set of a building face with the serial number j; Based on the data of the building surfaces which are not intersected, the equation of the straight line which intersects each other and the data of the angle between the building surfaces are calculated. } and the data of the angle between the building surfaces are calculated. } and the data of the angle between the building surfaces are calculated. } are expressed as: in, for The normal vector, for The normal vector, ij, represents the building surface index, and is obtained by combining the following... and Equations, to obtain the equations of intersecting lines { }

Images