A method for calculating and displaying the working status of construction machinery in real time

Abstract

The present invention relates to the technical field of calculating and displaying the working status of construction machinery in real time, and discloses a method for calculating and displaying the working status of construction machinery in real time, which comprises the following steps: establishing a priori database of construction machinery parameters and training an engineering machinery instance segmentation model, establishing a two-dimensional digital base plate of a site containing fixed camera position registration information, and using a deep learning algorithm to detect and obtain the corresponding pixel points of the construction machinery in the field of view of the video at any time, as well as the angle between the boom and the machinery body, and the movement of the construction machinery; the present invention realizes automated inspection of the efficiency of daily construction machinery, maximizes the workload generated by the construction machinery in a safe state, and automatically supervises the working status of the construction machinery, thereby improving the efficiency of building construction and the level of daily machinery management of construction projects, and realizing fast, efficient and accurate all-round and full-process supervision of the construction machinery. The present invention is applicable to various types of engineering machinery and has high compatibility.

Description

Technical Field

[0001] The present invention relates to the technical field of calculating and displaying the working status of construction machinery in real time, and in particular to a method for calculating and displaying the working status of construction machinery in real time. Background Art

[0002] Machinery management refers to the comprehensive management of machinery and equipment by construction companies, encompassing the entire process of equipment acquisition, operation, production, maintenance, upgrading, and disposal. It encompasses all management activities throughout the material and value processes of machinery and equipment. Managing construction machinery is a crucial tool for construction companies today. With increasingly fierce competition in the construction market and the deepening of enterprise reform, the role of construction machinery in modern construction has become increasingly prominent, and machinery and equipment management is receiving increasing attention within construction companies. However, due to various factors, construction companies still face many practical challenges in machinery and equipment management, such as the mismatch between outdated machinery and equipment and the increasing demands of construction, the disparity between weak professional and technical expertise and advanced machinery and equipment technology, and the incompatibility between low equipment management standards and the high level of mechanization in construction. As the old saying goes, "If you want to do your work well, you must first sharpen your tools." It can be seen from this that how to manage the machinery and equipment of construction enterprises well, correctly analyze and resolve various contradictions in the management process, and manage, use, maintain and repair construction machinery well, has very important practical significance for improving the equipment management and technical level of the enterprise, accelerating the construction progress, improving work efficiency, reducing labor intensity, and enhancing the market competitiveness of the enterprise.

[0003] Machinery management is an important part of engineering construction management. In addition to daily machinery safety management, it should also include daily inspections of construction machinery efficiency to maximize the workload of construction machinery in a safe state. However, due to the current simplification of management personnel by construction units, the workload of daily inspections is large, which consumes a lot of time and energy. As a result, management personnel cannot supervise the working status of construction machinery in the first place and throughout the entire process, resulting in reduced construction efficiency. In order to improve the level of daily machinery management in construction projects, the project urgently needs a new management method to quickly, efficiently and accurately supervise construction machinery in all aspects and throughout the entire process. Therefore, we propose a method to calculate and display the working status of construction machinery in real time to solve the above problems. Summary of the Invention

[0004] The present invention proposes a method for calculating and displaying the working status of construction machinery in real time, which solves the problems in the background technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for calculating and displaying the working status of a construction machine in real time comprises the following steps:

[0007] Step S1, establishing a priori database of construction machinery parameters and training a construction machinery instance segmentation model;

[0008] Step S2, creating a two-dimensional digital baseplate of the site including fixed camera position registration information;

[0009] Step S3: Using a deep learning algorithm, detect and obtain the corresponding pixel points of the construction machinery within the field of view at any time in the video, as well as the angle between the boom and the machine body;

[0010] Step S4, determining the movement, working and non-working status of the construction machinery;

[0011] In step S5, the distance between the machine and the camera center is calculated using a depth estimation algorithm assisted by mechanical parameters; the working status of the machine is dynamically updated to the background database and displayed in real time on a two-dimensional digital baseboard.

[0012] As a preferred technical solution of the present invention, the step S1 of establishing a priori database of construction machinery parameters and training a construction machinery instance segmentation model specifically includes the following steps:

[0013] S101, crawling construction machinery parameters from websites including but not limited to domestic and foreign second-hand machinery trading websites, public machinery parameter databases, etc., including but not limited to the following fields: machinery pictures, machinery type, machinery model, machinery dimensions such as length, width and height;

[0014] S102: Manual cleaning of database data, including but not limited to deduplication of low-quality images, merging images of machines of the same model, and correction of incorrect machine parameters;

[0015] S103, labeling the machine in the above picture using instance segmentation mask, with the label category being the machine model;

[0016] S104: training a deep learning model for instance segmentation of the above-mentioned engineering machinery.

[0017] As a preferred technical solution of the present invention, the step S2 of establishing a two-dimensional digital baseplate of the site including the fixed camera position registration information specifically includes the following steps:

[0018] S201, obtaining a site plan including but not limited to a construction site layout drawing and an aerial photo of the construction site;

[0019] S202, obtaining the relative pixel coordinates (x, y) of the site camera installation location, including but not limited to the location of the click action captured by the front-end page, and then recording the camera ID;

[0020] S203, calibrating the internal and external parameters of the camera to determine the mapping matrix a from the monitoring pixels within the camera's field of view to the pixels on the site plan;

[0021] S204, storing the camera IDi, relative pixel coordinate position (xi, yi), and mapping matrix ai into the camera information database.

[0022] As a preferred technical solution of the present invention, step S3 uses a deep learning algorithm to detect and obtain the corresponding pixel points of the construction machinery and the angle between the boom and the machinery body within the field of view at any time in the video, which specifically includes the following steps:

[0023] S301, identifying the construction machinery within the field of view at any time, and obtaining the size and coordinate attributes of the construction machinery object detection frame;

[0024] S302, using the detection frame described in step 1 as the search area, detecting the mechanical body and the mechanical arm within the detection frame, writing attribute information indicating that the body and the arm belong to the same machine, and obtaining the pixel coordinates of the center points of the body and the arm;

[0025] S303: Calculating the position information of the vehicle body and boom belonging to the same machine at any time, using the surface-line intersection method;

[0026] S304: Based on the position information of the vehicle body and the boom at any time described in step S3, calculate and collect: 1) the position A of the vehicle body at any time, 2) the angle θ between the vehicle body and the boom.

[0027] As a preferred technical solution of the present invention, the step S4, determining the movement, working and non-working states of the construction machinery, specifically includes the following steps:

[0028] S401: Divide the calculation period into different calculation periods according to different machines. The period can be divided equally or unequally. Assume that the length of a certain period is t;

[0029] S402: Calculate the position A of the vehicle body and the angle θ between the vehicle body and the boom to determine the start and end time of the above period, the calculation method being in accordance with claim 4;

[0030] S403: Calculate the change rate of the vehicle body position A and the angle θ between the vehicle body and the boom during the above time period, the position change rate v, and the angle change rate ω, which is the end time minus the start time divided by the time period duration t;

[0031] S404: Determine the motion state of the machine during the above period. If ω > ω0, it is determined that the machine is in the working state; if ω < ω0 and v > v0, it is determined that the machine is in the moving state; if ω < ω0 and v < v0, it is determined that the machine is in the non - working state.

[0032] As a preferred technical solution of the present invention, in step S5, the distance between the machine and the camera center is calculated using a depth estimation algorithm assisted by mechanical parameters; the working state of the machine is dynamically updated to the background database and is rendered and displayed on the two - dimensional digital floor in real - time, which specifically includes the following steps:

[0033] S501, Use a depth estimation algorithm guided by semantics to accurately segment the edge of the construction machine area mask at time t and estimate its average depth from the camera.

[0034] S502, According to the machine model identified by the instance segmentation model, retrieve and match the shape and size parameters of the machine from the prior database of construction machine parameters described in feature (1) of claim 1.

[0035] S503, Use the shape and size parameters to correct the depth of the construction machine from the camera, obtain the depth of the lowest point of the mask vertically downward from the centroid of the mask, so as to obtain the accurate pixel coordinates of the construction machine in the camera's field of view.

[0036] S504, Use the relative pixel coordinate positions (xi, yi) and the mapping matrix ai in feature (3) of claim 1 to obtain the mechanical pixel position at time t in the two - dimensional digital floor.

[0037] S505, Render the pixel position at time t + 1 on the front - end page, and continuously refresh it subsequently to obtain the dynamic trajectory of the machine.

[0038] The beneficial effects of the present invention are:

[0039] The present invention realizes the automatic inspection of the efficiency of daily construction machinery. The construction machinery generates the maximum workload in a safe state, automatically supervises the working state of the construction machinery, improves the construction efficiency, and at the same time improves the daily mechanical management level of the construction project, realizing the all - around and whole - process supervision of the construction machinery quickly, efficiently and accurately. It is applicable to various construction machinery and has high compatibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flowchart of a method for calculating and real - time displaying the working state of construction machinery according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0042] Reference Figure 1 , a method for calculating and displaying the working status of construction machinery in real time, comprising the following steps:

[0043] Step S1: Establish a priori database of construction machinery parameters and train a construction machinery instance segmentation model, specifically including the following steps:

[0044] S101, crawling construction machinery parameters from websites including but not limited to domestic and foreign second-hand machinery trading websites, public machinery parameter databases, etc., including but not limited to the following fields: machinery pictures, machinery type, machinery model, machinery dimensions such as length, width and height;

[0045] S102: Manual cleaning of database data, including but not limited to deduplication of low-quality images, merging images of machines of the same model, and correction of incorrect machine parameters;

[0046] S103, labeling the machine in the above picture using instance segmentation mask, with the label category being the machine model;

[0047] S104, training the instance segmentation deep learning model of the above-mentioned construction machinery;

[0048] Step S2, establishing a two-dimensional digital baseplate of the site including the fixed camera position registration information, specifically includes the following steps:

[0049] S201, obtaining a site plan including but not limited to a construction site layout drawing and an aerial photo of the construction site;

[0050] S202, obtaining the relative pixel coordinates (x, y) of the site camera installation location, including but not limited to the location of the click action captured by the front-end page, and then recording the camera ID;

[0051] S203, calibrating the internal and external parameters of the camera to determine the mapping matrix a from the monitoring pixels within the camera's field of view to the pixels on the site plan;

[0052] S204, storing the camera IDi, relative pixel coordinate position (xi, yi), and mapping matrix ai into the camera information database;

[0053] Step S3 uses a deep learning algorithm to detect and obtain the corresponding pixel points of the construction machinery and the angle between the boom and the machinery body within the field of view at any time in the video. Specifically, the following steps are included:

[0054] S301. Identify the construction machinery within the field of view at any time, and obtain the size and coordinate attributes of the detection frame of the construction machinery object.

[0055] S302. Taking the detection frame described in step 1 as the retrieval area, detect the mechanical vehicle body and the mechanical boom within the detection frame, write the attribute information belonging to the same machinery for the vehicle body and the boom, and at the same time obtain the pixel coordinates of the center point of the vehicle body and the boom.

[0056] S303: Calculate the position information of the vehicle body and the boom belonging to the same machinery at any time, and the calculation method is the plane-line intersection method.

[0057] S304: Based on the position information of the vehicle body and the boom at any time described in step S3, calculate and summarize: 1) the position A of the vehicle body at any time, 2) the angle θ between the vehicle body and the boom.

[0058] Step S4. Determine the motion, working, and non-working states of the construction machinery, which specifically includes the following steps:

[0059] S401: Divide the calculation period, which is divided into different calculation periods according to different machineries. This period can be equally divided or not equally divided. Assume that the duration of a certain period is t.

[0060] S402: Calculate and determine the position A of the vehicle body and the angle θ between the vehicle body and the boom at the start time and end time of the above period, and the calculation method is as described in claim 4.

[0061] S403: Calculate the change rates of the position A of the vehicle body and the angle θ between the vehicle body and the boom in the above period, the change rate v of the position, and the change rate ω of the angle, that is, subtract the quantity at the start time from the quantity at the end time, and then divide by the duration t of the time period.

[0062] S404: Determine the motion state of the machinery in the above period. If ω > ω0, it is determined that the machinery is in the working state; if ω < ω0 and v > v0, it is determined that the machinery is in the motion state; if ω < ω0 and v < v0, it is determined that the machinery is in the non-working state.

[0063] Step S5. Use the depth estimation algorithm assisted by mechanical parameters to calculate the distance between the machinery and the camera center; dynamically update the working state of the machinery to the background database and render it on the two-dimensional digital floor in real time, which specifically includes the following steps:

[0064] S501. Use the depth estimation algorithm guided by semantics to accurately segment the edge of the construction machinery area mask at time t and estimate its average depth from the camera.

[0065] S502, based on the machine model identified by the instance segmentation model, retrieve the outer dimension parameters matching the machine from the construction machine parameter priori database described in feature (1) of claim 1;

[0066] S503, using the external dimension parameters to correct the depth of the construction machine from the camera, obtaining the depth of the lowest point of the mask vertically downward from the centroid of the mask, thereby obtaining the precise pixel coordinates of the construction machine in the camera's field of view;

[0067] S504, using the relative pixel coordinate position (xi, yi) and the mapping matrix ai in feature (3) of claim 1, obtaining the mechanical pixel position at time t in the two-dimensional digital baseplate;

[0068] S505: Pixel position at the front-end page rendering time t+1 is continuously refreshed to obtain the dynamic trajectory of the machine.

[0069] The present invention realizes automated inspection of daily construction machinery efficiency, enables construction machinery to generate maximum workload in a safe state, performs automated supervision on the working status of construction machinery, improves building construction efficiency, and at the same time improves the level of daily machinery management of construction projects, and realizes fast, efficient and accurate all-round and full-process supervision of construction machinery. It is applicable to various types of engineering machinery and has high compatibility.

[0070] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein with equivalents. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for calculating and displaying the working status of construction machinery in real time, characterized in that: It includes the following steps: Step S1: Establish a prior database of construction machinery parameters and train a construction machinery instance segmentation model; Step S2: Establish a two-dimensional digital floor plan of the site including fixed camera position registration information, which specifically includes the following steps: S201: Obtain site plan drawings including but not limited to construction site leveling layout drawings, aerial photos of the construction site, etc.; S202: Obtain the relative pixel coordinates (x, y) of the camera installation position on the site, including but not limited to the position of the click action captured on the front-end page, etc., and then record the camera ID; S203: Calibrate the internal and external parameters of the camera to determine the mapping matrix a from the monitored pixel points within the camera's field of view to the pixel points of the site plane image; S204: Store the camera IDi, relative pixel coordinate position (xi, yi), and mapping matrix ai in the camera information database; Step S3: Use a deep learning algorithm to detect and obtain the corresponding pixel points of the construction machinery within the field of view of the video at any time and the angle between the boom and the machine body, which specifically includes the following steps: S301: Identify the construction machinery within the field of view at any time and obtain the size and coordinate attributes of the detection frame of the construction machinery object; S302: Take the detection frame described in step 1 as the retrieval area, detect the machine body and the machine boom within the detection frame, write the attribute information belonging to the same machine to the body and the boom, and at the same time obtain the pixel coordinates of the center point of the body and the boom; S303: Calculate the position information of the body and the boom belonging to the same machine at any time, and the calculation method is the plane-line intersection method; S304: Based on the position information of the body and the boom described in step S3 at any time, calculate and summarize: 1) the position A of the body at any time, 2) the angle θ between the body and the boom; Step S4: Determine the motion, working, and non-working states of the construction machinery, which specifically includes the following steps: S401: Divide the calculation period, which is divided into different calculation periods according to different machines. This period can be equally divided or unequally divided. Assume that the duration of a certain period is t; S402: Calculate and determine the position A of the body and the angle θ between the body and the boom at the start time and end time of the above period, and the calculation method is as described in step S3; S403: Calculate the change rates of the position A of the body and the angle θ between the body and the boom in the above period, the change rate v of the position, and the change rate ω of the angle, that is, subtract the quantity at the start time from the quantity at the end time, and then divide by the duration t of the period; S404: Determine the motion state of the machinery in the above period. If ω>ω0, it is determined that the machinery is in the working state; if ω<ω0 and v>v0, it is determined that the machinery is in the motion state; if ω<ω0 and v<v0, it is determined that the machinery is in the non-working state; Step S5: Use a depth estimation algorithm assisted by mechanical parameters to calculate the distance between the machinery and the camera center; dynamically update the working state of the machinery to the background database and render and display it on the two-dimensional digital floor plan in real time.

2. A method for calculating and displaying the working status of construction machinery in real time according to claim 1, characterized in that: The specific steps of step S1, establishing a prior database of construction machinery parameters and training a construction machinery instance segmentation model, are as follows: S101, crawling construction machinery parameters from websites including but not limited to domestic and foreign second-hand machinery trading websites, public machinery parameter databases, etc., including but not limited to the following fields: machinery pictures, machinery type, machinery model, machinery dimensions such as length, width and height; S102: Manual cleaning of database data, including but not limited to deduplication of low-quality images, merging images of machines of the same model, and correction of incorrect machine parameters; S103, labeling the machine in the above picture using instance segmentation mask, with the label category being the machine model; S104: training a deep learning model for instance segmentation of the above-mentioned engineering machinery.

3. The method for calculating and displaying the working status of construction machinery in real time according to claim 1, characterized in that: Step S5, which uses a depth estimation algorithm assisted by mechanical parameters to calculate the distance between the machine and the camera center; dynamically updates the machine working status to the backend database, and renders it in real time on a two-dimensional digital baseboard, specifically includes the following steps: S501, using a semantically guided depth estimation algorithm, accurately segment the mask edge of the construction machinery area at time t and estimate its average depth from the camera; S502, based on the machine model identified by the instance segmentation model, searching for the outer dimension parameters matching the machine from the construction machine parameter priori database described in step S1; S503, using the external dimension parameters to correct the depth of the construction machine from the camera, obtaining the depth of the lowest point of the mask vertically downward from the centroid of the mask, thereby obtaining the precise pixel coordinates of the construction machine in the camera's field of view; S504, using the relative pixel coordinate position (xi, yi) and the mapping matrix ai in step S2, obtaining the mechanical pixel position at time t in the two-dimensional digital baseplate; S505: Pixel position at the front-end page rendering time t+1 is continuously refreshed to obtain the dynamic trajectory of the machine.

Images