Construction equipment state supervision method and device based on deep learning, equipment and medium

Abstract

The application discloses a construction equipment state supervision method and device based on deep learning, equipment and medium. The method comprises the following steps: acquiring image data of a construction site and positioning data of each construction equipment, the image data being collected by high-altitude panoramic monitoring cameras arranged at the construction site; performing space-time registration on the image data and the positioning data; establishing a target association mapping relationship between the positioning data of the construction equipment and the corresponding construction equipment in the image data of the construction site; performing multi-level nest work judgment on each construction equipment based on the target association mapping relationship; determining real nest work equipment in each construction equipment according to the multi-level nest work judgment result; statistically analyzing the current comprehensive state of the real nest work equipment; and supervising the real nest work equipment based on the current comprehensive state. The application solves the technical problems of high misjudgment rate and weak scene adaptability of traditional construction equipment nest work supervision.

Description

Technical Field

[0001] This application relates to the field of deep learning, and in particular to a method, device, equipment and medium for monitoring the status of construction equipment based on deep learning. Background Technology

[0002] Monitoring the operational status of construction equipment (excavators, loaders, bulldozers, etc.) at construction sites is a core aspect of construction progress control, cost management, and personnel scheduling. Accurately identifying idle equipment is crucial for improving construction efficiency, reducing equipment downtime costs, and optimizing site resource allocation. Currently, the industry primarily uses three methods for monitoring the status of construction equipment: manual inspection, single-location monitoring, and purely visual video monitoring.

[0003] Traditional manual inspection methods rely on on-site management personnel to periodically inspect and record equipment operating status. This approach suffers from limited inspection frequency, significant subjective judgment errors, high labor costs, and an inability to achieve 24 / 7 comprehensive monitoring. It is highly susceptible to missed inspections, delayed inspections, and inaccurate records, resulting in extremely low monitoring efficiency and accuracy, failing to meet the refined management needs of modern smart construction sites. Currently, the mainstream automated monitoring method is single-location monitoring. This involves equipping various construction equipment with BeiDou or GNSS positioning terminals, relying on the logic of determining equipment inactive status by observing prolonged periods of inactivity and near-zero movement speed. While this method achieves automated data collection and preliminary judgment, it suffers from core technical flaws and extremely poor adaptability. Construction sites often involve numerous fixed-point operations, such as excavators continuously digging in a fixed position, foundation pit breaking, site leveling, and loaders piling soil at fixed points. In similar working conditions, the overall geographical location of the equipment does not shift significantly, and the positioning data remains static for a long time. However, the equipment is in normal construction operation and is not idle. Single positioning monitoring cannot distinguish between "stationary operation" and "idle idle operation," which can easily lead to large-scale misjudgments and distorted regulatory data. Existing pure visual monitoring methods rely on construction site surveillance cameras to collect video images and use image recognition algorithms to identify equipment status. However, construction sites use high-altitude wide-angle panoramic cameras, which focus on large-scale, all-area monitoring. This approach suffers from problems such as wide monitoring field of view, extremely low pixel ratio of individual construction equipment images, blurred features of distant equipment, dust, obstruction, and variable lighting. Traditional visual algorithms can only identify the presence of equipment, but cannot distinguish between equipment operation and idle status, let alone match the corresponding individual equipment. This results in a high misjudgment rate, inability to monitor in batches, and poor real-time performance.

[0004] In summary, existing technologies such as single-person manual monitoring, single-location monitoring, and pure visual monitoring all have significant shortcomings. Location data and visual data are independent and fragmented, lacking a unified judgment logic, and cannot adapt to the complex working conditions of fixed-point operations on construction sites. Furthermore, traditional visual models consume significant computational power and have high latency, making it difficult to meet the real-time monitoring needs of construction sites. Therefore, there is an urgent need for a new method for monitoring the idle status of construction equipment that can simultaneously utilize location data and visual image data to achieve multi-modal data spatiotemporal fusion, multi-level hierarchical accurate judgment, and lightweight real-time inference. Summary of the Invention

[0005] The main purpose of this application is to provide a method, device, equipment and medium for monitoring the status of construction equipment based on deep learning, which aims to solve the technical problems of existing technologies such as single location monitoring being unable to distinguish between equipment in fixed-point operation and idle work, pure visual monitoring having weak adaptability to large scenes and being unable to identify small operating parts.

[0006] To achieve the above objectives, this application provides a deep learning-based method for monitoring the status of construction equipment, the method comprising the following steps: Image data of the construction site and positioning data of each construction equipment are acquired. The image data is collected by high-altitude panoramic monitoring cameras deployed at the construction site, and the positioning data is collected by positioning devices installed on the construction equipment. Spatiotemporal registration is performed on the image data and the positioning data, and a target association mapping relationship is established between the positioning data of the construction equipment and the corresponding construction equipment in the image data of the construction site. Based on the target association mapping relationship, multi-level idle time judgment is performed on each construction equipment. The actual idle time equipment in each construction equipment is determined according to the multi-level idle time judgment results. The multi-level idle time judgment includes global motion judgment, local motion judgment and surrounding environment judgment. The current overall status of the actual idle equipment is statistically analyzed, and the actual idle equipment is monitored based on the current overall status, which includes idle duration, idle frequency, and idle period.

[0007] Optionally, the step of performing spatiotemporal registration on the image data and the positioning data, and establishing a target association mapping relationship between the positioning data of the construction equipment and the corresponding construction equipment in the image data of the construction site, includes: The image data is calibrated and corrected to obtain standard image data; The spatial correlation mapping relationship is obtained by mapping and transforming the actual geographic coordinate system of the construction site corresponding to the positioning data with the image pixel coordinate system corresponding to the standard image data. Based on the timestamp information of the positioning data and the standard image data, the positioning data and the standard image data are time-aligned to obtain a time-related mapping relationship; Assign a corresponding electronic identifier to each construction equipment target in the standard image data; Based on the electronic identifier, the spatial association mapping relationship, and the temporal association mapping relationship, a target association mapping relationship is established between each construction equipment target in the image data and the positioning coordinates of each construction equipment in the positioning data.

[0008] Optionally, the global motion determination includes: The idle time and total motion of each construction equipment are calculated based on the target association mapping relationship, and the idle time and total motion of each construction equipment are compared with preset thresholds. The total motion includes displacement, motion speed and position offset amplitude, and the preset thresholds include preset time thresholds and preset total motion thresholds. If the idle time of the construction equipment is greater than the preset time threshold and the total amount of exercise is less than the preset total amount of exercise threshold, then the construction equipment is marked as the first suspected idle equipment.

[0009] Optionally, local motion determination includes: Based on the target association mapping relationship, obtain multiple frames of image data corresponding to the first suspected idle equipment; Based on the multi-frame image data, it is determined whether the first suspected idle equipment is in a completely stationary state. If the first suspected idle equipment is in a completely stationary state, then the first suspected idle equipment is marked as the second suspected idle equipment.

[0010] Optionally, the determination of the surrounding environment includes: Based on the target association mapping relationship, local image data of the second suspected idle equipment is obtained, and the local image data of the second suspected idle equipment is input into a pre-trained target scene semantic recognition model. The target scene semantic recognition model is configured to recognize the surrounding environment of the second suspected idle equipment. If the surrounding environment of the second suspected idle equipment is not identified as a valid construction working area, then the second suspected idle equipment will be marked as a real idle equipment.

[0011] Optionally, before inputting the local image data of the second suspected idle equipment into the pre-trained target scene semantic recognition model, the following steps are included: The initial scene semantic recognition model is compressed and modified to obtain an intermediate scene semantic recognition model. The model compression and modification includes structured pruning, channel simplification and quantization compression. Configure a frame interval adaptive inference mechanism and a local ROI priority inference strategy for the intermediate scene semantic recognition model to obtain the target scene semantic recognition model.

[0012] Optionally, the monitoring of the actual idle equipment based on the current overall status includes: The current overall status will be output in the form of multiple reports and early warning information will be pushed. The multiple reports include single equipment operation efficiency analysis reports and overall equipment management and control analysis data of the construction site.

[0013] Furthermore, to achieve the above objectives, this application also proposes a deep learning-based construction equipment status monitoring device that applies the deep learning-based construction equipment status monitoring method described above, wherein the deep learning-based construction equipment status monitoring device includes: The data acquisition module is used to acquire image data of the construction site and positioning data of each construction equipment. The image data is collected by high-altitude panoramic monitoring cameras deployed at the construction site, and the positioning data is collected by positioning devices installed on the construction equipment. The mapping relationship construction module is used to perform spatiotemporal registration of the image data and the positioning data, and to establish a target association mapping relationship between the positioning data of the construction equipment and the corresponding construction equipment in the image data of the construction site. The idle time judgment module is used to perform multi-level idle time judgment on each construction equipment based on the target association mapping relationship, and to determine the actual idle time equipment in each construction equipment according to the multi-level idle time judgment results. The multi-level idle time judgment includes global motion judgment, local motion judgment and surrounding environment judgment. The status monitoring module is used to statistically analyze the current comprehensive status of the actual idle equipment and monitor the actual idle equipment based on the current comprehensive status, which includes idle duration, idle frequency, and idle period.

[0014] Furthermore, to achieve the above objectives, this application also proposes a deep learning-based construction equipment status monitoring device, the device comprising: a memory, a processor, and a deep learning-based construction equipment status monitoring program stored in the memory, the processor being used to run the deep learning-based construction equipment status monitoring program, the computer program being configured to implement the steps of the deep learning-based construction equipment status monitoring method described above.

[0015] In addition, to achieve the above objectives, this application also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the deep learning-based construction equipment status monitoring method described above.

[0016] This application acquires image data from the construction site and positioning data of various construction equipment, establishes a target association mapping relationship by performing spatiotemporal registration on the image data and positioning data, and performs multi-level idle status judgment based on the target association mapping relationship, including global motion judgment, local motion judgment, and surrounding environment judgment, and statistically analyzes the current comprehensive status of the idle equipment for monitoring. Because this application deeply integrates positioning and visual dual-modal data, combined with whole-machine time-series dynamic recognition and scene semantic verification, it accurately distinguishes between the fixed-point operation status and idle status of construction equipment, solving the core pain point of misjudgment in single positioning monitoring. At the same time, through multiple optimizations such as pre-positioning coarse screening, model lightweight transformation, adaptive computing power scheduling, and local priority inference, it significantly reduces system computing power consumption and inference latency, realizes 24 / 7 uninterrupted real-time monitoring, and significantly improves the accuracy and real-time performance of construction equipment idle status judgment. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the first embodiment of the deep learning-based construction equipment status monitoring method of this application. Figure 2 This is a flowchart illustrating the second embodiment of the deep learning-based construction equipment status monitoring method of this application. Figure 3 This is a flowchart illustrating the third embodiment of the deep learning-based construction equipment status monitoring method of this application. Figure 4 This is a flowchart illustrating the fourth embodiment of the deep learning-based construction equipment status monitoring method of this application. Figure 5 This is a schematic diagram of the module structure of the deep learning-based construction equipment status monitoring device according to an embodiment of this application; Figure 6 This is a schematic diagram of the hardware operating environment involved in the deep learning-based construction equipment status monitoring method in this application embodiment.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] It should be noted that the executing entity of this application embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a computer or server, or an electronic device capable of performing the above functions. The following uses a deep learning-based construction equipment status monitoring device as an example to describe this embodiment and the subsequent embodiments.

[0024] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0025] Based on this, the first embodiment of this application provides a construction equipment status monitoring method based on deep learning, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the deep learning-based construction equipment status monitoring method of this application.

[0026] In this embodiment, the deep learning-based construction equipment status monitoring method includes: Step S10: Acquire image data of the construction site and positioning data of each construction equipment. The image data is collected by high-altitude panoramic monitoring cameras deployed at the construction site.

[0027] It should be noted that image data refers to visual data reflecting the appearance and outline of the construction site and construction equipment, as well as the surrounding environment. This image data is collected by high-altitude panoramic monitoring cameras deployed at the construction site. These cameras can be video surveillance devices erected at high altitudes such as tower crane booms, lighthouses, and high points on slopes, using wide-angle lenses to capture a large area of ​​the construction site. They output image data by continuously collecting real-time video streams of the entire construction site 24 hours a day. The image data can be a continuous sequence of video frames in the video stream. Because high-altitude panoramic monitoring cameras focus on large-scale, comprehensive monitoring, they have a wide field of view, a low pixel ratio for individual construction equipment in the image, and blurred features for distant equipment. Positioning data refers to data reflecting the location and movement status of construction equipment in a spatial dimension. This positioning data is collected by positioning devices installed on the construction equipment. These devices can be Beidou or GNSS positioning terminals mounted on construction equipment such as excavators, loaders, and bulldozers. The positioning data includes the equipment's real-time latitude and longitude coordinates, elevation, movement speed, displacement trajectory, and timestamps.

[0028] It is understandable that image data and location data are two types of heterogeneous data with different sources and different forms of representation. Image data is good at reflecting the appearance of the device and its surroundings, while location data is good at reflecting the device's position and movement trajectory. This step, by simultaneously collecting both types of data, enables subsequent judgments to utilize both the motion quantification capability of location tracking and the state discrimination capability of vision, overcoming the shortcomings of single manual supervision, single location supervision, and pure vision supervision in terms of simplistic judgment logic and susceptibility to misjudgment.

[0029] In some embodiments, image data and positioning data can be collected synchronously at a preset collection frequency. For example, image data is collected at the original video frame rate, and positioning data is reported at a frequency of 1Hz to 10Hz. A timestamp of the collection time is added to each frame of image data and each piece of positioning data to ensure that the two types of data can correspond to each other in the time dimension.

[0030] Step S20: Perform spatiotemporal registration on the image data and the positioning data, and establish a target association mapping relationship between the positioning data of the construction equipment and the corresponding construction equipment in the image data of the construction site.

[0031] It should be noted that spatiotemporal registration refers to the process of aligning image data and positioning data in both spatial and temporal dimensions, encompassing both spatial and temporal registration. Spatial registration eliminates the coordinate system differences between the image pixel coordinate system used in the image data and the actual geographic coordinate system of the construction site used in the positioning data. Temporal registration eliminates time discrepancies caused by asynchronous clocks of the acquisition devices and different transmission link delays. Target association mapping refers to the mapping information that characterizes the one-to-one correspondence between the positioning data of construction equipment and the corresponding construction equipment targets in the image data of the construction site. Through this mapping relationship, the positioning coordinates of any construction equipment at any given time can be associated with the corresponding pixel target and local image area in the image data, and conversely, any equipment target in the image data can also be associated with its positioning coordinates.

[0032] It is understandable that, since image data uses pixel coordinates as the spatial reference and positioning data uses latitude, longitude or elevation geographic coordinates as the spatial reference, and there are differences in the acquisition frequency and transmission latency of the surveillance camera and the positioning terminal, if the two types of data are used together directly without spatiotemporal registration, problems such as misalignment between the device position and the target in the picture and mismatch between the status judgment object will occur.

[0033] Further, step S20 above includes: Step S201: Perform parameter calibration and image correction on the image data to obtain standard image data.

[0034] It should be noted that parameter calibration refers to the process of measuring and determining the imaging parameters of a high-altitude panoramic monitoring camera, which includes two aspects: intrinsic parameter calibration and extrinsic parameter calibration. Intrinsic parameter calibration is used to determine the camera's internal imaging parameters such as focal length, principal point coordinates, radial distortion coefficient, and tangential distortion coefficient; extrinsic parameter calibration is used to determine the camera's external attitude parameters such as its position and orientation in the actual geographic coordinate system of the construction site.

[0035] Image calibration refers to the process of correcting geometric distortions in image data based on calibration parameters obtained from parameter calibration. It is used to eliminate image distortions introduced by wide-angle lenses and shooting angles. Standard image data refers to image data obtained after parameter calibration and image calibration, in which geometric distortions have been eliminated and conform to the ideal imaging model. The standard image data can serve as an accurate image reference for subsequent target positioning and coordinate system mapping transformation.

[0036] Understandably, high-altitude panoramic surveillance cameras deployed at construction sites typically employ wide-angle lenses for comprehensive, wide-area monitoring. The raw image data they acquire inevitably contains geometric distortions such as barrel distortion and perspective distortion. If target localization and coordinate mapping are directly performed using this raw image data, a systematic deviation will occur between the pixel position of the target and its actual position, thus affecting the accuracy of subsequent spatial mapping. This step obtains the camera's intrinsic and extrinsic parameters through parameter calibration and corrects the image accordingly, eliminating geometric distortions and restoring the true geometric proportions of the target's positional relationships within the image. This provides an accurate and distortion-free image reference for subsequent coordinate system mapping transformations.

[0037] In practice, the monitoring equipment collects GIS map data from the construction site to calibrate the internal and external parameters of the high-altitude camera: the internal parameter calibration uses the Zhang Zhengyou calibration method, which calculates the camera's focal length, principal point coordinates, and radial or tangential distortion coefficients by taking multiple images of a calibration board of known dimensions; the external parameter calibration uses several control points with known coordinates on the ground to calculate the camera's position and orientation in the construction site coordinate system. Subsequently, based on the calibration parameters, the monitoring equipment sequentially performs wide-angle distortion correction and perspective transformation on the original image data, compensating for pixel shifts caused by the wide-angle lens and transforming the oblique view into a frontal view aligned with the construction site ground, thus obtaining standard image data.

[0038] Step S202: Map and transform the actual geographic coordinate system of the construction site corresponding to the positioning data with the image pixel coordinate system corresponding to the standard image data to obtain the spatial association mapping relationship.

[0039] It should be noted that the construction site's real geographic coordinate system refers to a coordinate system established based on the actual geographic space of the construction site. It is used to represent the spatial location of positioning data and can be expressed in latitude and longitude or elevation coordinates, or in the form of engineering coordinates of the construction site. The positioning coordinates such as equipment latitude and longitude, elevation, and displacement trajectory collected by the positioning device are all based on this coordinate system. The image pixel coordinate system refers to a coordinate system established based on the pixel array of standard image data. It is used to represent the position of the target in the image, usually with the upper left corner of the image as the origin and pixel coordinates... The term "mapping transformation" refers to the process of establishing a mathematical conversion relationship between the actual geographic coordinate system of the construction site and the pixel coordinate system of the image. This includes the forward projection from geographic coordinates to pixel coordinates and the reverse projection from pixel coordinates to geographic coordinates. Spatial correlation mapping refers to mapping information that characterizes the correspondence between geographic coordinates and pixel coordinates. Through this mapping relationship, any location coordinate can be converted into a pixel position in the image, and any pixel position can also be converted into its corresponding geographic coordinates.

[0040] It is understandable that location data and image data are two types of heterogeneous data with different spatial references. The former uses geographic coordinates as the spatial reference, while the latter uses pixel coordinates as the spatial reference. If there is no conversion relationship between the two, the device's location coordinates cannot be projected onto the screen, and the device target in the screen cannot be calculated back to the geographic space. The location and vision will not be able to corroborate each other in the spatial dimension.

[0041] In some embodiments, the mapping transformation can be implemented based on the camera imaging geometry model: during forward projection, the geographic coordinates of the device target are first transformed to the camera coordinate system using extrinsic parameters (rotation matrix and translation vector), and then the coordinates in the camera coordinate system are transformed to pixel coordinates using intrinsic parameters (focal length, principal point coordinates); during reverse projection, the pixel coordinates are back-projected along the camera optical axis, and the corresponding geographic coordinates are solved in conjunction with ground elevation constraints. For construction sites with large terrain undulations, the ground elevation model can also be divided into several grids, and mapping transformation can be performed grid by grid to improve the mapping accuracy of undulating terrain areas.

[0042] Step S203: Based on the timestamp information of the positioning data and the standard image data, perform time alignment on the positioning data and the standard image data to obtain a time-related mapping relationship.

[0043] It should be noted that timestamp information refers to the acquisition time marker attached to each piece of positioning data and each frame of standard image data. It uniquely indicates the acquisition time of that data and serves as the basis for time alignment processing. Time alignment refers to the process of eliminating time discrepancies between positioning data and standard image data based on timestamp information, ensuring that positioning data and image data at the same time correspond to each other. The time-related mapping relationship refers to the mapping information characterizing the correspondence between positioning data and standard image data in the time dimension. Through this mapping relationship, any frame of standard image data can be associated with positioning data at the same time, and vice versa.

[0044] It is understandable that, due to the different acquisition frequencies, data transmission links, and processing delays of the positioning device and the high-altitude panoramic monitoring camera, there may be millisecond-level or even larger time deviations between the positioning data and the image data during the same acquisition process. If they are used together directly without time alignment, the positioning coordinates of the device will correspond to the image frame of the previous moment or the next moment, causing a mismatch between the positioning target and the image target. This mismatch is particularly obvious when the device is in motion.

[0045] Step S204: Assign a corresponding electronic identifier to each construction equipment target in the standard image data.

[0046] It should be noted that the construction equipment target refers to the image region target detected and extracted from standard image data, which corresponds one-to-one with each construction equipment. Each construction equipment target represents the visual presentation of a construction equipment in the image. The electronic tag refers to the unique identification code (electronic ID) assigned to each construction equipment target, used to uniquely identify and continuously track the corresponding construction equipment in the system; the electronic tag corresponds one-to-one with the terminal identifier of the positioning device mounted on the construction equipment, thereby realizing the identity binding between the visual target and the physical equipment.

[0047] It is understandable that multiple pieces of the same or different types of construction equipment are often present simultaneously in construction site footage. If each piece of equipment is not identified, subsequent multi-level assessments of idle work will lead to confusion in the assessment targets and the inability to assign records to specific equipment. This step assigns a unique electronic identifier to each piece of construction equipment, enabling the image data, location data, and idle work records of the same equipment to be uniformly associated and traced under the same identifier, providing an identification basis for the status monitoring of each piece of equipment.

[0048] In practice, the monitoring equipment uses target detection algorithms (such as the YOLO series algorithms) to detect standard image data, extract each construction equipment target and identify the equipment type (such as excavator, loader, bulldozer, etc.), and then combines the spatial location matching results (i.e. the theoretical pixel position calculated by forward projection from the positioning coordinates) to associate the detected image targets with the physical equipment. A unique electronic ID is assigned to each construction equipment target, and this electronic ID corresponds one-to-one with the positioning terminal ID carried by the equipment, so as to realize the identity binding between visual targets and physical equipment.

[0049] Step S205: Based on the electronic identifier, the spatial association mapping relationship, and the temporal association mapping relationship, establish a target association mapping relationship between each construction equipment target in the image data and the positioning coordinates of each construction equipment in the positioning data.

[0050] It should be noted that the target association mapping relationship refers to the one-to-one precise mapping information established between each construction equipment target in the image data and the positioning coordinates of each construction equipment in the positioning data, with electronic identification as the identity link, spatial association mapping relationship as the spatial link, and temporal association mapping relationship as the temporal link.

[0051] Understandably, electronic identification solves the problem of identifying "which device," spatial mapping solves the problem of spatial correspondence of "where in the image and geographically," and temporal mapping solves the problem of temporal correspondence of "at what moment." This step integrates these three aspects, enabling the location coordinates of any construction equipment at any given moment to be uniquely determined along with its pixel target and local image area in the image data. This achieves a precise one-to-one mapping between physical equipment, location data, and visual images, providing a unified judgment object and data query entry point for subsequent multi-level idle work assessments.

[0052] Step S30: Based on the target association mapping relationship, perform multi-level idle time judgment on each construction equipment, and determine the actual idle time equipment among each construction equipment according to the multi-level idle time judgment results.

[0053] It should be noted that "idle work" refers to a state in which construction equipment is idle, stopped, and has no effective operational movements. In contrast, the state in which the overall geographical location of construction equipment does not shift significantly, but the working parts are in continuous motion, such as excavators continuously digging in a fixed position, foundation pit breaking, site leveling, and loaders piling soil at fixed points. These are considered normal construction operations and are not considered idle work.

[0054] Understandably, the multi-level idle work judgment includes global motion judgment, local motion judgment, and surrounding environment judgment. The first level, global motion judgment, is based entirely on positioning data calculation, without the need to call visual models, resulting in low computational load. It can perform high-frequency global scanning of all construction equipment and quickly eliminate equipment that is obviously in motion. The second level, local motion judgment, only performs whole-machine time-series dynamic analysis on the suspected idle work equipment screened out in the first level, to distinguish between "stationary fixed-point operation" and "idle idle work". The third level, surrounding environment judgment, further verifies whether the equipment is in an effective construction condition area using scene semantic information. The three levels of judgment are progressive and mutually corroborative, which not only significantly reduces the system's computing power consumption but also significantly improves the accuracy of idle work judgment, solving the core pain point of single positioning monitoring being unable to distinguish between stationary operation and idle idle work.

[0055] Step S40: Calculate the current overall status of the actual idle equipment and monitor the actual idle equipment based on the current overall status.

[0056] It should be noted that the current comprehensive status refers to the set of status information that characterizes the actual idle status of equipment, including idle duration, idle frequency, and idle period.

[0057] Understandably, by statistically analyzing and monitoring the current overall status of idle equipment, the results of idle equipment assessments can be transformed into quantifiable, traceable, and decision-making control data. This helps managers understand the idle patterns of individual pieces of equipment and the overall equipment utilization level on the construction site, providing precise data support for resource scheduling, equipment management, personnel allocation, schedule optimization, and cost control at the construction site. This reduces equipment idle costs and improves the level of intelligent and refined management at the construction site.

[0058] Further, step S40 above includes: Step S401: Output the current overall status in the form of multiple reports and push early warning information.

[0059] It should be noted that "multiple reports" refers to a collection of statistical results set up according to different statistical subjects and different control dimensions and output in report form. In this embodiment, the multiple reports include two categories: single equipment operation efficiency analysis reports and overall site equipment control analysis data. The single equipment operation efficiency analysis report is an analysis report generated and output with a single construction piece of equipment as the statistical subject. It reflects the operation and idle time of that equipment within the statistical period. Its content is generated based on the current comprehensive status of the actual idle equipment and may include idle time indicators such as idle time duration, idle frequency, and idle periods. It can also reflect the utilization rate and operation efficiency of the equipment in conjunction with the operation time. The overall site equipment control analysis data refers to analysis data generated and output by summarizing all construction equipment on the construction site as the statistical subject. It reflects the overall operation and idle time control level of the site equipment. Its content may include summary indicators such as overall equipment idle rate, equipment utilization rate, idle equipment list, and idle time distribution.

[0060] Understandably, the single-equipment operation efficiency analysis report provides a basis for the assessment and individual scheduling of each piece of equipment from a "point" perspective, while the overall equipment management and control analysis data of the construction site provides a basis for the overall resource allocation and management decisions of the construction site from a "surface" perspective. The combination of point and surface perspectives complements each other and can meet the decision-making needs of different management levels. At the same time, pushing early warning information shifts the information on idle work from post-event statistics to real-time perception. Managers can intervene and deal with idle work in a timely manner during the occurrence of idle work, thus forming a closed loop of management and control: "status determination - real-time early warning - statistical analysis - scheduling optimization". This step realizes the transformation of idle work determination results into management and control applications, so that supervision is not only "identification" but also serves "handling" and "optimization", empowering the refined and intelligent management and control of the construction site.

[0061] In its implementation, the monitoring equipment marks construction equipment identified as truly idle equipment in real time, issues pop-up warnings on the front-end interface, and retains the data. It simultaneously records the start and end times of idle time and the duration of idle time, and automatically calculates the idle time, frequency, and idle periods for each piece of equipment, forming the current comprehensive status. Based on this, the monitoring equipment generates and outputs individual equipment operation efficiency analysis reports and overall site equipment management analysis data according to a set statistical period (e.g., daily). The individual equipment operation efficiency analysis report records the idle time, frequency, idle periods, and operation efficiency indicators for each piece of equipment within the statistical period. The overall site equipment management analysis data summarizes the idle rate, equipment utilization rate, list of idle equipment, and distribution of idle periods for all equipment on the entire site. Simultaneously, the monitoring equipment pushes warning information, including the equipment's electronic identifier, equipment type, idle start time, current idle time, and location area, to the management personnel's terminal and the monitoring platform, providing site management personnel with a basis for scheduling decisions.

[0062] This embodiment acquires image data from the construction site and positioning data of each construction device. It performs spatiotemporal registration of the image and positioning data and establishes a target association mapping relationship. Based on this mapping relationship, it performs multi-level idle status assessments on each construction device, including global motion judgment, local motion judgment, and surrounding environment judgment, to identify the actual idle devices. It also statistically analyzes the current comprehensive status of the actual idle devices for monitoring. Because this embodiment deeply integrates positioning and visual dual-modal data, combined with dynamic temporal recognition of the entire machine and scene semantic verification, it accurately distinguishes between the fixed-point operation status and the idle status of construction equipment, completely solving the core pain point of misjudgment in single-location monitoring. Simultaneously, through pre-positioning coarse screening, lightweight model modification, adaptive computing power scheduling, and local ROI-priority inference optimization, it significantly reduces system computing power consumption and inference latency, achieving automated, high-precision, low-latency, and comprehensive intelligent monitoring of the idle status of construction equipment.

[0063] Based on the first embodiment described above, in the second embodiment of this application, the same or similar content as the first embodiment can be referred to the above description, and will not be repeated hereafter.

[0064] Based on this, please refer to Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the deep learning-based construction equipment status monitoring method of this application. In this embodiment, step S30 further includes: Step S301: Calculate the idle time and total movement of each construction equipment according to the target association mapping relationship, and compare the idle time and total movement of each construction equipment with the preset threshold.

[0065] It should be noted that idle time refers to the duration during which construction equipment is in a static or near-static state. This time can be accumulated from the moment the equipment enters a static state and is used to measure the duration of the equipment's stillness from a time perspective. Comprehensive motion quantity refers to a set of motion indicators that comprehensively characterize the motion state of construction equipment. This comprehensive motion quantity includes displacement, speed, and positional offset.

[0066] Among them, displacement refers to the distance the device changes in geographical location within a unit of time or a preset time window, which can be calculated by the difference in positioning coordinates between adjacent times; motion speed refers to the real-time motion rate of the device, which can be directly read from positioning data or obtained by the displacement versus time difference; position offset amplitude refers to the degree of offset of the device's current position relative to a reference position (such as the starting position of the time window or the center of the historical position), which is used to reflect the cumulative offset or wandering range of the device within the time window.

[0067] The preset threshold refers to a pre-set criterion used to compare with idle time and comprehensive exercise volume. The preset threshold includes a preset time threshold and a preset comprehensive exercise volume threshold.

[0068] Understandably, the full-domain motion judgment is the first-level lightweight pre-screening in the multi-level idle work judgment. It is calculated entirely based on positioning data, without the need to call a visual recognition model, and has a small computational load. Therefore, it can perform high-frequency, full-domain real-time scanning of all construction equipment on the construction site.

[0069] In practical implementation, the monitoring equipment reads the positioning data of all construction equipment in real time according to the target association mapping relationship, calculates the displacement of the equipment within a unit time (e.g., 5 seconds) through a differential algorithm, and then obtains the movement speed and position displacement amplitude. The equipment accumulates the duration of low movement to obtain the idle time. Combined with the adaptive setting of the displacement judgment threshold (e.g., 0.2m) of the construction site operation area, the preset comprehensive movement threshold is formed, and a preset time threshold is set. Then, the idle time of each construction equipment is compared with the preset time threshold, and the comprehensive movement is compared with the preset comprehensive movement threshold, and the comparison results of each equipment are output.

[0070] Step S302: If the idle time of the construction equipment is greater than the preset time threshold and the total amount of exercise is less than the preset total amount of exercise threshold, then the construction equipment is marked as the first suspected idle equipment.

[0071] It should be noted that the first suspected idle equipment refers to construction equipment screened out by the whole-domain motion judgment, whose idle time is greater than the preset time threshold and whose comprehensive motion amount is less than the preset comprehensive motion amount threshold, that is, equipment that is in a fixed-point static state for a long time; the first suspected idle equipment is the screening result of the first level of the multi-level idle equipment judgment, and is not the final judgment conclusion. It still needs to be further refined and verified by subsequent local motion judgment and surrounding environment judgment.

[0072] Understandably, this step employs a dual-condition (logical AND) judgment method that simultaneously satisfies both "idle time greater than a preset time threshold" and "comprehensive motion volume less than a preset comprehensive motion volume threshold": the time condition is used to filter short-term stationary situations such as brief stops, temporary parking, and short stops during shift changes, avoiding misclassifying short-term stationary situations as suspected idle time; the motion volume condition is used to filter equipment that is clearly in motion, such as continuous displacement, relocation, and dynamic operations, avoiding misclassifying normal operating equipment as suspected idle time; the combination of the two filters out only equipment that has been stationary for a long time as the first suspected idle time equipment, completing the preliminary lightweight coarse screening, significantly reducing the number of targets for subsequent visual inference, and reducing system computing power consumption and inference latency; at the same time, the first suspected idle time equipment is only "suspected" rather than the final conclusion, still retaining the room for subsequent local motion judgment to distinguish between "stationary stationary operation" and "idle idle time stationary operation", and for scene semantic verification by judging the surrounding environment, reflecting the design logic of multi-level idle time judgment, step-by-step screening, and gradual refinement.

[0073] In practice, the monitoring equipment marks construction equipment with idle time exceeding a preset time threshold and total physical activity less than a preset total physical activity threshold as the first suspected idle equipment based on its electronic tag, and includes it in the sample set to be visually judged. The start time and current idle time of the idle equipment are recorded simultaneously. Construction equipment that does not meet the above two conditions is judged to be in normal operation or transfer state and is directly filtered out without entering the subsequent level of judgment. If the filtered equipment subsequently re-enters a static state and meets the two conditions again, it is re-marked as the first suspected idle equipment.

[0074] This embodiment calculates the idle time of each construction device and the comprehensive motion amount including displacement, movement speed, and positional offset based on the target association mapping relationship. It compares this calculation with preset time thresholds and preset comprehensive motion amount thresholds. Construction devices with idle time greater than the preset time threshold and comprehensive motion amount less than the preset comprehensive motion amount threshold are marked as first suspected idle devices. Lightweight calculation based entirely on positioning data achieves full-domain pre-screening, quickly filtering out devices that are obviously in dynamic operation or transfer state with extremely low computational cost. Only devices that have been stationary for a long time are retained for subsequent visual fine judgment. This significantly reduces the number of targets for subsequent visual inference, reduces system computing power consumption and inference latency, and provides a unified and reproducible judgment benchmark for multi-level idle device judgment with dual quantitative conditions of time and motion amount, thereby improving the real-time performance and full-domain coverage of supervision.

[0075] Based on the first embodiment described above, please refer to Figure 3 , Figure 3 This is a flowchart illustrating the third embodiment of the deep learning-based construction equipment status monitoring method of this application. In this third embodiment, step S30 further includes: Step S303: Obtain multi-frame image data corresponding to the first suspected idle equipment based on the target association mapping relationship.

[0076] It should be noted that multi-frame image data refers to a set of multi-frame image data that are continuous in time and correspond to the first suspected idle equipment. The multi-frame image data can be continuously extracted from the video stream at a preset frame rate or preset frame interval based on the local screen area (local ROI video area) of the equipment determined by the target association mapping relationship. It covers a preset time window and is used to reflect the state evolution process of the equipment within the time window.

[0077] Understandably, local motion judgment belongs to the second level of visual fine judgment in multi-level idle work judgment. Its judgment object is only the first suspected idle work equipment screened out by the first level of global motion judgment, rather than all construction equipment on the construction site. Therefore, the number of targets for visual reasoning has been greatly reduced by the pre-screening, and the computing power consumption is controllable. At the same time, "whether the whole machine is stationary" is a state quantity in time interval. A single frame image can only reflect the image presentation of the equipment at a certain moment and cannot represent whether the equipment is continuously moving. Therefore, this step obtains multi-frame image data instead of a single frame image to provide time-series samples for subsequent analysis of the dynamic changes of the whole machine. The acquisition is based on the target association mapping relationship to ensure that the acquired multi-frame image data corresponds one-to-one with the first suspected idle work equipment and is not confused with other equipment or background areas.

[0078] In specific implementation, the monitoring equipment determines the pixel position and local image area of ​​the first suspected idle equipment in the standard image data based on the target association mapping relationship, automatically captures the local ROI video image of the location of the equipment based on the binding mapping relationship, and continuously extracts multiple frames (e.g., 10 frames) of time-series image data at a preset frame rate (e.g., a high frame rate of 20 frames / second for suspected stationary equipment) to obtain the multi-frame image data corresponding to the first suspected idle equipment.

[0079] Step S304: Determine whether the first suspected idle equipment is in a completely stationary state based on the multi-frame image data. If the first suspected idle equipment is in a completely stationary state, mark the first suspected idle equipment as the second suspected idle equipment.

[0080] It should be noted that a completely stationary state refers to a state in which the overall outline, circumscribed rectangle, pixel grayscale distribution, and body posture of the construction equipment remain unchanged within the time window covered by multiple frames of image data. In contrast, a state in which the overall position of the equipment does not shift significantly, but the working parts continue to move, causing the body to exhibit slight shaking, posture deviation, and other dynamic characteristics, belongs to a fixed-point operation state and does not belong to a completely stationary state. The second suspected idle equipment refers to construction equipment that has passed the two-level screening of global motion judgment and local motion judgment, that is, equipment that has been stationary for a long time and is completely stationary. The second suspected idle equipment is not the final judgment. It still needs to be determined as real idle equipment after scene semantic verification by judging the surrounding environment.

[0081] Understandably, to address the issue of extremely low pixel counts and the inability to identify small working components such as buckets and robotic arms in high-altitude, wide-angle, large-scene images, this step abandons traditional component feature recognition schemes and adopts whole-machine time-series dynamic analysis: by comparing the time-series features of multiple frames of image data, the dynamic changes in the overall outline of the equipment, the circumscribed rectangle, pixel grayscale distribution, and body posture between frames are compared. If the equipment exhibits continuous subtle posture shifts or body shaking, it indicates that its working components are still in motion and are in normal fixed-point operation, thus excluding idle equipment. If the overall outline, circumscribed rectangle, and pixel distribution remain unchanged across multiple frames, the equipment is in a completely stationary state, retaining the suspected idle equipment and marking it as the second suspected idle equipment. This allows for precise visual differentiation between "stationary fixed-point operation" and "idle idle equipment," overcoming the core deficiency of single-location monitoring in distinguishing between the two types of stationary states. Furthermore, it further compresses the set of suspected idle equipment on the basis of the first-level coarse screening, providing accurate verification objects for subsequent surrounding environment judgment.

[0082] This embodiment obtains multi-frame image data corresponding to the first suspected idle equipment based on the target association mapping relationship, and determines whether the equipment is in a completely stationary state based on the multi-frame image data. The first suspected idle equipment in a completely stationary state is marked as the second suspected idle equipment. The whole machine time-series dynamic analysis replaces the traditional feature recognition of small working parts, which overcomes the recognition problem of small equipment pixel ratio and missing part features in high-altitude large scene images. It accurately distinguishes between stationary fixed operation and idle idle operation at the visual level, further compresses the set of suspected idle equipment, reduces the computational load of subsequent verification, and provides precise judgment support for multi-level idle judgment, which significantly improves the accuracy of idle judgment and scene adaptability.

[0083] Based on the first embodiment described above, please refer to Figure 4 , Figure 4 This is a flowchart illustrating the fourth embodiment of the deep learning-based construction equipment status monitoring method of this application. In this fourth embodiment, step S30 further includes: Step S305: Obtain local image data of the second suspected idle equipment based on the target association mapping relationship, and input the local image data of the second suspected idle equipment into the pre-trained target scene semantic recognition model.

[0084] It should be noted that the local image data refers to the image data of the local screen area corresponding to the second suspected idle equipment and its surrounding environment. Unlike the multi-frame image data used for whole machine dynamic analysis in the local motion judgment, which focuses on the equipment body, the local image data in this step focuses on the environmental screen covering the equipment (such as the working face, earth pile, crushed material, open space, road, etc.) for scene semantic recognition.

[0085] Understandably, the surrounding environment judgment belongs to the third level of scene semantic verification in the multi-level idle work judgment: the first two levels of judgment have completed the coarse screening of "positioned and stationary" and the visual fine judgment of "completely stationary machine". However, equipment that is "stationary and the whole machine is stationary" may still be in a short-term situation such as waiting for construction instructions or process connection within the effective construction working condition area. If it is directly judged as idle work, it will still produce misjudgment. This step introduces scene semantic information of the surrounding environment of the equipment as the basis for the third level of verification. It judges whether the equipment is in the effective construction working condition area from the scene dimension. Together with the first two levels of judgment, it constitutes the progressive judgment logic of "positioned and stationary - whole machine stationary - scene verification". At the same time, using local image data (rather than the whole scene) as the model input can make the recognition field focus on the surrounding environment of the equipment, which reduces the interference of irrelevant areas in the image on the recognition results and reduces the computational load of model inference, ensuring the real-time performance of the verification process.

[0086] In its specific implementation, the monitoring equipment determines the pixel position of the second suspected idle equipment in the standard image data based on the target association mapping relationship, extracts local image data containing the equipment and its surrounding area, and inputs the local image data into a pre-trained target scene semantic recognition model. The model performs semantic feature recognition on the surrounding environment of the second suspected idle equipment and outputs the scene category recognition result of the surrounding environment of the equipment. For example, it determines whether the area where the equipment is located is a valid construction condition area such as a foundation pit operation area, earthwork stockpile area, site leveling area, or crushing operation area, or a non-construction area such as an open space, road, or parking area.

[0087] Furthermore, prior to step S305 above, the following steps are also included: Step S3051: Compress and modify the initial scene semantic recognition model to obtain an intermediate scene semantic recognition model.

[0088] It should be noted that the initial scene semantic recognition model refers to the original scene semantic recognition model without compression or modification. Its network structure is complete, and its parameter scale is large, possessing the ability to semantically recognize construction scene categories. However, the number of parameters and computational power consumption per inference are high, making it the starting point for model compression and modification. Model compression and modification refers to the process of reducing the parameter scale and computational power consumption of the model while maintaining its scene semantic recognition capabilities. This model compression and modification includes three methods: structured pruning, channel simplification, and quantization compression. Structured pruning refers to a pruning method that removes redundant structural units in the network that contribute little to recognition accuracy, using structural units such as convolutional kernels, channels, and network layers as granularity. Unlike unstructured pruning, which only resets individual weights to zero, the model after structured pruning retains a regular network structure, facilitating hardware acceleration and deployment.

[0089] Channel simplification refers to reducing the number of channels in convolutional layers and lowering the dimension of feature maps, thereby reducing computational load and feature redundancy between channels and further reducing the computational complexity of the model. Quantization compression refers to converting model weights and / or activation values ​​from high-precision floating-point representation (e.g., FP32 floating-point numbers) to low-precision numerical representation (e.g., INT8 integers), thereby reducing the model's storage space and computational load and improving inference throughput per unit of computing power. The intermediate scene semantic recognition model refers to the scene semantic recognition model obtained after model compression and modification. Its parameter count and computational consumption are significantly reduced compared to the initial scene semantic recognition model, while retaining scene semantic recognition capabilities, serving as the object for subsequent configuration of inference mechanisms and inference strategies.

[0090] In the specific implementation, the monitoring equipment performs structured pruning on the initial scene semantic recognition model, eliminating redundant convolutional parameters based on the contribution of each convolutional kernel and structural unit to the recognition accuracy; it also performs channel simplification on the convolutional layers to reduce the number of channels and reduce the dimension of the feature map; and it uses INT8 quantization compression to convert the model weights from FP32 floating-point numbers to 8-bit integer representations.

[0091] Step S3052: Configure the frame interval adaptive inference mechanism and the local ROI priority inference strategy for the intermediate scene semantic recognition model to obtain the target scene semantic recognition model.

[0092] It should be noted that the frame interval adaptive inference mechanism refers to an inference scheduling mechanism that adaptively adjusts the inference frame rate (i.e., frame interval) according to the state of the construction equipment. Specifically, low frame rate inference is used for equipment that is moving normally and operating normally, while high frame rate inference is used for equipment that is suspected of being stationary or idle, so that computing resources are dynamically tilted towards equipment that needs precise judgment. The local ROI priority inference strategy refers to a strategy that prioritizes inference on local ROIs (Regions of Interest), where the local ROI refers to the local screen area where the suspected idle equipment is located. The local ROI priority inference strategy means that high frame rate inference is only performed on the local ROI area of ​​the suspected idle equipment, while the inference frame rate is reduced or no inference is performed on other areas of the screen, avoiding high frame rate processing for the entire screen. The target scene semantic recognition model refers to the final scene semantic recognition model obtained after configuring the frame interval adaptive inference mechanism and the local ROI priority inference strategy on the intermediate scene semantic recognition model, that is, the pre-trained model used in step S305 to identify the surrounding environment of the second suspected idle equipment.

[0093] Understandably, model compression addresses the issue of "high cost per inference," while frame interval adaptive inference mechanism and local ROI priority inference strategy address the issues of "inference frequency" and "inference range." By adaptively adjusting the inference frame rate according to device status, the computational waste caused by indiscriminately applying equal frame rate inference to all devices is avoided, allowing high frame rate computational power to be concentrated on the precise judgment needs of suspected idle equipment. By performing high frame rate inference only on local ROI areas of suspected idle equipment, invalid calculations caused by high frame rate processing of the entire screen are avoided. The two types of inference mechanisms, combined with model compression, maximize the compression of inference latency while ensuring recognition accuracy, enabling real-time inference on the edge and meeting the needs of 24 / 7 uninterrupted real-time monitoring.

[0094] Step S306: If the surrounding environment of the second suspected idle equipment is not identified as a valid construction working area, then the second suspected idle equipment is marked as a real idle equipment.

[0095] It should be noted that the effective construction working condition area refers to the working area where construction equipment can carry out construction operations normally, such as the foundation pit operation area, earthwork stockpiling area, site leveling area, and crushing operation area. Equipment in such areas may be operating normally or waiting for construction instructions or process connections. "True idle equipment" refers to construction equipment that is ultimately confirmed to be in an idle state after passing three levels of judgment: overall motion judgment, local motion judgment, and surrounding environment judgment. This means equipment that has been stationary for a long time, is completely stationary, and whose surrounding environment does not belong to the effective construction working condition area. The "true idle equipment" is the final output result of the multi-level idle equipment judgment and serves as the object of subsequent comprehensive status statistics and monitoring.

[0096] In its implementation, the monitoring equipment compares the scene category recognition results output by the target scene semantic recognition model with the preset set of effective construction condition area categories. If the surrounding environment of the second suspected idle equipment does not belong to the effective construction condition area such as the foundation pit operation area, earthwork stockpile area, site leveling area, or crushing operation area (e.g., identified as an open space, road, or parking area), then the second suspected idle equipment is marked as a real idle equipment, and its idle start time and scene category are recorded simultaneously for subsequent current comprehensive status statistics and monitoring. If the surrounding environment belongs to the effective construction condition area, it is not marked as a real idle equipment for the time being.

[0097] This embodiment obtains local image data of the second suspected idle equipment based on the target association mapping relationship and inputs it into a pre-trained target scene semantic recognition model to identify the surrounding environment. The second suspected idle equipment whose surrounding environment does not belong to the effective construction condition area is marked as real idle equipment. The scene semantic information is used as the third-level verification basis, which together with the global motion judgment and the local motion judgment constitutes a complete judgment closed loop of "positioning static - whole machine static - scene invalid". This further eliminates the misjudgment situation of normal operation at fixed point and short-term waiting in the effective construction condition area, accurately locks the real idle equipment, and makes the idle equipment judgment result more realistic and reliable, providing accurate judgment objects for subsequent current comprehensive status statistics and intelligent supervision.

[0098] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the deep learning-based construction equipment status monitoring method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0099] This application also provides a construction equipment status monitoring device based on deep learning, please refer to... Figure 5 , Figure 5 This is a schematic diagram of the module structure of a deep learning-based construction equipment status monitoring device, which includes: The data acquisition module 10 is used to acquire image data of the construction site and positioning data of each construction equipment. The image data is collected by a high-altitude panoramic monitoring camera deployed at the construction site, and the positioning data is collected by a positioning device installed on the construction equipment. The mapping relationship construction module 20 is used to perform spatiotemporal registration on the image data and the positioning data, and to establish a target association mapping relationship between the positioning data of the construction equipment and the corresponding construction equipment in the image data of the construction site. The idle time judgment module 30 is used to perform multi-level idle time judgment on each construction equipment based on the target association mapping relationship, and to determine the actual idle time equipment in each construction equipment according to the multi-level idle time judgment results. The multi-level idle time judgment includes global motion judgment, local motion judgment and surrounding environment judgment. The status monitoring module 40 is used to statistically analyze the current comprehensive status of the real idle equipment and monitor the real idle equipment based on the current comprehensive status, which includes idle duration, idle frequency, and idle period.

[0100] This embodiment acquires image data of the construction site and positioning data of each construction device through a data acquisition module, providing a unified data source integrating visual and positioning modalities for status monitoring. A mapping relationship construction module performs spatiotemporal registration on the image data and positioning data, establishing a target association mapping relationship between the positioning data of the construction device and the corresponding construction device within the image data of the construction site. This achieves a precise one-to-one mapping between physical equipment, positioning data, and visual images, breaking down data silos between the two types of heterogeneous data. An idle work judgment module performs multi-level idle work judgment on each construction device based on the target association mapping relationship, including global motion judgment, local motion judgment, and surrounding environment judgment. Based on the results of the multi-level idle work judgment, it identifies the actual idle equipment among the construction devices, using a hierarchical judgment logic of progressive screening and gradually increasing precision to accurately distinguish between the fixed-point operation status and the idle work status of the construction equipment. This approach completely resolves the core pain point of misjudgment in single-location monitoring. Through a status monitoring module, it statistically analyzes the current comprehensive status of the actual idle equipment, including idle duration, idle frequency, and idle periods. Based on this comprehensive status, it monitors the actual idle equipment, transforming idle equipment judgment results into quantifiable, traceable, and decision-making control data, forming a closed-loop control system of "status judgment—real-time early warning—statistical analysis—scheduling optimization." Because this embodiment achieves deep fusion of positioning and visual dual-modal data, multi-level hierarchical accurate judgment, and status statistical early warning monitoring through the division of labor and collaboration among various modules, it organically unifies these technologies. Furthermore, by employing optimization methods such as pre-positioning coarse screening and local priority inference, it significantly reduces system computing power consumption and inference latency. This enables automated, high-precision, low-latency, and comprehensive intelligent monitoring of construction equipment idle status, significantly improving the level of intelligent and refined management at construction sites and reducing equipment idle costs.

[0101] The deep learning-based construction equipment status monitoring device provided in this application employs the deep learning-based construction equipment status monitoring method described in the above embodiments, and can solve the technical problems to be addressed by the deep learning-based construction equipment status monitoring method. Compared with the prior art, the beneficial effects of the deep learning-based construction equipment status monitoring device provided in this application are the same as those of the deep learning-based construction equipment status monitoring method provided in the above embodiments, and other technical features in the deep learning-based construction equipment status monitoring device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0102] This application provides a construction equipment status monitoring device based on deep learning. The construction equipment status monitoring device based on deep learning includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the construction equipment status monitoring method based on deep learning in any of the above embodiments.

[0103] The following is for reference. Figure 6 This document illustrates a structural diagram of a deep learning-based construction equipment status monitoring device suitable for implementing embodiments of this application. The deep learning-based construction equipment status monitoring device in this application embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Android Devices), PMPs (Portable Media Players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The deep learning-based construction equipment status monitoring device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0104] like Figure 6As shown, the deep learning-based construction equipment status monitoring device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM 1002 (Read Only Memory) or a program loaded from storage device 1003 into RAM 1004 (Random Access Memory). RAM 1004 also stores various programs and data required for the operation of the deep learning-based construction equipment status monitoring device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. I / O interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the deep learning-based construction equipment status monitoring device to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows deep learning-based construction equipment status monitoring devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0105] The deep learning-based construction equipment status monitoring device provided in this application employs the deep learning-based construction equipment status monitoring method described in the above embodiments, and can solve the technical problems to be addressed by the deep learning-based construction equipment status monitoring method. Compared with the prior art, the beneficial effects of the deep learning-based construction equipment status monitoring device provided in this application are the same as the beneficial effects of the deep learning-based construction equipment status monitoring method provided in the above embodiments, and other technical features in this deep learning-based construction equipment status monitoring device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0106] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0107] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0108] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the deep learning-based construction equipment status monitoring method in the above embodiments.

[0109] The computer-readable storage medium provided in this application can be a tangible medium capable of storing computer programs, such as a hard disk, random access memory, read-only memory, flash memory, optical storage device, or magnetic storage device. When the computer program is executed by a processor, it implements the aforementioned deep learning-based construction equipment status monitoring method.

[0110] The aforementioned computer-readable storage medium may be included in a deep learning-based construction equipment status monitoring device; or it may exist independently and not be assembled into a deep learning-based construction equipment status monitoring device.

[0111] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the deep learning-based construction equipment status monitoring device, the deep learning-based construction equipment status monitoring device performs the deep learning-based construction equipment status monitoring method as described in the above embodiments.

[0112] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0113] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0114] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned deep learning-based construction equipment status monitoring method, and is capable of solving the technical problems to be solved by the aforementioned deep learning-based construction equipment status monitoring method. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the deep learning-based construction equipment status monitoring method provided in the above embodiments, and will not be repeated here.

[0115] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for monitoring the status of construction equipment based on deep learning, characterized in that, The deep learning-based construction equipment status monitoring method includes: Image data of the construction site and positioning data of each construction equipment are acquired. The image data is collected by high-altitude panoramic monitoring cameras deployed at the construction site, and the positioning data is collected by positioning devices installed on the construction equipment. Spatiotemporal registration is performed on the image data and the positioning data, and a target association mapping relationship is established between the positioning data of the construction equipment and the corresponding construction equipment in the image data of the construction site. Based on the target association mapping relationship, multi-level idle time judgment is performed on each construction equipment. The actual idle time equipment in each construction equipment is determined according to the multi-level idle time judgment results. The multi-level idle time judgment includes global motion judgment, local motion judgment and surrounding environment judgment. The current overall status of the actual idle equipment is statistically analyzed, and the actual idle equipment is monitored based on the current overall status, which includes idle duration, idle frequency, and idle period.

2. The deep learning-based construction equipment status monitoring method as described in claim 1, characterized in that, The step of performing spatiotemporal registration of the image data and the positioning data, and establishing a target association mapping relationship between the positioning data of the construction equipment and the corresponding construction equipment in the image data of the construction site, includes: The image data is calibrated and corrected to obtain standard image data; The spatial correlation mapping relationship is obtained by mapping and transforming the actual geographic coordinate system of the construction site corresponding to the positioning data and the image pixel coordinate system corresponding to the standard image data. Based on the timestamp information of the positioning data and the standard image data, the positioning data and the standard image data are time-aligned to obtain a time-related mapping relationship; Assign a corresponding electronic identifier to each construction equipment target in the standard image data; Based on the electronic identifier, the spatial association mapping relationship, and the temporal association mapping relationship, a target association mapping relationship is established between each construction equipment target in the image data and the positioning coordinates of each construction equipment in the positioning data.

3. The deep learning-based construction equipment status monitoring method as described in claim 2, characterized in that, The global motion determination includes: The idle time and total motion of each construction equipment are calculated based on the target association mapping relationship, and the idle time and total motion of each construction equipment are compared with preset thresholds. The total motion includes displacement, motion speed and position offset amplitude, and the preset thresholds include preset time thresholds and preset total motion thresholds. If the idle time of the construction equipment is greater than the preset time threshold and the total amount of exercise is less than the preset total amount of exercise threshold, then the construction equipment is marked as the first suspected idle equipment.

4. The deep learning-based construction equipment status monitoring method as described in claim 3, characterized in that, The local motion determination includes: Based on the target association mapping relationship, obtain multiple frames of image data corresponding to the first suspected idle equipment; Based on the multi-frame image data, it is determined whether the first suspected idle equipment is in a completely stationary state. If the first suspected idle equipment is in a completely stationary state, then the first suspected idle equipment is marked as the second suspected idle equipment.

5. The deep learning-based construction equipment status monitoring method as described in claim 4, characterized in that, The assessment of the surrounding environment includes: Based on the target association mapping relationship, local image data of the second suspected idle equipment is obtained, and the local image data of the second suspected idle equipment is input into a pre-trained target scene semantic recognition model. The target scene semantic recognition model is configured to recognize the surrounding environment of the second suspected idle equipment. If the surrounding environment of the second suspected idle equipment is not identified as a valid construction working area, then the second suspected idle equipment will be marked as a real idle equipment.

6. The deep learning-based construction equipment status monitoring method as described in claim 5, characterized in that, Before inputting the local image data of the second suspected idle equipment into the pre-trained target scene semantic recognition model, the following steps are included: The initial scene semantic recognition model is compressed and modified to obtain an intermediate scene semantic recognition model. The model compression and modification includes structured pruning, channel simplification and quantization compression. Configure a frame interval adaptive inference mechanism and a local ROI priority inference strategy for the intermediate scene semantic recognition model to obtain the target scene semantic recognition model.

7. The deep learning-based construction equipment status monitoring method as described in claim 5, characterized in that, The monitoring of the actual idle equipment based on the current overall status includes: The current overall status will be output in the form of multiple reports and early warning information will be pushed. The multiple reports include single equipment operation efficiency analysis reports and overall equipment management and control analysis data of the construction site.

8. A deep learning-based construction equipment status monitoring device that applies the deep learning-based construction equipment status monitoring method according to any one of claims 1 to 7, characterized in that, The device includes: The data acquisition module is used to acquire image data of the construction site and positioning data of each construction equipment. The image data is collected by high-altitude panoramic monitoring cameras deployed at the construction site, and the positioning data is collected by positioning devices installed on the construction equipment. The mapping relationship construction module is used to perform spatiotemporal registration of the image data and the positioning data, and to establish a target association mapping relationship between the positioning data of the construction equipment and the corresponding construction equipment in the image data of the construction site. The idle time judgment module is used to perform multi-level idle time judgment on each construction equipment based on the target association mapping relationship, and to determine the actual idle time equipment in each construction equipment according to the multi-level idle time judgment results. The multi-level idle time judgment includes global motion judgment, local motion judgment and surrounding environment judgment. The status monitoring module is used to statistically analyze the current comprehensive status of the actual idle equipment and monitor the actual idle equipment based on the current comprehensive status, which includes idle duration, idle frequency, and idle period.

9. A construction equipment status monitoring device based on deep learning, characterized in that, The deep learning-based construction equipment status monitoring device includes: a memory, a processor, and a deep learning-based construction equipment status monitoring program stored in the memory. The processor is used to run the deep learning-based construction equipment status monitoring program, which is configured to implement the deep learning-based construction equipment status monitoring method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a deep learning-based construction equipment status monitoring program, which, when executed by a processor, implements the deep learning-based construction equipment status monitoring method as described in any one of claims 1 to 7.

Images