Image-based method for supervising the approach of a construction equipment

Abstract

The application discloses a kind of based on image's building equipment access supervision method, belong to building equipment technical field.The method includes: before access, equipment and delivery order image are collected, automatically identify information and complete quantity check and appearance preliminary inspection;In unloading, equipment is continuously tracked by image network, and real-time picture is compared with initial image automatically, realize real-time monitoring and alarm of damage;After access, nameplate information is automatically identified, and three-way comparison is carried out with delivery order, preset information to complete model check.The method can also realize intelligent damage identification based on packaging, large equipment hoisting safety analysis and compliance pre-inspection based on BIM.The application realizes the automation, intelligent closed-loop supervision of equipment access whole process, significantly improves the supervision efficiency, accuracy and safety.

Description

Technical Field

[0001] This invention belongs to the field of building equipment technology, specifically relating to an image-based method for monitoring the entry of building equipment. Background Technology

[0002] In the construction industry, the arrival of equipment (such as large electromechanical equipment, steel structural components, and piping modules) is a crucial step in ensuring the smooth progress of a project, and the effectiveness of its supervision directly affects equipment quality, construction safety, and project schedule. Traditional supervision of construction equipment arrival mainly relies on manual methods.

[0003] However, the aforementioned traditional manual supervision model has revealed many significant defects and limitations in practical applications: Heavy reliance on human resources and low coordination efficiency: The smooth progress of the process depends heavily on the simultaneous presence of personnel from multiple departments. The temporary absence or delay of any party will cause the entire process to be interrupted or delayed. This is especially true for large projects and multi-shift operations, where personnel coordination is difficult and seriously affects the efficiency of material flow.

[0004] Manual verification is inefficient, error-prone, and time-consuming: Faced with a large number of equipment of various models arriving on site, manual counting and verification is arduous and time-consuming. Not only is the verification speed slow, affecting subsequent installation procedures, but also, under conditions of fatigue, negligence, or poor lighting and angles, it is very easy to miss or misidentify items, creating hidden dangers for project quality.

[0005] Process monitoring suffers from blind spots, making real-time and objectivity difficult: Supervision of the unloading process relies entirely on human eyes, leading to visual fatigue and distraction, making continuous, comprehensive monitoring of equipment movement impossible. For stacked or partially obscured equipment, effective quantity counting and visual inspection are impossible. Furthermore, manual damage assessment is subjective and lacks objective, traceable records.

[0006] Information verification is cumbersome, and data silos are a serious problem: Model verification involves manual comparison of multiple sources of information, such as delivery notes, physical nameplates, and design drawings, which is tedious and prone to errors. The results of each stage of inspection are mostly in the form of paper records or scattered electronic documents, making it difficult to form a structured, interconnected, and traceable unified digital archive, which is not conducive to subsequent problem tracing and responsibility determination.

[0007] Although information technologies such as image recognition and the Internet of Things have been applied in the industrial sector in recent years, a key technical challenge remains to be solved in the specific and complex on-site scenario of construction equipment entry. This challenge is how to develop an intelligent monitoring method that can adapt to complex stacking conditions, overcome environmental interference, replace multi-disciplinary manual collaboration, and achieve closed-loop data management throughout the entire process. Current technologies lack a systematic solution that can deeply integrate intelligent image analysis technology with the professional business processes of construction equipment entry to completely resolve the aforementioned pain points regarding efficiency, accuracy, and collaboration. Summary of the Invention

[0008] In view of this, the purpose of the present invention is to provide an image-based method for monitoring the entry of construction equipment.

[0009] To achieve the above objectives, the present invention provides the following technical solution: An image-based method for monitoring the entry of construction equipment divides the entry process of target equipment into three stages: appearance and quantity inspection, unloading process monitoring, and model verification, and performs the following steps in sequence: S1. Pre-entry appearance and quantity inspection: Use image acquisition equipment to capture images of the target equipment and delivery note and upload them to the data processing center; The data processing center recognizes the uploaded images, extracts the delivery note information and counts the number of equipment, and feeds back the quantity comparison results and the preliminary analysis results of the equipment appearance to the terminal equipment; S2. Uninterrupted monitoring of the unloading process: Deploy an image acquisition network in the unloading area, transportation channel and placement area to continuously acquire and track images of the target equipment during the unloading process; throughout the unloading, transportation and placement process of the target equipment, compare its current appearance image with the initial appearance image before unloading in real time, and feed the real-time comparison results back to the terminal equipment. S3. Model verification upon arrival: Use image acquisition equipment to photograph the nameplate of the target equipment and upload it to the data processing center; The data processing center identifies and extracts the nameplate information, compares the nameplate information with the delivery note information and the preset equipment information, and feeds back the comparison results to the terminal equipment.

[0010] As a further preferred embodiment of the present invention, S1 specifically includes the following steps: S11. Image information acquisition: Use image acquisition equipment to take pictures of the target equipment arriving at the construction site, obtain panoramic images containing all target equipment, and at the same time take pictures of the delivery note, and upload the panoramic image and the delivery note image to the data processing center. S12. Processing Delivery Note Information: The data processing center receives the delivery note image and performs the following sub-steps: S12a. Perform semantic segmentation on the delivery note image to locate and segment the image region containing information on the equipment model and quantity. S12b: Perform text recognition on each segmented image region, extract a structured list of equipment models and quantity information, and generate delivery note data; S13. Processing Equipment Image Information: The data processing center receives the panoramic image and performs the following sub-steps: S13a. Perform semantic segmentation on the panoramic image to identify the outlines of individual devices in the image; S13b: Count the identified individual devices to obtain a statistical value of the number of devices on site; S13c. Based on the identified individual device outline images, perform a preliminary analysis of appearance integrity. S14. Information Comparison and Feedback: Compare the quantity information in the delivery note extracted in S12b with the on-site equipment quantity statistics obtained in S13b to generate quantity verification results; integrate the quantity verification results with the preliminary equipment appearance analysis results in S13c to form an inspection report and feed it back to the receiving personnel's terminal equipment.

[0011] As a further preferred embodiment of the present invention, S2 specifically includes the following steps: S21. Deployment and Initialization of Monitoring: Deploy multiple image acquisition devices in the unloading vehicle parking area, unloading equipment transportation channel and equipment temporary placement area to form an image acquisition network covering the entire unloading process area; S22. Lock the target equipment and acquire an initial appearance image: When the target equipment starts unloading, the target equipment is identified and locked through the image acquisition network, and before it is blocked or moved by the unloading equipment, images of the equipment from multiple perspectives are acquired and fused to generate an initial appearance image of the target equipment. S23. Continuous monitoring and real-time comparison: During the entire process of the target equipment being unloaded from the vehicle and moved through the transportation channel to the designated placement location, images of the equipment are continuously acquired through an image acquisition network, and the current appearance image acquired in real time is compared with the initial appearance image to monitor the consistency of the appearance status. S24. Anomaly detection and alarm: If, during the real-time comparison process in S23, the feature similarity between the current appearance image and the initial appearance image is lower than a preset threshold, it is determined to be an appearance anomaly. An unloading damage alarm containing the target equipment identifier and the time of the anomaly is generated and sent to the terminal device of the supervisor. S25. Monitoring Termination Confirmation: When the target equipment is placed in the designated location and remains stationary, if continuous monitoring reaches the preset time and no further appearance abnormalities are detected, then the monitoring of the unloading process of the target equipment is confirmed to have ended.

[0012] As a further preferred embodiment of the present invention, S3 specifically includes the following steps: S31. Acquire nameplate images: For each target device that has been unloaded and placed, use an image acquisition device to take a picture of its nameplate, acquire the nameplate image, and upload the nameplate image to the data processing center. S32. Processing and Extracting Nameplate Information: The data processing center receives the nameplate image and performs the following sub-steps: S32a. Preprocess the nameplate image to enhance the text area in the image; S32b: Perform text recognition on the preprocessed nameplate image, extract the equipment model, specifications and serial number information contained on the nameplate, and form structured nameplate data; S33. Perform third-party information comparison: The data processing center performs the following comparison operations: S33a. Obtain the delivery note data that has been extracted and stored in S1; S33b, Obtain the preset design drawing equipment information table or equipment warehousing information table; S33c, Compare the nameplate data extracted in S32b with the corresponding information in the delivery note data and the equipment information table or equipment warehousing information table in the design drawings; S34. Generate and provide feedback on verification results: Based on the comparison results in S33c, generate a model verification report for each device. The report includes details of whether the information is consistent, inconsistent, or missing. Send the model verification report to the terminal device of the inspector.

[0013] As a further preferred embodiment of the present invention, the unloading process monitoring also includes a hoisting safety analysis step for the identified large-scale construction electromechanical equipment: when the data processing center identifies the target equipment as large-scale electromechanical equipment to be hoisted based on the initial appearance image, the following safety analysis process is executed: Lifting point and center of gravity feature recognition: Based on the initial appearance image, identify the special lifting lugs, center of gravity symbols or asymmetric structural features of the target equipment body; Real-time monitoring of hoisting status: During the hoisting process, the actual connection status of the slings and lugs is identified from the real-time image stream, and the real-time attitude and tilt angle of the target equipment in the air are calculated; Safety Status Analysis and Early Warning: Based on the location of the dedicated lifting lugs or center of gravity markers, the connection status of the slings, and the real-time attitude and tilt angle, the force distribution of each lifting point is estimated through a preset mechanical balance model. If the analysis results indicate that there is a single point of excessive force, severely uneven force, or overturning risk exceeding the threshold, a graded safety alarm is immediately generated and sent to the terminal devices of the lifting command personnel and safety monitoring personnel.

[0014] As a further preferred embodiment of the present invention, the preliminary analysis of appearance integrity in step S13c specifically includes the following steps: Equipment packaging type identification: Based on the outline image of the individual equipment identified in the panoramic image, analyze its surface texture, color distribution and edge features to determine the packaging type of the equipment. Packaging types include wooden box packaging, bare metal surface and waterproof film wrapping. Call the corresponding damage feature library: The data processing center has a pre-stored damage feature library corresponding to the packaging type, including: For wooden crate packaging, the damage feature library contains feature templates for broken panels, deformed frames, and loose or missing nails; For bare metal surfaces, the damage feature library contains feature templates for dents, scratches, and corrosion. For waterproof membrane wrapping, the damage feature library includes feature templates for membrane tears, punctures, and edge delamination. Targeted defect detection and rating: Based on the identified packaging type, the corresponding damage feature library is called to perform matching detection within the individual equipment contour image area to identify whether there is a corresponding specific defect; for the identified defects, a comprehensive damage impact rating is calculated based on their size, quantity, and location on the equipment surface.

[0015] As a further preferred embodiment of the present invention, prior to the appearance and quantity inspection stage before the arrival of S1, a pre-inspection step for compliance based on the construction BIM model is also included, specifically including: BIM Model Data Extraction: Based on the construction schedule, the data processing center extracts the spatial boundary dimensions of the target equipment's intended installation location, the cross-sectional dimensions of the transport channel from the unloading area to the installation point, and the preset handling path from the project's Building Information Model (BIM). Visual estimation of on-site equipment dimensions: After acquiring panoramic images containing the target equipment in the S1 stage, the data processing center estimates the key external dimensions of the target equipment based on the panoramic images and using known reference object proportions or preset camera calibration parameters. The key external dimensions include the maximum length, width and height of the equipment. Automatic spatial compliance comparison: The estimated key external dimensions of the equipment are compared and analyzed with the corresponding spatial boundary dimensions of the installation location and the cross-sectional dimensions of the transportation channel extracted from the BIM model; Conflict warning and decision support: If the comparative analysis results show that the equipment size exceeds the allowable range of the reserved space or the channel size, the data processing center will determine that there is a risk of spatial conflict and generate spatial conflict warning information containing the specific conflict location and size deviation. This information will be sent to the relevant terminal equipment before unloading begins to support the entry decision.

[0016] The beneficial effects of this invention are as follows: This invention divides the process into three stages and automates them using images and a centralized platform. Inspections at each stage (appearance and quantity, unloading monitoring, and model verification) are automatically completed by image acquisition equipment and a data processing center, with results fed back to terminal devices. This significantly reduces the reliance on multiple departments being present simultaneously and on-site manual monitoring throughout the entire process. Staff can receive reports and alarms via terminals, achieving desynchronization and remote monitoring of the process, greatly alleviating personnel coordination pressure and improving the overall efficiency of material flow.

[0017] In terms of quantity verification, this invention automatically extracts the quantity of delivery notes and identifies and counts individual devices in the image through image semantic segmentation and optical character recognition (OCR), replacing manual counting and verification. This method is faster, more accurate, and avoids missed or incorrect inspections due to fatigue, negligence, or environmental factors. In terms of appearance inspection, it automatically matches, detects, and rates damage based on a smart damage feature library of packaging type, providing a more objective and standardized preliminary appearance integrity analysis than human visual inspection, reducing subjective misjudgments.

[0018] This invention constructs an image acquisition network covering the entire process of unloading, transportation, and placement, enabling uninterrupted and continuous image acquisition and tracking of equipment. By comparing real-time images with initial images for feature analysis, it can automatically and in real-time monitor changes in appearance, and automatically trigger an alarm once an anomaly is detected (similarity below a threshold). This completely eliminates visual blind spots and attention interruptions inherent in manual supervision, achieving comprehensive, objective, and traceable digital monitoring of the unloading process.

[0019] This invention proposes an automated three-way comparison of nameplate information, delivery note information, and pre-set equipment information (such as BIM model data). This automates the tedious verification process that previously required manually reviewing multiple documents, quickly generating verification reports and significantly reducing the probability of errors. The entire method revolves around a data processing center, uniformly processing and structurally storing the entire chain of images and data (delivery note data, equipment images, nameplate data, BIM data, etc.) from before to after equipment arrival, naturally forming a unified, structured digital archive of equipment arrival. The results at each stage are interconnected, providing a clear and complete chain of evidence for problem tracing and responsibility determination, effectively breaking down traditional data silos.

[0020] The added intelligent analysis of hoisting safety, through image recognition of hoisting points, monitoring of posture, and estimation of stress, enables proactive safety warnings for high-risk operations, compensating for the shortcomings of traditional supervision that relies solely on experience. The introduced BIM model-based pre-inspection of on-site compliance can provide early warnings of dimensional and spatial conflicts before equipment unloading, avoiding the passive situation where equipment cannot be installed or transported after arrival. This method constructs a complete, closed-loop intelligent supervision process from pre-inspection (BIM) → on-site inspection (appearance and quantity) → process monitoring (unloading) → final verification (model), deeply integrating intelligent image analysis into all aspects of the business, systematically improving the intelligence level, operational efficiency, accuracy, and safety of construction equipment on-site supervision. Attached Figure Description

[0021] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration: Figure 1 This is a schematic diagram of the construction equipment entry process of the present invention; Figure 2 This is a schematic diagram of the entrance shooting for the present invention; Figure 3 This is a schematic diagram of the unloading process area of ​​the present invention. Detailed Implementation

[0022] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the following embodiments are for illustrative purposes only and are not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0023] like Figures 1-3 As shown, this invention discloses an image-based method for monitoring the entry of construction equipment. The method divides the entry process of the target equipment into three stages: appearance and quantity inspection, unloading process monitoring, and model verification, and executes the following steps sequentially: S1. Pre-delivery Appearance and Quantity Inspection: Image acquisition equipment is used to capture images of the target equipment and delivery note, which are then uploaded to the data processing center. The data processing center identifies the uploaded images, extracts delivery note information, and counts the number of equipment. The quantity comparison results and preliminary analysis of the equipment appearance are fed back to the terminal equipment. Upon arrival at the construction site, receiving personnel use image acquisition equipment to photograph the equipment to be received, capturing images of all equipment from any angle. After photographing, the images are uploaded to the data processing equipment for quantity and appearance inspection. The inspection results are returned to the receiving personnel's terminal equipment. If the returned results confirm that the equipment appearance and quantity are correct, the next stage can proceed. If problems exist, manual processing is required. Manual processing also allows the option to proceed to the next stage of the process.

[0024] S2. Uninterrupted monitoring of the unloading process: Deploy an image acquisition network in the unloading area, transport channel, and placement area to continuously acquire and track images of the target equipment during the unloading process; throughout the entire unloading, transport, and placement process of the target equipment, compare its current appearance image with its initial appearance image before unloading in real time, and feed the real-time comparison results back to the terminal equipment. In the unloading process area (e.g., Figure 2 Several mobile image acquisition devices are deployed to provide real-time monitoring of the area from all angles. During the transportation process, the program uses the received images to identify and lock onto the target equipment to be unloaded, providing continuous monitoring from unloading to placement on the ground.

[0025] Equipment unloading: Multi-angle image acquisition equipment is installed on the unloading side of the delivery vehicle to continuously monitor the equipment on the vehicle. When the unloading equipment enters the image monitoring range and obstructs the monitored equipment, the background program extracts the image data before obstruction as the initial first frame image data of the target equipment. When the unloading equipment unloads the target equipment, the target equipment moves. After detecting the movement, the program locks the mobile device as the detection object for continuous detection, while acquiring images from other perspectives. The program locks the target in each perspective and acquires the initial appearance image data of the equipment, thus obtaining the initial appearance image data of the target equipment during transportation.

[0026] Equipment Transportation: Unloading equipment transports the locked target equipment, while several image acquisition devices along the route continuously acquire image data of the target equipment. During transportation, the background program continuously compares the appearance image data with the initial image data of the equipment. If the equipment image changes, the program will notify management personnel to inspect the equipment's appearance.

[0027] Equipment Placement: After the unloading equipment arrives at the unloading area, the equipment is placed. Once placed, the equipment will not move again. When the background program detects that the equipment is no longer moving, it will save the final image data of the target equipment and compare it with the equipment image at the start of unloading. If the equipment image has changed and no manual intervention was performed during transportation, the program will also notify the management personnel to inspect the equipment's appearance.

[0028] S3. Model Verification Upon Arrival: Image acquisition equipment is used to photograph the nameplate of the target equipment and upload the image to the data processing center. The data processing center identifies and extracts the nameplate information, performs a three-way comparison with the delivery note information and preset equipment information, and feeds the comparison results back to the terminal equipment. Inspectors use image acquisition equipment to acquire image data from the equipment nameplate. After data upload, the background program automatically parses and compares the data from the photographed equipment, the delivery list, and the equipment information table on the drawings, returning the comparison results to the inspectors' terminal equipment.

[0029] The implementation of the method described in this invention depends on the following hardware and software systems: Image acquisition equipment includes, but is not limited to, high-definition cameras, industrial cameras, drones, or mobile terminals with camera functions, deployed at key locations on the construction site (such as gates, unloading areas, passageways, and storage areas). It is used to acquire image data of the target area or target object, including mobile and fixed equipment; the target detection area includes transport vehicle areas, unloading areas, and transport passageways; the target object includes delivery notes, construction machinery and equipment, and equipment nameplates; the image data includes image data and video data; all acquired image data will be uploaded to data processing equipment, and the background program will analyze the relevant content according to the on-site entry procedures.

[0030] Data processing center: This can be a cloud server or a local server, equipped with sufficient computing and storage resources to run core algorithm software. The center receives image data and performs image recognition, analysis, comparison, and logical judgment. Data processing equipment uses the acquired image data and image recognition programs to perform various required processing on the acquired image data. When necessary, the program will stitch together multiple image data to obtain complete target image data.

[0031] Visual Inspection and Quantity Check Stage: Delivery Note Information Analysis. Image acquisition equipment collects image data from the paper delivery note and uploads it to the data processing equipment. The background program performs semantic segmentation on the obtained image data, that is, it segments important information such as equipment model and quantity on the delivery note into regions. The segmented region image data is assigned corresponding information tags by the program. After each region is tagged, the program automatically recognizes the image text information within each region and converts it into formatted data. After text recognition is completed, the relevant results are saved for subsequent steps. If the delivery party provides an electronic delivery note, the information from the paper version and the electronic version will be verified.

[0032] Equipment Appearance and Quantity Analysis: Delivery vehicle image data is used to obtain equipment appearance information and perform an appearance inspection. Simultaneously, the quantity is verified by comparing the data with the delivery note image. The program uses semantic segmentation to identify equipment in the images, counts the number of identified devices, inspects the equipment appearance, and compares the statistical data with the delivery note data.

[0033] During the unloading phase, the pre-defined "unloading-transportation-placement" unloading process area is monitored by images or videos. The background program continuously identifies the equipment during the unloading process to ensure that the equipment is under image monitoring throughout the unloading process and to monitor the integrity of the equipment appearance in real time. The image algorithm uses anchor boxes to lock onto the target equipment in the unloading phase from multiple image perspectives, monitors the entire transportation process of the target equipment, and continuously judges the appearance of the equipment to ensure that the appearance of the equipment is under continuous monitoring during the unloading phase.

[0034] During the model verification phase, the program performs semantic segmentation and information recognition of equipment parameters on the image data. The resulting text information is then compared with the equipment information in the existing design drawings and the delivery note information. The results from each stage are sent to the terminal device for notification, and any anomalies will prompt for manual inspection.

[0035] Terminal equipment: including smartphones, tablets or dedicated handheld terminals held by on-site management personnel, supervisors and operators, used to receive inspection reports, alarms and verification results from the data processing center.

[0036] Communication network: Ensures stable and high-speed transmission of image data, commands, and feedback information between field devices, data centers, and terminals.

[0037] Image recognition algorithm: Image recognition software based on YOLO, serving information related to building equipment. The main objects of recognition are: equipment type, equipment appearance, equipment nameplate, and nameplate information. The program identifies and segments the target image; the segmented image data is the basic recognition object for subsequent steps.

[0038] Text recognition algorithm: Text recognition software based on Paddle OCR for building equipment information. The main object of recognition is relevant text information in image data.

[0039] The two algorithms above are combined to form a software method for the entry of construction equipment, which is applicable to the entire process of construction equipment entry.

[0040] The steps of this invention will be explained in detail below: S11: Image Information Acquisition: After the transported equipment arrives at the designated area of ​​the construction site (such as the inspection area outside the gate), on-site personnel use image acquisition equipment (such as handheld terminals or fixed cameras) to take pictures. Two types of images need to be acquired: Panoramic images: Adjust the viewing angle and focus to ensure that a single image or a panoramic stitched image clearly and completely covers all target equipment on the transport vehicle. Delivery note images: Photograph the paper or electronic delivery note provided with the vehicle, ensuring that the text (especially the equipment model and quantity) is clearly legible. After acquisition, the panoramic images and delivery note images are uploaded to the data processing center via wireless network.

[0041] S12: Processing Delivery Note Information: After receiving the delivery note image, the data processing center performs the following automated processing: S12a: Semantic Segmentation to Locate Key Regions. A pre-trained deep learning semantic segmentation model (such as U-Net or DeepLab series) is used to analyze the delivery note image, automatically locating and segmenting table areas or text blocks containing key information such as "equipment model / name" and "quantity." This step effectively eliminates interfering information such as document headers, company logos, and irrelevant remarks. S12b: Text Recognition to Extract Structured Data. Optical Character Recognition (OCR) technology is used to recognize the text in each segmented text region. Subsequently, rule matching or natural language processing techniques are used to parse the recognized text into structured "delivery note data."

[0042] For example: Delivery note data = {Equipment A: {Model: 'Pump-1000', Quantity: 3}, Equipment B: {Model: 'Valve-DN200', Quantity: 10}, …} S13: Processing Equipment Image Information: The data processing center simultaneously processes the received panoramic images of the equipment. S13a: Semantic segmentation for individual equipment identification. An instance segmentation model optimized for industrial equipment detection (such as Mask R-CNN) is used to analyze panoramic images, identify each individual equipment in the image, and generate its precise pixel-level contour mask.

[0043] S13b: Automatic counting. Count the outline of each individual device identified in S13a to obtain the "statistical value of the number of field devices".

[0044] S13c: Preliminary analysis of appearance integrity, details are as follows: a. Equipment Packaging Type Identification: For each individual equipment contour image obtained in S13a, extract its color histogram, texture features (such as LBP, HOG), and contour shape features. Input the feature vector into a multi-classifier (such as SVM, Random Forest, or Convolutional Neural Network) to determine its packaging type. {Wooden crate packaging, bare metal surface, wrapped with waterproof membrane}Which category?

[0045] b. Access the corresponding damage feature library: The data processing center pre-stores "damage feature libraries" corresponding to three packaging types. These libraries contain feature templates or descriptions of various typical damage types on the image, for example: Characteristics of damage to wooden crates: These include broken wooden planks forming edges and areas of abrupt color changes.

[0046] Characteristics of metal dents: manifested as abnormal local reflections and irregular shadows on smooth curved surfaces.

[0047] Characteristics of waterproof membrane tearing: It manifests as linear cracks on the membrane surface and interruption of texture continuity.

[0048] c. Targeted Defect Detection and Rating: Detection: Based on the identified packaging type, within the equipment outline area, pattern matching is performed using the corresponding damage feature library, or a trained defect detection model is used for scanning to identify areas with suspected damage. Rating: For each detected suspected defect, its pixel area is calculated. Relative position coordinates on the surface of the equipment Define a comprehensive "damage impact rating". : in, It is the total area of ​​the equipment outline. It is the number of defects of the same type. It is a weighting function determined based on the location of the defect (such as its location in a stress-bearing area, corner, etc.). This is an adjustable weighting coefficient. Rating The higher the value, the more severe the potential damage.

[0049] S14: Information Comparison and Feedback: Quantity Comparison: Automatically compare the total quantity or itemized quantity in the delivery note extracted in S12b with the on-site equipment quantity statistics obtained in S13b. Result Integration: Integrate the quantity comparison results (e.g., "consistent," "missing X units / pieces") with the preliminary appearance analysis results of each piece of equipment generated in S13c (including packaging type, presence of defects, and damage rating R) to generate a structured "Pre-Arrival Inspection Report." Feedback: Push the report to the terminal devices of relevant personnel at the receiving party in real time. If there is a serious discrepancy in quantity or a high-rated (R exceeding the threshold) appearance damage, the report will highlight or alert you.

[0050] It enables automated, non-contact, and rapid inspection of the quantity and appearance of incoming equipment. Utilizing OCR and instance segmentation technology, it replaces manual counting and verification, offering speed and high accuracy. Intelligent damage analysis based on packaging type recognition provides a more objective and quantifiable preliminary appearance assessment standard than human visual inspection, reducing subjective misjudgments and providing an initial appearance benchmark for subsequent unloading process monitoring.

[0051] Uninterrupted monitoring of the unloading process: This stage aims to provide continuous and intelligent visual monitoring of the entire process of equipment unloading, handling and placement to prevent damage during the process.

[0052] S21: Deployment and Initialization Monitoring Before unloading operations begin, image acquisition equipment (cameras) are deployed in the following areas based on the site layout, forming an "image acquisition network" covering the entire workflow: the unloading vehicle parking area (multi-angle coverage); the transportation route from the vehicles to the temporary placement point; and the temporary equipment placement area. All camera timestamps are synchronized and connected to the data processing center via the network.

[0053] S22: Lock onto target equipment and acquire initial appearance image: When a piece of equipment begins to be contacted by a lifting device or forklift in preparation for unloading: 1. Target Locking: The data processing center analyzes the real-time video stream from the unloading point cameras and uses motion detection and target tracking algorithms (such as KCF and SORT) to identify and lock onto the target equipment that is about to move.

[0054] 2. Initial Image Acquisition: Before the device is completely obscured by the hoist or begins to move significantly, multiple cameras in the network are triggered to take rapid, continuous shots from different angles, acquiring a set of multi-view images. .

[0055] 3. Image Fusion: Through image registration and fusion techniques, these multi-view images are combined into an "initial appearance image" that contains more surface information about the device. And extract its global feature vector. (e.g., feature maps based on deep learning).

[0056] S23: Continuous monitoring and real-time comparison: throughout the entire process of lifting, transporting, and lowering the equipment: 1. Continuous tracking: The image acquisition network continuously acquires images from the device, and the target tracking algorithm ensures smooth switching and continuous tracking of the target between multiple camera views, generating the device's movement trajectory.

[0057] 2. Real-time feature extraction and comparison: For each frame (or keyframe) image acquired in real-time during the tracking process... Extract its feature vector .

[0058] 3. Calculate feature similarity The similarity between the current frame features and the initial features is calculated using metrics such as cosine similarity or the reciprocal of Euclidean distance. The closer the value is to 1, the more consistent the appearance.

[0059] S24: Anomaly Detection and Alarm Set a similarity threshold (e.g., 0.85). During real-time comparison, if multiple consecutive frames of images... If the device exhibits an abnormal appearance at that moment (such as a bump or scratch causing changes in surface features), the data processing center immediately generates an "unloading damage alarm," which includes: the device number / identifier, the timestamp of the abnormality, and the possible location of the abnormality (determined based on the camera's perspective). This alarm is then pushed to the on-site supervisor's terminal via the network, triggering an audible and visual alarm.

[0060] S25: Monitoring Termination Confirmation Once the device is placed in the designated location and remains stationary for a period of time (e.g., 30 seconds), the target tracking algorithm confirms that its status is stable. If no abnormal alarm is triggered during this period and within a preset time period thereafter (e.g., 2 minutes), the data processing center confirms that the monitoring of the unloading process of the device is over, records the image of its final placement location, and archives the monitoring video and log.

[0061] Safety analysis for lifting large equipment: For targets identified as large electromechanical equipment through the initial S22 image, the following additional safety analysis is performed during the lifting process: 1. Lifting point and center of gravity feature recognition: based on Image recognition technology is used to locate the dedicated lifting lugs or center of gravity markers on the equipment body. If there are no obvious markers, the asymmetric structural features and theoretical center of gravity position are estimated by analyzing the equipment's 3D point cloud (which can be reconstructed from multi-view images or a known model).

[0062] 2. Real-time monitoring of hoisting status: Connection status identification: From real-time images of the hoisting process, identify whether the slings (wire ropes, slings) are correctly threaded or connected to the lifting lugs, and determine whether there are dangerous conditions such as unhooking or single strand being stressed.

[0063] Attitude tilt angle calculation: By identifying the bounding rectangle of the device in the image and combining it with the known length, width, and height dimensions of the device, the real-time attitude tilt angle of the device relative to the horizontal plane in the air is calculated. .

[0064] 3. Safety Status Analysis and Early Warning: Establish a simplified mechanical equilibrium model. Assume the equipment is a rigid body, and the coordinates of the lifting points are known (measured from the image or obtained from the model). Total weight of equipment Center of gravity Under the static equilibrium assumption of neglecting acceleration, the theoretical forces at each suspension point are... The equilibrium equations should be satisfied.

[0065] Risk assessment: based on real-time attitude tilt angle 1. Adjust the model input based on the sling connection status. Calculate if the following applies: a) Estimating the force at any lifting point (Safe load threshold); b) The difference in force at each lifting point exceeds the proportional threshold (e.g., maximum / minimum force ratio > 3:1); c) Attitude and tilt angle (Safe tilt angle threshold).

[0066] If any risk condition is met, a graded safety alarm (such as early warning or emergency stop) will be generated immediately and sent to the terminal of the hoisting commander and safety monitor.

[0067] It achieves full coverage, blind-spot-free, and automated monitoring of the unloading process. Through real-time image comparison, it can automatically alarm at the first sign of damage, overcoming the lag of traditional manual supervision. For high-risk hoisting of large equipment, it adds vision-based proactive safety analysis, which can effectively warn of risks such as uneven stress and tipping over, improving operational safety. The entire process is digitalized and traceable.

[0068] Model verification upon arrival: This stage aims to automatically and accurately verify the nameplate information of each arriving piece of equipment to ensure consistency with the delivery note and design requirements.

[0069] S31: Acquire nameplate image Once the equipment is in place, operators use image acquisition equipment (such as a mobile terminal with macro capabilities) to take close-up photos of the nameplate of each device, ensuring that all text on the nameplate is clear. The photos of the nameplates are then uploaded to the data processing center.

[0070] S32: Processing and Extracting Nameplate Information: The data processing center processes the nameplate image. S32a: Image preprocessing: Perform operations such as grayscale conversion, contrast enhancement, binarization, and noise reduction on the image to highlight text areas and improve recognition conditions.

[0071] S32b: Text Recognition and Structuring: OCR technology is used to recognize all text in the preprocessed image. Using a predefined nameplate information template (typically containing keywords such as "model," "specification," "serial number," and "power"), keyword matching and context analysis are used to parse the identified jumbled text into structured "nameplate data," for example: Nameplate data = {Model: 'Pump-1000', Serial Number: 'SN20231028001', Power: '75kW', …} S33: Perform three-party information comparison.

[0072] The data processing center automatically retrieves data from three sources for comparison: S33a: Delivery note data (already extracted and stored in S12b).

[0073] S33b: Preset equipment information: Retrieve the "Design Drawing Equipment Information Table" or "Planned Inbound Information Table" related to this batch of equipment from the project management system or database.

[0074] S33c: Automatic three-party comparison: The "nameplate data" of the current device extracted by S32b is compared with: The model and quantity of the corresponding equipment items in the delivery note data are compared for consistency.

[0075] The model and specifications of the corresponding equipment items in the preset equipment information table are compared for consistency.

[0076] The comparison includes: whether the model number is consistent, whether the key specifications are within the allowable error range, and whether the serial number is in the expected list.

[0077] S34: Generating and Feedback of Verification Results Based on the comparison results, a "Type Verification Report" is generated for each device: Consistent: Marked as passed.

[0078] Discrepancy: Specifically list which information is inconsistent (e.g., "Nameplate model is X, delivery note record is Y").

[0079] Missing items: Marks information that is present in the preset information but is missing from the actual product nameplate or delivery note.

[0080] Compile all equipment verification reports and send them to the inspector's terminal. For equipment with "discrepancies" or missing key information, mark the report prominently and indicate that manual review or contacting the supplier is required.

[0081] The tedious and error-prone manual information verification process is fully automated. Through OCR technology and structured data comparison, cross-verification of nameplate information with multi-source documents is completed quickly and accurately, greatly improving verification efficiency and accuracy, and preventing incorrectly modeled equipment from entering the installation process due to human error.

[0082] BIM-based pre-inspection of site compliance: This solution, as an optional or preliminary step, predicts the compatibility of equipment with the site space before unloading.

[0083] 1. BIM Model Data Extraction: Based on the construction plan, the data processing center extracts the spatial boundary dimensions (length, width, height, and weight) of the target equipment's intended installation location from the project's BIM (Building Information Model) via API interface or file import. ,Width ,high ), and the planned minimum cross-sectional dimensions (such as minimum width) of the transport channel from the unloading area to the installation point. Minimum height ).

[0084] 2. Visual estimation of equipment dimensions on site: After acquiring panoramic images of the equipment in stage S1, in addition to quantity checks, the images are also used to estimate dimensions.

[0085] Method: Locate a reference object of known size in the image (such as a standard pallet or vehicle sideboard), or utilize calibrated camera parameters (intrinsic and extrinsic parameters). Estimate the key external dimensions of the target device, including maximum length, by comparing the pixel dimensions of the device outline in the image with those of the reference object, or by using perspective projection geometry. Maximum width Maximum height .

[0086] 3. Automatic Space Compliance Comparison: The estimated equipment dimensions are automatically compared and analyzed with the dimensions extracted from the BIM database. Check if the following conditions are met: Check transportation feasibility: 4. Conflict Warning and Decision Support: If any of the above conditions are not met, the data processing center determines that there is a spatial conflict risk. An early warning message is immediately generated, detailing the conflict type (e.g., "Equipment width exceeds installation space by X millimeters," "Equipment height cannot pass through Y-channel"), and attaching a schematic diagram of the conflict location in the BIM model. This warning is sent to the terminals of project management and technical personnel before unloading begins, providing them with decision support (e.g., coordinating changes to storage locations, adjusting handling routes, or contacting suppliers).

[0087] By moving the supervision process forward and utilizing visual measurement and BIM data, potential dimensional conflicts can be warned in advance before equipment is unloaded. This transforms "discovering problems after the fact" into "predicting risks before they occur," avoiding significant project delays and cost waste caused by equipment being unable to be transported or installed after it has arrived on site. This demonstrates the predictive value of intelligent supervision.

[0088] In summary, this invention, through the organic combination of the above-mentioned solutions and their detailed steps, constructs a complete, closed-loop, and intelligent construction equipment entry supervision system, encompassing pre-inspection, entry, process monitoring, and final verification. Data flows smoothly among the various solutions, and results are interconnected, collectively achieving a comprehensive improvement in efficiency compared to traditional manual supervision methods.

[0089] In this embodiment, a mobile phone with a shooting function (hereinafter referred to as a mobile phone) is a simple example of a mobile image data acquisition device and a terminal device; a camera capable of high-resolution shooting, video recording and audio recording (hereinafter referred to as a camera) is a simple example of a general image data acquisition device; and a server device with data processing function (hereinafter referred to as a server) is a data processing device.

[0090] Step 1: During the appearance and quantity inspection stage, use a mobile phone to photograph the transport vehicle carrying the equipment to be delivered, taking pictures of the equipment's appearance from various angles and from all directions (e.g., ...). Figure 2 This ensures that all appearance data of the device being photographed is recorded. The mobile phone uploads the captured images to the server, where an image recognition program checks the device's appearance based on the images and assesses its integrity. The results are then returned to the mobile phone for administrators to view.

[0091] Step 2: Unloading Stage. Taking the full video recording of unloading a single piece of equipment as an example, a number of cameras are used to monitor the unloading vehicle, transport channel, and unloading area continuously and from all angles throughout the entire process. Before unloading begins, cameras near the vehicle mark the target equipment, conduct appearance inspections, and record image data. After transport begins, the background program continuously inspects the target equipment from various angles. If the program detects a change in the appearance of the target equipment, it will notify the management personnel via mobile phone message to check the equipment status. When the equipment is transported to the unloading area, the program continues to monitor the target equipment until it is placed and no longer moved. The unloading stage monitoring of the equipment ends, and the program will return the monitoring results of this stage to the mobile phone for management personnel to view.

[0092] Step 3: During the model verification phase, the electromechanical installation unit uses a mobile phone to photograph the nameplate of each piece of equipment. After the image data is uploaded to the server, the program automatically extracts relevant information such as equipment information and parameters from the equipment nameplate, cross-compares it with the delivery note, factory order form, and equipment information sheet, and returns the results to the mobile device for management personnel to view. The phase results and image data in the above steps will become data samples for subsequent model training to improve the accuracy and robustness of the visual model.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring the entry of construction equipment based on images, characterized in that, The process for receiving the target equipment is divided into three stages: visual quantity inspection, unloading process monitoring, and model verification, and the following steps are performed sequentially: S1. Pre-entry appearance and quantity inspection: Use image acquisition equipment to capture images of the target equipment and delivery note and upload them to the data processing center; The data processing center recognizes the uploaded images, extracts the delivery note information and counts the number of equipment, and feeds back the quantity comparison results and the preliminary analysis results of the equipment appearance to the terminal equipment; S2. Uninterrupted monitoring of the unloading process: Deploy an image acquisition network in the unloading area, transportation channel and placement area to continuously acquire and track images of the target equipment during the unloading process; throughout the unloading, transportation and placement process of the target equipment, compare its current appearance image with the initial appearance image before unloading in real time, and feed the real-time comparison results back to the terminal equipment. S3. Model verification upon arrival: Use image acquisition equipment to photograph the nameplate of the target equipment and upload it to the data processing center; The data processing center identifies and extracts the nameplate information, compares the nameplate information with the delivery note information and the preset equipment information, and feeds back the comparison results to the terminal equipment.

2. The image-based monitoring method for the entry of construction equipment according to claim 1, characterized in that: S1 specifically includes the following steps: S11. Image information acquisition: Use image acquisition equipment to take pictures of the target equipment arriving at the construction site, obtain panoramic images containing all target equipment, and at the same time take pictures of the delivery note, and upload the panoramic image and the delivery note image to the data processing center. S12. Processing Delivery Note Information: The data processing center receives the delivery note image and performs the following sub-steps: S12a. Perform semantic segmentation on the delivery note image to locate and segment the image region containing information on the equipment model and quantity. S12b: Perform text recognition on each segmented image region, extract a structured list of equipment models and quantity information, and generate delivery note data; S13. Processing Equipment Image Information: The data processing center receives the panoramic image and performs the following sub-steps: S13a. Perform semantic segmentation on the panoramic image to identify the outlines of individual devices in the image; S13b: Count the identified individual devices to obtain a statistical value of the number of devices on site; S13c. Based on the identified individual device outline images, perform a preliminary analysis of appearance integrity. S14. Information Comparison and Feedback: Compare the quantity information in the delivery note extracted in S12b with the on-site equipment quantity statistics obtained in S13b to generate quantity verification results; integrate the quantity verification results with the preliminary equipment appearance analysis results in S13c to form an inspection report and feed it back to the receiving personnel's terminal equipment.

3. The image-based monitoring method for the entry of construction equipment according to claim 1, characterized in that: S2 specifically includes the following steps: S21. Deployment and Initialization of Monitoring: Deploy multiple image acquisition devices in the unloading vehicle parking area, unloading equipment transportation channel and equipment temporary placement area to form an image acquisition network covering the entire unloading process area; S22. Lock the target equipment and acquire an initial appearance image: When the target equipment starts unloading, the target equipment is identified and locked through the image acquisition network, and before it is blocked or moved by the unloading equipment, images of the equipment from multiple perspectives are acquired and fused to generate an initial appearance image of the target equipment. S23. Continuous monitoring and real-time comparison: During the entire process of the target equipment being unloaded from the vehicle and moved through the transportation channel to the designated placement location, images of the equipment are continuously acquired through an image acquisition network, and the current appearance image acquired in real time is compared with the initial appearance image to monitor the consistency of the appearance status. S24. Anomaly detection and alarm: If, during the real-time comparison process in S23, the feature similarity between the current appearance image and the initial appearance image is lower than a preset threshold, it is determined to be an appearance anomaly. An unloading damage alarm containing the target equipment identifier and the time of the anomaly is generated and sent to the terminal device of the supervisor. S25. Monitoring Termination Confirmation: When the target equipment is placed in the designated location and remains stationary, if continuous monitoring reaches the preset time and no further appearance abnormalities are detected, then the monitoring of the unloading process of the target equipment is confirmed to have ended.

4. The image-based monitoring method for the entry of construction equipment according to claim 1, characterized in that: S3 specifically includes the following steps: S31. Acquire nameplate images: For each target device that has been unloaded and placed, use an image acquisition device to take a picture of its nameplate, acquire the nameplate image, and upload the nameplate image to the data processing center. S32. Processing and Extracting Nameplate Information: The data processing center receives the nameplate image and performs the following sub-steps: S32a. Preprocess the nameplate image to enhance the text area in the image; S32b: Perform text recognition on the preprocessed nameplate image, extract the equipment model, specifications and serial number information contained on the nameplate, and form structured nameplate data; S33. Perform third-party information comparison: The data processing center performs the following comparison operations: S33a. Obtain the delivery note data that has been extracted and stored in S1; S33b, Obtain the preset design drawing equipment information table or equipment warehousing information table; S33c, Compare the nameplate data extracted in S32b with the corresponding information in the delivery note data and the equipment information table or equipment warehousing information table in the design drawings; S34. Generate and provide feedback on verification results: Based on the comparison results in S33c, generate a model verification report for each device. The report includes details of whether the information is consistent, inconsistent, or missing. Send the model verification report to the terminal device of the inspector.

5. The image-based monitoring method for the entry of construction equipment according to claim 3, characterized in that: The unloading process monitoring also includes a safety analysis step for the hoisting of identified large construction electromechanical equipment: When the data processing center identifies the target equipment as large electromechanical equipment requiring hoisting based on the initial appearance image, the following safety analysis process is executed: Lifting point and center of gravity feature recognition: Based on the initial appearance image, identify the special lifting lugs, center of gravity symbols or asymmetric structural features of the entire equipment body on the target equipment body; Real-time monitoring of hoisting status: During the hoisting process, the actual connection status of the slings and lugs is identified from the real-time image stream, and the real-time attitude and tilt angle of the target equipment in the air is calculated; Safety Status Analysis and Early Warning: Based on the location of the dedicated lifting lugs or center of gravity markers, the connection status of the slings, and the real-time attitude and tilt angle, the force distribution of each lifting point is estimated through a preset mechanical balance model. If the analysis results indicate that there is a single point of excessive force, severely uneven force, or overturning risk exceeding the threshold, a graded safety alarm is immediately generated and sent to the terminal devices of the lifting command personnel and safety monitoring personnel.

6. The image-based monitoring method for the entry of construction equipment according to claim 2, characterized in that: The preliminary analysis of appearance integrity in S13c specifically includes the following steps: Equipment packaging type identification: Based on the outline image of the individual equipment identified in the panoramic image, analyze its surface texture, color distribution and edge features to determine the packaging type of the equipment. Packaging types include wooden box packaging, bare metal surface and waterproof film wrapping. Call the corresponding damage feature library: The data processing center has a pre-stored damage feature library corresponding to the packaging type, including: For wooden crate packaging, the damage feature library contains feature templates for broken panels, deformed frames, and loose or missing nails; For bare metal surfaces, the damage feature library contains feature templates for dents, scratches, and corrosion. For waterproof membrane wrapping, the damage feature library includes feature templates for membrane tears, punctures, and edge delamination. Targeted defect detection and rating: Based on the identified packaging type, the corresponding damage feature library is called to perform matching detection within the individual equipment contour image area to identify whether there is a corresponding specific defect; for the identified defects, a comprehensive damage impact rating is calculated based on their size, quantity, and location on the equipment surface.

7. The image-based monitoring method for the entry of construction equipment according to claim 2, characterized in that: Prior to the appearance and quantity inspection phase before S1's arrival, there is also a pre-inspection step for compliance based on the construction BIM model, which specifically includes: BIM Model Data Extraction: Based on the construction schedule, the data processing center extracts the spatial boundary dimensions of the target equipment's intended installation location, the cross-sectional dimensions of the transport channel from the unloading area to the installation point, and the preset handling path from the project's Building Information Model (BIM). Visual estimation of on-site equipment dimensions: After acquiring panoramic images containing the target equipment in the S1 stage, the data processing center estimates the key external dimensions of the target equipment based on the panoramic images and using known reference object proportions or preset camera calibration parameters. The key external dimensions include the maximum length, width and height of the equipment. Automatic spatial compliance comparison: The estimated key external dimensions of the equipment are compared and analyzed with the corresponding spatial boundary dimensions of the installation location and the cross-sectional dimensions of the transportation channel extracted from the BIM model; Conflict warning and decision support: If the comparative analysis results show that the equipment size exceeds the allowable range of the reserved space or the channel size, the data processing center will determine that there is a risk of spatial conflict and generate spatial conflict warning information containing the specific conflict location and size deviation. This information will be sent to the relevant terminal equipment before unloading begins to support the entry decision.

Images