A building construction monitoring method and system based on image recognition

Abstract

This application provides a construction monitoring method and system based on image recognition. By spatially decoupling personnel, machinery, and operational structure elements in continuous construction images, corresponding element sub-image sets are obtained. The pixel proportion of each construction element within the construction area is determined based on these sub-image sets. The stability of construction behavior during continuous operation is determined using all pixel proportions. Based on this stability, staged image recognition of the construction process is performed, and the deviation between the actual and planned construction states is determined according to the trajectory of construction behavior changes within each stage. The existence of abnormal construction states at the construction site is determined based on the deviation states, and the construction monitoring results are output. This solution enables structural decoupling of different construction elements in construction images under dynamic multi-element changes at construction sites, thereby achieving accurate determination of deviations in the construction process state.

Description

Technical Field

[0001] This application relates to the field of construction monitoring technology, and more specifically, to a construction monitoring method and system based on image recognition. Background Technology

[0002] Construction monitoring refers to the process of continuously observing and managing personnel activities, machinery operation, work structure status, and construction progress at the construction site through technical means during the construction process. Its purpose is to ensure construction safety, standardize construction behavior, and improve construction organization efficiency.

[0003] However, existing image- or video-based construction monitoring solutions primarily focus on identifying single targets such as personnel, machinery, or safety equipment. They typically employ single-frame image analysis or isolated event detection, lacking a systematic portrayal of the dynamic processes at construction sites. On one hand, current technologies struggle to quantify the spatial distribution of different construction elements, resulting in monitoring results that remain at the level of "whether a target has appeared," failing to reflect the overall state of construction activity. On the other hand, existing solutions do not adequately address the continuous operation characteristics and stage evolution features of the construction process, making it difficult to determine whether construction activity is stable or whether the construction progress deviates from the plan, thus exhibiting lag and uncertainty in identifying abnormal construction states. Furthermore, existing construction monitoring results often lack clear image-based correlation with construction specifications, making it difficult for the identification results to directly support construction management decisions. Therefore, how to structurally decouple different construction elements in construction images within the context of dynamic changes across multiple elements at construction sites to accurately determine deviations in the construction process state has become a significant challenge for the industry. Summary of the Invention

[0004] This application provides a construction monitoring method and system based on image recognition, which can decouple different construction elements in construction images in scenarios where multiple elements are dynamically changing at the construction site, so as to accurately determine the deviation of the construction process status.

[0005] In a first aspect, this application provides a construction monitoring method based on image recognition, comprising the following steps: Collect continuous construction images from multiple perspectives at the building construction site; Spatially decouple the personnel, machinery, and work structure elements in the continuous construction images to obtain corresponding element sub-image sets, and then determine the pixel proportion of the corresponding construction element in the construction area based on the pixel regions in each element sub-image set. By measuring the changes in the proportion of all pixels over time, the stability of construction activities during continuous operation can be determined. Based on the stability during continuous operation, the construction process is subjected to staged image recognition, and the deviation between the actual construction state and the planned construction state is determined according to the trajectory of construction behavior changes in each construction stage. The system matches the deviation state with the image judgment boundary corresponding to the preset construction specification constraints to determine whether there is an abnormal construction state at the construction site and outputs the construction monitoring results.

[0006] In conjunction with the first aspect, in one possible implementation, spatial decoupling of personnel elements, machinery elements, and work structure elements in the continuous construction images to obtain a corresponding set of element sub-graphs specifically includes: The continuous construction images are segmented to identify and distinguish personnel, machinery, and work structure areas in the images; Based on the results obtained from instance segmentation, the pixel regions occupied by each construction element are extracted to generate corresponding personnel element sub-maps, machinery element sub-maps, and work structure element sub-maps. Each element subgraph is optimized to obtain the corresponding set of element subgraphs.

[0007] In conjunction with the first aspect, in one possible implementation, determining the pixel proportion of the corresponding construction element within the construction area based on the pixel regions in each element sub-map set specifically includes: For each time frame in each set of feature sub-maps, extract the non-background valid pixel regions from the personnel, machinery, and work structure feature sub-maps within that frame; Determine the total pixel area range of the construction area in the image corresponding to each time frame; The pixel proportion of each construction element in this time frame is calculated from the non-background effective pixel area of ​​each construction element and the total pixel area of ​​the corresponding construction area. Iterate through all time frames to generate the pixel percentage of each construction element within the construction area.

[0008] In conjunction with the first aspect, one possible implementation method involves determining the stability of construction behavior during continuous operation by analyzing the changes in the proportion of all pixels over time. This specifically includes: Define and apply a sliding time window to the time series data corresponding to the pixel proportion of each construction element; Determine the dispersion index of the pixel proportion of each construction element within each sliding time window; The stability of construction behavior during continuous operation is determined based on all dispersion indicators.

[0009] In conjunction with the first aspect, in one possible implementation, performing staged image recognition of the construction process based on the stability during continuous operation specifically includes: The stability during continuous operation is preprocessed to smooth out random fluctuations in the data; Based on the numerical characteristics of the stability sequence obtained after preprocessing, the continuous construction time axis is divided into multiple construction stages with different stability characteristics. Each construction stage is assigned a stage identifier and associated with its corresponding start and end timestamps and image frame range to complete staged image recognition.

[0010] In conjunction with the first aspect, in one possible implementation, determining the deviation between the actual construction state and the planned construction state based on the trajectory of changes in construction behavior within each construction stage specifically includes: Obtain the planned construction status corresponding to each construction stage; For each construction stage, the actual construction behavior change trajectory corresponding to that construction stage is extracted from the actual pixel proportion time series collected. By comparing the actual construction behavior change trajectory within the same construction phase with the corresponding expected change trajectory in the planned construction state, the difference between the two in terms of time sequence and spatial distribution is calculated to determine the deviation state of the construction phase.

[0011] In conjunction with the first aspect, in one possible implementation, the deviation state is matched with the image judgment boundary corresponding to the preset construction specification constraints to determine whether there is an abnormal construction state at the construction site, and the construction monitoring results are output, specifically including: Obtain the preset construction specification constraints and quantify them into image judgment boundaries that match the deviation state dimension; The deviation state corresponding to each construction stage is matched and compared with the image judgment boundary; Based on the matching comparison results, it is determined whether there are abnormal construction conditions at the construction site, and structured construction monitoring results are generated.

[0012] Secondly, this application provides a construction monitoring system based on image recognition, comprising: The acquisition module is used to acquire continuous construction images from multiple perspectives at the construction site. The processing module is used to spatially decouple personnel elements, mechanical elements and work structure elements in the continuous construction images to obtain corresponding element sub-image sets, and then determine the pixel proportion of the corresponding construction element in the construction area based on the pixel area in each element sub-image set. The processing module is also used to determine the stability of construction behavior during continuous operation by measuring the changes in the proportion of all pixels over time. The processing module is also used to perform phased image recognition of the construction process based on the stability during continuous operation, and to determine the deviation between the actual construction state and the planned construction state based on the trajectory of construction behavior changes in each construction stage. The execution module is used to match the deviation state with the image judgment boundary corresponding to the preset construction specification constraints, determine whether there is an abnormal construction state at the construction site, and output the construction monitoring results.

[0013] Thirdly, this application provides a computer device, which includes a memory and a processor. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device performs the above-described image recognition-based construction monitoring method.

[0014] Fourthly, this application provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the aforementioned image recognition-based construction monitoring method.

[0015] The technical solution provided in this application has the following beneficial effects: This application's solution, firstly, spatially decouples personnel, machinery, and operational structure elements in the continuous construction images to obtain corresponding element sub-image sets. This allows each type of construction element to have clear spatial boundaries and independent data carriers within the image, thus laying a structured foundation for construction status analysis in scenarios with dynamic changes in multiple elements. Secondly, by analyzing the changes in the proportion of all pixels over time, the stability of construction behavior during continuous operation is determined. This directly reflects whether construction behavior is continuous, balanced, or frequently changing, enabling construction monitoring to have a holistic perception capability of the dynamic evolution trend of multiple elements. Then, based on the trajectory of construction behavior changes within each construction stage, the deviation between the actual construction state and the planned construction state is determined. This transforms construction monitoring from simple state detection to process segmentation analysis that conforms to construction logic, systematically identifying the construction process over time. The consistency deviations in sequence and spatial distribution transform the construction status from a vague subjective judgment into an objective judgment based on stage characteristics and behavioral trajectories. This effectively improves the accuracy of identifying deviations in the construction process status under dynamic, multi-element environments. Finally, by matching the deviation status with the image judgment boundary corresponding to the preset construction specification constraints, it is determined whether there is an abnormal construction status at the construction site. This allows abnormal construction status to be identified promptly and objectively and output as monitoring results, forming a complete closed loop of structural decoupling, stability analysis, and deviation judgment. This enables accurate judgment of deviations in the construction process status under complex, multi-element construction scenarios. In summary, this solution can structurally decouple different construction elements in construction images under dynamic, multi-element scenarios at construction sites to achieve accurate judgment of deviations in the construction process status. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is an exemplary flowchart of a construction monitoring method based on image recognition, according to some embodiments of this application; Figure 2 This is an exemplary flowchart illustrating the implementation of spatial decoupling according to some embodiments of this application; Figure 3 This is an exemplary flowchart illustrating the determination of stability during a continuous operation process according to some embodiments of this application; Figure 4This is a structural schematic diagram of a building construction monitoring system based on image recognition, according to some embodiments of this application; Figure 5 This is a schematic diagram of the structure of a computer device for implementing an image recognition-based construction monitoring method according to some embodiments of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] refer to Figure 1 The figure is an exemplary flowchart of a construction monitoring method based on image recognition, according to some embodiments of this application. The construction monitoring method based on image recognition mainly includes the following steps: In step 101, continuous construction images from multiple perspectives of the building construction site are acquired.

[0020] In specific implementation, the acquisition of continuous construction images from multiple perspectives at a construction site can be achieved in the following way: First, multiple fixed or mobile image acquisition devices are deployed at key locations in the construction area to form a visual sensing network covering the main work surface; all acquisition devices are synchronized in time via the network; then, the central server controls each device to synchronously start continuous image acquisition at a preset frame rate, and the mobile devices can adjust the viewing angle according to preset rules; the acquired multi-channel image streams are transmitted to the server in real time or near real time, and are aligned and packaged according to device identifiers and timestamps to form time-aligned multi-view image data packets; finally, the server performs standardized preprocessing on the images from each perspective, which may include resolution unification, distortion correction, and brightness equalization, and adds perspective and timestamp metadata, outputting spatiotemporally aligned continuous construction images; other methods can also be used for acquisition in other embodiments, which are not specifically limited here.

[0021] It should be noted that, in this application, multi-view continuous construction images refer to a sequence of construction site images composed of consecutive frames acquired synchronously or quasi-synchronously from multiple spatial perspectives in chronological order.

[0022] In step 102, the personnel elements, machinery elements and work structure elements in the continuous construction image are spatially decoupled to obtain the corresponding element sub-image set, and then the pixel proportion of the corresponding construction element in the construction area is determined according to the pixel area in each element sub-image set.

[0023] In some embodiments, reference Figure 2 As shown, this figure is an exemplary flowchart for implementing spatial decoupling in some embodiments of this application. In this embodiment, the spatial decoupling of personnel elements, mechanical elements, and work structure elements in the continuous construction image to obtain the corresponding element sub-graph set can be achieved by the following steps: First, in step 1021, the continuous construction images are segmented to identify and distinguish personnel, machinery, and work structure areas in the images; Secondly, in step 1022, based on the results obtained from instance segmentation, the pixel regions occupied by each construction element are extracted to generate corresponding personnel element sub-maps, machinery element sub-maps, and work structure element sub-maps. Finally, in step 1023, each feature subgraph is optimized to obtain the corresponding feature subgraph set.

[0024] In specific implementation, instance segmentation processing is performed on the continuous construction images to identify and distinguish personnel, machinery, and work structure regions in the images. This can be achieved in the following way: the spatiotemporally aligned continuous construction images can be input frame by frame into a pre-trained instance segmentation neural network model, such as a model based on a masked region convolutional neural network architecture and finely tuned for the construction scene. This model predicts each pixel in the input image and outputs instance masks with different labels corresponding to the categories of "personnel," "machinery," "work structure," and "background." Each instance mask is a binary matrix used to identify the precise pixel position of a specified target instance in the image, thereby distinguishing the construction elements in the image that have different semantics and spatial locations. Other methods can also be used in other embodiments, which are not limited here.

[0025] In specific implementation, based on the results of instance segmentation, the pixel regions occupied by each construction element are extracted to generate corresponding personnel element sub-images, machinery element sub-images, and work structure element sub-images. This can be achieved in the following way: According to the category labels and instance masks output by the instance segmentation model, the pixels in the original construction image are classified and extracted; for each type of element, such as "personnel", all instance masks labeled as that type in the frame image are traversed, and the original image pixels corresponding to each mask region are copied to the same coordinate position on a new blank canvas with a solid black background, thereby generating a single-element image containing only the visual information of that type of element; this operation is performed on all "personnel" type instances and composited onto the same canvas to obtain the personnel element sub-image of that frame; in the same way, the "machinery" and "work structure" categories are processed separately, and the corresponding machinery element sub-images and work structure element sub-images are generated synchronously; this process is repeated for each frame of the continuous image sequence, and finally, a sub-image sequence grouped by element category corresponding to the original image sequence frames is obtained.

[0026] In specific implementation, optimizing each element subgraph to obtain the corresponding element subgraph set can be achieved in the following way: post-processing each generated element subgraph to improve its quality and the reliability of subsequent analysis; firstly, morphological operations, such as opening operations with erosion followed by dilation, can be applied to eliminate small noise points caused by inaccurate segmentation, such as missegmented isolated pixels, and to smooth element edges; secondly, to address potential issues such as internal voids within the same element instance or adhesion between different instances, algorithms based on connected component analysis can be used to filter regions with excessively small pixel areas and to separate spatially adjacent and similarly characterized adhesion regions based on preset geometric or color difference thresholds; finally, the three types of element subgraphs in each frame after optimization, namely personnel, machinery, and work structure, are organized to form a data structure indexed sequentially by time frame, with each frame containing three channelized subgraphs, which serves as the corresponding element subgraph set; other methods can also be used in other embodiments to determine this, which are not limited here.

[0027] It should be noted that, in this application, construction elements refer to the categories of visual objects representing the core activity entities at the construction site in continuous construction images, including construction scenes such as personnel, machinery, and work structures; the element sub-graphs in this application refer to images containing only visual information of a single category of construction elements obtained by spatial decoupling from the original construction images; and the element sub-graph set in this application refers to a structured dataset organized in time frame order, containing three categories of element sub-graphs: personnel, machinery, and work structures corresponding to each frame. It is used to systematically encapsulate the spatial state snapshots of all element categories at each moment during the entire continuous construction process, forming a spatiotemporally aligned multi-channel data cube.

[0028] In some embodiments, determining the pixel proportion of a corresponding construction element within a construction area based on pixel regions in each feature sub-map set can be achieved using the following steps: For each time frame in each set of feature sub-maps, extract the non-background valid pixel regions from the personnel, machinery, and work structure feature sub-maps within that frame; Determine the total pixel area range of the construction area in the image corresponding to each time frame; The pixel proportion of each construction element in this time frame is calculated from the non-background effective pixel area of ​​each construction element and the total pixel area of ​​the corresponding construction area. Iterate through all time frames to generate the pixel percentage of each construction element within the construction area.

[0029] In specific implementation, for each time frame in each element sub-graph set, the extraction of non-background valid pixel regions from the personnel, machinery, and work structure element sub-graphs within that frame can be achieved in the following way: For each time frame in each element sub-graph set, read the three element sub-graphs for personnel, machinery, and work structure respectively; since the background of the element sub-graph is a preset solid color, such as black, with an RGB value of (0, 0, 0), by traversing all pixels in the element sub-graph, the areas with pixel values ​​not equal to the background color are determined as the non-background valid pixel regions of the corresponding construction element; for each element category, use an algorithm based on connected component analysis to count the total number of pixels contained in all non-background valid pixel regions, which is taken as the effective pixel area of ​​the construction element in the current frame.

[0030] In specific implementation, the total pixel range of the construction area in the image corresponding to each time frame can be determined in the following way: the original construction image can be processed based on the construction site layout map acquired synchronously with the continuous construction images or through a pre-trained semantic segmentation model to identify the pixel range of the construction area in the image; this range is a static or slowly changing binary mask, in which the pixel value marked as the construction area is 1, and the non-construction area, such as outside the wall or living area, is 0; for each time frame, this mask is directly applied or the mask boundary is dynamically adjusted according to the image of the frame, and the total number of pixels with a value of 1 in the mask is counted. This value is the total pixel range of the construction area in the image corresponding to the current frame.

[0031] In specific implementation, the pixel proportion of each construction element in a given time frame can be calculated by the non-background effective pixel area of ​​each construction element and the total pixel area of ​​the corresponding construction area. This can be achieved in the following way: the effective pixel area of ​​each construction element in the current frame (i.e., personnel, machinery, and work structure) obtained in the non-background effective pixel area of ​​each construction element in the first step is divided by the total pixel area of ​​the construction area in the same frame obtained in the second step, i.e., the total pixel area of ​​the construction area corresponding to the same frame in the image. The three ratios obtained are usually decimals or percentages between 0 and 1, which are used as the pixel proportion of each construction element in that time frame. Similarly, the pixel proportion of each construction element in the construction area is obtained by traversing all time frames. Other methods can also be used to determine this in other embodiments, which are not limited here.

[0032] It should be noted that, in this application, the non-background effective pixel region refers to the connected region composed of all pixels that do not belong to the preset background color in the feature sub-image of a certain time frame. It is used to quantitatively characterize the actual visual space occupied by a specified type of construction element, such as personnel, machinery, or work structure, in the image of that frame. In this application, the total pixel area of ​​the construction area refers to all pixel areas in the image defined for each time frame that belong to the construction activity to be monitored. In this application, the pixel proportion refers to the ratio of the area of ​​the non-background effective pixel region of a certain construction element, such as personnel, machinery, or work structure, to the area of ​​the total pixel area of ​​the construction area within a given single time frame. It is used to characterize the spatial distribution density or visual salience of various construction elements within the effective construction area at that specified moment in a standardized numerical form.

[0033] In step 103, the stability of the construction behavior during continuous operation is determined by the change in the proportion of all pixels over time.

[0034] In some embodiments, reference Figure 3 As shown, this figure is an exemplary flowchart of determining the stability during continuous operation in some embodiments of this application. The stability of construction behavior during continuous operation can be determined by the changes in the proportion of all pixels over time using the following steps: Define and apply a sliding time window to the time series data corresponding to the pixel proportion of each construction element; Determine the dispersion index of the pixel proportion of each construction element within each sliding time window; The stability of construction behavior during continuous operation is determined based on all dispersion indicators.

[0035] In specific implementation, defining and applying a sliding time window to the time series data corresponding to the pixel proportion of each construction element can be achieved in the following way: the pixel proportion sequence of each construction element arranged in chronological order is used as input; a sliding time window containing N consecutive sampling points (e.g., N=30, corresponding to 30 frames or a physical duration) is set and slid sequentially along the time axis with a preset step size (e.g., 1 sampling point); for each slide, N consecutive proportion values ​​falling into the window are extracted from the time series data corresponding to the pixel proportion of each construction element, forming an independent window proportion quantum sequence for each construction element under the time window.

[0036] In specific implementation, the dispersion index of the pixel proportion of each construction element within each sliding time window can be determined in the following way: For each sliding time window, the dispersion of the quantum sequence of pixel proportion of each construction element, personnel, machinery, and work structure within that window is calculated; the dispersion index can be obtained by calculating the standard deviation or coefficient of variation of the subsequence; wherein, the standard deviation directly reflects the fluctuation range of the proportion value relative to the mean within the window, while the coefficient of variation, i.e., the standard deviation divided by the mean, eliminates the influence of the mean of different elements and is used to characterize the relative intensity of the fluctuation; each time window finally outputs three values, representing the dispersion of the pixel proportion changes of personnel, machinery, and work structure within that window period.

[0037] In specific implementation, the stability of construction behavior during continuous operation can be determined based on all dispersion indicators in the following way: the dispersion indicators of each construction element obtained in each time window are comprehensively processed and mapped into a stability value that characterizes the concentration of changes in the overall construction behavior within that window period. Specifically, the average of the dispersion indicators of the three elements within the window can be calculated, and then the reciprocal of the average can be taken or a preset linear mapping function can be used to transform it into a stability score between 0 and 1, where a higher score indicates more concentrated changes and more stable behavior. This process is executed continuously, and a stability score is output for each sliding window, thereby forming a time series describing the change in the stability of construction behavior over time. This time series serves as the stability of construction behavior during continuous operation. Other methods can also be used to determine this in other embodiments, which are not limited here.

[0038] It should be noted that the sliding time window in this application refers to dividing a continuous construction process into a series of overlapping or continuous local time periods in the time dimension; the dispersion index in this application refers to a numerical measure used to quantify the fluctuation or dispersion of the pixel proportion value of a certain construction element around its average value within a predetermined sliding time window, which can characterize the variation range or consistency of the spatial distribution density of each construction element within that local time period; the stability of construction behavior in the continuous operation process in this application refers to an evaluation index used to characterize the degree of concentration of changes in construction behavior within a predetermined time window.

[0039] In step 104, the construction process is subjected to staged image recognition based on the stability during continuous operation, and the deviation between the actual construction state and the planned construction state is determined based on the trajectory of construction behavior changes in each construction stage.

[0040] In some embodiments, the staged image recognition of the construction process based on the stability during continuous operation can be achieved using the following steps: The stability during continuous operation is preprocessed to smooth out random fluctuations in the data; Based on the numerical characteristics of the stability sequence obtained after preprocessing, the continuous construction time axis is divided into multiple construction stages with different stability characteristics. Each construction stage is assigned a stage identifier and associated with its corresponding start and end timestamps and image frame range to complete staged image recognition.

[0041] In specific implementation, the stability during continuous operation is preprocessed to smooth out random fluctuations in the data. This can be achieved by taking the time series corresponding to the stability during continuous operation as input and smoothing it using a sliding window averaging filter or Gaussian filter algorithm. For example, a smoothing window containing M continuous stability sampling points, such as M=5, is set. The arithmetic mean of the data within the window is calculated sequentially as the new stability value at the center point of the window. The window is then slid with a step size of 1 until the entire sequence is traversed, thereby generating a smooth stability curve that filters out short-term high-frequency fluctuations.

[0042] In specific implementation, based on the numerical characteristics of the stability sequence obtained after preprocessing, dividing the continuous construction timeline into multiple construction stages with different stability characteristics can be achieved in the following way: Based on the smoothed stability curve, a threshold-based state machine or change point detection algorithm is used for stage segmentation; for example: first, a high stability threshold and a low stability threshold are set, such as 0.7 and 0.4 respectively. When the stability curve value is continuously higher than the high stability threshold for more than a preset time, such as 10 consecutive sampling points, the time period is determined to belong to the high stability stage, such as the stable operation stage; when the stability curve value is continuously lower than the low stability threshold for more than a preset time, it is determined to be the low stability stage, such as the intermittent or preparation stage; the area between the two or the transition area where the threshold is crossed is determined to be the medium stability stage, such as the adjustment or transition stage; by detecting the crossing and persistence status of the stability curve relative to these thresholds, the entire construction timeline is automatically divided into multiple time periods with different stability characteristics that appear alternately, and each continuous time period is a preliminary construction stage.

[0043] In specific implementation, assigning a stage identifier to each construction stage and associating it with its corresponding start and end timestamps and image frame range can be achieved in the following way: Based on the time period results obtained from the previous stage division, generate a unique stage identifier for each construction stage, such as "Phase_001" or "High Stability Stage_1", and record the start and end indices of the stage in the stability sequence; according to the timestamp-image frame index mapping table, convert the above start and end sequence indices into precise start and end timestamps, and further locate the number range of all continuous construction image frames contained in the time period; finally, output a stage list, each item in the list containing a stage identifier, start and end timestamps, start and end image frame numbers, and the stability feature classification of the stage, thereby structurally dividing the entire continuous image sequence into multiple construction stages and completing staged image recognition; other methods can also be used in other embodiments, which are not limited here.

[0044] It should be noted that, in this application, construction stages with different stability characteristics refer to a continuous time period in which the internal construction behavior changes are similarly concentrated, dividing the continuous construction process on the time axis; the stage identifier in this application refers to a label or code used to uniquely identify and represent an identified construction stage, which can be used as an index and association key to bind the divided construction stage with its corresponding start and end times, image frame range, stability characteristics and other metadata, so as to accurately and unambiguously reference and manipulate all relevant data within the specified stage.

[0045] In some embodiments, determining the deviation between the actual construction state and the planned construction state based on the trajectory of changes in construction behavior within each construction stage can be achieved using the following steps: Obtain the planned construction status corresponding to each construction stage; For each construction stage, the actual construction behavior change trajectory corresponding to that construction stage is extracted from the actual pixel proportion time series collected. By comparing the actual construction behavior change trajectory within the same construction phase with the corresponding expected change trajectory in the planned construction state, the difference between the two in terms of time sequence and spatial distribution is calculated to determine the deviation state of the construction phase.

[0046] In practice, the planned construction status corresponding to each construction stage can be obtained in the following way: extract the planned information corresponding to the current monitoring task from the 4D construction simulation data of the building information model or the preset digital construction schedule; the planned information includes the time range of each planned construction stage and the corresponding expected spatial distribution density standard of each construction element (i.e., personnel, machinery, and work structure) in the construction area. This standard is pre-converted into the expected time series of the pixel proportion of each construction element that matches the resolution of the monitoring image; according to the start and end timestamps of the stage identification output, the planned time axis is aligned with the actual stage time axis, so as to match and obtain the corresponding expected change trajectory data as a benchmark for each actual construction stage, that is, the planned construction status.

[0047] In practice, for each construction stage, the extraction of the actual construction behavior change trajectory corresponding to that construction stage from the actual collected pixel proportion time series can be achieved in the following way: based on the start and end image frame numbers of each construction stage output by the stage identification, extract all continuous pixel proportion data points that are completely within the time range of that stage from the generated pixel proportion time series of each construction element; for the three elements of personnel, machinery, and work structure, arrange their proportion sequences in the stage in chronological order to form a multi-dimensional time series, which serves as a quantitative representation of the actual construction behavior change trajectory within that construction stage.

[0048] In practice, comparing the actual construction behavior change trajectory within the same construction phase with the corresponding expected change trajectory in the planned construction state, and calculating the difference between the two in terms of time sequence and spatial distribution, can determine the deviation state of that construction phase. This can be achieved as follows: For each construction phase, align its actual construction behavior change trajectory with the corresponding expected change trajectory in the planned construction state on the time axis, for example, using a dynamic time warping algorithm to allow non-linear time scaling matching; subsequently, based on the time alignment, calculate the difference between the actual and expected proportion sequences of personnel, machinery, and work structure at each corresponding time point, and then calculate the difference between these sequences. Spatial distribution differences can be obtained by calculating the Euclidean distance or mean absolute error of the difference vectors of each construction element throughout the entire construction stage. Temporal sequence differences can be measured by analyzing the time offset of key change points such as peaks and inflection points in the actual and expected sequences. Finally, the calculated spatial distribution differences and temporal sequence differences are weighted and fused, and mapped to a comprehensive deviation score through a preset normalization function, such as a value between 0 and 1. This score is used as the deviation state of the construction stage. The higher the score, the greater the consistency deviation between the actual and the planned in time and space. Other methods can also be used to determine this in other embodiments, which are not limited here.

[0049] It should be noted that, in this application, the planned construction state refers to a pre-defined digital description of the ideal state of the construction process, which defines the expected standards that construction elements such as personnel, machinery, and work structures should achieve in spatial distribution during each planned construction period; the actual construction behavior change trajectory in this application refers to the real dynamic evolution process characterizing the spatial distribution density of various construction elements at the construction site within a specific construction stage; the expected change trajectory corresponding to the planned construction state in this application refers to a subset of planned construction state data matching a specific construction stage, specifically manifested as the ideal change curve of the pixel proportion of each construction element in the construction stage according to the time sequence; the spatial distribution difference in this application refers to the overall deviation between the actual construction trajectory and the expected construction trajectory in terms of element spatial density; the temporal sequence difference in this application refers to the degree of misalignment between the actual construction trajectory and the expected construction trajectory in terms of the occurrence sequence or change rhythm of key construction behavior events; and the deviation state in this application refers to an evaluation index used to characterize the degree of deviation between the actual construction progress and the plan in terms of spatiotemporal coordination consistency.

[0050] In step 105, the deviation state is matched with the image judgment boundary corresponding to the preset construction specification constraints to determine whether there is an abnormal construction state at the construction site, and the construction monitoring result is output.

[0051] In some embodiments, the following steps can be used to determine whether there is an abnormal construction state at the construction site and output the construction monitoring results by matching the deviation state with the image judgment boundary corresponding to the preset construction specification constraints: Obtain the preset construction specification constraints and quantify them into image judgment boundaries that match the deviation state dimension; The deviation state corresponding to each construction stage is matched and compared with the image judgment boundary; Based on the matching comparison results, it is determined whether there are abnormal construction conditions at the construction site, and structured construction monitoring results are generated.

[0052] In specific implementation, obtaining preset construction specification constraints and quantifying them into image judgment boundaries that match the deviation state dimension can be achieved in the following way: extracting constraint clauses related to personnel distribution density, machinery operation range, structural spatial state, and process connection sequence from relevant construction safety standards, quality management procedures, or specific project operation instructions; converting each clause into a calculable rule, for example, quantifying "material stacking must not occupy safety passages" into an upper limit threshold for the percentage of pixels of work structure elements in the passage area, or quantifying "key process conversion must be completed within a predetermined time window" into an allowable range for the change in stability or time sequence between stages; finally, organizing all quantified rules into a rule base, where each rule defines its applicable construction stage type or all stages and one or more judgment boundary values ​​compared with the deviation state. This rule base and its set of boundary values ​​constitute the image judgment boundary. Other methods can also be used to determine this boundary in other embodiments, which are not specifically limited here.

[0053] In specific implementation, the deviation state corresponding to each construction stage can be matched and compared with the image judgment boundary in the following way: traverse all construction stages obtained by stage-based identification, and for each stage, read its deviation state and its corresponding spatial distribution difference and temporal sequence difference; according to the type identifier of the construction stage, such as "high stability stage" or key element features, retrieve all applicable judgment rules from the rule base corresponding to the image judgment boundary; compare the difference data of the current stage with the boundary value set by each retrieved rule one by one to determine whether it exceeds the boundary, such as whether the spatial distribution difference is greater than the safety threshold or whether the temporal sequence difference exceeds the allowable time difference; this matching and comparison process outputs a list of violations for the current construction stage, which records all triggered rules and their degree of exceeding the limit.

[0054] In specific implementation, determining whether there is an abnormal construction state at the construction site based on the matching comparison results and generating a structured construction monitoring result can be achieved in the following way: if the list of violations in any construction stage is not empty, then it is determined that there is an abnormal construction state at the construction site; integrate the judgment results of all construction stages to generate a structured construction monitoring result; this construction monitoring result is usually output in the form of a report, and its structure includes at least: the overall conclusion of whether there is an anomaly, the stage identifier and start and end time corresponding to each abnormal construction stage, the specific standard clauses triggered, the quantified over-limit data, such as the actual proportion exceeding the planned value by 15%, and preliminary suggested measures based on the rule base predefined, such as "issue an area intrusion alarm" or "prompt the process delay"; this construction monitoring result can be displayed through a visual interface, send alarm messages, or be written to a database for management personnel to make decisions; other methods can also be used in other embodiments, which are not specifically limited here.

[0055] It should be noted that the construction specification constraints in this application refer to mandatory or guiding rules and regulations from industry standards, safety procedures, or project requirements that should be followed by personnel, machinery, materials, and procedures at the construction site; the image judgment boundary in this application refers to the conversion and quantification of textual construction specification constraints into numerical thresholds or rule sets that can be directly compared with deviation states and their components; the abnormal construction state in this application refers to the site conditions characterized by a construction stage that, after matching and comparison, is determined to be a deviation state or whose components exceed the allowable range of the image judgment boundary. It is used to comprehensively and clearly indicate that an unexpected, high-risk, or inefficient operational situation has occurred at the construction site that is inconsistent with the plan and violates established specifications.

[0056] In another aspect, in some embodiments, this application provides a construction monitoring system based on image recognition, see reference. Figure 4 The figure is a schematic diagram of the structure of a construction monitoring system based on image recognition according to some embodiments of this application. The construction monitoring system 400 based on image recognition includes: a data acquisition module 401, a processing module 402, and an execution module 403, which are described below: Acquisition module 401, in this application, is mainly used to acquire continuous construction images from multiple perspectives at the construction site; Processing module 402, in this application, is mainly used to spatially decouple personnel elements, mechanical elements and work structure elements in the continuous construction image to obtain corresponding element sub-image sets, and then determine the pixel proportion of the corresponding construction element in the construction area based on the pixel area in each element sub-image set. The processing module 402 described in this application is also used to determine the stability of the construction behavior during continuous operation by measuring the changes in the proportion of all pixels over time. The processing module 402 described in this application is also used to perform phased image recognition of the construction process based on the stability during continuous operation, and to determine the deviation between the actual construction state and the planned construction state based on the trajectory of changes in construction behavior within each construction stage. The execution module 403 in this application is mainly used to match the deviation state with the image judgment boundary corresponding to the preset construction specification constraints, determine whether there is an abnormal construction state at the construction site, and output the construction monitoring results.

[0057] The foregoing has detailed examples of the image recognition-based construction monitoring method and system provided in this application. It is understood that the corresponding apparatus, in order to achieve the above functions, includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0058] In some embodiments, this application also provides a computer device, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for calling and running the computer program from the memory, so that the computer device performs the above-described image recognition-based building construction monitoring method.

[0059] In some embodiments, reference Figure 5 The dashed lines in the figure indicate that the unit or module is optional. This figure is a structural schematic diagram of the computer equipment implementing the image recognition-based construction monitoring method of this application. The image recognition-based construction monitoring method in the above embodiments can... Figure 5 The computer device 500 shown is used to implement this, and the computer device 500 includes at least one processor 501, a memory 502 and at least one communication unit 505. The computer device 500 may be a terminal device, a server or a chip.

[0060] The processor 501 can be a general-purpose processor or a special-purpose processor. For example, the processor 501 can be a central processing unit (CPU). The CPU can be used to control the computer device 500, execute software programs, and process data from the software programs. The computer device 500 may also include a communication unit 505 for inputting (receiving) and outputting (transmitting) signals.

[0061] For example, computer device 500 may be a chip, communication unit 505 may be the input and / or output circuit of the chip, or communication unit 505 may be the communication interface of the chip, and the chip may be a component of terminal device, network device or other device.

[0062] For example, computer device 500 may be a terminal device or a server, and communication unit 505 may be a transceiver of the terminal device or the server, or communication unit 505 may be a transceiver circuit of the terminal device or the server.

[0063] The computer device 500 may include one or more memories 502 storing a program 504. The program 504 can be executed by a processor 501 to generate instructions 503, causing the processor 501 to perform the methods described in the above method embodiments according to the instructions 503. Optionally, the memory 502 may also store data (such as a target audit model). Optionally, the processor 501 may also read data stored in the memory 502, which may be stored at the same storage address as the program 504, or the data may be stored at a different storage address than the program 504.

[0064] The processor 501 and memory 502 can be configured separately or integrated together, for example, integrated on the system-on-chip (SOC) of the terminal device.

[0065] It should be understood that each step of the above method embodiment can be completed by hardware logic circuits or software instructions in the processor 501. The processor 501 can be a CPU, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0066] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0067] For example, in some embodiments, this application also provides a computer-readable storage medium storing instructions or code that, when executed on a computer, cause the computer to implement the above-described image recognition-based construction monitoring method.

[0068] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0069] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A construction monitoring method based on image recognition, characterized in that, Includes the following steps: Collect continuous construction images from multiple perspectives at the building construction site; Spatially decouple the personnel, machinery, and work structure elements in the continuous construction images to obtain corresponding element sub-image sets, and then determine the pixel proportion of the corresponding construction element in the construction area based on the pixel regions in each element sub-image set. By measuring the changes in the proportion of all pixels over time, the stability of construction activities during continuous operation can be determined. Based on the stability during continuous operation, the construction process is subjected to staged image recognition, and the deviation between the actual construction state and the planned construction state is determined according to the trajectory of construction behavior changes in each construction stage. The system matches the deviation state with the image judgment boundary corresponding to the preset construction specification constraints to determine whether there is an abnormal construction state at the construction site and outputs the construction monitoring results.

2. The method as described in claim 1, characterized in that, Spatially decoupling the personnel, machinery, and work structure elements in the continuous construction images to obtain the corresponding set of element sub-images specifically includes: The continuous construction images are segmented to identify and distinguish personnel, machinery, and work structure areas in the images; Based on the results obtained from instance segmentation, the pixel regions occupied by each construction element are extracted to generate corresponding personnel element sub-maps, machinery element sub-maps, and work structure element sub-maps. Each element subgraph is optimized to obtain the corresponding set of element subgraphs.

3. The method as described in claim 1, characterized in that, Determining the pixel proportion of the corresponding construction element within the construction area based on the pixel regions in each element sub-map set specifically includes: For each time frame in each set of feature sub-maps, extract the non-background valid pixel regions from the personnel, machinery, and work structure feature sub-maps within that frame; Determine the total pixel area range of the construction area in the image corresponding to each time frame; The pixel proportion of each construction element in this time frame is calculated from the non-background effective pixel area of ​​each construction element and the total pixel area of ​​the corresponding construction area. Iterate through all time frames to generate the pixel percentage of each construction element within the construction area.

4. The method as described in claim 1, characterized in that, Determining the stability of construction activities during continuous operation by analyzing the changes in the proportion of all pixels over time includes: Define and apply a sliding time window to the time series data corresponding to the pixel proportion of each construction element; Determine the dispersion index of the pixel proportion of each construction element within each sliding time window; The stability of construction behavior during continuous operation is determined based on all dispersion indicators.

5. The method as described in claim 1, characterized in that, The phased image recognition of the construction process based on the stability during continuous operation specifically includes: The stability during continuous operation is preprocessed to smooth out random fluctuations in the data; Based on the numerical characteristics of the stability sequence obtained after preprocessing, the continuous construction time axis is divided into multiple construction stages with different stability characteristics. Each construction stage is assigned a stage identifier and associated with its corresponding start and end timestamps and image frame range to complete staged image recognition.

6. The method as described in claim 1, characterized in that, The deviation between the actual construction state and the planned construction state is determined based on the trajectory of changes in construction behavior within each construction stage. This deviation specifically includes: Obtain the planned construction status corresponding to each construction stage; For each construction stage, the actual construction behavior change trajectory corresponding to that construction stage is extracted from the actual pixel proportion time series collected. By comparing the actual construction behavior change trajectory within the same construction phase with the corresponding expected change trajectory in the planned construction state, the difference between the two in terms of time sequence and spatial distribution is calculated to determine the deviation state of the construction phase.

7. The method as described in claim 1, characterized in that, The system matches the deviation state with the image judgment boundary corresponding to the preset construction specification constraints to determine whether there is an abnormal construction state at the construction site, and outputs the construction monitoring results, specifically including: Obtain the preset construction specification constraints and quantify them into image judgment boundaries that match the deviation state dimension; The deviation state corresponding to each construction stage is matched and compared with the image judgment boundary; Based on the matching comparison results, it is determined whether there are abnormal construction conditions at the construction site, and structured construction monitoring results are generated.

8. A construction monitoring system based on image recognition, characterized in that, include: The acquisition module is used to acquire continuous construction images from multiple perspectives at the construction site. The processing module is used to spatially decouple personnel elements, mechanical elements and work structure elements in the continuous construction images to obtain corresponding element sub-image sets, and then determine the pixel proportion of the corresponding construction element in the construction area based on the pixel area in each element sub-image set. The processing module is also used to determine the stability of construction behavior during continuous operation by measuring the changes in the proportion of all pixels over time. The processing module is also used to perform phased image recognition of the construction process based on the stability during continuous operation, and to determine the deviation between the actual construction state and the planned construction state based on the trajectory of construction behavior changes in each construction stage. The execution module is used to match the deviation state with the image judgment boundary corresponding to the preset construction specification constraints, determine whether there is an abnormal construction state at the construction site, and output the construction monitoring results.

9. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to call and run the computer programs from the memory, so that the computer device performs the construction monitoring method based on image recognition as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions or code that, when executed on a computer, cause the computer to implement the image recognition-based construction monitoring method as described in any one of claims 1 to 7.

Images