Arch dam cable machine hoisting task visual recognition method
Patent Information
- Application Number
- CN202610392340.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-27
- Publication Date
- 2026-10-23
AI Technical Summary
然而，在实际施工过程中，不同吊运任务在载荷变化、运动特征上存在重叠，难以形成清晰可分的判别边界；且传感器数据易受到施工工况波动、环境噪声等因素干扰，导致任务识别精度受限
本发明提供了一种拱坝缆机吊运任务视觉识别方法，包括：1）获取拱坝施工场景下缆机作业视频序列；2）基于步骤1）获取得到的缆机作业视频序列，通过运动检测提取拱坝施工场景中的稳定运动前景特征；3）识别视频帧中的缆机吊钩，并以缆机吊钩的检测框为基准，在缆机吊钩的检测框下方动态构造运动前景约束区域ROI；4）将步骤2）得到的稳定运动前景特征与步骤3）得到的运动前景约束区域ROI进行空间匹配，若存在匹配成功的运动区域，则判定所述缆机吊钩下方存在被吊运目标物；5）判断被吊运目标物是否为吊罐，以输出缆机处于混凝土吊运状态或辅助吊运状态，完成拱坝缆机吊运任务的识别。本发明将缆机吊运任务识别问题转化为缆机吊钩下方是否存在目标物与目标物是否为吊罐的两阶段判别，仅需获取缆机吊钩以及吊罐相关特征信息即可完成三类作业状态输出，无需对钢筋、模板、设备、渣斗等大量辅助吊运目标进行逐类标注训练，显著降低数据标注成本与模型迭代成本，能够适应现场目标类别不断变化和长尾目标频繁出现的情况。针对固定摄像机视角下的大尺度施工场景以及吊运过程中目标物尺度随时间显著变化的特点，本发明以缆机吊钩作为关键视觉锚点，根据缆机吊钩检测框在视频序列中的位置与尺度变化，同步动态调整所构建的感兴趣区域（ROI），使ROI在空间位置与覆盖范围上均具备自适应性，从而实现对动态视频中与吊运任务密切相关区域的精准约束与提取。上述基于吊钩锚点的动态ROI构造机制适用于施工场景中摄像机视角固定、作业目标远近变化频繁的施工视频分析任务，具有良好的鲁棒性、稳定性和工程适用性。针对拱坝施工场景大尺度、背景复杂、远距离人员车辆与设备运动干扰强等问题，本发明通过三帧差分法提取运动前景特征，并进一步结合ROI区域约束进行空间匹配，对视频中与吊运作业无关的背景噪声及无关运动目标进行多层次筛除，从而提高吊钩下方目标物检测的可靠性。通过上述方法，使后续的吊运任务判别过程聚焦于缆机吊钩作业直接相关的关键区域，避免在全局范围内进行特征学习而引入大量背景噪声及无关目标，从而提高吊运任务识别的稳定性与准确性。
Smart Images
Figure CN122967191A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of arch dam construction and visual inspection, and relates to a visual recognition method for arch dam cable crane hoisting tasks, and more particularly to a visual recognition method for arch dam cable crane hoisting tasks based on motion foreground constraints. Background Technology
[0002] Dam projects are mostly located in high mountain and canyon areas, with complex construction environments, limited work sites, and restricted transportation conditions, posing numerous challenges to the organization and transportation of construction resources. As a key transportation equipment in the construction of arch dams, cable cranes not only undertake concrete hoisting tasks but also need to perform auxiliary hoisting tasks (transporting equipment, materials, slag, etc., and eliminating defects). The types of objects they hoist are diverse, and the operation modes are complex, which places high demands on the accurate identification and refined management of hoisting tasks.
[0003] For identifying targets in cable crane hoisting tasks, existing technologies employ visual perception-based target detection methods. These methods assume a pre-determined set of target categories that can be fully covered by the detection model, and use manually labeled samples for supervised training, with the detected target category serving as the basis for task identification. However, in arch dam construction scenarios, the types of auxiliary hoisting targets are numerous, with significant morphological differences and uneven sample distribution. Constructing a full-category training dataset and labeling it is labor-intensive and costly. Furthermore, when adding new hoisting objects, samples need to be re-collected and the model updated, resulting in poor engineering adaptability. Simultaneously, in large-scale complex scenes with a fixed viewpoint, the scale of hoisting targets varies greatly, and their appearance features are easily affected by lighting and environmental background. Visual detection methods relying on appearance features are prone to false positives or false negatives, limiting overall adaptability and stability. In summary, these methods rely on the prior condition of the hoisting object's category, failing to fully utilize the engineering motion patterns and spatial relationships inherent in cable crane hoisting tasks, making it difficult to achieve stable and reliable task identification in complex construction environments.
[0004] In addition, some studies have identified hoisting tasks based on cable crane operating parameters (sensor data such as load, displacement, or speed). These methods do not require visual recognition of the hoisted object, reducing the complexity of image annotation and model training to some extent. However, in actual construction, different hoisting tasks overlap in load changes and motion characteristics, making it difficult to establish clear and distinguishable boundaries. Furthermore, sensor data is easily affected by fluctuations in construction conditions and environmental noise, limiting the accuracy of task identification. Compared to visual perception methods, these methods can only achieve relatively coarse-grained task differentiation, failing to meet the refined management needs of complex hoisting tasks. Therefore, effectively combining visual perception with the motion characteristics and spatial relationships between key operational objects during cable crane hoisting can not only reduce the reliance on comprehensive labeling of auxiliary hoisting targets but also improve the stability and engineering applicability of cable crane hoisting task identification. Summary of the Invention
[0005] In order to solve the above-mentioned technical problems in the background art, the present invention provides a visual recognition method for arch dam cable crane hoisting tasks that can improve the reliability of target object detection below the hook and realize the automatic identification of cable crane hoisting tasks.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: A visual recognition method for arch dam cable crane hoisting tasks, characterized in that: the visual recognition method for arch dam cable crane hoisting tasks includes the following steps: 1) Obtain video sequences of cable crane operations during arch dam construction; 2) Based on the cable crane operation video sequence obtained in step 1), stable motion foreground features in the arch dam construction scene are extracted through motion detection; 3) Identify the cable crane hook in the video frame, and dynamically construct a motion foreground constraint region (ROI) below the detection frame of the cable crane hook, using the detection frame of the cable crane hook as a reference. 4) Spatial matching is performed between the stable motion foreground features obtained in step 2) and the motion foreground constraint region ROI obtained in step 3). If a successfully matched motion region exists, it is determined that there is a target object to be hoisted below the cable crane hook. 5) Determine whether the target object being hoisted is a hoisting container, and identify whether the output cable machine is in concrete hoisting or auxiliary hoisting state to complete the identification of the arch dam cable machine hoisting task.
[0007] Preferably, the video sequence used in step 1) of the present invention is obtained by a fixed camera with a resolution of not less than 3840×2160 and a frame rate of not less than 30 frames per second.
[0008] Preferably, the specific implementation of step 2) in this invention is as follows: 2.1) Perform grayscale processing on three consecutive original images to obtain P. t-1 P t and P t+1 Calculate the difference between two adjacent frames to obtain difference maps D1 and D2: D1=|P t-1 -P t | D2=|P t -P t+1 | 2.2) Based on the difference maps D1 and D2 obtained in step 2.1), a binary motion mask M(x, y) is obtained through thresholding. Then, morphological processing combining opening and closing operations is used to remove isolated noise, fill regional holes, and enhance connectivity. Finally, a logical AND operation is performed on the processed results to obtain the final motion mask M. t (x, y); in: (x, y) represents the pixel coordinates in the image; D(x, y) represents the gray-level difference in the difference image at pixel coordinates (x, y); T is a preset difference threshold used to distinguish between moving foreground and background; 2.3) The final motion mask M obtained in step 2.2) t Perform connected component analysis on (x, y) and calculate the area A of each connected component. t and the center of mass (x) i y i ), to obtain the set of moving foreground features; the area A of the connected region t The total number of pixels in the corresponding connected component; the centroid (x i y i ) is the geometric center of all pixel coordinates within the connected region, obtained by averaging the coordinates of all pixels within the connected region; 2.4) In multiple consecutive time frames, perform temporal consistency judgment on the motion foreground feature set of each frame, count the number of occurrences of the same connected region, and only when its consecutive occurrence reaches a preset number of frames is the connected region determined as a stable motion foreground feature.
[0009] Preferably, the specific implementation of step 2.2) in this invention is as follows: 2.2.1) Threshold the difference map to obtain binary motion masks M1(x,y) and M2(x,y); 2.2.2) Morphological enhancement is performed on the binary motion masks M1(x,y) and M2(x,y) obtained in step 2.2.1) to simultaneously suppress isolated noise generated after thresholding, fill in holes inside the motion region, and enhance connectivity, resulting in the enhanced mask M. d1 (x, y) and M d2 (x, y): in: M1(x,y) is a binary motion mask for the difference map D1; M2(x,y) is a binary motion mask for the difference map D2; Bo is an open operation structure element; Bc is a closed-operation structure element; It is an opening operation, which first performs an erosion operation and then a dilation operation to remove small noise; It is a closing operation, first performing an expansion operation, then an erosion operation to fill the voids; 2.2.3) Take the M obtained in step 2.2.2) d1 (x, y) and M d2 Perform a logical AND operation on (x, y) to obtain the final motion mask M. t (x, y): M t (x, y) = M d1 (x, y) ∩ M d2 (x, y).
[0010] Preferably, the method for constructing the motion foreground constraint region (ROI) in step 3) of the present invention is as follows: 3.1) Use the trained cable crane hook detection model to detect each video frame in the video sequence obtained in step 1), and obtain the bounding box B of the first cable crane hook. t : B t = (x t y t w t h t ) in: (x) t y t () represents the center coordinates of the cable crane hook boundary frame; (w) t h t () represents the width and height of the boundary frame of the cable crane hook; 3.2) Boundary box B of the cable crane hook t Based on the reference, within the boundary box B of the cable crane hookt Construct a rectangular ROI below ; The lower edge of the cable crane hook boundary frame ordinate : Construct a rectangular ROI: in: These are the center coordinates of the ROI region; These are the width and height of the ROI region; , , , These are the preset parameters; Control the horizontal coverage of ROI; Control the vertical coverage of the ROI; Horizontal offset; The ROI center position is shifted downward in the vertical direction; 3.3) For the other cable crane hooks except the first cable crane hook, repeat steps 3.1) and 3.2) to generate the corresponding motion foreground constraint region ROI in each frame and construct the ROI time sequence corresponding to each cable crane hook in chronological order.
[0011] Preferably, the trained cable crane hook detection model used in step 3) of the present invention is designed for cable crane hook targets in arch dam construction scenarios. A dedicated training dataset is constructed, and the YOLOv8 target detection model is trained based on the training dataset to finally obtain a trained cable crane hook detection model for identifying cable crane hooks.
[0012] Preferably, the process for determining whether there is a target object below the hook in step 4) of the present invention is as follows: 4.1) Based on the stable motion foreground features obtained in step 2) and the motion foreground constrained region of interest (ROI) obtained in step 3); 4.2) Determine the centroid of each connected component Does it fall within the threshold range of the ROI constrained region of motion? If it does, then the connected component motion mask... With the motion foreground constraint region ROI If the coverage ratio is greater than the threshold, the connected component is associated with the cable crane hook; otherwise, it is determined that the connected component is not associated with the hook, and it is assumed that there is no target object below the cable crane hook, and the output cable crane is in a waiting state.
[0013] Preferably, the specific implementation of step 4.2) in this invention is as follows: 4.2.1) Construct a lower-biased matching band C within the region of interest (ROI) constrained by motion. The expression for the lower-biased matching band C is: in: These are the center coordinates of the ROI region; These are the width and height of the ROI region; Control the left and right contraction of the matching band; Control the matching to capture the lower region of the ROI; 4.2.2) If the centroid of the connected domain satisfy If the connection is found to be unrelated to the cable crane hook, then the connected component is retained and the overlap ratio is determined; otherwise, the connected component is considered to be unrelated to the cable crane hook and is discarded. 4.2.3) Determine the binary mask for the motion foreground constraint region ROI : in: Represents the pixel coordinates in the image; This represents the coordinates of the lower left corner of the constrained region of interest (ROI). This represents the coordinates of the upper right corner of the constrained region ROI; 4.2.4) Based on the results of step 4.2.3), calculate the coverage ratio of connected components to ROIs: in: , is the coverage ratio threshold parameter; A binary mask representing a connected component; A binary mask representing the region of interest (ROI) constrained by motion. This represents the area covered by the connected components and the ROI. Represents connected components The total area; If the above conditions are met, the connected domain is determined to be associated with the cable crane hook, indicating that there is a target object below the cable crane hook; otherwise, it is determined to be unassociated, indicating that there is no target object below the cable crane hook, and the output cable crane is in a waiting state.
[0014] Preferably, the specific implementation of step 5) in this invention is as follows: 5.1) Obtain construction video data for model training, label the suspended tanks below the cable crane hooks in the construction video data, and construct a training dataset; 5.2) Train a YOLOv8 object detection model based on labeled data to obtain a trained can detection model; 5.3) Input the ROI sequence corresponding to the target object below the cable crane hook obtained in step 4) into the hoisting tank detection model for detection. If the hoisting tank is detected, output concrete hoisting; if the hoisting tank is not detected, output auxiliary hoisting.
[0015] The beneficial effects of this invention are: This invention provides a visual recognition method for arch dam cable crane hoisting tasks, comprising: 1) acquiring a video sequence of cable crane operations in an arch dam construction scene; 2) based on the cable crane operation video sequence acquired in step 1), extracting stable motion foreground features in the arch dam construction scene through motion detection; 3) identifying the cable crane hook in the video frame, and using the detection frame of the cable crane hook as a reference, dynamically constructing a motion foreground constraint region (ROI) below the detection frame of the cable crane hook; 4) spatially matching the stable motion foreground features obtained in step 2) with the motion foreground constraint region (ROI) obtained in step 3), and if a successfully matched motion region exists, determining that there is a target object to be hoisted below the cable crane hook; 5) determining whether the target object to be hoisted is a hoisting container, and outputting whether the cable crane is in a concrete hoisting state or an auxiliary hoisting state, thus completing the recognition of the arch dam cable crane hoisting task. This invention transforms the problem of identifying cable crane lifting tasks into a two-stage discrimination: whether a target exists below the cable crane hook and whether the target is a hoisting container. Only the relevant feature information of the cable crane hook and the hoisting container is needed to complete the output of three types of operational states. This eliminates the need for training by classifying and labeling numerous auxiliary lifting targets such as reinforcing bars, formwork, equipment, and slag hoppers, significantly reducing data labeling and model iteration costs. It can adapt to situations where target categories are constantly changing and long-tail targets frequently appear. Addressing the characteristics of large-scale construction scenes from a fixed camera perspective and the significant time-varying scale changes of target objects during lifting, this invention uses the cable crane hook as a key visual anchor point. Based on the position and scale changes of the cable crane hook detection box in the video sequence, the constructed Region of Interest (ROI) is dynamically adjusted synchronously, making the ROI adaptive in both spatial location and coverage. This enables precise constraint and extraction of areas closely related to the lifting task in dynamic videos. The aforementioned dynamic ROI construction mechanism based on the hook anchor point is suitable for construction video analysis tasks in construction scenarios where the camera perspective is fixed and the distance of the work target changes frequently, exhibiting good robustness, stability, and engineering applicability. To address the challenges of large-scale, complex backgrounds, and strong interference from the movement of personnel, vehicles, and equipment at long distances during arch dam construction, this invention extracts moving foreground features using a three-frame difference method and further combines this with ROI (Region of Interest) constraints for spatial matching. This multi-level filtering process removes background noise and irrelevant moving targets unrelated to the hoisting operation, thereby improving the reliability of target detection below the hook. This method allows subsequent hoisting task identification to focus on key areas directly related to the cable crane hook operation, avoiding the introduction of significant background noise and irrelevant targets through global feature learning, thus improving the stability and accuracy of hoisting task recognition. Attached Figure Description
[0016] Figure 1 This is the overall flowchart provided by the present invention; Figure 2 It is the motion foreground feature mask provided by the present invention; Figure 3 This is a schematic diagram of the ROI structure provided by the present invention; Figure 4 This is a schematic diagram of motion foreground and ROI space matching provided by the present invention; Figure 5 This is a flowchart of the two-stage task recognition process provided by the present invention. Detailed Implementation
[0017] See Figure 1 This invention provides a visual recognition method for arch dam cable crane hoisting tasks, comprising the following steps: Step 1: Obtain the video sequence of cable crane operation captured by a fixed camera; Cameras were installed at suitable locations on the dam abutment. The cameras used had a resolution of 3840×2160 and a frame rate of 30 frames per second. The cameras were required to operate stably under various lighting and weather conditions, and their viewpoints were required to capture the entire cable crane operation process. To ensure data diversity, cable crane hoisting images were extracted from videos collected at different times. These images were required to cover different viewpoints, target sizes, lighting conditions, and motion states.
[0018] Step 2: Based on the cable crane operation video sequence obtained in Step 1), stable motion foreground features in the arch dam construction scene are obtained through motion detection; Specifically, the process of extracting moving foreground features is as follows: 2.1) Calculation of three-frame differential response: The three consecutive original images are converted to grayscale to obtain Calculate the difference between two adjacent frames to obtain difference maps D1 and D2: (1) (2) 2.2) Motion Mask Generation: Thresholding is performed on the difference maps D1 and D2 to obtain a binary motion mask. Furthermore, morphological enhancements are used to remove noise, fill holes, and improve regional connectivity. Finally, a motion mask is obtained through logical operations. ; (3) in: Represents the pixel coordinates in the image; Represents the difference map in pixel coordinates The grayscale difference at the location; The preset difference threshold is used to distinguish between moving foreground and background.
[0019] Thresholding the difference image using the above formula yields a binary motion mask. Furthermore, morphological enhancement is performed on the binary motion mask. Since the target object's motion varies in speed between adjacent frames, holes appear after differential extraction when adjacent frames overlap. To simultaneously suppress isolated noise generated after thresholding and fill holes within the motion region, thus enhancing connectivity, this embodiment uses opening and closing operations in series to obtain... : (4) in: M1(x,y) is a binary motion mask for the difference map D1; M2(x,y) is a binary motion mask for the difference map D2; It is an open operation structure element; It is a closed-operation structure element; It is an opening operation, which first performs an erosion operation and then a dilation operation to remove small noise; It is a closing operation, which first performs an expansion operation and then an erosion operation to fill the voids.
[0020] After processing Perform logical AND operations to obtain the final motion mask. The final motion foreground mask image is as follows: Figure 2 As shown; (5) 2.3) Connected component analysis and structural feature extraction: for Perform connected component analysis and calculate the area of each connected component. and center of mass This yields a set of motion foreground features.
[0021] The area of the connected region The total number of pixels in the corresponding connected component; centroid The geometric center of all pixel coordinates within the connected region is obtained by averaging the coordinates of all pixels within the connected region.
[0022] 2.4) Temporal persistence constraint: In multiple consecutive time frames, the temporal consistency of the motion foreground feature set of each frame is judged, and the occurrence frequency of the same connected region is counted. Only when it occurs continuously for a preset number of frames is it judged as a stable motion foreground feature.
[0023] Step 3: Use the trained cable crane hook detection model to identify cable crane hooks in video frames, and dynamically construct a motion foreground constraint region (ROI) below the detection frame of the cable crane hook as a reference. To reduce the interference of complex backgrounds (large-scale scenes, long-distance movement of personnel and vehicles, and cross-operation of machinery and equipment) in the arch dam construction scenario on target feature learning, this embodiment uses the cable crane hook as the key anchor point and constructs the "motion foreground constraint region below the cable crane hook" as the region of interest (ROI), such as... Figure 3 As shown, this allows subsequent state discrimination to focus on the target area directly related to the cable crane hook operation, thereby significantly suppressing background noise and irrelevant motion interference, and improving recognition stability and accuracy.
[0024] Specifically, the ROI construction process is as follows: 3.1) Use the cable crane hook detection model to detect each video frame in the video sequence obtained in step 1) to obtain the bounding box of the cable crane hook: in The center coordinates of the cable crane hook boundary frame. This refers to the width and height of the boundary frame of the cable crane hook.
[0025] For multiple cable cranes present at the construction site, multi-target tracking is performed on the detected multiple cable crane hooks, and a unique ID is assigned to each cable crane hook, thereby generating a stable cable crane hook trajectory and realizing independent identification and management of multiple cable crane hooks.
[0026] 3.2) ROI Construction Principles: Since the hoisted target is always located below the cable crane hook and maintains a spatial relationship with it during cable crane operation, this invention uses the "area below the cable crane hook" as the motion foreground constraint area (ROI), and its construction follows these principles: a) The ROI is strongly bound to the position of the cable crane hook and is dynamically updated as the cable crane hook moves; b) The ROI covers the main working area where the hoisting target may appear; c) Background areas above and far from the cable crane hook are excluded as much as possible to reduce irrelevant motion interference; d) The size of the ROI varies proportionally with the size of the cable crane hook bounding box to adapt to different shooting distances and angles.
[0027] 3.3) Using the bounding box of the cable crane hook as a reference, construct a rectangular ROI below it. .
[0028] The ordinate of the lower edge of the cable crane hook boundary frame: (6) Construct a rectangular ROI: (7) (8) (9) (10) in , , , These are preset parameters; Controlling the lateral coverage of ROI ( (Preferably, 3) Controlling the vertical coverage of ROI ( Preferably, 3); Horizontal offset (used for compensation when the viewpoint is tilted or the camera is not facing directly, set to 0); The offset is lowered (to avoid targeting the cable crane hook and sling due to swaying), preferably... ).
[0029] Define the bottom left corner of the ROI rectangular region Top right corner : 3.4) For the other cable crane hooks except the first cable crane hook, repeat steps 3.1) and 3.2) to generate the corresponding motion foreground constraint region ROI in each frame and construct the ROI time sequence corresponding to each cable crane hook in chronological order.
[0030] Step 4: Spatial association matching is performed between the stable motion foreground features and the dynamic ROI. If a successfully matched motion region exists, it is determined that there is a target object to be hoisted below the cable crane hook. Specifically, such as Figure 5 As shown, the discrimination process is as follows: 4.1) Data: Motion foreground features, ROI time series; 4.2) Matching the foreground motion with the ROI space: such as Figure 4 As shown, determine the centroid of each connected component. Does it fall within the ROI threshold range? If it does, then the connected components... With ROI If the coverage ratio is greater than the threshold, then the connected component is associated with the cable crane hook; Considering the spatial relationship between the cable crane hook and the suspended object, the centroid of the suspended object is spatially adjacent to the center of the hook detection frame in the horizontal direction. Simultaneously, considering that the cable crane hook and the suspended object are connected by a sling with a non-zero length, the centroid of the motion connectivity region will shift downwards relative to the ROI region. Therefore, a downward-biased matching band C is constructed within the ROI. (11) in, The invention controls the left and right contraction of the matching band. Preferably, the invention takes... ; The control matches the lower region of the ROI. Preferably, the present invention selects... .
[0031] If the following conditions are met: If the connected component is not found to be associated with the cable crane hook, it is retained and proceeded to the overlap ratio determination; otherwise, the connected component is considered to be unrelated to the cable crane hook and is discarded; if none of the connected components meet the above conditions, the output cable crane is in a waiting state.
[0032] Binary mask of ROI : (12) in: Represents the pixel coordinates in the image; This represents the coordinates of the lower left corner of the constrained region of interest (ROI). This represents the coordinates of the upper right corner of the constrained region ROI; Calculate connected components With ROI Coverage ratio: (13) in, , is the coverage ratio threshold parameter. Preferably, the present invention selects ; A binary mask representing a connected component; A binary mask representing the region of interest (ROI) constrained by motion. This represents the coverage area value (i.e., the total number of pixels covered) of the connected components and the ROI. Represents connected components The total area (i.e., the total number of connected pixels); If the above conditions are met, the match is successful, and the connected component is determined to be associated with the cable crane hook, indicating that there is a target object below the cable crane hook; otherwise, it is determined to be unassociated, indicating that there is no target object below the cable crane hook, and the output cable crane is in a waiting state.
[0033] Step 5: Use the hoisting tank detection model to determine whether the target object is a hoisting tank, thereby identifying whether the cable crane is in a waiting state or is performing concrete hoisting or auxiliary hoisting tasks.
[0034] Since there are many types of auxiliary hoisting targets, this step does not further classify the auxiliary targets, but only determines whether it is a hoisting container, and then outputs "concrete hoisting" or "auxiliary hoisting".
[0035] Specifically, such as Figure 5 As shown, the discrimination process is as follows: 5.1) Caving tank annotation and YOLOv8 model training: Obtain construction video data for model training, annotate the caving tanks below the cable crane hooks in the construction videos, and construct a training dataset; train the YOLOv8 object detection model based on the annotated data to obtain the caving tank detection model.
[0036] 5.2) ROI Input Detection: Input the ROI sequence corresponding to the target object determined in step 5 below the cable crane hook into the hanging tank detection model for detection.
[0037] 5.3) If a hoisting tank is detected, output concrete hoisting; if no hoisting tank is detected, output auxiliary hoisting.
Claims
1. A visual recognition method for arch dam cable crane hoisting tasks, characterized in that: The visual recognition method for arch dam cable crane hoisting tasks includes the following steps: 1) Obtain video sequences of cable crane operations during arch dam construction; 2) Based on the cable crane operation video sequence obtained in step 1), stable motion foreground features in the arch dam construction scene are extracted through motion detection; 3) Identify the cable crane hook in the video frame, and use the detection frame of the cable crane hook as a reference to dynamically construct a motion foreground constraint region (ROI) below the detection frame of the cable crane hook; 4) Spatial matching is performed between the stable motion foreground features obtained in step 2) and the motion foreground constraint region ROI obtained in step 3). If a successfully matched motion region exists, it is determined that there is a target object to be hoisted below the cable crane hook. 5) Determine whether the target object being hoisted is a hoisting container, and identify whether the output cable machine is in concrete hoisting or auxiliary hoisting state to complete the identification of the arch dam cable machine hoisting task.
2. The visual recognition method for arch dam cable crane hoisting tasks according to claim 1, characterized in that: The video sequence in step 1) is obtained by capturing data from a fixed camera with a resolution of not less than 3840×2160 and a frame rate of not less than 30 frames per second.
3. The visual recognition method for arch dam cable crane hoisting tasks according to claim 2, characterized in that: The specific implementation method of step 2) is as follows: 2.1) Perform grayscale processing on three consecutive original images to obtain P. t-1 P t and P t+1 Calculate the difference between two adjacent frames to obtain difference maps D1 and D2: D1=|P t-1 -P t | D2=|P t -P t+1 | 2.2) Based on the difference maps D1 and D2 obtained in step 2.1), a binary motion mask M(x, y) is obtained through thresholding. Then, morphological processing combining opening and closing operations is used to remove isolated noise, fill regional holes, and enhance connectivity. Finally, a logical AND operation is performed on the processed results to obtain the final motion mask M. t (x, y); in: (x, y) represents the pixel coordinates in the image; D(x, y) represents the gray-level difference of the difference map at pixel coordinates (x, y); T is a preset difference threshold used to distinguish between moving foreground and background; 2.3) The final motion mask M obtained in step 2.2) t Perform connected component analysis on (x, y) and calculate the area A of each connected component. t and the center of mass (x) i y i ), to obtain the set of moving foreground features; the area A of the connected region t The total number of pixels in the corresponding connected component; the centroid (x i y i ) is the geometric center of all pixel coordinates within the connected domain, obtained by averaging the coordinates of all pixels within the connected domain; 2.4) In multiple consecutive time frames, perform temporal consistency judgment on the motion foreground feature set of each frame, count the number of occurrences of the same connected region, and only when its consecutive occurrence reaches a preset number of frames is the connected region determined as a stable motion foreground feature.
4. The visual recognition method for arch dam cable crane hoisting tasks according to claim 3, characterized in that: The specific implementation method of step 2.2) is as follows: 2.2.1) Threshold the difference map to obtain binary motion masks M1(x,y) and M2(x,y); 2.2.2) Morphological enhancement is performed on the binary motion masks M1(x,y) and M2(x,y) obtained in step 2.2.1) to obtain the enhanced mask M. d1 (x, y) and M d2 (x, y): in: M1(x,y) is a binary motion mask for the difference map D1; M2(x,y) is a binary motion mask for the difference map D2; Bo is an open operation structure element; Bc is a closed-operation structure element; It is an opening operation, which first performs an erosion operation and then a dilation operation to remove small noise; It is a closing operation, first performing an expansion operation, then an erosion operation to fill the voids; 2.2.3) Take the M obtained in step 2.2.2) d1 (x, y) and M d2 Perform a logical AND operation on (x, y) to obtain the final motion mask M. t (x, y): M t (x,y)=M d1 (x,y)∩M d2 (x,y)。 5. The visual recognition method for arch dam cable crane hoisting tasks according to claim 4, characterized in that: The method for constructing the motion foreground constraint region (ROI) in step 3) is as follows: 3.1) Use the trained cable crane hook detection model to detect each video frame in the video sequence obtained in step 1), and obtain the bounding box B of the first cable crane hook. t : B t =(x t ,y t ,w t ,h t ) in: (x) t y t () represents the center coordinates of the cable crane hook boundary frame; (w) t h t () represents the width and height of the boundary frame of the cable crane hook; 3.2) Boundary box B of the cable crane hook t Based on the reference, within the boundary box B of the cable crane hook t Construct a rectangular ROI below ; The lower edge of the cable crane hook boundary frame ordinate : Construct a rectangular ROI: in: These are the center coordinates of the ROI region; These are the width and height of the ROI region; , , , These are the preset parameters; Control the horizontal coverage of ROI; Control the vertical coverage of the ROI; Horizontal offset; It is a downward offset of the ROI center position in the vertical direction; 3.3) For the other cable crane hooks except the first cable crane hook, repeat steps 3.1) and 3.2) to generate the corresponding motion foreground constraint region ROI in each frame and construct the ROI time sequence corresponding to each cable crane hook in chronological order.
6. The visual recognition method for arch dam cable crane hoisting tasks according to claim 5, characterized in that: In step 3), the trained cable crane hook detection model is designed for cable crane hook targets in arch dam construction scenarios. A dedicated training dataset is constructed, and the YOLOv8 target detection model is trained based on the training dataset to finally obtain a trained cable crane hook detection model for identifying cable crane hooks.
7. The visual recognition method for arch dam cable crane hoisting tasks according to claim 6, characterized in that: The process for determining whether there is a target object below the hook in step 4) is as follows: 4.1) Based on the stable motion foreground features obtained in step 2) and the motion foreground constrained region of interest (ROI) obtained in step 3); 4.2) Determine the centroid of each connected component Does it fall within the threshold range of the ROI constrained region of motion? If it does, then the connected component motion mask... With the motion foreground constraint region ROI If the coverage ratio is greater than the threshold, the connected component is associated with the cable crane hook; otherwise, it is determined that the connected component is not associated with the hook, and it is assumed that there is no target object below the cable crane hook, and the output cable crane is in a waiting state.
8. The visual recognition method for arch dam cable crane hoisting tasks according to claim 7, characterized in that: The specific implementation method of step 4.2) is as follows: 4.2.1) Construct a lower-biased matching band C within the region of interest (ROI) constrained by motion. The expression for the lower-biased matching band C is: in: These are the center coordinates of the ROI region; These are the width and height of the ROI region; Control the left and right contraction of the matching band; Control the matching to capture the lower region of the ROI; 4.2.2) If the centroid of the connected domain satisfy If the connection is found to be unrelated to the cable crane hook, then the connected component is retained and the overlap ratio is determined; otherwise, the connected component is considered to be unrelated to the cable crane hook and is discarded. 4.2.3) Determine the binary mask for the motion foreground constraint region ROI : in: Represents the pixel coordinates in the image; This represents the coordinates of the lower left corner of the constrained region of interest (ROI). This represents the coordinates of the upper right corner of the constrained region ROI; 4.2.4) Based on the results of step 4.2.3), calculate the coverage ratio of connected components to ROIs: in: , is the coverage ratio threshold parameter; A binary mask representing a connected component; A binary mask representing the region of interest (ROI) constrained by motion. This represents the area covered by the connected components and the ROI. Represents connected components The total area; If the above conditions are met, the connected domain is determined to be associated with the cable crane hook, indicating that there is a target object below the cable crane hook; otherwise, it is determined to be unassociated, indicating that there is no target object below the cable crane hook, and the output cable crane is in a waiting state.
9. The visual recognition method for arch dam cable crane hoisting tasks according to claim 8, characterized in that: The specific implementation method of step 5) is as follows: 5.1) Obtain construction video data for model training, label the suspended tanks below the cable crane hooks in the construction video data, and construct a training dataset; 5.2) Train a YOLOv8 object detection model based on labeled data to obtain a trained can detection model; 5.3) Input the ROI sequence corresponding to the target object below the cable crane hook obtained in step 4) into the hoisting tank detection model for detection. If the hoisting tank is detected, output concrete hoisting; if the hoisting tank is not detected, output auxiliary hoisting.