Crane track settlement real-time detection method based on image recognition

Abstract

The application discloses a crane track settlement real-time detection method based on image recognition, comprising the following steps: collecting image data of a track area and performing image preprocessing; extracting a track target area and constructing a track feature point set; constructing a space-time track feature map, identifying and selecting a reference frame image, and generating a reference frame index sequence; registering and comparing the track feature point set with feature points in the corresponding frame of the reference frame index sequence, and constructing a three-dimensional displacement vector; estimating a three-dimensional local deformation variable field vector of the track boundary and constructing a track settlement deformation map; calculating the overall displacement change rate of the track and the local deformation fluctuation amplitude, and forming a track stability score; setting a weak settlement response threshold and a sudden settlement alarm threshold, and triggering an early warning prompt. The application combines image recognition and three-dimensional dynamic analysis, realizes real-time and accurate monitoring and intelligent early warning of crane track settlement, and has the advantages of automation, timeliness and high precision.

Description

Technical Field

[0001] This invention relates to the field of track settlement detection, and more particularly to a real-time detection method for crane track settlement based on image recognition. Background Technology

[0002] Currently, the settlement detection and safety monitoring of crane tracks mainly rely on fixed-point physical sensors, manual inspections, and traditional image processing methods. Common methods such as laser displacement gauges, electronic levels, and single-frame image comparison detection can monitor track settlement to a certain extent, but they have problems such as long detection cycles, untimely response, low spatial resolution, and poor environmental adaptability. Some image processing-based solutions only perform two-dimensional image analysis on single frames or a few key frames, lacking the ability to dynamically track the entire track operation process and failing to achieve continuous, three-dimensional, and refined observation of track settlement.

[0003] Existing technologies have not yet formed a complete track settlement detection method that integrates the spatiotemporal features of multi-frame images, structured analysis of track feature point sets, dynamic calculation of three-dimensional displacement, and multi-level early warning linkage. They lack the ability to dynamically model the entire cycle and global dependence of the track settlement process, cannot accurately locate spatial heterogeneous partitions and deformation hotspots, and have not achieved graded response and automatic control for weak settlement and sudden settlement. Summary of the Invention

[0004] One objective of this invention is to propose a real-time detection method for crane track settlement based on image recognition. This invention integrates image recognition and three-dimensional dynamic analysis to achieve real-time and accurate monitoring and intelligent early warning of crane track settlement, and has the advantages of automation, timeliness and high precision.

[0005] A real-time detection method for crane track settlement based on image recognition according to an embodiment of the present invention includes the following steps: Image data of the track area during crane operation is collected to obtain a set of original track images, and image preprocessing is performed to generate a set of standard track images; Extract the target region of the track from the standard track image set, and extract the track boundary contour to construct a set of track feature points for each frame of the image; The set of orbit feature points is constructed into a spatiotemporal orbit feature map. The graph structure is analyzed. When the state change amplitude and the adjacent edge weight change rate meet the adaptive criterion, the corresponding frame is identified and selected as the reference frame image, and a reference frame index sequence is generated. The set of orbit feature points is registered and compared with the feature points in the corresponding frames of the reference frame index sequence to construct a three-dimensional displacement vector for each feature point. The three-dimensional local deformation field vector of the track boundary is estimated based on the three-dimensional displacement vector, and the track settlement deformation map is constructed. Based on the track settlement deformation diagram, the overall track displacement change rate and local deformation fluctuation amplitude are calculated to form a track stability score and generate a track stability score sequence. A dynamic dual-threshold discrimination strategy is constructed based on the track stability scoring sequence. A weak settlement response threshold and a sudden settlement alarm threshold are set to trigger weak settlement early warning and sudden settlement alarm respectively.

[0006] Optionally, the image preprocessing includes median filtering, grayscale normalization, brightness equalization, and edge enhancement.

[0007] Optionally, the construction of the orbital feature point set includes: For the set of standard track images, the target region of track is extracted for each frame of the standard track image. The threshold segmentation method is used to compare the gray value of all pixels in the standard track image with the set segmentation threshold. When the gray value of a pixel is higher than the segmentation threshold, the corresponding pixel is included in the target region of track. All pixels that meet the conditions are marked with three-dimensional spatial coordinates to form a set of target regions of track. Based on the set of target regions of the track, the track boundary contour is extracted for each frame of the standard track image, the gray level gradient of all pixels is calculated, and for pixels whose gray level gradient is greater than the boundary discrimination threshold, the corresponding three-dimensional spatial coordinates are recorded to the set of track boundary contours. The track boundary profile is fitted with a polynomial curve using a set of track boundary profiles, and the polynomial coefficients are determined by the least squares method to obtain a three-dimensional polynomial curve model describing the track boundary profile. On the three-dimensional polynomial curve model of the track boundary profile, the distribution range of feature points is set according to the principle of equal interval sampling. The track boundary profile is divided into several equally spaced intervals within the minimum and maximum three-dimensional coordinate intervals. The three-dimensional spatial coordinates of the sampling points in each interval are calculated to obtain the set of track feature points. The number of all track feature points and the sampling interval are configured according to preset parameters.

[0008] Optionally, the construction of the reference frame index sequence specifically includes: The set of track feature points for each frame of the standard track image is obtained and marked one by one according to the acquisition time and frame number. All the sets of track feature points form a time-series track feature point set sequence. The three-dimensional spatial coordinates, pixel grayscale values, and edge response values ​​of the orbit feature points are collected at each moment, and the orbit feature point state vector is constructed based on these values. Construct a spatiotemporal orbit feature map from a sequence of temporal orbit feature point sets; Traverse all nodes in the spatiotemporal orbit feature map, calculate the state change amplitude and adjacent edge weight change rate of each node during continuous acquisition time, and calculate the average of the state change amplitude and adjacent edge weight change rate as the global state change amplitude and global adjacent edge weight change rate, and compare the global state change amplitude and global adjacent edge weight change rate with the adaptive criterion respectively. When the magnitude of the global state change and the rate of change of the global adjacent edge weight both exceed the adaptive threshold at the acquisition time, the standard track image with the frame number corresponding to the acquisition time is selected as the reference frame image, and all acquisition times and frame numbers that meet the conditions are included in the reference frame index. For each reference frame image, the corresponding acquisition time and frame number are recorded to form a reference frame index sequence.

[0009] Optionally, the construction of the three-dimensional displacement vector specifically includes: For each acquisition time of the standard orbit image, all orbit feature points are extracted from the orbit feature point set and marked one-to-one according to the orbit feature point number. Based on the acquisition time and frame number of each reference frame in the reference frame index sequence, the corresponding reference frame image is extracted from the track standard image set, the track feature point set at the acquisition time of the reference frame is obtained, and the track feature point set at the current acquisition time is registered one-to-one with the number. For each track feature point with the same number, the three-dimensional spatial coordinates at the current acquisition time and the reference frame acquisition time are recorded respectively. The difference between the horizontal coordinate of the track feature point at the current acquisition time and the horizontal coordinate of the same numbered track feature point at the reference frame acquisition time, the difference between the vertical coordinate and the vertical coordinate of the same numbered track feature point at the reference frame acquisition time, and the difference between the height coordinate and the height coordinate of the same numbered track feature point at the reference frame acquisition time are respectively used as the three-dimensional displacement components of the track feature point in three-dimensional space, and are combined to form the three-dimensional displacement vector of the track feature point.

[0010] Optionally, the construction of the orbital settlement deformation map specifically includes: For each acquisition moment, the three-dimensional displacement vectors of all track feature points are obtained, and an adaptive clustering method based on spatial distance and the change of three-dimensional displacement components is used for the three-dimensional displacement components corresponding to the number of all track feature points. In the three-dimensional space, the track feature points are divided into several spatial heterogeneous partitions to form local adaptive sub-intervals. Within each spatial heterogeneity partition, for all orbital boundary points, a weighted spatial interpolation method is used to adaptively weight the three-dimensional displacement components corresponding to the number of all orbital feature points in the corresponding local adaptive sub-interval, and to calculate the three-dimensional local deformation field vector of each orbital boundary point. The three-dimensional local deformation field vectors of all track boundary points are arranged in an orderly manner according to the spatial coordinates of the track boundary points to establish a track settlement deformation map. The track settlement deformation map uses the spatial coordinates of the track boundary as an index to store the three-dimensional local deformation field vectors corresponding to each spatial index.

[0011] Optionally, the local adaptive sub-intervals are formed as follows: Calculate the spatial Euclidean distance between any two track feature point numbers in three-dimensional space, and combine it with the numerical changes of the three-dimensional displacement components. Track feature point numbers whose spatial Euclidean distance is less than a preset spatial distance threshold and whose three-dimensional displacement components have the same trend are automatically classified into the same spatial heterogeneity partition. After completing the partitioning of all track feature point numbers, the set of track feature point numbers in each spatial heterogeneity partition is defined as a local adaptive sub-interval.

[0012] Optionally, the generation of the orbital stability score sequence specifically includes: For each acquisition time, the three-dimensional local deformation field components of all track feature point numbers at the current acquisition time are extracted from the track settlement deformation map. The difference between the three-dimensional local deformation field components of each track feature point number at the current acquisition time and the previous acquisition time is calculated. The change amplitude of the three-dimensional local deformation field components of all track feature point numbers is averaged to obtain the overall displacement change rate of the track at the corresponding acquisition time. At the same acquisition time, based on the spatial heterogeneity partition, all orbital feature points in each spatial heterogeneity partition are numbered, the variation amplitude of the three-dimensional local deformation field components is calculated, the variance of the variation amplitude of all three-dimensional local deformation field components in the spatial heterogeneity partition is calculated, and the variance is used as the local deformation fluctuation amplitude of the spatial heterogeneity partition. For each acquisition moment, the overall displacement change rate of the track and the local deformation fluctuation amplitude of all spatial heterogeneous zones are multiplied by preset weighting coefficients and then summed. The resulting value is used as the track stability score for the current acquisition moment. The orbit stability scores at all acquisition times are arranged in the order of acquisition time to form an orbit stability score sequence.

[0013] Optionally, the triggering of the weak settlement early warning and sudden settlement alarm specifically includes: For the track stability score sequence, a sliding window distribution statistical method is used to dynamically set the weak settlement response threshold and the sudden settlement alarm threshold. The weak settlement response threshold is always less than the sudden settlement alarm threshold, and the two are automatically adjusted according to the current track stability score sequence distribution. The score value at each acquisition time in the track stability score sequence is judged by two thresholds. If the score value is greater than or equal to the weak settlement response threshold and less than the sudden settlement alarm threshold, a weak settlement early warning prompt is immediately triggered. If the score is greater than or equal to the sudden settlement alarm threshold, a sudden settlement alarm will be triggered immediately.

[0014] Optionally, the dynamic setting of the weak settlement response threshold and the sudden settlement alarm threshold specifically includes: In the orbit stability scoring sequence, for the current acquisition time, a continuous scoring value including the current acquisition time and the previous N-1 acquisition times is selected to form a sliding window scoring subsequence of length N; Sort all scores within the sliding window scoring subsequence, and extract the scores corresponding to the P1 percentile and P2 percentile respectively, which are used as the weak settlement response threshold and sudden settlement alarm threshold within the current sliding window.

[0015] The beneficial effects of this invention are: (1) This invention improves the automation, precision and intelligence of crane track settlement monitoring by introducing a real-time detection method for track settlement throughout the entire process based on image recognition. By acquiring and standardizing multiple track images and combining them with three-dimensional structured modeling of track feature point sets, the system achieves accurate extraction of track boundaries and feature points and dynamic tracking throughout the entire process. Based on the structural analysis of the spatiotemporal track feature map and the dynamic selection of the reference frame index sequence, the system can adaptively perceive the evolution of track status, accurately grasp the settlement trend and local anomalies, and effectively solve the shortcomings of traditional methods in terms of dynamism and global correlation.

[0016] (2) This invention further improves the spatial resolution and local analysis capability of track settlement deformation by constructing three-dimensional displacement vectors and spatial heterogeneity partitions. By using weighted interpolation of spatial distance and historical three-dimensional displacement changes, it achieves high-precision estimation of the local deformation field of the track boundary, ensuring the expressive power and visualization effect of the track settlement deformation map. On this basis, the generation of track stability scoring sequence organically integrates the overall displacement change rate of the track with the local deformation fluctuation amplitude, realizing the dynamic stability quantitative assessment of the entire track operation process.

[0017] (3) By using a dynamic dual-threshold discrimination strategy based on the track stability scoring sequence, the system can adaptively set the weak settlement response threshold and the sudden settlement alarm threshold according to the actual operating status, which effectively improves the real-time performance and accuracy of early warning and alarm. The graded response mechanism for weak settlement and sudden settlement enhances the intelligent decision-making capability of safety linkage. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1This is a flowchart of a real-time detection method for crane track settlement based on image recognition proposed in this invention; Figure 2 This is a schematic diagram of the three-dimensional displacement vector registration and spatial heterogeneity partitioning of track feature points in a real-time crane track settlement detection method based on image recognition proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figure 1 and Figure 2 A real-time detection method for crane track settlement based on image recognition includes the following steps: Image data of the track area during crane operation is collected to obtain a set of original track images. Image preprocessing is then performed on the original track image set to generate a set of standard track images. Extract the target region of the track from the standard track image set, and extract the track boundary contour to construct a set of track feature points for each frame of the image; The set of orbit feature points is constructed into a spatiotemporal orbit feature map. The graph structure is analyzed. When the magnitude of state change and the rate of change of adjacent edge weights meet the adaptive criterion, the corresponding frame is identified and selected as the reference frame image, and a globally dependent reference frame index sequence is generated. The set of orbit feature points is registered and compared with the feature points in the corresponding frames of the reference frame index sequence to construct a three-dimensional displacement vector for each feature point. The three-dimensional local deformation field vector of the track boundary is estimated based on the three-dimensional displacement vector, and the track settlement deformation map is constructed based on the three-dimensional local deformation field vector; Based on the track settlement deformation diagram, the overall track displacement change rate and local deformation fluctuation amplitude are calculated to form a track stability score and generate a track stability score sequence. A dynamic dual-threshold discrimination strategy is constructed based on the track stability scoring sequence. A weak settlement response threshold and a sudden settlement alarm threshold are set to trigger weak settlement early warning and sudden settlement alarm respectively.

[0021] The Figure 1The flowchart describes the overall steps of the present invention, which are the steps of the above-mentioned real-time detection method for crane track settlement based on image recognition. The steps include: acquiring image data of the track area and performing image preprocessing; extracting the target area of ​​the track and constructing a set of track feature points; constructing a spatiotemporal track feature map, identifying and selecting reference frame images, and generating a reference frame index sequence; registering and comparing the track feature point set with the feature points in the corresponding frames of the reference frame index sequence to construct a three-dimensional displacement vector; estimating the three-dimensional local deformation field vector of the track boundary and constructing a track settlement deformation map; calculating the overall track displacement change rate and the local deformation fluctuation amplitude to form a track stability score; and setting a weak settlement response threshold and a sudden settlement alarm threshold to trigger an early warning.

[0022] The Figure 2 This paper describes a schematic diagram of the registration of three-dimensional displacement vectors of track feature points and the partitioning of spatial heterogeneity in this invention. First, a set of track feature points is extracted from a set of standard track images, and each feature point is assigned a unique number and three-dimensional spatial coordinates. At the same time, a reference frame index sequence is constructed to provide a spatiotemporal reference for subsequent registration. Then, the feature point sets at the current acquisition time and the acquisition time of the reference frame are numbered and registered one-to-one, and the three-dimensional displacement vector of each track feature point is calculated. After the three-dimensional displacement vectors of all track feature points are uniformly summarized, spatial heterogeneity partitioning is achieved through joint analysis of spatial Euclidean distance and three-dimensional displacement changes. Finally, the feature point number set of each spatial heterogeneity partition is output, providing a solid data foundation for local fine modeling and anomaly identification of track settlement deformation, ensuring the consistency of data structure and accurate expression of track spatial state.

[0023] In this embodiment, the image preprocessing includes median filtering, grayscale normalization, brightness equalization, and edge enhancement.

[0024] In this embodiment, the construction of the orbit feature point set includes: For the set of standard track images, the target region of track is extracted for each frame of the standard track image. The threshold segmentation method is used to compare the gray value of all pixels in the standard track image with the set segmentation threshold. When the gray value of a pixel is higher than the segmentation threshold, the corresponding pixel is included in the target region of track. All pixels that meet the conditions are marked with three-dimensional spatial coordinates to form a set of target regions of track. Based on the set of track target regions, track boundary contours are extracted for each frame of the track standard image, and the gray-level gradient of all pixels is calculated. For pixels whose gray-level gradient is greater than the boundary discrimination threshold, the corresponding three-dimensional spatial coordinates are recorded in the track boundary contour set, which reflects the external shape of the track target region. The track boundary profile is fitted with a polynomial curve using a set of track boundary profiles, and the polynomial coefficients are determined by the least squares method to obtain a three-dimensional polynomial curve model describing the track boundary profile. On the three-dimensional polynomial curve model of the track boundary profile, the distribution range of feature points is set according to the principle of equal interval sampling. The track boundary profile is divided into several equally spaced intervals within the minimum and maximum three-dimensional coordinate intervals. The three-dimensional spatial coordinates of the sampling points in each interval are calculated to obtain the set of track feature points. The number of all track feature points and the sampling interval are configured according to preset parameters.

[0025] In this embodiment, the construction of the reference frame index sequence specifically includes: The set of track feature points for each frame of the standard track image is obtained and marked one by one according to the acquisition time and frame number. All the sets of track feature points form a time-series track feature point set sequence. The three-dimensional spatial coordinates, pixel grayscale values, and edge response values ​​of the orbit feature points are collected at each moment, and the orbit feature point state vector is constructed based on these values. The edge response value is obtained by applying the Canny operator to the coordinates of the track feature points on the track standard image to obtain the horizontal gradient component and the vertical gradient component, and then taking the square root of the sum of the squares of the two components. All components in the state vector of the orbit feature points have been normalized to ensure that each component is within a uniform dimension range, thus ensuring that the state vector has consistency, comparability and numerical stability when calculating the state change amplitude of the orbit feature points and when performing registration and comparison of the orbit feature point set. Construct a spatiotemporal orbit feature map from a sequence of temporal orbit feature point sets; The spatiotemporal orbit feature map uses orbit feature points at each acquisition time as nodes, and the node attribute is the orbit feature point state vector. A directed edge is established between the same orbit feature point at any adjacent acquisition time, and the edge attribute is the magnitude of the orbit feature point state change and the change in Euclidean distance between adjacent orbit feature point sets. The state change amplitude of the orbit feature points refers to the difference between the state vector of each orbit feature point in the orbit feature point set and the state vector of the corresponding feature point in the previous acquisition time, and the Euclidean distance is calculated for each acquisition time. The Euclidean distance value is used as the state change amplitude of the orbit feature points and is used for the node state evolution analysis in the spatiotemporal orbit feature map and the subsequent reference frame selection criterion. The change in Euclidean distance between adjacent sets of orbit feature points refers to the change in the Euclidean distance between the same number of orbit feature points in the set of orbit feature points corresponding to two adjacent acquisition times in the spatiotemporal orbit feature map, measuring the magnitude of the spatial position change of the orbit feature points in the time dimension. Traverse all nodes in the spatiotemporal orbit feature map, calculate the state change amplitude and adjacent edge weight change rate of each node during continuous acquisition time, and calculate the average of the state change amplitude and adjacent edge weight change rate as the global state change amplitude and global adjacent edge weight change rate, and compare the global state change amplitude and global adjacent edge weight change rate with the adaptive criterion respectively. The rate of change of adjacent edge weights refers to the change in the weight of adjacent edges of the same numbered track feature point node in the spatiotemporal orbit feature map between the current acquisition time and the previous acquisition time, divided by the acquisition time interval. The adjacent edge weights are obtained by weighting and combining the magnitude of the change in the state of the track feature point with the change in the Euclidean distance between the sets of adjacent track feature points. This is used to quantify the speed of the dynamic evolution of track feature point nodes in the time series, providing a core criterion for track settlement anomaly detection and reference frame selection. When the magnitude of the global state change and the rate of change of the global adjacent edge weight both exceed the adaptive threshold at the acquisition time, the standard track image with the frame number corresponding to the acquisition time is selected as the reference frame image, and all acquisition times and frame numbers that meet the conditions are included in the reference frame index. The adaptive threshold is dynamically set using a sliding window statistical method. Specifically, at each acquisition time, the mean and standard deviation of the change amplitude of the state of the track feature points or the change rate of the adjacent edge weights in the previous N consecutive frames are calculated. The adaptive threshold is set as the weighted sum of the mean and standard deviation of the change amplitude of the state or the change rate of the adjacent edge weights within the window. For each reference frame image, the corresponding acquisition time and frame number are recorded to form a reference frame index sequence containing global spatiotemporal dependencies; Complete the construction of a reference frame index sequence based on the evolution of the spatiotemporal orbit feature map structure and global dependency driving, so as to ensure that the standard orbit image set can dynamically and adaptively select the most representative reference frame image that best reflects the evolution of the orbit settlement state throughout the entire orbit operation process, and provide a globally dependent optimized data index foundation for the registration and comparison of the orbit feature point set and the orbit feature point set in the corresponding frame of the reference frame index sequence, as well as the calculation of the displacement vector set.

[0026] This invention constructs a spatiotemporal orbit feature map and establishes a globally dependent reference frame index sequence based on the magnitude of changes in the state of orbit feature points and the rate of change in the weights of adjacent edges. The method can dynamically select the most representative and informative reference frames, realizing multi-dimensional spatiotemporal data modeling and adaptive updating of the entire orbit operation process. The dynamic construction of the reference frame index sequence effectively improves the accuracy of feature point registration and displacement calculation, providing a more timely and robust index data foundation for in-depth analysis of the evolution trend of orbit settlement and anomaly early warning.

[0027] In this embodiment, the construction of the three-dimensional displacement vector specifically includes: For each acquisition time of the standard orbital image, all orbital feature points are extracted from the orbital feature point set and marked one-to-one according to the orbital feature point number to ensure that the orbital feature point number has a unique correspondence at all acquisition times. Based on the acquisition time and frame number of each reference frame in the reference frame index sequence, the corresponding reference frame image is extracted from the track standard image set, the track feature point set at the acquisition time of the reference frame is obtained, and the track feature point set at the current acquisition time is registered one-to-one with the number. For each track feature point with the same number, the three-dimensional spatial coordinates at the current acquisition time and the reference frame acquisition time are recorded respectively. The difference between the horizontal coordinate of the track feature point at the current acquisition time and the horizontal coordinate of the same numbered track feature point at the reference frame acquisition time, the difference between the vertical coordinate and the vertical coordinate of the same numbered track feature point at the reference frame acquisition time, and the difference between the height coordinate and the height coordinate of the same numbered track feature point at the reference frame acquisition time are respectively used as the three-dimensional displacement components of the track feature point in three-dimensional space, and are combined to form the three-dimensional displacement vector of the track feature point.

[0028] This invention achieves the construction of a three-dimensional displacement vector for each track feature point with a consistent number by accurately registering the track feature point set with the track feature points in the reference frame index sequence. The method not only ensures the unique correspondence of the track feature point numbers at all acquisition times, but also improves the accuracy and consistency of track feature point displacement tracking. The high-precision construction of the three-dimensional displacement vector lays a solid foundation for subsequent deformation field analysis and settlement trend discrimination, and enhances the spatiotemporal tracking capability and spatial resolution of the overall detection system.

[0029] In this embodiment, the construction of the track settlement deformation map specifically includes: For each acquisition moment, the three-dimensional displacement vectors of all track feature points are obtained, and an adaptive clustering method based on spatial distance and the change of three-dimensional displacement components is used for the three-dimensional displacement components corresponding to the number of all track feature points. In the three-dimensional space, the track feature points are divided into several spatial heterogeneous partitions to form local adaptive sub-intervals. When performing spatial heterogeneity partitioning, for each orbit feature point number, the three-dimensional displacement components at continuous acquisition times are statistically analyzed. Correlation analysis is performed on the three-dimensional displacement component sequences within a sliding window. By comparing the Pearson correlation coefficients of the three-dimensional displacement component sequences of different orbit feature point numbers within the sliding window, if the correlation coefficient is greater than the preset correlation threshold, it is determined that the three-dimensional displacement components within the change trend window have the same trend and are classified into the same spatial heterogeneity partition. The preset correlation threshold is set based on the statistical analysis of historical track settlement monitoring data. Specifically, under normal operating conditions, the Pearson correlation coefficient distribution of all track feature point numbers in the three-dimensional displacement component sequence within the sliding window is statistically analyzed, and the correlation coefficient value corresponding to the 95th percentile is taken as the correlation threshold. Within each spatial heterogeneity partition, for all orbital boundary points, a weighted spatial interpolation method is used to adaptively weight the three-dimensional displacement components corresponding to the number of all orbital feature points in the corresponding local adaptive sub-interval, and to calculate the three-dimensional local deformation field vector of each orbital boundary point. The adaptive weighting method calculates the three-dimensional spatial distance between the orbit boundary point and each orbit feature point within the spatial heterogeneity partition, and statistically analyzes the three-dimensional displacement change amplitude of the orbit feature point in the most recent collection time. The interpolation weight is weighted and combined according to the normalized value of the inverse of the distance and the normalized value of the historical three-dimensional displacement change, which comprehensively reflects the spatial proximity and the representativeness of historical deformation, and realizes adaptive weighted fusion within the spatial heterogeneity partition. Arrange the three-dimensional local deformation field vectors of all track boundary points in an orderly manner according to the spatial coordinates of the track boundary points to establish a track settlement deformation map. The track settlement deformation map uses the spatial coordinates of the track boundary as an index to store the three-dimensional local deformation field vectors corresponding to each spatial index. By employing spatial structure similarity clustering and dynamic trend detection, deformation hotspots are automatically labeled, enabling visualization and real-time marking of spatial subsidence patterns and local abrupt change points.

[0030] In this embodiment, the local adaptive sub-interval is formed as follows: Calculate the spatial Euclidean distance between any two track feature point numbers in three-dimensional space, and combine it with the numerical changes of the three-dimensional displacement components. Track feature point numbers whose spatial Euclidean distance is less than a preset spatial distance threshold and whose three-dimensional displacement components have the same trend are automatically classified into the same spatial heterogeneity partition. After completing the partitioning of all track feature point numbers, the set of track feature point numbers in each spatial heterogeneity partition is defined as a local adaptive sub-interval.

[0031] This invention achieves high-precision estimation of the three-dimensional local deformation field of the track boundary and construction of the track settlement deformation map by using adaptive clustering and spatial interpolation methods based on three-dimensional displacement vectors. The method can fully reflect the local deformation characteristics of different spatial locations of the track boundary, and realizes visualization and real-time annotation of track settlement spatial patterns and abnormal areas through spatial heterogeneity partitioning and hotspot detection. Compared with traditional static analysis methods, this invention improves the spatial fineness and dynamic response capability of track settlement assessment.

[0032] In this embodiment, the generation of the orbital stability scoring sequence specifically includes: For each acquisition time, the three-dimensional local deformation field components of all track feature point numbers at the current acquisition time are extracted from the track settlement deformation map. The difference between the three-dimensional local deformation field components of each track feature point number at the current acquisition time and the previous acquisition time is calculated. The change amplitude of the three-dimensional local deformation field components of all track feature point numbers is averaged to obtain the overall displacement change rate of the track at the corresponding acquisition time. At the same acquisition time, based on the spatial heterogeneity partition, all orbital feature points in each spatial heterogeneity partition are numbered, the variation amplitude of the three-dimensional local deformation field components is calculated, the variance of the variation amplitude of all three-dimensional local deformation field components in the spatial heterogeneity partition is calculated, and the variance is used as the local deformation fluctuation amplitude of the spatial heterogeneity partition. For each acquisition moment, the overall displacement change rate of the track and the local deformation fluctuation amplitude of all spatial heterogeneous zones are multiplied by preset weighting coefficients and then summed. The resulting value is used as the track stability score for the current acquisition moment. The orbit stability scores at all acquisition times are arranged in the order of acquisition time to form an orbit stability score sequence.

[0033] This invention establishes a dynamic evaluation mechanism for track stability scoring and scoring sequence by comprehensively analyzing the overall displacement change rate of the track and the local deformation fluctuation amplitude of spatial heterogeneity based on track settlement deformation diagram. It can accurately reflect the stability change trend of the track at different times and in different spatial regions, and realize the time-series dynamic quantitative monitoring of the entire track settlement process.

[0034] In this embodiment, the triggering of the weak settlement early warning and the sudden settlement alarm specifically includes: For the track stability score sequence, a sliding window distribution statistical method is used to dynamically set the weak settlement response threshold and the sudden settlement alarm threshold. The weak settlement response threshold is always less than the sudden settlement alarm threshold, and the two are automatically adjusted according to the current track stability score sequence distribution. As the data collection progresses, the sliding window continues to slide forward. Each new data collection moment dynamically updates the weak settlement response threshold and the sudden settlement alarm threshold based on the latest score distribution within the window, achieving real-time adaptive adjustment of the thresholds. The sliding window length N and percentiles P1 and P2 can be set according to actual monitoring needs. The score value at each acquisition time in the track stability score sequence is judged by two thresholds. If the score value is greater than or equal to the weak settlement response threshold and less than the sudden settlement alarm threshold, a weak settlement early warning prompt is immediately triggered. If the score is greater than or equal to the sudden settlement alarm threshold, a sudden settlement alarm will be triggered immediately.

[0035] In this embodiment, the dynamic setting of the weak settlement response threshold and the sudden settlement alarm threshold specifically includes: In the orbit stability scoring sequence, for the current acquisition time, a continuous scoring value including the current acquisition time and the previous N-1 acquisition times is selected to form a sliding window scoring subsequence of length N; Sort all scores within the sliding window scoring subsequence, and extract the scores corresponding to the P1 percentile and P2 percentile respectively, which are used as the weak settlement response threshold and sudden settlement alarm threshold within the current sliding window.

[0036] Example 1

[0037] To verify the feasibility of this invention in a real-world scenario, it was applied to the daily operation and maintenance monitoring of a container crane track at a port. This port is located in a coastal economically developed area, where cranes operate frequently, placing extremely high demands on the safety and timeliness of track settlement monitoring and maintenance. Due to the complex soil structure and significant variations in foundation bearing capacity in the track laying area, traditional methods using manual inspections and point sensors are insufficient to cover the entire operating section and are easily affected by weather, terrain, and human oversights, making it difficult to detect and intervene in track settlement anomalies in a timely manner.

[0038] In the practical application of this invention, dedicated camera equipment is deployed along the entire track to continuously collect image data of the track area during crane operation. The image preprocessing module performs median filtering, grayscale normalization, and brightness equalization on the original track image set, eliminating interference from changes in ambient lighting and stray reflections. This improves the stability of subsequent feature point extraction and boundary analysis. The system automatically extracts the target track area and track boundary contour from the standard track image set, and uses grayscale gradient and polynomial curve fitting to achieve equidistant distribution modeling of track feature points in three-dimensional space. The feature point number, spatial coordinates, and boundary information at each moment are synchronized to the track feature point database in real time.

[0039] As the daily scheduling of cranes and track operation progress, the system automatically constructs a spatiotemporal track feature map. By calculating the change amplitude of the state vector of track feature points and the change rate of adjacent edge weights, it intelligently selects representative reference frames. Based on the reference frame index sequence, it performs precise registration and comparison of the track feature point set at all acquisition times. Based on the unique correspondence of the numbers and three-dimensional spatial coordinates, the system calculates the three-dimensional displacement vector of each track feature point with high precision, providing a detailed data foundation for dynamic settlement modeling of track boundaries.

[0040] Based on the three-dimensional displacement vector, spatial Euclidean distance and historical displacement change analysis are used to partition all track feature points into spatial heterogeneity zones. Sub-intervals with consistent local settlement trends are automatically identified and marked, achieving fine stratification of the spatial distribution of track settlement. Through weighted spatial interpolation and deformation field calculation, the system automatically generates track settlement deformation maps and marks settlement hotspots and abnormal areas based on spatial structural similarity and dynamic trends, making it convenient for on-site maintenance personnel to locate key areas of concern as soon as possible.

[0041] Furthermore, the system calculates the overall track displacement change rate and the local deformation fluctuation amplitude of each spatially heterogeneous zone in real time, dynamically generating a track stability score sequence. Based on the score sequence, a dynamic dual-threshold discrimination strategy is constructed, enabling graded responses to weak settlement and sudden settlement: when the track stability score approaches the warning threshold, the system automatically pushes warning information to the maintenance center; once the score exceeds the sudden settlement threshold, a linkage command is automatically generated, and the control system realizes the dynamic limit, scheduling adjustment, or shutdown protection of the crane.

[0042] During the implementation of this invention, all monitoring, analysis, early warning, and response are automated and continuously executed without manual intervention, effectively improving the timeliness and accuracy of track settlement anomaly detection. Compared with the previous methods that relied on discrete point sensors or inspections, the solution based on image recognition and three-dimensional dynamic analysis greatly improves spatial coverage and dynamic response capabilities. The system identifies local track settlement trends multiple times during continuous operation cycles and issues early warning prompts. Combined with subsequent maintenance and limiting measures, it reduces potential safety hazards, ensures the efficient and safe operation of cranes, and provides solid data support and decision-making basis for track maintenance and operation management.

[0043] To verify the performance of the present invention in practice, it was compared with traditional methods, and the results are shown in Table 1.

[0044] Table 1. Performance Comparison of Image Recognition-Based Crane Rail Settlement Detection Method and Traditional Methods Detection accuracy 92.6% 83.2% Average response time 12 minutes 2 hours Spatial resolution 0.5 meters 5 meters Warning timeliness 98% timely 65% timely Artificial dependence Low high False alarm rate 3.2% 9.5% Continuous monitoring capability Supports 24 / 7 full process Only scheduled or random checks Settlement hotspot identification capability Supports precise spatial positioning Difficult to achieve As can be seen from Table 1, the image recognition-based real-time detection method for crane track settlement of the present invention outperforms traditional manual inspection and point sensor solutions in all key performance indicators. In terms of detection accuracy, the present invention achieves a high accuracy of 92.6% through multi-frame image three-dimensional dynamic analysis and feature point structured registration, which is far higher than the traditional solution. The average response time is significantly shortened, and abnormal information can be automatically detected and pushed within 12 minutes. In contrast, traditional methods mostly rely on manual or periodic data reporting, and the response time is generally on the order of several hours, which cannot meet the real-time monitoring needs of emergencies.

[0045] In terms of spatial resolution, this invention employs high-density feature point sampling and full-track coverage, improving accuracy to 0.5 meters. This effectively supports the identification of local hotspots and minute deformations, achieving a warning timeliness of up to 98%. It can quickly link with the control system to implement warnings and protection. Traditional solutions, limited by inspection frequency and sensor deployment density, have lower timeliness. Because the core process is fully automated, the invention has extremely low reliance on manual intervention, effectively avoiding human error and subjective misjudgment. The false alarm rate is controlled at 3.2%, lower than the traditional 9.5%, reducing interference from invalid alarms.

[0046] In addition, this invention enables all-weather, continuous monitoring, enhancing the system's ability to manage the entire life cycle of the track. Settlement hotspots can be precisely located in space, making it convenient for maintenance personnel to carry out targeted repairs and adjustments. Traditional methods, due to their limited coverage, often fail to detect potential hazards in a timely and comprehensive manner.

[0047] In summary, the core reason for the performance improvement lies in the fact that this invention utilizes three-dimensional image recognition, spatial heterogeneity partitioning, dynamic threshold discrimination, and full-process automation technologies to achieve high-density, high-timeliness, and high-intelligence track settlement monitoring and early warning, thereby improving safety protection capabilities and operation and maintenance efficiency.

[0048] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for real-time detection of crane track settlement based on image recognition, characterized in that, Includes the following steps: Image data of the track area during crane operation is collected to obtain a set of original track images, and image preprocessing is performed to generate a set of standard track images; Extract the target region of the track from the standard track image set, and extract the track boundary contour to construct a set of track feature points for each frame of the image; The set of orbit feature points is constructed into a spatiotemporal orbit feature map. The graph structure is analyzed. When the state change amplitude and the adjacent edge weight change rate meet the adaptive criterion, the corresponding frame is identified and selected as the reference frame image, and a reference frame index sequence is generated. The set of orbit feature points is registered and compared with the feature points in the corresponding frames of the reference frame index sequence to construct a three-dimensional displacement vector for each feature point. The three-dimensional local deformation field vector of the track boundary is estimated based on the three-dimensional displacement vector, and the track settlement deformation map is constructed. Based on the track settlement deformation diagram, the overall track displacement change rate and local deformation fluctuation amplitude are calculated to form a track stability score and generate a track stability score sequence. A dynamic dual-threshold discrimination strategy is constructed based on the track stability scoring sequence. A weak settlement response threshold and a sudden settlement alarm threshold are set to trigger weak settlement early warning and sudden settlement alarm respectively.

2. The real-time detection method for crane track settlement based on image recognition according to claim 1, characterized in that, The image preprocessing includes median filtering, grayscale normalization, brightness equalization, and edge enhancement.

3. The real-time detection method for crane track settlement based on image recognition according to claim 1, characterized in that, The construction of the orbit feature point set includes: For the set of standard track images, the target region of track is extracted for each frame of the standard track image. The threshold segmentation method is used to compare the gray value of all pixels in the standard track image with the set segmentation threshold. When the gray value of a pixel is higher than the segmentation threshold, the corresponding pixel is included in the target region of track. All pixels that meet the conditions are marked with three-dimensional spatial coordinates to form a set of target regions of track. Based on the set of target regions of the track, the track boundary contour is extracted for each frame of the standard track image, the gray level gradient of all pixels is calculated, and for pixels whose gray level gradient is greater than the boundary discrimination threshold, the corresponding three-dimensional spatial coordinates are recorded to the set of track boundary contours. The track boundary profile is fitted with a polynomial curve using a set of track boundary profiles, and the polynomial coefficients are determined by the least squares method to obtain a three-dimensional polynomial curve model describing the track boundary profile. On the three-dimensional polynomial curve model of the track boundary profile, the distribution range of feature points is set according to the principle of equal interval sampling. The track boundary profile is divided into several equally spaced intervals within the minimum and maximum three-dimensional coordinate intervals. The three-dimensional spatial coordinates of the sampling points in each interval are calculated to obtain the set of track feature points. The number of all track feature points and the sampling interval are configured according to preset parameters.

4. The real-time detection method for crane track settlement based on image recognition according to claim 1, characterized in that, The construction of the reference frame index sequence specifically includes: The set of track feature points for each frame of the standard track image is obtained and marked one by one according to the acquisition time and frame number. All the sets of track feature points form a time-series track feature point set sequence. The three-dimensional spatial coordinates, pixel grayscale values, and edge response values ​​of the orbit feature points are collected at each moment, and the orbit feature point state vector is constructed based on these values. Construct a spatiotemporal orbit feature map from a sequence of temporal orbit feature point sets; Traverse all nodes in the spatiotemporal orbit feature map, calculate the state change amplitude and adjacent edge weight change rate of each node during continuous acquisition time, and calculate the average of the state change amplitude and adjacent edge weight change rate as the global state change amplitude and global adjacent edge weight change rate, and compare the global state change amplitude and global adjacent edge weight change rate with the adaptive criterion respectively. When the magnitude of the global state change and the rate of change of the global adjacent edge weight both exceed the adaptive threshold at the acquisition time, the standard track image with the frame number corresponding to the acquisition time is selected as the reference frame image, and all acquisition times and frame numbers that meet the conditions are included in the reference frame index. For each reference frame image, the corresponding acquisition time and frame number are recorded to form a reference frame index sequence.

5. The real-time detection method for crane track settlement based on image recognition according to claim 1, characterized in that, The construction of the three-dimensional displacement vector specifically includes: For each acquisition time of the standard orbit image, all orbit feature points are extracted from the orbit feature point set and marked one-to-one according to the orbit feature point number. Based on the acquisition time and frame number of each reference frame in the reference frame index sequence, the corresponding reference frame image is extracted from the track standard image set, the track feature point set at the acquisition time of the reference frame is obtained, and the track feature point set at the current acquisition time is registered one-to-one with the number. For each track feature point with the same number, the three-dimensional spatial coordinates at the current acquisition time and the reference frame acquisition time are recorded respectively. The difference between the horizontal coordinate of the track feature point at the current acquisition time and the horizontal coordinate of the same numbered track feature point at the reference frame acquisition time, the difference between the vertical coordinate and the vertical coordinate of the same numbered track feature point at the reference frame acquisition time, and the difference between the height coordinate and the height coordinate of the same numbered track feature point at the reference frame acquisition time are respectively used as the three-dimensional displacement components of the track feature point in three-dimensional space, and are combined to form the three-dimensional displacement vector of the track feature point.

6. The real-time detection method for crane track settlement based on image recognition according to claim 1, characterized in that, The construction of the orbital settlement deformation map specifically includes: For each acquisition moment, the three-dimensional displacement vectors of all track feature points are obtained, and an adaptive clustering method based on spatial distance and the change of three-dimensional displacement components is used for the three-dimensional displacement components corresponding to the number of all track feature points. In the three-dimensional space, the track feature points are divided into several spatial heterogeneous partitions to form local adaptive sub-intervals. Within each spatial heterogeneity partition, for all orbital boundary points, a weighted spatial interpolation method is used to adaptively weight the three-dimensional displacement components corresponding to the number of all orbital feature points in the corresponding local adaptive sub-interval, and to calculate the three-dimensional local deformation field vector of each orbital boundary point. The three-dimensional local deformation field vectors of all track boundary points are arranged in an orderly manner according to the spatial coordinates of the track boundary points to establish a track settlement deformation map. The track settlement deformation map uses the spatial coordinates of the track boundary as an index to store the three-dimensional local deformation field vectors corresponding to each spatial index.

7. The real-time detection method for crane track settlement based on image recognition according to claim 6, characterized in that, The formation of the local adaptive sub-interval is as follows: Calculate the spatial Euclidean distance between any two track feature point numbers in three-dimensional space, and combine it with the numerical changes of the three-dimensional displacement components. Track feature point numbers whose spatial Euclidean distance is less than a preset spatial distance threshold and whose three-dimensional displacement components have the same trend are automatically classified into the same spatial heterogeneity partition. After completing the partitioning of all track feature point numbers, the set of track feature point numbers in each spatial heterogeneity partition is defined as a local adaptive sub-interval.

8. The real-time detection method for crane track settlement based on image recognition according to claim 1, characterized in that, The generation of the orbital stability score sequence specifically includes: For each acquisition time, the three-dimensional local deformation field components of all track feature point numbers at the current acquisition time are extracted from the track settlement deformation map. The difference between the three-dimensional local deformation field components of each track feature point number at the current acquisition time and the previous acquisition time is calculated. The change amplitude of the three-dimensional local deformation field components of all track feature point numbers is averaged to obtain the overall displacement change rate of the track at the corresponding acquisition time. At the same acquisition time, based on the spatial heterogeneity partition, all orbital feature points in each spatial heterogeneity partition are numbered, the variation amplitude of the three-dimensional local deformation field components is calculated, the variance of the variation amplitude of all three-dimensional local deformation field components in the spatial heterogeneity partition is calculated, and the variance is used as the local deformation fluctuation amplitude of the spatial heterogeneity partition. For each acquisition moment, the overall displacement change rate of the track and the local deformation fluctuation amplitude of all spatial heterogeneous zones are multiplied by preset weighting coefficients and then summed. The resulting value is used as the track stability score for the current acquisition moment. The orbit stability scores at all acquisition times are arranged in the order of acquisition time to form an orbit stability score sequence.

9. The real-time detection method for crane track settlement based on image recognition according to claim 1, characterized in that, The triggering of the weak settlement early warning and sudden settlement alarm specifically includes: For the track stability score sequence, a sliding window distribution statistical method is used to dynamically set the weak settlement response threshold and the sudden settlement alarm threshold. The weak settlement response threshold is always less than the sudden settlement alarm threshold, and the two are automatically adjusted according to the current track stability score sequence distribution. The score value at each acquisition time in the track stability score sequence is judged by two thresholds. If the score value is greater than or equal to the weak settlement response threshold and less than the sudden settlement alarm threshold, a weak settlement early warning prompt is immediately triggered. If the score is greater than or equal to the sudden settlement alarm threshold, a sudden settlement alarm will be triggered immediately.

10. A real-time detection method for crane track settlement based on image recognition according to claim 9, characterized in that, The dynamic setting of the weak settlement response threshold and the sudden settlement alarm threshold specifically includes: In the orbit stability scoring sequence, for the current acquisition time, a continuous scoring value including the current acquisition time and the previous N-1 acquisition times is selected to form a sliding window scoring subsequence of length N; Sort all scores within the sliding window scoring subsequence, and extract the scores corresponding to the P1 percentile and P2 percentile respectively, which are used as the weak settlement response threshold and sudden settlement alarm threshold within the current sliding window.

Images