Car coupler centering state video detection method suitable for low-quality image

Through binocular camera and image processing technology, real-time detection of the hook centering status is achieved, the problems of low manual detection efficiency and poor accuracy are solved, and the automation level and operation efficiency of flexible train marshalling are improved.

CN119068387AActive Publication Date: 2024-12-03TONGJI UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411059489.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-04
Publication Date
2024-12-03
Estimated Expiration
2044-08-04

AI Technical Summary

Technical Problem

In the prior art, the detection of the hook pair mainly relies on manual assistance, which is low in efficiency and high in cost, cannot meet the needs of flexible marshalling of urban rail transit, and it is difficult to achieve accurate detection under low-quality image conditions.

Method used

The binocular camera collects images for pre-processing. Through Gaussian blur, edge extraction, expansion and clustering, the characteristic points of the hook are screened out, combined with K-means clustering and data smoothing, real-time detection of the hook centering state is achieved. Euro-style distance and reference line angle are used to evaluate the status of the train couple.

Benefits of technology

It significantly improves the automation level of hook detection, realizes high reliability detection under low-quality image conditions, and promotes flexible marshaling and operation efficiency of online joint hooking and decomposition of trains.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068387A_ABST
    Figure CN119068387A_ABST
Patent Text Reader

Abstract

The invention relates to a coupler centering state video detection method suitable for a low-quality image. The method comprises the following steps: collecting and preprocessing a coupler video; using a K-means clustering method to lock an interested interval, setting an initial position in a target interval according to a clustering result, and performing iterative preheating to obtain a reliable position; based on the preheating result, detecting the position of a target point in the target area in real time and comparing the position with a reliable position to obtain a feature point 1; detecting a feature point 2 in real time by using a minimum rectangular region detection algorithm; according to the initial reliable positions of the point 1 and the point 2, connecting a datum line of the coupler in the video; in each frame of detection, the real-time positions of the point 1 and the point 2 are detected, and a connecting line of the two points is used as a target line. Calculating an included angle between the reference line and the target line; and the coupler centering state is judged and fed back.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of intelligent operation and maintenance of rail transit, and relates to a video detection method for coupler centering status suitable for low-quality images. Background Art

[0002] The online train unmarshaling technology enables trains to complete coupling and unmarshaling tasks at any position on the main line, improves the flexibility and operability of train marshaling operations, and saves manpower, material resources and time costs; this technology can enable flexible marshaling and mixed operation of lines, improve the efficiency of urban rail transit operations, enhance residents' travel experience, and save energy and reduce emissions. In the application theory of online train unmarshaling technology, the core requirement is to detect the status of the couplers between trains anytime and anywhere, determine the real-time status of the couplers, and then determine whether the train has the conditions for coupling and unmarshaling based on the coupler status.

[0003] At present, the operation of coupler centering is mainly carried out through manual assistance on the ground, which requires the driver to get off the vehicle to check the status of the coupler. The manually assisted centering method has low automation, low efficiency, high cost, and high site requirements, which is not conducive to meeting the requirements of flexible formation and operation of urban rail transit. Therefore, improving the level of intelligence in the coupler alignment detection process has become a future development trend. Summary of the invention

[0004] A method for detecting a vehicle coupler centering state using a video of a low-quality image, comprising the following steps:

[0005] S1, pre-process the image collected by the binocular camera, and perform Gaussian blur processing on the image I1 with a pixel size of x*y to obtain I 1-gus , extract edge I 1-edgp And perform expansion operation to obtain edge set I 1-edgA , search I 1-edgA Get contour set I 1-conA , filter contour set I 1-conA The contour set I is obtained by taking the contour whose length-width ratio of the smallest circumscribed regular rectangle is within a certain range. 1-filA , the center point of the smallest circumscribed regular rectangle is written into point set I 1-poi , and can be combined with the contour set I 1-filA One to one correspondence;

[0006] S2, for I 1-filA Cluster the contours in to obtain several clusters I 1-clu As a clustering result, the position of the centroid of each cluster is recorded in the point set I 1-ctr ;

[0007] S3, because the camera is facing the hook, the hook in the image must be near the central axis of the image. At the same time, the cluster with more than 450 points in the cluster is considered to be a valid cluster. Therefore, the cluster I with the centroid closest to the horizontal central axis x / 2 of I1 and the number of cluster points exceeding 450 is selected. 1-int , let the centroid of the cluster be I 1-ict ;

[0008] S4. I 1-int The contours corresponding to the points in are taken out one by one and written into the contour set I 1-ic , calculate the minimum circumscribed rectangular area frame of the contour, and the average width and height are represented as I 1-w and I 1-h , through I 1-ict ,I 1-w、 I 1-h Determine the interval of interest I 1-roi1 ;

[0009] S5. Select the first frame cluster centroid coordinate I 1-int1 As the initial value, the target point position is detected frame by frame based on the Euclidean distance, and the reliable position is updated according to the result, iterating 100 times;

[0010] S6. Use a warm-up algorithm based on historical reliable results to smooth the extraction results of each frame, and the obtained point is the feature point A;

[0011] S7, select another region of interest I in I1 1-roiB , perform Gaussian blur processing and grayscale processing on the image of the region, extract its edge, and dilate the edge to obtain the edge set I 1-edgB , search I 1-edgB , get the contour set I 1-conB ; Filter contour set I 1-conB The contours with aspect ratios within a certain range are obtained as contour set I 1-filB , traverse the contour set I 1-filB Find the contour with the smallest perimeter I 1-minB , contour I 1-minB The center point is the feature point B;

[0012] S8, record the characteristic points A and B in the coupling alignment state as reference points A1 and B1, connect A1 and B1 to obtain the reference line L1 in the coupling alignment state, the reference line is fixed during the whole detection process, and record the characteristic points A and B in each frame as the target point A i and B i , connect to get the target line L i , calculate L1, L i The angle θ is used to evaluate the current coupler state. When θ is less than a certain angle, it is judged that the coupler is in the center state, otherwise.

[0013] Furthermore, in the image preprocessing process described in S1, in order to prevent the loss of necessary edge information after blurring, a smaller convolution kernel is used while ensuring that the canny recognition result is relatively accurate, and the convolution kernel size is 1*1; after extracting the edge, according to the characteristics of the image, the size of the structural element of the expansion processing is set to a convolution kernel of 10*10, and the following operation is performed:

[0014]

[0015] Where B represents the structural element, z is the pixel position in the image, represents the expansion operation, represents the reflection and translation of the structural element B about point z. If (B) is at (z) and (I 1-gus ) is not empty, that is, (I 1-gus ) is highlighted (value is 1) under the coverage of (B), then in the expanded image, point (z) will also be set to be highlighted.

[0016] Furthermore, three data points are selected as the initial cluster centers and the K-means clustering method is used for iteration. The goal is to minimize the sum of the intra-cluster distances, that is, the sum of the distances from each point to its cluster center.

[0017] Its mathematical formula can be expressed as:

[0018]

[0019] (J) is the cost function, (I 1-clu ) represents the allocation of clusters, (I 1-ctr ) represents the cluster center, (k) is the number of clusters, is the set of points in the (i)th cluster, is the center of the (i)th cluster and (x) is a point in the cluster.

[0020] Further, the cluster of interest I described in S3 1-int According to the geometric position relationship between the camera and the coupler installation, the point set I is selected. 1-ctr The cluster where the points closer to the horizontal center axis x / 2 of I1 are located; at the same time, in order to ensure the distribution stability of the points and the universality in the video, the number of points passing through the cluster is required to exceed 450 within 500 frames.

[0021] Further, the interested interval I in S4 1-roi1 I 1-int The contours are taken out frame by frame and written into the contour set I 1-ic , calculate the average width and height of the rectangular outline respectively 1-w and I 1-h, the interval of interest is a rectangle with two sides parallel to the horizontal line, and the coordinates of a pair of diagonal points a and b are:

[0022]

[0023]

[0024] x ict ,y ict Cluster I in S4 1-int The center of mass I 1-ict The horizontal and vertical coordinates.

[0025] Furthermore, the updating of the reliable position described in S5 selects the first frame cluster centroid coordinate I 1-int1 As the initial value, the Euclidean distance between the target point and the reliable position in the interval of interest is calculated frame by frame, and the points close to the reliable position are screened out. These points are weighted averaged with the reliable position to obtain the updated reliable position. This process is iterated 100 times to complete the warm-up operation and obtain the initial reliable position.

[0026] Furthermore, the determination of feature point A described in S6 is based on the reliable position obtained by iterative updating in S5, and the Euclidean distance between the identified point in the interval of interest of each frame and the reliable position is calculated, and the point with the smallest distance among the points with a Euclidean distance less than or equal to 20 pixels is taken as the recognition result of the frame, that is, feature point A.

[0027] Further, the characteristic point A and the characteristic point B in the coupler centering state in S8 are connected to obtain the reference line L1 in the coupler alignment state. Indicates the direction of L1, the feature point A of each frame i and feature point B i Connect the target line L2 to To express the direction of L2, use the following formula:

[0028] θ = arctan 2(det,dot);

[0029] Used to evaluate the current coupler status. When |θ|<1°, Status:Good is fed back, indicating that the coupler is in the centered state. When |θ|>1°, Status:Bad is fed back, indicating that the angle is too large and the coupler is not centered.

[0030] Due to the limitations of stains and rust on the coupler, shooting light, environment and other conditions, the accurate identification of feature points is affected. The present invention adopts K-means clustering and data smoothing technology in data processing, as well as reliability judgment and result correction based on historical results, to achieve a high-reliability train coupler alignment real-time detection method based on video recognition technology. The present invention has a wide range of applications and can significantly improve the level of automation of flexible train marshaling and promote the development and application of online train coupling and uncoupling. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a real shot effect diagram of the detection object of the present invention.

[0032] Figure 2 It is a block diagram of the working principle of the present invention.

[0033] Figure 3 It is a schematic diagram of clusters obtained by the clustering step in a specific implementation manner of the present invention. DETAILED DESCRIPTION

[0034] The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, and are not intended to limit the protection scope of the present invention.

[0035] First, the binocular camera is installed on the reverse slope of the train head. After the device is installed and started normally, the 25 frames / s video information collected by the binocular camera is preheated with the first 500 frames for 20 seconds. Each frame of the image is preprocessed, and the image I1 with a pixel size of 1920*1080 is subjected to Gaussian blur processing with a convolution kernel of (1×1) to obtain I 1-gus ; Use Canny edge detection algorithm to extract edge I 1-edgp , and perform dilation processing with a convolution kernel of (10×10).

[0036] Use findContours function to get the contour set I 1-edgA , frame by frame search I 1-edgA And write it into a list to get the contour set I 1-conA , filter contour set I 1-conA The contours of the smallest circumscribed regular rectangle with an aspect ratio range of 0.67-1.5 and an area range of 40000-200000 pixels are obtained as contour set I 1-filA The minimum circumscribed regular rectangle refers to the smallest rectangle that can exactly contain the contour and whose two sides are respectively perpendicular and parallel to the horizontal line.

[0037] For contour set I 1-filA For each contour in the minimum circumscribed regular rectangle, the center point is written into the point set I 1-poi , and can be combined with the contour set I 1-filA One to one correspondence.

[0038] like Figure 2 As shown, in S2, for point set I 1-poi , K-means algorithm is used for clustering.

[0039] K-means clustering requires a preset number of clusters. For each point in the data set, calculate its distance from each cluster center and assign it to the cluster represented by the nearest cluster center. Recalculate the center point of each cluster, that is, the mean of all points in the cluster, and repeat the above steps until the termination condition is met, such as the cluster center no longer changes. The goal of K-means clustering is to minimize the sum of the distances within the cluster.

[0040] Theoretically, for the same device, the relative position of the interval of interest collected after installation has almost no change. Therefore, after the experiment, three clusters were preset for clustering, and cluster I was obtained. 1-clu As a clustering result, the centroid position of each cluster is recorded in the point set I 1-ctr , and mark the cluster number to which each point belongs.

[0041] like Figure 2 As shown, in S3, since the camera shooting position is facing the coupler, the position of the coupler in the image must be near the central axis of the image.

[0042] Since the data set used for clustering is all the contour centers detected in each frame within 500 frames, for pictures of the same position in different time periods, ideally there should be 500 points corresponding to the static position in the picture. Considering the actual conditions, in order to ensure the effectiveness of the cluster, the occurrence rate of the points in the cluster should be greater than 90% of the number of video frames, so the cluster with more than 450 points in the cluster is considered to be a valid cluster.

[0043] like Figure 3 As shown, the cluster 0 whose centroid is closest to the horizontal center axis x / 2 of I1 and whose number of cluster midpoints exceeds 450 is selected. The centroid coordinates of this cluster are:

[0044] I 1-ict :(1114.06747405 913.82352941);

[0045] like Figure 2 As shown, in S4, I 1-int The contours corresponding to the points in are taken out one by one and written into the contour set I 1-ic , calculate the minimum circumscribed rectangular area frame of the contour, and the average width and height are represented as I 1-w and I 1-h , through I 1-ict ,I 1-w ,I 1-h Determine the interval of interest I 1-roi1, the interval of interest is a rectangle with two sides parallel to the horizontal line, and the coordinates of a pair of diagonal points a and b are:

[0046] a: (942,763); b (1285,1064).

[0047] like Figure 2 As shown, in S5, within the interval of interest I 1-roi1 ,To ensure the reliability of the detection results and facilitate the ,smoothing of the data, for each frame of real-time detection results, the ,reliable position is set as the historical reliable detection result of the ,device, and the value is weighted by the reliable results of all ,frames before the frame.

[0048] For reliable positions, the first 100 frames of video information are preheated using the first 4 seconds of detection. 1-ict As the initial value, in I 1-roi1 For each frame of the image, contour detection is performed, and contours with an area range of 400-1500 are screened, and the centroid of its minimum circumscribed regular rectangle is counted into the point set I 1-ppr , in I 1-ppr In the above example, the feature point position is detected frame by frame based on the Euclidean distance from the reliable position, and the weighted average of all points with the minimum distance or a distance not greater than 10 is taken as the reliable result I of the frame. 1-rst , and based on the weighted average of the result and the reliable position, the reliable position is updated.

[0049] like Figure 2 As shown, in S6, the above method iterates the reliable position 100 times; this reliable position can be considered as a smoothed high-reliability recognition result. The preheating algorithm based on historical reliable results is used to smooth the extraction results of each frame. The smoothing method is the same as S5. The obtained point is the feature point A, as shown in Figure 1 shown.

[0050] like Figure 2 As shown, in S7, another region of interest I in I1 is selected 1-roiB The image in this area is subjected to Gaussian blurring with a convolution kernel of (5×5), grayscale processing, and then the Canny detection algorithm is used to extract the edge. The edge is then dilated with a convolution kernel of (3×3) to obtain the edge set I 1-edgB , search I 1-edgB , get the contour set I 1-conB ; Filter contour set I 1-conB The contours whose minimum circumscribed rectangle aspect ratio is within 1 to 1.5 and whose minimum circumscribed rectangle area is between 600 and 1500 pixels are obtained as contour set I. 1-filB , traverse the contour set I 1-filB Find the contour with the smallest perimeter I 1-minB , contour I 1-minBThe center point of the smallest circumscribed rectangle is the feature point B, such as Figure 1 shown.

[0051] like Figure 1 As shown, in S8, the characteristic point A and the characteristic point B in the coupler centering state are connected to obtain the reference line L1 in the coupler alignment state. The characteristic point A of each frame i and feature point B i The target line L2 is connected and the angle values ​​of L1 and L2 are calculated using vector coordinates. It is used to evaluate the current coupler status. When |θ|<1°, Status:Good is fed back, indicating that the coupler is in the centered state; when |θ|>1°, Status:Bad is fed back, indicating that the angle is too large and the coupler is not centered. Figure 1 As shown, at this time |θ| is 7°, the coupler is not aligned, and the device alarms.

[0052] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting the centering status of a coupler using a video of a low-quality image, characterized in that: The following steps are involved: S1, pre-process the image collected by the binocular camera, and perform Gaussian blur processing on the image I1 with an image resolution of x*y to obtain I 1-gus , extract edge I frame by frame within a certain number of video frames 1-edgp , dilate all edges extracted from all frames and record them in edge set I 1-edgA , search I 1-edgA Get contour set I 1-conA , filter contour set I 1-conA The contour set I is obtained by taking the contour whose length-width ratio of the smallest circumscribed regular rectangle is within a certain range. 1-filA , the center point of the smallest circumscribed regular rectangle is written into point set I 1-poi , and can be combined with the contour set I 1-filA One to one correspondence; S2, for I 1-filA Cluster the contours in to obtain several clusters I 1-clu As a clustering result, the position of the centroid of each cluster is recorded in the point set I 1-ctr ; S3. Since the camera shooting position is facing the coupler, the position of the coupler in the image must be near the central axis of the image. At the same time, under ideal conditions, for the same position, the contour recognition result of each frame has only one point corresponding to it, and the contour appearance rate is 100%. At this time, the number of points in the corresponding position cluster should be the same as the number of video frames. However, due to the limitations of actual conditions, it is considered that the cluster with a contour appearance rate of 90%, that is, the number of points in the cluster ≥ the number of preheated video frames × 0.9 is a valid cluster. Therefore, the effective cluster I with the cluster centroid closest to the horizontal center axis x / 2 of I1 is selected. 1-int , let the centroid of the cluster be I 1-ict ; S4. I 1-int The contours corresponding to the points in are taken out one by one and written into the contour set I 1-ic , calculate the minimum circumscribed rectangular area frame of the contour, and the average width and height are represented as I 1-w and I 1-h , through I 1-ict ,I 1-w、 I 1-h Determine the interval of interest I 1-roi1 ; S5. Select the first frame cluster centroid coordinate I 1-int1 As the initial value, the target point position is detected frame by frame based on the Euclidean distance, and the reliable position is updated according to the result until the coordinate pixel distance between the frame coordinate result and the last reliable position is less than 1.5; S6. Use a warm-up algorithm based on a reliable position to smooth the extraction result of each frame, and the obtained point is the feature point A; S7, select another region of interest I in I1 1-roiB , perform Gaussian blur processing and grayscale processing on the image of the region, extract its edge, and dilate the edge to obtain the edge set I 1-edgB , search I 1-edgB , get the contour set I 1-conB ; Filter contour set I 1-conB The contours with aspect ratios within a certain range are obtained as contour set I 1-filB , traverse the contour set I 1-filB Find the contour with the smallest perimeter I 1-minB , contour I 1-minB The center point is the feature point B; S8, record the characteristic points A and B in the coupling alignment state as reference points A1 and B1, connect A 1、 B1 obtains the reference line L1 in the coupling alignment state. The reference line is fixed during the entire detection process. The feature points A and B in each frame are recorded as the target point A. i and B i , connect to get the target line L i , calculate L1, L i The angle θ is used to evaluate the current coupler state. When θ is less than 1°, the coupler is considered to be in a safe state.

2. The method for detecting the centering state of a coupler using a video of a low-quality image according to claim 1, characterized in that: In the image preprocessing process described in S2, in order to prevent the loss of necessary edge information after blurring, a smaller convolution kernel is used while ensuring that the canny recognition result is relatively accurate. The convolution kernel size is 1*1. After extracting the edge, according to the characteristics of the image, the size of the structural element of the expansion process is set to a convolution kernel of 10*10, and the following operation is performed: Where B represents the structural element, z is the pixel position in the image, and ⊕ represents the dilation operation. represents the reflection and translation of the structural element B about point z; if At (z) and (I 1-gus ) is not empty, that is, (I 1-gus ) has at least one element in is highlighted (value is 1) under the coverage, then in the expanded image, point (z) will also be set to be highlighted.

3. The method for detecting the centering status of a coupler using a video of a low-quality image according to claim 1, characterized in that: The clustering process of the outline described in S3 selects 3 data points as the initial cluster centers and uses the K-means clustering method for iteration, with the goal of minimizing the sum of intra-cluster distances, that is, the sum of the distances from each point to its cluster center.

4. The method for detecting the centering state of a coupler using a video of a low-quality image according to claim 1, characterized in that: As described in S4, cluster I 1-int According to the geometric position relationship between the camera and the coupler installation, the point set I is selected. 1-ctr The cluster where the points closer to the horizontal center axis x / 2 of the middle distance I1 are located; at the same time, in order to ensure the distribution stability of the point and the universality of the point in the video, the appearance rate of the contour at this position is required to be 90%, that is, the number of points in the cluster ≥ the number of preheated video frames × 0.

9.

5. The method for detecting the centering state of a coupler using a video of a low-quality image according to claim 1, characterized in that: S5 describes the interval of interest I 1-roi1 I 1-int The contours are taken out frame by frame and written into the contour set I 1-ic , calculate the average width and height of the rectangular outline respectively 1-w and I 1-h , the interval of interest is a rectangle with two sides parallel to the horizontal line, and the coordinates of a pair of diagonal points a and b are: x ict ,y ict Cluster I in S4 1-int The center of mass I 1-ict The horizontal and vertical coordinates of .

6. The method for detecting the centering status of a coupler using a video of a low-quality image according to claim 1, characterized in that: The update of the reliable position described in S5 selects the first frame cluster centroid coordinate I 1-int1 As the initial value, the Euclidean distance between the target point and hist in the interval of interest is calculated frame by frame, and the points close to the reliable position are screened out. These points are weighted averaged with the hist to obtain the updated hist, until the coordinate result of this frame is less than 1.5 pixels away from the coordinate of the last reliable position, the warm-up operation is completed, and the initial reliable position is obtained.

7. The method for detecting the centering state of a coupler using a video of a low-quality image according to claim 1, characterized in that: The determination of feature point A described in S6 is based on the reliable position obtained by iterative updating in S5, and the Euclidean distance between the identified point in the interval of interest of each frame and the reliable position is calculated, and the point with the smallest distance among the points with a Euclidean distance less than or equal to 20 pixels is taken as the recognition result of the frame, that is, feature point A.

Citation Information

Patent Citations

  • Horizontal centring measuring device and method for assembly of sliding bearing

    CN104759875A

  • Coupler centering automatic measurement method

    CN110470248A

  • Car coupler joist breaking detection method, system and device based on image processing

    CN112233094A

  • Method and system for selecting uniform view angle frame sequence in equipment inspection video

    CN118247702A

  • Image processing method, image processing apparatus, and non-transitory storage medium

    US20240078646A1