A wire rope derailment warning method and system for container lifting pulley block

Abstract

The present invention discloses a wire rope derailment warning method and system for container lifting pulley block, including: collecting real-time images of pulley block and wire rope, realizing automatic monitoring of wire rope running state through feature point detection and geometric transformation estimation; and using structural knowledge of pulley block, accurate derailment detection is performed in the case of a small number of samples or even almost no samples, reducing the cost of data preparation and model training. The present invention is easy to operate, and the user only needs to manually specify the mask of the pulley area, beam area and wire rope area, and mark the key structural feature points, and the system can automatically extract features and perform subsequent automatic detection, with good user-friendliness and adaptability.

Description

Technical Field

[0001] The invention belongs to the technical field of container detection, and relates to a wire rope derailment early warning method and system for a container hoisting pulley block. Background Art

[0002] In port container lifting equipment, pulley blocks and wire ropes are key load-bearing and moving components. Traditional derailment detection methods mainly rely on regular manual inspections and monitoring systems based on mechanical sensors. However, manual inspections have the following disadvantages: high labor intensity, low detection efficiency, and strong subjectivity. Although the monitoring system based on mechanical sensors can achieve automated detection, it also has problems such as complex installation, difficult maintenance, and complex data processing. With the development of port automation and intelligence, higher requirements are placed on the detection methods of lifting equipment, and existing technologies are difficult to meet the needs of efficient, real-time, and accurate derailment detection. Summary of the invention

[0003] The purpose of the present invention is to solve the shortcomings of the prior art in manual inspection, such as high labor intensity, low detection efficiency, strong subjectivity, etc.; when the sensor performs detection, there is a problem of complex data processing, and to provide a wire rope derailment warning method and system for a container lifting pulley block.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] A wire rope derailment warning method for container hoisting pulley block, comprising:

[0006] Collect real-time video data, capture static images from the video stream, and pre-process the static images;

[0007] Based on the preprocessed static image, the pulley area mask, the beam area mask and the wire rope area mask are obtained, and then the feature points of the respective areas are obtained;

[0008] Linear fitting is performed on the characteristic points of the pulley area mask and the wire rope area mask to obtain the fitted pulley groove straight line and wire rope straight line respectively;

[0009] Based on the current frame The feature points of the middle beam area and the pulley groove area are matched with the feature points of the beam area and the pulley groove area set at the time of initialization, and then the geometric transformation matrix is ​​obtained. and the geometric transformation matrix ;

[0010] Based on geometric transformation matrix , geometric transformation matrix , the pulley groove straight line at the time of initialization, the wire rope area mask at the time of initialization and the wire rope straight line at the time of initialization, and obtain the updated wire rope fitting straight line and the pulley groove fitting straight line;

[0011] A linearity check and a parallelism check are performed based on the updated wire rope fitting straight line and the pulley groove fitting straight line. If the linearity check or the parallelism check fails, it is determined that the wire rope is derailed or abnormal.

[0012] A further improvement of the present invention is:

[0013] Furthermore, the static image is preprocessed, specifically: the initialization frame of the static image Divide and obtain a static image of the pulley area , static image of beam area , static image of wire rope area , and then the pulley area static image , static image of beam area , static image of wire rope area Perform grayscale conversion to obtain grayscale images of the pulley area , grayscale image of the beam area and the grayscale image of the wire rope area , and the grayscale image of the pulley area , grayscale image of the beam area and the grayscale image of the wire rope area Gaussian blur processing is performed separately;

[0014] Among them, the grayscale image of the pulley area for:

[0015] (1)

[0016] in, Indicates converting a color image into a grayscale image;

[0017] Grayscale image of the pulley area Perform Gaussian blur processing, specifically:

[0018] (2)

[0019] in, Indicates Gaussian blur processing;

[0020] Grayscale image of the beam area for:

[0021] (3)

[0022] in, Indicates converting a color image into a grayscale image;

[0023] Grayscale image of the beam area Perform Gaussian blur processing, specifically:

[0024] (4)

[0025] in, Indicates Gaussian blur processing;

[0026] Grayscale image of wire rope area for:

[0027] (5)

[0028] in, Indicates converting a color image into a grayscale image;

[0029] Grayscale image of the wire rope area Perform Gaussian blur processing, specifically:

[0030] (6)

[0031] in, is the standard deviation, Indicates Gaussian blur processing.

[0032] Further, based on the pre-processed static image, the pulley area mask, beam area mask and wire rope area mask are obtained, specifically: manually specify the area where the pulley block, beam and wire rope are located, and generate the pulley area mask , beam area mask and wire rope area mask ;

[0033] The obtaining of the feature points of the respective regions is specifically as follows:

[0034] Masking in the pulley area based on the ORB algorithm Internal detection feature points:

[0035] (7)

[0036] in, Indicates the pulley area mask The number of key feature points detected within The value range is between 400 and 800;

[0037] Masking in the beam area based on the ORB algorithm Internal detection feature points:

[0038] (8)

[0039] in, Represents the beam area mask The number of key feature points detected within The value range is between 400 and 800;

[0040] Masking in the wire rope area In the example, the image edges are extracted based on the Canny edge detection algorithm:

[0041] (9)

[0042] Among them, threshold1 and threshold2 represent two different thresholds in the Canny edge detection function. Pixels with gradient values ​​greater than threshold2 are marked as edges, pixels between threshold1 and threshold2 are considered potential edges, and pixels with gradient values ​​less than threshold1 are not edges.

[0043] Detect straight line segments based on Hough transform and extract straight line feature points:

[0044] (10)

[0045] in, Indicates the extraction of straight line feature points; Indicates the number of extracted straight line feature points, The value range is between 10 and 50.

[0046] Furthermore, linear fitting is performed on the feature points of the pulley area mask and the wire rope area mask to obtain the fitted pulley groove straight line and wire rope straight line, respectively, specifically:

[0047] The pulley groove straight line is the key structural feature point at the bottom of the pulley groove selected from the feature points of the pulley area mask. , by fitting the characteristic points at the bottom of the pulley groove The straight line obtained is recorded as ;in, is the number of key structural feature points at the bottom of the pulley groove, The value range is between 400 and 800;

[0048] The wire rope straight line is based on the least square method to the characteristic points Perform linear fitting and get a straight line , specifically:

[0049] (11)

[0050] in, Indicates fitting the straight line feature points into a straight line.

[0051] Furthermore, based on the current frame The feature points of the middle beam area and the pulley groove area are matched with the feature points of the beam area and the pulley groove area set at the time of initialization, and then the geometric transformation matrix is ​​obtained. and the geometric transformation matrix , specifically:

[0052] Based on the FLANN matching algorithm, the current frame Feature points in the middle beam area The initialization beam area feature point set Perform matching and obtain matching pairs:

[0053] (12)

[0054] in, Indicates that feature points are matched to obtain matching pairs;

[0055] Based on the RANSAC algorithm, the geometric transformation matrix of the beam area is estimated by matching pairs. , specifically:

[0056] (13)

[0057] in, Indicates that the geometric transformation matrix is ​​estimated based on the matching pairs;

[0058] Based on the FLANN matching algorithm, the current frame Feature points in the middle pulley groove area The initialization feature point set of the pulley groove area To match:

[0059] (14)

[0060] in, Indicates that feature points are matched to obtain matching pairs;

[0061] Based on the RANSAC algorithm, the geometric transformation matrix of the pulley groove area is estimated using matching pairs. , specifically:

[0062] (15)

[0063] in, Indicates that the geometric transformation matrix is ​​estimated based on the matching pairs.

[0064] Furthermore, based on the geometric transformation matrix , geometric transformation matrix , the pulley groove straight line at the time of initialization, the wire rope area mask at the time of initialization and the wire rope straight line at the time of initialization, and obtain the updated wire rope fitting straight line and pulley groove fitting straight line, specifically:

[0065] Set the pulley groove straight line at initialization Through the geometric transformation matrix Map to the current frame to get the pulley groove straight line of the current frame :

[0066] (16)

[0067] Based on geometric transformation matrix The wire rope area mask at initialization Map to the position of the current frame to get the wire rope area mask of the current frame , specifically:

[0068] (17)

[0069] In the current frame Wire rope area Re-check the wire rope lines , make sure the wire rope is in line with the pulley groove consistency.

[0070] Furthermore, a linearity check and a parallelism check are performed based on the updated wire rope fitting line and the pulley groove fitting line, specifically: based on the average distances between the updated wire rope fitting line and the pulley groove fitting line and the feature points in their respective regions, it is judged whether the average distances from the feature points in their respective regions to their respective fitting lines are less than a preset linearity threshold; if so, it is judged that the pulley groove feature points and the wire rope feature points satisfy the linearity check; it is judged again whether the angle between the pulley groove fitting line direction vector and the wire rope fitting line direction vector is greater than a preset angle threshold; if so, it is judged that the fixed pulley groove line is not parallel to the wire rope line; if so, it is judged that the pulley groove line is parallel to the wire rope line.

[0071] Further, it is determined whether the angle between the pulley groove fitting straight line direction vector and the wire rope fitting straight line direction vector is greater than a preset angle threshold, specifically:

[0072] The pulley groove straight line Direction vector for:

[0073] (18)

[0074] The wire rope is straight Direction vector for:

[0075] (19)

[0076] The direction vector With direction vector Angle for:

[0077] (20)

[0078] in, The straight line of the pulley groove The slope of The wire rope is straight The slope of

[0079] Set the angle threshold ;like , it is determined that the pulley groove straight line is not parallel to the wire rope straight line.

[0080] A wire rope derailment warning system for container hoisting pulley block, comprising:

[0081] A preprocessing module, wherein the preprocessing module collects real-time video data, captures static images from the video stream, and preprocesses the static images;

[0082] A first acquisition module, which acquires a pulley region mask, a beam region mask, and a wire rope region mask based on the preprocessed static image, and further acquires feature points of the respective regions;

[0083] A linear fitting module, wherein the linear fitting module performs linear fitting on the characteristic points of the pulley region mask and the wire rope region mask to obtain fitted pulley groove straight lines and wire rope straight lines respectively;

[0084] A matching module, wherein the matching module is based on the current frame The feature points of the middle beam area and the pulley groove area are matched with the feature points of the beam area and the pulley groove area set at the time of initialization, and then the geometric transformation matrix is ​​obtained. and the geometric transformation matrix ;

[0085] The second acquisition module is based on a geometric transformation matrix , geometric transformation matrix , the pulley groove straight line at the time of initialization, the wire rope area mask at the time of initialization and the wire rope straight line at the time of initialization, and obtain the updated wire rope fitting straight line and the pulley groove fitting straight line;

[0086] A verification module performs a linearity check and a parallelism check based on the updated wire rope fitting straight line and the pulley groove fitting straight line. If the linearity check or the parallelism check fails, it is determined that the wire rope is derailed or abnormal.

[0087] Compared with the prior art, the present invention has the following beneficial effects:

[0088] The present invention realizes automatic monitoring of the running status of the wire rope by collecting real-time images of the pulley block and the wire rope through feature point detection and geometric transformation estimation. At the same time, the present invention makes full use of the structural knowledge of the pulley block, and can still achieve efficient and accurate derailment detection with a small number of samples or even almost no samples, reducing the cost of data preparation and model training. The present invention is easy to operate. The user only needs to manually specify the masks of the pulley area, beam area and wire rope area, and mark the key structural feature points. The system can automatically extract features and perform subsequent automated detection, and has good user-friendliness and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0090] Figure 1 It is a schematic flow chart of a wire rope derailment warning method for a container hoisting pulley block of the present invention;

[0091] Figure 2 It is a structural schematic diagram of a wire rope derailment warning system for a container hoisting pulley block of the present invention;

[0092] Figure 3 It is a structural diagram of the steel wire rope derailment detection device of the present invention;

[0093] Figure 4 It is a schematic diagram of data flow in a wire rope derailment detection device;

[0094] Figure 5 A schematic diagram of a process for obtaining characteristic points of pulley groove straight line, wire rope straight line and beam area;

[0095] Figure 6 Another schematic diagram of a process for obtaining characteristic points of pulley groove straight line, wire rope straight line and beam area;

[0096] Figure 7 Schematic diagram of the process for geometric transformation estimation;

[0097] Figure 8 This is a flow chart of derailment detection. DETAILED DESCRIPTION

[0098] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0099] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0100] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0101] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use, it is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0102] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0103] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0104] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0105] See also Figure 1 The present invention discloses a wire rope derailment warning method for a container hoisting pulley block, comprising:

[0106] S101, collecting real-time video data, capturing static images from the video stream, and preprocessing the static images;

[0107] Initialization frame for static images Divide and obtain a static image of the pulley area , static image of beam area , static image of wire rope area , and then the pulley area static image , static image of beam area , static image of wire rope area Perform grayscale conversion to obtain grayscale images of the pulley area , grayscale image of the beam area and the grayscale image of the wire rope area , and the grayscale image of the pulley area , grayscale image of the beam area and the grayscale image of the wire rope area Gaussian blur processing is performed separately;

[0108] Among them, the grayscale image of the pulley area for:

[0109] (1)

[0110] in, Indicates converting a color image into a grayscale image;

[0111] Grayscale image of the pulley area Perform Gaussian blur processing, specifically:

[0112] (2)

[0113] in, Indicates Gaussian blur processing;

[0114] Grayscale image of the beam area for:

[0115] (3)

[0116] in, Indicates converting a color image into a grayscale image;

[0117] Grayscale image of the beam area Perform Gaussian blur processing, specifically:

[0118] (4)

[0119] in, Indicates Gaussian blur processing;

[0120] Grayscale image of wire rope area for:

[0121] (5)

[0122] in, Indicates converting a color image into a grayscale image;

[0123] Grayscale image of the wire rope area Perform Gaussian blur processing, specifically:

[0124] (6)

[0125] in, is the standard deviation, Indicates Gaussian blur processing.

[0126] S102, based on the preprocessed static image, obtaining a pulley region mask, a beam region mask, and a wire rope region mask, and then obtaining feature points of the respective regions;

[0127] Manually specify the areas where the pulley blocks, beams and wire ropes are located to generate pulley area masks , beam area mask and wire rope area mask ;

[0128] The obtaining of the feature points of the respective regions is specifically as follows:

[0129] Masking in the pulley area based on the ORB algorithm Internal detection feature points:

[0130] (7)

[0131] in, Indicates the pulley area mask The number of key feature points detected within The value range is between 400 and 800;

[0132] Masking in the beam area based on the ORB algorithm Internal detection feature points:

[0133] (8)

[0134] in, Represents the beam area mask The number of key feature points detected within The value range is between 400 and 800;

[0135] Masking in the wire rope area In the example, the image edges are extracted based on the Canny edge detection algorithm:

[0136] (9)

[0137] Among them, threshold1 and threshold2 represent two different thresholds in the Canny edge detection function. Pixels with gradient values ​​greater than threshold2 are marked as edges, pixels between threshold1 and threshold2 are considered potential edges, and pixels with gradient values ​​less than threshold1 are not edges.

[0138] Detect straight line segments based on Hough transform and extract straight line feature points:

[0139] (10)

[0140] in, Indicates the extraction of straight line feature points; Indicates the number of extracted straight line feature points, The value range is between 10 and 50.

[0141] S103, performing linear fitting on the feature points of the pulley region mask and the wire rope region mask to obtain fitted pulley groove straight lines and wire rope straight lines respectively;

[0142] The pulley groove straight line is the key structural feature point at the bottom of the pulley groove selected from the feature points of the pulley area mask. , by fitting the characteristic points at the bottom of the pulley groove The straight line obtained is recorded as ;in, is the number of key structural feature points at the bottom of the pulley groove, The value range is between 400 and 800.

[0143] The wire rope straight line is based on the least square method to the characteristic points Perform linear fitting and get a straight line , specifically:

[0144] (11)

[0145] in, Indicates fitting the straight line feature points into a straight line.

[0146] S104, based on the current frame The feature points of the middle beam area and the pulley groove area are matched with the feature points of the beam area and the pulley groove area set at the time of initialization, and then the geometric transformation matrix is ​​obtained. and the geometric transformation matrix ;

[0147] Based on the FLANN matching algorithm, the current frame Feature points in the middle beam area The initialization beam area feature point set Perform matching and obtain matching pairs:

[0148] (12)

[0149] in, Indicates that feature points are matched to obtain matching pairs;

[0150] Based on the RANSAC algorithm, the geometric transformation matrix of the beam area is estimated by matching pairs. , specifically:

[0151] (13)

[0152] in, Indicates that the geometric transformation matrix is ​​estimated based on the matching pairs;

[0153] Based on the FLANN matching algorithm, the current frame Feature points in the middle pulley groove area The initialization feature point set of the pulley groove area To match:

[0154] (14)

[0155] in, Indicates that feature points are matched to obtain matching pairs;

[0156] Based on the RANSAC algorithm, the geometric transformation matrix of the pulley groove area is estimated using matching pairs. , specifically:

[0157] (15)

[0158] in, Indicates that the geometric transformation matrix is ​​estimated based on the matching pairs;

[0159] S105, based on geometric transformation matrix , geometric transformation matrix , the pulley groove straight line at the time of initialization, the wire rope area mask at the time of initialization and the wire rope straight line at the time of initialization, and obtain the updated wire rope fitting straight line and the pulley groove fitting straight line;

[0160] Set the pulley groove straight line at initialization Through the geometric transformation matrix Map to the current frame to get the pulley groove straight line of the current frame :

[0161] (16)

[0162] Based on geometric transformation matrix The wire rope area mask at initialization Map to the position of the current frame to get the wire rope area mask of the current frame , specifically:

[0163] (17)

[0164] In the current frame Wire rope area Re-check the wire rope lines , make sure the wire rope is in line with the pulley groove consistency.

[0165] S106, performing a linearity check and a parallelism check based on the updated wire rope fitting line and the pulley groove fitting line. If the linearity check or the parallelism check fails, it is determined that the wire rope is derailed or abnormal.

[0166] Based on the average distances between the updated wire rope fitting line and the pulley groove fitting line and the feature points in their respective regions, determine whether the average distances from the feature points in their respective regions to their respective fitting lines are less than the preset linearity threshold; if so, determine that the pulley groove feature points and the wire rope feature points satisfy the linearity check; again determine whether the angle between the pulley groove fitting line direction vector and the wire rope fitting line direction vector is greater than the preset angle threshold; if so, determine that the fixed pulley groove line is not parallel to the wire rope line; if so, determine that the pulley groove line is parallel to the wire rope line.

[0167] Determine whether the angle between the pulley groove fitting straight line direction vector and the wire rope fitting straight line direction vector is greater than the preset angle threshold, specifically:

[0168] The pulley groove straight line Direction vector for:

[0169] (18)

[0170] The wire rope is straight Direction vector for:

[0171] (19)

[0172] The direction vector With direction vector Angle for:

[0173] (20)

[0174] in, The straight line of the pulley groove The slope of The wire rope is straight The slope of

[0175] Set the angle threshold ;like , it is determined that the pulley groove straight line is not parallel to the wire rope straight line.

[0176] See also Figure 2 The present invention discloses a wire rope derailment warning system for a container hoisting pulley block, comprising:

[0177] A preprocessing module, wherein the preprocessing module collects real-time video data, captures static images from the video stream, and preprocesses the static images;

[0178] A first acquisition module, which acquires a pulley region mask, a beam region mask, and a wire rope region mask based on the preprocessed static image, and further acquires feature points of the respective regions;

[0179] A linear fitting module, wherein the linear fitting module performs linear fitting on the characteristic points of the pulley region mask and the wire rope region mask to obtain fitted pulley groove straight lines and wire rope straight lines respectively;

[0180] A matching module, wherein the matching module is based on the current frame The feature points of the middle beam area and the pulley groove area are matched with the feature points of the beam area and the pulley groove area set at the time of initialization, and then the geometric transformation matrix is ​​obtained. and the geometric transformation matrix ;

[0181] The second acquisition module is based on a geometric transformation matrix , geometric transformation matrix , the pulley groove straight line at the time of initialization, the wire rope area mask at the time of initialization and the wire rope straight line at the time of initialization, and obtain the updated wire rope fitting straight line and the pulley groove fitting straight line;

[0182] A verification module performs a linearity check and a parallelism check based on the updated wire rope fitting straight line and the pulley groove fitting straight line. If the linearity check or the parallelism check fails, it is determined that the wire rope is derailed or abnormal.

[0183] Example:

[0184] See also Figure 3 , the present invention discloses a wire rope derailment warning device for container hanging pulley block, comprising: a camera module, a main controller, an alarm and display module, a monitoring system and a power supply module;

[0185] The camera module is responsible for collecting image information of the pulley block and wire rope of the container lifting equipment in real time;

[0186] The main controller receives the image data collected by the camera module, pre-processes the image data, detects the pre-processed image feature points, performs geometric transformation on the detected image feature points, determines the running state of the wire rope, and issues an alarm signal when an abnormality is detected. The main controller feeds back the system state and detection results to the monitoring system for real-time viewing and recording by the operator.

[0187] When the alarm and display module detects that the wire rope is derailed or abnormal, it will issue an alarm to the operator through visual, sound and other means, and present relevant detection information on the display screen.

[0188] The power supply module supplies power to the camera module, the main controller, the alarm and display module and the monitoring system.

[0189] The working process of a wire rope derailment warning device for container lifting pulley block is as follows:

[0190] See also Figure 4 , the real-time video stream captured by the camera module ,in Indicates The frame image is transmitted to the main controller for image processing through a high-speed data transmission interface (such as USB 3.0, Ethernet). The main controller processes each frame image. Perform preprocessing and feature point detection to extract feature point sets , and perform geometric transformation estimation to generate processed data According to the detection algorithm Analyze and determine the wire rope status. If the main controller detects derailment or abnormality, it will generate an alarm signal The main controller feeds back the system status and detection results to the monitoring system. , so that operators can view and record in real time.

[0191] After receiving the video stream captured by the camera module, the main controller takes the following approach:

[0192] See also Figure 5 and Figure 6 , collect real-time video data, capture static images from the video stream, and pre-process the static images;

[0193] Initialization frame for static images Divide and obtain a static image of the pulley area , static image of beam area , static image of wire rope area , and then the pulley area static image , static image of beam area , static image of wire rope area Perform grayscale conversion to obtain grayscale images of the pulley area , grayscale image of the beam area and the grayscale image of the wire rope area , and the grayscale image of the pulley area , grayscale image of the beam area and the grayscale image of the wire rope area Gaussian blur processing is performed separately;

[0194] Among them, the grayscale image of the pulley area for:

[0195] (1)

[0196] in, Indicates converting a color image into a grayscale image;

[0197] Grayscale image of the pulley area Perform Gaussian blur processing, specifically:

[0198] (2)

[0199] in, Indicates Gaussian blur processing;

[0200] Grayscale image of the beam area for:

[0201] (3)

[0202] in, Indicates converting a color image into a grayscale image;

[0203] Grayscale image of the beam area Perform Gaussian blur processing, specifically:

[0204] (4)

[0205] in, Indicates Gaussian blur processing;

[0206] Grayscale image of wire rope area for:

[0207] (5)

[0208] in, Indicates converting a color image into a grayscale image;

[0209] Grayscale image of the wire rope area Perform Gaussian blur processing, specifically:

[0210] (6)

[0211] in, is the standard deviation, Indicates Gaussian blur processing.

[0212] Based on the preprocessed static image, the pulley area mask, the beam area mask and the wire rope area mask are obtained, and then the feature points of the respective areas are obtained;

[0213] Manually specify the areas where the pulley blocks, beams and wire ropes are located to generate pulley area masks , beam area mask and wire rope area mask ;

[0214] The obtaining of the feature points of the respective regions is specifically as follows:

[0215] Masking in the pulley area based on the ORB algorithm Internal detection feature points:

[0216] (7)

[0217] in, Indicates the pulley area mask The number of key feature points detected within The value range is between 400 and 800. The specific value should be adjusted according to the computing resources of the application, the requirements for real-time performance and accuracy.

[0218] Masking in the beam area based on the ORB algorithm Internal detection feature points:

[0219] (8)

[0220] in, Represents the beam area mask The number of key feature points detected within The value range is between 400 and 800.

[0221] Masking in the wire rope area In the example, the image edges are extracted based on the Canny edge detection algorithm:

[0222] (9)

[0223] Among them, threshold1 and threshold2 represent two different thresholds in the Canny edge detection function, which are used to determine which pixels are strong edges and which are weak edges. Pixels with gradient values ​​greater than threshold2 are marked as edges, pixels between threshold1 and threshold2 are considered potential edges, and pixels with gradient values ​​less than threshold1 are not edges.

[0224] Detect straight line segments based on Hough transform and extract straight line feature points:

[0225] (10)

[0226] in, Indicates the extraction of straight line feature points; Indicates the number of extracted straight line feature points, which depends on the complexity of the image, the number of straight lines, and the need for real-time processing. The value range is between 10 and 50.

[0227] Linear fitting is performed on the characteristic points of the pulley area mask and the wire rope area mask to obtain the fitted pulley groove straight line and wire rope straight line respectively;

[0228] The pulley groove straight line is the key structural feature point at the bottom of the pulley groove selected from the feature points of the pulley area mask. , by fitting the characteristic points at the bottom of the pulley groove The straight line obtained is recorded as .in, is the number of key structural feature points at the bottom of the pulley groove, The value range is between 400 and 800.

[0229] The wire rope straight line is based on the least square method to the characteristic points Perform linear fitting and get a straight line , specifically:

[0230] (11)

[0231] in, Indicates fitting the straight line feature points into a straight line.

[0232] See also Figure 7 , based on the current frame The feature points of the middle beam area and the pulley groove area are matched with the feature points of the beam area and the pulley groove area set at the time of initialization, and then the geometric transformation matrix is ​​obtained. and the geometric transformation matrix ;

[0233] Based on the FLANN matching algorithm, the current frame Feature points in the middle beam area The initialization beam area feature point set Perform matching and obtain matching pairs:

[0234] (12)

[0235] in, Indicates that feature points are matched to obtain matching pairs;

[0236] Based on the RANSAC algorithm, the geometric transformation matrix of the beam area is estimated by matching pairs. , specifically:

[0237] (13)

[0238] in, Indicates that the geometric transformation matrix is ​​estimated based on the matching pairs;

[0239] Based on the FLANN matching algorithm, the current frame Feature points in the middle pulley groove area The initialization feature point set of the pulley groove area To match:

[0240] (14)

[0241] in, Indicates that feature points are matched to obtain matching pairs;

[0242] Based on the RANSAC algorithm, the geometric transformation matrix of the pulley groove area is estimated using matching pairs. , specifically:

[0243] (15)

[0244] in, Indicates that the geometric transformation matrix is ​​estimated based on the matching pairs;

[0245] Based on geometric transformation matrix , geometric transformation matrix , the pulley groove straight line at the time of initialization, the wire rope area mask at the time of initialization and the wire rope straight line at the time of initialization, and obtain the updated wire rope fitting straight line and the pulley groove fitting straight line;

[0246] Set the pulley groove straight line during initialization Through the geometric transformation matrix Map to the current frame to get the pulley groove straight line of the current frame :

[0247] (16)

[0248] Based on geometric transformation matrix The wire rope area mask at initialization Map to the position of the current frame to get the wire rope area mask of the current frame , specifically:

[0249] (17)

[0250] In the current frame Wire rope area Re-check the lines of the wire rope , make sure the wire rope is in line with the pulley groove consistency.

[0251] See also Figure 8 , based on the updated wire rope fitting straight line and pulley groove fitting straight line, linearity check and parallelism check are performed. If the linearity check or parallelism check fails, it is determined that the wire rope is derailed or abnormal.

[0252] Based on the average distances between the updated wire rope fitting line and the pulley groove fitting line and the feature points in their respective regions, determine whether the average distances from the feature points in their respective regions to their respective fitting lines are less than the preset linearity threshold; if so, determine that the pulley groove feature points and the wire rope feature points satisfy the linearity check; again determine whether the angle between the pulley groove fitting line direction vector and the wire rope fitting line direction vector is greater than the preset angle threshold; if so, determine that the fixed pulley groove line is not parallel to the wire rope line; if so, determine that the pulley groove line is parallel to the wire rope line.

[0253] Based on the average distances between the updated wire rope fitting line and the pulley groove fitting line and the feature points in their respective regions, it is determined whether the average distances between the feature points in their respective regions and their respective fitting lines are less than the preset linearity threshold, specifically:

[0254] Extract feature point set from the pulley groove area of ​​the current frame ; Based on the least squares method, the characteristic points of the pulley groove area Perform linear fitting to obtain the pulley groove straight line of the current frame .

[0255] Based on the least squares method, the characteristic points of the pulley groove area are Perform linear fitting, specifically:

[0256] (18)

[0257] in, is the slope, is the intercept.

[0258] Calculate the average distance from the feature point to the fitting line :

[0259] (19)

[0260] Setting the linearity threshold ;like , then the pulley groove characteristic points are considered to satisfy the linearity check.

[0261] Extract feature point set from the wire rope area of ​​the current frame , based on the least squares method, the characteristic points of the wire rope area Perform linear fitting to obtain the wire rope straight line of the current frame .

[0262] Based on the least squares method, the characteristic points of the wire rope area are Perform linear fitting, specifically:

[0263] (20)

[0264] in, is the slope, is the intercept.

[0265] Calculate the average distance from the feature point to the fitting line :

[0266] (twenty one)

[0267] Setting the linearity threshold ;like , it is considered that the characteristic points of the wire rope meet the linearity check.

[0268] Determine whether the angle between the pulley groove fitting straight line direction vector and the wire rope fitting straight line direction vector is greater than the preset angle threshold, specifically:

[0269] The pulley groove straight line Direction vector for:

[0270] (twenty two)

[0271] The wire rope is straight Direction vector for:

[0272] (twenty three)

[0273] The direction vector With direction vector Angle for:

[0274] (twenty four)

[0275] in, is the vector dot product,

[0276] (25)

[0277] The magnitude of a vector is:

[0278] (26)

[0279] in, The straight line of the pulley groove The slope of The wire rope is straight The slope of

[0280] Set the angle threshold ;like , it is determined that the pulley groove straight line is not parallel to the wire rope straight line.

[0281] Taking the pulley area as an example, the two-dimensional Gaussian function of the pulley area is defined as:

[0282]

[0283] where (x, y) are coordinates relative to the center of the nucleus; is the standard deviation, which controls the degree of blur; σ=1.0 is a commonly used value that can provide a moderate blur effect;

[0284] The kernel size is usually 6σ+1 (upward odd numbers), because the values ​​outside the range of 3σ are close to 0; when σ=1.0, the kernel size is usually 3×3 or 5×5;

[0285] Taking a 3×3 kernel as an example, the 3×3 kernel matrix is ​​expressed as:

[0286]

[0287] Specific calculation example

[0288] Suppose there is a 3×3 local region of a grayscale image:

[0289]

[0290] The center point The fuzzy result is:

[0291]

[0292] Specific implementation examples:

[0293] A large port container terminal is equipped with multiple container lifting equipment. Take one of the equipment as an example. The pulley block is installed on the crossbeam of the equipment, which is responsible for controlling the lifting and lowering of the wire rope to ensure the stable lifting of the container. In order to monitor the running status of the wire rope, the system fixed a high-resolution industrial camera on the crossbeam of the equipment to collect images of the pulley block and wire rope in real time.

[0294] Environmental conditions: sufficient natural light during the day, supplemented by the device's own LED lighting at night. The pulley block moves continuously under normal working conditions, and the wire rope runs smoothly in the pulley groove. The camera is installed in a fixed position to avoid vibration and deviation affecting the image quality.

[0295] The camera module is an industrial-grade high-definition camera with a resolution of: Pixels and above, frame rate: 30FPS and above; Installation position: fixedly installed in the center of the equipment beam, the viewing angle covers the pulley block and wire rope area.

[0296] The main controller is based on Ubuntu's real-time operating system and Ethernet interface, supporting high-speed communication with the monitoring system and alarm module.

[0297] The alarm and display module includes an LED indicator, a buzzer, and a 15-inch high-definition touch screen; when an abnormality is detected, the LED indicator flashes, the buzzer sounds an alarm, and the display shows the detection information and images in real time.

[0298] The power module is connected to each module via a standard cable to ensure stable power supply.

[0299] In order to ensure the stable operation of the system in the actual port environment, the following protective measures are taken:

[0300] Dustproof and waterproof: The camera and main controller housing adopts dustproof and waterproof design (IP67 level) to prevent dust and water vapor from entering the internal components.

[0301] Shockproof design: All modules are installed on shockproof brackets to reduce the impact of equipment vibration on detection accuracy.

[0302] Power voltage stabilization: The power module is equipped with a voltage-stabilized power supply to ensure that each module operates under a stable voltage and prevent system failures caused by voltage fluctuations.

[0303] Algorithm parameter settings

[0304] ORB feature point detection parameters: the number of feature points is 500; the FAST threshold is 20;

[0305] Descriptor matching: Hamming distance is used with a distance threshold of 30.

[0306] The Hough transform parameters include: minimum line length of 30 pixels, maximum gap of 10 pixels, and accumulator threshold of 50.

[0307] The RANSAC algorithm parameters include: the number of iterations is 1000 and the tolerance is 5 pixels.

[0308] Linear Fit Parameters: Linearity Threshold 1.5 pixels; linearity threshold is 1.5 pixels; angle threshold 2 degrees;

[0309] The number of feature points and FAST threshold include: 500 feature points can cover the main features of the pulley area and the beam area, and the FAST threshold of 20 can effectively detect obvious corner points.

[0310] The Hough transform and LSD parameters include: minimum line length of 30 pixels and maximum gap of 10 pixels to ensure that continuous and obvious wire rope segments are detected.

[0311] The RANSAC parameters are: 1000 iterations and 5 pixel tolerance, which improve the robustness of the algorithm while ensuring the accuracy of transformation matrix estimation.

[0312] Linear fitting thresholds include: 1.5 pixel linearity threshold and The angle threshold can effectively distinguish normal operation from derailment or abnormal status in practical applications.

[0313] The camera starts to collect video streams, and the image processing unit and main controller are initialized.

[0314] The operator manually specifies the pulley area in the initialization interface , beam area And wire rope area Mask and mark the key feature points at the bottom of the pulley groove .

[0315] The system initializes the frame according to the specified mask. Perform feature point detection and extraction , , , and is obtained by least squares fitting and .

[0316] The main controller continuously receives and processes feature point data from the image processing unit and estimates the geometric transformation matrix and , and update and .

[0317] The system performs a straightness check and a parallelism check. If an angle is detected or fitting error , , it is judged as derailment or abnormality.

[0318] Test results: In actual application, the system successfully detected several minor derailments of the wire rope. Whenever the wire rope slightly jumped out of the pulley groove, the system immediately triggered an alarm, the LED indicator flashed, the buzzer sounded an alarm, and the display screen showed the abnormal image and related parameters in real time.

[0319] Example data:

[0320] Normal state detection: fitting error is 1.2 pixels, is 1.3 pixels; angle The value is 1.8 degrees, which is lower than the threshold. 2 degrees; Result: Normal operation, no alarm.

[0321] Abnormal state detection: fitting error is 2.0 pixels, 2.1 pixels; angle 3.5 degrees, exceeding the threshold It is 2 degrees; Result: The alarm is triggered and the operator takes immediate measures to prevent the accident.

[0322] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A wire rope derailment warning method for container lifting pulley block, characterized in that: include: Collect real-time video data, capture static images from the video stream, and pre-process the static images; Based on the preprocessed static image, the pulley groove area mask, the beam area mask and the wire rope area mask at the time of initialization are obtained, and then the feature points of the respective area masks at the time of initialization are obtained; Performing linear fitting on the feature points of the pulley groove area mask and the feature points of the wire rope area mask at the time of initialization, respectively obtaining the fitted pulley groove straight line and wire rope straight line at the time of initialization; Based on the feature points of the beam area mask and the feature points of the pulley groove area in the current frame t, the feature points of the beam area mask and the feature points of the pulley groove area mask at the time of initialization are matched respectively, and then the geometric transformation matrix T of the beam area is obtained. b and the geometric transformation matrix T of the pulley groove area p ; Based on the geometric transformation matrix T of the beam area b , the geometric transformation matrix T of the pulley groove area p , the pulley groove straight line at the time of initialization, the wire rope area mask at the time of initialization and the wire rope straight line at the time of initialization, and obtain the updated wire rope straight line and the updated pulley groove straight line; specifically: Set the pulley groove straight line at initialization By the geometric transformation matrix T of the beam area b Map to the current frame to get the updated pulley groove straight line Geometric transformation matrix T based on pulley groove area p The wire rope area mask at initialization Map to the position of the current frame to get the updated wire rope area mask Specifically: The updated wire rope area mask Recheck the updated wire rope straightness Ensure that the updated wire rope straight line is aligned with the updated pulley groove straight line consistency; A linearity check and a parallelism check are performed based on the updated wire rope straight line and the updated pulley groove straight line. If the linearity check or the parallelism check fails, it is determined that the wire rope is derailed or abnormal.

2. The wire rope derailment warning method for container lifting pulley block according to claim 1 is characterized in that: The static image is preprocessed as follows: the initialization frame F0 of the static image is divided to obtain a static image F1 of the pulley groove area, a static image F2 of the beam area, and a static image F3 of the wire rope area, and then the static image F1 of the pulley groove area, the static image F2 of the beam area, and the static image F3 of the wire rope area are gray-scale converted to obtain gray-scale images G of the pulley groove area. pulley , grayscale image G of the beam area ref and the grayscale image G of the wire rope area wire , and the grayscale image G of the pulley groove area pulley , grayscale image G of the beam area ref and the grayscale image G of the wire rope area wire Gaussian blur processing is performed separately; Among them, the grayscale image G of the pulley groove area pulley for: G pulley =Grayscale(F1) (1) Among them, Grayscale() means converting a color image into a grayscale image; The grayscale image G of the pulley groove area pulley Perform Gaussian blur processing, specifically: G′ pulley =GaussianBlur(G pulley ,σ=1.0) (2) Among them, GaussianBlur() represents Gaussian blur processing; Grayscale image G of the beam area ref for: G ref =Grayscale(F2) (3) Among them, Grayscale() means converting a color image into a grayscale image; The grayscale image G of the beam area ref Perform Gaussian blur processing, specifically: G′ ref =GaussianBlur(G ref ,σ=1.0) (4) Among them, GaussianBlur() represents Gaussian blur processing; Grayscale image G of the wire rope area wire for: G wire =Grayscale(F3) (5) Among them, Grayscale() means converting a color image into a grayscale image; For the grayscale image G of the wire rope area wire Perform Gaussian blur processing, specifically: G′ wire =GaussianBlur(G wire ,σ=1.0) (6) Among them, σ is the standard deviation, and GaussianBlur() represents Gaussian blur processing.

3. The wire rope derailment warning method for container hoist pulley block according to claim 2 is characterized in that: The method is to obtain the pulley groove area mask, beam area mask and wire rope area mask at the time of initialization based on the preprocessed static image, specifically: manually specify the area where the pulley block, beam and wire rope are located, and generate the pulley groove area mask at the time of initialization. Beam Area Mask and wire rope area mask The feature points of the respective regional masks obtained during initialization are specifically: Masking in the pulley groove area based on ORB algorithm Internal detection feature points: Where n represents the pulley groove area mask The number of key feature points detected in the image, n ranges from 400 to 800; Masking in the beam area based on the ORB algorithm Internal detection feature points: Where l represents the beam area mask The number of key feature points detected in the image, l ranges from 400 to 800; Masking in the wire rope area In the example, the image edges are extracted based on the Canny edge detection algorithm: E wire =Canny(G′ wire ,threshold1=50,threshold2=150) (9) Among them, threshold1 and threshold2 represent two different thresholds in the Canny edge detection function. Pixels with gradient values ​​greater than threshold2 are marked as edges, pixels between threshold1 and threshold2 are considered potential edges, and pixels with gradient values ​​less than threshold1 are not edges. Detect straight line segments based on Hough transform and extract straight line feature points: Among them, HoughTransform() represents the extraction of straight line feature points; k represents the number of extracted straight line feature points, and the value range of k is between 10 and 50.

4. The wire rope derailment warning method for container lifting pulley block according to claim 3 is characterized in that: The linear fitting is performed on the characteristic points of the pulley groove area mask and the characteristic points of the wire rope area mask at the time of initialization to obtain the fitted pulley groove straight line and wire rope straight line at the time of initialization, specifically: The pulley groove straight line is a key structural feature point {s1, s2, ..., s m }, by fitting the characteristic points {s1,s2,…,s m }, the pulley groove straight line obtained during the initialization of the fitting is recorded as Among them, m is the number of key structural characteristic points at the bottom of the pulley groove, and the value range of m is between 400 and 800; The wire rope straight line at the time of initialization of the fitting is based on the least square method for the characteristic point P wire Perform linear fitting to obtain the wire rope straight line at the time of fitting initialization Specifically: Among them, FitLine() means fitting the straight line feature points into a straight line.

5. The wire rope derailment warning method for container lifting pulley block according to claim 4 is characterized in that: The feature points of the beam area mask and the feature points of the pulley groove area in the current frame t are matched with the feature points of the beam area mask and the feature points of the pulley groove area mask at the time of initialization, thereby obtaining the geometric transformation matrix T of the beam area b and the geometric transformation matrix T of the pulley groove area p , specifically: Based on the FLANN matching algorithm, the feature points of the beam area mask in the current frame t are The feature point P of the beam area mask at the time of initialization ref Perform matching and obtain matching pairs: Among them, FLANN() means matching feature points to obtain matching pairs; Based on the RANSAC algorithm, the geometric transformation matrix T of the beam area is estimated by matching pairs. b , specifically: T b =EstimateTransform(Matches1,RANSAC) (13) Among them, EstimateTransform() means estimating the geometric transformation matrix based on the matching pair; Based on the FLANN matching algorithm, the feature points of the pulley groove area mask in the current frame t are The feature point P of the pulley groove area mask at the time of initialization pulley To match: Among them, FLANN() means matching feature points to obtain matching pairs; Based on the RANSAC algorithm, the geometric transformation matrix T of the pulley groove area is estimated using matching pairs. p , specifically: T p =EstimateTransform(Matches2,RANSAC) (15) Among them, EstimateTransform() means estimating the geometric transformation matrix based on the matching pairs.

6. The wire rope derailment warning method for container lifting pulley block according to claim 5 is characterized in that: The linearity check and parallelism check based on the updated wire rope straight line and the updated pulley groove straight line are specifically as follows: based on the average distances between the updated wire rope straight line and the updated pulley groove straight line and the feature points between the respective regions, it is judged whether the average distances from the feature points between the respective regions to the respective fitting straight lines are less than a preset linearity threshold; if less than, it is judged that the feature points of the pulley groove region mask and the feature points of the wire rope region mask meet the linearity check; it is judged again whether the angle between the updated pulley groove straight line direction vector and the updated wire rope straight line direction vector is greater than a preset angle threshold; if greater than, it is judged that the updated pulley groove straight line is not parallel to the updated wire rope straight line; If it is less than, it is judged that the updated pulley groove straight line is parallel to the updated wire rope straight line.

7. The wire rope derailment warning method for container hoist pulley block according to claim 6, characterized in that: The step of judging whether the angle between the updated pulley groove linear direction vector and the updated wire rope linear direction vector is greater than a preset angle threshold is specifically as follows: The updated pulley groove straight line Direction vector for: The updated wire rope straight line Direction vector for: The direction vector With direction vector The angle θ is: Among them, k p The updated pulley groove straight line The slope of k w The updated wire rope straight line The slope of Set the angle threshold θ max ; If |θ|>θ max , it is determined that the updated pulley groove straight line is not parallel to the updated wire rope straight line.

8. A wire rope derailment warning system for container hoisting pulley block, characterized in that: include: A preprocessing module, wherein the preprocessing module collects real-time video data, captures static images from the video stream, and preprocesses the static images; A first acquisition module, which acquires the pulley groove area mask, the beam area mask and the wire rope area mask at the time of initialization based on the preprocessed static image, and further acquires the feature points of the respective area masks at the time of initialization; A linear fitting module, wherein the linear fitting module performs linear fitting on the characteristic points of the pulley groove area mask and the characteristic points of the wire rope area mask at the time of initialization, and obtains the fitted pulley groove straight line and wire rope straight line at the time of initialization respectively; A matching module, wherein the matching module matches the feature points of the beam area mask and the feature points of the pulley groove area in the current frame t with the feature points of the beam area mask and the feature points of the pulley groove area mask at the time of initialization, respectively, and then obtains the geometric transformation matrix T of the beam area b and the geometric transformation matrix T of the pulley groove area p ; The second acquisition module is based on the geometric transformation matrix T of the beam area. b , the geometric transformation matrix T of the pulley groove area p , the pulley groove straight line at the time of initialization, the wire rope region mask at the time of initialization and the wire rope straight line at the time of initialization, and obtain the updated wire rope straight line and the updated pulley groove straight line; Set the pulley groove straight line at initialization By the geometric transformation matrix T of the beam area b Map to the current frame to get the updated pulley groove straight line Geometric transformation matrix T based on pulley groove area p The wire rope area mask at initialization Map to the position of the current frame to get the updated wire rope area mask Specifically: The updated wire rope area mask Recheck the updated wire rope straightness Ensure that the updated wire rope straight line is aligned with the updated pulley groove straight line consistency; A verification module performs a linearity check and a parallelism check based on the updated wire rope straight line and the updated pulley groove straight line. If the linearity check or the parallelism check fails, it is determined that the wire rope is derailed or abnormal.

Images