Truss identification method, device and system suitable for crane

Abstract

The invention discloses a truss identification method, device and system suitable for a crane, and the method comprises the steps: obtaining a target image, the target image comprises a plurality of trusses, and the installation sequence of the trusses is a target installation sequence; obtaining a to-be-recognized image in real time, wherein the to-be-recognized image comprises a plurality of trusses; based on the target image and the to-be-recognized image, performing first judgment on correctness of the truss mounting sequence; calculating the distance between each truss and the crane body based on time difference data generated by a wireless positioning sensor; based on the distance between each truss and the crane body, carrying out second judgment on the correctness of the truss mounting sequence; and based on the results of the first judgment and the second judgment, generating a final truss installation sequence identification result. According to the invention, positioning of the truss and detection of the installation sequence can be realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of engineering machinery, and in particular relates to a truss identification method, device and system suitable for a crane. Background Art

[0002] The truss structure of a crane is an important part of the crane, which provides the necessary support and strength for the crane. The truss is a triangular frame structure composed of straight rods. The combination of these triangular frame structures gives the truss a high stability and load-bearing capacity. The truss can be flat or spatial, depending on the design and use requirements of the crane.

[0003] During the installation of a crane, the installation of the truss is a key step. For example, for a gantry crane, its truss is usually composed of a main beam, an end beam, and a supporting structure. The main beam is the main part that carries the load, while the end beam connects the main beam and provides additional support. During installation, it is necessary to ensure that the various parts of the truss are correctly aligned and firmly connected together using high-strength bolts or welding.

[0004] For crawler cranes, the installation of lattice booms is also a complex process. This usually involves precisely docking and fixing the various sections of the lattice boom. During the installation process, special lifting equipment and equipment may be required to ensure the stability and safety of the lattice boom. Different trusses need to correspond to corresponding installation positions. If the installation sequence is wrong, it will cause a high safety risk.

[0005] In the Chinese invention patent application with publication number CN115636358A, a telescopic truss boom inter-section self-locking device and crane are disclosed, which abandons the inter-section locking method of the boom locked by the pin shaft, avoids the operation of inserting and pulling the pin, and when the inner boom section moves to the required position for the operation, it can automatically lock without additional auxiliary power and detection device, thereby transmitting axial force and realizing the load-bearing function of the truss boom. In this scheme, only the mechanical connection structure of the truss is described, and its position sequence detection is not mentioned.

[0006] In the Chinese invention patent application with publication number CN116239029A, an assembled crane for installing large pipes in buildings is disclosed, which includes two groups of combined columns on the left and right, walking wheels, a bottom frame and a lifting device. A fixed frame and a sleeve frame are respectively installed on the top of the two groups of combined columns, a movable frame is installed on one side of the fixed frame, and a power actuator is installed between the movable frame and the sleeve frame; a sleeve rope is installed between the bottom ends of the two recovery ropes, a disconnection mechanism is configured on the sleeve rope, and a tapping detection plate is installed at the connection between the sleeve rope and the recovery rope, and a tightening mechanism is installed between the top of the recovery rope and the truss; a positioning wheel is installed on the surface of the end of the movable frame away from the fixed frame, a secondary sling is movably installed on one side of the lifting device, and a limited clamping mechanism is installed on the clamp. The scheme does not mention the sequential recognition of the truss position. Summary of the invention

[0007] In view of the above problems, the present invention proposes a truss identification method, device and system suitable for a crane, which can realize the position positioning of the truss and the detection of the installation sequence.

[0008] In order to achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions:

[0009] In a first aspect, the present invention provides a truss identification method applicable to a crane, comprising:

[0010] Acquire a target image, wherein the target image includes a plurality of trusses, and the installation order of each truss is a target installation order;

[0011] Acquire an image to be identified in real time, wherein the image to be identified includes a plurality of trusses;

[0012] Based on the target image and the image to be identified, a first judgment is made on whether the truss installation sequence is correct or not;

[0013] Based on the time difference data generated by the wireless positioning sensor, the distance between each truss and the crane body is calculated;

[0014] A second judgment is made on the correctness of the truss installation sequence based on the distance between each truss and the crane body;

[0015] Based on the results of the first judgment and the second judgment, a final truss installation sequence recognition result is generated.

[0016] In combination with the first aspect, optionally, the first judgment of whether the truss installation sequence is correct based on the target image and the image to be identified includes:

[0017] Extracting a plurality of target sub-images from the target image, wherein each target sub-image contains only one truss;

[0018] Perform feature operator extraction on each target sub-image respectively to obtain the first corner point corresponding to each target sub-image;

[0019] Extracting a plurality of sub-images to be identified from the image to be identified, wherein one sub-image to be identified contains only one truss;

[0020] Perform feature operator extraction on each sub-image to be identified, and obtain a second corner point corresponding to each sub-image to be identified;

[0021] Based on the first corner point and the second corner point, the correctness of the truss installation sequence is judged.

[0022] In combination with the first aspect, optionally, the method for acquiring the corner point includes:

[0023] Use the Sobel operator to calculate the gradient of the image in the x and y directions;

[0024] Calculate the square and product of the gradients and construct the autocorrelation matrix;

[0025] The Harris response value is calculated by the eigenvalues ​​of the autocorrelation matrix;

[0026] Points with Harris response values ​​higher than the set threshold are regarded as candidate corner points;

[0027] Perform non-maximum suppression on candidate corner points to generate the final corner points.

[0028] In combination with the first aspect, optionally, the target sub-image and the sub-image to be identified are both obtained by the following method:

[0029] The multi-scale features of the trusses in the image are extracted through multiple layers of network structures of different sizes, and the multi-scale features are fused;

[0030] Make bounding box predictions for trusses;

[0031] Perform non-maximum suppression on all bounding boxes to ensure that each truss is accurately marked by only one bounding box;

[0032] The marked bounding box is given a category and a confidence score, and finally several sub-images are obtained, each of which contains only one truss.

[0033] In combination with the first aspect, optionally, judging whether the truss installation sequence is correct based on the first corner point and the second corner point includes:

[0034] Compare the SIFT descriptor of the second corner point of a sub-image to be identified with the SIFT descriptor of the first corner point of the target sub-image corresponding to the sub-image to be identified;

[0035] If the similarity between the two SIFT descriptors is greater than the set threshold, it means that the installation order of the trusses in the sub-image to be identified is wrong;

[0036] Among them, the formula used for comparison is:

[0037]

[0038]

[0039]

[0040] in, is the corner point in the target sub-image SIFT descriptor, is the corner point in the sub-image to be identified SIFT descriptor, Indicates the number of dimensions of the SIFT descriptor, is the difference between two eigenvalues.

[0041] In combination with the first aspect, optionally, a positioning transmitter is respectively provided on each truss, and a plurality of positioning receivers are provided at preset position points of the crane body, and the truss identification method applicable to the crane further includes:

[0042] Based on the time difference data generated by multiple positioning receivers, the distance between each truss and the crane body is calculated;

[0043] Based on the actual distance between each truss and the crane body and the preset target distance between each truss and the crane body, the correctness of the truss installation sequence is judged.

[0044] In combination with the first aspect, optionally, if the difference between the actual distance between a certain truss and the crane body and the preset target distance between the truss and the crane body is greater than a set threshold, it is determined that the installation sequence of the truss is wrong.

[0045] In a second aspect, the present invention provides a truss identification device suitable for a crane, comprising an image collector, a wireless positioning sensor and a controller, wherein the wireless positioning sensor comprises a positioning transmitter and a positioning receiver; a positioning transmitter is respectively arranged on each truss, and a positioning receiver is respectively arranged at a plurality of preset position points of the crane body;

[0046] The image collector is used to collect target images and images to be identified;

[0047] The controller is configured to perform the following steps:

[0048] Acquire a target image, wherein the target image includes a plurality of trusses, and the installation order of each truss is a target installation order;

[0049] Acquire an image to be identified in real time, wherein the image to be identified includes a plurality of trusses;

[0050] Based on the target image and the image to be identified, a first judgment is made on whether the truss installation sequence is correct or not;

[0051] Based on the time difference data generated by the wireless positioning sensor, the distance between each truss and the crane body is calculated;

[0052] A second judgment is made on the correctness of the truss installation sequence based on the distance between each truss and the crane body;

[0053] Based on the results of the first judgment and the second judgment, a final truss installation sequence recognition result is generated.

[0054] In a third aspect, the present invention provides a truss identification device suitable for a crane, comprising:

[0055] A target image acquisition module is used to acquire a target image, wherein the target image includes a plurality of trusses, and the installation order of each truss is a target installation order;

[0056] An image acquisition module to be identified, used for acquiring an image to be identified in real time, wherein the image to be identified includes a plurality of trusses;

[0057] A sub-image extraction module to be identified, used to extract a plurality of sub-images to be identified from the image to be identified, wherein one sub-image to be identified contains only one truss;

[0058] A first judgment module, used for making a first judgment on whether the truss installation sequence is correct or not based on the target image and the image to be recognized;

[0059] A distance acquisition module is used to calculate the distance between each truss and the crane body based on the time difference data generated by the wireless positioning sensor;

[0060] A second judgment module, used for making a second judgment on whether the truss installation sequence is correct or not based on the distance between each truss and the crane body;

[0061] The identification module is used to generate a final truss installation sequence identification result based on the results of the first judgment and the second judgment.

[0062] In a fourth aspect, the present invention provides a truss identification system applicable to a crane, comprising a storage medium and a processor;

[0063] The storage medium is used to store instructions;

[0064] The processor is configured to operate according to the instructions to execute the method according to any one of the first aspects.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] The present invention adopts a wireless positioning sensor to obtain the installation position of the truss, and detects the installation order of the truss by extracting the corner points in the target image and the image to be identified, and then fuses and matches the two parts of data, thereby indirectly completing the installation order measurement of the truss position.

[0067] Multiple wireless positioning sensors installed on the crane body and trusses can provide different perspectives and data on the same phenomenon, and multi-sensor data can improve the accuracy of estimates. If one sensor fails or provides inaccurate data, the data from other sensors can supplement, thereby improving the reliability of the entire system. Multi-sensor systems are better able to cope with environmental changes and changes in sensor performance because it does not rely on a single data source. Different sensors may focus on different spatial areas or spectral ranges, and this data can expand the coverage of monitoring and perception. In some cases, data collected by multiple sensors can reduce the amount of computing required because it allows the system to process only the most important information, thereby speeding up the decision-making process. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor, among which:

[0069] Figure 1 The present invention is a flowchart of a truss identification method applicable to a crane according to an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0072] Example 1

[0073] The present invention provides a truss identification method applicable to a crane, such as Figure 1 As shown, the following steps are included:

[0074] (1) Acquire a target image, wherein the target image includes a plurality of trusses, and the installation order of each truss is a target installation order;

[0075] (2) acquiring an image to be identified in real time, wherein the image to be identified includes a plurality of trusses;

[0076] (3) performing a first judgment on whether the truss installation sequence is correct based on the target image and the image to be identified;

[0077] (4) Calculate the distance between each truss and the crane body based on the time difference data generated by the wireless positioning sensor;

[0078] (5) Based on the distance between each truss and the crane body, a second judgment is made on whether the truss installation sequence is correct;

[0079] (6) Based on the results of the first judgment and the second judgment, a final truss installation sequence recognition result is generated.

[0080] The truss recognition method suitable for cranes in the embodiment of the present invention integrates the positioning method based on wireless positioning sensors and the image feature extraction algorithm of machine vision, and simultaneously realizes the position positioning of the truss and the detection of the installation sequence with high recognition accuracy.

[0081] In a specific implementation of the embodiment of the present invention, Figure 1 As shown, the first judgment of whether the truss installation sequence is correct based on the target image and the image to be identified includes:

[0082] Extracting a plurality of target sub-images from the target image, wherein each target sub-image contains only one truss;

[0083] Perform feature operator extraction on each target sub-image respectively to obtain the first corner point corresponding to each target sub-image;

[0084] Extracting a plurality of sub-images to be identified from the image to be identified, wherein one sub-image to be identified contains only one truss;

[0085] Perform feature operator extraction on each sub-image to be identified, and obtain a second corner point corresponding to each sub-image to be identified;

[0086] Based on the first corner point and the second corner point, the correctness of the truss installation sequence is judged.

[0087] In a specific implementation of the embodiment of the present invention, the target sub-image and the sub-image to be identified are obtained by the following method:

[0088] The multi-scale features of the trusses in the image are extracted through multiple layers of network structures of different sizes;

[0089] Fuse multi-scale features;

[0090] Make bounding box predictions for trusses;

[0091] Perform non-maximum suppression on all bounding boxes to ensure that each truss is accurately marked by only one bounding box;

[0092] The marked bounding box is given a category and a confidence score, and finally several sub-images are obtained, each of which contains only one truss.

[0093] In a specific implementation of the embodiment of the present invention, a method for obtaining corner points includes:

[0094] Use the Sobel operator to calculate the gradient of the image in the x and y directions;

[0095] Calculate the square and product of the gradients and construct the autocorrelation matrix;

[0096] The Harris response value is calculated by the eigenvalues ​​of the autocorrelation matrix;

[0097] Points with Harris response values ​​higher than the set threshold are regarded as candidate corner points;

[0098] Perform non-maximum suppression on candidate corner points to generate the final corner points.

[0099] In a specific implementation of the embodiment of the present invention, the judging of the correctness of the truss installation sequence based on the first corner point and the second corner point includes:

[0100] SIFT feature descriptor calculation of the first corner point and the second corner point:

[0101] 1. Scale space extreme value detection

[0102] (1) Gaussian filtering

[0103] ;

[0104] in, represents the pixel coordinates, is the scale parameter that controls the width of the Gaussian kernel. Generates a Gaussian kernel function that is used to smooth the image, reduce noise, and retain the main features of the image.

[0105] (2) Gaussian Pyramid

[0106] ;

[0107] in, represents the original image, is the Gaussian kernel number, by the original image Perform Gaussian filtering to generate a scale of Gaussian Pyramid .

[0108] (3) Gaussian Difference Pyramid

[0109] ;

[0110] in, The scale is The Gaussian pyramid layers, The scale is The Gaussian pyramid layers, is the scale factor, usually 1.6, generating a scale of Gaussian Difference Pyramid, highlighting edges and corners in images.

[0111] (4) Extreme points

[0112] If in If a point is a local maximum or minimum in its 3x3x3 neighborhood, then the point is considered to be an extreme point, that is, a key point

[0113] 2. Key point positioning

[0114] (1) Sub-pixel positioning

[0115] Sub-pixel positioning via Taylor expansion:

[0116] ;

[0117] represents the gradient vector at the key point, Represents the Hessian matrix at the key point, sub-pixel position It provides more accurate key point locations and improves the positioning accuracy of key points for subsequent feature description and matching.

[0118] 3. Direction assignment of key points

[0119] (1) Gradient calculation

[0120] ;

[0121] Indicates the key point The gradient of the direction, Indicates the key point Directional gradient, by calculating the gradient, obtain the change direction and intensity of the pixels around the key point

[0122] ;

[0123] By using the inverse tangent function, the direction of the gradient is calculated , used to determine the main direction of the key point

[0124] (2) Gradient Histogram

[0125] The gradient direction is divided into 36 intervals (one interval every 10 degrees), a histogram is constructed, and the peak value is selected as the main direction of the key point to achieve rotation invariance.

[0126] 4. Feature Description

[0127] (1) Gradient calculation

[0128] ;

[0129] and Represents the gradient of the pixels around the key point. By calculating the gradient, the change direction and intensity of the pixels around the key point are obtained

[0130] ;

[0131] Calculate the direction of the gradient and amplitude , by calculating the direction and magnitude of the gradient, detailed information of the pixels around the key point is obtained.

[0132] (2) Gradient Histogram

[0133] The area around the key point is divided into 16 sub-regions, the gradient direction and amplitude are calculated for the pixels in each sub-region, and the gradient histogram in 8 directions of each sub-region is constructed. The 8-direction histograms of the 16 sub-regions are spliced ​​together to form a 128-dimensional SIFT descriptor.

[0134] The feature value of the second corner point of the sub-image to be identified is compared with the feature value of the first corner point of the target sub-image corresponding to the sub-image to be identified:

[0135] 1. Calculate corner features

[0136] By calculating the above features, we can get the corner points in the target sub-image (template image) SIFT descriptor:

[0137] ;

[0138] Corner points in the sub-image to be identified (real-time image) SIFT descriptor:

[0139] ;

[0140] 2. Feature Matching

[0141] ;

[0142] Calculate the Euclidean distance between two SIFT descriptors , which measures the similarity of the two SIFT descriptors. If the similarity of the two SIFT descriptors is less than the set threshold, it means that the corner points in the target sub-image and the angles in the sub-image to be identified match, indicating that the installation order of the trusses in the sub-image to be identified is correct. If the similarity of the two SIFT descriptors is greater than the set threshold, it means that the corner points in the target sub-image and the angles in the sub-image to be identified do not match, indicating that the installation order of the trusses in the sub-image to be identified is wrong.

[0143] In a specific implementation of the embodiment of the present invention, a positioning transmitter is respectively arranged on each truss, and a plurality of positioning receivers are arranged at preset positions of the crane body, such as Figure 1 As shown, the truss identification method applicable to the crane also includes:

[0144] Based on the time difference data generated by multiple positioning receivers, the distance between each truss and the crane body is calculated;

[0145] Based on the actual distance between each truss and the crane body and the preset target distance between each truss and the crane body, the correctness of the truss installation sequence is judged.

[0146] In a specific implementation of an embodiment of the present invention, if the difference between the actual distance between a certain truss and the crane body and the preset target distance between the truss and the crane body is greater than a set threshold, the truss installation sequence is determined to be wrong. If the difference between the actual distance between a certain truss and the crane body and the preset target distance between the truss and the crane body is less than or equal to the set threshold, the truss installation sequence is determined to be correct. Only when the truss installation sequence is determined to be correct based on both the image recognition result and the distance recognition result, the truss installation sequence is considered to be correct. If one of the recognition results is that the truss installation sequence is wrong, re-recognition is required.

[0147] Example 2

[0148] In an embodiment of the present invention, a truss identification device suitable for a crane is provided, comprising an image collector, a wireless positioning sensor and a controller, wherein the wireless positioning sensor comprises a positioning transmitter and a positioning receiver; a positioning transmitter is respectively arranged on each truss, and a positioning receiver is respectively arranged at a plurality of preset position points of the crane body;

[0149] The image collector is used to collect target images and images to be identified;

[0150] The controller is configured to perform the following steps:

[0151] Acquire a target image, wherein the target image includes a plurality of trusses, and the installation order of each truss is a target installation order;

[0152] Acquire an image to be identified in real time, wherein the image to be identified includes a plurality of trusses;

[0153] Based on the target image and the image to be identified, a first judgment is made on whether the truss installation sequence is correct or not;

[0154] Based on the time difference data generated by the wireless positioning sensor, the distance between each truss and the crane body is calculated;

[0155] A second judgment is made on the correctness of the truss installation sequence based on the distance between each truss and the crane body;

[0156] Based on the results of the first judgment and the second judgment, a final truss installation sequence recognition result is generated.

[0157] The truss identification device applicable to a crane in an embodiment of the present invention is described in detail below in conjunction with a specific implementation manner.

[0158] The target image and the image to be identified are collected by an image collector; the target image contains a plurality of trusses, and the installation order of each truss is the target installation order; the image to be identified is an image acquired in real time, the image to be identified contains a plurality of trusses, and the installation order of each truss may be wrong;

[0159] Image acquisition is the process of converting analog or digital image signals into digital signals that can be processed by computers. This technology has applications in many fields, including but not limited to video surveillance, medical imaging, industrial inspection, scientific research, etc. The following are some key information about image acquisition:

[0160] Image acquisition principle: Image acquisition involves the illumination of the target object, image acquisition and image digitization. Illumination highlights the features of the object, image acquisition generally captures the image through a camera, and image digitization converts the analog image signal into a digital image signal.

[0161] Image collectors: including cameras, scanners, mobile phone cameras, etc. These devices affect the quality and application scenarios of image data depending on the resolution, frame rate and sensor type.

[0162] The controller is used to pre-process the collected images, specifically:

[0163] Image preprocessing is an important step in computer vision and image processing, which aims to improve image quality for subsequent analysis and recognition. Image preprocessing is a process of performing a series of transformations and adjustments on raw image data, with the aim of eliminating irrelevant information in the image, restoring useful real information, and enhancing detectability, thereby improving the reliability of feature extraction, image segmentation, and recognition.

[0164] Grayscale transformation: including brightness and contrast adjustment, gamma correction, etc., aimed at improving the visual effect of the image.

[0165] Filtering: such as mean filtering, Gaussian filtering and median filtering, used to remove noise from images.

[0166] Histogram equalization: Enhances the contrast of an image by redistributing pixel values ​​to make the histogram more uniform.

[0167] Binarization: Convert the image into a black and white image by setting a threshold. Commonly used algorithms include the Otsu method.

[0168] Morphological processing: such as dilation and erosion, is mainly used to process the shape characteristics of images.

[0169] The correctness of the truss installation sequence is determined by object detection, which is an important task in the field of computer vision. It involves identifying and locating objects of interest in an image. This technology has a wide range of applications in many fields, including but not limited to face recognition, pedestrian detection, intelligent driving, medical image analysis, etc.

[0170] Object detection algorithms are mainly divided into two categories: two-stage algorithms and one-stage algorithms.

[0171] Two-stage algorithms: These algorithms first generate candidate regions, and then classify and regress bounding boxes for these regions. Representative algorithms include the R-CNN series (including Fast R-CNN and Faster R-CNN), which usually have high detection accuracy but relatively slow detection speed.

[0172] Single-stage algorithms: These algorithms directly detect objects on the entire image without generating candidate regions. Representative algorithms include YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector). These algorithms are characterized by fast detection speed and are suitable for real-time applications, but their accuracy is usually slightly lower than that of two-stage algorithms.

[0173] With the development of deep learning technology, target detection algorithms are also constantly improving, and new algorithms and improved methods are emerging to adapt to different application requirements and challenges. For example, the YOLO series of algorithms have improved detection speed and accuracy through continuous iteration and optimization, while the Faster R-CNN algorithm improves the efficiency of candidate region generation by introducing the Region Proposal Network (RPN).

[0174] The present invention uses the YOLO algorithm to identify and detect the trusses in the image, and the specific method steps are as follows:

[0175] (1) Data preprocessing: resize the image and normalize the data;

[0176] (2) Feature extraction: Extract multi-scale features of the trusses in the image through multiple layers of network structures of different sizes

[0177] (3) Feature fusion: Combine multi-scale features to capture target information at different scales and achieve feature fusion

[0178] (4) Predict the bounding box of the truss;

[0179] (5) Perform non-maximum suppression on all bounding boxes to ensure that each truss is accurately marked by only one bounding box;

[0180] (6) Give the marked bounding box a category and confidence score, and finally obtain several sub-images, each of which contains only one truss;

[0181] (7) Finally, several sub-images to be identified are output, and there is only one truss in a sub-image to be identified.

[0182] Perform feature operator extraction on each sub-image to obtain the corresponding corner points.

[0183] Feature operator extraction is an important task in computer vision. It aims to detect feature points from images that remain stable under image transformations. These points usually have unique properties, such as being able to be recognized under rotation, scaling, or lighting changes. Corner points are the intersection of two edges in an image and are the most common type of image features.

[0184] Harris corner detection is a classic corner extraction algorithm that identifies corners by calculating the gradient and second-order moment of an image. The basic steps of the Harris algorithm include:

[0185] (1) Calculate gradient: Use the Sobel operator to calculate the gradient of the image in the x and y directions.

[0186] (2) Construct the autocorrelation matrix: Calculate the square sum and product of the gradients to construct the autocorrelation matrix.

[0187] (3) Calculate the response value: Calculate the Harris response value through the eigenvalue of the autocorrelation matrix.

[0188] (4) Threshold processing: Set a threshold and take points with response values ​​higher than the threshold as corner point candidates.

[0189] (5) Non-maximum suppression: Non-maximum suppression is performed in local areas to ensure accurate positioning of corner points.

[0190] The advantages of Harris corner detection algorithm are simplicity and efficiency, but it may not work well when the corner points are unevenly distributed. To solve this problem, more advanced algorithms such as SIFT (Scale Invariant Feature Transform) can be used, which detects key points by constructing a scale space and assigns a direction to each key point to make it invariant to rotation. If the feature values ​​of the corner points in the two sub-images differ greatly, it is determined that the installation order of the truss in the sub-image is wrong, and the lifting operation is stopped and an alarm is issued.

[0191] The short-range wireless positioning sensor uses UWB (Ultra Wide Band) positioning technology, which is a communication technology that uses extremely narrow pulses for wireless transmission. It measures the distance and position with centimeter-level accuracy by measuring the flight time of radio signals. The advantages of UWB technology are its high accuracy, high security, low power consumption, strong penetration and strong anti-multipath ability, which make it particularly popular in the field of indoor positioning.

[0192] The main implementation methods of UWB positioning technology include:

[0193] Time Difference of Arrival (TDoA): Similar to the principle of GPS positioning, the location of an object is determined by measuring the time difference of arrival of signals between different base stations. This method requires precise time synchronization between base stations.

[0194] Two-way ranging (TWR): relies on two-way communication between two devices to measure the distance between them. This method does not require time synchronization between base stations, but the power consumption of the devices is relatively high.

[0195] Phase Difference of Arrival (PDoA): Combines the measurements of range and bearing to calculate the relative position of two devices without the need for additional infrastructure.

[0196] In the truss sequence detection, the specific steps of using wireless positioning sensors to identify the installation sequence include:

[0197] 1. Install a positioning receiver at the preset position of the vehicle body and install a positioning transmitter on each truss;

[0198] 2. Calculate the position of the truss and the vehicle body by the arrival time difference of the signal;

[0199] 3. If the distance between the truss position and the vehicle body is different from the preset distance, an alarm will be issued;

[0200] 4. Re-determine whether the installation sequence of the trusses is correct based on the alarm sound.

[0201] The present invention adopts wireless positioning sensors to detect the spatial position of the truss, arranges multiple positioning receivers on the vehicle body, and arranges positioning transmitters on each truss, and measures the distance of each truss relative to the vehicle body through the time difference data generated by the multiple positioning receivers.

[0202] The present invention first detects the installation order of the trusses through the target image and the image to be identified to determine whether each truss is installed correctly. However, there will be some errors in the image detection. For safety reasons, the wireless positioning sensor technology is further used to detect the distance of each truss relative to the vehicle body, and the distance is judged to detect whether the installation order is correct. If the installation order is correct, the hoisting operation is performed. If there is an error in the installation order, the correct order of the trusses is restored. It can effectively improve the efficiency and accuracy of truss installation order recognition.

[0203] Example 3

[0204] Based on the same inventive concept as that of Embodiment 1, an embodiment of the present invention provides a truss identification device applicable to a crane, comprising:

[0205] A target image acquisition module is used to acquire a target image, wherein the target image includes a plurality of trusses, and the installation order of each truss is a target installation order;

[0206] An image acquisition module to be identified, used for acquiring an image to be identified in real time, wherein the image to be identified includes a plurality of trusses;

[0207] A sub-image extraction module to be identified, used to extract a plurality of sub-images to be identified from the image to be identified, wherein one sub-image to be identified contains only one truss;

[0208] A first judgment module, used for making a first judgment on whether the truss installation sequence is correct or not based on the target image and the image to be recognized;

[0209] A distance acquisition module is used to calculate the distance between each truss and the crane body based on the time difference data generated by the wireless positioning sensor;

[0210] A second judgment module, used for making a second judgment on whether the truss installation sequence is correct or not based on the distance between each truss and the crane body;

[0211] The identification module is used to generate a final truss installation sequence identification result based on the results of the first judgment and the second judgment.

[0212] The rest are the same as in Example 1.

[0213] Example 4

[0214] Based on the same inventive concept as in Example 1, an embodiment of the present invention provides a truss identification system applicable to a crane, including a storage medium and a processor;

[0215] The storage medium is used to store instructions;

[0216] The processor is configured to operate according to the instructions to execute the method according to any one of the first aspects.

[0217] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0218] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0219] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0220] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0221] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

[0222] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A truss identification method suitable for a crane, characterized in that: include: Acquire a target image, wherein the target image includes a plurality of trusses, and the installation order of each truss is a target installation order; Acquire an image to be identified in real time, wherein the image to be identified includes a plurality of trusses; Based on the target image and the image to be identified, a first judgment is made on whether the truss installation sequence is correct or not; Based on the time difference data generated by the wireless positioning sensor, the distance between each truss and the crane body is calculated; A second judgment is made on the correctness of the truss installation sequence based on the distance between each truss and the crane body; Based on the results of the first judgment and the second judgment, a final truss installation sequence recognition result is generated.

2. A truss identification method suitable for a crane according to claim 1, characterized in that: The first determination of whether the truss installation sequence is correct or not based on the target image and the image to be identified includes: Extracting a plurality of target sub-images from the target image, wherein each target sub-image contains only one truss; Perform feature operator extraction on each target sub-image respectively to obtain the first corner point corresponding to each target sub-image; Extracting a plurality of sub-images to be identified from the image to be identified, wherein one sub-image to be identified contains only one truss; Perform feature operator extraction on each sub-image to be identified, and obtain a second corner point corresponding to each sub-image to be identified; Based on the first corner point and the second corner point, the correctness of the truss installation sequence is judged.

3. A truss identification method applicable to a crane according to claim 2, characterized in that: The methods for obtaining corner points include: Use the Sobel operator to calculate the gradient of the image in the x and y directions; Calculate the square and product of the gradients and construct the autocorrelation matrix; The Harris response value is calculated by the eigenvalues ​​of the autocorrelation matrix; Points with Harris response values ​​higher than the set threshold are regarded as candidate corner points; Perform non-maximum suppression on candidate corner points to generate the final corner points.

4. A truss identification method applicable to a crane according to claim 2, characterized in that: The target sub-image and the sub-image to be identified are obtained by the following method: The multi-scale features of the trusses in the image are extracted through multiple layers of network structures of different sizes, and the multi-scale features are fused; Make bounding box predictions for trusses; Perform non-maximum suppression on all bounding boxes to ensure that each truss is accurately marked by only one bounding box; The marked bounding box is given a category and a confidence score, and finally several sub-images are obtained, each of which contains only one truss.

5. The truss identification method applicable to a crane according to claim 2, characterized in that: The determining of whether the truss installation sequence is correct based on the first corner point and the second corner point includes: Compare the SIFT descriptor of the second corner point of a sub-image to be identified with the SIFT descriptor of the first corner point of the target sub-image corresponding to the sub-image to be identified; If the similarity between the two SIFT descriptors is greater than the set threshold, it means that the installation order of the trusses in the sub-image to be identified is wrong; Among them, the formula used for comparison is: ； ； ； in, is the corner point in the target sub-image SIFT descriptor, is the corner point in the sub-image to be identified SIFT descriptor, Indicates the number of dimensions of the SIFT descriptor, is the difference between two eigenvalues.

6. A truss identification method applicable to a crane according to claim 1, characterized in that: A positioning transmitter is respectively arranged on each truss, and a plurality of positioning receivers are arranged at preset positions of the crane body. The truss identification method applicable to the crane also includes: Based on the time difference data generated by multiple positioning receivers, the distance between each truss and the crane body is calculated; Based on the actual distance between each truss and the crane body and the preset target distance between each truss and the crane body, the correctness of the truss installation sequence is judged.

7. A truss identification method applicable to a crane according to claim 6, characterized in that: If the difference between the actual distance between a certain truss and the crane body and the preset target distance between the truss and the crane body is greater than a set threshold, it is determined that the installation sequence of the truss is wrong.

8. A truss identification device suitable for a crane, characterized in that: It includes an image collector, a wireless positioning sensor and a controller. The wireless positioning sensor includes a positioning transmitter and a positioning receiver. A positioning transmitter is respectively arranged on each truss, and a positioning receiver is respectively arranged at multiple preset position points of the crane body. The image collector is used to collect target images and images to be identified; The controller is configured to perform the following steps: Acquire a target image, wherein the target image includes a plurality of trusses, and the installation order of each truss is a target installation order; Acquire an image to be identified in real time, wherein the image to be identified includes a plurality of trusses; Based on the target image and the image to be identified, a first judgment is made on whether the truss installation sequence is correct or not; Based on the time difference data generated by the wireless positioning sensor, the distance between each truss and the crane body is calculated; A second judgment is made on the correctness of the truss installation sequence based on the distance between each truss and the crane body; Based on the results of the first judgment and the second judgment, a final truss installation sequence recognition result is generated.

9. A truss identification device suitable for a crane, characterized in that: include: A target image acquisition module is used to acquire a target image, wherein the target image includes a plurality of trusses, and the installation order of each truss is a target installation order; An image acquisition module to be identified, used for acquiring an image to be identified in real time, wherein the image to be identified includes a plurality of trusses; A sub-image extraction module to be identified, used to extract a plurality of sub-images to be identified from the image to be identified, wherein one sub-image to be identified contains only one truss; A first judgment module, used for making a first judgment on whether the truss installation sequence is correct or not based on the target image and the image to be recognized; A distance acquisition module is used to calculate the distance between each truss and the crane body based on the time difference data generated by the wireless positioning sensor; A second judgment module is used to make a second judgment on whether the truss installation sequence is correct or not based on the distance between each truss and the crane body; The identification module is used to generate a final truss installation sequence identification result based on the results of the first judgment and the second judgment.

10. A truss identification system suitable for a crane, characterized in that: including storage media and processors; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the method according to any one of claims 1-7.

Images