A real-time distance measurement method for gantry crane hoisting objects based on video
Through video surveillance combined with deep learning technology, real-time distance measurement of gantry crane objects is achieved, safety accidents caused by foreign object intrusion are solved, fast, intelligent and accurate distance measurement methods are provided, and operational safety and efficiency are improved. Reduced costs.
Patent Information
- Application Number
- CN202111580789.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-22
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-12-22
AI Technical Summary
During the operation, the gantry crane is prone to invasion of foreign objects due to overload or misoperation, causing safety accidents. The existing technology has problems such as low accuracy, lag and relying on manual identification in terms of hanging object distance measurement, making it difficult to achieve fast intelligent distance measurement and Real-time notification.
Using the video-based real-time distance measurement method of door machine hanging objects, the feature extraction network and YOLOV3 model are used to detect hanging objects and foreign objects, and the video images are collected in real time using the camera, and the model is trained for automated detection, and the distance between foreign objects and hanging objects is calculated, and the distance between foreign objects and hanging objects is calculated through The final result is calculated by geometric principles to achieve real-time and accurate ranging.
It realizes rapid and intelligent distance measurement of different hanging objects and foreign objects in complex outdoor environments, reduces the dependence of manual identification, improves safety and work efficiency, reduces costs, and has strong anti-interference. It is suitable for a wide range of applications. Scene.
Smart Images

Figure CN114463693B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of image target detection, and more specifically, relates to a video-based real-time distance measurement method for gantry crane hoisted objects. Background Art
[0002] Gantry crane is a variation of bridge crane, also called gantry crane. It is mainly used for loading and unloading of outdoor cargo yards, material yards and bulk cargo. Its metal structure is like a door-shaped frame, with two legs installed under the main beam, which can walk directly on the track on the ground, and the two ends of the main beam can have outward cantilever beams.
[0003] Compared with other lifting equipment, gantry cranes have simple structures, are easy to use, have a large operating range, are widely adaptable, and have high safety and reliability. They are widely used in port transportation and shipbuilding industries. However, since most gantry cranes are overloaded during on-site operations, accidents caused by overloading and misoperation occur frequently. When the gantry crane is operating, especially when the operator is lifting objects with a hook, foreign objects should be avoided as much as possible. Considering the particularity of the site environment where the gantry crane is located, foreign objects are generally divided into people, vehicles, large mechanical equipment, etc.; foreign object intrusion will not only hinder normal lifting operations, but also cause a certain degree of damage to foreign objects, which may lead to major safety accidents in severe cases. Therefore, it is very important to quickly and intelligently measure distance and notify the operator.
[0004] The real-time distance measurement of the hoisted objects of the gantry crane based on video requires first learning the characteristics of a large number of hoisted objects and foreign objects through a deep learning network, and then performing detection on this basis. This can effectively eliminate the defects of low accuracy and large lag in manual observation. The method of using video monitoring proposed in the present invention is not only safe and reliable, but also low in cost and has a wide range of applications. Summary of the invention
[0005] In view of the many limitations of existing methods in measuring the distance of objects hoisted by gantry cranes, the present invention proposes a real-time distance measurement method for objects hoisted by gantry cranes based on video to overcome these problems.
[0006] The basic principle of the target detection model adopted in the present invention is: first use the feature extraction network to extract the features of the image, so as to obtain the corresponding n*n feature map, and then divide the image into an n*n unit network format based on this, and then use the unit cell to predict the target whose center coordinates fall within it. Each cell will predict a certain number of bounding boxes, and select the bounding box with the largest intersection-over-union ratio with the true box for prediction.
[0007] The two dimensions in the resulting feature map are the extracted features, one is the plane, such as n*n; the other is the depth, such as S*(5+T), where S represents the number of bounding boxes predicted by a cell, T represents the number of categories corresponding to the bounding box, and 5 represents 1 bounding box confidence and 4 coordinate information.
[0008] The method proposed in the present invention is aimed at the problem of real-time distance measurement of gantry crane hoisted objects under video surveillance. The method can detect different types of hoisted objects and foreign objects, is less affected by complex outdoor factors such as rain and light, has strong anti-interference ability, does not need to rely on manual auxiliary identification, can realize automatic detection, and at the same time, the real-time and accuracy of the detection meet the requirements.
[0009] In order to achieve the above object, the technical solution adopted by the present invention is:
[0010] A video-based real-time distance measurement method for gantry crane hoisting objects comprises the following steps:
[0011] S1: Set up a camera, aim the camera's monitoring range at the hanging object, and capture video images in real time;
[0012] S2: Collect sample images of hanging objects and amplify them; annotate the real detection frame and category of the hanging objects in each sample image, and use the annotated sample images as the hanging object training set; use the CoCo dataset as the foreign object training set;
[0013] S3: Establish a YOLOV3 model, and use the suspended object training set and the foreign object training set obtained in step S2 to train the model, so as to obtain a trained suspended object foreign object detection model;
[0014] The YOLOV3 model includes a network input layer, a Darknet-53 network, and a multi-scale prediction layer. The sample image is used as the input of the network input layer to obtain a sample image of uniform size. The Darknet-53 network is trained using the sample image of uniform size to output the feature map of the sample image. Finally, the multi-scale prediction layer uses three scales to annotate the feature map with a prediction box, which is compared with the true prediction box, and the prediction box and confidence are output. The foreign body training process is the same.
[0015] S4: using a camera to collect a video of the suspended object when the gantry crane is working in real time, extracting n frames at equal intervals from the video corresponding to the current moment as the image to be detected at the current moment, using the image to be detected as the input of the trained suspended foreign object detection model obtained in step S3, and obtaining a detection result, the detection result including the target confidence and the detection frame coordinates, and on this basis calculating the distance from the foreign object to the suspended object according to the geometric principle;
[0016] S5: taking the average of the distances from the foreign object to the hanging object corresponding to the n frames of images at the current moment, and comparing it with the set threshold, and outputting the comparison result as the final detection result;
[0017] S6: Repeat steps S4 to S5 to perform the detection of the gantry crane hoisting distance at the next moment to achieve real-time detection.
[0018] Compared with the prior art, the advantages of the present invention are:
[0019] (1) The present invention combines monitoring data and video images, no longer displays technical points separately, saves hard disk recorders, and integrates video monitoring technology into industrial control configuration technology, which saves costs economically and alleviates the narrow space in the cockpit on the one hand;
[0020] (2) The YOLOV3 deep learning network constructed by the present invention can effectively and fully learn the characteristics of hanging objects and foreign objects. The network has good stability and strong anti-interference ability and can be put into use for a long time.
[0021] (3) The YOLOV3-based automatic ranging algorithm constructed in the present invention ensures the safety of the gantry crane during lifting operations, improves work efficiency, and reduces labor burden. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 The present invention provides a flow chart of a method for real-time distance measurement of objects hoisted by a gantry crane.
[0023] Figure 2 is a data set expansion diagram provided by an embodiment of the present invention
[0024] Figure 3 It is a Darknet-53 network structure diagram provided by an embodiment of the present invention.
[0025] Figure 4 It is a Feature Pyramid Network diagram provided by an embodiment of the present invention.
[0026] Figure 5 It is a geometric principle diagram of suspended object distance measurement provided by an embodiment of the present invention.
[0027] Figure 6 This is a diagram of the real-time distance measurement results of the gantry crane hoisting object provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0028] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention is described in detail below in conjunction with specific embodiments. Specific embodiments are described below to simplify the present invention. However, it should be recognized that the present invention is not limited to the illustrated embodiments, and various modifications of the present invention are possible without departing from the basic principles, and these equivalent forms also fall within the scope defined by the claims attached to the present application.
[0029] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0030] like Figure 1 As shown, the present invention provides a video-based real-time distance measurement method for gantry crane hoisting objects, which mainly includes the following steps:
[0031] Step 1: Set up a camera, aim the camera's monitoring range at the hanging object, and shoot video images in real time; the number of cameras for detecting hanging objects in the present invention can be one or more, preferably multiple. For example, multiple cameras are installed in different parts of the door machine to shoot the hanging object and its surrounding areas from multiple angles. In actual work, when the real-time hanging object video or image shot by any camera is judged by the method of the present invention to need an alarm, an alarm is issued to avoid missed detection.
[0032] Step 2: Collect sample images of hanging objects and amplify them; label the real detection frame and category of the hanging objects in each sample image, and use the labeled sample images as the hanging object training set; use the CoCo dataset as the training set for foreign objects. The CoCo dataset is as follows: Figure 2 As shown,
[0033] Step 3: Establish a YOLOV3 model, and use the suspended object training set and foreign object training set obtained in step S2 to train the model to obtain a trained suspended object foreign object detection model;
[0034] The YOLOV3 model includes a network input layer, a Darknet-53 network, and a multi-scale prediction layer. The sample image is used as the input of the network input layer to obtain a sample image of uniform size. The Darknet-53 network is trained using the sample image of uniform size to output the feature map of the sample image. Finally, the multi-scale prediction layer uses three scales to annotate the feature map with a prediction box, which is compared with the true prediction box, and the prediction box and confidence are output. The foreign body training process is the same.
[0035] Step 4: The video of the suspended object when the gantry crane is working is collected in real time by the camera, and n frames are equally spaced from the video corresponding to the current moment as the image to be detected at the current moment. The image to be detected is used as the input of the trained suspended foreign object detection model obtained in step S3 to obtain the detection result, which includes the target confidence and the detection frame coordinates. On this basis, the distance from the foreign object to the suspended object is calculated according to the geometric principle and compared with the threshold distance to output the final result;
[0036] Step 5: Take the average of the threshold comparison results corresponding to the n frames of images at the current moment as the final detection result output;
[0037] Step 6: Repeat steps S4 to S5 to perform the detection of the gantry crane hoisting distance at the next moment to achieve real-time detection.
[0038] The specific implementation method is introduced below.
[0039] Step 1: Set a suitable location to shoot surveillance video in real time;
[0040] Step 2: Configure the detection environment, build the data set, and annotate the samples:
[0041] Using Anaconda as the platform, we built a real-time distance measurement environment for gantry cranes using Keras 2.1.5 and TensorFlow-GPU2.1 to collect sample images.
[0042] On the basis of existing data, random cropping is adopted, that is, cropping part of the original image, such as cropping the four corners, the center, or the upper and lower parts, etc., to increase the data volume; the data volume is increased by flipping the mirror and other operations; the data volume is expanded by adjusting the image brightness and contrast and other operations.
[0043] Step 3: Build a deep learning network architecture based on the YOLOV3 algorithm, which mainly consists of three parts: network input, Darknet-53 and multi-scale prediction.
[0044] 3-1: The specific implementation method of network input is:
[0045] Since YOLOV3 has 5 upsamplings and each sampling step is 2, the maximum stride of the network (the stride refers to the input size of the layer divided by the output) is 2^5=32, so the image size of the network input must be adjusted to an integer multiple of 32, here the size is set to 256x256.
[0046] 3-2: Darknet-53 The entire network is mainly composed of a series of 1x1 and 3x3 convolutional layers. Each convolutional layer is followed by a BN (Batch Normalization) layer and a LeakyReLU layer. Because there are 53 convolutional layers in the network, this part is called Darknet-53. Figure 3 shown.
[0047] a) For an input image of size 256x256, it first passes through a 3x3x32 convolutional layer to output 256x256x32;
[0048] b) Then it passes through a 3x3x64 convolutional layer with stride=2 and the output is 128x128x64;
[0049] c) After a residual block, the output is 128x128x64;
[0050] d) After a 3x3x128 convolutional layer with stride=2, the output is 64x64x128;
[0051] e) After 2 residual blocks, the output is 64x64x128;
[0052] f) After a 3x3x256, stride=2 convolutional layer, the output is 32x32x256;
[0053] g) After 8 residual blocks, the output is 32x32x256;
[0054] h) The output is 16x16x512 after passing through a 3x3x512 convolutional layer with stride=2.
[0055] i) After 8 residual blocks, the output is 16x16x512;
[0056] j) The output is 8x8x1024 through a 3x3x1024 convolutional layer with stride=2;
[0057] k) After 4 residual blocks, the output is 8x8x1024;
[0058] l) After the pooling fully connected layer and the sigmoid activation function layer, the final feature map and prediction probability are output.
[0059] Adjust the parameters of the above network architecture:
[0060] a) Randomly initialize the weight parameter W of the network structure;
[0061] b) Randomly select a batch of sample images as iterative sample image input;
[0062] c) Image feature extraction is performed through the Darknet-53 network, and the probability output is obtained by forward transmission. The node with the largest probability value among the output nodes is selected as the prediction result;
[0063] d) Use the prediction results to calculate the sigmoid cross entropy loss function of this iteration. The calculation formula of the sigmoid cross entropy loss function is:
[0064] H(p,q)=-∑(p(x)logq(x)+(1-p(x))log(1-q(x)))
[0065] Among them, probability distribution p(x) is the expected output, probability distribution q(x) is the actual output, and H(p,q) is the cross entropy;
[0066] e) Update the parameters W in the model using the error back propagation BP algorithm and Adam optimization algorithm;
[0067] f) Repeat steps b) to e) until a preset number of iterations is completed.
[0068] The step of updating the parameter W in the deep learning model using the error back propagation BP algorithm and the Adam optimization algorithm dynamically adjusts the learning rate of each parameter using the first-order moment estimation and the second-order moment estimation of the gradient, and iteratively updates each parameter W until the optimal solution is obtained:
[0069] W′=W+learning rate
[0070] Where learning rate is the learning rate and W′ is the updated parameter.
[0071] When training the model, corresponding measures are taken to adjust the model, and finally a detection model with the lowest loss value and the highest accuracy is obtained.
[0072] 3-3: Multi-scale prediction:
[0073] Different from the single-label classification of YOLOV2, YOLOV3 uses multi-label classification, and the classifier is changed from Softmax to Logistic. This is mainly because the former can only classify an object into one category, while the latter can classify it into multiple categories. This improvement of YOLOV3 solves the multi-label problem while ensuring that the accuracy is not affected, making it very useful in complex fields such as open image datasets.
[0074] In the YOLOV3 algorithm, the FPN structure is used, such as Figure 4 As shown in the figure. By using the upsampling and fusion method, the prediction is performed by fusion of feature maps of three scales (13*13, 26*26, 52*52), which is very effective for detecting small targets. In the FPN structure, the feature map of the current layer upsamples and utilizes the feature map of the future layer to obtain the information of the future layer, fuse the low-level and high-level features, and improve the detection accuracy.
[0075] There are three scales of feature maps output in YOLOV3, and three prior bounding boxes are set for cells of feature maps of different scales, so there are a total of nine prior bounding boxes.
[0076] The specific formula for solving the YOLOV3 bounding box is as follows:
[0077] b x =σ(t x )+c x
[0078] b y =σ(t y )+c y
[0079]
[0080]
[0081] Pr(object)*IOU(b,object)=σ(t o )
[0082] Among them, c x and c y Indicates the horizontal and vertical distance of a pixel from the upper left corner; σ(t x ) and σ(t y ) represent the offset of the prediction box in the x-coordinate and y-coordinate respectively; (b x ,b y ,b w ,b h ) represent the x-coordinate, y-coordinate, width and height of the prediction box center respectively; (t x ,t y ,t w ,t h )∈(0,1) are the parameters learned by the network, which respectively represent the offset of the center position x of the prediction box, the offset of y, the scale offset of the width, and the scale offset of the height; p w ,p h They represent the width and height of the prior box relative to the feature map respectively; Pr(object) represents the probability of the target existing; IOU(b,object) represents the intersection-over-union ratio, that is, the overlap ratio of the target prediction box window and the original label window; σ(t o ) indicates the confidence level.
[0083] 3-4: Calculation of the distance of hanging objects:
[0084] like Figure 5 As shown in the figure, the detected hanging object ROI area is a trapezoid ABCD, and the parameter information of the trapezoid sides AD and BC are obtained by the Hough line detection method. When the foreign object is located on the left side of the hanging object, its detection box can be abstracted into point G (x G ,y G ):
[0085]
[0086] Similarly, when the foreign object is located on the right side of the hanging object, its detection frame can be abstracted into point H (x H ,y H ):
[0087]
[0088] Among them, left, right, top, and bottom are the four single coordinates of the detection box.
[0089] The connection points G and H intersect the trapezoid sides AD and BC at points E and F respectively; considering the pixel distance and the actual distance, the calculation formula of the hanging object distance h can be obtained by using the principle of similar triangles:
[0090]
[0091] Where d is the actual width of the hanging object, which can be obtained through field measurement; x E 、x F It can also be converted after obtaining the trapezoid side AD and BC parameters.
[0092] Taking into account the inevitable vibration of the camera caused by external environmental factors, the threshold distance can be taken as 1.2m. When h is less than the threshold, it is determined that foreign objects have invaded the hanging operation, and an alarm is issued to the operator.
[0093] Figure 6 This is the result of real-time distance measurement of gantry crane hoisted objects. The detected hoisted objects are marked with dotted frames, and foreign objects (people) are marked with solid frames. The recognized category and confidence level are displayed above the detection frame, and the real-time distance between the foreign object and the hoisted object is displayed below. At this time, the distance is 1.46m, which is not less than the threshold, so no alarm is required, which is consistent with the manual detection result.
[0094] Compared with the prior art, the method of the embodiment of the present invention is less affected by complex outdoor factors, has strong anti-interference ability, does not need to rely on manual auxiliary identification during detection, can realize intelligent detection, and at the same time, the real-time and accuracy of the detection meet the requirements.
[0095] The accompanying drawings shown in the embodiments of the present invention can make the purpose, technical solutions and advantages of the present invention more clearly described. It should be noted that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. All equivalent replacements, improvements, etc. made within the method ideas and principles provided by the present invention should be included in the protection scope of the present invention.
Claims
1. A video-based real-time distance measurement method for gantry crane hoisting objects, characterized in that: The steps include: S1: Set up a camera, aim the camera's monitoring range at the hanging object, and shoot video images in real time; S2: Collect sample images of hanging objects and amplify them; annotate the real detection frame and category of the hanging objects in each sample image, and use the annotated sample images as the hanging object training set; use the CoCo dataset as the foreign object training set; S3: Establish a YOLOV3 model, and use the suspended object training set and the foreign object training set obtained in step S2 to train the model to obtain a trained suspended object foreign object detection model; The YOLOV3 model includes a network input layer, a Darknet-53 network, and a multi-scale prediction layer. The sample image is used as the input of the network input layer to obtain a sample image of uniform size. The Darknet-53 network is trained using the sample image of uniform size to output the feature map of the sample image. Finally, the multi-scale prediction layer uses three scales to annotate the feature map with a prediction box, compares it with the true prediction box, and outputs the prediction box and confidence. The foreign object training process is similar. S4: using a camera to collect a video of the suspended object when the gantry crane is working in real time, extracting n frames at equal intervals from the video corresponding to the current moment as the image to be detected at the current moment, using the image to be detected as the input of the trained suspended foreign object detection model obtained in step S3, and obtaining a detection result, the detection result including the target confidence and the detection frame coordinates, and on this basis calculating the distance from the foreign object to the suspended object according to the geometric principle; The detected hanging object ROI area is a trapezoid, and the parameter information of the straight lines where the two waists are located is obtained by the Hough line detection method; when the foreign object is located on the left side of the hanging object, its detection frame is abstracted into point G (x G ,y G ): Similarly, when the foreign object is located on the right side of the hanging object, its detection frame is abstracted into point H (x H ,y H ): Where left, right, top, and bottom are the four single coordinates of the detection box; The connection points G and H intersect the two sides of the trapezoid at points E and F respectively; considering the pixel distance and the actual distance, the calculation formula of the hanging object distance h is obtained by using the principle of similar triangles: Where d is the actual width of the hanging object, obtained through field measurement; x E 、x F It is obtained by conversion after obtaining the parameters of the two sides of the trapezoid; S5: taking the average of the distances from the foreign object to the hanging object corresponding to the n frames of images at the current moment, and comparing it with the set threshold, and outputting the comparison result as the final detection result; S6: Repeat steps S4 to S5 to perform the detection of the gantry crane hoisting distance at the next moment to achieve real-time detection.
2. The video-based real-time distance measurement method for gantry crane hoisting objects according to claim 1 is characterized in that: The method of enlarging the sample image in step S2 includes one or more of random cropping, mirror flipping, and random adjustment of image parameters, wherein: The random cropping is: randomly cropping the four corners, the center, or the upper and lower parts of the original image, and using the cropped image as a new sample image; The mirror flipping is: mirror flipping the original image, and using the flipped image as a new sample image; The randomly adjusting image parameters is: randomly adjusting the brightness and contrast of the original image, and using the adjusted image as a newly added sample image.
3. The video-based real-time distance measurement method for gantry crane hoisting objects according to claim 1 is characterized in that: The real detection frame described in step S2 is represented by the coordinate information of the upper left and lower right points of the target.
4. The video-based real-time distance measurement method for gantry crane hoisting objects according to claim 1 is characterized in that: The network input layer in the YOLOV3 model resizes the input image to an integer multiple of 32.
5. The video-based real-time distance measurement method for gantry crane hoisting objects according to claim 4 is characterized in that: The input image size is 256x256, and the Darknet-53 network in the YOLOV3 model processes the input image as follows: 1) For an input image of size 256x256, it first passes through a 3x3x32 convolutional layer to output 256x256x32; 2) Then it passes through a 3x3x64, stride=2 convolutional layer and the output is 128x128x64; 3) After a residual block, the output is 128x128x64; 4) After passing through a 3x3x128, stride=2 convolutional layer, the output is 64x64x128; 5) After 2 residual blocks, the output is 64x64x128; 6) After a 3x3x256, stride=2 convolutional layer, the output is 32x32x256; 7) After 8 residual blocks, the output is 32x32x256; 8) After passing through a 3x3x512 convolutional layer with stride=2, the output is 16x16x512; 9) After 8 residual blocks, the output is 16x16x512; 10) After passing through a 3x3x1024 convolutional layer with stride=2, the output is 8x8x1024; 11) After 4 residual blocks, the output is 8x8x1024; 12) After the pooling fully connected layer and the sigmoid activation function layer, the final feature map and prediction probability are output.
6. The video-based real-time distance measurement method for gantry crane hoisting objects according to claim 5 is characterized in that: The process of training the model in step S3 is: a) Randomly initialize the weight parameter W of the network structure; b) Randomly select a batch of sample images as iterative sample image input; c) Image feature extraction is performed through the Darknet-53 network, and the probability output is obtained by forward transmission. The node with the largest probability value among the output nodes is selected as the prediction result; d) Use the prediction results to calculate the sigmoid cross entropy loss function of this iteration. The calculation formula of the sigmoid cross entropy loss function is: H(p,q)=-Σ(p(x)logp(x)+(1-p(x))log(1-q(x))) Among them, probability distribution p(x) is the expected output, probability distribution q(x) is the actual output, and H(p,q) is the cross entropy; e) Update the parameters W in the model using the error back propagation BP algorithm and Adam optimization algorithm; f) Repeat steps b) to e) until a preset number of iterations is completed.
7. The video-based real-time distance measurement method for gantry crane hoisting objects according to claim 6 is characterized in that: Between steps e) and f), the learning rate of each parameter is dynamically adjusted using the first-order moment estimation and the second-order moment estimation of the gradient, and each parameter W is iteratively updated until the optimal solution is obtained: W′=W+learning rate Where learning rate is the learning rate and W′ is the updated parameter.
8. The video-based real-time distance measurement method for gantry crane hoisting objects according to claim 1 is characterized in that: The multi-scale prediction layer in the YOLOV3 model adopts the FPN structure, and uses the fusion of feature maps of three sizes, 13*13, 26*26, and 52*52, for prediction. The cells of feature maps of different scales are provided with three prior bounding boxes, so there are a total of nine prior bounding boxes. In the FPN structure, the feature map of the current layer upsamples and fuses the feature map of the future layer, and the final output bounding box solution formula is: b x =σ(t x )+c x b y =σ(t y )+c y Pr(object)*IOU(b,object)=σ(t o ) Among them, c x and c y Indicates the horizontal and vertical distance of a pixel from the upper left corner; σ(t x ) and σ(t y ) represent the offset of the prediction box in the x-coordinate and y-coordinate respectively; (b x ,b y ,b w ,b h ) represent the x-coordinate, y-coordinate, width and height of the prediction box center respectively; (t x ,t y ,t w ,t h )∈(0,1) are the parameters learned by the network, which respectively represent the offset of the center position x of the prediction box, the offset of y, the scale offset of the width, and the scale offset of the height; p w ,p h They represent the width and height of the prior box relative to the feature map respectively; Pr(object) represents the probability of the target existing; IOU(b,object) represents the intersection-over-union ratio, that is, the overlap ratio of the target prediction box window and the original label window; σ(t o ) indicates the confidence level.
9. The video-based real-time distance measurement method for gantry crane hoisting objects according to claim 1 is characterized in that: There are multiple cameras for detecting suspended objects, which are installed in different parts of the gantry crane to take multi-angle shots to avoid missed detections.
