A tracking method and system for port lifting equipment sent from the cloud

Abstract

The present invention discloses a tracking method and system for port crane equipment sent from the cloud. The method receives crane equipment information sent from the cloud, splits the crane equipment into two supporting legs, and then corrects the position and orientation information of each supporting leg of the crane equipment in combination with the characteristics of the crane equipment during operation and high-precision map information. Then, each supporting leg of the crane equipment is tracked to obtain more accurate position and speed information of the target. Finally, the tracking target and the detection target perceived by the vehicle are checked for coincidence. The tracking target with high coincidence is fused and output with the vehicle perception, and the tracking target with low coincidence is directly output to the downstream for use, thereby making full use of the crane equipment information sent from the cloud and improving the efficiency and interactive safety of port operations.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent driving technology and relates to the tracking of heavy lifting equipment such as quay cranes and tire cranes, and specifically to a tracking method and system for port lifting equipment sent from the cloud. Background Art

[0002] At present, the labor cost in the field of port transportation is constantly rising, and the operating cost is also rising. At the same time, with the increasing development of global trade, the cargo throughput of ports has increased accordingly, and ports urgently need to improve efficiency. Under the dual pressure of reducing operating costs and improving efficiency, the use of autonomous driving in ports has received attention from all parties. The port's autonomous driving transfer vehicles can operate unmanned 24 hours a day, which improves the utilization rate of port equipment, significantly improves efficiency while reducing operating costs.

[0003] There are various container lifting equipment in the port (referring to large lifting equipment with two legs such as quay cranes and tire cranes). Among them, mobile lifting equipment such as bridge cranes and tire cranes need to be paid special attention. Many operations in the operation of autonomous driving vehicles require interaction with lifting equipment, and attention should also be paid to avoiding lifting equipment during operations. At present, the perception of lifting equipment is mainly single-vehicle perception, but lifting equipment such as bridge cranes are relatively large, and the span between the two legs is relatively large. Single-vehicle perception sometimes cannot fully perceive the two legs of the bridge crane; and in the container area, due to the obstruction of containers, there are blind spots in the field of vision, and sometimes it is impossible to fully perceive equipment such as tire cranes and rail cranes. This requires lifting equipment information sent from the cloud to make up for the shortcomings of single-vehicle perception.

[0004] The information of cranes sent from the cloud at the port is usually the overall size and location of the equipment for port dispatching. There is no individual location information of each support leg, and there are certain errors in the location and orientation information, which cannot be used directly by autonomous vehicles. Therefore, it is necessary to track the cranes sent from the cloud more accurately to improve the efficiency and safety of autonomous driving. Summary of the invention

[0005] In response to the above problems, the main purpose of the present invention is to design a tracking method and system for port lifting equipment sent from the cloud, so as to solve the problem that the lifting equipment information sent from the cloud cannot be used directly and has low accuracy, and to improve the efficiency and safety of port automatic driving.

[0006] In order to achieve the above purpose, the present invention adopts the following technical solutions:

[0007] A method for tracking port lifting equipment sent from the cloud comprises the following steps:

[0008] Receive information data of lifting equipment sent from the cloud, where each frame of information data sent from the cloud includes information of multiple lifting equipment, and the information of each lifting equipment includes: measurement time, target number, center point position, and four corner point information;

[0009] Filter the received lifting equipment information to obtain effective cloud-based information;

[0010] Based on the filtered information, extract the supporting structure information on both sides of the lifting equipment, including the center point, direction, length, and width;

[0011] Combined with the map information and the characteristics of the lifting equipment operating along the track, the position and orientation angle of the supporting structure are corrected to obtain the accurate position and orientation information of the lifting equipment supporting structure;

[0012] Track the corrected support structure to obtain the target stable speed;

[0013] The support structure sent down from the cloud after tracking is checked for overlap with the support structure perceived by the vehicle. If the overlap is greater than the threshold, the support structure sent down from the cloud after tracking and the support structure perceived by the vehicle are fused and output. If the overlap is less than the threshold, the support structure sent down from the cloud after tracking is directly sent to the downstream for use.

[0014] As a further description of the present invention, the method of screening the lifting equipment information includes: time verification and data validity verification;

[0015] The time verification method is to use the measurement time t of the data frame sent by the cloud at the current time i i , and the measured time t received by the previous frame i-1 i-1 Check if t i =t i-1 , then the frame data is considered as duplicate data and discarded;

[0016] The data validity verification method is to obtain the latitude and longitude origin O of the port project to which the lifting equipment belongs project , and the maximum operating range R of the lifting equipment project ;

[0017] Traverse all the lifting equipment information data sent from the cloud, and calculate the distance from the observation center point and corner point of each lifting equipment to the origin O project If the calculated distance is greater than the above operating range R project , the observed center point or corner point is considered invalid and discarded.

[0018] As a further description of the present invention, the supporting structure is two supporting legs of the lifting equipment, and the information of the supporting structure includes: the center point of the left leg, the center point of the right leg, the direction of the left leg, the direction of the right leg, the length of the left leg and the right leg, and the width of the left leg and the right leg.

[0019] As a further description of the present invention, according to the four corner points of the lifting device [c 1 ,c 2 ,c 3 ,c 4 ] Calculate the length and direction of the two supporting legs, the left leg c 1 c 2 and right leg c 3 c 4 The lengths of are the same, both are l, and the expression is:

[0020] l=(||c 1 -c 2 ||+||c 4 -c 3 ||)*0.5;

[0021] Left leg 1 c 2 Direction h left and right leg c 3 c 4 Direction h right , the expression is:

[0022]

[0023] Among them, (c 1 -c 2 ) x is the vector c 1 -c 2 The x-component of 1 -c 2 ) y is the vector c 1 -c 2 The y-direction component of 4 -c 3 ) x is the vector c 4 -c 3 The x-component of 4 -c 3 ) y is the vector c 4 -c 3 The y-direction component;

[0024] According to the prior width of the left and right legs and the corner point information, calculate the center point c of the left leg left and the center point c of the right leg rightCoordinates, the expression is:

[0025]

[0026] Among them, w leg is the a priori width of each supporting leg of the lifting equipment;

[0027] Adjust the target ID of each supporting leg to ID+′L′ and ID+′R′ respectively to distinguish the left and right legs of the lifting equipment.

[0028] As a further description of the present invention, the position and orientation angle correction of each supporting leg of the lifting device comprises the following steps:

[0029] Get the track information from the map information, query the track information closest to each support leg of the lifting equipment, and use the starting point r of the track as the starting point. start and the end point r end The form is stored in the track Rail;

[0030] Calculate the heading information of the track aligned , the expression is:

[0031]

[0032] Among them, (r start -r end ) x is the vector r start -r end The x-component of start -r end ) y is the vector r start -r end The y-direction component;

[0033] The vector from the center point of each support leg of the lifting equipment to the starting point of the track is The vector from the end point of the track to the starting point of the track is Calculate the center point position P of each supporting leg of the lifting equipment center Projection point position P to the track Rail aligned , the expression is:

[0034]

[0035] Calculate the center point position P of each supporting leg of the lifting equipment center Projection point position P to the track Rail aligned The distance, dist = ||P aligned -P center ||, when the distance dist of each support leg of the lifting equipment from the track is less than the threshold Dth , make corrections to the position of each supporting leg: P center =P aligned , where P aligned is the projection point position, P center The center point of the supporting leg; make corrections to the direction: heading = heading aligned , where heading aligned is the orientation information of the track, and heading is the orientation angle of the supporting legs;

[0036] Through the center point position P of each supporting leg of the lifting equipment center The position relationship with the road determines whether it is in the transfer state. When the center point position P of each supporting leg of the lifting equipment center Crossing the road edge and entering the road indicates that the lifting equipment is in a transfer state and no position and orientation correction is made.

[0037] As a further description of the present invention, the supporting structure is two supporting legs of the lifting equipment, and the tracking of the supporting structure is to obtain the speed information according to the corrected position information of each supporting leg and the timestamp of each frame of data, wherein each supporting leg sent by the cloud after correction is a detection target, denoted as detectObject, and the attribute of the detection target is [ID,P center ,t meas ,heading,l,w leg ], the tracking target list is trackList, the continuous tracking of the detection target forms a tracking target, recorded as trackObject, and the attributes of each tracking target are [ID,P center ,t meas ,velocity,heading,l,w leg ];

[0038] Where ID represents the target number, P center represents the center point of the supporting leg, t meas represents the measurement time, velocity represents the speed, heading represents the direction angle of the support leg, l represents the length of the support leg, and w leg represents the a priori width of each supporting leg of the lifting equipment;

[0039] The tracking process includes: matching associations, status updates, initializing targets, and target lifecycle management.

[0040] As a further description of the present invention, the tracking process of each supporting leg sent from the cloud after correction includes the following steps:

[0041] Matching association: Each supporting leg of each lifting device has a unique target ID. The detection target detectObject is matched with the tracking target trackObject in the tracking list according to the target ID.

[0042] If the match is successful, that is, trackObject.ID = detectObject.ID, the state of the detected target is updated. If the match is unsuccessful, a new tracking target is created and the information of the tracking target is initialized;

[0043] State update: Assign the position, orientation, and size information of the detected target to the tracked target. The expression is:

[0044]

[0045] Calculate the update time difference between the detected target and the tracked target. The expression is:

[0046] ΔT = detectObject.t meas -trackObject.t meas ;

[0047] Among them, detectObject.t meas TrackObject.t is the measurement time of the detected target. meas Measuring time for tracking target;

[0048] Based on the displacement within the above time difference, the instantaneous velocity is calculated, and the expression is:

[0049]

[0050] Among them, detectObject.P center To detect the center point of the target's supporting leg, trackObject.P center To track the center point of the target’s supporting leg, (detectObject.P center -trackObject.P center ) is the displacement within the time difference;

[0051] Using weighted average filtering, the instantaneous speed is merged with the tracking speed of the previous frame to obtain a new tracking speed, which is expressed as:

[0052] trackObject.velocity=λ*tempVelocity+(1-λ)*trackObject.velocity;

[0053] Among them, λ is the weight;

[0054] Initialize the target: Initialize the tracking target trackObject according to the position, orientation, and size information of the detected target, and set the speed of the initialized target to 0. The expression is:

[0055]

[0056] Target lifecycle management: Delete tracking targets that have not been updated for a long time. The expression is:

[0057] currentTime-trackObject.t meas >ΔT th ;

[0058] Among them, trackObject.t meas is the measurement time of the tracking target, currentTime is the current system time, ΔT th is the preset threshold. If the above conditions are met, the tracking target will be deleted.

[0059] As a further description of the present invention, the cloud sends the output of the tracking target, including the following steps:

[0060] The area of ​​the target sent from the cloud and the area of ​​the target perceived by the vehicle are calculated according to the shape of the target, and the area of ​​the overlapping area is determined based on the shape of the target sent from the cloud and the shape of the target perceived by the vehicle;

[0061] The overlap between the tracking target sent from the cloud and the tracking target perceived by the vehicle is calculated, and the expression is:

[0062]

[0063] Among them, area overlap is the area of ​​the overlapped region, area cloud The area of ​​the tracking target sent to the cloud. ego The area of ​​the target that the vehicle perceives and tracks;

[0064] Based on the overlapRate and the preset threshold Rate th The output method of sending tracking targets from the cloud is determined by comparing the overlapRate and the rate. th , then the overlap is high, and the tracking target is fused with the self-vehicle perception tracking target and then output, overlapRate<Rate th , then the overlap is low and the tracking target is directly sent to the downstream for use.

[0065] A cloud-based tracking system for port lifting equipment, the system is used to execute the above-mentioned tracking method, including a data receiving module, a data processing module, a data extraction module, a data correction module, a data tracking module, and a data output module;

[0066] The data receiving module is used to receive the lifting equipment information data sent from the cloud, each frame of data includes information of multiple lifting equipment, and the information of each lifting equipment includes measurement time, target number, center point position, and four corner point information;

[0067] The data processing module filters the information received by the data receiving module based on time verification and data validity verification to obtain valid cloud-delivered information;

[0068] The data extraction module extracts the information of the support structure from the information processed by the data processing module, and obtains the center point of the left leg, the center point of the right leg, the direction of the left leg, the direction of the right leg, the length of the left leg and the right leg, and the width of the left leg and the right leg of the support structure;

[0069] The data correction module corrects the support structure position and orientation angle of the lifting equipment obtained by the data extraction module to obtain accurate position and orientation information of the lifting equipment;

[0070] The data tracking module tracks the support structure obtained by the data correction module to obtain stable speed information;

[0071] The data output module verifies the overlap between the support structure perceived by the vehicle and the support structure sent from the cloud after tracking, fuses and outputs the tracked targets with high overlap, and directly outputs the targets with low overlap to the downstream for use.

[0072] Compared with the prior art, the technical effects of the present invention are:

[0073] The present invention provides a method and system for tracking port crane equipment sent from the cloud. The method receives crane equipment information sent from the cloud, splits the crane equipment into two supporting legs, and then corrects the position and orientation information of each supporting leg of the crane equipment in combination with the characteristics of the crane equipment during operation and high-precision map information. Then, each supporting leg of the crane equipment is tracked to obtain more accurate position and speed information of the target. Finally, the tracking target and the detection target perceived by the vehicle are checked for overlap, and the tracking target with high overlap is fused and output with the vehicle perception, and the target with low overlap is directly output to the downstream for use, thereby making full use of the crane equipment information sent from the cloud and improving the efficiency and interactive safety of port operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is a schematic diagram of the overall method flow of the present invention;

[0075] Figure 2 A top view of information sent down by the lifting equipment of the present invention;

[0076] Figure 3 It is a schematic diagram of the calculation of the projection points of the lifting equipment track of the present invention;

[0077] Figure 4 It is a schematic diagram of the transfer state of the lifting equipment of the present invention;

[0078] Figure 5 This is a view for checking the overlap of the lifting equipment of the present invention. DETAILED DESCRIPTION

[0079] The present invention is described in detail below in conjunction with the accompanying drawings:

[0080] In one embodiment of the present invention, a method for tracking port crane equipment sent from the cloud is disclosed, referring to Figure 1-5 As shown, the method receives the lifting equipment information sent from the cloud, and then extracts the relatively accurate position, speed, and direction information of the lifting equipment in combination with the characteristics of the lifting equipment during operation and the high-precision map information, thereby improving the efficiency and safety of the operation. Specifically, the method includes the following steps:

[0081] Receive information data of lifting equipment sent from the cloud, where each frame of information data sent from the cloud includes information of multiple lifting equipment, and the information of each lifting equipment includes: measurement time, target number, center point position, and four corner point information;

[0082] Filter the received lifting equipment information to obtain effective cloud-based information;

[0083] Based on the filtered information, extract the supporting structure information on both sides of the lifting equipment, including the center point, direction, length, and width;

[0084] Combined with the map information and the characteristics of the lifting equipment operating along the track, the position and orientation angle of the supporting structure are corrected to obtain the accurate position and orientation information of the lifting equipment supporting structure;

[0085] Track the corrected support structure to obtain the target stable speed;

[0086] The support structure sent from the cloud is checked for overlap with the support structure perceived by the vehicle. If the overlap is greater than the threshold, the tracked support structure sent from the cloud is fused with the support structure perceived by the vehicle and then output. If the overlap is less than the threshold, the tracked support structure sent from the cloud is directly sent to the downstream for use.

[0087] It should be noted that the above-mentioned supporting structure is usually two supporting legs of the lifting equipment, but is not limited to two supporting legs, and also includes any other form and structure for supporting the lifting equipment. Specifically, this embodiment is described with respect to the above steps, and the details are as follows:

[0088] The position and shape information of the lifting equipment sent from the cloud is used to recover the two supporting legs of the lifting equipment. Combined with the map information and the fact that lifting equipment generally operates along tracks, the position and orientation angle of each supporting leg are corrected. Each supporting leg is then tracked to obtain accurate position, speed and orientation information for downstream use.

[0089] The cloud organizes all the current lifting equipment information and sends it down. Each frame of data includes information of multiple lifting equipment. The information of a certain lifting equipment in the i-th frame of the cloud-sent information usually includes: measurement time Target ID, center point location And the four corner point information

[0090] Measuring time It refers to the time when the lifting equipment sends this frame information;

[0091] The target number ID is the identifier of the lifting equipment. This identifier is unique and is used to distinguish it from other equipment. The previous and next frame data can be queried based on the target number.

[0092] Center point location And the four corner point information It is usually given in the form of (x, y) coordinates, where x generally represents longitude and y represents latitude. It can also be other coordinate representations.

[0093] 1. Filter valid information

[0094] The lifting equipment information sent by the port has problems such as delay, stagnation, and repeated sending. It is necessary to extract valid information from the sent information. The screening methods of lifting equipment information include: time verification and data validity verification;

[0095] (1) The time verification method is to use the measurement time t of the data frame sent by the cloud at the current time i i , and the measured time t received by the previous frame i-1 i-1 Check, if the time is exactly the same, that is, t i =t i-1 , then the frame data is considered duplicate data and discarded.

[0096] (2) The data validity verification method is to obtain the latitude and longitude origin O of the port project to which the lifting equipment belongs.project , and the maximum operating range R of the lifting equipment project ; Traverse all the lifting equipment information data sent from the cloud, and calculate the distance from the observation center point and corner point of each lifting equipment to the origin O project If the calculated distance is greater than the above range R project , the observed center point or corner point is considered invalid and discarded.

[0097] In this embodiment, the observation center point of each lifting equipment is calculated. To the origin O project If the distance is greater than the above range R project , then the observation of this center point is considered invalid, that is Discard the image; similarly, judge the validity of corner point observation.

[0098] 2. Extract the support structure position and orientation of the lifting equipment based on the information sent

[0099] In this embodiment, the supporting structure is two supporting legs of the lifting equipment, and the information of the supporting structure includes: the center point of the left leg, the center point of the right leg, the direction of the left leg, the direction of the right leg, the length of the left leg and the right leg, and the width of the left leg and the right leg.

[0100] Specifically, the corner point information sent by lifting equipment such as quay cranes and rail cranes is usually the corner point of the largest box of the entire lifting equipment, that is, the box including the two supporting legs of the lifting equipment, such as Figure 2 As shown; in this embodiment, according to the four corner points of the lifting equipment [c 1 ,c 2 ,c 3 ,c 4 ] Calculate the length and direction of the two supporting legs, the left leg c 1 c 2 and right leg c 3 c 4 The lengths of are the same, both are l, and the expression is:

[0101] l=(||c 1 -c 2 ||+||c 4 -c 3 ||)*0.5;

[0102] Left leg 1 c 2 Direction h left and right leg c 3 c 4 Direction h right , the expression is:

[0103]

[0104] Among them, (c 1 -c 2 ) x is the vector c 1 -c 2 The x-component of 1 -c 2 ) y is the vector c 1 -c 2 The y-direction component of 4 -c 3 ) x is the vector c 4 -c 3 The x-component of 4 -c 3 ) y is the vector c 4 -c 3 y-direction component.

[0105] According to the prior width of the left and right legs, and the two corner points c of the left leg 1 ,c 2 Calculate its midpoint and then add the vector perpendicular to the long side Normalize and combine the prior width to get the coordinates of the center point of the left leg. The calculation of the center point of the right leg is the same. Specifically, calculate the center point c of the left leg left and the center point c of the right leg right Coordinates, the expression is:

[0106]

[0107] Among them, w leg is the a priori width of each supporting leg of the lifting equipment.

[0108] Through the above, we can get all the information of each supporting leg of the lifting equipment, left leg: center point c left , towards h left , length l, width w leg ; Right leg: center point c right , towards h right , length l, width w leg ; and adjust the target ID of each supporting leg to ID+′L′ and ID+′R′ respectively to distinguish the left leg and right leg of the lifting equipment.

[0109] 3. Correction of the position and orientation of each support leg of the lifting equipment

[0110] Since the positioning device of the lifting equipment is usually not a high-precision positioning device, the position and other information it sends usually have certain errors. However, the actual port lifting equipment moves along a fixed track, and its movement is constrained by the track. Its center point will not deviate from the track, and its direction is also along the track, so the position and direction of each support leg of the lifting equipment can be corrected according to the track information.

[0111] Specifically, the position and orientation correction of each supporting leg of the lifting equipment includes the following steps:

[0112] Get the track information from the map information, query the track information closest to each support leg of the lifting equipment, and use the starting point r of the track as the starting point. start and the end point r end The form is stored in Rail;

[0113] Calculate the heading information of the track aligned , the expression is:

[0114]

[0115] Among them, (r start -r end ) x is the vector r start -r end The x-component of start -r end ) y is the vector r start -r end The y-direction component;

[0116] The vector from the center point of each support leg of the lifting equipment to the starting point of the track is The vector from the end point of the track to the starting point of the track is Calculate the center point position P of each supporting leg of the lifting equipment center Projection point position P to the track Rail aligned , the expression is:

[0117]

[0118] like Figure 3 As shown, calculate the center point position P of each supporting leg of the lifting equipment center Projection point position P to the track Rail aligned The distance, dist = ||P aligned -P center ||, when the distance dist of each support leg of the lifting equipment from the track is less than the threshold D th , make corrections to the position of each supporting leg: P center=P aligned , where P aligned is the projection point position, P center The center point of the supporting leg; make corrections to the direction: heading = heading aligned , where heading aligned is the orientation information of the track, and heading is the orientation angle of the supporting legs;

[0119] During the actual operation of the lifting equipment, it will leave the track and enter the road for transfer movement. At this time, the support legs cannot be bound to the running track. In this embodiment, the center point position P of each support leg of the lifting equipment is center The position relationship with the road determines whether it is in the transfer state. When the center point position P of each supporting leg of the lifting equipment center Crossing the road edge and entering the road indicates that the lifting equipment is in a transfer state and no position and orientation correction is performed, such as Figure 4 shown.

[0120] 4. Tracking of each supporting leg of the lifting equipment

[0121] The information of each supporting leg of the lifting equipment after the above correction only contains relevant information about the position, direction, and shape, but no speed information. Stable and accurate speed information enables the autonomous driving vehicle to accurately predict the movement trend of the lifting equipment and avoid collision. Therefore, it is necessary to track each supporting leg and obtain the speed information based on the corrected position information and the timestamp of each frame of data. Among them, each supporting leg sent by the cloud after correction is a detection target, denoted as detectObject, and the attributes of the detection target are [ID,P center ,t meas ,heading,l,w leg ], the tracking target list is trackList, the continuous tracking of the detection target forms a tracking target, recorded as trackObject, and the attributes of each tracking target are [ID,P center ,t meas ,velocity,heading,l,w leg ]; where ID represents the target number, P center represents the center point of the supporting leg, t meas represents the measurement time, velocity represents the speed, heading represents the direction angle of the support leg, l represents the length of the support leg, and w leg Represents the prior width of each supporting leg of the lifting equipment; the tracking process includes: matching association, status update, initialization target and target lifecycle management.

[0122] Specifically, in this embodiment, the tracking process of each supporting leg sent from the cloud after correction has the following specific steps:

[0123] Matching association:

[0124] Each supporting leg of each lifting equipment has a unique target number ID. The detection target detectObject is matched with the tracking target trackObject in the tracking list according to the target number ID. If the match is successful, that is, trackObject.ID = detectObject.ID, where trackObject.ID is the target number of the tracking target and detectObject.ID is the target number of the detection target, the status of the detection target is updated, including the position, orientation, size, speed and other states. If the match is unsuccessful, a new tracking target is created and the information of the tracking target is initialized.

[0125] State update: Assign the position, orientation, and size information of the detected target to the tracked target. The expression is:

[0126]

[0127] Calculate the update time difference between the detected target and the tracked target. The expression is:

[0128] ΔT = detectObject.t meas -trackObject.t meas ;

[0129] Based on the displacement within the above time difference, the instantaneous velocity is calculated, and the expression is:

[0130]

[0131] Among them, detectObject.P center To detect the center point of the target's supporting leg, trackObject.P center To track the center point of the target’s supporting leg, (detectObject.P center -trackObject.P center ) is the displacement within the time difference; using weighted average filtering, the instantaneous speed is merged with the tracking speed of the previous frame to obtain a new tracking speed, which is expressed as:

[0132] trackObject.velocity=λ*tempVelocity+(1-λ)*trackObject.velocity;

[0133] Among them, λ is the weight.

[0134] Initialize the target: Initialize the tracking target trackObject according to the position, orientation, and size information of the detected target, and set the speed of the initialized target to 0. The expression is:

[0135]

[0136] Target lifecycle management: Delete tracking targets that have not been updated for a long time. The expression is:

[0137] currentTime-trackObject.t meas >ΔT th ;

[0138] Among them, trackObject.t meas is the measurement time of the tracking target, currentTime is the current system time, ΔT th is the preset threshold. If the above conditions are met, the tracking target will be deleted.

[0139] 5. Lifting equipment output (tracking target output)

[0140] In this embodiment, the cloud sends the output of the tracking target, including the following steps:

[0141] The area of ​​the target sent from the cloud and the area of ​​the target perceived by the vehicle are calculated according to the shape of the target, and the area of ​​the overlapping area is determined based on the shape of the target sent from the cloud and the shape of the target perceived by the vehicle, such as Figure 5 As shown;

[0142] The overlap between the tracking target sent from the cloud and the tracking target perceived by the vehicle is calculated, and the expression is:

[0143]

[0144] Among them, area overlap is the area of ​​the overlapped region, area cloud The area of ​​the tracking target sent to the cloud. ego The area of ​​the target that the vehicle perceives and tracks;

[0145] Based on the overlapRate and the preset threshold Rate th The output method of sending tracking targets from the cloud is determined by comparing the overlapRate and the rate. th , then the overlap is high, and the tracking target is fused with the self-vehicle perception tracking target and then output, overlapRate<Rate th , then the overlap is low and the tracking target is directly sent to the downstream for use.

[0146] It should be noted that the above-mentioned fusion method is any fusion method in the prior art. In this embodiment, Kalman filtering is selected to treat the tracked target sent from the cloud as an observation, input the Kalman filter, and update the position and speed of the target perceived by the vehicle.

[0147] The tracking method of the present invention is disclosed through the above content. Compared with the prior art, the present invention has the following advantages:

[0148] 1. The tracking method of the present invention separates the issued lifting equipment into two supporting legs for tracking, rather than tracking it as a whole, so that the autonomous driving vehicle can pass and operate between the two supporting legs, thereby improving the efficiency and safety of the operation;

[0149] 2. The tracking method of the present invention obtains track information based on the operating characteristics of port cranes and high-precision maps, and corrects the position and heading of the target based on the track to obtain a more accurate target position and heading. At the same time, considering the situation of the cranes being transferred to other places, no relevant correction is made for the cranes that are off the track and running on the road area;

[0150] 3. The tracking method of the present invention tracks the target, obtains the stable speed of the target, and improves the safety of the interaction of the lifting equipment;

[0151] 4. The tracking method of the present invention verifies the overlap between the tracking target and the detection target perceived by the vehicle. By setting a preset threshold, the tracking target with high overlap is merged and output with the detection target perceived by the vehicle, thereby improving the accuracy of target perception. At the same time, the tracking target with low overlap is directly output to the downstream for use to ensure safety.

[0152] In another embodiment of the present invention, a cloud-based tracking system for port crane equipment is disclosed, the system is used to execute the above-mentioned tracking method, and includes a data receiving module, a data processing module, a data extraction module, a data correction module, a data tracking module, and a data output module;

[0153] The data receiving module is used to receive the lifting equipment information data sent from the cloud, each frame of data includes information of multiple lifting equipment, and the information of each lifting equipment includes measurement time, target number, center point position, and four corner point information;

[0154] The data processing module filters the information received by the data receiving module based on time verification and data validity verification to obtain valid cloud-delivered information;

[0155] The data extraction module extracts the information of the support structure from the information processed by the data processing module, and obtains the center point of the left leg, the center point of the right leg, the direction of the left leg, the direction of the right leg, the length of the left leg and the right leg, and the width of the left leg and the right leg of the support structure;

[0156] The data correction module corrects the support structure position and orientation angle of the lifting equipment obtained by the data extraction module to obtain accurate position and orientation information of the lifting equipment;

[0157] The data tracking module tracks the support structure obtained by the data correction module to obtain stable speed information;

[0158] The data output module verifies the overlap between the support structure perceived by the vehicle and the support structure sent from the cloud after tracking, fuses and outputs the tracked targets with high overlap, and directly outputs the targets with low overlap to the downstream for use.

[0159] The above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Other modifications or equivalent substitutions made to the technical solution of the present invention by ordinary technicians in this field should be included in the scope of the claims of the present invention as long as they do not depart from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for tracking port crane equipment sent from the cloud, characterized in that: The steps include: Receive information data of lifting equipment sent from the cloud, where each frame of information data sent from the cloud includes information of multiple lifting equipment, and the information of each lifting equipment includes: measurement time, target number, center point position, and four corner point information; Filter the received lifting equipment information to obtain effective cloud-based information; Based on the filtered information, extract the supporting structure information on both sides of the lifting equipment, including the center point, direction, length, and width; Combined with the map information and the characteristics of the lifting equipment operating along the track, the position and orientation angle of the supporting structure are corrected to obtain the accurate position and orientation information of the lifting equipment supporting structure; Track the corrected support structure to obtain the target stable speed; The support structure sent down from the cloud after tracking is checked for overlap with the support structure perceived by the vehicle. If the overlap is greater than the threshold, the support structure sent down from the cloud after tracking and the support structure perceived by the vehicle are fused and output. If the overlap is less than the threshold, the support structure sent down from the cloud after tracking is directly sent to the downstream for use.

2. According to claim 1, a method for tracking port crane equipment sent from the cloud is characterized in that: The screening methods of lifting equipment information include: time verification and data validity verification; The time verification method is to receive the measurement time t of the data frame i sent by the cloud at the current moment i , and the measured time t received by the previous frame i-1 i-1 Check if t i =t i-1 , then the frame data is considered as duplicate data and discarded; the data validity verification method is to obtain the latitude and longitude origin O of the port project to which the lifting equipment belongs project , and the maximum operating range R of the lifting equipment project ; Traverse all the lifting equipment information data sent from the cloud, and calculate the distance from the observation center point and corner point of each lifting equipment to the origin O project If the calculated distance is greater than the above operating range R project , the observed center point or corner point is considered invalid and discarded.

3. The method for tracking port crane equipment sent from the cloud according to claim 1 is characterized by: The supporting structure is two supporting legs of the lifting equipment, and the information of the supporting structure includes: the center point of the left leg, the center point of the right leg, the direction of the left leg, the direction of the right leg, the length of the left leg and the right leg, and the width of the left leg and the right leg.

4. A method for tracking port crane equipment issued from the cloud according to claim 3, characterized in that: The length and orientation of the two supporting legs are calculated based on the four corner points [c1, c2, c3, c4] of the lifting equipment. The length of the left leg c1c2 and the right leg c3c4 are the same, both l, and the expression is: l=(||c1-c2||+||c4-c3||)*0.5; Left leg c1c2 direction h left and the right leg c3c4 towards h right , the expression is: Among them, (c1-c2) x is the x-component of the vector c1-c2, (c1-c2) y is the y-component of the vector c1-c2; (c4-c3) x is the x-component of vector c4-c3, (c4-c3) y is the y-direction component of the vector c4-c3; according to the prior width of the left and right legs and the corner point information, calculate the center point c of the left leg left and the center point c of the right leg right Coordinates, the expression is: Among them, w leg is the a priori width of each supporting leg of the lifting equipment; Adjust the target ID of each supporting leg to ID+′L′ and ID+′R′ respectively to distinguish the left and right legs of the lifting equipment.

5. A method for tracking port crane equipment sent from the cloud according to claim 4, characterized in that: The position and orientation angle correction of each supporting leg of the lifting equipment includes the following steps: Get the track information from the map information, query the track information closest to each support leg of the lifting equipment, and use the starting point r of the track as the starting point. start and the end point r end The form is stored in the track Rail; Calculate the heading information of the track aligned , the expression is: Among them, (r start -r end ) x is the vector r start -r end The x-component of start -r end ) y is the vector r start -r end The y-direction component; The vector from the center point of each support leg of the lifting equipment to the starting point of the track is The vector from the end point of the track to the starting point of the track is Calculate the center point position P of each supporting leg of the lifting equipment center Projection point position P to the track Rail aligned , the expression is: Calculate the center point position P of each supporting leg of the lifting equipment center Projection point position P to the track Rail aligned The distance, dist = ||P aligned -P center ||, when the distance dist of each support leg of the lifting equipment from the track is less than the threshold D th , make corrections to the position of each supporting leg: P center =P aligned , where P aligned is the projection point position, P center The center point of the supporting leg; make corrections to the direction: heading = heading aligned , where heading aligned is the orientation information of the track, and heading is the orientation angle of the supporting legs; Through the center point position P of each supporting leg of the lifting equipment center The position relationship with the road determines whether it is in the transfer state. When the center point position P of each supporting leg of the lifting equipment center Crossing the road edge and entering the road indicates that the lifting equipment is in a transfer state and no position and orientation correction is made.

6. The method for tracking port crane equipment sent from the cloud according to claim 1 is characterized by: The support structure is the two support legs of the lifting equipment. The tracking of the support structure is to obtain the speed information according to the corrected position information of each support leg and the timestamp of each frame of data. Among them, each support leg sent by the cloud after correction is a detection target, denoted as detectObject, and the attributes of the detection target are [ID,P center ,t meas ,heading,l,w leg ], the tracking target list is trackList, the continuous tracking of the detection target forms a tracking target, recorded as trackObject, and the attributes of each tracking target are [ID,P center ,t meas ,velocity,heading,l,w leg ]; Where ID represents the target number, P center represents the center point of the supporting leg, t meas represents the measurement time, velocity represents the speed, heading represents the direction angle of the support leg, l represents the length of the support leg, and w leg represents the a priori width of each supporting leg of the lifting equipment; The tracking process includes: matching associations, status updates, initializing targets, and target lifecycle management.

7. A method for tracking port crane equipment issued from the cloud according to claim 6, characterized in that: The tracking process of each supporting leg sent from the cloud after correction includes the following steps: Matching association: Each supporting leg of each lifting device has a unique target ID. The detection target detectObject is matched with the tracking target trackObject in the tracking list according to the target ID. If the match is successful, that is, trackObject.ID = detectObject.ID, where trackObject.ID is the target number of the tracked target and detectObject.ID is the target number of the detected target, then the state of the detected target is updated. If the match is unsuccessful, a new tracked target is created and the information of the tracked target is initialized. Status update: assign the position, orientation, length, and width information of the detected target to the position, orientation, length, and width of the tracked target respectively; Calculate the update time difference between the detected target and the tracked target. The expression is: ΔT=detectObject.t meas -trackObject.t meas ; Among them, detectObject.t meas The measurement time for detecting the target, trackObject.t meas Measuring time for tracking target; Based on the displacement within the above time difference, the instantaneous velocity is calculated, and the expression is: Among them, detectObject.P center To detect the center point of the target's supporting leg, trackObject.P center To track the center point of the target’s supporting leg, (detectObject.P center -trackObject.P center ) is the displacement within the time difference; Use weighted average filtering to merge the instantaneous velocity with the previous frame tracking velocity to obtain a new tracking velocity. Specifically, take the instantaneous velocity tempVelocity and the tracking velocity trackObject.velocity of the tracking target as the weight and perform weighted average to obtain a new tracking velocity, namely: λ*tempVelocity+(1-λ)*trackObject.velocity; Assigning the new tracking speed obtained by weighted average to the tracking speed of the tracking target, completing the tracking speed update of the tracking target; Initialize the target: assign the position, orientation, length, and width information of the detected target to the position, orientation, length, and width of the initialized tracking target, respectively, and set the tracking speed of the initialized tracking target to 0; Target lifecycle management: Delete tracking targets that have not been updated for a long time. The expression is: currentTime-trackObject.t meas >ΔT th ; Among them, trackObject.t meas is the measurement time of the tracking target, currentTime is the current system time, ΔT th is the preset threshold. If the above conditions are met, the tracking target will be deleted.

8. The method for tracking port crane equipment sent from the cloud according to claim 7 is characterized by: The cloud sends the output of the tracking target, including the following steps: The area of ​​the target sent from the cloud and the area of ​​the target perceived by the vehicle are calculated according to the shape of the target, and the area of ​​the overlapping area is determined based on the shape of the target sent from the cloud and the shape of the target perceived by the vehicle; The overlap between the tracking target sent from the cloud and the tracking target perceived by the vehicle is calculated, and the expression is: Among them, area overlap is the area of ​​the overlapped region, area cloud The area of ​​the tracking target sent to the cloud. ego The area of ​​the target that the vehicle perceives and tracks; Based on the overlapRate and the preset threshold Rate th The output method of sending tracking targets from the cloud is determined by comparing the overlapRate and the rate. th , then the overlap is high, and the tracking target is fused with the self-vehicle perception tracking target and then output, overlapRate<Rate th , then the overlap is low and the tracking target is directly sent to the downstream for use.

9. A tracking system for port crane equipment sent from the cloud according to the tracking method according to any one of claims 1 to 8, characterized in that: The system includes a data receiving module, a data processing module, a data extracting module, a data correcting module, a data tracking module, and a data output module; The data receiving module is used to receive the lifting equipment information data sent from the cloud, each frame of data includes information of multiple lifting equipment, and the information of each lifting equipment includes measurement time, target number, center point position, and four corner point information; The data processing module filters the information received by the data receiving module based on time verification and data validity verification to obtain valid cloud-delivered information; The data extraction module extracts the information of the support structure from the information processed by the data processing module, and obtains the center point of the left leg, the center point of the right leg, the direction of the left leg, the direction of the right leg, the length of the left leg and the right leg, and the width of the left leg and the right leg of the support structure; The data correction module corrects the support structure position and orientation angle of the lifting equipment obtained by the data extraction module to obtain accurate position and orientation information of the lifting equipment; The data tracking module tracks the support structure obtained by the data correction module to obtain stable speed information; The data output module verifies the overlap between the support structure perceived by the vehicle and the support structure sent from the cloud after tracking, fuses and outputs the tracked targets with high overlap, and directly outputs the targets with low overlap to the downstream for use.

Images