Port mobile machinery intelligent sensing system and method integrated with multiple sensors

Abstract

The invention relates to an integrated port mobile machinery intelligent sensing system and method fusing multiple sensors. A mobile machinery comprises an edge calculation unit and multiple sensors. The edge calculation unit comprises an operation subunit and a control subunit; the sensor comprises a speed sensor, a high-precision positioning sensor, a radar and a camera; the method comprises the following steps: acquiring a target position and an initial position of the flowing machine; periodically acquiring the current position, the current movement speed and the current acceleration of the flowing machine; determining a sensing distance according to the current movement speed, the current acceleration, the destination position, the initial position and the current position; and the sensing distance is sent to a control subunit of the mobile machinery, so that the mobile machinery performs auxiliary driving based on the sensing distance, the operation efficiency and safety of the port mobile machinery are improved, and automation and intelligence of the dry bulk port are promoted.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of auxiliary driving vehicle control, in particular to a port mobile machine intelligent perception system and method integrated with multiple sensors. BACKGROUND

[0002] With the vigorous development of global ports, the application of mobile machines is common in large-scale dry bulk cargo loading and unloading sites. However, there are several characteristics in the application of port mobile machines in safety management. First, single-person operation has great randomness and is difficult to control. Second, the operation environment is variable, and it is difficult to control safety. Most operation sites mainly rely on man-machine and machine-machine cooperation, and the mechanical flow is frequent, the operation environment changes rapidly, and the driver needs to have a high and difficult requirement for the surrounding safety environment. Third, external factors have a significant impact. The safety awareness of external personnel is weak, and it is difficult to cooperate with the operation. Non-standard parking, speeding, queuing, and other illegal behaviors of external vehicles often occur, which greatly affects the safety operation. Fourth, it is difficult to dynamically control the mental state of the operation personnel. Due to the poor working environment in the port area and heavy tasks, the driver is prone to fatigue, which may cause accidents.

[0003] Therefore, in order to improve the operation efficiency and safety of port mobile machines and promote the automation and intelligentization of dry bulk cargo ports, it is an urgent need to promote intelligent auxiliary driving. SUMMARY

[0004] (I) Technical problems to be solved

[0005] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present application provides a port mobile machine intelligent perception system and method integrated with multiple sensors.

[0006] (II) Technical solutions

[0007] In order to achieve the above-mentioned purposes, the main technical solutions adopted by the present application include:

[0008] In a first aspect, the present application provides a port mobile machine intelligent perception method integrated with multiple sensors. The mobile machine includes an edge computing unit and multiple sensors.

[0009] The edge computing unit includes a computing subunit and a control subunit.

[0010] The sensors include a speed sensor, a high-precision positioning sensor, a radar, and a camera.

[0011] The method includes:

[0012] Obtaining the destination position and the starting position of the mobile machine.

[0013] Periodically acquiring a current position, a current motion speed, and a current acceleration of the flow machine, wherein the current position is determined based on a high-precision positioning sensor of the flow machine, and the current motion speed and the current acceleration are determined based on a speed sensor of the flow machine;

[0014] Determining a perception distance according to the current motion speed, the current acceleration, the target position, the start position, and the current position;

[0015] Sending the perception distance to a control subunit of the flow machine, so that an operation subunit of the flow machine performs obstacle perception within the perception distance according to data collected by a radar and a camera, and the control subunit of the flow machine controls motion of the flow machine according to a result of the obstacle perception.

[0016] Optionally, determining the perception distance according to the current motion speed, the current acceleration, the target position, the start position, and the current position comprises:

[0017] If a distance between the current position and a comparison position is less than a comparison distance, and the comparison position is not the target position, then the perception distance is determined.

[0018] If a speed after running forward for a first time length under a condition that an initial speed is the current motion speed and an acceleration is the current acceleration is not higher than a highest speed limit, then the comparison distance is a distance that the flow machine runs forward for the first time length in a first running mode, otherwise, the comparison distance is a distance that the flow machine runs forward for the first time length in a second running mode.

[0019] The first running mode is an acceleration running mode with the initial speed being the current motion speed and the acceleration being the current acceleration, and the second running mode is a variable speed running mode with the initial speed being the current motion speed and the acceleration being the current acceleration running until the highest speed limit, and then running at the highest speed limit at a constant speed, and a sum of a length of the acceleration running and a length of the constant speed running is the first time length.

[0020] An initial position of the comparison position is the start position, and the comparison position is updated after a new perception distance is determined.

[0021] Optionally, if the candidate position does not reach the target position, then the comparison position is updated to the candidate position, otherwise, the comparison position is updated to the target position.

[0022] The candidate position is a position of the flow machine that perceives the perception distance forward from the current position.

[0023] Optionally, determining the perception distance comprises:

[0024] Determining a type of the flow machine, a start work time, an end work time, a work amount, and a priority weight of a work object.

[0025] The perception priority is determined based on the start time, end time, workload, and priority weight of the task object;

[0026] Determine the path nodes based on the destination location and the current location;

[0027] The sensing distance is determined based on the type of mobile machinery, sensing priority, and path nodes.

[0028] Optionally, the perception priority is determined based on the start time, end time, workload, and priority weight of the task object, including:

[0029] The adjustment value is determined based on the relationship between the workload and the start and end times of the work.

[0030] The perception priority is determined based on the adjustment value, adjustment ratio, and priority weight of the task object.

[0031] Optionally, the perception priority is determined based on the start time, end time, workload, and priority weight of the task object, including:

[0032] The adjustment value is determined based on the relationship between the current time and the start and end times of the work, as well as the workload;

[0033] The perception priority is determined based on the adjustment value, adjustment ratio, and priority weight of the task object.

[0034] Optionally, the adjustment ratio is determined based on the priority weight of the work object and the relationship between the current time and the start time and end time of the work.

[0035] Optionally, the sensing distance is determined based on the type of mobile machinery, sensing priority, and path nodes, including:

[0036] The initial distance is determined based on the radar detection range of the mobile machinery and the depth of field of the camera;

[0037] Among the path nodes, identify the target path node located within the initial distance the mobile machinery travels forward from its current position;

[0038] The impact value of each target path node is determined based on the type of mobile machinery and the perception priority;

[0039] The specific adjustment value is determined based on the impact value of each target path node;

[0040] The sensing distance is determined based on the initial distance and specific adjustment values.

[0041] Optionally, the impact value of each target path node is determined based on the type of mobile machinery and the sensing priority, including:

[0042] For any target path node, determine the total number of other mobile machines currently located at the target path node, the dwell time of each other mobile machine at the target path node, and the type of each other mobile machine; determine the impact value of any target path node based on the total number of other mobile machines currently located at the target path node, the dwell time of each other mobile machine at the target path node, and the type of each other mobile machine.

[0043] Secondly, embodiments of the present invention provide an integrated multi-sensor intelligent sensing system for port mobile machinery, the system comprising: at least one mobile machine and a central control platform;

[0044] The mobile machinery includes an edge computing unit and multiple sensors;

[0045] The edge computing unit includes: a computation subunit and a control subunit;

[0046] Sensors include: speed sensors, high-precision positioning sensors, radar, and cameras;

[0047] The central control platform is used to execute the method described in the first aspect above to intelligently sense each mobile machine.

[0048] (III) Beneficial Effects

[0049] The beneficial effects of this invention are as follows: This invention relates to an integrated intelligent sensing system and method for port mobile machinery, incorporating multiple sensors. The mobile machinery includes an edge computing unit and multiple sensors. The edge computing unit includes a computation subunit and a control subunit. The sensors include a speed sensor, a high-precision positioning sensor, radar, and a camera. The method includes: acquiring the destination position and starting position of the mobile machinery; periodically acquiring the current position, current speed, and current acceleration of the mobile machinery; determining the sensing distance based on the current speed, current acceleration, destination position, starting position, and current position; sending the sensing distance to the control subunit of the mobile machinery, so that the computation subunit of the mobile machinery can perceive obstacles within the sensing distance based on the data collected by the radar and camera; and the control subunit of the mobile machinery controls the movement of the mobile machinery based on the obstacle perception results. This invention's method, based on an edge computing unit and multiple sensors, enables assisted driving of mobile machinery, improves the operational efficiency and safety of port mobile machinery, and promotes the automation and intelligence of dry bulk cargo ports. Attached Figure Description

[0050] Figure 1 A schematic diagram of a mobile machine provided by the present invention;

[0051] Figure 2 A schematic diagram of another type of mobile machinery provided by the present invention;

[0052] Figure 3 A flowchart illustrating the integrated multi-sensor intelligent sensing method for port mobile machinery provided by this invention. Detailed Implementation

[0053] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] With the booming development of ports worldwide, the application of mobile machinery is common in large-scale dry bulk cargo loading and unloading operations. However, the application of mobile machinery in ports generally presents several characteristics in terms of safety management: First, individual operations are highly arbitrary, making safety control difficult; second, the working environment is highly variable, making safety pre-control difficult. Most work sites rely on human-machine or machine-machine coordination, with frequent machinery movement and rapidly changing working environments, requiring drivers to have a high level of understanding of the surrounding safety environment, which is challenging, as safety hazards are ever-present and constantly changing; third, it is significantly affected by external factors. External personnel entering the port have a weak safety awareness, making cooperation difficult, and violations such as improper parking, speeding, cutting in line, and racing by outside vehicles occur frequently, significantly impacting safe operations; fourth, it is difficult to dynamically monitor the mental state of operators. Due to the harsh working environment and heavy workload in port areas, drivers are prone to fatigue, leading to accidents.

[0055] Therefore, in order to improve the efficiency and safety of port mobile machinery operations, promote the automation and intelligence of dry bulk cargo ports, and popularize intelligent assisted driving, it has become an urgent need.

[0056] Therefore, this invention relates to an integrated intelligent sensing system and method for port mobile machinery, incorporating multiple sensors. The mobile machinery includes an edge computing unit and multiple sensors. The edge computing unit includes a computation subunit and a control subunit. The sensors include a speed sensor, a high-precision positioning sensor, radar, and a camera. The method includes: acquiring the destination position and starting position of the mobile machinery; periodically acquiring the current position, current speed, and current acceleration of the mobile machinery; determining the sensing distance based on the current speed, current acceleration, destination position, starting position, and current position; sending the sensing distance to the control subunit of the mobile machinery, so that the computation subunit of the mobile machinery can perceive obstacles within the sensing distance based on the data collected by the radar and camera; and controlling the movement of the mobile machinery based on the obstacle perception results. This invention's method, based on an edge computing unit and multiple sensors, enables assisted driving of mobile machinery, improves the operational efficiency and safety of port mobile machinery, and promotes the automation and intelligence of dry bulk cargo ports.

[0057] This embodiment provides an integrated multi-sensor intelligent perception method for port mobile machinery, which can be executed by a central control platform in an integrated multi-sensor intelligent perception system for port mobile machinery. The central control platform uses this integrated multi-sensor intelligent perception method to intelligently perceive each piece of mobile machinery, enabling each piece of machinery to perform assisted driving based on the intelligent perception from the central control platform.

[0058] This integrated intelligent sensing system for port mobile machinery, which incorporates multiple sensors, includes at least one mobile machine and a central control platform.

[0059] Each mobile machine includes an edge computing unit and multiple sensors.

[0060] The edge computing unit includes a processing subunit and a control subunit. The edge computing unit also features a protocol conversion gateway function.

[0061] Sensors include: speed sensors, high-precision positioning sensors, radar, and cameras.

[0062] In practical implementation, the speed sensor and high-precision positioning sensor can be the same device, such as an Inertial Measurement Unit (IMU). The IMU achieves autonomous navigation and motion state monitoring by measuring the acceleration and angular velocity of the moving vehicle. It consists of sensors such as gyroscopes and accelerometers, capable of measuring linear acceleration and rotational angular velocity in three directions, and obtaining the vehicle's attitude, velocity, and displacement information through analysis. The IMU can provide real-time multi-dimensional information such as velocity, position, and attitude, ensuring the stability and safety of mobile machinery in unmanned operation. Figure 1 An exemplary mobile machine is shown, in which radar and cameras are not present. Figure 1 This is reflected in the text.

[0063] In addition, there can be multiple radars and cameras. For example, radars can be multiple lidars, multiple millimeter-wave radars, multiple ultrasonic radars, etc., and cameras can be multiple vehicle-mounted cameras. Examples include forward-facing lidars, long-range millimeter-wave radars and cameras, rearward-facing long-range millimeter-wave radars and cameras, side-facing lidars and short-range millimeter-wave radars, and lidars on the roof of the vehicle. Figure 2 An exemplary flowing machine is shown, in which the speed sensor and edge computing unit are not located within the machine. Figure 2 This is reflected in the text.

[0064] Figure 2The various speed sensors, radars, and cameras in the system can be connected via vehicle Ethernet and connected to high-precision positioning sensors via serial ports. After acquiring image and point cloud data, the edge computing unit can complete obstacle perception through sensor fusion detection network, target tracking, and corresponding algorithms, and complete driving scene modeling through scene segmentation network.

[0065] By employing speed sensors, high-precision positioning sensors, radar, and cameras, one or more combinations of high-precision positioning from the Global Positioning System (GPS), laser positioning, and visual positioning are achieved to acquire high-precision positioning signals; combined with high-precision maps, a global situational awareness model is generated; and transmitted to the edge computing unit via the vehicle Ethernet through the central gateway.

[0066] By collaborating with multiple sensors such as LiDAR, cameras, millimeter-wave radar, and ultrasonic radar, it is possible to cover all-around perception needs from long distance to short distance.

[0067] In addition, each mobile machine also includes an external information interaction domain controller.

[0068] The external information interaction domain controller can connect to the port operation system via a dedicated wireless network, receive operation task information, and transmit information such as the status, trajectory, and attitude of the mobile machinery to the central control platform. This enables the central control platform to monitor the mobile machinery in real time. Communication uses a dedicated network and encrypted protocol to ensure communication security and prevent external interference and intrusion. It can also directly exchange information with the onboard information platform of the port's mobile machinery through a wireless communication network to obtain the machinery's working status, complete operations with the mobile machinery, and avoid collisions in complex operating environments. It can also provide auxiliary environmental perception and positioning information data for intelligent assisted driving functions.

[0069] The central control platform is used to execute the integrated multi-sensor intelligent sensing method for port mobile machinery provided in this embodiment, and to intelligently sense each piece of mobile machinery.

[0070] For example, any mobile machinery can be intelligently sensed using the integrated multi-sensor intelligent sensing method for port mobile machinery provided in this embodiment.

[0071] See Figure 3 The implementation process of the integrated multi-sensor intelligent sensing method for port mobile machinery provided in this embodiment is as follows:

[0072] 301, Obtain the destination and starting positions of the mobile machinery.

[0073] The destination position and the starting position here are related to the direction of movement of the mobile machinery. The starting point of the direction of movement is the starting position, and the ending point is the destination position.

[0074] For example, if a mobile machine is used to transport goods from unloading area 1 in a port to warehouse 1, then during the process of the mobile machine traveling from warehouse 1 to unloading area 1, the location of warehouse 1 is the starting position, and the location of unloading area 1 is the destination position. Similarly, during the process of the mobile machine traveling from unloading area 1 back to warehouse 1, the location of unloading area 1 is the starting position, and warehouse 1 is the destination position.

[0075] The destination and starting positions of the mobile machinery can be determined in advance based on its operating trajectory, which is then used to determine the starting and ending positions. In step 301, the corresponding destination and starting positions are obtained based on the direction of movement of the mobile machinery.

[0076] The work content is pre-reported by the mobile machinery. For example, when the mobile machinery enters the relevant sensing area of ​​the port, it reports the type of the mobile machinery, relevant parameters of the mobile machinery (such as the detection range of the radar and the depth of field of the camera), work content, work object, workload C0, end time T1, start time T0 and other relevant information, so as to ensure the safety of its operation through the integrated multi-sensor intelligent sensing method for port mobile machinery provided in this embodiment.

[0077] Furthermore, in practical implementation, the distance between the starting position and the destination position can be long or short, as long as it meets the most basic operational requirement. For example, in addition to the distance between the warehouse and the unloading area mentioned above, it could also be the distance between one unloading point in the unloading area and another unloading point. This embodiment does not limit the distance between the destination position and the starting position, as long as the destination position and the starting position reflect the single-operation requirements of the mobile machinery.

[0078] If mobile machinery involves multiple operational needs, such as traveling from the east gate of the port to an unloading point in the unloading area, then from that unloading point to another unloading point, and finally from that unloading point to the warehouse, then these needs can be divided into multiple operational requirements: traveling from the east gate of the port to an unloading point in the unloading area, traveling from that unloading point to another unloading point, and traveling from that unloading point to the warehouse. Each operational requirement is controlled using the integrated multi-sensor intelligent sensing method for port mobile machinery provided in this embodiment. The only difference is that the starting and destination positions of this integrated multi-sensor intelligent sensing method for port mobile machinery differ each time it is executed, corresponding to the operational requirement.

[0079] Among them, the operational requirement is the need to travel normally from one location to another, and the need to stop midway without any working purpose.

[0080] For example, in step 302, the central control platform acquires the destination position L of any mobile machinery (such as mobile machinery M). E and starting position L B .

[0081] 302, periodically acquire the current position, current speed, and current acceleration of the mobile machinery.

[0082] The current position is determined by a high-precision positioning sensor of the mobile machinery, while the current speed and acceleration are determined by a speed sensor of the mobile machinery.

[0083] After acquiring their positions, the high-precision positioning sensors of each piece of mobile machinery send them to the central control platform. Similarly, the speed sensors of each piece of mobile machinery acquire their speed and acceleration, and send them to the central control platform. In this way, the central control platform can periodically acquire the current position, current speed, and current acceleration of the mobile machinery. The period length is a relatively short length, such as acquiring the current position, current speed, and current acceleration of the mobile machinery every 1 minute.

[0084] For example, in step 302, the central control platform acquires the current position L of any mobile machinery (such as mobile machinery M) in the t-th cycle. t Current speed v t Current acceleration a t .

[0085] 303. Determine the sensing distance based on the current speed, current acceleration, target position, starting position, and current position.

[0086] In step 303, the central control platform determines the current speed v of any mobile machinery (such as mobile machinery M). t Current acceleration a t Target location L E Starting position L B and current position L t Determine the sensing distance D.

[0087] The implementation process of step 303 is as follows:

[0088] 1. If the current position is L t Comparison position L C The distance between them is less than the alignment distance, and the alignment position L C Non-target location L E Then the perception distance D is determined.

[0089] Where the initial velocity is the current velocity v t The acceleration is the current acceleration a. tIf the speed after the first preset time Δt1 of forward movement under the condition is not higher than the maximum speed limit, then the comparison distance is the distance of the mobile machinery moving forward for the first time Δt1 using the first operating mode; otherwise, the comparison distance is the distance of the mobile machinery moving forward for the first time Δt1 using the second operating mode.

[0090] The first operating mode is where the initial speed is the current speed v. t The acceleration is the current acceleration a. t The first mode is accelerated operation. The second mode is when the initial speed is the current speed v. t The acceleration is the current acceleration a. t The system operates at its maximum speed limit, then maintains a constant speed at that limit, with the sum of the acceleration duration and the constant speed duration equal to the first duration Δt1.

[0091] Comparison position L C The initial position is the starting position L. B Compare position L C It is updated after the new sensing distance D is determined.

[0092] The update process is as follows: if the candidate position does not reach the target position L E Then update the comparison position L. C If a candidate position is selected, the alignment position L is updated. C For the target location L E .

[0093] Among them, the candidate position is the current position L of the mobile machinery. t Perceive the position at a distance D in front.

[0094] The following example uses any mobile machine (such as mobile machine M) starting from position L. B Drive to destination L E Taking this as an example, the implementation process of step 303 will be explained in detail.

[0095] 1) Compare position L C The initial position is the starting position L. B (i.e. L) C =L B );

[0096] Then compare position L C It will be updated after the new sensing distance D is determined. That is, at the comparison position L C Assign an initial value (i.e., the initial position L) B After that, as any mobile machine (such as mobile machine M) moves, based on the current position L... t Comparison position L CThe relationship between the distance between and the comparison distance is used to repeatedly determine the perceptual distance D (i.e., whenever the current position L is reached). t Comparison position L C When the distance between them is less than the comparison distance, the sensing distance D will be redefined until any mobile machine (such as mobile machine M) reaches the target position L. E Each time the perception distance D is redefined, the comparison position L is also updated. C Therefore, based on the current position L t With the new alignment position L C The relationship between the distance between them and the comparison distance is repeatedly used to determine the sensing distance D. This process is repeated to achieve the goal of starting from any mobile machine (such as mobile machine M) and moving (i.e., from the starting position L). B Starting from this point, a sensing distance D is obtained, which ensures the safe operation of any mobile machine (such as mobile machine M) within this sensing distance D. Simultaneously, a new comparison position L is obtained based on the sensing distance D. C When any mobile machine (such as mobile machine M) travels to a distance L from the new comparison position C When the distance is less than the comparison distance, a new sensing distance D is obtained again. This ensures the safe operation of any mobile machinery (such as mobile machinery M) within the new sensing distance D. At the same time, a new comparison position L is obtained based on the new sensing distance D. C ... by continuously updating the perceived distance D and the comparison position L C Ensure that any mobile machinery (such as mobile machinery M) safely travels to its destination location L. E .

[0097] By cyclically updating the sensing distance and comparing the position, the sensor's sensing distance limit is met, ensuring the safe operation of the mobile machinery within the sensor's effective sensing range, while also avoiding resource waste caused by frequent sensing.

[0098] The ultimate goal of the integrated multi-sensor intelligent sensing method for port mobile machinery provided in this embodiment is to ensure that any mobile machinery (such as mobile machinery M) safely operates to its destination position L. E Therefore, update the alignment position L. C At that time, it will reach the destination position L after traveling a sensing distance D in any mobile machine (such as mobile machine M). E Or beyond the target position L E Stop updating under these circumstances; further updates are pointless at this point, as the mobile machinery has reached its destination L. E That's it. Based on this analysis, a candidate position can be obtained (i.e., the candidate position is the current position L of the flowing machinery). tThe candidate position is determined by sensing a distance D forward (where "forward" refers to the direction of movement towards any mobile machine (e.g., mobile machine M)). This candidate position describes the position after sensing a distance D from any mobile machine (e.g., mobile machine M). If this position reaches the target position L... E Or beyond the target position L E If so, then stop updating.

[0099] The update process is as follows: if the candidate position does not reach the target position L E Then update the comparison position L. C If it is a candidate position, then no update is needed; simply set the alignment position L directly. C The target location L has been determined. E That is, the last alignment position L. C The target location L must be E .

[0100] 2) Determine the current position L t Comparison position L C The distance between them (for ease of description, this distance is denoted as distance1);

[0101] 3) Determine if distance1 is less than the comparison distance.

[0102] Among them, based on the pre-set roads in the port and the current location L t Determine the maximum speed limit on the road where any mobile machinery (such as mobile machinery M) is currently located, such as 15 km / h.

[0103] Then, the initial velocity is determined to be v. t The acceleration is a t Under the condition of moving forward (i.e., moving towards L) E The velocity after a time interval Δt1 (for ease of description, this velocity is denoted as speed1), i.e., speed1 = v t +a t ×Δt1.

[0104] Here, Δt1 is the minimum time before sensing is not required, determined based on the sensing accuracy; the higher the accuracy, the smaller the value of Δt1. For example, Δt1 is 5 minutes, meaning that any mobile machinery (such as mobile machinery M) must undergo a new sensing operation within 5 minutes. If sensing is not performed, the currently sensed data cannot guarantee its safe operation after 5 minutes. This reflects the duration for which sensing needs to be repeated, as required by the integrated multi-sensor intelligent sensing method for port mobile machinery provided in this embodiment.

[0105] speed1 reflects the speed of any mobile machine (such as mobile machine M) after 5 minutes.

[0106] If the speed does not exceed the maximum speed limit, it means that the speed of any mobile machine (such as mobile machine M) is within the normal range after 5 minutes. Therefore, the comparison distance is the distance between the mobile machine (such as mobile machine M) and the first operating mode (i.e., the initial speed is the current speed v). t The acceleration is the current acceleration a. t The distance traveled forward for the first duration Δt1 (for ease of description, this distance is denoted as distance2), i.e., In other words, if the speed of any mobile machine (such as mobile machine M) meets the standard after 5 minutes, then the distance traveled by any mobile machine (such as mobile machine M) within 5 minutes is determined as the comparison distance.

[0107] If the speed exceeds the maximum speed limit, it indicates that any mobile machine (such as mobile machine M) has exceeded the speed limit after 5 minutes. The comparison distance is the distance between the mobile machine (such as mobile machine M) and the operating mode (i.e., the initial speed is the current speed v). t The acceleration is the current acceleration a. t The machine operates until it reaches its maximum speed limit, then maintains that maximum speed at a constant speed, with the sum of the acceleration duration and the constant speed duration being the first duration Δt1. It then moves forward a distance of the first duration Δt1 (denoted as distance2 for ease of description). If the initial speed of any mobile machine (such as mobile machine M) is the current speed v... t The acceleration is the current acceleration a. t If the time taken to run until the maximum speed limit is Δt2, then In other words, if any mobile machine (such as mobile machine M) exceeds the speed limit after 5 minutes, it will continue to travel at its maximum speed after reaching the maximum speed limit. It is the distance traveled by any mobile machinery (such as mobile machinery M) when it reaches the maximum speed limit. Maximum speed limit × (Δt2-Δt1) is the distance traveled at the maximum speed limit. The distance traveled in this way within 5 minutes is determined as the comparison distance.

[0108] Therefore, the comparison distance actually reflects the distance that any mobile machinery (such as mobile machinery M) travels within Δt1 time under the condition of complying with traffic rules. This distance is also the maximum distance that the integrated multi-sensor port mobile machinery intelligent perception method provided in this embodiment needs to perceive again. That is, it must be perceived again within this distance, otherwise it is no longer guaranteed that any mobile machinery (such as mobile machinery M) is traveling safely.

[0109] If the current position is L t Comparison position LC If the distance between them, distance1, is less than the comparison distance, it means that any mobile machine (such as mobile machine M) needs to be sensed again to ensure the safe operation of any mobile machine (such as mobile machine M). If the current position L t Comparison position L C If the distance 'distance1' between them is not less than the comparison distance, it means that any mobile machine (such as mobile machine M) can operate safely according to the current parameters and there is no need to perform sensing again. By comparing the distance 'distance1', the sensing frequency can be effectively controlled, avoiding unnecessary frequent sensing that would waste computing resources, and also avoiding excessively long periods without sensing that would increase the safety risk of any mobile machine (such as mobile machine M).

[0110] 4) If the current position L t Comparison position L C If the distance between them (distance1) is less than the comparison distance, the comparison position (L) will also be determined. C Is this the destination location L? E .

[0111] If comparing position L C For the target location L E This means that any mobile machine (such as mobile machine M) can safely reach its destination L by operating at the current sensing distance D. E There is no need to update the perceived distance D.

[0112] If comparing position L C Non-target location L E This means that any mobile machine (such as mobile machine M) cannot reach its destination L even when operating safely at its current sensing distance D. E The sensing distance D needs to be updated again.

[0113] 5) If the current position L t Comparison position L C The distance between them is less than the alignment distance, and the alignment position L C Non-target location L E This means that any mobile machine (such as mobile machine M) cannot reach its destination L even when operating safely at its current sensing distance D. E Any mobile machine (such as mobile machine M) needs to be sensed again to ensure the safe operation of any mobile machine (such as mobile machine M), so the sensing distance D will be determined.

[0114] If the distance between the current position and the comparison position is not less than the comparison distance, it means that any mobile machinery (such as mobile machinery M) can operate safely according to the current parameters, and there is no need to sense again. Therefore, the sensing distance is uncertain.

[0115] If the compared position is the target position, it means that any mobile machine (such as mobile machine M) can safely reach the target position L by operating at the current sensing distance D. E Since there is no need to update the perception distance D, the perception distance is uncertain.

[0116] Furthermore, the process of determining the perceived distance D is as follows:

[0117] 401. Determine the type of mobile machinery, start time, end time, workload, and priority of the work objects.

[0118] The type of mobile machinery, workload C0, end time T1, and start time T0 are all reported in advance for the work objects of any mobile machinery (such as mobile machinery M).

[0119] Port safety personnel pre-determine the right-of-way priority weights for each work object based on the work sequence. This priority indicates which work object has the right-of-way if multiple work objects are simultaneously located at the same position (the work object with the higher priority weight has the right-of-way, and other work objects must give way). This right-of-way priority weight is the priority weight of the work object. In other words, when any mobile machinery (such as mobile machinery M) reports a work object, the priority weight of that work object for that mobile machinery (such as mobile machinery M) has already been determined based on the pre-determined priority weights of each work object.

[0120] The priority weight of any work object pre-determined by port safety personnel can be a single value, meaning that the work object corresponds to the same priority weight, or it can be multiple values ​​(which can be consecutive or discrete). If there are multiple values, other distinguishing conditions can be set, such as different values ​​corresponding to different workloads and / or different values ​​corresponding to different work durations. This embodiment does not limit the distinguishing conditions or the corresponding relationships.

[0121] If the priority weight of any work object predetermined by the port-related safety personnel is a single value, then the priority weight of the work object in step 401 is the priority weight corresponding to the work object involved by any mobile machinery (such as mobile machinery M); if the priority weight of any work object predetermined by the port-related safety personnel is multiple values, then the priority weight of the work object in step 401 is determined from multiple values ​​based on the actual situation of the work object involved by any mobile machinery (such as mobile machinery M) (where the selection dimension is the same as the distinguishing dimension of different values).

[0122] 402. Determine the perception priority based on the start time, end time, workload, and priority weight of the task object.

[0123] Step 402 determines the adjustment value Adj based on the relationship between the current time t, the start time T0, the end time T1, and the workload C0. The perception priority pre is then determined based on the adjustment value Adj, the adjustment ratio α, and the priority weight w0 of the task object. t .

[0124] For adjusting values, it can be done through... If the adjustment value is represented by Adj, then

[0125] Where C0 is the workload, C t T1 represents the remaining workload, T0 represents the start time of the workload, and t represents the current time.

[0126] Current remaining workload C t This is obtained through real-time assessment of the remaining workload and work objects of any mobile machinery (such as mobile machinery M). For example, if the workload of any mobile machinery (such as mobile machinery M) is all containers on terminal A, and the work objects are containers, then the number of containers on terminal A is constantly monitored by pre-installed image acquisition equipment (such as surveillance cameras) in the port. Based on this number, the current remaining workload C can be obtained. t .

[0127] exist middle, Let T1-t be the amount of work per unit time during the planning stage, and T1-t be the remaining working time of any mobile machine (such as mobile machine M). Therefore, The remaining workload that should be planned for any mobile machine (such as mobile machine M). t If the remaining workload is [the current workload], then The ratio of the actual remaining workload to the planned remaining workload is denoted as Adj. A value of 1 indicates that the work is currently being completed as planned, a value greater than 1 indicates that the current progress is lagging behind, and a value less than 1 indicates that the progress is ahead of schedule. In this way, Adj actually represents the work progress of any mobile machinery (such as mobile machinery M), that is, the perceived priority is adjusted based on the work progress.

[0128] For adjusting values, you can also use... If the adjustment value is represented by Adj, then

[0129] exist In this context, T1-t represents the remaining operating time of any mobile machine (e.g., mobile machine M), and T1-T0 represents the planned operating time of any mobile machine (e.g., mobile machine M). Therefore, This represents the proportion of remaining operating time to planned operating time for any mobile machine (such as mobile machine M). The smaller (i.e.) The larger the value, the more urgent the task of any mobile machine (e.g., mobile machine M) is, indicating that it is about to reach the end time of its operation. The duration of operation reflects the urgency of the operation required for any mobile machinery (such as mobile machinery M). The percentage of remaining work to total work indicates the amount of work remaining and the greater the need for immediate action. This percentage reflects the urgency of work required for any mobile machine (e.g., mobile machine M) from a workload perspective. In this way, Adj actually characterizes the urgency of work for any mobile machine (e.g., mobile machine M), meaning that perceived priority is adjusted based on the urgency of work.

[0130] When determining the perception priority based on the adjustment value Adj, the adjustment ratio α, and the priority weight of the task object, the formula pre can be used. t =w0+α×softmax{Adj} is implemented.

[0131] Where w0 is the priority weight of the task object, α is the adjustment ratio, and softmax{·} is the normalization function.

[0132] The adjustment ratio is determined based on the priority weight of the work object and the relationship between the current time and the start and end times of the work. In other words, the adjustment ratio is determined based on the priority weight of the work object and the already completed work time of any mobile machinery (such as mobile machinery M).

[0133] Such as adjusting the ratio

[0134] Where β is the time ratio,

[0135] Among them, w max Let w be the highest priority weight that the task object can possibly obtain. If the priority weight of any task object is predetermined by the port's relevant safety personnel to be a single value, then w... max w represents the maximum priority weight of all work objects; if the port safety personnel have predetermined multiple priority weights for any work object, then w... max It is the maximum value among all values ​​of the work objects involved in any mobile machinery (such as mobile machinery M).

[0136] w max -w0 represents the maximum adjustment space of the work object involved in any mobile machinery (such as mobile machinery M). It represents the proportion of time that any mobile machine (such as mobile machine M) has been in operation; the longer the operation time, the higher the percentage of time that has been in operation. The larger the value, the shorter the remaining working time. A shorter remaining time allows for a slight increase in the adjustment ratio, enabling any mobile machine (such as mobile machine M) to complete its work within that short time. The larger the value, the smaller the β value and the larger the α value.

[0137] w0 serves as the base priority. Based on this, the priority is dynamically adjusted according to the progress of any mobile machinery (such as mobile machinery M) or the urgency of the task, to obtain the final perceived priority. The more delayed the task is, the more the priority needs to be increased; the greater the urgency of the task, the more the priority needs to be increased.

[0138] Perception Priority t The method can be flexibly determined based on the operating conditions of any mobile machine (such as mobile machine M), which can ensure that any mobile machine (such as mobile machine M) can complete the operation safely and on time.

[0139] 403, based on the destination position L2 and the current position L t Identify the pathway nodes.

[0140] Multiple nodes are pre-set within the sensing area of ​​mobile machinery at the port. Each node corresponds to an important attribute. For example, each warehouse is designated as the first node, each mobile machinery docking station (gas station, charging station, rest station, etc.) is also designated as a node, each unloading area is also designated as a node, and each area prone to danger (such as accident-prone areas) is also designated as a node. The specific areas designated as nodes can be determined by relevant personnel based on specific circumstances, and this embodiment and subsequent embodiments do not limit this.

[0141] In 403, the existing navigation scheme will be used to determine the destination location L2 and the current location L. t The driving path is determined, and all nodes on the driving path are identified as pathway nodes.

[0142] 404. Determine the sensing distance D based on the type of mobile machinery, sensing priority, and path nodes.

[0143] The implementation process of step 404 is as follows:

[0144] 1. Determine the initial distance based on the radar detection range of the mobile machinery and the depth of field of the camera.

[0145] The radar detection range and camera depth of field of the mobile machinery are pre-reported by any mobile machinery (e.g., mobile machinery M), and the reported parameters can be directly obtained here. For example, if the radar detection range is related to the radar model, its detection range is determined after the radar is manufactured and / or installed on the mobile machinery and adjusted; for example, the detection range d1 is 200 meters. Similarly, the camera depth of field is related to the camera model; its depth of field is determined after the camera is manufactured and / or installed on the mobile machinery and adjusted; for example, the depth of field d2 is 2 meters.

[0146] After determining the radar detection range d1 and the camera depth of field d2 of the mobile machinery, the initial distance d3 = min{d1,d2} can be determined.

[0147] Where min{·} is the function for finding the minimum value.

[0148] In other words, the minimum value between the radar's detection range d1 and the camera's depth of field d2 is used as the initial distance, which ensures that both the radar and the camera can detect normally and with high precision within this distance.

[0149] 2. Among the path nodes, identify the target path node located within the initial distance the mobile machinery travels forward from its current position.

[0150] The path node obtained in 403 is from the current position L. t All nodes between the destination L2 and the destination L2, but not the current location L. t The region between the destination location L2 and the destination location L2 (it should be noted that the region in this embodiment may be a two-dimensional region, i.e., the current location L) t The area of ​​the travel path from the destination L2 can also be a three-dimensional area, i.e., the current position L. t Based on the travel path between L2 and the destination location, extend a certain height into the air (this height is the same as the maximum operating height of all mobile machinery, such as the operating height of a gantry crane, etc.). This is the area for intelligent sensing of the port mobile machinery intelligent sensing method provided in this embodiment, which integrates multiple sensors. This is mainly affected by the radar detection range and the camera depth of field. If the current location L... t The area between the destination L2 and the target location L2 exceeds the area corresponding to the radar detection range and the camera depth of field. Therefore, at this point, the path node will be determined to be located at the current position L of the mobile machinery. t The target path nodes within a forward distance of d3 are the path nodes within the largest area that can be accurately detected by radar and cameras.

[0151] 3. Determine the impact value of each target path node based on the type of mobile machinery and the perception priority.

[0152] For any target path node (such as target path node i), its influence value can be determined by the following steps.

[0153] 1) Determine the total number of other mobile machines currently located at any target path node, the dwell time of each other mobile machine at any target path node, and the type of each other mobile machine.

[0154] All mobile machinery entering the port's intelligent sensing area will be intelligently sensed by the integrated multi-sensor port mobile machinery intelligent sensing method provided in this embodiment. Therefore, the central control platform executing the integrated multi-sensor port mobile machinery intelligent sensing method provided in this embodiment will obtain the operation data of each mobile machinery. Based on this data (and also referencing data collected by other sensing devices in the port's intelligent sensing area, such as data collected by monitoring cameras installed at each target path node, data collected by mobile machinery identification collection devices installed at each target path node, etc.), the total number of other mobile machinery currently located at any target path node (such as target path node i), the dwell time of each other mobile machinery at any target path node, and the type of each other mobile machinery can be obtained.

[0155] For example, the central control platform uses the operational data of each mobile machine, data collected by the monitoring camera at any target path node (e.g., target path node i), and other data collected by the monitoring camera at any target path node (e.g., target path node i) to determine all other mobile machines currently located in the area of ​​any target path node (e.g., target path node i) and whose dwell time exceeds a preset time threshold (e.g., 3 minutes). If a mobile machine X is currently located in the area of ​​any target path node (e.g., target path node i), but its dwell time does not exceed the preset time threshold (e.g., 3 minutes), it is considered to have temporarily passed through the area of ​​any target path node (e.g., target path node i) and is not identified as another mobile machine in this location. In other words, in this embodiment and subsequent embodiments, other mobile machines are mobile machines that are not any particular mobile machine (e.g., mobile machine M), and the dwell time of such mobile machine in the area of ​​any target path node (e.g., target path node i) exceeds the preset time threshold (e.g., 3 minutes).

[0156] 2) Determine the impact value of any target path node based on the total number of other mobile machines currently located at any target path node, the dwell time of each other mobile machine at any target path node, and the type of each other mobile machine.

[0157] For example, the influence value of any target path node (such as target path node i).

[0158] in, This represents the total number of other mobile machines currently located at any target path node (e.g., target path node i).

[0159] n2 represents the position L at the current position. t Initially, the total number of all mobile machines within an initial distance d3 in the direction of travel of the mobile machine is the sum of the number of other mobile machines in all target path nodes. For example, if there are two target path nodes, target path node 1 and target path node i, and the total number of other mobile machines currently located at target path node 1 is 1, then...

[0160] This represents the total number of other mobile machines of the same type as any given mobile machine (e.g., mobile machine M) currently located at any target path node (e.g., target path node i). If the type of any mobile machine (e.g., mobile machine M) is type k, then... This represents the total number of other mobile machines of type k that are currently located at any target path node (e.g., target path node i).

[0161] This represents the minimum number of other mobile machines of each type currently located at any target path node (e.g., target path node i). For example, if the mobile machines currently located at any target path node (e.g., target path node i) involve three types: type k, type 2, and type 3. If there are any mobile machines currently located at any target path node (e.g., target path node i)... There are 2 types of special mobile machinery, each of type k; there are 2 types of special mobile machinery, each of type 2; and there are 3 types of special mobile machinery, each of type 3. (wherein, ).So for The minimum value among quantity 2 and quantity 3.

[0162] This represents the maximum number of other mobile machines of each type currently located at any target path node (e.g., target path node i). For example, if the mobile machines currently located at any target path node (e.g., target path node i) involve three types: type k, type 2, and type 3, then... There are 2 types of special mobile machinery, each of type k; there are 2 types of special mobile machinery, each of type 2; and there are 3 types of special mobile machinery, each of type 3. (wherein, ).So for The maximum value among quantity 2 and quantity 3.

[0163] This refers to the average dwell time of a mobile machine identical to any other mobile machine (e.g., mobile machine M) at any target path node (e.g., target path node i). For example, there is... If all k types of mobile machinery (excluding any mobile machinery such as mobile machinery M) have stopped at any target path node (such as target path node i), then for The average dwell time of a mobile machine at any target path node (e.g., target path node i).

[0164] This represents the minimum average dwell time of all types of mobile machinery currently located at any target path node (e.g., target path node i) at that node. For example, the mobile machinery currently located at any target path node (e.g., target path node i) involves three types: type k, type 2, and type 3. All k types of mobile machinery (excluding any single mobile machinery such as mobile machinery M) have stopped at any target path node (such as target path node i), and the The average residence time of each type k of mobile machinery is Five types of type 2 mobile machines (excluding any single mobile machine (e.g., mobile machine M)) all stopped at any target path node (e.g., target path node i), and the average dwell time of these five types of type 2 mobile machines was 2. Six types of type 3 mobile machines (excluding any single mobile machine (e.g., mobile machine M)) all stopped at any target path node (e.g., target path node i), and the average dwell time of these six types of type 3 mobile machines was 3. Then... for The minimum values ​​of average stay time 2 and average stay time 3.

[0165] This represents the maximum average dwell time of all types of mobile machinery currently located at any target path node (e.g., target path node i) at that node. For example, the mobile machinery currently located at any target path node (e.g., target path node k) involves three types: type k, type 2, and type 3. All k types of mobile machinery (excluding any single mobile machinery such as mobile machinery M) have stopped at any target path node (such as target path node i), and the The average residence time of each type k of mobile machinery is Five types of type 2 mobile machines (excluding any single mobile machine (e.g., mobile machine M)) all stopped at any target path node (e.g., target path node i), and the average dwell time of these five types of type 2 mobile machines was 2. Six types of type 3 mobile machines (excluding any single mobile machine (e.g., mobile machine M)) all stopped at any target path node (e.g., target path node i), and the average dwell time of these six types of type 3 mobile machines was 3. Then... for The maximum values ​​of average stay time 2 and average stay time 3.

[0166] This represents the maximum duration of other types of mobile machinery that remain at any target path node (e.g., target path node i). For example, besides... In addition to a flow of type k (excluding any flow of machine M) stopping at any target path node (e.g., target path node i), there are 7 flow of machines not of type k that also stopped at any target path node (e.g., target path node i). The longest time that a number of 7 non-type k mobile machines stay at any target path node (such as target path node i) is.

[0167] This represents the minimum dwell time at each node among all types visited at any given target path node (e.g., target path node i). For example, besides... In addition to a type k mobile machine (excluding any mobile machine such as mobile machine M) stopping at any target path node (such as target path node i), there are 7 non-type k mobile machines that have also stopped at any target path node (such as target path node i), and these 7 non-type k mobile machines involve 4 types, namely type 2, type 3, type 4 and type 5. The minimum stay duration for all mobile machines of type 2 (excluding any mobile machine (e.g., mobile machine M) in all nodes (nodes are not limited to target path nodes and path nodes, but all nodes within the port's intelligent sensing area) is minimum duration 2. The minimum stay duration for all mobile machines of type 3 (excluding any mobile machine (e.g., mobile machine M)) in all nodes is minimum duration 3. The minimum stay duration for all mobile machines of type 4 (excluding any mobile machine (e.g., mobile machine M)) in all nodes is minimum duration 4. The minimum stay duration for all mobile machines of type 5 (excluding any mobile machine (e.g., mobile machine M)) in all nodes is minimum duration 5. It is the minimum value among minimum duration 2, minimum duration 3, minimum duration 4, and minimum duration 5.

[0168] This represents the maximum dwell time at each node across all types visited at any given target path node (e.g., target path node i). For example, besides... In addition to a type k mobile machine (excluding any mobile machine such as mobile machine M) stopping at any target path node (such as target path node i), there are 7 non-type k mobile machines that have also stopped at any target path node (such as target path node i), and these 7 non-type k mobile machines involve 4 types, namely type 2, type 3, type 4 and type 5. The maximum duration of stay for all mobile machines of type 2 (excluding any mobile machine (e.g., mobile machine M) in all nodes (nodes are not limited to target path nodes and path nodes, but all nodes within the port's intelligent sensing area) is a maximum duration of 2. The maximum duration of stay for all mobile machines of type 3 (excluding any mobile machine (e.g., mobile machine M)) in all nodes is a maximum duration of 3. The maximum duration of stay for all mobile machines of type 4 (excluding any mobile machine (e.g., mobile machine M)) in all nodes is a maximum duration of 4. The maximum duration of stay for all mobile machines of type 5 (excluding any mobile machine (e.g., mobile machine M)) in all nodes is a maximum duration of 5. It is the maximum value among the maximum durations of 2, 3, 4, and 5.

[0169] It should be noted that the "stay" mentioned in this embodiment and subsequent embodiments refers to a stay duration greater than a duration threshold (e.g., 3 minutes). If the stay duration is not greater than the duration threshold (e.g., 3 minutes), it is considered not a stay.

[0170] This represents the proportion of other mobile machinery currently located at any target path node (e.g., target path node i). The larger this proportion, the more important any target path node (e.g., target path node i) is to any mobile machinery (e.g., mobile machinery M) within the initial distance d3 in the forward direction. The greater the likelihood that this node will affect the safe operation of any mobile machinery (e.g., mobile machinery M).

[0171] This reflects the proportion of other mobile machines of the same type as any mobile machine (e.g., mobile machine M) that stay in any target path node (e.g., target path node i). The larger the proportion, the greater the probability of mobile machines of the same type (e.g., mobile machine M) staying in that node. An increase in the number of mobile machines of the same type staying in the same node will lead to more mobile machines of the same priority gathering in the same node. The conflict caused by the gathering of mobile machines of the same priority will affect the safe operation of any mobile machine (e.g., mobile machine M).

[0172] therefore, The number and type of mobile machinery reflect the impact of the target path node (such as target path node i) on any mobile machinery (such as mobile machinery M).

[0173] This reflects the proportion of time that mobile machinery of any type (e.g., mobile machinery M) stays at any target route node (e.g., target route node i). The larger the proportion, the longer the mobile machinery of any type (e.g., mobile machinery M) stays at that node. In this case, more mobile machinery is likely to gather at any target route node (e.g., target route node i), and more mobile machinery will affect the safe operation of any mobile machinery (e.g., mobile machinery M).

[0174] This reflects the proportion of time that non-any type of mobile machinery (such as mobile machinery M) stays at any target path node (such as target path node i). The larger the proportion, the longer the non-any type of mobile machinery (such as mobile machinery M) stays at that node. In this case, more mobile machinery is likely to gather at any target path node (such as target path node i), and more mobile machinery will affect the safe operation of any mobile machinery (such as mobile machinery M).

[0175] therefore, The dwell time of mobile machinery reflects the impact of any target path node (such as target path node i) on any mobile machinery (such as mobile machinery M).

[0176] It is the impact of any target path node (such as target path node i) on any mobile machine (such as mobile machine M) obtained from the perspective of the number, type and dwell time of mobile machines. The larger the value, the greater the influence of any target path node (such as target path node i) on any mobile machine (such as mobile machine M).

[0177] 4. Determine the specific adjustment value based on the impact value of each target path node.

[0178] For example, determining the specific adjustment value

[0179] Where d4 represents any mobile machine (such as mobile machine M) with an initial velocity of v and the current velocity v. t The acceleration is the current acceleration a. t Under the condition of moving forward a distance of Δr3.

[0180] Δr3 is a pre-set third duration, which is the unit adjustment duration for a mobile machine. It is used to control the range of adjustment values, meaning that adjustments can only be made within the operating distance of Δr3. This value is an empirical value, determined by relevant personnel based on actual work experience (such as the actual situation of safety risks of the mobile machine). The lower the safety risk of the mobile machine, the larger this value can be, thus allowing for a larger adjustment scale and a wider range of adjustment for the sensing distance.

[0181] δ is the maximum adjustment threshold, an empirical value, which represents the maximum adjustment value to prevent... An excessively large value can lead to over-adjustment. This value is determined by relevant personnel based on their practical experience (such as the actual situation of safety risks of mobile machinery). The smaller the safety risk of mobile machinery, the larger this value can be, thus allowing for a larger adjustment scale and a wider range of adjustment for the sensing distance.

[0182] `max{·}` is the function to find the maximum value.

[0183] pre t The sensing priority for any mobile machinery (such as mobile machinery M) is determined in step 402.

[0184] pre(min) is the minimum perception priority of all mobile machines on the travel path of any mobile machine (such as mobile machine M), and pre(max) is the maximum perception priority of all mobile machines on the travel path of any mobile machine (such as mobile machine M).

[0185] This reflects the priority of any mobile machine (such as mobile machine M) on the travel path. The larger the value, the higher the priority of any mobile machine (such as mobile machine M) on the travel path. d4 is the speed of any mobile machine (such as mobile machine M) at its current speed v. t Current acceleration a t The distance traveled forward by the sensing unit is Δt3. This represents the maximum impact value of all target path nodes, which is the maximum impact value within the initial distance of any mobile machine (such as mobile machine M).

[0186] Using d4 as the base adjustment distance, and considering the maximum influence value within the initial distance of any mobile machine (e.g., mobile machine M) and its priority on the travel path, the adjustment distance is obtained. This makes the specific adjustment value Δd the distance adjusted accordingly based on the node situation within the initial distance and the priority of any mobile machine (e.g., mobile machine M) on the base adjustment distance. This distance can better reflect the influence of nodes and mobile machines within the initial distance of any mobile machine (e.g., mobile machine M), allowing for more precise adjustment of the sensing distance.

[0187] Specifically, the greater the influence value of a node within the initial distance of any mobile machine (such as mobile machine M), the greater the specific adjustment value Δd; the higher the priority of any mobile machine (such as mobile machine M) on the travel path, the greater the specific adjustment value Δd.

[0188] 5. Determine the sensing distance based on the initial distance and the specific adjustment value.

[0189] For example, the perceived distance D = d3 - Δd is determined.

[0190] d3 represents the maximum accurately perceptible distance considering the radar detection range d1 and the camera depth of field d2 of the mobile machinery. Based on this, if the influence value of nodes within the initial distance of any mobile machinery (e.g., mobile machinery M) is larger, then more careful assistance is needed for that mobile machinery (e.g., mobile machinery M), thus reducing the perception distance. This allows for more granular perception; therefore, the larger Δd is, the smaller D is. Conversely, if any mobile machinery (e.g., mobile machinery M) has a higher priority on its travel path, then better protection for its travel safety is required, further reducing the perception distance and allowing for more granular perception. Therefore, the larger Δd is, the smaller D is.

[0191] The sensing distance D is a distance that can be determined comprehensively based on the conditions of the mobile machinery (such as priority, radar and camera parameters, etc.) and the direction of operation of the mobile machinery (such as the influence of target path nodes, etc.). This distance can help to more accurately ensure the safe operation of the mobile machinery.

[0192] As can be seen from the above explanation, in step 303, the central control platform will determine the current speed v of any mobile machinery (such as mobile machinery M). t Destination position L2, starting position L1, and current position L t Determine the sensing distance D.

[0193] 304. Send the sensing distance to the control subunit of the mobile machinery so that the computing subunit of the mobile machinery can sense obstacles within the sensing distance based on the data collected by radar and camera, and the control subunit of the mobile machinery can control the movement of the mobile machinery based on the obstacle sensing results.

[0194] In step 304, the central control platform sends the sensing distance D to the control subunit of any mobile machine (such as mobile machine M). This enables the computing subunit of any mobile machine (such as mobile machine M) to perceive obstacles within the sensing distance based on data collected by radar and cameras, and the control subunit of the mobile machine controls the movement of the mobile machine based on the obstacle perception results.

[0195] For example, (1) the central control platform sends the sensing distance D to the control subunit of any mobile machinery (such as mobile machinery M).

[0196] (2) The control subunit of any mobile machine (such as mobile machine M) controls the radar and camera to collect data within the area corresponding to the sensing distance D.

[0197] (3) Any mobile machine (such as mobile machine M) can perform obstacle perception within the perception range based on the data collected by radar and camera.

[0198] Data fusion techniques can be used to integrate information from different sensors into a unified environmental model, compensating for the limitations of a single sensor (e.g., LiDAR data can provide high-precision 3D point clouds, but it's difficult to distinguish object types, while cameras can supplement visual information and enhance the system's semantic recognition capabilities). Complex algorithms can extract valuable information from raw data, such as detecting the boundaries of mobile machinery, segmenting pedestrian contours, identifying obstacles, and recognizing road signs, thereby achieving feature extraction. Existing deep learning methods can then be used to perform obstacle detection and tracking, driving scene understanding, and perception of obstacles within the sensing distance based on the extracted features.

[0199] (4) The control subunit of the mobile machinery controls the movement of the mobile machinery (such as obstacle avoidance) based on the obstacle perception results.

[0200] The radar and camera data acquisition process, the obstacle perception process within the sensing distance based on the radar and camera data, and the motion control process based on the obstacle perception results are all implemented using existing solutions, which will not be elaborated on here. You can refer to existing autonomous driving solutions for further information.

[0201] The integrated multi-sensor intelligent perception method for port mobile machinery provided in this embodiment enables intelligent assisted driving of mobile machinery. Firstly, by including an edge computing unit and multiple sensors on each piece of mobile machinery, the perception capability of the mobile machinery is enhanced, allowing for a comprehensive understanding of static and dynamic elements in the surrounding environment, including traffic signs, other vehicles, pedestrians, and potential obstacles. Through accurate perception of these elements, the mobile machinery can construct a clear environmental model in complex traffic scenarios, providing a reliable basis for subsequent driving decisions. Secondly, the edge computing unit is a core component of assisted autonomous driving for mobile machinery and the foundation for achieving high-level autonomous driving functions. The mobile machinery collects external environmental information through various sensors and transforms this data into a form that machines can understand and analyze, supporting the intelligent assisted driving control of the mobile machinery. The integrated multi-sensor intelligent perception method for port mobile machinery provided in this embodiment can perceive the dynamic environment in real time and accurately, thereby helping the vehicle make safe and effective decisions.

[0202] The integrated multi-sensor intelligent perception method for port mobile machinery provided in this embodiment can realize environmental perception of the working environment and obstacle identification and positioning functions, providing a reliable basis for subsequent driving decisions and improving operational efficiency and safety.

[0203] The integrated multi-sensor intelligent perception method for port mobile machinery provided in this embodiment can provide technical support in terms of electronic and electrical structure for intelligent assisted driving of port mobile machinery, and promote the implementation of intelligent port mobile machinery.

[0204] The integrated multi-sensor intelligent perception method for port mobile machinery provided in this embodiment replaces manual driving with intelligent assisted driving technology, reducing the risk of personal injury accidents and lowering the port's labor costs.

[0205] The integrated multi-sensor intelligent perception method and intelligent assisted driving technology for port mobile machinery provided in this embodiment make the use of mobile machinery more reasonable and precise, effectively avoid collisions during operation, reduce the maintenance cost of mobile machinery, and improve the service life of vehicles.

[0206] The integrated multi-sensor intelligent perception method for port mobile machinery provided in this embodiment is not only applicable to the intelligent assisted driving retrofit of existing mobile machinery, but also to the electronic and electrical structure planning of newly manufactured mobile machinery, and has strong applicability.

[0207] The integrated multi-sensor intelligent perception method for port mobile machinery provided in this embodiment fully considers the functional characteristics of port mobile machinery operation in terms of intelligent assisted driving technology, and provides a better implementation plan for the implementation of intelligent assisted driving for port mobile machinery.

[0208] This embodiment provides an integrated, multi-sensor intelligent sensing method for port mobile machinery. The mobile machinery includes an edge computing unit and multiple sensors. The edge computing unit includes a computation subunit and a control subunit. The sensors include a speed sensor, a high-precision positioning sensor, radar, and a camera. The method includes: acquiring the destination position and starting position of the mobile machinery; periodically acquiring the current position, current speed, and current acceleration of the mobile machinery; determining the sensing distance based on the current speed, current acceleration, destination position, starting position, and current position; sending the sensing distance to the control subunit of the mobile machinery, so that the computation subunit of the mobile machinery can perceive obstacles within the sensing distance based on the data collected by the radar and camera, and the control subunit of the mobile machinery can control the movement of the mobile machinery based on the obstacle perception results. This embodiment's method, based on an edge computing unit and multiple sensors, enables assisted driving of mobile machinery, improves the operational efficiency and safety of port mobile machinery, and promotes the automation and intelligence of dry bulk cargo ports.

[0209] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the systems / devices based on the methods described in the above embodiments of the present invention, and therefore will not be repeated here. All systems / devices used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.

[0210] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0211] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0212] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.

[0213] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0214] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0215] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.

Claims

1. A method for intelligent sensing of port mobile machinery integrating multiple sensors, characterized in that, The mobile machinery includes an edge computing unit and multiple sensors; The edge computing unit includes: a computation subunit and a control subunit; The sensors include: a speed sensor, a high-precision positioning sensor, radar, and a camera; The method includes: Obtain the destination and starting positions of the mobile machinery; The current position, current speed, and current acceleration of the mobile machinery are periodically acquired; wherein the current position is determined based on a high-precision positioning sensor of the mobile machinery, and the current speed and current acceleration are determined based on a speed sensor of the mobile machinery. The sensing distance is determined based on the current speed, current acceleration, target position, starting position, and current position. The sensing distance is sent to the control subunit of the mobile machinery so that the computing subunit of the mobile machinery can sense obstacles within the sensing distance based on the data collected by radar and camera. The control subunit of the mobile machinery then controls the movement of the mobile machinery based on the obstacle sensing results.

2. The method according to claim 1, characterized in that, Determining the sensing distance based on the current speed, current acceleration, target position, starting position, and current position includes: If the distance between the current position and the comparison position is less than the comparison distance, and the comparison position is not the target position, then the sensing distance is determined; Wherein, if the initial speed is the current speed and the acceleration is the current acceleration, and the speed after moving forward for a preset first time does not exceed the maximum speed limit, then the comparison distance is the distance the mobile machinery moves forward for the first time using the first operating mode; otherwise, the comparison distance is the distance the mobile machinery moves forward for the first time using the second operating mode. The first operating mode is an acceleration mode where the initial speed is the current speed and the acceleration is the current acceleration. The second operating mode is a variable speed mode where the initial speed is the current speed and the acceleration is the current acceleration until the maximum speed limit is reached, and then the speed is maintained at the maximum speed limit. The sum of the acceleration duration and the constant speed duration is the first duration. The initial position of the comparison location is the starting position, which is updated after the new sensing distance is determined.

3. The method according to claim 2, characterized in that, If the candidate position does not reach the target position, then update the comparison position to the candidate position; otherwise, update the comparison position to the target position. The candidate position is the position of the mobile machinery forward from its current position by the sensed distance.

4. The method according to claim 2, characterized in that, Determining the sensing distance includes: Determine the type of mobile machinery, start time, end time, workload, and priority of the work objects; The perception priority is determined based on the start time, end time, workload, and priority weight of the work object. Determine the path nodes based on the destination location and the current location; The sensing distance is determined based on the type of mobile machinery, sensing priority, and path nodes.

5. The method according to claim 4, characterized in that, The process of determining the sensing priority based on the start time, end time, workload, and priority weight of the work object includes: The adjustment value is determined based on the relationship between the workload and the start and end times of the work. The perception priority is determined based on the adjustment value, adjustment ratio, and priority weight of the task object.

6. The method according to claim 4, characterized in that, The process of determining the sensing priority based on the start time, end time, workload, and priority weight of the work object includes: The adjustment value is determined based on the relationship between the current time and the start and end times of the work, as well as the workload; The perception priority is determined based on the adjustment value, adjustment ratio, and priority weight of the task object.

7. The method according to claim 5 or 6, characterized in that, The adjustment ratio is determined based on the priority weight of the work objects and the relationship between the current time and the start and end times of the work.

8. The method according to claim 4, characterized in that, The step of determining the sensing distance based on the type of mobile machinery, sensing priority, and path nodes includes: The initial distance is determined based on the radar detection range of the mobile machinery and the depth of field of the camera; Among the path nodes, a target path node located within the initial distance from the current position of the mobile machinery is identified; The impact value of each target path node is determined based on the type and sensing priority of the mobile machinery. The specific adjustment value is determined based on the impact value of each target path node; The sensing distance is determined based on the initial distance and the specific adjustment value.

9. The method according to claim 8, characterized in that, The step of determining the impact value of each target path node based on the type and sensing priority of the mobile machinery includes: For any target path node, determine the total number of other mobile machines currently located at the target path node, the dwell time of each other mobile machine at the target path node, and the type of each other mobile machine; determine the influence value of the target path node based on the total number of other mobile machines currently located at the target path node, the dwell time of each other mobile machine at the target path node, and the type of each other mobile machine.

10. An integrated multi-sensor intelligent sensing system for port mobile machinery, characterized in that, The system includes: at least one mobile machine and a central control platform; The mobile machinery includes an edge computing unit and multiple sensors; The edge computing unit includes: a computation subunit and a control subunit; The sensors include: a speed sensor, a high-precision positioning sensor, radar, and a camera; The central control platform is used to perform the method as described in any one of claims 1-9, and to intelligently sense each mobile machine.

Images