Method and device for determining driving path of autonomous vehicle and storage medium

Abstract

The invention provides a method and device for determining a driving path of an autonomous vehicle and a storage medium, and the method comprises the steps: obtaining a three-dimensional coordinate of a suspended object, and determining the point cloud data of the suspended object according to the three-dimensional coordinate of the suspended object; determining the category and boundary information of the suspended object at least according to the point cloud data of the suspended object; according to the boundary information of the suspended object, velocity components of the suspended object in all directions are determined, and the velocity components at least comprise a longitudinal velocity component, a transverse velocity component and a vertical velocity component; and determining a ground projection area corresponding to the suspended object according to the point cloud data of the suspended object, and determining a driving path of the autonomous vehicle according to the speed component of the suspended object in each direction, the category of the suspended object and the ground projection area. According to the method, the suspended containers are detected, classified and tracked, and the autonomous vehicle is controlled to bypass the ground projection area of the suspended containers, so that the potential collision risk is avoided.

Description

Technical Field

[0001] The present application relates to the technical field of automated terminals, and more specifically, to a method for determining a driving path of an autonomous driving vehicle, a device for determining a driving path of an autonomous driving vehicle, a computer-readable storage medium, and an electronic device. Background Art

[0002] It is common to see containers suspended in mid-air by vertical operating equipment in ports. Therefore, port operating vehicles need to bypass the ground projection area corresponding to the suspended containers during driving to avoid collision accidents caused by blind spots.

[0003] In the existing technical solution, a yard interactive scanning prompt device is set in the interactive area of ​​the container terminal transport channel. The device obtains vehicle information by scanning the external horizontal transport equipment, and issues a prompt message for the external horizontal transport equipment to enter the interactive area based on the vehicle information. After obtaining the prompt message, the internal horizontal transport equipment automatically interacts with the transport vehicle according to the navigation positioning.

[0004] Existing solutions mainly use yard interactive scanning devices to scan vehicles, rather than vehicle sensors to scan the environment, so it is difficult for the vehicle to be aware of obstacles such as suspended containers; and existing solutions do not involve control of autonomous driving vehicles, so it is difficult to avoid the risk of collision between autonomous driving vehicles and obstacles such as suspended containers. Summary of the invention

[0005] The main purpose of the present application is to provide a method for determining the driving path of an autonomous driving vehicle, a device for determining the driving path of an autonomous driving vehicle, a computer-readable storage medium and an electronic device, so as to at least solve the problem in the prior art that it is difficult to avoid the risk of collision between the autonomous driving vehicle and obstacles such as suspended containers.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present application, a method for determining a driving path of an autonomous driving vehicle is provided, comprising: obtaining the three-dimensional coordinates of a suspended object, and determining the point cloud data of the suspended object based on the three-dimensional coordinates of the suspended object; determining the category and boundary information of the suspended object at least based on the point cloud data of the suspended object; determining the velocity components of the suspended object in each direction based on the boundary information of the suspended object, the velocity components including at least a longitudinal velocity component, a lateral velocity component and a vertical velocity component; determining the ground projection area corresponding to the suspended object based on the point cloud data of the suspended object, and determining the driving path of the autonomous driving vehicle based on the velocity components of the suspended object in each direction, the category of the suspended object and the ground projection area.

[0007] Optionally, obtaining the three-dimensional coordinates of a suspended object includes: obtaining first image data of the suspended object within a first preset time period, the first image data being acquired by using a high-resolution wide-angle camera; obtaining second image data of the suspended object within the first preset time period, the second image data being acquired by using a fisheye camera; and determining the three-dimensional coordinates of the suspended object based on the first image data and the second image data and using a three-dimensional target detection algorithm.

[0008] Optionally, the point cloud data of the suspended object is determined according to the three-dimensional coordinates of the suspended object, including: according to the three-dimensional coordinates of the suspended object, determining the area including the suspended object as a target area, the volume of the target area is less than or equal to a preset volume; using a laser radar to acquire environmental point cloud data of the target area; according to the three-dimensional coordinates of the suspended object, extracting the point cloud data of the suspended object from the environmental point cloud data.

[0009] Optionally, based on the boundary information of the suspended object, the velocity components of the suspended object in each direction are determined, including: obtaining the boundary information of the suspended object within a second preset time period; determining the moving speed and moving direction of the suspended object based on the boundary information and point cloud data of the suspended object within the second preset time period; based on the moving speed and moving direction of the suspended object, determining that the velocity component of the suspended object in the direction of advance of the autonomous driving vehicle is the longitudinal velocity component, determining that the velocity component of the suspended object in the direction perpendicular to the direction of advance of the autonomous driving vehicle is the lateral velocity component, and determining that the velocity component of the suspended object in the direction perpendicular to the ground is the vertical velocity component.

[0010] Optionally, the driving path of the autonomous driving vehicle is determined according to the velocity components of the suspended object in various directions, the category of the suspended object and the ground projection area, including: when the category of the suspended object is a suspended sling that has not grasped a container, controlling the autonomous driving vehicle to travel along an original driving path, and the original driving path overlaps or does not overlap with the ground projection area; when the category of the suspended object is a suspended container, updating the driving path of the autonomous driving vehicle according to the velocity components of the suspended container in various directions and the ground projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle, and the updated driving path does not overlap with the ground projection area.

[0011] Optionally, the driving path of the autonomous driving vehicle is updated according to the speed components of the suspended container in various directions and the ground projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle, including: obtaining the ground projection area of ​​the suspended container at the current moment as the current projection area; predicting the predicted projection area of ​​the suspended container at the next moment according to the speed components of the suspended container in various directions and the current projection area of ​​the suspended container; obtaining the current driving speed of the autonomous driving vehicle; updating the driving path of the autonomous driving vehicle according to the current driving speed of the autonomous driving vehicle and the predicted projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle.

[0012] Optionally, the autonomous driving vehicle is equipped with a high-resolution wide-angle camera, a blind spot laser radar and a fisheye camera. The vertical field angle of the high-resolution wide-angle camera is greater than or equal to a first preset angle, the vertical field angle of the blind spot laser radar and the vertical field angle of the fisheye camera are both greater than or equal to a second preset angle, and the second preset angle is greater than the first preset angle.

[0013] According to another aspect of the present application, a device for determining a driving path of an autonomous driving vehicle is provided, comprising: an acquisition unit, configured to acquire three-dimensional coordinates of a suspended object, and determine point cloud data of the suspended object based on the three-dimensional coordinates of the suspended object; a first determination unit, configured to determine the category and boundary information of the suspended object at least based on the point cloud data of the suspended object; a second determination unit, configured to determine velocity components of the suspended object in various directions based on the boundary information of the suspended object, the velocity components including at least a longitudinal velocity component, a lateral velocity component and a vertical velocity component; a third determination unit, configured to determine a ground projection area corresponding to the suspended object based on the point cloud data of the suspended object, and determine the driving path of the autonomous driving vehicle based on the velocity components of the suspended object in various directions, the category of the suspended object and the ground projection area.

[0014] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the methods for determining the driving path of the autonomous driving vehicle.

[0015] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for determining a driving path of any one of the autonomous driving vehicles described.

[0016] Applying the technical solution of the present application, the method for determining the driving path of the above-mentioned autonomous driving vehicle first obtains the three-dimensional coordinates of the suspended object, and determines the point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object; then determines the category and boundary information of the suspended object at least according to the point cloud data of the suspended object; then determines the velocity components of the suspended object in each direction according to the boundary information of the suspended object, and the velocity components at least include the longitudinal velocity component, the lateral velocity component and the vertical velocity component; finally, determines the ground projection area corresponding to the suspended object according to the point cloud data of the suspended object, and determines the driving path of the autonomous driving vehicle according to the velocity components of the suspended object in each direction, the category of the suspended object and the ground projection area. The method configures the autonomous driving truck with on-board sensors such as blind laser radar, high-resolution wide-angle camera, fisheye camera, etc., detects, classifies and tracks the suspended container through the above sensors, and controls the autonomous driving vehicle to bypass the ground projection area of ​​the suspended container according to the above perception results, so as to solve the problem that it is difficult to avoid the collision risk between the autonomous driving vehicle and obstacles such as the suspended container in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings constituting part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A hardware structure block diagram of a mobile terminal for executing a method for determining a driving path of an autonomous driving vehicle provided in an embodiment of the present application is shown;

[0019] Figure 2 A schematic diagram of a flow chart of a method for determining a driving path of an autonomous driving vehicle provided according to an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram of the field of view of a high-resolution wide-angle camera, a blind spot-filling laser radar, and a fisheye camera installed on an autonomous driving vehicle according to an embodiment of the present application is shown;

[0021] Figure 4 A schematic flow chart of another method for determining a driving path of an autonomous driving vehicle provided according to an embodiment of the present application is shown;

[0022] Figure 5 A structural block diagram of a device for determining a driving path of an autonomous driving vehicle provided according to an embodiment of the present application is shown.

[0023] The above drawings include the following reference numerals:

[0024] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0025] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0026] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] As introduced in the background technology, the existing solutions mainly use the yard interactive scanning device to scan the vehicle, rather than the vehicle sensor to scan the environment, so it is difficult for the vehicle to be aware of obstacles such as suspended containers; and the existing solutions do not involve the control of the autonomous driving vehicle, so it is difficult to avoid the risk of collision between the autonomous driving vehicle and obstacles such as suspended containers. In order to solve the problem that it is difficult to avoid the risk of collision between the autonomous driving vehicle and obstacles such as suspended containers in the prior art, the embodiments of the present application provide a method for determining the driving path of an autonomous driving vehicle, a device for determining the driving path of an autonomous driving vehicle, a computer-readable storage medium, and an electronic device.

[0029] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0030] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal of a method for determining a driving path of an autonomous driving vehicle according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0031] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the driving path of the autonomous driving vehicle in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. The transmission device 106 is used to receive or send data via a network. The above network specific examples may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0032] In this embodiment, a method for determining the driving path of an autonomous driving vehicle running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0033] Figure 2 FIG. 1 is a flow chart of a method for determining a driving path of an autonomous driving vehicle according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0034] Step S201, obtaining the three-dimensional coordinates of the suspended object, and determining the point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object;

[0035] Specifically, based on the image data generated in real time by the wide-angle camera and the fisheye camera, the area where the suspended container is located in the image is determined by a 3D target detection algorithm, the 3D position of the suspended container in the real world is calculated, and the blind lidar point cloud data of the position is used to confirm the precise position and boundary of the suspended container. The autonomous driving vehicle can be an autonomous driving truck.

[0036] Among them, the above-mentioned autonomous driving vehicle is equipped with a high-resolution wide-angle camera, a blind spot laser radar and a fish-eye camera. The vertical field angle of the above-mentioned high-resolution wide-angle camera is greater than or equal to a first preset angle, and the vertical field angle of the above-mentioned blind spot laser radar and the vertical field angle of the above-mentioned fish-eye camera are both greater than or equal to a second preset angle, and the second preset angle is greater than the first preset angle.

[0037] Specifically, through the configuration and angle optimization of on-board sensors such as blind spot lidar, high-resolution wide-angle camera, fisheye camera, etc., it can effectively support the detection, classification and tracking of suspended containers in the port. The detection height of suspended containers can be as low as 20 meters and as high as 30 meters.

[0038] In some instances, such as Figure 3 As shown in the figure, a high-resolution wide-angle camera is installed in the cabin of the self-driving truck and deployed behind the front windshield through a bracket. The resolution of the camera should reach 5 million pixels, and the vertical field of view should not be less than 60 degrees; a blind spot laser radar and a fisheye camera are installed near the center of the front bumper of the self-driving truck. The vertical field of view of the blind spot laser radar and the fisheye camera should not be less than 120 degrees. Figure 3As shown in the figure, the first sector area A is the field of view of the laser radar detection, the second sector area B is the field of view of the high-resolution wide-angle camera, and the third sector area C is the field of view of the fisheye camera. In order to ensure the detection effect of high-level suspended containers, the installation pitch angle of the above sensors can be appropriately adjusted (0 to 30 degrees). The above sensor configuration ensures that the minimum height area coverage is 20 meters (H_min) and the maximum height area coverage is 30 meters (H_max) within the longitudinal direction of 25 meters.

[0039] Wherein, obtaining the three-dimensional coordinates of the suspended object includes the following steps:

[0040] Step S301, obtaining first image data of the suspended object within a first preset time period, wherein the first image data is acquired by using a high-resolution wide-angle camera;

[0041] Step S302, obtaining second image data of the suspended object within the first preset time period, wherein the second image data is acquired by using a fisheye camera;

[0042] Step S303, determining the three-dimensional coordinates of the suspended object according to the first image data and the second image data and using a three-dimensional target detection algorithm.

[0043] Specifically, by fusing the two types of image data, the three-dimensional coordinates of the suspended object can be accurately determined, preventing the problem of inaccurate coordinate positioning caused by one of the high-resolution wide-angle camera or the fisheye camera not fully capturing the suspended object. In addition, the combination of the high resolution of the wide-angle camera and the high depth detection of the fisheye camera makes binocular recognition more accurate.

[0044] Wherein, determining the point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object comprises the following steps:

[0045] Step S401, according to the three-dimensional coordinates of the suspended object, determining the area including the suspended object as a target area, wherein the volume of the target area is less than or equal to a preset volume;

[0046] Step S402, using a laser radar to obtain environmental point cloud data of the target area;

[0047] Step S403 , extracting the point cloud data of the suspended object from the environmental point cloud data according to the three-dimensional coordinates of the suspended object.

[0048] Specifically, by first reducing the detection range of the lidar through the three-dimensional coordinates of the suspended object, the system computing power can be saved.

[0049] Step S202, determining the category and boundary information of the suspended object at least according to the point cloud data of the suspended object;

[0050] Specifically, the point cloud data of suspended objects can effectively support the detection, classification and tracking of suspended containers in the port. The detection height of suspended containers can be as low as 20 meters and as high as 30 meters.

[0051] Step S203, determining the velocity components of the suspended object in various directions according to the boundary information of the suspended object, wherein the velocity components at least include a longitudinal velocity component, a lateral velocity component and a vertical velocity component;

[0052] Specifically, according to the boundary information of the suspended object, the velocity components of the suspended object in each direction can be accurately determined, thereby accurately determining the ground projection area of ​​the suspended container.

[0053] Wherein, determining the velocity components of the suspended object in each direction according to the boundary information of the suspended object comprises the following steps:

[0054] Step S2031, obtaining boundary information of the suspended object within a second preset time period;

[0055] Step S2032, determining the moving speed and moving direction of the suspended object according to the boundary information and point cloud data of the suspended object within the second preset time period;

[0056] Step S2033, based on the moving speed and moving direction of the above-mentioned suspended object, determine the velocity component of the above-mentioned suspended object in the forward direction of the above-mentioned autonomous driving vehicle as the above-mentioned longitudinal velocity component, determine the velocity component of the above-mentioned suspended object in the direction perpendicular to the forward direction of the above-mentioned autonomous driving vehicle as the above-mentioned lateral velocity component, and determine the velocity component of the above-mentioned suspended object in the direction perpendicular to the ground as the above-mentioned vertical velocity component.

[0057] Specifically, according to the position and boundary of the suspended container in continuous time, the speed magnitude and direction of the suspended container are calculated, and the speed components of the container in the longitudinal, transverse and vertical directions are output, which can provide a data basis for the subsequent determination of the ground projection area of ​​the suspended object. The second preset time period can be set to within 5 seconds to 10 seconds.

[0058] Step S204, determining the ground projection area corresponding to the suspended object based on the point cloud data of the suspended object, and determining the driving path of the autonomous driving vehicle based on the velocity components of the suspended object in each direction, the type of the suspended object and the ground projection area.

[0059] Specifically, the autonomous driving vehicle is controlled to bypass the ground projection area of ​​the suspended container, thereby avoiding potential collision risks.

[0060] Wherein, determining the driving path of the autonomous driving vehicle according to the velocity components of the suspended object in each direction, the type of the suspended object and the ground projection area comprises the following steps:

[0061] Step S501, when the category of the suspended object is a suspended spreader that does not grab a container, controlling the automatic driving vehicle to travel according to an original driving path, and the original driving path overlaps or does not overlap with the ground projection area;

[0062] Step S502, when the category of the above-mentioned suspended object is a suspended container, the driving path of the above-mentioned autonomous driving vehicle is updated according to the velocity components of the above-mentioned suspended container in various directions and the ground projection area of ​​the above-mentioned suspended container to obtain an updated driving path of the above-mentioned autonomous driving vehicle, and the above-mentioned updated driving path does not overlap with the above-mentioned ground projection area.

[0063] Specifically, when the suspended object is a suspended container, the autonomous driving vehicle is controlled to detour the ground projection area of ​​the suspended container to avoid potential collision risks. Among them, when the suspended object is a suspended spreader without grabbing a container, since the spreader does not grab any container, there is no risk of the container falling on the ground and colliding with the autonomous driving vehicle. Therefore, when the suspended object is a suspended spreader without grabbing a container, the autonomous driving vehicle does not need to detour and can continue to maintain the current operating state.

[0064] In addition, the categories of suspended containers also include different container sizes such as 40-foot containers or 20-foot containers, which will also affect the driving path of the autonomous vehicle.

[0065] According to the velocity components of the suspended container in each direction and the ground projection area of ​​the suspended container, the driving path of the autonomous driving vehicle is updated to obtain the updated driving path of the autonomous driving vehicle, including the following steps:

[0066] Step S5021, obtaining the ground projection area of ​​the suspended container at the current moment as the current projection area;

[0067] Step S5022, predicting the predicted projection area of ​​the suspended container at the next moment according to the velocity components of the suspended container in each direction and the current projection area of ​​the suspended container;

[0068] Step S5023, obtaining the current driving speed of the autonomous driving vehicle;

[0069] Step S5024: updating the driving path of the autonomous driving vehicle according to the current driving speed of the autonomous driving vehicle and the predicted projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle.

[0070] Specifically, this can control the autonomous driving vehicle to bypass the ground projection area of ​​the suspended container, thereby avoiding potential collision risks. Among them, combined with the location, category and speed information of the suspended container, the autonomous driving vehicle is controlled to select the appropriate bypass direction and spacing to bypass the ground projection area corresponding to the suspended container. If the lateral speed of the suspended container is to the left and there is ample space on the right, the autonomous driving vehicle should choose to bypass from the right, and vice versa; if the lateral speed of the suspended container is zero, the autonomous driving vehicle should choose to bypass the side with the most ample space and in compliance with port operation regulations.

[0071] The method for determining the driving path of the above-mentioned autonomous driving vehicle of the present application first obtains the three-dimensional coordinates of the suspended object, and determines the point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object; then determines the category and boundary information of the suspended object at least according to the point cloud data of the suspended object; then determines the velocity components of the suspended object in each direction according to the boundary information of the suspended object, and the velocity components at least include the longitudinal velocity component, the lateral velocity component and the vertical velocity component; finally, determines the ground projection area corresponding to the suspended object according to the point cloud data of the suspended object, and determines the driving path of the autonomous driving vehicle according to the velocity components of the suspended object in each direction, the category of the suspended object and the ground projection area. The method configures the autonomous driving truck with on-board sensors such as blind laser radar, high-resolution wide-angle camera, fisheye camera, etc., detects, classifies and tracks the suspended container through the above sensors, and controls the autonomous driving vehicle to bypass the ground projection area of ​​the suspended container according to the above perception results, so as to solve the problem that it is difficult to avoid the collision risk between the autonomous driving vehicle and obstacles such as the suspended container in the prior art.

[0072] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the method for determining the driving path of the autonomous driving vehicle of the present application will be described in detail below in combination with specific embodiments.

[0073] This embodiment relates to a specific method for determining a driving path of an autonomous driving vehicle, such as Figure 4 As shown, the following steps are included:

[0074] Step S0: Equip the self-driving truck with a forward-facing high-resolution wide-angle camera, blind spot-filling laser radar, and fisheye camera, and appropriately increase their installation pitch angles;

[0075] Step S1: Based on the image and point cloud data, the position and boundary of the suspended container are determined by a three-dimensional object detection algorithm;

[0076] Step S2: Classify the suspended containers (40-foot container, 20-foot container, suspended spreader that has not yet grabbed the container) according to the image and point cloud features;

[0077] Step S3: Calculate the velocity magnitude and direction of the suspended container according to the position and boundary of the suspended container in continuous time, and output the velocity components of the container in the longitudinal, transverse and vertical directions;

[0078] Step S4: Based on the location, type and speed information of the suspended container, the autonomous driving vehicle is controlled to select an appropriate detour direction and spacing to detour the ground projection area corresponding to the suspended container.

[0079] The embodiment of the present application also provides a device for determining the driving path of an autonomous driving vehicle. It should be noted that the device for determining the driving path of an autonomous driving vehicle in the embodiment of the present application can be used to execute the method for determining the driving path of an autonomous driving vehicle provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and those that have been described will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0080] The following is an introduction to the device for determining the driving path of an autonomous driving vehicle provided in an embodiment of the present application.

[0081] Figure 5 Schematic diagram of a device for determining a driving path of an autonomous driving vehicle according to an embodiment of the present application. Figure 5 As shown, the device includes an acquisition unit 10, a first determination unit 20, a second determination unit 30 and a third determination unit 40, the acquisition unit 10 is used to acquire the three-dimensional coordinates of the suspended object, and determine the point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object; the first determination unit 20 is used to determine the category and boundary information of the suspended object at least according to the point cloud data of the suspended object; the second determination unit 30 is used to determine the velocity components of the suspended object in various directions according to the boundary information of the suspended object, and the velocity components at least include the longitudinal velocity component, the lateral velocity component and the vertical velocity component; the third determination unit 40 is used to determine the ground projection area corresponding to the suspended object according to the point cloud data of the suspended object, and determine the driving path of the autonomous driving vehicle according to the velocity components of the suspended object in various directions, the category of the suspended object and the ground projection area.

[0082] The device for determining the driving path of the above-mentioned autonomous driving vehicle of the present application includes an acquisition unit, a first determination unit, a second determination unit and a third determination unit. The acquisition unit is used to acquire the three-dimensional coordinates of the suspended object, and determine the point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object; the first determination unit is used to determine the category and boundary information of the suspended object at least according to the point cloud data of the suspended object; the second determination unit is used to determine the velocity components of the suspended object in various directions according to the boundary information of the suspended object, and the velocity components include at least longitudinal velocity components, lateral velocity components and vertical velocity components; the third determination unit is used to determine the ground projection area corresponding to the suspended object according to the point cloud data of the suspended object, and determine the driving path of the autonomous driving vehicle according to the velocity components of the suspended object in various directions, the category of the suspended object and the ground projection area. The device equips the self-driving truck with on-board sensors such as blind spot laser radar, high-resolution wide-angle camera, and fisheye camera. It detects, classifies, and tracks suspended containers through the above sensors, and controls the self-driving vehicle to bypass the ground projection area of ​​the suspended containers based on the above perception results, solving the problem that it is difficult to avoid the risk of collision between the self-driving vehicle and obstacles such as suspended containers in the existing technology.

[0083] In some examples, the acquisition unit includes a first acquisition module, a second acquisition module and a first determination module, the first acquisition module is used to acquire the first image data of the suspended object within a first preset period of time, the first image data is acquired by using a high-resolution wide-angle camera; the second acquisition module is used to acquire the second image data of the suspended object within the first preset period of time, the second image data is acquired by using a fisheye camera; the first determination module is used to determine the three-dimensional coordinates of the suspended object based on the first image data and the second image data and using a three-dimensional target detection algorithm. By fusing the two types of image data, the three-dimensional coordinates of the suspended object can be accurately determined, preventing the problem of inaccurate coordinate positioning caused by the high-resolution wide-angle camera or the fisheye camera not fully capturing the suspended object.

[0084] In some examples, the acquisition unit includes a second determination module, a third acquisition module and an extraction module. The second determination module is used to determine the area including the suspended object as the target area according to the three-dimensional coordinates of the suspended object, and the volume of the target area is less than or equal to the preset volume; the third acquisition module is used to use a laser radar to acquire the environmental point cloud data of the target area; the extraction module is used to extract the point cloud data of the suspended object from the environmental point cloud data according to the three-dimensional coordinates of the suspended object. Firstly, narrowing the detection range of the laser radar by the three-dimensional coordinates of the suspended object can save system computing power.

[0085] In some examples, the second determination unit includes a fourth acquisition module, a third determination module and a fourth determination module, wherein the fourth acquisition module is used to acquire the boundary information of the suspended object within the second preset period; the third determination module is used to determine the moving speed and moving direction of the suspended object according to the boundary information and point cloud data of the suspended object within the second preset period; the fourth determination module is used to determine the velocity component of the suspended object in the forward direction of the autonomous driving vehicle as the longitudinal velocity component, the velocity component of the suspended object in the direction perpendicular to the forward direction of the autonomous driving vehicle as the lateral velocity component, and the velocity component of the suspended object in the direction perpendicular to the ground as the vertical velocity component according to the moving speed and moving direction of the suspended object. This can provide a data basis for the subsequent determination of the ground projection area of ​​the suspended object.

[0086] In this embodiment, the third determination unit includes a first control module and a first update module. The first control module is used to control the autonomous driving vehicle to travel according to the original driving path when the category of the suspended object is a suspended sling that does not grab a container. The original driving path overlaps or does not overlap with the ground projection area. The first update module is used to update the driving path of the autonomous driving vehicle according to the velocity components of the suspended container in each direction and the ground projection area of ​​the suspended container when the category of the suspended object is a suspended container, so as to obtain an updated driving path of the autonomous driving vehicle, and the updated driving path does not overlap with the ground projection area. When the category of the suspended object is a suspended container, the autonomous driving vehicle is controlled to bypass the ground projection area of ​​the suspended container to avoid potential collision risks.

[0087] An optional solution, the first update module includes a first acquisition submodule, a prediction submodule, a second acquisition submodule and an update submodule, the first acquisition submodule is used to acquire the ground projection area of ​​the above-mentioned suspended container at the current moment as the current projection area; the prediction submodule is used to predict the predicted projection area of ​​the above-mentioned suspended container at the next moment according to the speed components of the above-mentioned suspended container in each direction and the current projection area of ​​the above-mentioned suspended container; the second acquisition submodule is used to acquire the current driving speed of the above-mentioned autonomous driving vehicle; the update submodule is used to update the driving path of the above-mentioned autonomous driving vehicle according to the current driving speed of the above-mentioned autonomous driving vehicle and the predicted projection area of ​​the above-mentioned suspended container, and obtain the updated driving path of the above-mentioned autonomous driving vehicle. In this way, the autonomous driving vehicle can be controlled to bypass the ground projection area of ​​the suspended container, thereby avoiding potential collision risks.

[0088] As an optional solution, the autonomous driving vehicle is equipped with a high-resolution wide-angle camera, a blind spot laser radar and a fisheye camera. The vertical field of view of the high-resolution wide-angle camera is greater than or equal to a first preset angle, and the vertical field of view of the blind spot laser radar and the vertical field of view of the fisheye camera are both greater than or equal to a second preset angle, and the second preset angle is greater than the first preset angle. Through the configuration and angle optimization of on-board sensors such as blind spot laser radar, high-resolution wide-angle camera, and fisheye camera, it can effectively support the detection, classification and tracking of suspended containers in the port.

[0089] The above-mentioned device for determining the driving path of the autonomous driving vehicle includes a processor and a memory, and the above-mentioned acquisition unit and the like are stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to implement corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned modules are located in different processors in any combination.

[0090] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the problem of the difficulty in avoiding the risk of collision between the autonomous driving vehicle and obstacles such as suspended containers in the existing technology can be solved by adjusting the kernel parameters.

[0091] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0092] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the method for determining the driving path of the autonomous driving vehicle.

[0093] Specifically, the method for determining the driving path of the autonomous driving vehicle includes:

[0094] Step S201, obtaining the three-dimensional coordinates of the suspended object, and determining the point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object;

[0095] Specifically, based on the image data generated in real time by the wide-angle camera and the fisheye camera, the area where the suspended container is located in the image is determined by a 3D target detection algorithm, the 3D position of the suspended container in the real world is calculated, and the blind lidar point cloud data of the position is used to confirm the precise position and boundary of the suspended container. The autonomous driving vehicle can be an autonomous driving truck.

[0096] Step S202, determining the category and boundary information of the suspended object at least according to the point cloud data of the suspended object;

[0097] Specifically, the point cloud data of suspended objects can effectively support the detection, classification and tracking of suspended containers in the port. The detection height of suspended containers can be as low as 20 meters and as high as 30 meters.

[0098] Step S203, determining the velocity components of the suspended object in various directions according to the boundary information of the suspended object, wherein the velocity components at least include a longitudinal velocity component, a lateral velocity component and a vertical velocity component;

[0099] Specifically, according to the boundary information of the suspended object, the velocity components of the suspended object in each direction can be accurately determined, thereby accurately determining the ground projection area of ​​the suspended container.

[0100] Step S204, determining the ground projection area corresponding to the suspended object based on the point cloud data of the suspended object, and determining the driving path of the autonomous driving vehicle based on the velocity components of the suspended object in each direction, the type of the suspended object and the ground projection area.

[0101] Specifically, the autonomous driving vehicle is controlled to bypass the ground projection area of ​​the suspended container, thereby avoiding potential collision risks.

[0102] Optionally, obtaining the three-dimensional coordinates of the suspended object includes: obtaining first image data of the suspended object within a first preset time period, the first image data being acquired by using a high-resolution wide-angle camera; obtaining second image data of the suspended object within the first preset time period, the second image data being acquired by using a fisheye camera; and determining the three-dimensional coordinates of the suspended object based on the first image data and the second image data and using a three-dimensional target detection algorithm.

[0103] Optionally, the point cloud data of the suspended object is determined according to the three-dimensional coordinates of the suspended object, including: according to the three-dimensional coordinates of the suspended object, determining the area including the suspended object as the target area, the volume of the target area is less than or equal to a preset volume; using a laser radar to acquire environmental point cloud data of the target area; according to the three-dimensional coordinates of the suspended object, extracting the point cloud data of the suspended object from the environmental point cloud data.

[0104] Optionally, based on the boundary information of the suspended object, the velocity components of the suspended object in various directions are determined, including: obtaining the boundary information of the suspended object within a second preset time period; determining the moving speed and moving direction of the suspended object based on the boundary information and point cloud data of the suspended object within the second preset time period; determining the velocity component of the suspended object in the forward direction of the autonomous driving vehicle as the longitudinal velocity component, determining the velocity component of the suspended object in the direction perpendicular to the forward direction of the autonomous driving vehicle as the lateral velocity component, and determining the velocity component of the suspended object in the direction perpendicular to the ground as the vertical velocity component according to the moving speed and moving direction of the suspended object.

[0105] Optionally, the driving path of the autonomous driving vehicle is determined according to the velocity components of the suspended object in various directions, the category of the suspended object and the ground projection area, including: when the category of the suspended object is a suspended sling that has not grasped a container, controlling the autonomous driving vehicle to drive according to the original driving path, and the original driving path overlaps or does not overlap with the ground projection area; when the category of the suspended object is a suspended container, updating the driving path of the autonomous driving vehicle according to the velocity components of the suspended container in various directions and the ground projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle, and the updated driving path does not overlap with the ground projection area.

[0106] Optionally, the driving path of the autonomous driving vehicle is updated according to the speed components of the suspended container in various directions and the ground projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle, including: obtaining the ground projection area of ​​the suspended container at the current moment as the current projection area; predicting the predicted projection area of ​​the suspended container at the next moment according to the speed components of the suspended container in various directions and the current projection area of ​​the suspended container; obtaining the current driving speed of the autonomous driving vehicle; updating the driving path of the autonomous driving vehicle according to the current driving speed of the autonomous driving vehicle and the predicted projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle.

[0107] Optionally, the autonomous driving vehicle is equipped with a high-resolution wide-angle camera, a blind spot laser radar and a fisheye camera. The vertical field angle of the high-resolution wide-angle camera is greater than or equal to a first preset angle, and the vertical field angle of the blind spot laser radar and the vertical field angle of the fisheye camera are both greater than or equal to a second preset angle, and the second preset angle is greater than the first preset angle.

[0108] An embodiment of the present invention provides a processor, which is used to run a program, wherein the method for determining the driving path of the autonomous driving vehicle is executed when the program is running.

[0109] Specifically, the method for determining the driving path of the autonomous driving vehicle includes:

[0110] Step S201, obtaining the three-dimensional coordinates of the suspended object, and determining the point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object;

[0111] Specifically, based on the image data generated in real time by the wide-angle camera and the fisheye camera, the area where the suspended container is located in the image is determined by a 3D target detection algorithm, the 3D position of the suspended container in the real world is calculated, and the blind lidar point cloud data of the position is used to confirm the precise position and boundary of the suspended container. The autonomous driving vehicle can be an autonomous driving truck.

[0112] Step S202, determining the category and boundary information of the suspended object at least according to the point cloud data of the suspended object;

[0113] Specifically, the point cloud data of suspended objects can effectively support the detection, classification and tracking of suspended containers in the port. The detection height of suspended containers can be as low as 20 meters and as high as 30 meters.

[0114] Step S203, determining the velocity components of the suspended object in various directions according to the boundary information of the suspended object, wherein the velocity components at least include a longitudinal velocity component, a lateral velocity component and a vertical velocity component;

[0115] Specifically, according to the boundary information of the suspended object, the velocity components of the suspended object in each direction can be accurately determined, thereby accurately determining the ground projection area of ​​the suspended container.

[0116] Step S204, determining the ground projection area corresponding to the suspended object based on the point cloud data of the suspended object, and determining the driving path of the autonomous driving vehicle based on the velocity components of the suspended object in each direction, the type of the suspended object and the ground projection area.

[0117] Specifically, the autonomous driving vehicle is controlled to bypass the ground projection area of ​​the suspended container, thereby avoiding potential collision risks.

[0118] Optionally, obtaining the three-dimensional coordinates of the suspended object includes: obtaining first image data of the suspended object within a first preset time period, the first image data being acquired by using a high-resolution wide-angle camera; obtaining second image data of the suspended object within the first preset time period, the second image data being acquired by using a fisheye camera; and determining the three-dimensional coordinates of the suspended object based on the first image data and the second image data and using a three-dimensional target detection algorithm.

[0119] Optionally, the point cloud data of the suspended object is determined according to the three-dimensional coordinates of the suspended object, including: according to the three-dimensional coordinates of the suspended object, determining the area including the suspended object as the target area, the volume of the target area is less than or equal to a preset volume; using a laser radar to acquire environmental point cloud data of the target area; according to the three-dimensional coordinates of the suspended object, extracting the point cloud data of the suspended object from the environmental point cloud data.

[0120] Optionally, based on the boundary information of the suspended object, the velocity components of the suspended object in various directions are determined, including: obtaining the boundary information of the suspended object within a second preset time period; determining the moving speed and moving direction of the suspended object based on the boundary information and point cloud data of the suspended object within the second preset time period; determining the velocity component of the suspended object in the forward direction of the autonomous driving vehicle as the longitudinal velocity component, determining the velocity component of the suspended object in the direction perpendicular to the forward direction of the autonomous driving vehicle as the lateral velocity component, and determining the velocity component of the suspended object in the direction perpendicular to the ground as the vertical velocity component according to the moving speed and moving direction of the suspended object.

[0121] Optionally, the driving path of the autonomous driving vehicle is determined according to the velocity components of the suspended object in various directions, the category of the suspended object and the ground projection area, including: when the category of the suspended object is a suspended sling that has not grasped a container, controlling the autonomous driving vehicle to drive according to the original driving path, and the original driving path overlaps or does not overlap with the ground projection area; when the category of the suspended object is a suspended container, updating the driving path of the autonomous driving vehicle according to the velocity components of the suspended container in various directions and the ground projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle, and the updated driving path does not overlap with the ground projection area.

[0122] Optionally, the driving path of the autonomous driving vehicle is updated according to the speed components of the suspended container in various directions and the ground projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle, including: obtaining the ground projection area of ​​the suspended container at the current moment as the current projection area; predicting the predicted projection area of ​​the suspended container at the next moment according to the speed components of the suspended container in various directions and the current projection area of ​​the suspended container; obtaining the current driving speed of the autonomous driving vehicle; updating the driving path of the autonomous driving vehicle according to the current driving speed of the autonomous driving vehicle and the predicted projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle.

[0123] Optionally, the autonomous driving vehicle is equipped with a high-resolution wide-angle camera, a blind spot laser radar and a fisheye camera. The vertical field angle of the high-resolution wide-angle camera is greater than or equal to a first preset angle, and the vertical field angle of the blind spot laser radar and the vertical field angle of the fisheye camera are both greater than or equal to a second preset angle, and the second preset angle is greater than the first preset angle.

[0124] An embodiment of the present invention provides a device, the device including a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:

[0125] Step S201, obtaining the three-dimensional coordinates of the suspended object, and determining the point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object;

[0126] Step S202, determining the category and boundary information of the suspended object at least according to the point cloud data of the suspended object;

[0127] Step S203, determining the velocity components of the suspended object in various directions according to the boundary information of the suspended object, wherein the velocity components at least include a longitudinal velocity component, a lateral velocity component and a vertical velocity component;

[0128] Step S204, determining the ground projection area corresponding to the suspended object based on the point cloud data of the suspended object, and determining the driving path of the autonomous driving vehicle based on the velocity components of the suspended object in each direction, the type of the suspended object and the ground projection area.

[0129] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0130] Optionally, obtaining the three-dimensional coordinates of the suspended object includes: obtaining first image data of the suspended object within a first preset time period, the first image data being acquired by using a high-resolution wide-angle camera; obtaining second image data of the suspended object within the first preset time period, the second image data being acquired by using a fisheye camera; and determining the three-dimensional coordinates of the suspended object based on the first image data and the second image data and using a three-dimensional target detection algorithm.

[0131] Optionally, the point cloud data of the suspended object is determined according to the three-dimensional coordinates of the suspended object, including: according to the three-dimensional coordinates of the suspended object, determining the area including the suspended object as the target area, the volume of the target area is less than or equal to a preset volume; using a laser radar to acquire environmental point cloud data of the target area; according to the three-dimensional coordinates of the suspended object, extracting the point cloud data of the suspended object from the environmental point cloud data.

[0132] Optionally, based on the boundary information of the suspended object, the velocity components of the suspended object in various directions are determined, including: obtaining the boundary information of the suspended object within a second preset time period; determining the moving speed and moving direction of the suspended object based on the boundary information and point cloud data of the suspended object within the second preset time period; determining the velocity component of the suspended object in the forward direction of the autonomous driving vehicle as the longitudinal velocity component, determining the velocity component of the suspended object in the direction perpendicular to the forward direction of the autonomous driving vehicle as the lateral velocity component, and determining the velocity component of the suspended object in the direction perpendicular to the ground as the vertical velocity component according to the moving speed and moving direction of the suspended object.

[0133] Optionally, the driving path of the autonomous driving vehicle is determined according to the velocity components of the suspended object in various directions, the category of the suspended object and the ground projection area, including: when the category of the suspended object is a suspended sling that has not grasped a container, controlling the autonomous driving vehicle to drive according to the original driving path, and the original driving path overlaps or does not overlap with the ground projection area; when the category of the suspended object is a suspended container, updating the driving path of the autonomous driving vehicle according to the velocity components of the suspended container in various directions and the ground projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle, and the updated driving path does not overlap with the ground projection area.

[0134] Optionally, the driving path of the autonomous driving vehicle is updated according to the speed components of the suspended container in various directions and the ground projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle, including: obtaining the ground projection area of ​​the suspended container at the current moment as the current projection area; predicting the predicted projection area of ​​the suspended container at the next moment according to the speed components of the suspended container in various directions and the current projection area of ​​the suspended container; obtaining the current driving speed of the autonomous driving vehicle; updating the driving path of the autonomous driving vehicle according to the current driving speed of the autonomous driving vehicle and the predicted projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle.

[0135] Optionally, the autonomous driving vehicle is equipped with a high-resolution wide-angle camera, a blind spot laser radar and a fisheye camera. The vertical field angle of the high-resolution wide-angle camera is greater than or equal to a first preset angle, and the vertical field angle of the blind spot laser radar and the vertical field angle of the fisheye camera are both greater than or equal to a second preset angle, and the second preset angle is greater than the first preset angle.

[0136] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0137] Step S201, obtaining the three-dimensional coordinates of the suspended object, and determining the point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object;

[0138] Step S202, determining the category and boundary information of the suspended object at least according to the point cloud data of the suspended object;

[0139] Step S203, determining the velocity components of the suspended object in various directions according to the boundary information of the suspended object, wherein the velocity components at least include a longitudinal velocity component, a lateral velocity component and a vertical velocity component;

[0140] Step S204, determining the ground projection area corresponding to the suspended object based on the point cloud data of the suspended object, and determining the driving path of the autonomous driving vehicle based on the velocity components of the suspended object in each direction, the type of the suspended object and the ground projection area.

[0141] Optionally, obtaining the three-dimensional coordinates of the suspended object includes: obtaining first image data of the suspended object within a first preset time period, the first image data being acquired by using a high-resolution wide-angle camera; obtaining second image data of the suspended object within the first preset time period, the second image data being acquired by using a fisheye camera; and determining the three-dimensional coordinates of the suspended object based on the first image data and the second image data and using a three-dimensional target detection algorithm.

[0142] Optionally, the point cloud data of the suspended object is determined according to the three-dimensional coordinates of the suspended object, including: according to the three-dimensional coordinates of the suspended object, determining the area including the suspended object as the target area, the volume of the target area is less than or equal to a preset volume; using a laser radar to acquire environmental point cloud data of the target area; according to the three-dimensional coordinates of the suspended object, extracting the point cloud data of the suspended object from the environmental point cloud data.

[0143] Optionally, based on the boundary information of the suspended object, the velocity components of the suspended object in various directions are determined, including: obtaining the boundary information of the suspended object within a second preset time period; determining the moving speed and moving direction of the suspended object based on the boundary information and point cloud data of the suspended object within the second preset time period; determining the velocity component of the suspended object in the forward direction of the autonomous driving vehicle as the longitudinal velocity component, determining the velocity component of the suspended object in the direction perpendicular to the forward direction of the autonomous driving vehicle as the lateral velocity component, and determining the velocity component of the suspended object in the direction perpendicular to the ground as the vertical velocity component according to the moving speed and moving direction of the suspended object.

[0144] Optionally, the driving path of the autonomous driving vehicle is determined according to the velocity components of the suspended object in various directions, the category of the suspended object and the ground projection area, including: when the category of the suspended object is a suspended sling that has not grasped a container, controlling the autonomous driving vehicle to drive according to the original driving path, and the original driving path overlaps or does not overlap with the ground projection area; when the category of the suspended object is a suspended container, updating the driving path of the autonomous driving vehicle according to the velocity components of the suspended container in various directions and the ground projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle, and the updated driving path does not overlap with the ground projection area.

[0145] Optionally, the driving path of the autonomous driving vehicle is updated according to the speed components of the suspended container in various directions and the ground projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle, including: obtaining the ground projection area of ​​the suspended container at the current moment as the current projection area; predicting the predicted projection area of ​​the suspended container at the next moment according to the speed components of the suspended container in various directions and the current projection area of ​​the suspended container; obtaining the current driving speed of the autonomous driving vehicle; updating the driving path of the autonomous driving vehicle according to the current driving speed of the autonomous driving vehicle and the predicted projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle.

[0146] Optionally, the autonomous driving vehicle is equipped with a high-resolution wide-angle camera, a blind spot laser radar and a fisheye camera. The vertical field angle of the high-resolution wide-angle camera is greater than or equal to a first preset angle, and the vertical field angle of the blind spot laser radar and the vertical field angle of the fisheye camera are both greater than or equal to a second preset angle, and the second preset angle is greater than the first preset angle.

[0147] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0148] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0149] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0150] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0151] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0152] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0153] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0154] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0155] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0156] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0157] 1) The method for determining the driving path of the above-mentioned autonomous driving vehicle of the present application first obtains the three-dimensional coordinates of the suspended object, and determines the point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object; then determines the category and boundary information of the suspended object at least according to the point cloud data of the suspended object; then determines the velocity components of the suspended object in each direction according to the boundary information of the suspended object, and the velocity components at least include the longitudinal velocity component, the lateral velocity component and the vertical velocity component; finally, determines the ground projection area corresponding to the suspended object according to the point cloud data of the suspended object, and determines the driving path of the autonomous driving vehicle according to the velocity components of the suspended object in each direction, the category of the suspended object and the ground projection area. The method configures the autonomous driving truck with on-board sensors such as blind laser radar, high-resolution wide-angle camera, fisheye camera, etc., detects, classifies and tracks the suspended container through the above sensors, and controls the autonomous driving vehicle to bypass the ground projection area of ​​the suspended container according to the above perception results, so as to solve the problem that it is difficult to avoid the collision risk of the autonomous driving vehicle with obstacles such as the suspended container in the prior art.

[0158] 2) The device for determining the driving path of the above-mentioned autonomous driving vehicle of the present application includes an acquisition unit, a first determination unit, a second determination unit and a third determination unit. The acquisition unit is used to acquire the three-dimensional coordinates of the suspended object, and determine the point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object; the first determination unit is used to determine the category and boundary information of the suspended object at least according to the point cloud data of the suspended object; the second determination unit is used to determine the velocity components of the suspended object in various directions according to the boundary information of the suspended object, and the velocity components include at least longitudinal velocity components, lateral velocity components and vertical velocity components; the third determination unit is used to determine the ground projection area corresponding to the suspended object according to the point cloud data of the suspended object, and determine the driving path of the autonomous driving vehicle according to the velocity components of the suspended object in various directions, the category of the suspended object and the ground projection area. The device equips the self-driving truck with on-board sensors such as blind spot laser radar, high-resolution wide-angle camera, and fisheye camera. It detects, classifies, and tracks suspended containers through the above sensors, and controls the self-driving vehicle to bypass the ground projection area of ​​the suspended containers based on the above perception results, solving the problem that it is difficult to avoid the risk of collision between the self-driving vehicle and obstacles such as suspended containers in the existing technology.

[0159] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining a driving path of an autonomous driving vehicle, characterized in that: include: Acquire the three-dimensional coordinates of the suspended object, and determine the point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object; Determining the category and boundary information of the suspended object at least according to the point cloud data of the suspended object; Determine, according to the boundary information of the suspended object, the velocity components of the suspended object in each direction, wherein the velocity components at least include a longitudinal velocity component, a transverse velocity component, and a vertical velocity component; According to the point cloud data of the suspended object, the ground projection area corresponding to the suspended object is determined, and according to the velocity components of the suspended object in each direction, the type of the suspended object and the ground projection area, the driving path of the autonomous driving vehicle is determined.

2. The method according to claim 1, characterized in that Get the three-dimensional coordinates of the suspended object, including: Acquire first image data of the suspended object within a first preset time period, where the first image data is acquired by using a high-resolution wide-angle camera; Acquire second image data of the suspended object within the first preset time period, where the second image data is acquired by using a fisheye camera; The three-dimensional coordinates of the suspended object are determined according to the first image data and the second image data and by using a three-dimensional target detection algorithm.

3. The method according to claim 1, characterized in that Determining point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object includes: According to the three-dimensional coordinates of the suspended object, a region including the suspended object is determined as a target region, wherein a volume of the target region is less than or equal to a preset volume; Using laser radar to obtain environmental point cloud data of the target area; According to the three-dimensional coordinates of the suspended object, point cloud data of the suspended object is extracted from the environmental point cloud data.

4. The method according to claim 1, characterized in that Determining the velocity components of the suspended object in each direction according to the boundary information of the suspended object includes: Acquiring boundary information of the suspended object within a second preset time period; Determining a moving speed and a moving direction of the suspended object according to boundary information and point cloud data of the suspended object within the second preset time period; According to the moving speed and moving direction of the suspended object, the velocity component of the suspended object in the forward direction of the autonomous driving vehicle is determined as the longitudinal velocity component, the velocity component of the suspended object in the direction perpendicular to the forward direction of the autonomous driving vehicle is determined as the lateral velocity component, and the velocity component of the suspended object in the direction perpendicular to the ground is determined as the vertical velocity component.

5. The method according to claim 1, characterized in that Determining a driving path of the autonomous driving vehicle according to the velocity components of the suspended object in each direction, the type of the suspended object and the ground projection area includes: In a case where the category of the suspended object is a suspended spreader that does not grab a container, controlling the automatic driving vehicle to travel according to an original driving path, wherein the original driving path overlaps or does not overlap with the ground projection area; In the case where the category of the suspended object is a suspended container, the driving path of the autonomous driving vehicle is updated according to the velocity components of the suspended container in each direction and the ground projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle, and the updated driving path does not overlap with the ground projection area.

6. The method according to claim 5, characterized in that The method updates the driving path of the autonomous driving vehicle according to the velocity components of the suspended container in each direction and the ground projection area of ​​the suspended container to obtain an updated driving path of the autonomous driving vehicle, including: The ground projection area of ​​the suspended container at the current moment is obtained as the current projection area; Predicting a predicted projection area of ​​the suspended container at the next moment according to the velocity components of the suspended container in each direction and the current projection area of ​​the suspended container; Obtaining the current driving speed of the autonomous driving vehicle; According to the current driving speed of the autonomous driving vehicle and the predicted projection area of ​​the suspended container, the driving path of the autonomous driving vehicle is updated to obtain an updated driving path of the autonomous driving vehicle.

7. The method according to claim 1, characterized in that The autonomous driving vehicle is equipped with a high-resolution wide-angle camera, a blind spot laser radar and a fisheye camera. The vertical field of view of the high-resolution wide-angle camera is greater than or equal to a first preset angle, and the vertical field of view of the blind spot laser radar and the vertical field of view of the fisheye camera are both greater than or equal to a second preset angle, and the second preset angle is greater than the first preset angle.

8. A device for determining a driving path of an autonomous driving vehicle, characterized in that: include: An acquisition unit, used for acquiring the three-dimensional coordinates of the suspended object, and determining the point cloud data of the suspended object according to the three-dimensional coordinates of the suspended object; A first determining unit, configured to determine the category and boundary information of the suspended object at least according to the point cloud data of the suspended object; A second determining unit, configured to determine the velocity components of the suspended object in various directions according to the boundary information of the suspended object, wherein the velocity components at least include a longitudinal velocity component, a transverse velocity component, and a vertical velocity component; The third determination unit is used to determine the ground projection area corresponding to the suspended object based on the point cloud data of the suspended object, and determine the driving path of the autonomous driving vehicle based on the velocity components of the suspended object in each direction, the category of the suspended object and the ground projection area.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for determining the driving path of the autonomous driving vehicle as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for determining a driving path of an autonomous driving vehicle as described in any one of claims 1 to 7.

Images