A path optimization method and related apparatus

Abstract

A path optimization method is applied to the technical field of robots or the technical field of automatic driving. In the method, the motion state of each sampling position point of a driving device is obtained; the motion state is represented based on a plurality of flat variables or derivatives of the flat variables; the motion state of at least one sampling position point comprises a curvature and a curvature change rate; a target penalty function is determined according to the motion state information; the target penalty function is related to the driving feasibility and smoothness of the driving device when driving along the plurality of sampling position points; and the driving path is optimized according to the target penalty function. The present application can avoid solving differential equations in the trajectory optimization process, thereby reducing the difficulty of trajectory optimization, and directly describing the nonlinear constraint terms such as curvature and its change rate in the flat variable space, which can avoid repeated discretization of displacement or time difference.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of path optimization, in particular to a path optimization method and related device. BACKGROUND

[0002] Path optimization technology refers to that a driving device (for example, a robot) perceives a driving environment according to a sensor of the driving device, and automatically plans a safe motion path from a current point to a target point. In addition to positioning and mapping, path optimization is a key function of any autonomous navigation system, and mainly solves three problems: 1, connectivity, that is, the path must be able to connect the starting point and the ending point, and ensure safe obstacle avoidance throughout the journey; 2, dynamics and kinematics feasibility, that is, the path meets the constraints of the motion of the driving device and the constraints of dynamics, such as the speed of the driving device should not exceed the upper limit of the speed; 3, optimality, that is, under the premise of achieving the above two, the length of the path, the travel time and the energy consumption of the driving device should be optimized as much as possible.

[0003] In the prior art, a predicted driving path is optimized by using a penalty function, wherein the construction of the penalty function is related to information such as the curvature and the rate of change of the curvature of the path, wherein the curvature is determined by the linear velocity and the angular velocity, and belongs to a nonlinear constraint (involving arctan, sqrt, and exponential operation), and the rate of change of the curvature is more nonlinear and has a higher order (numerical operation involves jerk, 5th power of speed, etc., and values in the range of 10^10 often appear, which easily causes gradient explosion), resulting in high complexity of modeling of the rate of change of the curvature in terms of reduction degree and numerical stability.

[0004] Therefore, there is an urgent need for a more efficient and accurate path optimization method. SUMMARY

[0005] The first aspect of the present application provides a path optimization method, which is applied to a driving device. The method specifically comprises: obtaining motion state information of the driving device and a driving path to be optimized, the driving path comprising a plurality of sampling position points; the motion state information representing the motion state of each sampling position point; wherein the motion state is represented based on a plurality of flat variables or derivatives of the flat variables; each flat variable corresponds to the position of one dimension of the sampling position point; the motion state of at least one sampling position point comprises curvature and rate of change of the curvature; determining a target penalty function according to the motion state information; the target penalty function is related to the driving feasibility and smoothness of the driving device when driving along the plurality of sampling position points; wherein the driving feasibility is related to the curvature, and the smoothness is related to the curvature and the rate of change of the curvature; and optimizing the driving path according to the target penalty function.

[0006] For example, the drivability can be related to the curvature (e.g., a turning radius constraint can be constructed according to the curvature), and the smoothness can be related to the curvature and the rate of change of the curvature (e.g., the curvature itself is a kind of representation of the smoothness, and the rate of change of the curvature can be used to construct an angular acceleration constraint).

[0007] The application expresses the motion state through flat variables, which can avoid solving differential equations in the trajectory optimization process, and can reduce the dimension of the expression space, thereby reducing the difficulty of trajectory optimization. In addition, directly describing the nonlinear constraint terms such as curvature and its rate of change in the flat variable space can avoid repeated discretization of displacement or time difference, so as to reasonably constrain the curvature jump while making the overall steering consistency as high as possible and the amplitude as small as possible.

[0008] In a possible implementation, the optimization of the driving path according to the motion state information includes: determining a target penalty function according to the motion state information; the target penalty function is related to the driving performance of the driving device when driving along the plurality of sampling position points; and the driving path is optimized according to the target penalty function; wherein the target of the optimization of the driving path is to reduce the value of the target penalty function.

[0009] In a possible implementation, the obtaining of the motion state information of the driving device includes: obtaining initial motion state information of the driving device, the initial motion state information being expressed in a first space; and changing the expression space of the initial motion state information from the first space to a second space to obtain the motion state information, the second space having a dimension less than or equal to that of the first space, and each dimension of the second space corresponding to a flat variable or a derivative of a flat variable.

[0010] When the dimension of the second space is less than that of the first space, the number of variables used to represent the motion state is reduced, thereby reducing the dimension of the expression space and the difficulty of optimization.

[0011] In a possible implementation, the motion state is at least one of a speed, an acceleration, an angular velocity, an angular acceleration, a curvature, and a rate of change of the curvature.

[0012] In a possible implementation, the driving path is represented by a high-order polynomial with respect to a time variable, and the optimization of the driving path includes: optimizing the time variable and polynomial coefficients of the high-order polynomial.

[0013] In the existing implementation, a plurality of sampling points can be sampled for a driving path to be optimized, the driving path is represented by the positions of the sampling points, and when the path is optimized, the position of each sampling point is optimized. On the one hand, in order to ensure the optimization accuracy, the number of sampling points cannot be too small, and therefore the number of points to be optimized is also large, resulting in large computational overhead. On the other hand, the position of the sampling point is only an approximate expression of the path, and even if the number of sampling points is large, the optimization accuracy and effect of the path are still limited.

[0014] In the embodiments of the present application, the driving path to be optimized is represented by a high-order polynomial with respect to the time variable, and when the driving path is optimized, the time variable and the polynomial coefficients are optimized. Since the number of optimization objects is reduced, the computational overhead during optimization can be reduced, and the shape of the driving path can be more accurately represented by the high-order polynomial, thereby improving the optimization accuracy and effect.

[0015] In a possible implementation, the plurality of sampling points includes a first sampling point and a second sampling point, the corresponding curvature or curvature change rate of the first sampling point in the driving path is greater than that of the second sampling point, the first sampling point is obtained by sampling the driving path according to a first sampling resolution, the second sampling point is obtained by sampling the driving path according to a second sampling resolution, and the first sampling resolution is greater than the second sampling resolution.

[0016] In a possible implementation, each trajectory segment can be sampled according to the variable change rate, so that the sampling number of points with large curvature or curvature change rate is larger. Since the part with large curvature or curvature change rate is the object that needs to be focused on, the effect of subsequent optimization can be improved by the above-mentioned variable change rate sampling.

[0017] In a possible implementation, the driving path to be optimized is a path selected from a planned path of the driving device, and the two ends of the driving path are obtained by sampling the planned path based on the curvature and / or curvature change rate being less than a threshold.

[0018] When the driving trajectory is optimized, in order to ensure the optimization effect, the end point (which can also be referred to as an anchor point in the embodiments of the present application) of the segmented path cannot be a point with a curvature or curvature change rate greater than a threshold. The reason is that the end point is often not adjusted in the optimization algorithm.

[0019] In a possible implementation, the driving path includes a sampling point with a curvature and / or curvature change rate greater than a threshold.

[0020] In order to ensure the optimization effect, the positions with large curvature or large curvature change rate in the driving trajectory need to be adjusted when the driving trajectory is optimized. Therefore, at least one sampling position point (which can also be referred to as a bad point in the embodiments of the present application) with large curvature or large curvature change rate is included in each segmented path (including the driving path to be optimized in the embodiments of the present application) when the initial path is segmented.

[0021] In a possible implementation, the target penalty function is negatively related to the driving feasibility and / or smoothness of the driving device when driving along the plurality of sampling position points.

[0022] In a possible implementation, the method further includes: obtaining point cloud data of a driving environment where the plurality of sampling position points are located, the point cloud data being used to represent obstacles in the driving environment; obtaining collision risk information of the driving device at each sampling position point according to interaction of each point cloud in the point cloud data and an Euclidean distance field ESDF of the driving device; and optimizing the driving path according to the motion state information, including: optimizing the driving path according to the motion state information and the collision risk information.

[0023] In a possible implementation, the obtaining of the collision risk information of the driving device at each sampling position point according to the interaction of each point cloud in the point cloud data and the ESDF of the driving device includes: obtaining the collision risk information of the driving device at each sampling position point according to an ESDF gradient determined based on an ESDF value of a corresponding position of each point cloud in the point cloud data in the ESDF and ESDF values of nearby positions of the corresponding position.

[0024] In a possible implementation, the driving device includes a robot or a vehicle.

[0025] In a second aspect, the present application provides a path optimization device, including:

[0026] The obtaining module is configured to obtain motion state information of a driving device and a driving path to be optimized, the driving path including a plurality of sampling position points; the motion state information representing a motion state of each sampling position point; wherein the motion state is represented based on a plurality of flat variables or derivatives of the flat variables; each flat variable corresponding to a position in one dimension of the sampling position point; and the motion state of at least one sampling position point including curvature and curvature change rate.

[0027] The processing module is configured to determine a target penalty function according to the motion state information, the target penalty function being related to a drivability and a smoothness of the driving device when driving along the plurality of sampling position points, the drivability being related to the curvature, and the smoothness being related to the curvature and the curvature change rate.

[0028] The driving path is optimized according to the target penalty function.

[0029] In a possible implementation, the target of optimizing the driving path is to reduce the value of the target penalty function.

[0030] In a possible implementation, the processing module is specifically configured to:

[0031] determine a target penalty function according to the motion state information, the target penalty function being related to a driving performance of the driving device when driving along the plurality of sampling position points;

[0032] optimize the driving path according to the target penalty function, wherein the target of optimizing the driving path is to reduce the value of the target penalty function.

[0033] In a possible implementation, the obtaining module is specifically configured to:

[0034] obtain initial motion state information of the driving device, the initial motion state information being expressed in a first space;

[0035] convert the expression space of the initial motion state information from the first space to a second space to obtain the motion state information, the second space having a dimension less than or equal to that of the first space, and each dimension of the second space corresponding to a flat variable or a derivative of a flat variable.

[0036] In a possible implementation, the motion state is at least one of a speed, an acceleration, an angular speed, an angular acceleration, a curvature, and a curvature change rate.

[0037] In a possible implementation, the driving path is represented by a high-order polynomial with respect to a time variable, and the optimization of the driving path includes optimizing the time variable and polynomial coefficients of the high-order polynomial.

[0038] In a possible implementation, the plurality of sampling position points include a first sampling point and a second sampling point, the curvature or the curvature change rate corresponding to the first sampling point in the driving path being greater than that of the second sampling point, the first sampling point being obtained by sampling the driving path according to a first sampling resolution, and the second sampling point being obtained by sampling the driving path according to a second sampling resolution, the first sampling resolution being greater than the second sampling resolution.

[0039] In a possible implementation, the driving path to be optimized is a segment of path selected from a planned path of the driving device, and two ends of the driving path are sampled from the planned path based on a curvature and / or a curvature rate being less than a threshold.

[0040] In a possible implementation, the driving path includes a sampling point with a curvature and / or a curvature rate greater than a threshold.

[0041] In a possible implementation, the target penalty function is negatively related to driving feasibility and / or smoothness of the driving device when driving along the plurality of sampling position points.

[0042] In a possible implementation, the obtaining module is further configured to:

[0043] obtain point cloud data of a driving environment in which the driving device is located at the plurality of sampling position points, the point cloud data being used to represent obstacles in the driving environment;

[0044] The processing module is further configured to: obtain collision risk information of the driving device at each of the sampling position points according to interaction of each point cloud in the point cloud data and an Euclidean distance field (ESDF) of the driving device.

[0045] The processing module is specifically configured to:

[0046] optimize the driving path according to the motion state information and the collision risk information.

[0047] In a possible implementation, the processing module is specifically configured to:

[0048] obtain the collision risk information of the driving device at each of the sampling position points according to an ESDF value of a corresponding position of each point cloud in the point cloud data in the ESDF and an ESDF gradient determined according to the ESDF value of the corresponding position and ESDF values of positions near the corresponding position.

[0049] In a possible implementation, the driving device includes a robot or a vehicle.

[0050] The third aspect of the present application provides a driving device, which can include a processor, the processor being coupled to a memory, and the memory storing program instructions, when the program instructions stored in the memory are executed by the processor, the method of the first aspect or any implementation manner of the first aspect is implemented. For the processor to execute the steps in each possible implementation manner of the first aspect, specific details can be referred to the first aspect, which will not be repeated here.

[0051] The fourth aspect of the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, when the computer program is run on a computer, the computer executes the method of the first aspect or any implementation manner of the first aspect.

[0052] The fifth aspect of the present application provides a circuit system, comprising a processing circuit configured to perform the method of the first aspect or any implementation manner of the first aspect.

[0053] The sixth aspect of the present application provides a computer program product, which, when running on a computer, causes the computer to perform the method of the first aspect or any implementation manner of the first aspect.

[0054] The seventh aspect of the present application provides a chip system, comprising a processor for supporting a server or a threshold value obtaining device to implement the functions involved in the first aspect or any implementation manner of the first aspect, for example, sending or processing the data and / or information involved in the above method. In a possible design, the chip system further comprises a memory for storing necessary program instructions and data of the server or the communication device. The chip system can be composed of a chip, or can include the chip and other discrete devices.

[0055] The beneficial effects of the above-mentioned second aspect to the seventh aspect can refer to the introduction of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 A schematic diagram of a power transmission industry inspection robot provided by an embodiment of the present application;

[0057] Figure 2 A schematic diagram of a mine car running on a mine provided by an embodiment of the present application;

[0058] Figure 3 A structural schematic diagram of a robot 500 provided by an embodiment of the present application;

[0059] Figure 4 A structural schematic diagram of a vehicle 100 provided by an embodiment of the present application;

[0060] Figure 5 A flowchart of a path optimization method provided by an embodiment of the present application;

[0061] Figure 6 A structural schematic diagram of a driving device provided by an embodiment of the present application;

[0062] Figure 7 A structural schematic diagram of a driving device provided by an embodiment of the present application;

[0063] Figure 8 A structural schematic diagram of a chip provided by an embodiment of the present application;

[0064] Figure 9A structural schematic diagram of a computer readable storage medium provided by an embodiment of the present application. DETAILED DESCRIPTION

[0065] The embodiments of the present application will be described below with reference to the accompanying drawings. It is obvious to those skilled in the art that, as technology develops and new scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0066] The terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is merely a distinguishing way used in the description of the embodiments of the present application to describe the objects with the same attributes. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that the processes, methods, systems, products or devices containing a series of units do not have to be limited to those units, but can include other units that are not clearly listed or inherent to these processes, methods, products or devices.

[0067] In order to facilitate understanding, the technical terms related to the embodiments of the present application will be introduced first.

[0068] (1) Sign Distance Function (SDF)

[0069] SDF can also be referred to as an oriented distance function, which determines the distance of a point to the boundary of a region in space and defines the sign of the distance at the same time. If the point is inside the boundary of the region, the sign of the distance is positive; if the point is outside the boundary of the region, the sign of the distance is negative; if the point is on the boundary of the region, the distance is 0.

[0070] (2) Euclidean Space

[0071] Euclidean space, also known as Euclidean space, is a generalization of the two-dimensional and three-dimensional space studied by Euclid in mathematics. This generalization converts Euclid's concepts of distance, length and angle into a coordinate system of any number of dimensions.

[0072] (3) Euclidea Signed Distance Field (ESDF)

[0073] ESDF is the result of applying SDF in Euclidean space, representing the shortest distance of a point in Euclidean space to the surrounding obstacles.

[0074] (4) Truncated Signed Distance Function (TSDF)

[0075] TSDF is a penalty function obtained by adding a truncated zone to the SDF. Similarly, it determines the distance of a point to the boundary of a region in space and defines the sign of the distance at the same time. If the distance of the point to the boundary of the region is closer, the value of the TSDF is larger; if the distance of the point to the boundary of the region is farther, the value of the TSDF is smaller; if the distance of the point to the boundary of the region exceeds the value of the truncated zone (i.e. a certain threshold), the value of the TSDF is 0.

[0076] (5) Penalty Function

[0077] The penalty function refers to adding an obstacle function to the original objective function when solving an optimization problem (linear constraint optimization and nonlinear constraint optimization), thereby obtaining an augmented objective function. The function of the penalty function is to assign a very large value to a non-feasible point or a point that attempts to cross the boundary to escape the feasible region, i.e. to convert the constrained optimization problem into an unconstrained optimization problem.

[0078] (6) RANdom Sample Consensus (RANSAC)

[0079] RANSAC is an algorithm for calculating the mathematical model parameters of data from a sample data set containing abnormal data, and obtaining effective sample data.

[0080] (7) Bounding Box

[0081] A bounding box is an algorithm for solving the optimal bounding space of a discrete point set. The basic idea is to use a geometric body (called a bounding box) with slightly larger volume and simple characteristics to approximately replace a complex geometric object.

[0082] Common bounding box algorithms include Axis-aligned Bounding Box (AABB) bounding box, Oriented Bounding Box (OBB) bounding sphere, and Fixed Directions Hulls (FDH). Collision detection problems have wide applications in virtual reality, computer-aided design and manufacturing, games, and robots, and even become key technologies. The bounding box algorithm is one of the important methods for preliminary detection of collision interference.

[0083] (8) Point Cloud Data

[0084] Point cloud data refers to a set of vectors in a three-dimensional coordinate system, which can accurately reflect the true situation of the ground or object, such as ground state, ground feature reflection, etc. Generally, point cloud data is obtained by emitting laser from a laser radar to the object to be scanned, and then receiving the laser reflected by the object in the environment. The data obtained by the laser radar scanning the object is recorded in the form of points, and each point contains three-dimensional coordinates, and some may contain color information (RGB) or intensity information (Intensity).

[0085] (9) Laser Radar

[0086] Laser radar is a radar system that uses laser beams to detect the position, speed, and other characteristic quantities of a target. The working principle of laser radar is to emit a detection signal (laser beam) to the target, then compare and process the received signal (target echo) reflected from the target with the transmitted signal, and then obtain the relevant information of the target, such as target distance, direction, height, speed, attitude, and even shape, etc. parameters, so as to realize the detection of the ground or obstacles on the ground. Generally, laser radar can be composed of laser transmitter, optical receiver and information processing system, etc. The laser transmitter converts electrical pulses into optical pulses and transmits them, and the optical receiver restores the optical pulses reflected from the target into electrical pulses and sends them to the information processing system for processing.

[0087] (10) Inertial sensor

[0088] Inertial sensor is a kind of sensor, also known as inertial measurement unit (IMU). Inertial sensor is mainly used for detecting and measuring acceleration, rotation, inclination, impact, vibration and multi-degree-of-freedom (DoF) motion, and is an important component for solving navigation, orientation and motion carrier control. The principle of inertial sensor is realized by using inertial law, mainly including accelerometer and angular velocity meter (gyroscope), which can measure the acceleration and angular velocity of an object in three-dimensional coordinate system.

[0089] (11) Signal-to-noise ratio

[0090] Signal-to-noise ratio refers to the ratio of signal to noise in an electronic device or electronic system. The signal here refers to the electronic signal from the outside of the device that needs to be processed by the device, and the noise refers to the irregular additional signal (or information) that does not exist in the original signal after passing through the device, and this kind of signal does not change with the change of the original signal.

[0091] (12) Pose

[0092] Pose is the description of the position and orientation of an object in a specified coordinate system. The position refers to the location of the object in space, which can be represented by a 3x1 matrix for a rigid body, i.e., the position of the rigid body in the three-dimensional coordinate system. The orientation refers to the orientation of the object in space, which can also be represented by a 3x3 matrix for a rigid body, i.e., the orientation of the rigid body coordinate system in the base coordinate system.

[0093] (13) Grader

[0094] A grader is a kind of earth-moving machinery that uses a blade to level the ground. The blade is mounted between the front and rear axles of the machine and can be raised, tilted, rotated, and extended. It moves flexibly and accurately, and is easy to operate, providing high precision in leveling the ground. It is suitable for constructing roadbeds and ground surfaces, building slopes, excavating ditches, and can also be used for mixing ground materials, removing snow, pushing bulk materials, and maintaining soil and gravel roads.

[0095] (14) Elevation map

[0096] An elevation map refers to a map that represents the height (i.e., elevation) of a certain area.

[0097] (15) Covariance

[0098] Covariance is used in probability theory and statistics to measure the overall error of two variables. Variance is a special case of covariance, i.e., when the two variables are the same.

[0099] Covariance represents the overall error of two variables, which is different from variance, which only represents the error of one variable. If the trend of the change of the two variables is consistent, that is, if one is greater than its expected value, the other is also greater than its expected value, then the covariance between the two variables is positive. If the trend of the change of the two variables is opposite, that is, if one is greater than its expected value, the other is less than its expected value, then the covariance between the two variables is negative.

[0100] (16) Variance

[0101] Variance is a measure of the degree of dispersion of a random variable or a set of data in probability theory and statistics. In probability theory, variance is used to measure the degree of deviation between a random variable and its mathematical expectation (i.e., mean). In statistics, the variance (sample variance) is the average of the squared differences between each sample value and the average of all sample values. In many practical problems, studying variance, i.e., the degree of deviation, is of great significance.

[0102] (17) Curvature (kappa): Curvature is defined as the derivative of the unit tangent vector function, which is the rate of rotation of the tangent direction angle with respect to the arc length at a certain point on the curve. It is defined by differentiation and indicates the degree of deviation of the curve from a straight line.

[0103] (18) Rate of change of curvature (dkappa): Derivative of curvature.

[0104] In the embodiment, the path optimization method can be applied to a traveling device, such as a robot or a vehicle. Specifically, the robot is, for example, an indoor robot (such as a sweeping robot, a household robot, a welcome robot, etc.), a patrol robot in the power transmission industry (such as a patrol robot inside a substation), a carrying robot or a patrol robot in the coal mine industry, a carrying robot or an underground exploration robot in the logistics industry, etc. For these robots, the environment in which the robot travels is, for example, an indoor environment with irregular non-convex obstacles, or an outdoor environment with obstacles or uneven terrain. For example, please refer to Figure 1 , Figure 1 A schematic diagram of a patrol robot in the power transmission industry provided in the embodiment of the present application.

[0105] The vehicle is, for example, a mine car traveling on a mine, a truck traveling on a construction site, and a carrying vehicle traveling in a warehouse, etc. The environment in which these vehicles travel is prone to have uneven ground or obstacles. Therefore, in order to ensure the normal travel of the vehicle, the travel path of the vehicle needs to be planned to ensure that the vehicle can avoid obstacles. For example, please refer to Figure 2 , Figure 2 A schematic diagram of a mine car traveling on a mine provided in the embodiment of the present application.

[0106] In order to facilitate understanding of the present solution, the structure of the robot and the vehicle provided in the present application is introduced in the embodiment of the present application in combination with Figure 3 and Figure 4 .

[0107] Please refer to Figure 3 , Figure 3 A schematic diagram of the structure of a robot 500 provided in the embodiment of the present application. As shown in Figure 3 , the robot 500 can include an image acquisition module 501, a sensor 502, a processor 503, a memory 504, and a communication module 505.

[0108] The processor 503 can include one or more processing units, such as: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated in one or more processors.

[0109] The controller can be the nerve center and command center of the robot 500. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of instruction fetching and instruction execution.

[0110] The processor 503 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. The memory can save instructions or data that the processor has just used or repeatedly uses. If the processor needs to use the instructions or data again, it can directly call from the memory. This avoids repeated access and reduces the waiting time of the processor 503, thereby improving the efficiency of the system.

[0111] In some embodiments, the processor 503 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0112] The interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor can contain multiple sets of I2C buses. The processor can be coupled to sensors, cameras, etc. through different I2C bus interfaces, respectively.

[0113] The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is usually used to connect the processor 503 and the communication module 505. For example, the processor 503 communicates with the Bluetooth module in the wireless communication module through the UART interface to realize the Bluetooth function.

[0114] The IPI interface can be used to connect the processor and peripheral devices such as cameras. The MIPI interface includes a camera serial interface (CSI) and the like. In some embodiments, the processor and the camera communicate through the CSI interface to realize the shooting function of the robot 500.

[0115] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor and the camera, the wireless communication module, the sensor module, etc. The GPIO interface can also be configured as an I2C interface, a UART interface, a MIPI interface, etc.

[0116] The USB interface is an interface that meets the USB standard specification, which can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface can be used to connect a charger to charge the robot 500, or to transmit data between the robot 500 and peripheral devices. The interface can also be used to connect other robots 500, etc.

[0117] The image acquisition module 501 can acquire image information around the robot 500, such as taking photos or shooting videos, etc. The robot 500 can realize the image acquisition function through an ISP, a camera, a video codec, a GPU, and an application processor, etc.

[0118] ISP is used to process data fed back by the camera. For example, when taking a photo, the shutter is opened, light is transmitted through the lens to the camera photosensitive element, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and conversion into a visible image. ISP can also optimize the noise, brightness, etc. of the image. ISP can also optimize the exposure, color temperature, etc. of the shooting scene. In some embodiments, ISP can be provided in the camera.

[0119] The camera is used to capture still images or videos. Objects generate optical images through the lens and project onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into a standard RGB, YUV, etc. format image signal. In some embodiments, the robot 500 can include one or N cameras, where N is a positive integer greater than 1.

[0120] The sensor 502 can obtain information such as the moving speed, moving direction, and distance from surrounding objects of the robot 500. For example, the sensor 502 can include a gyroscope sensor, a speed sensor, an acceleration sensor, a distance sensor, etc.

[0121] The gyroscope sensor can be used to determine the motion posture of the robot 500. In some embodiments, the angular velocity of the robot 500 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor. The gyroscope sensor can be used for anti-shake. For example, when the robot 500 is collecting images, the gyroscope sensor detects the angle of the robot 500 shaking, calculates the distance that the lens module needs to compensate according to the angle, and lets the lens offset the shaking of the robot 500 by reverse movement to achieve anti-shake. The gyroscope sensor can also be used for navigation or calculating the concave-convex situation of the ground, determining whether the robot 500 is trapped, etc.

[0122] The speed sensor is used to measure the moving speed. In some embodiments, the robot 500 measures the moving speed at the current time through the speed sensor, which can be combined with the distance sensor to predict the environment where the robot 500 is located at the next time, etc.

[0123] The acceleration sensor can detect the magnitude of acceleration of the robot 500 in various directions (generally three axes). When the robot 500 is stationary, the magnitude and direction of gravity can be detected.

[0124] Distance sensor, for measuring distance. The robot 500 can measure distance by infrared or laser. In some embodiments, the scene is photographed, and the robot 500 can utilize the distance sensor to measure distance to achieve fast focusing.

[0125] The memory 504 can include an external memory and an internal memory. An external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the robot 500. The external memory card communicates with the processor through the external memory interface to achieve data storage functions. For example, a sample information file is saved in the external memory card.

[0126] The internal memory can be used to store computer executable program codes, which include instructions. The processor executes various functional applications and data processing of the robot 500 by running the instructions stored in the internal memory. The internal memory can include a program storage area and a data storage area. The program storage area can store an operating system and at least one application program required by a function. The data storage area can store data created during the use of the robot 500. In addition, the internal memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash memory (UFS), etc.

[0127] The wireless communication function of the robot 500 can be realized through the communication module 505. For example, through the communication module 505, the robot 500 can realize communication with other devices, such as communication with a server. As an example, the communication module 505 can include an antenna 1, an antenna 2, a mobile communication module, a wireless communication module, a modem processor, a baseband processor, etc.

[0128] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the robot 500 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example: the antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0129] The mobile communication module can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the robot 500. The mobile communication module can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module can receive electromagnetic waves from the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transfer the processed signals to the modem processor for demodulation. The mobile communication module can also amplify the signals modulated by the modem processor, and radiate the signals as electromagnetic waves through the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module can be disposed in the processor. In some embodiments, at least part of the functional modules of the mobile communication module can be disposed in the same device as at least part of the modules of the processor.

[0130] The wireless communication module can provide a solution for wireless communication including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), infrared (IR) technology, etc. applied to the robot 500. The wireless communication module can be one or more devices integrated with at least one communication processing module. The wireless communication module receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering on the electromagnetic wave signals, and transmits the processed signals to the processor. The wireless communication module can also receive signals to be transmitted from the processor, perform frequency modulation and amplification on the signals, and radiate the signals as electromagnetic waves through the antenna 2.

[0131] In some embodiments, the antenna 1 and the mobile communication module of the robot 500 are coupled, and the antenna 2 and the wireless communication module are coupled, so that the robot 500 can communicate with the server and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include global positioning system (GPS), global navigation satellite system (GLONASS), beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0132] It can be understood that the structure illustrated in the embodiments does not constitute a specific limitation on the robot 500. In other embodiments, the robot 500 can include more or fewer components than illustrated, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0133] Please refer to Figure 4 , Figure 4 A structural schematic diagram of the vehicle 100 provided in the embodiments of the present application is shown in FIG. 1.

[0134] As Figure 4As shown, in one embodiment, the vehicle 100 can be configured to operate in a fully or partially autonomous mode. For example, the vehicle 100 can control itself while in the autonomous mode and can determine a current state of the vehicle and its surrounding environment, determine a likely behavior of at least one other vehicle in the surrounding environment, and determine a confidence level corresponding to a likelihood that the other vehicle will perform the likely behavior by human operation, control the vehicle 100 based on the determined information, while in the autonomous mode. While the vehicle 100 is in the autonomous mode, the vehicle 100 can be placed to operate without human interaction.

[0135] The vehicle 100 can include various subsystems, such as a propulsion system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, and a power source 110, a computer system 101, and a user interface 116. Alternatively, the vehicle 100 can include more or fewer subsystems, and each subsystem can include multiple elements, such as each subsystem including multiple Electronic Control Units (ECUs). Additionally, each subsystem and element of the vehicle 100 can be interconnected by wires or wirelessly.

[0136] The propulsion system 102 can include components that provide powered movement for the vehicle 100. In one embodiment, the propulsion system 102 can include an engine 118, an energy source 119, a transmission 120, and wheels / tires 121. The engine 118 can be an internal combustion engine, an electric motor, an air compression engine, or other types of engine combinations, such as a hybrid engine composed of a gasoline engine and an electric motor, a hybrid engine composed of an internal combustion engine and an air compression engine. The engine 118 converts the energy source 119 into mechanical energy.

[0137] Examples of the energy source 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electrical power. The energy source 119 can also provide energy for other systems of the vehicle 100.

[0138] The transmission 120 can transmit mechanical power from the engine 118 to the wheels 121. The transmission 120 can include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission 120 can also include other devices, such as a clutch. The drive shaft can include one or more shafts that can be coupled to one or more wheels 121.

[0139] The sensor system 104 can include several sensors that sense information about the environment surrounding the vehicle 100. For example, the sensor system 104 can include a global positioning system 122 (the positioning system can be a GPS system, a Beidou system, or other positioning system), an inertial measurement unit (IMU) 124, a radar 126, a laser rangefinder 128, and a camera 130. The sensor system 104 can also include sensors that monitor internal systems of the vehicle 100 (e.g., an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their respective characteristics (location, shape, direction, velocity, etc.). Such detection and identification are key functions for the safe operation of the vehicle 100.

[0140] The global positioning system 122 can be used to estimate the geographic location of the vehicle 100. The IMU 124 is used to sense changes in the position and orientation of the vehicle 100 based on inertial acceleration. In one embodiment, the IMU 124 can be a combination of an accelerometer and a gyroscope.

[0141] The radar 126 can utilize radio signals to sense objects within the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing objects, the radar 126 can also be used to sense the velocity and / or heading of the objects.

[0142] The laser rangefinder 128 can utilize laser light to sense objects in the environment in which the vehicle 100 is located. In some embodiments, the laser rangefinder 128 can include one or more laser sources, a laser scanner, and one or more processing modules, among other system components.

[0143] The camera 130 can be used to capture multiple images of the surrounding environment of the vehicle 100. The camera 130 can be a still camera or a video camera.

[0144] The control system 106 controls the operation of the vehicle 100 and its components. The control system 106 can include various elements, including a steering system 132, a throttle 134, a braking unit 136, a computer vision system 140, a route control system 142, and an obstacle avoidance system 144.

[0145] The steering system 132 is operable to adjust the heading of the vehicle 100. For example, in one embodiment, the steering system 132 can be a steering wheel system.

[0146] The throttle 134 is used to control the operational speed of the engine 118 and, in turn, the speed of the vehicle 100.

[0147] The braking unit 136 is used to control the deceleration of the vehicle 100. The braking unit 136 can use friction to slow the wheels 121. In other embodiments, the braking unit 136 can convert the kinetic energy of the wheels 121 into electrical current. The braking unit 136 can also take other forms to slow the rotational speed of the wheels 121 to control the speed of the vehicle 100.

[0148] The computer vision system 140 can operate to process and analyze images captured by the camera 130 to identify objects and / or features in the environment surrounding the vehicle 100. The objects and / or features can include traffic signals, road boundaries, and obstacles. The computer vision system 140 can use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 140 can be used to map the environment, track objects, estimate the speed of objects, and so on.

[0149] The route control system 142 is used to determine the route of travel for the vehicle 100. In some embodiments, the route control system 142 can combine data from the GPS 122 and one or more predetermined maps to determine the route of travel for the vehicle 100.

[0150] The obstacle avoidance system 144 is used to identify, evaluate, and avoid or otherwise navigate around potential obstacles in the environment of the vehicle 100.

[0151] Of course, in one instance, the control system 106 can include additional components in addition to those shown and described, either in addition to or in place of those shown. Or some of the components shown above can be reduced.

[0152] The vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users through the peripherals 108. The peripherals 108 can include a wireless communication system 146, an on-board computer 148, a microphone 150, and / or a speaker 152.

[0153] In some embodiments, the peripherals 108 provide a means for a user of the vehicle 100 to interact with the user interface 116. For example, the on-board computer 148 can provide information to a user of the vehicle 100. The user interface 116 can also operate the on-board computer 148 to receive input from the user. The on-board computer 148 can be operated through a touch screen. In other cases, the peripherals 108 can provide a means for the vehicle 100 to communicate with other devices located within the vehicle. For example, the microphone 150 can receive audio (e.g., voice commands or other audio input) from a user of the vehicle 100. Similarly, the speaker 152 can output audio to a user of the vehicle 100.

[0154] The wireless communication system 146 can wirelessly communicate with one or more devices directly or via a communication network. For example, the wireless communication system 146 can use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE. Or 5G cellular communication. The wireless communication system 146 can communicate with a wireless local area network (WLAN) using WiFi. In some embodiments, the wireless communication system 146 can communicate directly with devices using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various vehicle communication systems, for example, the wireless communication system 146 can include one or more dedicated short range communications (DSRC) devices, which can include public and / or private data communications between vehicles and / or roadside stations.

[0155] The power source 110 can provide power to various components of the vehicle 100. In one embodiment, the power source 110 can be a rechargeable lithium-ion or lead-acid battery. One or more battery packs of such a battery can be configured to power the various components of the vehicle 100. In some embodiments, the power source 110 and the energy source 119 can be implemented together, such as in some all-electric vehicles.

[0156] Some or all of the functionality of the vehicle 100 is controlled by the computer system 101. The computer system 101 can include at least one processor 113 that executes instructions 115 stored in a non-transitory computer readable medium, such as the data storage 114. The computer system 101 can also be a plurality of computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.

[0157] The processor 113 can be any conventional processor, such as a commercially available CPU. Alternatively, the processor can be a special purpose device such as an ASIC or other hardware-based processor. Although Figure 4 Although functionally associated with the processor, the memory, and other elements of the computer 110 in the same block, one of ordinary skill in the art will appreciate that the processor, computer, or memory can actually include multiple processors, computers, or memories that can or can not be stored within the same physical housing.

[0158] For example, the memory can be a hard drive or other storage medium located in a housing different from the computer 110. Thus, references to a processor or computer will be understood to include references to a collection of processors or computers or memories, which can or can not operate in parallel, that can be co-located or distributed across different physical locations. Rather than using a single processor to perform the steps described herein, such as some of the components of the steering assembly and the deceleration assembly, each can have its own processor that performs only the computations related to the functions specific to that component.

[0159] In various aspects described herein, the processor can be located remotely from the vehicle and in wireless communication with the vehicle. In other aspects, some of the processes described herein are performed on a processor disposed within the vehicle while others are performed by a remote processor, including taking the necessary steps to perform a single maneuver.

[0160] In some embodiments, the memory 114 can include instructions 115 (e.g., program logic) that can be executed by the processor to perform various functions of the vehicle 100, including those described above. The data storage 114 can also include additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the propulsion system 102, the sensor system 104, the control system 106, and the peripherals 108.

[0161] In addition to the instructions 115, the memory 114 can store data, such as road maps, route information, the location, direction, speed, and other such vehicle data of the vehicle, as well as other information. Such information can be used by the vehicle 100 and the computer system 101 during operation of the vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0162] The user interface 116 is used to provide information to or receive information from a user of the vehicle 100. Optionally, the user interface 116 can include one or more input / output devices within the collection of peripherals 108, such as the wireless communication system 146, the vehicle-to-computer 148, the microphone 150, and the speaker 152.

[0163] The computer system 101 can control the functions of the vehicle 100 based on inputs received from various subsystems (e.g., the propulsion system 102, the sensor system 104, and the control system 106), as well as from the user interface 116. For example, the computer system 101 can utilize inputs from the control system 106 to control the steering unit 132 to avoid obstacles detected by the sensor system 104 and the obstacle avoidance system 144. In some embodiments, the computer system 101 can be operable to provide control over many aspects of the vehicle 100 and its subsystems.

[0164] Optionally, one or more of the components described above can be installed separately from or associated with the vehicle 100. For example, the memory 114 can exist partially or entirely separately from the vehicle 100. The components described above can be communicatively coupled together in a wired and / or wireless manner.

[0165] Optionally, the above components are only an example, and in actual applications, components in each module described above can be added or deleted according to actual needs, Figure 4 This should not be understood as a limitation on the embodiments of the present application.

[0166] An autonomous vehicle traveling on a road, such as the vehicle 100 described above, can identify objects within its surrounding environment to determine an adjustment to a current speed. The objects can be other vehicles, traffic control devices, or other types of objects. In some examples, each identified object can be considered independently, and based on the respective characteristics of the object, such as its current speed, acceleration, spacing from the vehicle, etc., can be used to determine a speed to which the autonomous vehicle is to adjust.

[0167] Optionally, the vehicle 100 or a computing device associated with the vehicle 100 can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each of the identified objects can be dependent on the behavior of the others, so the behavior of a single identified object can also be predicted by considering all of the identified objects together. The vehicle 100 can adjust its speed based on the predicted behavior of the identified objects.

[0168] In other words, the autonomous vehicle can determine what steady state the vehicle will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. In this process, other factors can also be considered to determine the speed of the vehicle 100, such as the lateral position of the vehicle 100 in the road it is traveling on, the curvature of the road, the proximity of static and dynamic objects, etc.

[0169] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of the vehicle 100 to cause the autonomous vehicle to follow a given trajectory and / or maintain a safe lateral and longitudinal distance from objects in the vicinity of the autonomous vehicle (e.g., a car in an adjacent lane on the road).

[0170] The vehicle 100 described above can be a car, a truck, a motorcycle, a bus, an entertainment vehicle, an amusement park vehicle, a construction device, a trolley, a golf cart, etc., and the embodiments of the present application are not particularly limited.

[0171] Figure 4In the vehicle 100 shown, an advanced driving assistance system for implementing automatic driving functions can be configured, and a large number of parameters in the advanced driving assistance system need to be calibrated. Specifically, the calibration implementation process of the advanced driving assistance system of the vehicle mainly includes the calibration of the parameters of the subsystems in the execution layer, the perception layer, and the function layer. The execution layer involves power system calibration, brake system calibration, steering system calibration, four-wheel positioning parameter calibration, and suspension system calibration, etc. The perception layer involves GNSS and INS (Initial Navigation System) calibration, camera calibration, laser radar calibration, millimeter wave radar calibration, ultrasonic radar calibration, etc. GNSS includes GPS (Global Position System), GLONASS (Global Navigation Satellite System), Galileo (Galileo navigation satellite system), and BDS (BeiDounavigation satellite system). The function layer involves vehicle longitudinal control module calibration, lateral control module calibration, ADAS basic function calibration, ADAS driving style calibration, etc. The longitudinal control mainly controls the speed, and controls the speed by controlling the brake, throttle, gear, etc. The lateral control mainly controls the heading, and makes the vehicle travel according to the desired heading by changing the size of the steering wheel torque or angle, etc. The basic functions of ADAS include ACC (Adaptive Cruising System), LCC (Lane Center Control), ALC (Auto Lane Change), etc. The driving style refers to the way or habitual driving method of driving, which includes the selection of driving speed, the selection of driving distance, etc. The driving style includes aggressive type, stable type, cautious type, etc.

[0172] The above introduces the scene and the driving device to which the method provided by the embodiments of the present application is applied. The path optimization method provided by the present application will be described in detail below.

[0173] Please refer to Figure 5 , Figure 5 The flowchart of the path optimization method provided by the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the path optimization method includes the following steps 501-503. Figure 5

[0174] ​501、acquire motion state information of a driving device and a driving path to be optimized, the driving path comprising a plurality of sampling position points; the motion state information representing a motion state of each of the sampling position points; wherein the motion state is represented based on a plurality of flat variables or derivatives of the flat variables; each of the flat variables corresponding to a position of one dimension of the sampling position point; the motion state of at least one of the sampling position points comprising a curvature and a curvature rate of change.

[0175] In a possible implementation, a driving path to be optimized can be acquired, wherein the driving path to be optimized can be a prediction result (for example, an initial prediction result or an intermediate result of an optimization process) of a driving path of a driving device.

[0176] In the prior implementation, a plurality of sampling points can be sampled for a driving path to be optimized, the driving path is represented by positions of the sampling points, and the path optimization is performed on the positions of each sampling position point. On the one hand, in order to ensure the optimization accuracy, the number of sampling position points cannot be too small, and therefore the number of points to be optimized is also large, resulting in large computational cost. On the other hand, the positions of the sampling points are only an approximate expression of the path, and even if the number of sampling points is large, the optimization accuracy and effect of the path are still limited.

[0177] In the embodiment of the application, the driving path to be optimized is represented by a high-order polynomial with respect to a time variable, and the time variable and polynomial coefficients are optimized when the driving path is optimized. Since the number of objects to be optimized is reduced, the computational cost of optimization can be reduced, and the shape of the driving path can be more accurately represented by the high-order polynomial, thereby improving the optimization accuracy and effect.

[0178] For example, for a driving path to be optimized, let the arc length be L, the expected average driving speed be v mean , and the endpoints of the driving path be anchor points qk and qk+1. The corresponding driving path duration T k is the path length N q is a compensation coefficient (determined by the expected speed and expected acceleration of the endpoints); and the corresponding polynomial coefficients are obtained by solving a trajectory planning algorithm (for example, MinimumSnap), thereby obtaining a parameterized representation result of the driving path to be optimized, which can be used as an initial value of the optimization process.

[0179] Since the original driving path to be optimized is long, the original path needs to be split into a plurality of sub-paths, and each sub-path is optimized in granularity.

[0180] In a possible implementation, the original path can be sampled (for example, randomly sampled) to obtain a plurality of position sampling points, and the splitting of the original path can be performed based on the features of the curvatures or curvature change rates of the position sampling points.

[0181] Next, how to divide the original path is introduced:

[0182] When optimizing the driving trajectory, in order to ensure the effect of optimization, the positions with large curvatures or large curvature change rates in the driving trajectory often need to be adjusted. Therefore, when the initial path is divided, each divided path (including the driving path to be optimized in the embodiments of the present application) can include at least one sampling position point (which can also be referred to as a bad point in the embodiments of the present application) with a large curvature or a large curvature change rate.

[0183] In a possible implementation, the driving path includes a sampling point with a curvature or a curvature change rate greater than a threshold value.

[0184] When optimizing the driving trajectory, in order to ensure the effect of optimization, the end points (which can also be referred to as anchor points in the embodiments of the present application) of the divided paths cannot be points with curvatures or curvature change rates greater than a threshold value, because the end points are often not adjusted in the optimization algorithm.

[0185] In a possible implementation, the driving path to be optimized is a path selected from a planned path of the driving device, and the two ends of the driving path are obtained by sampling the planned path based on curvatures or curvature change rates less than a threshold value.

[0186] Next, a flowchart of path division is introduced:

[0187] An initial path is taken, anchor points and time distribution are allocated according to the curvature change of the original path, and the original path is divided. For the obtained initial path, uniform sampling can be performed thereon at a certain step length, for each sampling point, the approximate curvature and the curvature change rate are calculated in combination with the adjacent front and rear points, if both of them exceed the preset upper threshold value, the sampling point is counted as a bad point, and grouping is performed according to the distance proximity. From the two ends of each group of bad points, the two sides are pushed until the points with curvatures and curvature change rates lower than the preset lower threshold value are found, which are marked as anchor points. The original path is divided into a plurality of road segments according to the order of the front and rear anchor points. It should be understood that if there is overlap between the road segments, the anchor points can be merged or cut according to the severity of the overlapping part until there is no overlap between the road segments. Then, each road segment can be time-distributed and polynomial coefficient parameterized, and the time distribution is performed according to the arc length of each segment trajectory, and the whole trajectory is initialized as a segmented polynomial trajectory.

[0188] The curvature-guided trajectory space-time parameterization is implemented as follows:

[0189] After obtaining the initial path, for each point except the first and the last, the two adjacent points are combined to approximate the curvature of the point by a second order polynomial:

[0190]

[0191]

[0192] The curvature and its rate of change of each position sampling point are calculated, and if or κ>κ u , the current point is marked as a bad point i; if two or more bad points are close, they are grouped together to ensure that there are no repeated bad points between groups; starting from both ends of the bad points in each group, the two directions are expanded by ds steps, until the actual curvature rate of change of the expanded point meets and κ<κ l ; the corresponding positions of the two end expansion points are respectively registered as anchor points q i and q i+1 , and the part between the two anchor points (including) is taken as the corresponding road segment of the bad point i; if there is an intersection between the front and rear road segments, the center point of the intersection part is used to replace the two end anchor points, and this iteration is repeated until all road segments are non-overlapping, and finally the arc length of each road segment is calculated

[0193] In one possible implementation, motion state information of a driving device can be obtained, and the driving path includes a plurality of sampling position points; the motion state information represents the motion state of each sampling position point.

[0194] For example, the motion state can be at least one of speed, acceleration, angular velocity, angular acceleration, curvature, and rate of change of curvature.

[0195] In the existing implementation, the motion state of each position sampling point is represented by the expression of all state quantities and input quantities obtained through the physical motion equation of the system (usually a system of ordinary differential equations, such as ).

[0196] Among them, the curvature is determined by the linear velocity and the angular velocity (Δθ / Δs), which belongs to a nonlinear constraint (involving arctan, sqrt, exponential operation), and the rate of change of curvature is more nonlinear and has a higher order (numerical operation involving jerk, speed to the 5th power, etc., which often appears in the range of 10^10, which is easy to cause gradient explosion), resulting in high complexity in modeling the rate of change of curvature, which is not easy to control in terms of reduction and numerical stability (discrete curvature calculation, 0.5m difference displacement may cause local curvature to change by more than 10 times), so precise curvature constraint modeling and efficient calculation means are needed.

[0197] In a possible implementation, after obtaining the driving path to be optimized, the driving path to be optimized can be sampled at position points, and the motion states of the sampled position points can be used to construct the penalty term.

[0198] First, the sampling method of the position sampling points is introduced:

[0199] In a possible implementation, the trajectory segments can be sampled at the variable curvature, so that the number of points with large curvature or curvature change rate is more. Since the part with large curvature or curvature change rate is the object that needs to be optimized, the above sampling method at the variable curvature can improve the effect of subsequent optimization.

[0200] Specifically, the plurality of sampling position points include first sampling points and second sampling points, the corresponding curvature or curvature change rate of the first sampling points is greater than that of the second sampling points, the first sampling points are obtained by sampling the driving path according to a first sampling resolution, the second sampling points are obtained by sampling the driving path according to a second sampling resolution, and the first sampling resolution is greater than the second sampling resolution.

[0201] In the embodiment of the application, the motion state is represented based on a plurality of flat variables or derivatives of the flat variables; each flat variable corresponds to the position of one dimension of the sampling position point.

[0202] The embodiment of the application can model the motion system by using differential flatness, and directly perform unified differential parameterization description on the motion state quantities such as curvature κ and curvature change rate κ, speed v, acceleration a, angular speed ω, and angular acceleration α in the flat variable space.

[0203] Expressing the motion state through the flat variable can avoid solving the differential equation in the trajectory optimization process, thereby reducing the optimization difficulty.

[0204] In a possible implementation, initial motion state information of the driving device can be obtained, the initial motion state information is expressed in a first space, and the expression space of the initial motion state information is changed from the first space to a second space, to obtain the motion state information, the dimension of the second space is less than or equal to the dimension of the first space, and each dimension of the second space corresponds to a flat variable or a derivative of a flat variable. When the dimension of the second space is less than the dimension of the first space, the number of variables used to represent the motion state is equivalent, thereby reducing the dimension of the expression space and reducing the optimization difficulty.

[0205] For example, the physical motion equation of the system (usually a system of ordinary differential equations, such as All state variables and input variables (i.e., motion states in the embodiments of the present application) are obtained, and a set of flat output variables y is found by mathematical transformation of the state variables and input variables, so that the original system variables can be expressed by the set of flat variables and their linearly independent derivatives of various orders A complete expression is obtained, thereby completing differential flat modeling of the original system.

[0206] In addition, differential flatness is used to model the motion system, and unified description of state variables such as velocity, acceleration, angular velocity, angular acceleration, curvature, and curvature rate is performed in a low-dimensional flat variable space, thereby avoiding the use of geometric difference and steering angle, yaw angle, and other state variables.

[0207] For example, the physical parameters of the motion body, such as the turning radius and wheelbase of the vehicle body, the constant in the kinematic model, and the basis vectors composed of the flat outputs and their derivatives of various orders are used to perform differential parameterization on key state variables such as curvature κ and curvature rate κ·, velocity v, acceleration a, angular velocity ω, and angular acceleration α of the motion trajectory Description.

[0208] For example, the vehicle system is used as an example, and a specific implementation of differential flat modeling is as follows:

[0209] The position, motion state, and wheelbase of the vehicle can be described according to the vehicle motion equation, and a set of flat outputs σ x ,σ y =p x ,p y Differential parameterization is performed on the main state variables and input variables of the original system, and unified description is performed as flat outputs and their derivatives of various orders:

[0210]

[0211] 502. Determine a target penalty function according to the motion state information; the target penalty function is related to the driving feasibility and smoothness of the driving device when driving along the plurality of sampling position points; wherein the driving feasibility is related to the curvature, and the smoothness is related to the curvature and the curvature rate;

[0212] 503. Optimize the driving path according to the target penalty function.

[0213] In one possible implementation, a target penalty function can be determined according to the motion state information; the target penalty function is related to the driving performance of the driving device when driving along the plurality of sampling position points, and the driving path is optimized according to the target penalty function; wherein the goal of optimizing the driving path is to reduce the value of the target penalty function.

[0214] In the target penalty function, multiple penalty terms can be included, and one or more of the penalty terms can be obtained according to the motion state information.

[0215] Next, the target penalty function in the embodiment of the present application is introduced:

[0216] In a possible implementation, the target penalty function (one or more penalty terms) is negatively related to the driving feasibility of the driving device when driving along the plurality of sampling position points.

[0217] In a possible implementation, the target penalty function (one or more penalty terms) is negatively related to the smoothness of the driving device when driving along the plurality of sampling position points.

[0218] For example, a nonlinear inequality constraint can be constructed directly for the curvature, the rate of change of the curvature, and the angular acceleration, and a regular term is set, and other constraints such as speed, acceleration, and angular velocity are combined to be converted into feasibility and smoothness penalty terms.

[0219] In addition, the curvature, the rate of change of the curvature, and the angular acceleration can also be used as hard constraint objects to construct inequality constraints, limit the upper and lower limits, and can be constrained by softening and relaxing (such as barrier function, Lagrange multiplier, augmented Lagrange, etc.) to be converted into kinematic penalty terms.

[0220] At the same time, the instantaneous curvature, the rate of change of the curvature / angular acceleration is regularized The cumulative result in the whole trajectory is used as a regular penalty term related to the change of the curvature in the whole interval

[0221] At the same time, hard inequality constraints are constructed for the dynamics-related variables such as speed, acceleration, angular velocity, and angular acceleration, and the upper and lower limits are limited, which can be constrained by softening and relaxing (such as barrier function, Lagrange multiplier, augmented Lagrange, etc.) to be converted into dynamics penalty terms.

[0222] The above kinematic, dynamic, and regularization penalty terms are linearly weighted as the overall feasibility and smoothness penalty terms.

[0223]

[0224] Next, the construction of the penalty term by the motion state represented by the flat variable is introduced:

[0225] In a possible implementation, the expression of the rate of change (derivative) can be obtained according to the parametric description of the angular velocity and the differential of the curvature:

[0226]

[0227] Apply upper and lower limit (inequality) constraints on curvature, curvature rate of change and / or angular velocity, angular acceleration, and convert them into kinematic penalty terms

[0228] The penalty term corresponding to the curvature constraint can be:

[0229]

[0230] The penalty term corresponding to the curvature rate of change constraint can be:

[0231]

[0232] The penalty term corresponding to the angular velocity constraint can be:

[0233]

[0234] The penalty term corresponding to the angular acceleration constraint can be:

[0235]

[0236] Regularization terms can be added to the curvature, curvature rate of change, angular velocity, angular acceleration (selected according to actual situation), such as: And the cumulative result within the entire trajectory is taken as the regularization penalty term for curvature change in the entire interval (from time 0 to time T) Nonlinear inequality constraints are directly constructed for curvature, curvature rate of change / angular acceleration, and regularization terms are set for curvature and its rate of change to jointly construct a smoothness penalty term, while constraining curvature jump, steering consistency and amplitude.

[0237] The above introduces a part of the penalty terms (about feasibility and smoothness). In addition to the above penalty terms, collision risk information related to driving safety needs to be constructed, and the corresponding safety penalty term is constructed based on the collision risk information.

[0238] Safety obstacle avoidance involves online collision detection near a large number of trajectory points (request frequency often reaches hundreds of thousands) and changes in path topological form, especially when the entity and the environment have non-convex configurations or complex structures, the results often have a great impact on optimization time, stability and trajectory smoothness, so the efficiency of collision gradient calculation and the selection of trajectory control points are more sensitive, and efficient collision constraint modeling and adaptive optimization of control points are needed.

[0239] In a possible implementation, point cloud data of a driving environment where the driving device is located at the plurality of sampling position points can be acquired, the point cloud data being used to represent obstacles in the driving environment, and collision risk information of the driving device at each of the sampling position points can be obtained according to interaction of each point cloud in the point cloud data with an Euclidean distance field ESDF of the driving device.

[0240] In this embodiment, the point cloud data is actually a collection of a plurality of points, and each point is used to represent coordinates of a ground surface or an obstacle surface in the driving environment. Therefore, based on the plurality of points in the point cloud data, coordinates of each point of the ground surface and the obstacle surface in the driving environment can be determined.

[0241] The point cloud data of the driving environment can be obtained by real-time scanning of the driving device using a laser radar during driving. The point cloud data of the driving environment can also be obtained by scanning of other driving devices in advance using a laser radar, and the embodiment does not limit the source of the point cloud data.

[0242] The driving device can be a robot or a vehicle, for example. The driving environment can be an indoor environment or an outdoor environment, for example, and the driving environment mainly depends on the environment in which the driving device works. For example, when the driving device is a cleaning robot, the driving environment can be a user's home, for example. When the driving device is a welcome robot, the driving environment can be an indoor public place such as a bank or a restaurant, for example. When the driving device is a patrol robot, the driving environment can be an outdoor environment such as a substation, for example. When the driving device is a vehicle, the driving environment can be an outdoor environment such as a mine, for example.

[0243] In a possible implementation, collision risk information of the driving device at each of the sampling position points can be obtained according to interaction of each point cloud in the point cloud data with an Euclidean distance field ESDF of the driving device.

[0244] The Euclidean distance field ESDF of the driving device can be constructed in the following manner, but is not limited thereto: an outer bounding box (generally a convex bounding box) of a surface profile thereof is found, and the actual profile of the body is ensured to be entirely enclosed therein, and methods such as an Axis Aligned Bounding Box (AABB), an Oriented Bounding Box (OBB), a Frustum Depth Heuristic (FDH), and a Convex Hull (CH) can be used (but are not limited thereto). The region inside the outer bounding box is further subdivided, and the region enclosed by the actual profile edge is divided into an ROI region, and the remaining region is a non-ROI region. An EDT algorithm is used to generate the ESDF, and the ROI region is set to be negative, and the non-ROI region is set to be non-negative.

[0245] In one possible implementation, the collision risk information of the driving device at each sampling location point can be obtained based on the ESDF value of each point cloud in the point cloud data at the corresponding position in the ESDF, and the ESDF gradient determined by the ESDF values ​​of nearby positions.

[0246] For example, the entire ESDF can be stored as a discretized lookup table. When in use, continuous ESDF values ​​can be obtained based on coordinate interpolation and the mobile body positioning information. (Location and pose information) The point cloud is registered in a grid map centered on the subject, and the occupancy status of the grid is maintained and updated in real time. The point cloud covered by the bounding box of the moving subject is detected, and the occupancy status of the grid corresponding to each point cloud i is queried. If the status is occupied, the ESDF value di corresponding to the point cloud is obtained by interpolation using a lookup table, and its gradient is calculated based on the ESDF values ​​of the neighborhood. The ESDF values ​​and gradients of all occupied point clouds within the bounding box are weighted and used as the collision cost and gradient corresponding to the current localization of the subject. The result is used as a safety penalty term.

[0247] Using the above method, a 3D potential field, such as the Euclidean Signed Distance Field (ESDF), is employed to provide a refined description of the collision trend within the moving body, constructing a safety penalty term. In the robot's coordinate system, for environmental obstacle points that intrude into its contour (confirmed through occupancy status), the ESDF costs and gradients of all intrusion points are queried and summed. This allows for the establishment of an intrusive 3D collision calculation model based on the environmental occupancy grid and the moving body's ESDF, thus constructing a safety penalty term.

[0248] In one possible implementation, the driving path can be optimized based on the target penalty function; wherein the goal of optimizing the driving path is to reduce the value of the target penalty function.

[0249] Wherein, when the driving path is represented by a higher-order polynomial with respect to the time variable, the modification of the driving path includes: optimizing the time variable and the polynomial coefficients of the higher-order polynomial.

[0250] When performing path optimization, optionally, an optimal control problem can be constructed with the objectives of minimizing control and optimizing time. The aforementioned penalty term can be added for weighting, transforming it into an unconstrained optimization problem, and then performing nonlinear optimization on the piecewise polynomial trajectory. For example, dynamic variational rate sampling can be performed based on the magnitude of the curvature and its rate of change on each segment of the trajectory. The aforementioned penalty term cost weighting and gradient calculation can be applied to all sampling points, propagating the temporal and spatial two-dimensional gradients, and simultaneously optimizing the time variable and polynomial coefficients to obtain the optimized path trajectory.

[0251] where the optimal control problem is constructed with the goal of minimum control and optimal time, and the above penalty terms are added to weight, and transformed into an unconstrained optimization problem, and the final form of the unconstrained optimization problem is as follows when the piecewise polynomial trajectory is nonlinearly optimized:

[0252]

[0253] where, is to minimize the control input, is to minimize the trajectory time;

[0254] is the safety, smoothness and feasibility penalty and regularization term.

[0255] where the specific implementation of the unconstrained optimization problem construction and solution is as follows:

[0256] (I) The energy penalty function makes the control input as smooth as possible, and ensures the energy optimality:

[0257]

[0258] (II) The total time penalty function makes the total time of the trajectory as short as possible:

[0259]

[0260] (III) The dynamics penalty function makes the expected body velocity and acceleration at any time on the trajectory meet the dynamics limit of the moving body itself:

[0261]

[0262]

[0263]

[0264] where, C(*)=max{*,0} 3 is a cubic penalty function, v max is the maximum speed limit, a max is the maximum acceleration limit, and K is the number of trajectory segments.

[0265] (IV) The collision safety penalty function is as follows:

[0266]

[0267] where, is the current pose obtained from the body contour ESDF ESDF The corresponding collision penalty term is calculated as the square of the sum of the ESDF values of all obstacle points that collide with the body contour at the current pose. i,j The ESDF value at p i,j is calculated by using the trilinear interpolation method.

[0268] (V) The smoothness penalty function is as follows:

[0269]

[0270] wherein, G g (p i,j ) is the kinematic penalty term (curvature, rate of change of curvature, angular velocity, angular acceleration, etc.) of the moving body at p i,j on the trajectory, and C is a self-defined cost-barrier function.

[0271] (VI) The regularization penalty function is as follows:

[0272]

[0273] That is, the sampling of the discrete points in the integral operation of the above regularization penalty term is replaced by implementation. Wherein, G ρ (p i,j ) is the norm combination weighting of the curvature, rate of change of curvature, angular velocity, and angular acceleration of the moving body at p i,j on the trajectory.

[0274] (VII) The trajectory of the moving body is represented by a quintic polynomial function. Let the trajectory have M segments, and the polynomial coefficients of the kth segment (k = 1, 2,..., M) be c k , which is a 6 × 1 vector, and the duration is T k , which is a one-dimensional vector, and β(t) = [1, t, t 2 , t 3 , t 4 , t 5 ] T is a function of time t, and the position of the kth segment trajectory at time t is Let c = [c1, c2,..., c M ], T = [T1, T2,..., T M ] T , and let be the *th derivative of P k (t), and β (*) be the *th derivative of β(t), then the finally modeled unconstrained optimization problem is:

[0275] minc,T λ e J e +λ t J t +λ d J d +λ r J r +λ g J g +λ ρ J ρ ;

[0276] where λ* is a weight coefficient of the term λ.

[0277] (VIII) solving the unconstrained optimization problem by using a quasi-Newton method or a trust region method.

[0278] (IX) performing safety detection on the generated trajectory of (II), that is, taking trajectory points every L c length on the generated trajectory, if a grid where a trajectory point is located is occupied; otherwise, the trajectory is successfully generated, and the generated trajectory is returned.

[0279] The embodiment of the present application realizes the optimization of the initial driving path into a trajectory with continuous curvature and no collision with the environment by efficiently and high-quality solving the nonlinear minimum control problem.

[0280] The present application expresses the motion state through flat variables, which can avoid solving differential equations in the trajectory optimization process on the one hand, and can reduce the dimension of the expression space on the other hand, thereby reducing the difficulty of trajectory optimization. In addition, directly describing the nonlinear constraint terms such as curvature and its rate of change in the flat variable space can avoid repeated discretization of displacement or time difference, so as to achieve reasonable constraint curvature jump while making the overall steering consistency as high as possible and the amplitude as small as possible.

[0281] The above introduces a path optimization method provided by the embodiment of the present application, and the following will introduce a driving device for executing the path optimization method.

[0282] Please refer to Figure 6 , Figure 6 a structural schematic diagram of a driving device provided by the embodiment of the present application. As Figure 6 shown, the driving device comprises:

[0283] The acquisition module 601 is configured to acquire motion state information of a driving device and a driving path to be optimized, the driving path comprising a plurality of sampling position points; the motion state information represents a motion state of each of the sampling position points; wherein the motion state is represented based on a plurality of flat variables or derivatives of the flat variables; each of the flat variables corresponds to a position of one dimension of the sampling position point; the motion state of at least one of the sampling position points comprises a curvature and a curvature rate of change.

[0284] The acquisition module 601 is configured to acquire motion state information of a driving device and a driving path to be optimized, the driving path comprising a plurality of sampling position points; the motion state information represents a motion state of each of the sampling position points; wherein the motion state is represented based on a plurality of flat variables or derivatives of the flat variables; each of the flat variables corresponds to a position of one dimension of the sampling position point; the motion state of at least one of the sampling position points comprises a curvature and a curvature rate of change.

[0285] The processing module 602 is configured to determine a target penalty function according to the motion state information; the target penalty function is related to driving feasibility and smoothness of the driving device when driving along the plurality of sampling position points; wherein the driving feasibility is related to the curvature, and the smoothness is related to the curvature and the curvature rate of change; and the driving path is optimized according to the target penalty function.

[0286] The processing module 602 is configured to determine a target penalty function according to the motion state information; the target penalty function is related to driving feasibility and smoothness of the driving device when driving along the plurality of sampling position points; wherein the driving feasibility is related to the curvature, and the smoothness is related to the curvature and the curvature rate of change; and the driving path is optimized according to the target penalty function.

[0287] In a possible implementation, a target of optimizing the driving path is to reduce a value of the target penalty function.

[0288] In a possible implementation, the processing module 602 is specifically configured to: determine a target penalty function according to the motion state information; the target penalty function is related to driving performance of the driving device when driving along the plurality of sampling position points; and optimize the driving path according to the target penalty function; wherein a target of optimizing the driving path is to reduce a value of the target penalty function.

[0289] In a possible implementation, the acquisition module 601 is specifically configured to: acquire initial motion state information of the driving device, the initial motion state information being expressed in a first space; and change an expression space of the initial motion state information from the first space to a second space to obtain the motion state information, a dimension of the second space being less than or equal to a dimension of the first space, and each dimension of the second space corresponding to a flat variable or a derivative of a flat variable.

[0290] In a possible implementation, the motion state is at least one of a speed, an acceleration, an angular speed, an angular acceleration, a curvature, and a curvature rate of change.

[0291] In a possible implementation, the driving path is represented by a high-order polynomial with respect to a time variable, and the optimization of the driving path comprises optimization of the time variable and polynomial coefficients of the high-order polynomial.

[0292] In a possible implementation, the plurality of sampling position points comprises a first sampling point and a second sampling point, the first sampling point corresponds to a larger curvature or curvature change rate than the second sampling point in the driving path, the first sampling point is obtained by sampling the driving path according to a first sampling resolution, and the second sampling point is obtained by sampling the driving path according to a second sampling resolution, the first sampling resolution being greater than the second sampling resolution.

[0293] In a possible implementation, the driving path to be optimized is a section of path selected from a planned path of the driving device, and two ends of the driving path are obtained by sampling the planned path based on a curvature and / or a curvature change rate being less than a threshold.

[0294] In a possible implementation, the driving path comprises a sampling point with a curvature and / or a curvature change rate greater than a threshold.

[0295] In a possible implementation, the target penalty function is negatively related to driving feasibility and / or smoothness of the driving device when driving along the plurality of sampling position points.

[0296] In a possible implementation, the obtaining module 601 is further configured to:

[0297] obtain point cloud data of a driving environment in which the driving device is located at the plurality of sampling position points, the point cloud data being used to represent an obstacle in the driving environment;

[0298] The processing module 602 is further configured to: obtain collision risk information of the driving device at each of the sampling position points according to interaction of each point cloud in the point cloud data and an Euclidean distance field (ESDF) of the driving device.

[0299] The processing module 602 is specifically configured to:

[0300] optimize the driving path according to the motion state information and the collision risk information.

[0301] In a possible implementation, the processing module 602 is specifically configured to:

[0302] obtain the collision risk information of the driving device at each of the sampling position points according to an ESDF value of a corresponding position of each point cloud in the point cloud data in the ESDF and an ESDF gradient determined according to the ESDF value of the corresponding position and ESDF values of positions near the corresponding position.

[0303] In a possible implementation, the traveling device includes a robot or a vehicle.

[0304] Next, a traveling device provided by an embodiment of the present application is introduced. Referring to FIG. 7, Figure 7 Figure 7 FIG. 7 is a structural schematic diagram of the traveling device provided by the embodiment of the present application. The traveling device 700 can specifically include a microcontroller unit (MCU), a chip, a chip system or a circuit system, etc., which is not limited here. Specifically, the traveling device 700 includes a transceiver 701, a processor 702 and a memory 703 (wherein the number of the processor 702 in the traveling device 700 can be one or more, and one processor is taken as an example in the figure), wherein the processor 702 can include an application processor 7021 and a communication processor 707. In some embodiments of the present application, the transceiver 701, the processor 702 and the memory 703 can be connected through a bus or other means. Figure 7

[0305] The memory 703 can include a read-only memory and a random access memory, and provide the processor 702 with instructions and data. A part of the memory 703 can also include a non-volatile random access memory (NVRAM). The memory 703 stores processor and operation instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the operation instructions can include various operation instructions for implementing various operations.

[0306] The processor 702 controls the operation of path optimization. In a specific application, various components of the traveling device are coupled together through a bus system, wherein the bus system can include a data bus, a power bus, a control bus and a state signal bus, etc. in addition to the data bus. However, for the purpose of clear illustration, various buses are referred to as a bus system in the figure.

[0307] ​​The method disclosed by the embodiments of the present application can be applied to the processor 702 or implemented by the processor 702. The processor 702 can be an integrated circuit chip having a signal processing capability. In the implementation process, the steps of the method can be completed by an integrated logic circuit or an instruction in the form of software in the processor 702. The processor 702 described above can be a general processor, a digital signal processor (DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The processor 702 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or the processor can also be any conventional processor or the like. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory 703, and the processor 702 reads the information in the memory 703 and combines the hardware to complete the steps of the method.

[0308] The transceiver 701 (for example, a network card) can be used to receive input digital or character information, and generate signal input related to the settings and function control of the travel device. The transceiver 701 can also be used to output digital or character information through the first interface; and send instructions to the disk group through the first interface to modify the data in the disk group.

[0309] The travel device provided by the embodiments of the present application can specifically include a chip, which includes a processing unit and a communication unit. The processing unit can be a processor, and the communication unit can be an input / output interface, a pin or a circuit, etc. The processing unit can execute computer execution instructions stored in a storage unit, so that the chip in the processing module executes the path optimization method described in the above embodiments. Alternatively, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit can also be a storage unit outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0310] In particular, refer to Figure 8 , Figure 8 A structural schematic diagram of a chip provided by an embodiment of the present application; the chip can be represented as a neural network processor NPU 800, which is mounted on a host CPU as a coprocessor and is assigned tasks by the host CPU. The core part of the NPU is an operation circuit 803, which extracts matrix data in a memory and performs multiplication operation under the control of a controller 804.

[0311] In some implementations, the operation circuit 803 internally includes a plurality of processing units (PEs). In some implementations, the operation circuit 803 is a two-dimensional systolic array. The operation circuit 803 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 803 is a general-purpose matrix processor.

[0312] For example, it is assumed that there are an input matrix A, a weight matrix B, and an output matrix C. The operation circuit takes the corresponding data of the matrix B from the weight memory 802 and buffers it on each PE in the operation circuit. The operation circuit takes the matrix A data from the input memory 801 and performs matrix operation with the matrix B, and the partial result or final result of the obtained matrix is saved in an accumulator 808.

[0313] The unified memory 806 is used to store input data and output data. The weight data is transferred to the weight memory 802 through a direct memory access controller (DMAC) 805. The input data is also transferred to the unified memory 806 through the DMAC.

[0314] The BIU is a bus interface unit 810, which is used for interaction between the AXI bus and the DMAC and the instruction fetch buffer (IFB) 809.

[0315] The bus interface unit 810 (BIU) is used for the instruction fetch buffer 809 to obtain instructions from an external memory, and is also used for the direct memory access controller 805 to obtain original data of the input matrix A or the weight matrix B from the external memory.

[0316] DMAC is mainly used to move input data in external memory DDR to unified memory 806 or move weight data to weight memory 802 or move input data to input memory 801.

[0317] Vector computation unit 807 includes multiple operation processing units, which further process the output of operation circuit 803 as needed, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / full connection layer network computation in neural network, such as Batch Normalization, pixel-level summation, up-sampling of feature plane, etc.

[0318] In some implementations, vector computation unit 807 can store the processed output vector to unified memory 806. For example, vector computation unit 807 can apply a linear function; or, a non-linear function to the output of operation circuit 803, such as linear interpolation on the feature plane extracted by the convolutional layer, and further, a vector of accumulated values to generate activation values. In some implementations, vector computation unit 807 generates normalized values, pixel-level summation values, or both. In some implementations, the processed output vector can be used as activation input to operation circuit 803, such as for use in subsequent layers in the neural network.

[0319] Controller 804 is connected to instruction fetch buffer 809 for storing instructions used by controller 804;

[0320] Unified memory 806, input memory 801, weight memory 802, and instruction fetch buffer 809 are all on-chip memories. External memory is private to the NPU hardware architecture.

[0321] Any of the above-mentioned processors can be a general central processing unit, microprocessor, ASIC, or one or more integrated circuits for controlling the execution of the above-mentioned programs.

[0322] Reference can be made to Figure 9 , Figure 9 A structural schematic diagram of a computer readable storage medium provided by an embodiment of the present application. The present application also provides a computer readable storage medium, and in some embodiments, the above-mentioned disclosed method can be implemented as computer program instructions encoded in a computer readable storage medium or other non-transitory medium or article in a machine readable format.

[0323] Figure 9A conceptual partial view of an example computer-readable storage medium arranged in accordance with at least some embodiments presented herein is shown schematically, the example computer-readable storage medium including a computer program for executing a computer process on a computing device.

[0324] In one embodiment, the computer-readable storage medium 900 is provided using a signal bearing medium 901. The signal bearing medium 901 can include the one or more program instructions 902 which, when executed by one or more processors, can provide the functionality or some of the functionality described above for embodiments. Additionally, or alternatively, the signal bearing medium 901 can include the program instructions 902 which, when executed by one or more processors, can provide the functionality or some of the functionality described above for embodiments. Figure 9 The program instructions 902 in the signal bearing medium 901 also describe example instructions.

[0325] In some examples, the signal bearing medium 901 can comprise a computer- readable medium 903, such as, but not limited to, a hard disk drive, a Compact Disk (CD), a Digital Video Disk (DVD), a memory, a ROM, a RAM, or the like.

[0326] In some embodiments, the signal bearing medium 901 can comprise a computer- recordable medium 904, such as, but not limited to, a memory, a Read-Write (R / W) CD, a R / W DVD, etc. In some embodiments, the signal bearing medium 901 can comprise a communications medium 905, such as, but not limited to, a digital and / or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communication link, a wireless communication link, etc.). Thus, for example, the signal bearing medium 901 can be conveyed by a wireless form of the communication medium 905 (e.g., a wireless communication medium complying with the IEEE 802 standards or other transmission protocols).

[0327] The one or more program instructions 902 can be, for example, computer- executable instructions, or logic-implementing instructions. In some examples, a computing device of a computing device can be configured to provide various operations, functions, or actions in response to the program instructions 902 conveyed to the computing device by one or more of the computer-readable medium 903, the computer-recordable medium 904, and / or the communication medium 905.

[0328] It should also be noted that the above-described apparatus embodiments are merely illustrative, where the units described as separate units can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purposes of the embodiments. In addition, the connection relationship between the modules in the apparatus embodiments provided in the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0329] Those skilled in the art can clearly understand that the application can be implemented by means of software plus necessary universal hardware, and of course can also be implemented by means of dedicated hardware including special integrated circuit, special CPU, special memory, special component, etc. Generally, any function completed by computer program can be easily implemented by corresponding hardware, and the specific hardware structure for implementing the same function can also be various, such as analog circuit, digital circuit or special circuit, etc. However, for the application, software program implementation is a better embodiment. Based on such understanding, the technical solution of the application or the part of the application which makes contribution to the prior art can be embodied in the form of software product, which is stored in readable storage medium, such as computer floppy disk, U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a plurality of instructions for making a computer device (which can be personal computer, training device or network device, etc.) execute the method described in various embodiments of the application.

[0330] In the above embodiments, the implementation can be achieved by software, hardware, firmware or any combination thereof, entirely or partially. When implemented by software, the implementation can be achieved in the form of a computer program product, entirely or partially.

[0331] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the flow or function described in the embodiments of the application is generated entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, training device or data center to another website, computer, training device or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be stored by a computer or a data storage device such as a training device, a data center, etc. integrated with one or more available media sets. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

Claims

1. A path optimization method characterized by, The method comprises: acquiring motion state information of a traveling device and a traveling path to be optimized, the traveling path comprising a plurality of sampling position points; the motion state information representing a motion state of each of the sampling position points; wherein the motion state is represented based on a plurality of flat variables or derivatives of the flat variables; each of the flat variables corresponding to a position of one dimension of the sampling position points; the motion state of at least one of the sampling position points comprising a curvature and a curvature rate of change; determining a target penalty function according to the motion state information; the target penalty function being related to a traveling feasibility and a smoothness of the traveling device when traveling along the plurality of sampling position points; wherein the traveling feasibility is related to the curvature, and the smoothness is related to the curvature and the curvature rate of change; optimizing the traveling path according to the target penalty function.

2. The method of claim 1, wherein, The optimization of the traveling path aims to reduce a value of the target penalty function.

3. The method according to claim 1 or 2, characterized in that, The acquiring of the motion state information of the traveling device comprises: acquiring initial motion state information of the traveling device, the initial motion state information being expressed in a first space; changing an expression space of the initial motion state information from the first space to a second space to obtain the motion state information, the second space having a dimension less than or equal to that of the first space, and each dimension of the second space corresponding to a flat variable or a derivative of a flat variable.

4. The method according to claim 1 or 2, characterized in that, The motion state is at least one of a speed, an acceleration, an angular speed, an angular acceleration, a curvature, and a curvature rate of change.

5. The method according to claim 1 or 2, characterized in that, The traveling path is represented by a high-order polynomial with respect to a time variable, and the optimization of the traveling path comprises optimizing the time variable and polynomial coefficients of the high-order polynomial.

6. The method of claim 1 or 2, wherein, The plurality of sampling position points comprise a first sampling point and a second sampling point, the curvature or the curvature rate of change corresponding to the first sampling point in the traveling path being greater than that of the second sampling point, the first sampling point being obtained by sampling the traveling path according to a first sampling resolution, the second sampling point being obtained by sampling the traveling path according to a second sampling resolution, the first sampling resolution being greater than the second sampling resolution.

7. The method according to claim 1 or 2, characterized in that, The traveling path to be optimized is a section of path selected from a planned path of the traveling device, and two ends of the traveling path are obtained by sampling the planned path based on a curvature and / or a curvature rate of change being less than a threshold value.

8. The method of claim 7, wherein, The traveling path comprises a sampling point with a curvature and / or a curvature rate of change greater than a threshold value.

9. The method of claim 2, wherein, The target penalty function is negatively related to the traveling feasibility and / or the smoothness of the traveling device when traveling along the plurality of sampling position points.

10. The method of claim 1 or 2, wherein, The method further comprises: acquiring point cloud data of a traveling environment in which the plurality of sampling position points are located, the point cloud data being used to represent obstacles in the traveling environment; obtaining collision risk information of the traveling device at each of the sampling position points according to interaction of each point cloud in the point cloud data with a 3D potential field of the traveling device; the optimization of the traveling path according to the motion state information comprises: The driving path is optimized according to the motion state information and the collision risk information.

11. The method of claim 10, wherein, The collision risk information of the driving device at each sampling position point is obtained according to the interaction of each point cloud in the point cloud data with an ESDF of the driving device. The collision risk information of the driving device at each sampling position point is obtained according to the ESDF value of each point cloud in the point cloud data at the corresponding position in the ESDF and the ESDF gradient determined according to the ESDF value of the nearby position of the corresponding position.

12. The method of claim 1 or 2, wherein, The driving device includes a robot or a vehicle.

13. A path optimization apparatus characterized by comprising: The method comprises: The acquisition module is configured to acquire motion state information of a driving device and a driving path to be optimized, the driving path comprising a plurality of sampling position points; the motion state information representing a motion state of each sampling position point; wherein the motion state is represented based on a plurality of flat variables or derivatives of the flat variables; each flat variable corresponding to a position in one dimension of the sampling position point; the motion state of at least one sampling position point comprising a curvature and a curvature change rate; The processing module is configured to determine a target penalty function according to the motion state information; the target penalty function being related to driving feasibility and smoothness of the driving device when driving along the plurality of sampling position points; wherein the driving feasibility is related to the curvature, and the smoothness is related to the curvature and the curvature change rate; The driving path is optimized according to the target penalty function.

14. The apparatus of claim 13, wherein, The target of optimizing the driving path is to reduce the value of the target penalty function.

15. The apparatus of claim 13 or 14, wherein, The acquisition module is specifically configured to: acquire initial motion state information of the driving device, the initial motion state information being expressed in a first space; change the expression space of the initial motion state information from the first space to a second space to obtain the motion state information, the dimension of the second space being less than or equal to the dimension of the first space, and each dimension of the second space corresponding to a flat variable or a derivative of a flat variable.

16. The apparatus of claim 13 or 14, wherein, The motion state is at least one of a speed, an acceleration, an angular speed, an angular acceleration, a curvature, and a curvature change rate.

17. The apparatus of claim 13 or 14, wherein, The driving path is represented by a high-order polynomial with respect to a time variable, and the optimization of the driving path comprises optimization of the time variable and polynomial coefficients of the high-order polynomial.

18. The apparatus of claim 13 or 14, wherein, The plurality of sampling position points comprise a first sampling point and a second sampling point, the curvature or curvature change rate corresponding to the first sampling point in the driving path being greater than that of the second sampling point, the first sampling point being obtained by sampling the driving path according to a first sampling resolution, the second sampling point being obtained by sampling the driving path according to a second sampling resolution, and the first sampling resolution being greater than the second sampling resolution.

19. The apparatus of claim 13 or 14, wherein, The driving path to be optimized is a section of path selected from a planned path of the driving device, and two ends of the driving path are obtained by sampling the planned path based on a curvature and / or a curvature change rate being less than a threshold value.

20. The apparatus of claim 19, wherein, The driving path includes a sampling point with a curvature and / or a curvature rate greater than a threshold value.

21. The apparatus of claim 13 or 14, wherein, The target penalty function is negatively related to driving feasibility and / or smoothness of the driving device when driving along the plurality of sampling position points.

22. The apparatus of claim 13 or 14, wherein, The acquisition module is further configured to: acquire point cloud data of a driving environment where the plurality of sampling position points are located, the point cloud data being used to represent obstacles in the driving environment; The processing module is further configured to obtain collision risk information of the driving device at each of the sampling position points according to interaction of each point cloud in the point cloud data with an Euclidean distance field (ESDF) of the driving device. The processing module is specifically configured to: optimize the driving path according to the motion state information and the collision risk information.

23. The apparatus of claim 22, wherein, The processing module is specifically configured to: obtain the collision risk information of the driving device at each of the sampling position points according to an ESDF gradient determined according to an ESDF value of a corresponding position of each point cloud in the point cloud data in the ESDF and ESDF values of nearby positions of the corresponding position.

24. The apparatus of claim 13 or 14, wherein, The driving device includes a robot or a vehicle.

25. A traveling device characterized by comprising: The device includes a memory and a processor; the memory stores code, and the processor is configured to execute the code; when the code is executed, the device performs the method of any one of claims 1 to 12.

26. A computer storage medium, comprising, The computer storage medium stores instructions, which, when executed by a computer, cause the computer to implement the method of any one of claims 1 to 12.

27. A computer program product, characterised in that, The computer program product stores instructions, which, when executed by a computer, cause the computer to implement the method of any one of claims 1 to 12.

Images