Intelligent driving obstacle avoidance method, device and equipment based on pedestrian attitude trajectory prediction

Abstract

The invention provides an intelligent driving obstacle avoidance method and device based on pedestrian attitude trajectory prediction, equipment and a storage medium, and belongs to the technical field of intelligent driving. Comprising the following steps: inputting an obtained four-channel RGB image, position information, speed, acceleration, direction, attitude and skeleton key points of a target pedestrian within a preset time length into a preset attitude trajectory prediction model, and predicting coordinates of the skeleton key points of the target pedestrian; and determining an intelligent obstacle avoidance strategy according to the coordinates of the skeleton key points and the driving path data of the target vehicle. According to the invention, the prediction accuracy of the intelligent driving system on pedestrian behaviors can be improved, and the safety of obstacle avoidance decision making is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, in particular to an intelligent driving obstacle avoidance method and device based on pedestrian posture trajectory prediction and equipment. BACKGROUND

[0002] With the continuous development of autonomous driving technology, the safety and obstacle avoidance capability of vehicles are important components of intelligent driving systems. Existing autonomous driving systems mainly rely on sensors such as lidar and cameras to obtain environmental information to make decisions. However, sensors such as lidar can only provide current position information of pedestrians and obstacles, while the behavior of pedestrians has high uncertainty and flexibility, and there is a certain risk in calculating obstacle avoidance only through static data. For example, when a pedestrian accelerates or suddenly turns, the vehicle may not be able to respond in time, posing a potential collision risk. Therefore, an obstacle avoidance system that can predict the next movement of pedestrians in advance is needed to improve the safety of intelligent driving and the ability to respond to dynamic environments. SUMMARY

[0003] The present application provides an intelligent driving obstacle avoidance method and device based on pedestrian posture trajectory prediction, equipment and computer readable storage medium, which can solve the safety risk problem caused by the inability of existing autonomous driving systems to effectively predict pedestrian motion trajectories.

[0004] In a first aspect, the present application provides an intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction, which comprises: By inputting the acquired RGB image, position information, speed, acceleration, direction, posture, and skeletal key points of the target pedestrian within a preset time length into a preset posture trajectory prediction model, the coordinates of the skeletal key points of the target pedestrian are predicted, wherein the RGB image is a 4-channel RGB image. According to the coordinates of the skeletal key points and the driving path data of the target vehicle, an intelligent obstacle avoidance strategy for the target vehicle is determined.

[0005] In combination with the first aspect, in an implementation manner, the determination of the intelligent obstacle avoidance strategy for the target vehicle according to the coordinates of the skeletal key points and the driving path data of the target vehicle comprises: According to the coordinates of the skeletal key points and the driving path data of the target vehicle, the collision risk values of the skeletal key points are obtained. According to each collision risk value, the intelligent obstacle avoidance strategy for the target vehicle is determined.

[0006] In combination with the first aspect, in an implementation manner, the obtaining the collision risk value of the skeleton key point according to the coordinate of the skeleton key point and the driving path data of the target vehicle comprises: calculating the distance from the skeleton key point to the target vehicle according to the coordinate of the skeleton key point and the driving path data of the target vehicle; calculating the collision risk score of the skeleton key point according to the distance from the skeleton key point to the target vehicle and the preset safety distance; generating the dynamic collision weight of the skeleton key point according to the preset basic collision weight of the skeleton key point and the coordinate of the skeleton key point; obtaining the collision risk value of the skeleton key point according to the collision risk score of the skeleton key point and the dynamic collision weight.

[0007] In combination with the first aspect, in an implementation manner, the generating the dynamic collision weight of the skeleton key point according to the preset basic collision weight of the skeleton key point and the coordinate of the skeleton key point comprises: obtaining the offset of the coordinate of the skeleton key point relative to the preset center point coordinate in real time; obtaining each standard offset by normalizing each offset; generating the dynamic collision weight of the skeleton key point according to each standard offset and the preset basic collision weight.

[0008] In combination with the first aspect, in an implementation manner, the determining the intelligent obstacle avoidance strategy of the target vehicle according to each collision risk value comprises: determining the maximum collision risk value by comparing each collision risk value; if the maximum collision risk value is within the low collision risk threshold range, determining the intelligent obstacle avoidance strategy of the target vehicle as the route adjustment strategy; if the maximum collision risk value is within the collision risk threshold range, determining the intelligent obstacle avoidance strategy of the target vehicle as the deceleration avoidance strategy; if the maximum collision risk value is within the high collision risk threshold range, determining the intelligent obstacle avoidance strategy of the target vehicle as the emergency braking strategy.

[0009] In combination with the first aspect, in an implementation manner, the obtaining method of the 4-channel RGB image comprises: obtaining the coordinate position of each skeleton key point of the target pedestrian according to the collected 3-channel RGB image; generating a heat map at the coordinate position of the skeleton key point; The 4-channel RGB image is generated by encoding and superimposing each of the heat maps and the 3-channel RGB image.

[0010] In combination with the first aspect, in an implementation, the obtaining of the coordinate positions of the skeletal key points of the target pedestrian from the collected 3-channel RGB image comprises: The target pedestrian image is identified from the collected 3-channel RGB image based on a preset target detection algorithm. The target pedestrian image block is generated by cropping the target pedestrian image. The coordinate positions of the skeletal key points of the target pedestrian are obtained from the target pedestrian image block based on a preset skeletal key point detection algorithm.

[0011] In the second aspect, the embodiments of the present application provide an intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction, which comprises: The prediction module is configured to input the obtained RGB image, position information, speed, acceleration, direction, posture, and skeletal key points of the target pedestrian within a preset time length into a preset posture trajectory prediction model, and predict the coordinate positions of the skeletal key points of the target pedestrian, wherein the RGB image is a 4-channel RGB image. The obstacle avoidance module is configured to determine an intelligent obstacle avoidance strategy of the target vehicle according to the coordinate positions of the skeletal key points and the obtained driving path data of the target vehicle.

[0012] In the third aspect, the embodiments of the present application provide an intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction, which comprises a processor, a memory, and an intelligent driving obstacle avoidance program based on pedestrian posture trajectory prediction stored in the memory and executable by the processor.

[0013] In the fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores an intelligent driving obstacle avoidance program based on pedestrian posture trajectory prediction.

[0014] The technical scheme provided by the embodiments of the present application has the following beneficial effects: 1. By taking 18 skeletal key point information as an additional input channel to form a 4-channel RGB image, the convolutional neural network can more accurately identify the pedestrian posture.

[0015] 2. Combined with pedestrian skeleton key point prediction information, more accurate pedestrian future position and relative position of each part of the body are obtained, and more detailed collision risk prediction is realized through dynamic weight calculation of different skeleton key points.

[0016] 3. By predicting the pedestrian motion trajectory and skeleton posture in advance, the system can predict the intention of the pedestrian before the pedestrian actually changes direction or speed, effectively reducing the collision risk between the pedestrian and the vehicle.

[0017] 4. It can handle the dynamic behavior of pedestrians suddenly accelerating, turning, etc., improve the adaptability of the system in complex traffic environment, and greatly improve the safety of the system.

[0018] 5. By identifying risks in advance, the system can take measures earlier, reduce emergency braking frequency, improve driving stability and passenger experience, and prolong the service life of the braking system. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 Flowchart of an embodiment of the intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction of the present application; Figure 2 Functional module diagram of an embodiment of the intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction of the present application; Figure 3 Hardware structure diagram of the intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction involved in the embodiment scheme of the present application. DETAILED DESCRIPTION

[0020] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0021] First, some technical terms in the present application are explained and described in order to facilitate the understanding of the present application by those skilled in the art.

[0022] In a first aspect, the embodiments of the present application provide an intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction.

[0023] In an embodiment, with reference to Figure 1 , Figure 1 Flowchart of the first embodiment of the intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction of the present application. As shown in Figure 1The intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction includes the following steps: Step S10: input the acquired RGB image, position information, speed, acceleration, direction, posture, and skeletal key points of the target pedestrian within a preset time length into a preset posture trajectory prediction model to predict the coordinates of the skeletal key points of the target pedestrian, wherein the RGB image is a 4-channel RGB image. For example, the coordinates of the skeletal key points of the target pedestrian are predicted by inputting the acquired RGB image, position information, speed, acceleration, direction, posture, and skeletal key points of the target pedestrian within a preset time length into a preset posture trajectory prediction model. The RGB image is a 4-channel RGB image, which includes three RGB channels and one skeletal key point heat map channel.

[0024] Specifically, the method for acquiring the 4-channel RGB image includes: acquiring the coordinate positions of the skeletal key points of the target pedestrian according to the acquired 3-channel RGB image; generating a heat map at the coordinate positions of the skeletal key points; and generating a 4-channel RGB image by encoding and superimposing each heat map and the 3-channel RGB image.

[0025] For example, first, the coordinate positions of the skeletal key points of the target pedestrian are acquired according to the acquired 3-channel RGB image; then, a heat map is generated at each coordinate position of the skeletal key points, and a two-dimensional Gaussian distribution heat map is generated at the corresponding coordinate position of the input image (size 128×128), and the Gaussian kernel standard deviation is set to 2 pixels, covering the area around the key point; finally, a 4-channel RGB image is generated by encoding and superimposing each heat map and the 3-channel RGB image.

[0026] Specifically, the method for acquiring the coordinate positions of the skeletal key points of the target pedestrian according to the acquired 3-channel RGB image includes: identifying a target pedestrian image from the acquired 3-channel RGB image based on a preset target detection algorithm; and generating a target pedestrian image block by cropping the target pedestrian image. The coordinate positions of the skeletal key points of the target pedestrian are acquired from the target pedestrian image block based on a preset skeletal key point detection algorithm.

[0027] For example, the target pedestrian image is identified from the acquired 3-channel RGB image based on a preset target detection algorithm. Input: each frame of RGB image (1920×1080×3) Output: pedestrian bounding box coordinates (x, y, w, h), where x and y are the top-left corner coordinates, w and h are the width and height; and pedestrian center coordinates ( , ), where , ; a category label of the pedestrian ("pedestrian").

[0028] By cropping the target pedestrian image, a target pedestrian image block is generated, and the cropping formula is:

[0029] The detected pedestrian region is cropped according to the above formula to generate a pedestrian image block, and scaling normalization processing (size adjustment to 128x128) is performed.

[0030] Based on a preset skeleton key point detection algorithm, the coordinate positions of each skeleton key point of the target pedestrian are obtained from the target pedestrian image block. These key points include head key points (nose, neck, left and right eyes, left and right ears), upper body key points (left and right shoulders, left and right elbows, left and right wrists), and lower body key points (left and right hips, left and right knees, left and right ankles), a total of 18 skeleton key points.

[0031] The preset posture trajectory prediction model adopts a gated recurrent unit (GRU) network structure, and the input layer receives a historical position coordinate sequence of the pedestrian, a historical position coordinate sequence of 18 skeleton key points, speed, acceleration, direction, and posture information. The hidden layer includes a GRU layer with 128 hidden units, and uses a tanh activation function. The output layer predicts the center coordinates and 18 skeleton key point coordinates of the pedestrian at a future time.

[0032] GRU network structure: Input layer: Historical position coordinate sequence of the pedestrian , historical position coordinate sequence of 18 key points

[0033] Speed , , acceleration , , direction , posture.

[0034] These input features are input into the GRU in a time series manner in order to capture the dynamic pattern of the pedestrian motion. Specifically, the historical motion information of the pedestrian (center coordinates, skeleton key point coordinates, speed, acceleration, direction, and posture of a certain number of time steps in the past) is used to predict the future motion trajectory.

[0035] The center coordinate and 18 skeleton keypoint coordinates provide both global and local motion information of the pedestrian. The center coordinate reflects the overall position of the pedestrian, while the keypoints reveal the relative positions of individual body parts, allowing for a more detailed representation of the pedestrian's posture and motion changes. By inputting the historical coordinate sequence into the GRU, the model can fully learn the dynamic changes of the pedestrian over the past few time steps, thereby better identifying the pedestrian's motion trends (walking, accelerating, decelerating, stopping, turning) in the short-term future and making more accurate trajectory predictions.

[0036] Velocity and acceleration information can provide the magnitude and rate of change of pedestrian motion, which is important for determining whether the pedestrian is about to slow down, speed up, or make a sharp turn. Direction and posture information further provide the model with clues about the pedestrian's motion intent. For example, when the direction and posture change, the pedestrian may be preparing to cross or turn, and the model can more accurately predict the pedestrian's future trajectory and posture changes based on this information. These dynamic features help the GRU better capture the pedestrian's motion trends, allowing the model to make real-time predictions of complex behaviors such as turning around, bending over, accelerating, and running.

[0037] The pedestrian historical coordinate sequence and the skeleton keypoint coordinate sequence are input in a time series manner, allowing the model to learn multi-level dynamic patterns. In the model's time sequence memory unit, these input features can be gradually stacked to form more in-depth motion patterns, effectively improving the model's ability to adapt to complex and dynamic scenarios.

[0038] Hidden layer:GRU layer with 128 hidden units using tanh activation GRU state update formula: Update gate : Determines how much information from the previous time step will be inherited in the current time step. The formula for the update gate is as follows:

[0039] where, is the Sigmoid function, is the weight matrix, is the hidden state of the previous time step, is the input of the current time step (position, velocity, etc.).

[0040] Reset gate : Controls how to forget the past information, determining how much information from the previous time step has an impact on the current time step. The formula for the reset gate is as follows:

[0041] Candidate hidden state Update the hidden state with current input and historical information:

[0042] Hidden state update: update the hidden state of the current time step using the update gate and the candidate hidden state:

[0043] where the hidden state is used to preserve important temporal information and guide the model's prediction of future trajectories.

[0044] Output layer: The last layer of the GRU outputs the predicted center coordinates of the pedestrian and the 18 key point coordinates of the skeleton at future time points (x .

[0045] By predicting the center coordinates of the pedestrian and the 18 key points of the skeleton, the position and posture changes of the pedestrian are described in more detail. This helps the vehicle to judge the movement trend of the pedestrian in advance and evaluate the spatial relationship of the key points at future time points, thereby improving the accuracy of collision risk assessment.

[0046] Model training: The mean square error (MSE) loss function is used to evaluate the accuracy of the prediction results, and the Adam optimizer is used to update the model weights. The performance of the model is evaluated by the validation set, and the number of GRU layers, the number of hidden units, and the learning rate are optimized as hyperparameters to improve the accuracy and stability of trajectory prediction.

[0047] Step S20: According to the coordinates of the key points and the obtained driving path data of the target vehicle, determine the intelligent obstacle avoidance strategy of the target vehicle.

[0048] According to the coordinates of the key points and the obtained driving path data of the target vehicle, the intelligent obstacle avoidance strategy of the target vehicle is determined. Specifically, it includes two sub-steps: first, according to the coordinates of the key points and the obtained driving path data of the target vehicle, the collision risk values of the key points are obtained; then, according to the collision risk values, the intelligent obstacle avoidance strategy of the target vehicle is determined.

[0049] The process of obtaining the collision risk values of the key points is as follows: first, according to the coordinates of the key points and the obtained driving path data of the target vehicle, the distances from the key points to the target vehicle are calculated, and the future center coordinate trajectory points and 18 key point coordinates of the target pedestrian are set as (x (i=0, 1,..., 18), (x is the center coordinate, and the future trajectory point of the vehicle is predicted by combining the speed (x For each future time t, the distance between the target vehicle and each key point is calculated : ) Then, according to the distance from each skeleton key point to the target vehicle and the preset safety distance , the collision risk score of each skeleton key point is calculated:

[0050] Then, according to the preset basic collision weight of each skeleton key point and the coordinates of each skeleton key point, the dynamic basic collision weight of each skeleton key point is generated; finally, according to the collision risk score and the dynamic basic collision weight of each skeleton key point, the collision risk value of each skeleton key point is obtained.

[0051] The process of generating the dynamic basic collision weight of each skeleton key point is: real-time acquisition of the offset of each skeleton key point coordinate relative to the preset center point coordinate; through normalization processing of each offset, the standard offset of each skeleton key point is obtained; according to the standard offset of each skeleton key point and the preset basic collision weight, the dynamic basic collision weight of each skeleton key point is generated.

[0052] The preset basic collision weight is set according to the principles of ergonomics and collision energy, including: the weight of the pedestrian center position coordinate is 1.0, the weight of the head key point (nose, neck, left and right eyes, left and right ears) is 1.0, the weight of the left and right shoulders is 1.0, the weight of the left and right elbows is 1.0, the weight of the left and right wrists is 0.9, the weight of the left and right hips is 1.2, the weight of the left and right knees is 1.2, and the weight of the left and right ankles is 0.8. The hip and knee are close to the vertical height of the vehicle, and the collision risk is large, so the initial weight is set to a high value; while the wrist and ankle key points are flexible, and are set to a relatively low initial weight.

[0053] Real-time acquisition of the offset of each skeleton key point coordinate relative to the preset center point coordinate:

[0054] Through normalization processing of each offset , the standard offset of each skeleton key point is obtained :

[0055] According to the standard offset of each skeleton key point and the preset basic collision weight , the dynamic basic collision weight of each skeleton key point is generated :

[0056] In the formula: k is the sensitivity coefficient.

[0057] According to the collision risk values, the process of determining the intelligent obstacle avoidance strategy of the target vehicle is: by comparing the collision risk values, the maximum collision risk value is determined; if the maximum collision risk value is within the low collision risk threshold range, the intelligent obstacle avoidance strategy of the target vehicle is determined as the route adjustment strategy; if the maximum collision risk value is within the medium collision risk threshold range, the intelligent obstacle avoidance strategy of the target vehicle is determined as the deceleration avoidance strategy; if the maximum collision risk value is within the high collision risk threshold range, the intelligent obstacle avoidance strategy of the target vehicle is determined as the emergency braking strategy.

[0058] Route adjustment strategy: when the pedestrian is close to the vehicle driving path but not completely coincides, the system actively adjusts the driving route while decelerating to avoid the pedestrian. The path adjustment logic includes dynamic path planning and priority adjustment to ensure the minimum impact on overall driving safety during obstacle avoidance.

[0059] Deceleration avoidance strategy: when the evaluation is medium risk, i.e., the relative position and speed of the pedestrian and the vehicle have the possibility of collision, the collision can be avoided by decelerating in advance. The system gradually reduces the vehicle speed to maintain a safe distance from the pedestrian and ensures smooth deceleration to reduce the impact on passengers.

[0060] Emergency braking strategy: when the evaluation is high risk, the system immediately triggers emergency braking to maximum deceleration to make the vehicle stop as soon as possible. Through the safety control system such as ABS (anti-lock braking system), the braking distance is reduced and the stability of the vehicle during braking is maintained.

[0061] In this embodiment, through fine prediction of pedestrian skeleton key points and accurate evaluation of collision risk, the intelligent driving system improves the prediction ability of pedestrian behavior, can take obstacle avoidance measures earlier, reduces the frequency of emergency braking, improves the stability of driving and passenger experience, prolongs the service life of the braking system, effectively reduces the collision risk between pedestrians and vehicles, and improves the safety of intelligent driving.

[0062] In a second aspect, the embodiments of the present application also provide an intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction.

[0063] In an embodiment, with reference to Figure 2 , Figure 2 is a functional module schematic diagram of an embodiment of the intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction of the present application. As Figure 2 shown, the intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction includes: A prediction module 01 is configured to input the acquired RGB image, position information, speed, acceleration, direction, posture, and skeleton key points of a target pedestrian within a preset time length into a preset posture trajectory prediction model to predict the coordinates of the skeleton key points of the target pedestrian, wherein the RGB image is a 4-channel RGB image. an obstacle avoidance module 02, configured to determine an intelligent obstacle avoidance strategy of the target vehicle according to the coordinates of the skeleton key points and the driving path data of the target vehicle.

[0064] Further, in an embodiment, the obstacle avoidance module 02 is configured to: obtain a collision risk value of each of the skeleton key points according to the coordinates of the skeleton key points and the driving path data of the target vehicle; determine the intelligent obstacle avoidance strategy of the target vehicle according to each of the collision risk values.

[0065] Further, in an embodiment, the obstacle avoidance module 02 is configured to: calculate a distance from each of the skeleton key points to the target vehicle according to the coordinates of the skeleton key points and the driving path data of the target vehicle; calculate a collision risk score of each of the skeleton key points according to the distance from each of the skeleton key points to the target vehicle and a preset safety distance; generate a dynamic collision weight of each of the skeleton key points according to a preset basic collision weight and the coordinates of each of the skeleton key points; obtain a collision risk value of each of the skeleton key points according to the collision risk score and the dynamic collision weight of each of the skeleton key points.

[0066] Further, in an embodiment, the obstacle avoidance module 02 is configured to: obtain an offset of each of the skeleton key points relative to a preset center point coordinate in real time; obtain a standard offset of each of the skeleton key points by normalizing each of the offsets; generate a dynamic collision weight of each of the skeleton key points according to each of the standard offsets and a preset basic collision weight.

[0067] Further, in an embodiment, the obstacle avoidance module 02 is configured to: determine a maximum collision risk value by comparing each of the collision risk values; if the maximum collision risk value is within a low collision risk threshold range, determine that the intelligent obstacle avoidance strategy of the target vehicle is a route adjustment strategy; if the maximum collision risk value is within a collision risk threshold range, determine that the intelligent obstacle avoidance strategy of the target vehicle is a deceleration avoidance strategy; if the maximum collision risk value is within a high collision risk threshold range, determine that the intelligent obstacle avoidance strategy of the target vehicle is an emergency braking strategy.

[0068] Further, in an embodiment, the prediction module 01 is configured to: According to the collected 3-channel RGB image, the coordinate positions of each skeletal key point of the target pedestrian are obtained; A heat map is generated at the coordinate positions of the skeletal key points; Each heat map and the 3-channel RGB image are encoded and superimposed to generate a 4-channel RGB image.

[0069] Further, in an embodiment, the prediction module 01 is configured to: Based on a preset target detection algorithm, a target pedestrian image is identified from the collected 3-channel RGB image; A target pedestrian image block is generated by cropping the target pedestrian image; Based on a preset skeletal key point detection algorithm, the coordinate positions of each skeletal key point of the target pedestrian are obtained from the target pedestrian image block.

[0070] The functions of each module in the above-described intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction correspond to the steps in the above-described embodiment of the intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction, and the functions and implementation processes will not be described here.

[0071] In a third aspect, the embodiments of the present application provide an intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction. The intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction can be a personal computer (PC), a notebook computer, a server, or other devices with data processing functions.

[0072] Reference Figure 3 , Figure 3 The hardware structure of the intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction involved in the embodiments of the present application is shown in the figure. In the embodiments of the present application, the intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction can include a processor, a memory, a communication interface, and a communication bus.

[0073] The communication bus can be of any type, used to interconnect the processor, the memory, and the communication interface.

[0074] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces, which are used to interconnect the devices inside the intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction, and are used to interconnect the intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction with other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber interface, an ATM interface, etc.; the user device can be a display (Display), a keyboard (Keyboard), etc.

[0075] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), and the like.

[0076] The processor can be a general-purpose processor, which can invoke the intelligent driving obstacle avoidance program based on pedestrian posture trajectory prediction stored in the memory and execute the intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction provided by the embodiments of the present application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the intelligent driving obstacle avoidance program based on pedestrian posture trajectory prediction is invoked can refer to various embodiments of the intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction of the present application, which will not be described here.

[0077] Those skilled in the art can understand that the hardware structure shown in the above-mentioned embodiments is not a limitation of the present application, and can include more or less components than the illustrated components, or combine certain components, or different component arrangements. Figure 3

[0078] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium.

[0079] The computer readable storage medium of the present application stores the intelligent driving obstacle avoidance program based on pedestrian posture trajectory prediction, wherein when the intelligent driving obstacle avoidance program based on pedestrian posture trajectory prediction is executed by the processor, the steps of the intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction as described above are implemented.

[0080] The method implemented when the intelligent driving obstacle avoidance program based on pedestrian posture trajectory prediction is executed can refer to various embodiments of the intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction of the present application, which will not be described here.

[0081] It should be noted that the above-mentioned sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0082] ​The terms “include,” “comprise,” “have,” and any variations thereof, in the Specification and in the Claims of the present application, and the above-mentioned drawings, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or device that includes a list of steps or units is not limited to the listed steps or units, but can optionally further include steps or units not listed, or can optionally further include other steps or units inherent to such processes, methods, products, or devices. The terms “first,” “second,” and “third” and the like descriptions are used to distinguish different objects, and do not represent a sequence or limit the types of “first,” “second,” and “third.”

[0083] In the description of the embodiments of the present application, “exemplary”, “for example”, or “for instance” is used to represent an example, an illustration, or a description. Any embodiment or design scheme described as “exemplary”, “for example”, or “for instance” in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the words “exemplary”, “for example”, or “for instance” are intended to present the relevant concept in a specific manner.

[0084] In the description of the embodiments of the present application, unless otherwise specified, “ / ” represents the meaning of or, for example, A / B can represent A or B; “and / or” in the text only represents a description of the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. In addition, in the description of the embodiments of the present application, “multiple” means two or more than two.

[0085] In some of the processes described in the embodiments of the present application, a plurality of operations or steps are included in a specific order, but it should be understood that these operations or steps can be executed or performed in parallel or in an order different from that in which they appear in the embodiments of the present application. The serial number of the operation is only used to distinguish different operations, and the serial number itself does not represent any execution order. In addition, these processes can include more or fewer operations, and these operations or steps can be executed in sequence or in parallel, and these operations or steps can be combined.

[0086] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and a general hardware platform as required, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, an optical disk) as described above, and includes a plurality of instructions for causing a terminal device to execute the methods described in the embodiments of the present application.

[0087] The above merely provides the preferred embodiments of the present application and not therefore limit the patent range of the present application, any equivalent structure or equivalent flow transformation made by using the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection range of the present application.

Claims

1. An intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction, characterized in that: The intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction includes: The coordinates of the target pedestrian's skeletal key points are predicted by inputting the acquired RGB image, position information, speed, acceleration, direction, posture, and skeletal key points of the target pedestrian within a preset time period into a preset posture trajectory prediction model, wherein the RGB image is a 4-channel RGB image; An intelligent obstacle avoidance strategy for the target vehicle is determined based on the coordinates of the skeleton key points and the acquired driving path data of the target vehicle.

2. The intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction according to claim 1, characterized in that: Determining the intelligent obstacle avoidance strategy of the target vehicle based on the coordinates of the skeleton key points and the acquired driving path data of the target vehicle includes: Obtaining a collision risk value of the skeleton key point according to the coordinates of the skeleton key point and the acquired driving path data of the target vehicle; An intelligent obstacle avoidance strategy for the target vehicle is determined based on each of the collision risk values.

3. The intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction according to claim 2, characterized in that: The step of obtaining the collision risk value of the skeleton key point according to the coordinates of the skeleton key point and the acquired driving path data of the target vehicle includes: Calculate the distance from the skeleton key point to the target vehicle based on the coordinates of the skeleton key point and the acquired driving path data of the target vehicle; Calculating a collision risk score of the skeleton key point based on the distance from the skeleton key point to the target vehicle and a preset safety distance; Generate a dynamic collision weight of the skeleton key point according to the preset basic collision weight of the skeleton key point and the coordinates of the skeleton key point; The collision risk value of the skeleton key point is obtained according to the collision risk score and the dynamic collision weight of the skeleton key point.

4. The intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction according to claim 3, characterized in that: Generating the dynamic collision weight of the skeleton key point according to the preset basic collision weight of the skeleton key point and the coordinates of the skeleton key point includes: Obtaining in real time the offset of the coordinates of the skeleton key points relative to the coordinates of the preset center point; By normalizing each of the offsets, each standard offset is obtained; The dynamic collision weights of the skeleton key points are generated according to the respective standard offsets and the preset basic collision weights.

5. The intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction according to claim 2, characterized in that: Determining the intelligent obstacle avoidance strategy of the target vehicle according to each of the collision risk values ​​includes: By comparing the respective collision risk values, a maximum collision risk value is determined; If the maximum collision risk value is within a low collision risk threshold range, determining that the intelligent obstacle avoidance strategy of the target vehicle is a route adjustment strategy; If the maximum collision risk value is within the collision risk threshold range, determining that the intelligent obstacle avoidance strategy of the target vehicle is a deceleration avoidance strategy; If the maximum collision risk value is within a high collision risk threshold range, the intelligent obstacle avoidance strategy of the target vehicle is determined to be an emergency braking strategy.

6. The intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction according to claim 1, characterized in that: The method for obtaining the 4-channel RGB image includes: According to the collected 3-channel RGB image, the coordinate position of each skeleton key point of the target pedestrian is obtained; Generate a heat map at the coordinate positions of the skeleton key points; A 4-channel RGB image is generated by encoding and superimposing each of the heat maps and the 3-channel RGB image.

7. The intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction according to claim 6, characterized in that: The method of obtaining the coordinate positions of the skeleton key points of the target pedestrian based on the collected 3-channel RGB image includes: Based on the preset target detection algorithm, the target pedestrian image is identified from the collected 3-channel RGB image; Generate a target pedestrian image block by cropping the target pedestrian image; Based on a preset skeleton key point detection algorithm, the coordinate position of each skeleton key point of the target pedestrian is obtained from the target pedestrian image block.

8. An intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction, characterized in that: The intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction includes: A prediction module is configured to predict the coordinates of the target pedestrian's skeletal key points by inputting the acquired RGB image, position information, speed, acceleration, direction, posture, and skeletal key points of the target pedestrian within a preset time period into a preset posture trajectory prediction model, wherein the RGB image is a 4-channel RGB image; The obstacle avoidance module is used to determine the intelligent obstacle avoidance strategy of the target vehicle based on the coordinates of the skeleton key points and the acquired driving path data of the target vehicle.

9. An intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction, characterized in that: The intelligent driving obstacle avoidance device based on pedestrian posture trajectory prediction includes a processor, a memory, and an intelligent driving obstacle avoidance program based on pedestrian posture trajectory prediction stored in the memory and executable by the processor, wherein when the intelligent driving obstacle avoidance program based on pedestrian posture trajectory prediction is executed by the processor, the steps of the intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores an intelligent driving obstacle avoidance program based on pedestrian posture trajectory prediction, wherein when the intelligent driving obstacle avoidance program based on pedestrian posture trajectory prediction is executed by the processor, the steps of the intelligent driving obstacle avoidance method based on pedestrian posture trajectory prediction as described in any one of claims 1 to 7 are implemented.

Images