Motion planning and control method and system of intelligent vehicle fused with machine vision

Abstract

The invention provides a motion planning and control method and system for an intelligent vehicle fused with machine vision, and the method comprises the steps: generating an original motion planning track through the mapping information and an automatic driving demand according to a preset task; processing a visual image perception result of the intelligent vehicle by using a deep neural network to obtain a road surface classification result of a front driving road section, and completing rough road surface classification; performing attachment condition judgment on the road surface classification result, secondarily classifying the road surface classification result into a high type, a middle type and a low type, and distributing corresponding road surface attachment coefficients; based on the road adhesion coefficient, the original motion planning track is re-judged and dynamically corrected, a secondary motion planning track result is obtained, motion control is carried out in combination with a vehicle dynamics model, and an automatic driving task is achieved. According to the method, the active safety of the vehicle can be effectively improved, and the motion safety and adaptability of the intelligent vehicle under the complex road surface condition are enhanced by utilizing multi-source sensing information and introducing a chassis secondary motion planning mechanism.

Description

Technical Field

[0001] This application relates to the technical field of intelligent vehicle motion planning and control, and particularly to a motion planning and control method and system for an intelligent vehicle integrating machine vision. Background Art

[0002] With the popularization and wide application of autonomous driving technology, the penetration rate of intelligent vehicles and traffic penetration rate are gradually increasing. To ensure the safety of future intelligent transportation systems, the investment of major automobile manufacturers' R & D centers in intelligent vehicle safety technology is increasing day by day.

[0003] The core unit modules of intelligent vehicle autonomous driving technology include motion planning and motion control technologies. Based on the environmental perception module, the generation of a reference motion trajectory is achieved through global path planning and local path planning. Then, precise tracking of the reference trajectory is realized through the motion control module.

[0004] As an important part of autonomous driving active safety technology, research has been carried out on the R & D of intelligent vehicle motion planning and control technology. For example, Patent CN114442630B proposes an intelligent vehicle planning control method based on reinforcement learning and model prediction, which uses an on-vehicle lidar sensor to obtain an obstacle grid map and combines the reinforcement learning algorithm of DDPG to achieve vehicle obstacle avoidance trajectory planning and path tracking control. Patent CN116243338A designs an intelligent vehicle based on lidar, which uses this lidar sensor to fuse data processing and control modules, intelligent planning modules, and core control modules to achieve intelligent vehicle motion planning and control. However, the existing lidar sensing scheme has an increase in point cloud noise under complex climate conditions such as rain and snow, and there is a risk of failure under extreme environments or complex road surfaces. Patent CN110161865B proposes an intelligent vehicle lane-changing trajectory planning method based on nonlinear model predictive control. However, this method mainly focuses on the original planned trajectory, lacks the correction and verification of the planned trajectory, and has poor adaptability to algorithms under complex and sudden environments.

[0005] The above intelligent vehicle sports meeting planning and control method mainly targets conventional driving conditions and roads with good adhesion. However, sudden changes in the external road adhesion environment and road conditions make the motion planning results unreasonable, leading to frequent interruptions and exits of the autonomous driving system. Existing adhesion coefficient estimation algorithms based on dynamic models are difficult to provide prior knowledge of a certain road environment. Vehicles often need to drive on specific roads and can only obtain the adhesion conditions of the current road section through dynamic response information, making it impossible to correct and verify the motion planning trajectory in advance, resulting in problems such as reduced vehicle motion control performance and dynamic instability. Therefore, there is an urgent need to introduce a real-time verification and dynamic adjustment mechanism for the original planning results in the existing autonomous driving system. With the introduction of additional sensor information and through the integration with the chassis motion control effect, the adaptability of autonomous driving technology to different road surface environments can be improved. Summary of the Invention

[0006] This application aims to solve at least one of the technical problems in the related art to some extent.

[0007] To this end, the first object of this application is to propose a motion planning and control method for an intelligent vehicle integrating machine vision.

[0008] The second object of this application is to propose a motion planning and control system for an intelligent vehicle integrating machine vision.

[0009] The third object of this application is to propose an electronic device.

[0010] The fourth object of this application is to propose a computer-readable storage medium.

[0011] The fifth object of this application is to propose a computer program product.

[0012] To achieve the above object, the first aspect embodiment of this application proposes a motion planning and control method for an intelligent vehicle integrating machine vision, including:

[0013] Generate an original motion planning trajectory according to a preset task, using mapping information and autonomous driving requirements;

[0014] Process the visual image perception result of the intelligent vehicle using a deep neural network to obtain the road surface classification result of the front driving section and complete the rough road surface classification;

[0015] Judge the adhesion conditions for the road surface classification result, reclassify it into three types: high, medium, and low, and assign corresponding road surface adhesion coefficients;

[0016] Based on the road surface adhesion coefficient, re-judge and dynamically correct the original motion planning trajectory to obtain the secondary motion planning trajectory result, and carry out motion control in combination with the vehicle dynamics model to achieve the autonomous driving task.

[0017] Optionally, using a deep neural network to process the visual image perception result of the intelligent vehicle to obtain the road surface classification result of the front driving section and complete the rough road surface classification, including:

[0018] Process the visual image perception result through an image segmentation network to extract the road surface area in the image;

[0019] Based on the extracted road surface area, use a data calibration method and a road surface adhesion pattern feature extraction network to extract the features of different road surface adhesion conditions;

[0020] Based on the extracted features, perform road surface classification of the front driving section through a road surface classification network to complete the rough road surface classification.

[0021] Optionally, judging the adhesion conditions of the road surface classification result, and secondarily classifying it into three types: high, medium, and low, and assigning corresponding road surface adhesion coefficients, including:

[0022] Based on the road surface classification result, comprehensively consider the current driving speed of the vehicle and the prior knowledge of the road surface material, and secondarily classify the road adhesion classification result into three types: high, medium, and low;

[0023] Assign values to the road adhesion classification results of each type to obtain the corresponding road surface adhesion coefficients.

[0024] Optionally, based on the road surface adhesion coefficient, re-judge and dynamically correct the original motion planning trajectory to obtain the secondary motion planning trajectory result, including:

[0025] Re-judge and dynamically correct the original motion planning trajectory through the chassis secondary motion planning module algorithm, where the chassis secondary motion planning module algorithm is designed as:

[0026]

[0027] In the formula, Y d is the original motion planning trajectory, is a reasonable reference trajectory that satisfies the tracking constraint conditions, the parameter κ > 0, the adjustment function Δ(t) ≥ 0 and satisfies the bounded condition, and η is a positive coefficient to prevent the chassis secondary motion planning module algorithm from having a singularity problem when;

[0028] Among them, the expression of the adjustment function Δ(t) is:

[0029]

[0030] In the formula, the weight coefficients α1, α2, α3, α4 in the adjustment function satisfy α1 + α2 + α3 + α4 = 1, k1, k2, k3, k4 are adjustment factors greater than 0, and the function S β , S γ , identifies the barrier function for a certain independent variable, and J Y is the lateral displacement tracking objective function, and its structure is in the following quadratic form:

[0031]

[0032] In the formula, the subscript Q Y is a weight coefficient greater than 0, and the function is the quadratic function of the control effect with as the coefficient, and is specifically written as:

[0033]

[0034] In the formula, the objective function J Y is the quadratic Lyapunov function for the actual tracking error and the virtual tracking error Y - Y d .

[0035] Optionally, the motion control is carried out in combination with the vehicle dynamics model to achieve the autonomous driving task, including:

[0036] The vehicle dynamics model designs a controller based on the model predictive control algorithm, wherein the cost function in the model predictive control algorithm is expressed as:

[0037]

[0038] Among them, the cost function includes the current lateral position Y of the vehicle, the actual tracking error the virtual tracking error e Y = Y - Y d , the vehicle yaw angle tracking attitude error e ψ = ψ - ψ d and the vehicle lateral stability co-optimization objective function. The objective functions are all in the form of quadratic convex functions. At the same time, the control input front wheel steering angle δ f is selected as the decision variable, and the control input front wheel steering angle δ f is minimized; in the cost function, Q Y , Q ψis the weight factor corresponding to the target of the quadratic form control variable, all of which are positive numbers greater than 0, R>0 is the cost function of the control input, and the parameter N p is the prediction step, N c is the control step, and the state vector is selected as x = [X Y ψ v y γ], the overline on the variable represents the upper bound, and the underline is the lower bound of the variable;

[0039] The constraint conditions that the tracking controller needs to satisfy are:

[0040]

[0041] In the solution of the controller, the constraint range of the input front wheel steering angle satisfies the following stability constraint conditions:

[0042]

[0043] Among them, is the lower bound of the front wheel steering angle, is the upper bound of the front wheel steering angle, is the upper bound of the rear wheel sideslip angle, γ, v y and v x respectively represent the yaw rate, lateral velocity, and longitudinal velocity at the centroid position in the vehicle body coordinate system, m is the vehicle mass, L f and L r respectively represent the distances from the vehicle centroid to the front and rear axles, are the upper bounds of the lateral forces on the front and rear axles.

[0044] To achieve the above object, the second aspect embodiment of the present application proposes a motion planning and control system for an intelligent vehicle integrating machine vision, including:

[0045] A global path planning module, which is used to generate an original motion planning trajectory according to a preset task, using mapping information and autonomous driving requirements;

[0046] A vision feedforward perception module, which is used to process the visual image perception results of the intelligent vehicle using a deep neural network to obtain the road surface classification result of the front driving section and complete the rough road surface classification;

[0047] An adhesion condition judgment module, which is used to judge the adhesion condition of the road surface classification result, secondarily classify it into three types: high, medium, and low, and assign corresponding road surface adhesion coefficients;

[0048] A chassis motion planning and control module, which is used to re-judge and dynamically correct the original motion planning trajectory based on the road surface adhesion coefficient, obtain a secondary motion planning trajectory result, and carry out motion control in combination with the vehicle dynamics model to achieve the autonomous driving task.

[0049] Optionally, the visual feedforward perception module includes:

[0050] An image segmentation module, configured to process the visual image perception result through an image segmentation network, and extract the road surface area in the image;

[0051] A feature extraction module, configured to extract features of different road adhesion conditions based on the extracted road surface area by using a data calibration method and a road adhesion pattern feature extraction network;

[0052] A pattern classification module, configured to classify the road surface of the front driving section based on the extracted features through a road surface classification network, and complete the rough road surface classification.

[0053] To achieve the above object, an embodiment of the third aspect of the present application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0054] The memory stores computer execution instructions;

[0055] The processor executes the computer execution instructions stored in the memory to implement the method according to any one of the above first aspects.

[0056] To achieve the above object, an embodiment of the fourth aspect of the present application provides a computer-readable storage medium, in which computer execution instructions are stored, and when the computer execution instructions are executed by a processor, they are used to implement the method according to any one of the above first aspects.

[0057] To achieve the above object, an embodiment of the fifth aspect of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the method according to any one of the above first aspects.

[0058] The technical solutions provided by the embodiments of the present application at least bring the following beneficial effects:

[0059] (1) The present application obtains the road adhesion coefficient through machine vision, and provides prior knowledge of three road adhesion modes of high, medium, and low adhesion through the front road adhesion mode identification module, so that the chassis system can verify and judge the rationality of the original motion planning trajectory, thereby improving the rationality and executability of the motion planning results of the existing autonomous driving system;

[0060] (2) The present application obtains the road adhesion conditions by using the method of machine vision, rather than based on the vehicle tire dynamics. And the adhesion mode is simplified into three mode classifications of high, medium, and low, and thus introduced into the motion planning and control system of the intelligent vehicle, simplifying the judgment rules of the intelligent vehicle autonomous driving system;

[0061] (3) In this application, a secondary motion planning mechanism is introduced during the chassis motion control process, ensuring the verification and correction of the original planned trajectory by the chassis system, and improving the safety and adaptability of the existing intelligent vehicle motion planning and control technology in complex environments.

[0062] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application. Brief Description of the Drawings

[0063] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, in which:

[0064] Figure 1 is a schematic flowchart of a motion planning and control method for an intelligent vehicle integrating machine vision provided by an embodiment of the present application;

[0065] Figure 2 is a schematic structural diagram of a motion planning and control system for an intelligent vehicle integrating machine vision provided by an embodiment of the present application. Detailed Description of the Embodiments

[0066] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the drawings, in which the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present application and should not be construed as limiting the present application.

[0067] In view of the problems existing in the prior art, the embodiments of the present application provide a motion planning and control method for an intelligent vehicle integrating machine vision. Figure 1 is a schematic flowchart of a motion planning and control method for an intelligent vehicle integrating machine vision provided by an embodiment of the present application. It can be understood that the vehicle motion planning and control method provided in this embodiment can be used not only in the field of vehicle active safety, but also in other fields.

[0068] As Figure 1 shown, the method includes the following steps:

[0069] Step 101, generate an original motion planning trajectory according to a preset task by using mapping information and autonomous driving requirements.

[0070] In the embodiments of the present application, the objective of step 101 is to generate an original motion planning trajectory based on preset task requirements and autonomous driving needs. First, according to the task setting, the driving route of the vehicle is determined by combining the starting point and the target point. At this time, the autonomous driving needs include vehicle driving speed limits, traffic rules along the way, road restrictions, etc. At the same time, dynamic factors in the environment need to be considered, such as traffic signal states, road congestion, etc.

[0071] When generating the motion trajectory, the embodiments of the present application utilize high-precision mapping information. The mapping information usually comes from sensors such as lidar, cameras, GPS, etc. These information cover important information such as road geometry, road markings, traffic signs, and intersections. Through these data, the map of the current environment can be accurately constructed to ensure the accuracy of motion planning.

[0072] In addition to the static mapping information, the embodiments of the present application also incorporate real-time environmental perception data. These data are collected in real time through sensors such as cameras and lidar, and can identify dynamic obstacles, pedestrians, other vehicles, etc. on the road. The combination of these real-time data and mapping information forms a dynamic environment model for more accurate guidance of motion planning.

[0073] Through the above process, the embodiments of the present application can effectively generate an original motion planning trajectory suitable for the current road environment according to the preset tasks and autonomous driving needs, in combination with mapping information and real-time perception data.

[0074] Step 102, using a deep neural network to process the visual image perception results of the intelligent vehicle to obtain the road surface classification results of the front driving section, and completing the rough road surface classification, including:

[0075] In the embodiments of the present application, the objective of step 102 is to process the visual image perception results collected by the intelligent vehicle through a deep neural network, so as to classify the road surface of the front driving section, complete the rough road surface classification, and provide a basis for subsequent adhesion condition judgment and trajectory optimization.

[0076] First, the present application processes the visual image perception results through an image segmentation network to extract the road surface area in the image. The image segmentation network is used to accurately identify the area belonging to the road from complex environmental images, and can be implemented by the currently widely used U-Net network or DeepLab-V3 network. These networks have good feature extraction capabilities and spatial resolution capabilities, which can effectively improve the accuracy of road surface area extraction.

[0077] It can be understood that relevant image segmentation technologies have been widely studied in existing literature, and specific technical details are not elaborated too much in the present application.

[0078] After obtaining the road surface area, the embodiment of the present application further extracts the image features of different road adhesion conditions through the method of data calibration based on the extraction result by using the road adhesion mode feature extraction network. The parameter calibration of the feature extraction network has been completed with a large amount of labeled data during the training phase. Therefore, in the embodiment of the present application, it can be directly used for forward inference without online training, thereby significantly reducing the computational burden of the autonomous driving system during operation.

[0079] Subsequently, based on the extracted adhesion features, the embodiment of the present application classifies the forward driving section by using the road surface classification network to complete the rough road surface classification operation. The classification network can be implemented by using a MobileNet network or other lightweight convolutional neural network (CNN) architectures, and has good operation efficiency and recognition accuracy. The classification results are mainly used to distinguish different types of road surface materials and states, such as asphalt, concrete, muddy ground, gravel, water accumulation or ice and snow coverage, etc., providing a preliminary basis for the next adhesion condition judgment and trajectory adjustment.

[0080] It can be understood that there are mature solutions for related classification algorithms, and the embodiment of the present application will not elaborate and limit the specific model here.

[0081] In summary, the embodiment of the present application efficiently processes visual perception data through a deep neural network, and completes the rough classification of the road surface of the forward driving section by combining key steps such as image segmentation, feature extraction and road surface classification, laying a foundation for the intelligent vehicle to realize environment-adaptive dynamic motion planning and control.

[0082] Step 103: Judge the adhesion condition of the road surface classification result, re-classify it into three types: high, medium and low, and assign corresponding road adhesion coefficients.

[0083] In the embodiment of the present application, the main objective of step 103 is to make a secondary judgment on different road adhesion conditions according to the road surface classification result, classify them into three adhesion types: high, medium and low according to the actual situation, and then assign corresponding road adhesion coefficients to each type.

[0084] First of all, based on the obtained road surface classification result, the embodiment of the present application judges the adhesion condition by comprehensively considering the driving speed of the current vehicle, the prior knowledge of the road surface material and the road adhesion characteristics.

[0085] In practical applications, the driving speed of the vehicle and the road surface material directly affect the road adhesion. For example, on a wet, waterlogged or icy road surface, the adhesion of the vehicle will be significantly reduced, while on a dry and solid road, the adhesion is higher. Therefore, it is necessary to divide different road surfaces into three categories according to the strength of adhesion: high adhesion, medium adhesion and low adhesion according to the road surface classification result.

[0086] Specifically, during the adhesion condition determination process, the embodiments of the present application perform secondary classification according to the road surface classification result in combination with the following factors:

[0087] High adhesion road surface: Generally includes dry asphalt or concrete road surfaces. On these road surfaces, the adhesion of the vehicle is strong, which can provide high stability.

[0088] Medium adhesion road surface: Such as wet asphalt road surface or incompletely dry concrete road surface. The adhesion of these road surfaces is medium, and more caution is required when the vehicle is driving.

[0089] Low adhesion road surface: For example, waterlogged road surface, ice and snow covered road surface or gravel road surface, with low adhesion, and the vehicle is prone to skidding or losing control.

[0090] After completing the secondary classification, the embodiments of the present application assign corresponding adhesion coefficients to each adhesion type to quantify the strength of adhesion under different road surface conditions. For example, the high adhesion type can be assigned a value of 0.8, indicating that the vehicle has strong adhesion on this road surface; the medium adhesion type can be assigned a value of 0.5, indicating moderate adhesion; and the low adhesion type is assigned a value of 0.2, indicating weak adhesion. In actual applications, these adhesion coefficients can be dynamically adjusted according to the actual road conditions, vehicle status and control requirements. Generally, however, through these assigned values, the differences in adhesion of different road surfaces can be effectively reflected.

[0091] Through this process, the embodiments of the present application can reasonably judge and classify the road surface adhesion conditions according to the road surface type and vehicle driving status of the front section, providing an important reference for subsequent movement trajectory adjustment and optimization of vehicle control strategies. This secondary classification and assignment of adhesion coefficients helps to ensure that the vehicle can drive stably and smoothly under different road conditions.

[0092] Step 104: Based on the road surface adhesion coefficient, re-judge and dynamically correct the original motion planning trajectory to obtain the secondary motion planning trajectory result, and carry out motion control in combination with the vehicle dynamics model to achieve the autonomous driving task.

[0093] In the embodiments of the present application, the core purpose of step 104 is to re-judge and dynamically correct the original motion planning trajectory through the chassis secondary motion planning module to adapt to different road surface adhesion conditions and ensure the smooth completion of the autonomous driving task. This step mainly solves the problem that the original motion planning trajectory is unreasonable under low adhesion road surface conditions, resulting in frequent triggering of safety functions, thereby avoiding system interruption or failure.

[0094] To solve the problem that the unreasonable original planned trajectory causes the vehicle to frequently trigger safety functions on unexpectedly low - adhesion road surfaces, resulting in the interruption and even failure of the autonomous driving system, the embodiment of this application re - judges and dynamically corrects the original motion planning trajectory through the algorithm of the chassis secondary motion planning module. The algorithm of the chassis secondary motion planning module is designed as follows:

[0095]

[0096] In the formula, Y d is the original motion planning trajectory, is a reasonable reference trajectory that meets the tracking constraint conditions, the parameter κ > 0, the adjustment function Δ(t) ≥ 0 and satisfies the bounded condition, η is a positive coefficient to prevent the singularity problem of the chassis secondary motion planning module algorithm when. This algorithm optimizes the original trajectory through the adjustment function, making the corrected trajectory more in line with the actual road conditions and the vehicle's motion ability.

[0097] The expression of the adjustment function Δ(t) is:

[0098]

[0099] In the formula, the weight coefficients α1, α2, α3, α4 in the adjustment function satisfy α1 + α2 + α3 + α4 = 1, k1, k2, k3, k4 are adjustment factors greater than 0, the function S β , S γ , identifies the obstacle function for a certain independent variable, J Y is the lateral displacement tracking objective function, and its construction is in the following quadratic form:

[0100]

[0101] In the formula, the subscript Q Y is a weight coefficient greater than 0, the function is the quadratic function of the control effect with as the coefficient, and is specifically written as:

[0102]

[0103] In the formula, the objective function J Y is the quadratic Lyapunov function for the actual tracking error and the virtual tracking error Y - Y d .

[0104] Next, the embodiments of the present application utilize a vehicle dynamics model based on Model Predictive Control (MPC) to design a controller. The core of the MPC algorithm is to predict the future behavior of the vehicle by optimizing the cost function and adjust it according to the minimized control input. In this algorithm, the cost function is usually expressed as:

[0105]

[0106] wherein, the cost function includes the current lateral position Y of the vehicle, the actual tracking error virtual tracking error e Y = Y - Y d , the yaw angle tracking attitude error e ψ = ψ - ψ d and the vehicle lateral stability co-optimize the objective function. The objective functions are all in the form of quadratic convex functions. At the same time, the control input front wheel steering angle δ f is selected as the decision variable, and the control input front wheel steering angle δ f is minimized; in the cost function, Q Y , Q ψ are the weight factors corresponding to the quadratic control variable objectives, all positive numbers greater than 0, R>0 is the cost function of the control input, and the parameter N p is the prediction step, N c is the control step. The state vector is selected as x = [X Y ψ v y γ]. The overline on the variable represents the upper bound, and the underline is the lower bound of the variable.

[0107] The constraint conditions that the tracking controller needs to satisfy are:

[0108]

[0109] In the controller solution, the constraint range of the input front wheel steering angle satisfies the following stability constraint conditions:

[0110]

[0111] wherein, is the lower bound of the front wheel steering angle, is the upper bound of the front wheel steering angle, is the upper bound of the rear wheel side slip angle, γ, v y and v x respectively represent the yaw angular velocity, lateral velocity, and longitudinal velocity at the centroid position in the body coordinate system. m is the vehicle mass, L f and L r respectively represent the distances from the vehicle centroid to the front and rear axles, are the upper bounds of the lateral forces on the front and rear axles.

[0112] Through model predictive control, control inputs such as the front wheel steering angle are optimized to minimize the lateral displacement and control error of the vehicle. The constraint conditions of the control inputs include the upper and lower bounds of the front wheel steering angle, the upper bound of the rear wheel sideslip angle, as well as the vehicle mass and centroid parameters, etc. All the constraint conditions ensure that the vehicle is always in a stable state during driving, avoiding vehicle out of control caused by too large or too small control inputs.

[0113] Through the comprehensive application of these steps, the vehicle can adapt to different road surface conditions during the motion trajectory correction and control process, ensuring the stable operation of the autonomous driving system in a changing road surface environment, and thus efficiently and reliably completing the autonomous driving task.

[0114] In summary, this application improves the adaptability of the current motion planning and control technology of intelligent vehicles to complex road surface environments. By using the road surface feedforward information provided by machine vision and combining with the designed secondary motion planning process, the original planned trajectory is rejudged and verified, making the planned trajectory better match the current chassis execution ability. In addition, the algorithm of this application integrates machine vision information and designs a chassis secondary motion planning mechanism, makes full use of the redundant perception information of the current intelligent vehicle and designs a verification mechanism for the self-driving planned trajectory, improving the adaptability of the intelligent vehicle motion planning and control technology in complex environments.

[0115] To implement the above embodiments, this application also proposes a motion planning and control system for an intelligent vehicle integrating machine vision. Figure 2 It is a schematic structural diagram of a motion planning and control system for an intelligent vehicle integrating machine vision provided by an embodiment of this application. As Figure 2 shown, the device includes:

[0116] A global path planning module 1, configured to generate an original motion planning trajectory according to a preset task, using mapping information and autonomous driving requirements;

[0117] A vision feedforward perception module 2, configured to process the visual image perception result of the intelligent vehicle using a deep neural network to obtain the road surface classification result of the front driving section and complete the rough road surface classification;

[0118] An adhesion condition judgment module 3, for judging the adhesion condition of the road surface classification result, reclassifying it into three types: high, medium, and low, and assigning corresponding road surface adhesion coefficients;

[0119] A chassis motion planning and control module 4, configured to rejudge and dynamically correct the original motion planning trajectory based on the road surface adhesion coefficient to obtain a secondary motion planning trajectory result, and carry out motion control in combination with the vehicle dynamics model to achieve the autonomous driving task.

[0120] The vision feedforward perception module 2 includes:

[0121] An image segmentation module 21, configured to process the visual image perception result through an image segmentation network and extract the road surface area in the image;

[0122] A feature extraction module 22, configured to, based on the extracted road surface area, use a method of data calibration and extract features of different road adhesion conditions through a road adhesion pattern feature extraction network;

[0123] A mode classification module 23, configured to classify the road surface of the forward driving section through a road surface classification network based on the extracted features and complete the rough road surface classification.

[0124] The adhesion condition determination module 3 further includes a secondary motion planning module 41, a vehicle dynamics model 42, and a chassis motion controller 43.

[0125] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0126] To implement the above embodiments, the present application also provides an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided in the foregoing embodiments.

[0127] To implement the above embodiments, the present application also provides a computer-readable storage medium storing computer-executable instructions, and the computer-executable instructions are used to implement the method provided in the foregoing embodiments when executed by a processor.

[0128] To implement the above embodiments, the present application also provides a computer program product, including a computer program, and the computer program implements the method provided in the foregoing embodiments when executed by a processor.

[0129] The collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved in the present application all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0130] It should be noted that personal information from users should be collected for legal and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after obtaining the informed consent of the user, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization including authorizing the relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others with access to the personal information data comply with their privacy policies and procedures.

[0131] This application is expected to provide an implementation scheme that allows users to selectively prevent the use or access of personal information data. That is, the present disclosure is expected to provide hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, the risk can be minimized by restricting data collection and deleting the data. In addition, when applicable, personal identifiers are removed from such personal information to protect the privacy of users.

[0132] In the description of the foregoing embodiments, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0133] In addition, the terms "first" and "second" are used only for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, "a plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0134] Any process or method description shown in the flowchart or described in other ways herein can be understood as representing a module, segment, or part of code including one or more executable instructions for implementing a customized logic function or process, and the scope of the preferred embodiments of this application includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in the reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of this application belong.

[0135] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or used in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then storing it in a computer memory.

[0136] It should be understood that various parts of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0137] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the method of implementing the above embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0138] In addition, each functional unit in various embodiments of the present application may be integrated into a processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0139] The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present application.

[0140] It should be understood that various forms of the flow shown above can be used, steps can be reordered, added or deleted. For example, the steps described in the present application can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present application can be achieved, and no limitation is imposed herein.

[0141] The above specific implementation manners do not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A motion planning and control method for an intelligent vehicle integrating machine vision, characterized in that, Including: Generating an original motion planning trajectory according to a preset task by using mapping information and autonomous driving requirements; Processing the visual image perception result of the intelligent vehicle by using a deep neural network to obtain the road surface classification result of the front driving section and completing the rough road surface classification; Judging the adhesion condition of the road surface classification result, reclassifying it into three types: high, medium, and low, and assigning corresponding road surface adhesion coefficients; Based on the road surface adhesion coefficient, rejudging and dynamically correcting the original motion planning trajectory to obtain a secondary motion planning trajectory result, and carrying out motion control in combination with the vehicle dynamics model to realize the autonomous driving task.

2. The method according to claim 1, wherein The processing the visual image perception result of the intelligent vehicle by using a deep neural network to obtain the road surface classification result of the front driving section and completing the rough road surface classification includes: Processing the visual image perception result through an image segmentation network to extract the road surface area in the image; Based on the extracted road surface area, extracting the features of different road surface adhesion conditions by using a road surface adhesion mode feature extraction network through a data calibration method; Based on the extracted features, classifying the road surface of the front driving section through a road surface classification network to complete the rough road surface classification.

3. The method according to claim 2, wherein The judging the adhesion condition of the road surface classification result, reclassifying it into three types: high, medium, and low, and assigning corresponding road surface adhesion coefficients includes: Based on the road surface classification result, comprehensively considering the current driving speed of the vehicle and the prior knowledge of the road surface material, reclassifying the road adhesion classification result into three types: high, medium, and low; Assigning values to the road adhesion classification results of each type to obtain corresponding road surface adhesion coefficients.

4. The method according to claim 3, characterized in that, The rejudging and dynamically correcting the original motion planning trajectory based on the road surface adhesion coefficient to obtain a secondary motion planning trajectory result includes: Rejudging and dynamically correcting the original motion planning trajectory through the chassis secondary motion planning module algorithm, where the chassis secondary motion planning module algorithm is designed as: where Y d is the original motion planning trajectory, is a reasonable reference trajectory that satisfies the tracking constraint conditions, the parameter κ > 0, the adjustment function Δ(t) ≥ 0 and satisfies the bounded condition, and η is a positive coefficient to prevent the singularity problem from occurring in the algorithm of the chassis secondary motion planning module when Among them, the expression of the adjustment function Δ(t) is: Wherein, the weight coefficients α1, α2, α3, α4 in the adjustment function satisfy α1 + α2 + α3 + α4 = 1, k1, k2, k3, k4 are adjustment factors greater than 0, and the function S β , S γ , identifies the barrier function for a certain independent variable, and J Y is the lateral displacement tracking objective function, and its construction is in the following quadratic form: In the formula, the subscript Q Y is a weight coefficient greater than 0, and the function is a quadratic function of the control effect with as the coefficient, and is specifically written as: In the formula, the objective function J Y is the quadratic Lyapunov function for the actual tracking error and the virtual tracking error Y-Y d .

5. The method according to claim 4, wherein The carrying out motion control in combination with the vehicle dynamics model to realize the autonomous driving task includes: The vehicle dynamics model designs a controller based on a model predictive control algorithm, where the cost function in the model predictive control algorithm is expressed as: Among them, the cost function includes the current lateral position Y of the vehicle and the actual tracking error virtual tracking error e Y = Y - Y d , the yaw angle tracking attitude error e ψ = ψ - ψ d and the lateral stability of the vehicle cooperative optimization objective function. The objective functions are all in the form of quadratic convex functions. At the same time, the control input front wheel steering angle δ f is selected as the decision variable, and the control input front wheel steering angle δ f is minimized; in the cost function, Q Y* , Q Y , Q ψ are the weight factors corresponding to the quadratic control variable objectives, all of which are positive numbers greater than 0, R > 0 is the cost function of the control input, and the parameter N p is the prediction step, N c is the control step. The state vector is selected as x = [X Y ψ v y γ], the overline of the variable represents the upper bound, and the underline is the lower bound of the variable; The constraint conditions that the tracking controller needs to satisfy are: In the controller solution, the constraint range of the input front wheel steering angle satisfies the following stability constraint conditions: Among them, δ f is the lower bound of the front wheel steering angle, is the upper bound of the front wheel steering angle, is the upper bound of the rear wheel sideslip angle, γ, v y and v x respectively represent the yaw rate, lateral velocity and longitudinal velocity at the center of mass position in the vehicle body coordinate system. m is the vehicle mass, L f and L r respectively represent the distances from the vehicle center of mass to the front and rear axles, are the upper bounds of the lateral forces of the front and rear axles.

6. An intelligent vehicle motion planning and control system integrating machine vision, characterized in that, Including: A global path planning module for generating an original motion planning trajectory according to a preset task by using mapping information and autonomous driving requirements; A visual feedforward perception module for processing the visual image perception result of the intelligent vehicle by using a deep neural network to obtain the road surface classification result of the front driving section and completing the rough road surface classification; An adhesion condition judgment module for judging the adhesion condition of the road surface classification result, reclassifying it into three types: high, medium, and low, and assigning corresponding road surface adhesion coefficients; The chassis motion planning and control module is used to re-judge and dynamically correct the original motion planning trajectory based on the road surface adhesion coefficient, obtain the secondary motion planning trajectory result, and carry out motion control in combination with the vehicle dynamics model to achieve the autonomous driving task.

7. The system according to claim 1, wherein The visual feedforward perception module includes: The image segmentation module is used to process the visual image perception result through an image segmentation network and extract the road surface area in the image; The feature extraction module is used to extract the features of different road surface adhesion conditions based on the extracted road surface area by means of data calibration and using a road surface adhesion mode feature extraction network; The mode classification module is used to classify the road surface of the front driving section through a road surface classification network based on the extracted features to complete the rough road surface classification.

8. An electronic device, characterized in that, It includes: A processor and a memory communicatively connected to the processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory to implement the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Computer execution instructions are stored in the computer-readable storage medium, and when the computer execution instructions are executed by a processor, they are used to implement the method according to any one of claims 1-6.

10. A computer program product, characterized in that, It includes a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1-6.

Images