A driver's personalized driving intention prediction method and system

Abstract

The present invention provides a method and system for predicting personalized driving intentions of a driver, and relates to the field of intelligent transportation technology. The method comprises: inputting an internal image and an external image of a vehicle, wherein the internal image is divided into a general group image and an expert group image; preprocessing the internal image to obtain the facial features of the driver, and preprocessing the external image to obtain the environmental features; inputting the facial features and environmental features of the driver into a generalized driving intention prediction network to generate a first prediction result and a confidence interval; learning the driver's intention preference features based on the facial features of the expert group image to obtain a second prediction result; after filtering through the confidence interval, selecting one as the final prediction result. The present invention fully considers the driver's personal intention preferences, improves the accuracy of predicting the intentions of different drivers, and improves the accuracy of the intention prediction task a few seconds in advance.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a method and system for predicting a driver's personalized driving intention. Background Art

[0002] Prediction of human behavior is an important research direction in the field of artificial intelligence. Behavior prediction has applications in various fields, such as intelligent transportation, security monitoring, health care, and social media. Especially in the field of intelligent transportation, driving behavior prediction is an important prerequisite for realizing autonomous driving technology.

[0003] As autonomous driving technology becomes more and more mature, more and more automakers and technology companies are investing in the research and development and production of autonomous vehicles. The application of autonomous vehicles has become popular and has entered the public eye. Surveys show that most road accidents are caused by human factors. In the context of the desire to improve road safety, autonomous driving systems combined with research in the field of artificial intelligence have attracted much attention, which is expected to prevent accidents and reduce driving-related stress.

[0004] However, at present, it may be risky and unsafe to let the autonomous driving system decide on some driving behaviors, such as changing lanes, accelerating and decelerating, and avoiding, etc., and the driver's participation is still required in this link. The purpose of the Advanced Driver Assistance System is to detect the driver's status and provide supplementary information in this link to improve the driver's perception of potential road hazards. The automotive industry has made significant progress in the active safety technology of advanced driver assistance systems, such as electronic stability systems and automatic emergency braking. However, when the advanced driver assistance system detects danger, the driver usually does not have time to react. If the system can predict the driver's intentions a few seconds in advance, the system can prepare for potential dangers before the driver performs the operation, and help the driver control the vehicle in time to prevent a collision. Therefore, how to accurately predict the driver's driving behavior intentions has become one of the important research directions in the development of autonomous driving technology.

[0005] In recent years, driving intention prediction research has obtained reliable data support using mature sensor technology. Reliable multimodal data provides sufficient information for analyzing and predicting driving intention, including vehicle dynamics, global positioning system coordinates, driver head posture, and visual gaze. These data types are integrated through complex algorithms to improve the effectiveness of driver intention analysis. Previous studies have mainly focused on directly fusing different modal features after data preprocessing, including the above data types. Supplementary information from outside the vehicle is usually considered, such as the distance and status between the car and the traffic lights, traffic conditions, and traffic density. In addition, some researchers use the driver cognitive paradigm to construct a driver intention model based only on visual information.

[0006] Despite recent progress in predicting driver intentions, differences in individual preferences have been neglected. Driving is a complex behavior that is influenced by multiple factors, including age, gender, health, psychological factors, and personality traits. Different personality traits lead to different behaviors, and personality traits are stable characteristics of an individual. Numerous studies have shown a link between driving behavior and personality traits. In fact, the intention preferences of different driver groups vary, which has been well demonstrated in experiments.

[0007] In actual driving scenarios, different drivers have different driving preferences, i.e., latent preferences, which leads to huge deviations in the displayed sequence features. When two drivers face the same road choice, their facial 2D trajectories show different characteristics. Different drivers show different feature sequences before performing actions. If the influence of this factor is not taken into account, previous models cannot provide accurate predictions. In fact, when encountering individuals that do not exist in the training dataset, the performance of the model deviates significantly from expectations. In practical applications, it is difficult to achieve stable prediction goals using traditional model design and training frameworks. At the same time, it is impractical to customize the data model for each car according to the owner's specifications before leaving the factory. This impracticality is mainly due to the limitations of privacy information, the uncertainty of the owner, and the huge cost of customization. Therefore, it is necessary to explore alternative solutions. Summary of the invention

[0008] In view of the above problems, the purpose of the present invention is to provide a driver personalized driving intention prediction method and system to solve the problems in the prior art that the driver's personal intention preferences are not taken into account, the intention prediction accuracy of different drivers is poor, and the prediction accuracy is poor a few seconds in advance.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0010] On the one hand, a method for predicting a driver's personalized driving intention is provided, the method comprising the following steps:

[0011] S1. Input the interior image and exterior image of the vehicle, where the interior image is divided into a general group image and an expert group image;

[0012] S2, preprocessing the internal image to obtain the driver's facial features, and preprocessing the external image to obtain environmental features;

[0013] S3, in the generalized driving intention prediction network, the driver's facial features and environmental features are respectively input into the gated recurrent unit GRU, and the data after GRU processing is input into the structural significance module SSM to generate the first prediction result and confidence interval;

[0014] S4. For the expert group image, learn the driver intention preference features based on facial features, obtain the preference vector through importance sampling and gradient calculation, calculate the reward function using the preference vector, generate sampling data consistent with the expert preference, and obtain the second prediction result;

[0015] S5. Evaluate the first prediction result and the second prediction result, and select one as the final prediction result after filtering through the confidence interval.

[0016] Optionally, in step S1, the interior image of the vehicle includes a facial image of the driver, and the exterior image of the vehicle includes an image of the environment that the driver is paying attention to.

[0017] Optionally, in step S2, face recognition processing is performed on the internal image to extract 2D facial trajectory information to obtain the driver's facial features.

[0018] Optionally, the step S3 specifically includes:

[0019] The driver’s facial features and environmental features are input into the multi-head attention module of GRU to update the state; the process is expressed as the following formula:

[0020]

[0021] Among them, ATT represents a multi-head attention module, which is used to receive facial features and environmental features and output updated states; facial features Input GRU to get , as Q and V input to the ATT module; environmental characteristics Input GRU to get , as K input to the ATT module; facial features and environmental features are input to the SSM module after passing through GRU;

[0022] At the same time, the loss function is defined as the cross entropy loss function: , Pred Grepresents the first prediction result, which is the prediction label here, and label represents the confidence interval, thereby jointly modeling the environmental features that the driver is concerned about and the driver's facial features;

[0023] Q represents the query vector, V represents the value vector, and K represents the key vector; represents updated facial features, represents the updated environmental characteristics, Indicates the characteristics of the old environment, represents the new joint feature, Represents the old joint feature; d k represents the hidden layer dimension, and T represents the matrix transpose.

[0024] Optionally, the step S4 specifically includes:

[0025] Based on relative entropy inverse reinforcement learning to derive the preference vector, the reward function R(s,a) is assumed to be a linear function that is linearly related to the observed features f(s,a) on the state-action pair:

[0026]

[0027] Among them, s represents the state and a represents the action. Assuming that there are differences in driving intention characteristics between different individuals, called state preference, θ T is the preference vector, also known as the reward weight parameter; therefore, the reward function of the Markov decision process trajectory is:

[0028]

[0029] where δ represents the trajectory of the Markov decision process, ; If trajectories are collected, the empirical feature expectation is estimated according to the above formula; then, the preference vector is calculated by solving the maximum entropy problem, and the maximum likelihood estimate of the preference vector is calculated using the standard gradient descent method according to the following formula:

[0030]

[0031] Since we cannot know the state-action transition function in the task, we cannot calculate the probability function P(δ); here we use relative entropy inverse reinforcement learning to derive it, use importance sampling to estimate, and transform the solution of the preference vector into the solution of the gradient descent problem:

[0032]

[0033] Among them, TR represents the set of all trajectories, represents a set of sampled trajectories; represents the expert experience feature value based on the sampling trajectory; ζ represents a single sampling trajectory, ζ' represents another single sampling trajectory; f (ζ) represents the expert experience feature value based on a single sampling trajectory, U(ζ) represents the uniform distribution of a single sampling trajectory, ϕ(ζ) represents the trajectory distribution of the sampling strategy of a single sampling trajectory; f (ζ') represents the expert experience feature value based on another single sampling trajectory, U(ζ') represents the uniform distribution of another single sampling trajectory, ϕ(ζ') represents the trajectory distribution of the sampling strategy of another single sampling trajectory.

[0034] Through the above transformation, the gradient descent problem is solved to achieve the driver intention preference learning for a certain expert sampling data.

[0035] Optionally, the step S5 specifically includes:

[0036] If the first prediction result is within the confidence interval, the first prediction result is taken as the final prediction result; if the first prediction result is not within the confidence interval, the second prediction result is taken as the final prediction result.

[0037] On the other hand, a driver personalized driving intention prediction system is provided for implementing any of the above methods, the system comprising:

[0038] An image input module is used to input an interior image and an exterior image of a vehicle, wherein the interior image is divided into a general group image and an expert group image;

[0039] An image processing module is used to preprocess the internal image to obtain the driver's facial features and to preprocess the external image to obtain the environmental features;

[0040] A generalized driving intention prediction module is used to input the driver's facial features and environmental features into the gated recurrent unit GRU in the generalized driving intention prediction network, and the data after GRU processing is input into the structural significance module SSM to generate a first prediction result and a confidence interval;

[0041] A driver intention preference feature learning module is used to learn the driver intention preference feature based on facial features for the expert group image, obtain the preference vector through importance sampling and gradient calculation, calculate the reward function using the preference vector, generate sampling data consistent with the expert preference, and obtain the second prediction result;

[0042] The prediction result determination module is used to evaluate the first prediction result and the second prediction result, and select one as the final prediction result after filtering through the confidence interval.

[0043] In another aspect, an electronic device is provided, the electronic device comprising:

[0044] processor;

[0045] A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are loaded and executed by the processor, the steps of the above-mentioned method for predicting the driver's personalized driving intention are implemented.

[0046] On the other hand, a computer-readable storage medium is provided, in which a program code is stored. The program code can be called by a processor to execute the steps of the above-mentioned driver personalized driving intention prediction method.

[0047] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0048] (1) This invention introduces an innovative personalized driving intention prediction network, which aims to address the impact of driver personality preferences on driving prediction accuracy. Surveys have revealed significant characteristic differences between drivers, and traditional generalized driving intention prediction networks tend to ignore these differences, resulting in predictions that are biased by personal preferences. This invention provides a method to transition from generalized methods to more personalized driving intention prediction models by leveraging deep learning and reinforcement learning techniques, which has been proven to be effective.

[0049] (2) The present invention introduces a method that combines inverse reinforcement learning with deep learning, which effectively evaluates whether the driver's behavior reflects general or personalized intentions, thereby further improving the accuracy of driving intention prediction.

[0050] (3) The present invention designs a time series network based on deep learning, which can effectively improve the prediction accuracy in the task of predicting intentions a few seconds in advance, and has more practical application significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 is a flow chart of a method for predicting a driver's personalized driving intention provided by an embodiment of the present invention;

[0053] Figure 2 It is a schematic diagram of the overall framework of a method for predicting a driver's personalized driving intention provided by an embodiment of the present invention;

[0054] Figure 3 is a schematic diagram of a prediction network structure provided by an embodiment of the present invention;

[0055] Figure 4 It is a structural schematic diagram of a driver's personalized driving intention prediction system provided by an embodiment of the present invention;

[0056] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a concrete way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two. In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings to be expressed are consistent.

[0059] The embodiment of the present invention provides a method for predicting a driver's personalized driving intention. Figure 1 and Figure 2 The flowchart and overall framework diagram of the method are respectively, and the method comprises the following steps:

[0060] S1. Input the interior image and exterior image of the vehicle, where the interior image is divided into a general group image and an expert group image.

[0061] The interior image of the vehicle includes the driver's facial image, and the exterior image of the vehicle includes the environment that the driver is concerned about, such as the environment in front of the vehicle. In order to better predict the driver's personalized intention, the collected interior images are divided into two groups according to different data partitioning protocols named for different drivers: general group images and expert group images.

[0062] S2. Preprocess the internal image to obtain the driver's facial features, and preprocess the external image to obtain environmental features.

[0063] Among them, face recognition processing is performed on all internal images to extract 2D facial trajectory information and obtain the driver's facial features. For external images, image recognition technology can be used for preprocessing to obtain environmental features. The facial features and environmental features are both time series data, so they can be used for the task of predicting intentions a few seconds in advance.

[0064] S3. In the generalized driving intention prediction network, the facial features of the driver in all internal images and the environmental features in the external images are respectively input into the gated recurrent unit GRU. The data after GRU processing is input into the structural significance module SSM to generate the first prediction result (prediction label) and confidence interval.

[0065] To gain a deeper understanding of driving intention, the state modeling of driver decision-making incorporates the external environmental features that the driver attends to before the action. These environmental features are encoded and stored in short-term memory, representing the driver's perception of the current scene. In addition, the driver maintains specific driving intentions and personal styles, which are reflected in long-term memory. This association is related to the transition of temporal features of facial states in the cockpit before the driver's action. The gated recurrent unit (GRU) is the basic recurrent unit that captures and updates the state.

[0066] like Figure 3 As shown, in this step, the driver's facial features and environmental features are input into the multi-head attention module of GRU to update the state; the process is expressed as the following formula:

[0067]

[0068] Among them, ATT represents a multi-head attention module, which is used to receive facial features and environmental features and output updated states; facial features Input GRU to get , as Q and V input to the ATT module; environmental characteristics Input GRU to get , as K input to the ATT module; facial features and environmental features are input to the SSM module after passing through GRU;

[0069] At the same time, the loss function is defined as the cross entropy loss function: , Pred G represents the first prediction result, which is the prediction label here, and label represents the confidence interval, thereby jointly modeling the environmental features that the driver is concerned about and the driver's facial features;

[0070] Q represents the query vector, V represents the value vector, and K represents the key vector; represents updated facial features, represents the updated environmental characteristics, Indicates the characteristics of the old environment, represents the new joint feature, Represents the old joint feature; d k represents the hidden layer dimension, and T represents the matrix transpose.

[0071] The generalized driving intention prediction network uses a sequence of pre-processed image features from inside and outside the vehicle to generate a first prediction result with a confidence interval. If the confidence of the first prediction result is lower than the confidence interval threshold, the network lacks confidence in the input action sequence. This information will be input into the personalized driving intention prediction network, combined with the driver intention preference feature learning to provide further predictions.

[0072] S4. For the expert group image, learn the driver intention preference features based on facial features, obtain the preference vector through importance sampling and gradient calculation, calculate the reward function using the preference vector, generate sampling data consistent with the expert preference, and obtain the second prediction result.

[0073] Considering that the driver has less driving data in the early stage, relative entropy inverse reinforcement learning is used here to avoid poor performance caused by small amount of data. The relative entropy inverse reinforcement learning algorithm proposes a widely used solution to find strategies through a model-free method, that is, designing demonstration behaviors with the highest entropy, that is, expert examples. The method of the present invention is based on the process of deriving the preference vector generation based on relative entropy inverse reinforcement learning. Constrained by the matching feature expectation and the distribution of expert example trajectories, the reward function R(s,a) is assumed to be a linear function, which is linearly related to the observed features f (s,a) on the state-action pair:

[0074]

[0075] Among them, s represents the state and a represents the action. Assuming that there are differences in driving intention characteristics between different individuals, called state preference, θ T is the preference vector, also known as the reward weight parameter; therefore, the reward function of the Markov decision process trajectory is:

[0076]

[0077] where δ represents the trajectory of the Markov decision process, ; If trajectories are collected, the empirical feature expectation is estimated according to the above formula; then, the preference vector is calculated by solving the maximum entropy problem, and the maximum likelihood estimate of the preference vector is calculated using the standard gradient descent method according to the following formula:

[0078]

[0079] Since we cannot know the state-action transition function in the task, we cannot calculate the probability function P(δ); here we use relative entropy inverse reinforcement learning to derive it, use importance sampling to estimate, and transform the solution of the preference vector into the solution of the gradient descent problem:

[0080]

[0081] Among them, TR represents the set of all trajectories, represents a set of sampled trajectories; represents the expert experience feature value based on the sampling trajectory; ζ represents a single sampling trajectory, ζ' represents another single sampling trajectory; f (ζ) represents the expert experience feature value based on a single sampling trajectory, U(ζ) represents the uniform distribution of a single sampling trajectory, ϕ(ζ) represents the trajectory distribution of the sampling strategy of a single sampling trajectory; f (ζ') represents the expert experience feature value based on another single sampling trajectory, U(ζ') represents the uniform distribution of another single sampling trajectory, ϕ(ζ') represents the trajectory distribution of the sampling strategy of another single sampling trajectory.

[0082] Through the above transformation, the gradient descent problem is solved to achieve the driver intention preference learning for a certain expert sampling data.

[0083] S5. Evaluate the first prediction result and the second prediction result, and select one as the final prediction result after filtering through the confidence interval.

[0084] If the first prediction result is within the confidence interval, the first prediction result is taken as the final prediction result; if the first prediction result is not within the confidence interval, the second prediction result is taken as the final prediction result.

[0085] In the embodiments of the present invention, a model combining deep learning and reinforcement learning is used to solve the impact of the driver's personal preferences on the accuracy of driving predictions. An evaluation method combining inverse reinforcement learning and deep learning is used to distinguish between the general intentions and personalized intentions in the driver's behavior, thereby improving the accuracy of intention prediction. The timing network design based on deep learning can improve the accuracy of driving intention prediction a few seconds in advance, which has more practical application value.

[0086] Accordingly, an embodiment of the present invention further provides a driver personalized driving intention prediction system, Figure 4 is a structural block diagram of a driver personalized driving intention prediction system according to an exemplary embodiment. Figure 4 As shown, the system includes:

[0087] An image input module 201 is used to input an interior image and an exterior image of a vehicle, wherein the interior image is divided into a general group image and an expert group image;

[0088] An image processing module 202 is used to pre-process the internal image to obtain the driver's facial features and to pre-process the external image to obtain environmental features;

[0089] A generalized driving intention prediction module 203 is used to input the driver's facial features and environmental features into the gated recurrent unit GRU in the generalized driving intention prediction network, and the data after GRU processing is input into the structural significance module SSM to generate a first prediction result and a confidence interval;

[0090] The driver intention preference feature learning module 204 is used to learn the driver intention preference feature based on the facial features of the expert group image, obtain the preference vector through importance sampling and gradient calculation, calculate the reward function using the preference vector, generate sampling data consistent with the expert preference, and obtain the second prediction result;

[0091] The prediction result determination module 205 is used to evaluate the first prediction result and the second prediction result, and select one as the final prediction result after filtering through the confidence interval.

[0092] For ease of explanation, Figure 4 Only the main components of the system are shown. The system of this embodiment can be used to perform Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0093] In an exemplary embodiment, the present invention further provides an electronic device, the electronic device comprising:

[0094] processor;

[0095] A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are loaded and executed by the processor, the steps of the above-mentioned method for predicting the driver's personalized driving intention are implemented.

[0096] Figure 5 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 5 As shown, the electronic device 300 may include a processor 3001 and a memory 3002. Optionally, the electronic device 300 may further include a transceiver 3003. The processor 3001, the memory 3002 and the transceiver 3003 may be connected, for example, via a communication bus. The memory 3002 stores computer-readable instructions, which, when executed by the processor 3001, implement the steps of the above-mentioned driver personalized driving intention prediction method.

[0097] In a specific implementation, as an embodiment, the processor 3001 may include one or more CPUs, such as Figure 5CPU0 and CPU1 are shown in FIG.

[0098] In a specific implementation, as an embodiment, the electronic device 300 may also include multiple processors, such as Figure 5 3001 and processor 3004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0099] The memory 3002 is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor 3001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0100] The transceiver 3003 is used to communicate with a network device or a terminal device.

[0101] Optionally, the transceiver 3003 may include a receiver and a transmitter, wherein the receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0102] Optionally, the transceiver 3003 may be integrated with the processor 3001 , or may exist independently and be coupled to the processor 3001 via an interface circuit of the electronic device 300 , which is not specifically limited in the embodiment of the present invention.

[0103] It should be noted that Figure 5 The structure of the electronic device 300 shown in the figure does not constitute a limitation on the electronic device. The actual electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In addition, the technical effects of the electronic device 300 can refer to the technical effects of the above-mentioned method embodiment, which will not be repeated here.

[0104] In an exemplary embodiment, the present invention further provides a computer-readable storage medium, wherein at least one instruction is stored in the computer-readable storage medium, and the at least one instruction is loaded and executed by a processor to implement the steps of the above-mentioned driver personalized driving intention prediction method. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0105] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0106] References in the specification to "one embodiment", "an embodiment", "an exemplary embodiment", "some embodiments", etc. indicate that the described embodiments may include a particular feature, structure, or characteristic, but not every embodiment may include the particular feature, structure, or characteristic. In addition, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the relevant art to implement such feature, structure, or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0107] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0108] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0109] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0110] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0111] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0112] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0113] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0114] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for predicting a driver's personalized driving intention, characterized in that: The following steps are involved: S1. Input the interior image and exterior image of the vehicle, where the interior image is divided into a general group image and an expert group image; S2, preprocessing the internal image to obtain the driver's facial features, and preprocessing the external image to obtain environmental features; S3, in the generalized driving intention prediction network, the driver's facial features and environmental features are respectively input into the gated recurrent unit GRU, and the data after GRU processing is input into the structural significance module SSM to generate the first prediction result and confidence interval; S4. For the expert group image, learn the driver intention preference features based on facial features, obtain the preference vector through importance sampling and gradient calculation, calculate the reward function using the preference vector, generate sampling data consistent with the expert preference, and obtain the second prediction result; The step S4 specifically includes: Based on relative entropy inverse reinforcement learning to derive the preference vector, the reward function R(s,a) is assumed to be a linear function, which is linearly related to the observed features f(s,a) on the state-action pair: R(s,a)=θ T ·f(s,a) Among them, s represents the state and a represents the action. Assuming that there are differences in driving intention characteristics between different individuals, called state preference, θ T is the preference vector, also known as the reward weight parameter; therefore, the reward function of the Markov decision process trajectory is: R(δ)=θ T ·f(δ) Where δ represents the trajectory of the Markov decision process, f(δ) = ∑ δ∈TR P(δ)f(δ); if trajectories are collected, the empirical feature expectation is estimated according to the above formula; then, the preference vector is calculated by solving the maximum entropy problem, and the maximum likelihood estimate of the preference vector is calculated using the standard gradient descent method according to the following formula: Relative entropy inverse reinforcement learning is used for derivation, and importance sampling is used for estimation, which transforms the solution of the preference vector into the solution of the gradient descent problem: Among them, TR represents the set of all trajectories, represents a set of sampled trajectories; represents the expert experience feature value based on the sampling trajectory; ζ represents a single sampling trajectory, ζ' represents another single sampling trajectory; f(ζ) represents the expert experience feature value based on a single sampling trajectory, U(ζ) represents the uniform distribution of a single sampling trajectory, φ(ζ) represents the trajectory distribution of the sampling strategy of a single sampling trajectory; f(ζ') represents the expert experience feature value based on another single sampling trajectory, U(ζ') represents the uniform distribution of another single sampling trajectory, φ(ζ') represents the trajectory distribution of the sampling strategy of another single sampling trajectory; Through the above transformation, the gradient descent problem is solved to achieve the driver intention preference learning for a certain expert sampling data; S5. Evaluate the first prediction result and the second prediction result, and select one as the final prediction result after filtering through the confidence interval.

2. The method for predicting driver's personalized driving intention according to claim 1, characterized in that: In step S1, the interior image of the vehicle includes the facial image of the driver, and the exterior image of the vehicle includes the environment image that the driver is concerned about.

3. The method for predicting driver's personalized driving intention according to claim 1, characterized in that: In step S2, face recognition processing is performed on the internal image to extract 2D facial trajectory information to obtain the driver's facial features.

4. The method for predicting driver's personalized driving intention according to claim 1, characterized in that: The step S3 specifically includes: The driver’s facial features and environmental features are input into the multi-head attention module of GRU to update the state; the process is expressed as the following formula: Among them, ATT represents a multi-head attention module, which is used to receive facial features and environmental features and output updated states; facial features Input GRU to get As Q and V input to ATT module; environmental characteristics Input GRU to get As K, it is input into the ATT module; facial features and environmental features are input into the SSM module after passing through GRU; at the same time, the loss function is defined as the cross entropy loss function: loss = CE (Pred G ,label),Pred G represents the first prediction result, which is the prediction label here, and label represents the confidence interval, thereby jointly modeling the environmental features that the driver is concerned about and the driver's facial features; Q represents the query vector, V represents the value vector, and K represents the key vector; Indicates facial features, Indicates environmental characteristics, represents the new joint feature, Represents the old joint feature; d k represents the hidden layer dimension, and T represents the matrix transpose.

5. The method for predicting driver's personalized driving intention according to claim 1, characterized in that: The step S5 specifically includes: If the first prediction result is within the confidence interval, the first prediction result is taken as the final prediction result; if the first prediction result is not within the confidence interval, the second prediction result is taken as the final prediction result.

6. A driver personalized driving intention prediction system, the system is used to implement the method according to any one of claims 1 to 5, characterized in that: The system comprises: An image input module is used to input an interior image and an exterior image of a vehicle, wherein the interior image is divided into a general group image and an expert group image; An image processing module is used to preprocess the internal image to obtain the driver's facial features and to preprocess the external image to obtain the environmental features; A generalized driving intention prediction module is used to input the driver's facial features and environmental features into the gated recurrent unit GRU in the generalized driving intention prediction network, and the data after GRU processing is input into the structural significance module SSM to generate a first prediction result and a confidence interval; A driver intention preference feature learning module is used to learn the driver intention preference feature based on facial features for the expert group image, obtain the preference vector through importance sampling and gradient calculation, calculate the reward function using the preference vector, generate sampling data consistent with the expert preference, and obtain the second prediction result; The prediction result determination module is used to evaluate the first prediction result and the second prediction result, and select one as the final prediction result after filtering through the confidence interval.

7. An electronic device, characterized in that: The electronic device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are loaded and executed by the processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 5.

Images