A driver intention recognition model establishment method, device, equipment and medium

Abstract

This invention provides a method, apparatus, device, and medium for establishing a driver intent recognition model, relating to the field of automotive intelligent interaction technology. The method for establishing the driver intent recognition model includes the following steps: collecting camera image data, radar data, and vehicle sensor data; processing the collected information to obtain sample data; feeding the sample data into a training model to train the driver intent recognition model; and after obtaining the driver intent recognition model, inputting new sample data to identify the driver's driving intent. This invention achieves accurate recognition of driver intent based on the driver intent recognition model and has strong practicality.

Description

Technical Field

[0001] This invention relates to the field of automotive intelligent interaction technology, and in particular to a method, apparatus, device, and medium for establishing a driver intent recognition model. Background Technology

[0002] In recent years, with the continuous development of deep learning technology, significant progress has been made in identifying semantics from text or images. This technology has been extended to other fields, such as car driving. From the moment a driver unlocks the vehicle, many scenarios with clear intentions occur during each use of the car. For example, when a driver unlocks the car, gets in, and starts the engine, their intention is highly likely to drive the vehicle out. The characteristics of these scenarios can be identified by analyzing changes in all vehicle information over a period of time. Current technologies typically rely on product managers defining a limited number of scenarios and predicting user intentions by listening to limited conditions. This method has the following problems: the scenarios defined by product managers are limited, and due to limited personal knowledge, they cannot represent general user intentions; scenarios judged by limited conditions cannot meet the personalized needs of each user. Summary of the Invention

[0003] In view of the shortcomings of the prior art described above, the present invention provides a method, apparatus, device and medium for establishing a driver intent recognition model to solve the above technical problems.

[0004] The present invention provides a method for establishing a driver intent recognition model, comprising the following steps:

[0005] Acquire data inside and outside the vehicle in different scenarios, including image data, radar data, and vehicle sensor data;

[0006] The image data is processed to generate different visual perception events, and the radar data is processed to generate perceived target and status data;

[0007] The visual perception events, the perceived targets, and the state data are encoded and fused with the vehicle sensor data to obtain sample data.

[0008] The sample data is used to train a neural network that incorporates an attention mechanism to obtain a driver intention recognition model.

[0009] In one embodiment of the present invention, the image data is processed to generate different visual perception events, specifically including:

[0010] The image data is processed to obtain the pixel count and frame rate supported by the image recognition algorithm.

[0011] The processed image data is input into the image recognition algorithm to obtain the visual perception event;

[0012] The visual perception events include whether the driver is present, whether the driver is fatigued, and whether there is a passenger in the front seat.

[0013] In one embodiment of the present invention, the radar data is processed to generate target perception and status data, specifically including:

[0014] Radar data is input into the radar identification algorithm to obtain the perceived target and its status data.

[0015] The perceived targets include other vehicles in the vicinity, and the status includes the distance of other vehicles from the current vehicle.

[0016] In one embodiment of the present invention, the vehicle sensor data specifically includes:

[0017] The vehicle sensor data includes the vehicle's ignition status, vehicle speed, window status, and the temperature or humidity inside and outside the vehicle.

[0018] In one embodiment of the present invention, the step of encoding the visual perception event, the perception target, and the state data, and fusing them with the vehicle sensor data to obtain sample data includes:

[0019] All data are segmented according to the preset time window width;

[0020] Process each data segment according to its data type, including taking the average value within a time window for continuous data;

[0021] For event-driven data, retrieve the latest state within the time window;

[0022] Apply constraints to the processed data;

[0023] The sample data is obtained by fusing the constrained data.

[0024] In one embodiment of the present invention, the constraint processing includes bucketing, normalization, or standardization.

[0025] In one embodiment of the present invention, the step of encoding the visual perception event, the perception target, and the state data specifically includes:

[0026] For the visual perception events obtained by the image recognition algorithm, a pre-established table of corresponding visual perception events is queried for encoding.

[0027] For the multiple sensing targets and their states closest to the vehicle obtained by the radar identification algorithm, a pre-established correspondence table of sensing targets and states is queried and encoded.

[0028] In one embodiment of the present invention, the driver intent recognition model introduces an attention mechanism to automatically learn and calculate the contribution of input data to output data, and automatically identifies the input data that has a greater impact on each scenario.

[0029] The present invention also provides a driver intent recognition model building device, comprising:

[0030] The data acquisition module is used to acquire data inside and outside the vehicle in different scenarios, including image data, radar data, and vehicle sensor data.

[0031] The data processing module is used to process the image data to generate different visual perception events and to process the radar data to generate perceived target and status data.

[0032] The data fusion module is used to encode the visual perception events, the perception targets, and the state data, and to fuse them with the vehicle sensor data to obtain sample data.

[0033] The training data module is used to train a neural network incorporating an attention mechanism using the sample data to obtain a driver intention recognition model.

[0034] The present invention also provides an electronic device, comprising:

[0035] One or more processors;

[0036] A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the driver intent recognition model building method as described in any of the above embodiments.

[0037] The present invention also provides a computer-readable storage medium, characterized in that it stores computer-readable instructions thereon, which, when executed by a computer's processor, cause the computer to perform the driver intent recognition model establishment method as described in any of the above embodiments.

[0038] The beneficial effects of this invention are as follows: Based on multimodal data information acquired from different scenarios, this invention processes and fuses the modal information and trains it on a training model to obtain a model that can accurately identify driver intentions. This method can overcome the limitations of human-defined scenarios and the limitations of individual cognition. Compared with previous driver intention recognition and prediction under fixed scenarios, this invention targets driver intention recognition in real-time scenarios. It processes and trains the model based on data acquired in real-time scenarios to obtain a trained driver intention recognition model. This model can effectively improve the accuracy of driver intention recognition, thereby better serving drivers and improving the accuracy and practicality of driver intention recognition.

[0039] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0041] Figure 1 This is a schematic diagram illustrating the implementation environment of establishing a driver intent recognition model, as shown in an exemplary embodiment of the present invention.

[0042] Figure 2 This is a flowchart illustrating the establishment of a driver intent recognition model, as shown in an exemplary embodiment of the present invention.

[0043] Figure 3 yes Figure 2 The flowchart of step S240 in the illustrated embodiment is shown in an exemplary embodiment;

[0044] Figure 4 This is a block diagram illustrating a driver intent recognition model building device according to an exemplary embodiment of the present invention;

[0045] Figure 5 A schematic diagram of a computer system suitable for implementing embodiments of the present invention is shown. Detailed Implementation

[0046] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0047] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0048] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0049] First, it's important to clarify that training a model involves using existing data to determine the parameters of a function through various methods. The function with determined parameters is the result of the training. Using the model involves substituting new data into the function to evaluate it. The embodiments of this invention relate to the establishment of a driver intent recognition model. This involves processing and fusing collected vehicle and other relevant data as training samples. These training samples are then input into the training model, resulting in a driver intent recognition model after training. The training model employs machine learning techniques to simulate and recognize driver intent.

[0050] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0051] In the process of establishing a driver intent recognition model, data from different modalities in different scenarios are aggregated and fused to obtain sample data. Based on the different sample data, the driver intent recognition model can obtain different driver intentions; the more sample data, the more accurate the intent recognition model. Taking the application scenario of a vehicle driving on a city street after rain as an example, because the driver's vision is obstructed in the rain, making it difficult to see specific information on the road, the driver intent recognition model obtains this sample data. This sample data is then used to train the driver intent recognition model to obtain prediction results, thereby predicting the driver's intention to turn on the low beam headlights. In other application scenarios, different predicted intentions can be generated based on different sample data, and different driver intentions can be obtained according to the actual situation. The embodiments of this application do not limit this.

[0052] Figure 1 This is a schematic diagram illustrating an implementation environment for establishing an intent recognition model, as shown in an exemplary embodiment of this application. Figure 1 As shown, the vehicle acquires data through the data collection device 110 on the vehicle during driving. That is, the data collection device 110 collects data around the vehicle and sends the collected data to the data processing server 120. The data processing server 120 processes the collected data to obtain sample data, and inputs the sample data into the training model for training. After training, the driver intention recognition model is obtained. The driver intention recognition model can predict the driver's intention based on the new input sample data.

[0053] in, Figure 1 The data collection device 110 shown can be any software or hardware that supports data collection, such as an in-vehicle thermometer, radar sensor, in-vehicle or out-of-vehicle camera, or Hall sensor, but is not limited to these. Figure 1 The data processing server 120 shown is a data processing server, which can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, or an in-vehicle server. No restrictions are placed on this. The data collection device 110 can communicate with the data processing server 120 via wireless networks such as 3G (third-generation mobile information technology), 4G (fourth-generation mobile information technology), and 5G (fifth-generation mobile information technology), or transmit data with the in-vehicle server via a data bus. No restrictions are placed on this either.

[0054] Existing intent recognition technologies typically define only a limited set of scenarios, predicting user intent by listening to a finite number of conditions. However, because these human-defined scenarios are limited, and individual perception is also limited, they cannot represent universal user intent. Scenario-based judgments based on limited conditions cannot meet the personalized needs of each user. The following concrete example illustrates the impact of limited scenario definitions on user experience:

[0055] For example, in a scenario where the outside temperature is suitable but the inside temperature of the car is relatively high, when the driver is driving, the cold air blowing from the outside initially lowers the outside temperature, so the driver closes the windows to keep the inside warm. As the journey progresses, the cold air gradually stops, and because the windows are closed, the air inside the car cannot circulate sufficiently with the outside air, causing the inside temperature to rise and the air to become stale. In a human-defined scenario, the driver's intention is to turn on the air conditioning to ventilate the car, based on the assumption that the inside temperature is rising and the air is stale. However, the driver's actual intention is to open the windows for ventilation. This human-defined scenario misjudges the driver's intention, leading to a lack of trust in the scenario prediction. This incorrect prediction of the intention may cause a certain degree of distraction for the driver, affecting driving safety.

[0056] The problems mentioned above are universally applicable in common travel scenarios. It can be seen that in a limited number of artificially defined scenarios, the driver's intent cannot be accurately matched, leading to misjudgment of the driver's intent, inaccurate operations, and various problems. To solve these problems, embodiments of this application propose a method for establishing a driver intent recognition model, a device for establishing a driver intent recognition model, an electronic device, a computer-readable storage medium, and a computer program product, which will be described in detail below.

[0057] Figure 2 This is a flowchart illustrating a method for establishing a driver intent recognition model, as shown in an exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown is specifically executed by the vehicle-mounted data collection device 110 and the data processing server 120 within that implementation environment. It should be understood that this method can also be applied to other exemplary implementation environments and executed by devices in other implementation environments; this embodiment does not limit the implementation environment to which the method is applicable.

[0058] For example, the data processing server 120 to which the driver intent recognition model establishment method disclosed in this embodiment is applicable can be configured with a training model, and the method disclosed in this embodiment is specifically implemented by training the training model to obtain the driver intent recognition model.

[0059] like Figure 2As shown, in an exemplary embodiment, the driver intent recognition model establishment method includes at least steps S200 to S250, which are described in detail below:

[0060] Step S200: Obtain vehicle interior and exterior data for different scenarios.

[0061] First, it should be noted that a scenario refers to a certain condition that generates a demand. This condition includes, but is not limited to, environment, time, location, space, etc. Only when the condition is met can the demand be established. In contrast, in-vehicle and out-of-vehicle data refers to data related to the driver in a scenario. In this embodiment, in-vehicle and out-of-vehicle data in different scenarios refers to camera data acquired by in-vehicle and out-of-vehicle cameras, radar data acquired by radar, and vehicle sensor data.

[0062] It should be noted that vehicle sensor data refers to data that can influence the driver's intentions. It usually refers to data around the vehicle, such as the temperature inside and outside the vehicle, the humidity inside and outside the vehicle, the vehicle speed, and the brightness of the light around the vehicle.

[0063] Step S210: Acquire camera visual perception events.

[0064] First, it's important to clarify that a camera refers to a device capable of capturing images. It typically refers to an input device that can acquire specific images, such as in-vehicle cameras, mobile phone cameras, and wearable devices. Correspondingly, visual perception events refer to visual perception events generated by cameras capturing data inside and outside the vehicle. These events are generated by processing the real-time image data into pixels and frame rates, and then using image recognition algorithms to further process the pixels and frame rates. Examples of visual perception events include whether the driver is present, whether the driver is fatigued, whether there is someone in the passenger seat, whether the driver is smoking, and whether the driver is drinking water.

[0065] Step S220: Obtain the radar-sensing target and its status.

[0066] First, it's important to clarify that radar is a device capable of identifying surrounding objects, such as vehicle-mounted radar and mobile phone radar. Correspondingly, the radar data acquired is input into radar perception algorithms to identify the perceived target and its status. For example, targets might be other vehicles or pedestrians in the vicinity, and statuses might include the distance between the vehicle and other vehicles, or the distance between pedestrians and other pedestrians.

[0067] It should also be mentioned that the processing of real-time acquired image data into pixels and frame rate, the processing of image recognition algorithms, and the identification of perceived targets and states by inputting radar data acquired by radar into radar perception algorithms can be completed on the vehicle-mounted system, on the data processing server 120, or on a processing unit with preset relevant algorithms. This embodiment does not limit this.

[0068] Step S230: Encode the visually perceived events, radar-sensed targets, and states.

[0069] All visual perception events are encoded to generate a mapping table of visual perception events. The latest visual perception event is obtained, and the corresponding code is found in the pre-established mapping table of visual perception events. The corresponding code is then assigned to the visual perception event.

[0070] The target and status data of all radars are encoded to generate a correspondence table of target and status. Multiple target and status data closest to the vehicle are selected, and the corresponding codes are found in the pre-established correspondence table of target and status data. The corresponding codes are then assigned to the multiple target and status data.

[0071] The encoding transforms data that is difficult for the training model to process into data that is easy to process. For example, whether a driver smokes is encoded as A1, whether a driver drinks water is encoded as B2, and whether a pedestrian on the road is making a phone call is encoded as C3.

[0072] Step S240: Within a preset time window, all data are merged.

[0073] like Figure 3 As shown, step S310 is executed, and all data are segmented according to a preset time window width. The preset time window width is set differently according to different scenarios. All data includes the encoded data of visual perception events obtained after processing camera data, the encoded data of targets and their states obtained after processing radar data, and the data of vehicle sensors.

[0074] After segmenting all data, step S320 is executed to process the segmented data according to its data type. For continuous data, the average value within a preset time window is taken, such as vehicle speed or interior temperature. For event-driven data, such as visual perception events or radar-sensed targets, the latest state within a preset time window is taken, such as whether the driver is fatigued or whether the driver is in position. The segmented data may require data constraint processing, which can be achieved through binning, normalization, or standardization to constrain the data range and prevent large data distribution differences from affecting model training results. Step S330 is then executed to fuse the constrained data to obtain sample data for training.

[0075] Step S250: Train the fused data.

[0076] The sample data obtained after fusion processing is trained on a preset neural network, automatically learning and continuously updating to obtain a driver intention recognition model. The driver intention recognition model introduces an attention mechanism to automatically learn and calculate the contribution of input data to output data, and automatically identify the input data that has a greater impact on each scenario. The attention mechanism is calculated as follows:

[0077]

[0078]

[0079] Where h represents the input sample data, i and j represent the number of groups, n represents the total number of sample data, q represents the task-related query vector, s represents the attention scoring function, softmax represents the normalization exponential function, and a represents the attention distribution. i Let h represent the attention distribution of the i-th data set. i Let h represent the sample data of the i-th input group. j Let j represent the sample data of the j-th input group, and context represent the content that the model should focus on at the moment.

[0080] Figure 4 This is a block diagram illustrating a driver intent recognition model device according to an exemplary embodiment of this application. The device can be applied to… Figure 1 The implementation environment shown is specifically configured in the vehicle-mounted data collection device 110. This device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0081] like Figure 4 As shown, the exemplary driver intent recognition model device includes:

[0082] The data acquisition module 410 is used to acquire data inside and outside the vehicle in different scenarios. The data includes image data, radar data and vehicle sensor data. The acquired data is sent to the data processing module 420.

[0083] The data processing module 420 is used to process image data to generate different visual perception events and to process radar data to generate perceived target and status data.

[0084] The data fusion module 430 is used to encode the visual perception event, the perceived target and the state data, and fuse them with the vehicle sensor data to obtain sample data.

[0085] Training data module 440 is used to train a neural network incorporating an attention mechanism using sample data to obtain a driver intention recognition model.

[0086] It should be noted that the driver intent recognition model device and the driver intent recognition model method provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs its operations have been described in detail in the method embodiments and will not be repeated here. In practical applications, the driver intent recognition model device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0087] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the driver intent recognition model establishment method provided in the above embodiments.

[0088] Figure 5 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 5 The computer system 500 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0089] like Figure 5 As shown, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 502 or programs loaded from storage portion 508 into Random Access Memory (RAM) 503, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in RAM 403. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An Input / Output (I / O) interface 505 is also connected to the bus 504.

[0090] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0091] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs various functions defined in the system of this application.

[0092] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0093] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0094] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0095] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the traffic condition refresh method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not incorporated into the electronic device.

[0096] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the traffic update method provided in the various embodiments described above.

[0097] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for establishing a driver intent recognition model, characterized in that, Includes the following steps: Acquire vehicle interior and exterior data in different scenarios. The vehicle interior and exterior data includes image data, radar data, and vehicle sensor data. The vehicle sensor data includes vehicle ignition status, vehicle speed, window status, vehicle interior and exterior temperature, vehicle interior and exterior humidity, and ambient light intensity. The image data is processed to generate different visual perception events, and the radar data is processed to generate perceived target and status data. The visual perception events include whether the driver is in place, whether the driver is fatigued, whether there is someone in the passenger seat, whether the driver is smoking, and whether the driver is drinking water. The visual perception events, the perceived targets, and the state data are encoded and fused with the vehicle sensor data to obtain sample data. The sample data is used to train a neural network that incorporates an attention mechanism to obtain a driver intention recognition model; The steps of encoding the visual perception event, the perceived target, and the state data, and fusing them with the vehicle sensor data to obtain sample data include: All data are segmented according to the preset time window width; Process each data segment according to its data type, including taking the average value within a time window for continuous data; For event-driven data, retrieve the latest state within the time window; Apply constraints to the processed data; The sample data is obtained by fusing the constrained data.

2. The method for establishing a driver intent recognition model according to claim 1, characterized in that, The image data is processed to generate different visual perception events, specifically including: The image data is processed to obtain the pixel count and frame rate supported by the image recognition algorithm. The processed image data is input into an image recognition algorithm to obtain the visual perception event.

3. The method for establishing a driver intent recognition model according to claim 1, characterized in that, The radar data is processed to generate target perception and status data, specifically including: Radar data is input into a radar identification algorithm to obtain the perceived target and its status data. The perceived targets include other vehicles in the vicinity, and the status includes the distance of other vehicles from the current vehicle.

4. The method for establishing a driver intent recognition model according to claim 1, characterized in that, The constraint processing includes bucketing, normalization, or standardization.

5. The method for establishing a driver intent recognition model according to claim 1, characterized in that, The steps of encoding the visual perception event, the perception target, and the state data specifically include: For the visual perception events obtained by the image recognition algorithm, a pre-established correspondence table of visual perception events is queried for encoding; For the multiple sensing targets and their states closest to the vehicle obtained by the radar identification algorithm, the pre-established correspondence table of sensing targets and states is queried and encoded.

6. A driver intent recognition model building device, characterized in that, include: The data acquisition module is used to acquire data inside and outside the vehicle in different scenarios. The data includes image data, radar data, and vehicle sensor data. The vehicle sensor data includes the vehicle ignition status, vehicle speed, window status, and the temperature or humidity inside and outside the vehicle. The data processing module is used to process the image data to generate different visual perception events and to process the radar data to generate perceived target and status data. The visual perception events include whether the driver is in place, whether the driver is fatigued, whether there is someone in the passenger seat, whether the driver is smoking, and whether the driver is drinking water. The data fusion module is used to encode the visual perception events, the perception targets, and the state data, and to fuse them with the vehicle sensor data to obtain sample data. The training data module is used to train a neural network incorporating an attention mechanism using the sample data to obtain a driver intention recognition model. The steps of encoding the visual perception event, the perceived target, and the state data, and fusing them with the vehicle sensor data to obtain sample data include: All data are segmented according to the preset time window width; Process each data segment according to its data type, including taking the average value within a time window for continuous data; For event-driven data, retrieve the latest state within the time window; Apply constraints to the processed data; The sample data is obtained by fusing the constrained data.

7. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the driver intent recognition model building method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by the computer's processor, cause the computer to perform the driver intent recognition model building method as claimed in any one of claims 1 to 5.

Images