Inertial navigation and machine vision assisted driving method, system, device and medium

Abstract

The present invention relates to an assisted driving method, system, device and medium of inertial navigation and machine vision, and belongs to the field of assisted driving technology. The method includes collecting historical assisted driving data, obtaining a historical vehicle posture data set through data annotation, and obtaining a historical vehicle posture standard set through data preprocessing, extracting an assisted driving visual feature set through a FlowNetSimple feature encoder, and extracting an assisted driving inertial feature set through a bidirectional LSTM network; obtaining an assisted driving fusion feature set through multi-head attention feature fusion based on the assisted driving visual feature set and the assisted driving inertial feature set, and obtaining an assisted driving posture model through Transformer training and testing; acquiring real-time assisted driving data and obtaining vehicle posture transformation parameters through the assisted driving posture model, performing assisted driving functions through a vehicle control system based on the vehicle posture transformation parameters, comprehensively perceiving the vehicle's surrounding environment and vehicle motion state, and improving the accuracy of the assisted driving system.

Description

Technical Field

[0001] The present invention belongs to the technical field of assisted driving, and in particular relates to an assisted driving method, system, device and medium of inertial navigation and machine vision. Background Art

[0002] In the field of assisted driving technology, inertial navigation systems and machine vision systems are two important technical means. The inertial navigation system can calculate the position and posture of the vehicle in real time by measuring the acceleration and angular velocity of the vehicle, combined with the initial position and direction information. The machine vision system obtains the image information of the vehicle's surrounding environment through camera sensors and performs image processing and analysis, thereby providing the assisted driving system with environmental information such as roads, vehicles, and pedestrians.

[0003] However, there are certain limitations when using either an inertial navigation system or a machine vision system alone. The inertial navigation system may affect the accuracy of positioning due to low precision and error accumulation, especially during long-term driving or in complex road conditions. On the other hand, the machine vision system is greatly affected by lighting and weather conditions, and its performance depends largely on manually designed feature extraction and matching algorithms. These limitations together lead to the problem of low accuracy of the assisted driving system. Summary of the invention

[0004] In order to solve the above-mentioned problems existing in the prior art, the present invention proposes an assisted driving method, system, device and medium of inertial navigation and machine vision, which respectively extracts assisted driving visual features and assisted driving inertial features through different feature extractors and performs feature fusion through multi-head attention, and obtains an assisted driving posture model through Transformer training and testing, thereby improving the robustness of the fused features, thereby more comprehensively perceiving the vehicle's surrounding environment and vehicle motion state, and improving the accuracy of the assisted driving system.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] An assisted driving method of inertial navigation and machine vision, comprising:

[0007] Collecting historical assisted driving data, the historical assisted driving data including historical inertial sensor data, historical vehicle surrounding environment image data, and historical vehicle posture transformation parameters, the historical inertial sensor data including historical vehicle acceleration data and historical vehicle angular velocity data;

[0008] A historical vehicle posture data set is obtained by data annotation based on the historical assisted driving data, and the data annotation includes:

[0009] Annotating the lane line position and obstacles around the vehicle by using an annotation tool according to the historical vehicle surrounding environment image data, annotating the vehicle acceleration and vehicle angular velocity by using an annotation tool according to the historical inertial sensor data, and annotating the vehicle steering angle and vehicle lateral displacement by using an annotation tool according to the historical vehicle posture transformation parameters;

[0010] Obtaining a historical vehicle posture standard set through data preprocessing according to the historical vehicle posture data set, wherein the historical vehicle posture standard set includes a historical vehicle surrounding environment image standard set and a historical inertial sensor data standard set;

[0011] Extracting an assisted driving visual feature set through a FlowNetSimple feature encoder based on the historical vehicle surrounding environment image standard set, and extracting an assisted driving inertial feature set through a bidirectional LSTM network based on the historical inertial sensor data standard set;

[0012] Obtaining an assisted driving fusion feature set by fusing the assisted driving visual feature set and the assisted driving inertial feature set through multi-head attention feature fusion;

[0013] According to the assisted driving fusion feature set, the assisted driving fusion feature training set and the assisted driving fusion feature test set are obtained by dividing the data set;

[0014] According to the assisted driving fusion feature training set, an assisted driving posture initial model is obtained by Transformer training; according to the assisted driving fusion feature test set, the assisted driving posture initial model is evaluated by evaluation indicators to obtain an assisted driving posture initial model evaluation result; according to the assisted driving posture initial model evaluation result, an assisted driving posture model is obtained by Transformer model iteration;

[0015] Real-time assisted driving data is acquired, vehicle posture transformation parameters are obtained through the assisted driving posture model according to the real-time assisted driving data, and assisted driving functions are performed through a vehicle control system according to the vehicle posture transformation parameters.

[0016] Preferably, the step of obtaining a historical vehicle posture standard set through data preprocessing based on the historical vehicle posture data set comprises:

[0017] Obtaining historical vehicle surrounding environment image standard data by preprocessing the vehicle surrounding environment image data according to the historical vehicle surrounding environment image data, wherein the vehicle surrounding environment image data preprocessing includes image enhancement, cropping, and data normalization;

[0018] The historical inertial sensor standard data is obtained by inertial sensor data preprocessing according to the historical inertial sensor data, wherein the inertial sensor data preprocessing includes filtering, denoising, and data normalization.

[0019] Preferably, the step of obtaining the assisted driving fusion feature set by fusing the assisted driving visual feature set and the assisted driving inertial feature set through multi-head attention features comprises:

[0020] According to the assisted driving visual feature set and the assisted driving inertial feature set, a visual inertial long sequence feature set is obtained by concatenating the feature sets;

[0021] Obtaining a visual inertia long sequence query vector, a visual inertia long sequence key vector, and a visual inertia long sequence value vector through a visual inertia long sequence linear transformation according to the visual inertia long sequence feature set;

[0022] The visual-inertial long sequence single-head attention weight is obtained by similarity mapping calculation based on the visual-inertial long sequence query vector and the visual-inertial long sequence key vector. The mathematical expression of the similarity mapping is:

[0023] ,

[0024] in, A is the single-head attention weight of the visual-inertial long sequence, Q is the visual-inertial long sequence query vector, K T is the transpose of the visual-inertial long sequence key vector, d k is the dimension of the visual-inertial long sequence key vector;

[0025] The visual-inertial long-sequence single-head attention weighted feature is obtained by point multiplication weighted calculation according to the visual-inertial long-sequence single-head attention weight and the visual-inertial long-sequence value vector. The mathematical expression of the point multiplication weighted is:

[0026] ,

[0027] in, H is the visual inertial long sequence single-head attention weighted feature, A is the single-head attention weight of the visual-inertial long sequence, Z is the visual-inertial long sequence value vector;

[0028] The auxiliary driving fusion feature set is obtained by multi-head splicing according to the visual-inertial long sequence single-head attention weighted features. The mathematical expression of the multi-head splicing is:

[0029] ,

[0030] in, F is the assisted driving fusion feature set, his the number of multi-head attention heads.

[0031] Preferably, the mathematical expression of the visual-inertial long sequence feature set is:

[0032] ,

[0033] ,

[0034] ,

[0035] in, V is the assisted driving visual feature set, v For the visual feature samples of assisted driving, J is the auxiliary driving inertial feature set, j For the inertial feature samples of assisted driving, S is the visual-inertial long-sequence feature set, s is the visual-inertial long-sequence feature sample, M is the total number of features in the assisted driving visual feature set, N is the total number of features in the auxiliary driving inertial feature set.

[0036] Preferably, the mathematical expression of the visual-inertial long sequence linear transformation is:

[0037] ,

[0038] in, Q is the visual-inertial long sequence query vector, K is the visual-inertial long sequence key vector, P is the visual-inertial long sequence value vector, S is the visual-inertial long sequence feature set, W q is the visual-inertial long sequence query vector weight, W k is the visual-inertial long sequence key vector weight, W p is the weight of the visual-inertial long sequence value vector.

[0039] An inertial navigation and machine vision assisted driving system, the system applies the above-mentioned inertial navigation and machine vision assisted driving method, including an inertial navigation module, a machine vision module, an assisted driving feature fusion module, and an assisted driving control module;

[0040] The inertial navigation module is used to capture inertial sensor data through an accelerometer and a gyroscope and perform inertial navigation data preprocessing and inertial navigation data feature extraction;

[0041] The machine vision module is used to obtain vehicle surrounding environment image data through a camera sensor and perform vehicle surrounding environment image data preprocessing and vehicle surrounding environment image data feature extraction;

[0042] The auxiliary driving feature fusion module is used to fuse the inertial navigation features and machine vision features output by the inertial navigation module and the machine vision module to obtain comprehensive vehicle status information;

[0043] The assisted driving control module is used to generate vehicle control instructions according to the comprehensive vehicle status information to realize the assisted driving function.

[0044] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned inertial navigation and machine vision assisted driving method when executing the computer program.

[0045] A storage medium containing computer executable instructions, which are used to execute the above-mentioned inertial navigation and machine vision assisted driving method when executed by a computer processor.

[0046] The beneficial effects of the present invention are:

[0047] (1) By performing image enhancement, cropping, and data normalization preprocessing on the vehicle surrounding environment image data, the vehicle surrounding environment image data is made easier to be recognized and processed by the model, thereby improving the efficiency of assisted driving visual feature extraction.

[0048] (2) By filtering, denoising, and normalizing the inertial sensor data, the quality of the inertial sensor data is improved, the errors caused by noise and interference are reduced, and the prediction accuracy of the assisted driving posture model is improved. Through data normalization, different types of data can be compared at the same scale, which is conducive to the training and prediction of the assisted driving posture model.

[0049] (3) The FlowNetSimple feature encoder is used to extract features from the image data of the vehicle's surrounding environment. It can automatically extract useful auxiliary driving visual features from the image, avoiding the complexity of manually designing feature extraction algorithms. At the same time, it enhances feature expression and can learn more complex and abstract feature representations.

[0050] (4) Through the multi-head attention mechanism, the assisted driving visual features and assisted driving inertial features are fused to obtain the assisted driving fusion feature set. The features from different sources are fused and weighted according to the importance of different features, which improves the robustness of the fused features, thereby more comprehensively perceiving the vehicle's surrounding environment and vehicle motion status, and improving the accuracy of the assisted driving system.

[0051] (5) Through real-time assisted driving data processing and assisted driving function execution, driving safety and comfort can be improved, while driving efficiency can be improved and the impact of human factors on driving can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0053] Figure 1 The figure is a flow chart of an inertial navigation and machine vision assisted driving method of the present invention. DETAILED DESCRIPTION

[0054] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0055] See also Figure 1 , an assisted driving method of inertial navigation and machine vision, comprising:

[0056] S1: Collecting historical assisted driving data, wherein the historical assisted driving data includes historical inertial sensor data, historical vehicle surrounding environment image data, and historical vehicle posture transformation parameters, wherein the historical inertial sensor data includes historical vehicle acceleration data and historical vehicle angular velocity data;

[0057] S2: obtaining a historical vehicle posture data set through data annotation according to the historical assisted driving data, and obtaining a historical vehicle posture standard set through data preprocessing according to the historical vehicle posture data set, wherein the historical vehicle posture standard set includes a historical vehicle surrounding environment image standard set and a historical inertial sensor data standard set;

[0058] S3: Obtaining an assisted driving posture model through Transformer training and testing according to the historical vehicle posture standard set;

[0059] S4: Acquire real-time assisted driving data, obtain vehicle posture transformation parameters through the assisted driving posture model according to the real-time assisted driving data, and perform assisted driving functions through the vehicle control system according to the vehicle posture transformation parameters, wherein the real-time assisted driving data includes real-time inertial sensor data and real-time vehicle surrounding environment image data, and the real-time inertial sensor data includes vehicle real-time acceleration data and vehicle real-time angular velocity data.

[0060] In step S2, obtaining a historical vehicle posture data set through data annotation according to the historical assisted driving data includes:

[0061] The lane line position and obstacles around the vehicle are annotated by an annotation tool based on the historical vehicle surrounding environment image data, the vehicle acceleration and vehicle angular velocity are annotated by an annotation tool based on the historical inertial sensor data, and the vehicle steering angle and vehicle lateral displacement are annotated by an annotation tool based on the historical vehicle posture transformation parameters.

[0062] In step S2, a historical vehicle posture standard set is obtained by data preprocessing according to the historical vehicle posture data set, including:

[0063] Obtaining historical vehicle surrounding environment image standard data by preprocessing the vehicle surrounding environment image data according to the historical vehicle surrounding environment image data, wherein the vehicle surrounding environment image data preprocessing includes image enhancement, cropping, and data normalization;

[0064] The historical inertial sensor standard data is obtained by inertial sensor data preprocessing according to the historical inertial sensor data, wherein the inertial sensor data preprocessing includes filtering, denoising, and data normalization.

[0065] The step S3 specifically comprises the following steps:

[0066] S301: extracting an assisted driving visual feature set through a FlowNetSimple feature encoder according to the historical vehicle surrounding environment image standard set, and extracting an assisted driving inertial feature set through a bidirectional LSTM network according to the historical inertial sensor data standard set;

[0067] S302: Obtain an assisted driving fusion feature set by fusing the assisted driving visual feature set and the assisted driving inertial feature set through multi-head attention features.

[0068] The step S302 specifically includes the following steps:

[0069] S302-1: Obtain a visual-inertial long-sequence feature set by concatenating the assisted driving visual feature set and the assisted driving inertial feature set. The mathematical expression of the visual-inertial long-sequence feature set is:

[0070] ,

[0071] ,

[0072] ,

[0073] in, V is the assisted driving visual feature set, v For the visual feature samples of assisted driving, J is the auxiliary driving inertial feature set, jFor the inertial feature samples of assisted driving, S is the visual-inertial long-sequence feature set, s is the visual-inertial long-sequence feature sample, M is the total number of features in the assisted driving visual feature set, N is the total number of features of the auxiliary driving inertial feature set;

[0074] S302-2: Obtain a visual inertial long sequence query vector, a visual inertial long sequence key vector, and a visual inertial long sequence value vector through a visual inertial long sequence linear transformation according to the visual inertial long sequence feature set. The mathematical expression of the visual inertial long sequence linear transformation is:

[0075] ,

[0076] in, Q is the visual-inertial long sequence query vector, K is the visual-inertial long sequence key vector, P is the visual-inertial long sequence value vector, S is the visual-inertial long sequence feature set, W q is the visual-inertial long sequence query vector weight, W k is the visual-inertial long sequence key vector weight, W p is the weight of the visual-inertial long sequence value vector;

[0077] S302-3: Calculate the visual-inertial long-sequence single-head attention weight through similarity mapping according to the visual-inertial long-sequence query vector and the visual-inertial long-sequence key vector. The mathematical expression of the similarity mapping is:

[0078] ,

[0079] in, A is the single-head attention weight of the visual-inertial long sequence, Q is the visual-inertial long sequence query vector, K T is the transpose of the visual-inertial long sequence key vector, d k is the dimension of the visual-inertial long sequence key vector;

[0080] S302-4: Obtain a visual-inertial long-sequence single-head attention weighted feature through a point multiplication weighted calculation based on the visual-inertial long-sequence single-head attention weight and the visual-inertial long-sequence value vector. The mathematical expression of the point multiplication weighted is:

[0081] ,

[0082] in, H is the visual inertial long sequence single-head attention weighted feature, A is the single-head attention weight of the visual-inertial long sequence, Z is the visual-inertial long sequence value vector;

[0083] S302-5: The auxiliary driving fusion feature set is obtained by multi-head splicing according to the visual-inertial long sequence single-head attention weighted features. The mathematical expression of the multi-head splicing is:

[0084] ,

[0085] in, F is the assisted driving fusion feature set, h is the number of multi-head attention heads.

[0086] S303: obtaining an assisted driving fusion feature training set and an assisted driving fusion feature test set by dividing the data set according to the assisted driving fusion feature set;

[0087] S304: obtaining an assisted driving posture initial model through Transformer training according to the assisted driving fusion feature training set, evaluating the assisted driving posture initial model through evaluation indicators according to the assisted driving fusion feature test set to obtain an assisted driving posture initial model evaluation result, and obtaining an assisted driving posture model through Transformer model iteration according to the assisted driving posture initial model evaluation result;

[0088] The evaluation result of the assisted driving posture initial model is obtained by evaluating the assisted driving fusion feature test set through the MSE indicator. The MSE indicator is the prediction error of the assisted driving posture initial model, and the mathematical expression is:

[0089] ,

[0090] Wherein, N is the total number of samples in the assisted driving fusion feature test set, yi is the true value of the i-th assisted driving fusion feature test sample, and ŷi is the predicted value of the i-th assisted driving fusion feature test sample;

[0091] In each Transformer model iteration, the Transformer model parameters are adjusted according to the evaluation results of the initial model of the assisted driving posture, and multiple epochs are iteratively trained until the Transformer model converges or reaches a preset number of iterations, thereby obtaining the assisted driving posture model.

[0092] An inertial navigation and machine vision assisted driving system includes an inertial navigation module, a machine vision module, an assisted driving feature fusion module, and an assisted driving control module;

[0093] The inertial navigation module is used to capture inertial sensor data through an accelerometer and a gyroscope and perform inertial navigation data preprocessing and inertial navigation data feature extraction;

[0094] The machine vision module is used to obtain vehicle surrounding environment image data through a camera sensor and perform vehicle surrounding environment image data preprocessing and vehicle surrounding environment image data feature extraction;

[0095] The auxiliary driving feature fusion module is used to fuse the inertial navigation features and machine vision features output by the inertial navigation module and the machine vision module to obtain comprehensive vehicle status information;

[0096] The assisted driving control module is used to generate vehicle control instructions according to the comprehensive vehicle status information to realize the assisted driving function.

[0097] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0098] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0099] The program code included on the computer readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or their combinations, and the programming language includes object-oriented programming languages-such as Java, Smalltalk, C++, and also includes conventional procedural programming languages-such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0100] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. An assisted driving method using inertial navigation and machine vision, characterized in that: include: Collecting historical assisted driving data, the historical assisted driving data including historical inertial sensor data, historical vehicle surrounding environment image data, and historical vehicle posture transformation parameters, the historical inertial sensor data including historical vehicle acceleration data and historical vehicle angular velocity data; Obtaining a historical vehicle posture data set through data annotation according to the historical assisted driving data, wherein the data annotation includes annotating the lane line position and obstacles around the vehicle through an annotation tool according to the historical vehicle surrounding environment image data, annotating the vehicle acceleration and vehicle angular velocity through an annotation tool according to the historical inertial sensor data, and annotating the vehicle steering angle and vehicle lateral displacement through an annotation tool according to the historical vehicle posture transformation parameters; Obtaining a historical vehicle posture standard set through data preprocessing according to the historical vehicle posture data set, wherein the historical vehicle posture standard set includes a historical vehicle surrounding environment image standard set and a historical inertial sensor data standard set; Extracting an assisted driving visual feature set through a FlowNetSimple feature encoder based on the historical vehicle surrounding environment image standard set, and extracting an assisted driving inertial feature set through a bidirectional LSTM network based on the historical inertial sensor data standard set; Obtaining an assisted driving fusion feature set by fusing the assisted driving visual feature set and the assisted driving inertial feature set through multi-head attention feature fusion; According to the assisted driving fusion feature set, the assisted driving fusion feature training set and the assisted driving fusion feature test set are obtained by dividing the data set; According to the assisted driving fusion feature training set, an assisted driving posture initial model is obtained by Transformer training; according to the assisted driving fusion feature test set, the assisted driving posture initial model is evaluated by evaluation indicators to obtain an assisted driving posture initial model evaluation result; according to the assisted driving posture initial model evaluation result, an assisted driving posture model is obtained by Transformer model iteration; Real-time assisted driving data is acquired, vehicle posture transformation parameters are obtained through the assisted driving posture model according to the real-time assisted driving data, and assisted driving functions are performed through a vehicle control system according to the vehicle posture transformation parameters.

2. The inertial navigation and machine vision assisted driving method according to claim 1, characterized in that: The historical vehicle posture standard set is obtained by data preprocessing according to the historical vehicle posture data set, including: Obtaining historical vehicle surrounding environment image standard data by preprocessing the vehicle surrounding environment image data according to the historical vehicle surrounding environment image data, wherein the vehicle surrounding environment image data preprocessing includes image enhancement, cropping, and data normalization; The historical inertial sensor standard data is obtained by inertial sensor data preprocessing according to the historical inertial sensor data, wherein the inertial sensor data preprocessing includes filtering, denoising, and data normalization.

3. The inertial navigation and machine vision assisted driving method according to claim 1, characterized in that: The assisted driving fusion feature set obtained by fusing the assisted driving visual feature set and the assisted driving inertial feature set through multi-head attention features includes: According to the assisted driving visual feature set and the assisted driving inertial feature set, a visual inertial long sequence feature set is obtained by concatenating the feature sets; Obtaining a visual inertia long sequence query vector, a visual inertia long sequence key vector, and a visual inertia long sequence value vector through a visual inertia long sequence linear transformation according to the visual inertia long sequence feature set; The visual-inertial long sequence single-head attention weight is obtained by similarity mapping calculation based on the visual-inertial long sequence query vector and the visual-inertial long sequence key vector. The mathematical expression of the similarity mapping is: ， in, A is the single-head attention weight of the visual-inertial long sequence, Q is the visual-inertial long sequence query vector, K T is the transpose of the visual-inertial long sequence key vector, d k is the dimension of the visual-inertial long sequence key vector; The visual-inertial long-sequence single-head attention weighted feature is obtained by point multiplication weighted calculation according to the visual-inertial long-sequence single-head attention weight and the visual-inertial long-sequence value vector. The mathematical expression of the point multiplication weighted is: ， in, H is the visual inertial long sequence single-head attention weighted feature, A is the single-head attention weight of the visual-inertial long sequence, Z is the visual-inertial long sequence value vector; The auxiliary driving fusion feature set is obtained by multi-head splicing according to the visual-inertial long sequence single-head attention weighted features. The mathematical expression of the multi-head splicing is: ， in, F is the assisted driving fusion feature set, h is the number of multi-head attention heads.

4. The inertial navigation and machine vision assisted driving method according to claim 3, characterized in that: The mathematical expression of the visual-inertial long sequence feature set is: ， ， ， in, V is the assisted driving visual feature set, v For the visual feature samples of assisted driving, J is the auxiliary driving inertial feature set, j For the inertial feature samples of assisted driving, S is the visual-inertial long-sequence feature set, s is the visual-inertial long-sequence feature sample, M is the total number of features in the assisted driving visual feature set, N is the total number of features in the auxiliary driving inertial feature set.

5. The inertial navigation and machine vision assisted driving method according to claim 3, characterized in that: The mathematical expression of the visual-inertial long sequence linear transformation is: ， in, Q is the visual-inertial long sequence query vector, K is the visual-inertial long sequence key vector, P is the visual-inertial long sequence value vector, S is the visual-inertial long sequence feature set, W q is the visual-inertial long sequence query vector weight, W k is the visual-inertial long sequence key vector weight, W p is the weight of the visual-inertial long sequence value vector.

6. An inertial navigation and machine vision assisted driving system, characterized in that: The system is applied to the assisted driving method of inertial navigation and machine vision as described in any one of claims 1 to 5, comprising an inertial navigation module, a machine vision module, an assisted driving feature fusion module, and an assisted driving control module; The inertial navigation module is used to capture inertial sensor data through an accelerometer and a gyroscope and perform inertial navigation data preprocessing and inertial navigation data feature extraction; The machine vision module is used to obtain vehicle surrounding environment image data through a camera sensor and perform vehicle surrounding environment image data preprocessing and vehicle surrounding environment image data feature extraction; The auxiliary driving feature fusion module is used to fuse the inertial navigation features and machine vision features output by the inertial navigation module and the machine vision module to obtain comprehensive vehicle status information; The assisted driving control module is used to generate vehicle control instructions according to the comprehensive vehicle status information to realize the assisted driving function.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the assisted driving method of inertial navigation and machine vision as described in any one of claims 1-5 is implemented.

8. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the inertial navigation and machine vision assisted driving method as described in any one of claims 1 to 5 when executed by a computer processor.

Images