Pedestrian head injury value prediction method and system based on multi-modal data interaction

Abstract

The invention discloses a pedestrian head injury value prediction method and system based on multi-modal data interaction, and relates to the technical field of automobile driving safety. The method comprises the following steps: acquiring experimental data in a collision process, and screening head injury standard related data to form a data set; the expert network comprises a gating network and a plurality of sub-networks, the gating network is used for guiding and classifying the data sets, the different sub-networks are used for processing different types of data, the gating network allocates weights to the different types of data, and the weights are distributed to the different types of data; and finally, integrating sub-network output to obtain a pedestrian head injury predicted value. And predicting to-be-predicted data by using the pedestrian head injury prediction model to obtain a pedestrian head injury value. According to the invention, for influence data of multi-modal HIC values, different features are extracted by using an expert network, and a plurality of data evaluation indexes are integrated to finally generate a pedestrian head injury value prediction result.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile driving safety, and in particular to a method and system for predicting pedestrian head injury values ​​based on multimodal data interaction. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The car hood is one of the important areas that pedestrians may come into contact with in a vehicle collision accident, so design optimization is crucial. Traditionally, automakers use the head injury criterion (HIC) to measure the risk of pedestrian head injuries. The traditional process of calculating the head injury criterion (HIC) is usually time-consuming and labor-intensive. First, complex mathematical calculations are required to determine the extent of head injury during a collision. This usually involves precise measurement and analysis of factors such as collision force and speed changes. Then, based on this data, complex formulas and algorithms are required to calculate the HIC value. These calculations can take a lot of time and resources, and require expertise to ensure accuracy. Therefore, the traditional calculation of HIC often requires a lot of manpower, time and money resources, and may have errors.

[0004] At present, many studies have trained neural network models through car hood data. This method can accurately predict the impact of different hood designs on pedestrian head injuries, provide a scientific basis for designing safer vehicles, improve traffic safety levels, and comply with relevant safety regulations and standards. However, the influencing data of HIC values ​​are diverse, and the HIC value results obtained by training only with car hood data have a large error with the actual value, which cannot be used as an effective reference for optimizing car parameters. However, the relationship between the influencing data of HIC values ​​is complex and in many forms, so there is a lack of methods in the prior art that can mine information and efficiently analyze the influencing data of multi-modal HIC values ​​to achieve accurate prediction of HIC values. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method and system for predicting pedestrian head injury values ​​based on multimodal data interaction. In the face of multimodal HIC value influencing data, different features are extracted separately by using an expert network, and a variety of data evaluation indicators are integrated to finally generate a pedestrian head injury value prediction result.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0007] The first aspect of the present invention provides a method for predicting pedestrian head injury value based on multimodal data interaction, comprising the following steps:

[0008] Acquire experimental data during the collision process and filter data related to head injury standards to form a data set;

[0009] The expert network is trained using the data set to obtain a pedestrian head injury prediction model. The expert network includes a gating network and several sub-networks. The gating network is used to guide the classification of the data set, which is divided into spatial feature data, mechanical structure data, and dynamic data. Different sub-networks are used to process different types of data respectively. The gating network assigns weights to different types of data. Finally, the sub-network outputs are integrated to obtain the pedestrian head injury prediction value.

[0010] The pedestrian head injury prediction model is used to predict the data to be predicted to obtain the pedestrian head injury value.

[0011] Furthermore, the data related to the head injury standard include car hood data, pedestrian data during the collision process, interior space layout data, and car driving data during the collision process.

[0012] Furthermore, the subnetwork includes a first model composed of a CNN model, a second model composed of a DNN, and a third model composed of MixConv and LSTM. The first model is used to process mechanical structure data, the second model is used to process spatial feature data, and the third model is used to process dynamic data.

[0013] Furthermore, the specific steps of processing the mechanical structure data using the first model are:

[0014] The mechanical structure data is normalized and converted into RGB three-dimensional data;

[0015] Perform mosaic processing on the RGB three-dimensional data to obtain a mosaic image;

[0016] Features in the mosaic image are extracted to obtain a first output.

[0017] Furthermore, the specific steps of mosaic processing of RGB three-dimensional data are as follows:

[0018] Create a grid of fixed size to form a matrix;

[0019] Traverse the RGB three-dimensional data and store the corresponding color data in the matrix;

[0020] Use library functions to convert matrices into images;

[0021] Process the non-collision point area data and fill it with background color;

[0022] Output mosaic image.

[0023] Furthermore, the specific steps of processing the spatial feature data using the second model are as follows:

[0024] Preprocess the spatial feature data;

[0025] Using an automatic encoder to extract spatial feature information from spatial feature data;

[0026] The hidden layer is used to learn the relationship between spatial feature information to obtain the second output.

[0027] Furthermore, the specific steps of processing the kinetic data using the third model are:

[0028] Using the MixConv layer, convolution kernels of different sizes are used to extract multi-scale features in the kinetic data;

[0029] Use depth-wise separable convolution to further extract deep features for multi-scale features;

[0030] The LSTM layer is used to capture the temporal dependency in the deep features, and the third output is obtained after the fully connected layer.

[0031] A second aspect of the present invention provides a pedestrian head injury value prediction system based on multimodal data interaction, comprising:

[0032] A data acquisition module is configured to acquire experimental data during the collision process and filter data related to head injury standards to form a data set;

[0033] The model building module is configured to train an expert network using a data set to obtain a pedestrian head injury prediction model, wherein the expert network includes a gated network and several sub-networks, and the gated network is used to guide the classification of the data set into spatial feature data, mechanical structure data, and dynamic data. Different sub-networks are used to process different categories of data respectively, and the gated network assigns weights to different categories of data. Finally, the sub-network outputs are integrated to obtain the pedestrian head injury prediction value;

[0034] The injury prediction module is configured to use the pedestrian head injury prediction model to predict the data to be predicted to obtain the pedestrian head injury value.

[0035] The third aspect of the present invention provides a medium having a program stored thereon, which, when executed by a processor, implements the steps in the method for predicting pedestrian head injury value based on multimodal data interaction as described in the first aspect of the present invention.

[0036] The fourth aspect of the present invention provides a device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for predicting pedestrian head injury value based on multimodal data interaction as described in the first aspect of the present invention are implemented.

[0037] One or more of the above technical solutions have the following beneficial effects:

[0038] The present invention discloses a pedestrian head injury value prediction method and system based on multimodal data interaction. In the face of multimodal HIC value influencing data, an expert network is used to extract different features respectively, and a variety of data evaluation indicators are integrated to finally generate a pedestrian head injury value prediction result. This method takes into account the diversity of influencing data in collision accidents, and the prediction results can more reasonably reflect the real damage in the collision process.

[0039] The car hood is one of the important areas that pedestrians may come into contact with in a vehicle collision accident. The present invention uses a CNN network to extract features from the car hood data to prevent the phenomenon of focus deviation caused by mixed analysis with other data. The analysis process converts complex numerical data into a mosaic image format, which can intuitively display the collision point features of the hood, facilitating image recognition and feature learning by the convolutional neural network.

[0040] The present invention utilizes a data set generated by simulation data to efficiently train a model, and the simulation model can simulate various extreme or complex collision scenarios that may be dangerous or infeasible in actual testing, thereby expanding the data diversity of model training.

[0041] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0043] Figure 1 This is a flow chart of a method for predicting pedestrian head injury value based on multimodal data interaction in Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0044] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0045] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "include" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or their combinations;

[0046] Embodiment 1:

[0047] Embodiment 1 of the present invention provides a method for predicting pedestrian head injury value based on multimodal data interaction, such as Figure 1 As shown, the following steps are included:

[0048] Step 1: Obtain experimental data during the collision process and filter data related to head injury standards to form a data set.

[0049] Head injury standard-related data include car hood data, pedestrian data during collision, in-car space layout data, and car driving data during collision. Car hood data, i.e., car engine hood data, includes the outer panel height, inner panel height, inner and outer panel spacing, outer panel to lower hard point distance, and inner and outer panel thickness corresponding to the hood collision point. Pedestrian data during collision includes pedestrian posture, pedestrian speed, and pedestrian facing direction. In-car space layout data includes seat spacing, seat position, and seat belt fixing point position. Car driving data during collision includes vehicle speed, collision angle, and ground adhesion coefficient.

[0050] In addition, it should be noted that the experimental data in the collision process in this embodiment is the data obtained by the simulation model. Specifically, by building a variety of different simulation models and conducting simulated collision tests, the required experimental data is obtained. In order to ensure the validity of the experimental data, the external CAS stage data can be used to implement early model verification prediction.

[0051] Step 2: Use the dataset to train the expert network and obtain a pedestrian head injury prediction model.

[0052] In this embodiment, the expert network includes a gating network and several sub-networks. The gating network is used to guide the classification of the data set and divide it into spatial feature data, mechanical structure data and dynamic data. Different sub-networks are used to process different categories of data respectively. The gating network assigns weights to different categories of data. Finally, the sub-network outputs are integrated to obtain the pedestrian head injury prediction value.

[0053] Among them, the spatial feature data is data that can reflect the spatial relationship, including seat spacing, seat belt fixing point position, seat position, etc. In this embodiment, the mechanical structure data is only the car hood data, which is used to reflect the particularity of the car hood in pedestrian head injuries in collision accidents. The dynamic data is the movement data of the collision parties that changes with time, including pedestrian posture, pedestrian speed, pedestrian facing direction, car speed, collision angle, ground adhesion coefficient, etc.

[0054] The sub-networks include a first model consisting of a CNN model, a second model consisting of a DNN, and a third model consisting of MixConv and LSTM.

[0055] The first model is used to process the mechanical structure data. By extracting the features of the car hood image, the influence of the car hood related data on the pedestrian head injury value is obtained. The output is converted into a unified gating score of the gating network for joint evaluation with other data.

[0056] The second model is used to process the spatial feature data, and the third model is used to process the dynamic data. In this embodiment, DNN is used to extract HIC-related spatial features, and the MixConv model with LSTM is used to extract HIC-related temporal features. By analyzing both temporal and spatial aspects simultaneously, a more accurate pedestrian head injury prediction value can be obtained.

[0057] Step 2.1: Use the first model to process the mechanical structure data to obtain a first output.

[0058] Step 2.1.1: Compile the automobile hood data corresponding to the hood collision point, such as the outer panel height, inner panel height, inner and outer panel spacing, distance from the outer panel to the lower hard point, and inner and outer panel thickness, into the specified table.

[0059] Step 2.1.2: Normalize the mechanical structure data and convert it into RGB three-dimensional data.

[0060] In a specific implementation, the table data is read by programming, and the data is normalized and converted into RGB three-dimensional data to generate a corresponding color mosaic image.

[0061] Writing a script to convert the collision point feature information on the hair cover into a mosaic image format in this embodiment involves mapping the collision point position information onto the image and using mosaic technology to blur these positions to protect sensitive information and facilitate subsequent processing.

[0062] Step 2.1.3: Perform mosaic processing on the RGB three-dimensional data to obtain a mosaic image.

[0063] Step 2.1.3.1: Create a grid of fixed size to form a matrix.

[0064] Step 2.1.3.2: Traverse the RGB three-dimensional data in the table and store the corresponding color data in the matrix.

[0065] Step 2.1.3.3: Convert the matrix into an image using library functions.

[0066] Step 2.1.3.4: Process the non-collision point area data and fill it with the background color.

[0067] Step 2.1.3.5: Output mosaic image.

[0068] Step 2.1.4: Extract features from the mosaic image and obtain the first output after passing through the output layer.

[0069] Step 2.2: Use the second model to process the spatial feature data.

[0070] Step 2.2.1: Preprocess the spatial feature data.

[0071] The spatial feature data such as seat spacing, seat belt fixing point position, seat position, etc. are cleaned to remove outliers, and then normalized and scaled to the [0,1] interval so that features of different dimensions can be compared at the same scale.

[0072] Step 2.2.2: Use an autoencoder to extract spatial feature information from the spatial feature data.

[0073] Step 2.2.3: Use the hidden layer to learn the relationship between spatial feature information and obtain the second output.

[0074] This embodiment uses multiple hidden layers, each of which contains multiple neurons, to extract complex relationships and patterns between spatial features. In addition, a ReLU activation function is used in the hidden layer to introduce nonlinearity and improve the expressiveness of the model. Finally, the second output is obtained through full connection.

[0075] Step 2.3: Process the kinetic data using the third model to obtain a third output.

[0076] Step 2.3.1: Use the MixConv layer to extract multi-scale features from the kinetic data using convolutional kernels of different sizes.

[0077] The MixConv layer is the first layer of the model, and uses convolution kernels of different sizes (such as 3x3, 5x5) to extract multi-scale features. These features contain the spatial information of the input data at different scales. Feature maps are obtained, which encode the multi-scale spatial features of each dynamic data.

[0078] Step 2.3.2: Use depth-wise separable convolution to further extract deep features for multi-scale features.

[0079] Depthwise Separable Convolution is a convolution operation used in deep learning to improve computational efficiency and reduce the number of parameters. It decomposes the traditional convolution into two smaller operations: Depthwise Convolution and Pointwise Convolution.

[0080] In the depth convolution step, a convolution kernel is applied independently to each input channel. This means that if there are M input channels, then M convolution kernels will be used, and each convolution kernel is responsible for the convolution operation of only one channel. The advantage of this is that the number of parameters and the amount of calculation can be significantly reduced. For example, for an input feature map with N channels, the depth convolution will use N convolution kernels, and each convolution kernel will operate on only one channel instead of all channels together.

[0081] After the depthwise convolution, the output of all channels is convolved with a 1x1 convolution kernel, also known as pointwise convolution or point convolution. The purpose of this step is to mix and combine the features of different channels to generate a new feature map. Pointwise convolution uses a 1x1 convolution kernel to linearly combine the output of the depthwise convolution in the channel dimension while keeping the size of the feature map unchanged, thereby enhancing the expressive power of the model.

[0082] In this embodiment, the depthwise separable convolution layer follows the MixConv layer and receives the output feature map of the MixConv layer. The depthwise convolution part convolves each channel independently to further extract spatial features; the pointwise convolution part mixes the features of different channels to generate a new feature map. The output of this layer is still a feature map, but compared with the output of the MixConv layer, they have stronger correlation between channels and less computation.

[0083] Step 2.3.3: Use the LSTM layer to capture the temporal dependency in the deep features and get the third output after passing through the fully connected layer.

[0084] The LSTM layer is used to process time series data and capture the changing relationship of dynamic data over time. The input received by the LSTM layer is the feature map processed by the MixConv layer and the depth-separable convolution layer. The LSTM layer outputs a series of hidden states that capture the temporal dependency of the time series data. In this embodiment, the LSTM layer receives the MixConv layer feature map, processes the feature map of each time step, and outputs a hidden state containing spatiotemporal features.

[0085] Step 2.3.4: Finally, the output after the LSTM layer passes through the fully connected layer to obtain the third output.

[0086] Step 2.4: The outputs of the first model, the second model, and the third model are converted into corresponding gating scores, and the gating network assigns weights to each output according to the importance. Finally, the first output, the second output, and the third output are added together to obtain the prediction result of the pedestrian head injury value.

[0087] Step 3: Use the pedestrian head injury prediction model to predict the data to be predicted and obtain the pedestrian head injury value.

[0088] Embodiment 2:

[0089] Embodiment 2 of the present invention provides a pedestrian head injury value prediction system based on multimodal data interaction, including:

[0090] A data acquisition module is configured to acquire experimental data during the collision process and filter data related to head injury standards to form a data set;

[0091] The model building module is configured to train an expert network using a data set to obtain a pedestrian head injury prediction model, wherein the expert network includes a gated network and several sub-networks, and the gated network is used to guide the classification of the data set into spatial feature data, mechanical structure data, and dynamic data. Different sub-networks are used to process different categories of data respectively, and the gated network assigns weights to different categories of data. Finally, the sub-network outputs are integrated to obtain the pedestrian head injury prediction value;

[0092] The injury prediction module is configured to use the pedestrian head injury prediction model to predict the data to be predicted to obtain the pedestrian head injury value.

[0093] Embodiment three:

[0094] Embodiment 3 of the present invention provides a medium on which a program is stored. When the program is executed by a processor, the steps in the method for predicting pedestrian head injury value based on multimodal data interaction as described in Embodiment 1 of the present invention are implemented.

[0095] Embodiment 4:

[0096] Embodiment 4 of the present invention provides a device, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for predicting pedestrian head injury values ​​based on multimodal data interaction as described in Embodiment 1 of the present invention are implemented.

[0097] The steps involved in the above embodiments 2, 3 and 4 correspond to the method embodiment 1. For the specific implementation methods, please refer to the relevant description part of embodiment 1.

[0098] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0099] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A pedestrian head injury value prediction method based on multimodal data interaction, characterized in that: The following steps are involved: Acquire experimental data during the collision process and filter data related to head injury standards to form a data set; The expert network is trained using the data set to obtain a pedestrian head injury prediction model. The expert network includes a gating network and several sub-networks. The gating network is used to guide the classification of the data set, which is divided into spatial feature data, mechanical structure data, and dynamic data. Different sub-networks are used to process different types of data respectively. The gating network assigns weights to different types of data. Finally, the sub-network outputs are integrated to obtain the pedestrian head injury prediction value. The pedestrian head injury prediction model is used to predict the data to be predicted to obtain the pedestrian head injury value.

2. The method for predicting pedestrian head injury value based on multimodal data interaction according to claim 1, characterized in that: The data related to the head injury standard include vehicle hood data, pedestrian data during the collision process, interior space layout data, and vehicle driving data during the collision process.

3. The method for predicting pedestrian head injury value based on multimodal data interaction according to claim 1, characterized in that: The subnetwork includes a first model composed of a CNN model, a second model composed of a DNN, and a third model composed of MixConv and LSTM. The first model is used to process mechanical structure data, the second model is used to process spatial feature data, and the third model is used to process dynamic data.

4. The method for predicting pedestrian head injury value based on multimodal data interaction as claimed in claim 3, characterized in that: The specific steps of processing the mechanical structure data using the first model are: The mechanical structure data is normalized and converted into RGB three-dimensional data; Perform mosaic processing on the RGB three-dimensional data to obtain a mosaic image; Features in the mosaic image are extracted to obtain a first output.

5. The method for predicting pedestrian head injury value based on multimodal data interaction as claimed in claim 4, characterized in that: The specific steps for mosaic processing of RGB three-dimensional data are as follows: Create a grid of fixed size to form a matrix; Traverse the RGB three-dimensional data and store the corresponding color data in the matrix; Use library functions to convert matrices into images; Process the non-collision point area data and fill it with background color; Output mosaic image.

6. The method for predicting pedestrian head injury value based on multimodal data interaction as claimed in claim 3, characterized in that: The specific steps of using the second model to process the spatial feature data are: Preprocess the spatial feature data; Using an automatic encoder to extract spatial feature information from spatial feature data; The hidden layer is used to learn the relationship between spatial feature information to obtain the second output.

7. The method for predicting pedestrian head injury value based on multimodal data interaction as claimed in claim 3, characterized in that: The specific steps of processing the kinetic data using the third model are: Using the MixConv layer, convolution kernels of different sizes are used to extract multi-scale features in the kinetic data; Use depth-wise separable convolution to further extract deep features for multi-scale features; The LSTM layer is used to capture the temporal dependency in the deep features, and the third output is obtained after the fully connected layer.

8. A pedestrian head injury value prediction system based on multimodal data interaction, characterized in that: include: A data acquisition module is configured to acquire experimental data during the collision process and filter data related to head injury standards to form a data set; The model building module is configured to train an expert network using a data set to obtain a pedestrian head injury prediction model, wherein the expert network includes a gated network and several sub-networks, and the gated network is used to guide the classification of the data set into spatial feature data, mechanical structure data, and dynamic data. Different sub-networks are used to process different categories of data respectively, and the gated network assigns weights to different categories of data. Finally, the sub-network outputs are integrated to obtain the pedestrian head injury prediction value; The injury prediction module is configured to use the pedestrian head injury prediction model to predict the data to be predicted to obtain the pedestrian head injury value.

9. A computer-readable storage medium, characterized in that: A plurality of instructions are stored therein, and the instructions are suitable for being loaded by a processor of a terminal device and executing the pedestrian head injury value prediction method based on multimodal data interaction described in any one of claims 1-7.

10. A terminal device, characterized in that: It includes a processor and a computer-readable storage medium, the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, and the instructions are suitable for being loaded by the processor and executing the pedestrian head injury value prediction method based on multimodal data interaction described in any one of claims 1-7.

Images