Multi-modal online car-hailing comfort evaluation network model and construction method thereof

Abstract

The invention provides a multi-modal online car-hailing comfort evaluation network model optimized by a genetic algorithm and a construction method of the multi-modal online car-hailing comfort evaluation network model. Constructing an end-to-end vision-odor multi-modal comfort evaluation network model, wherein the network model is composed of tidiness feature extraction, odor pseudo image feature extraction and multi-modal information evaluation; a genetic algorithm is combined with a back propagation learning method, and key parameters of the multi-modal evaluation neural network model are optimized; and a trained end-to-end multi-modal evaluation network model is used to carry out multi-dimensional quantitative evaluation on the comfort level of the in-vehicle environment. According to the method, odor pseudo image-tidiness image features can be effectively fused; compared with the method for evaluating the comfort level of the environment in the vehicle from the perspective of peculiar smell or cleanliness, the method has the advantages that the accuracy of the evaluation result after the cleanliness-peculiar smell information is effectively fused is obviously improved, and the evaluation result has higher robustness and reference value.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-sensor data fusion, and in particular to a multi-modal online car-hailing comfort evaluation network model and a construction method thereof. Background Art

[0002] Nowadays, under the concept of "low-carbon travel, mass transportation", online car-hailing has become the main means of transportation for people's daily travel. However, with the development of the times and the progress of society, people pay more attention to the experience and comfort of travel. The "comfort" of the interior environment of online car-hailing will directly affect the passengers' riding experience. The main influencing factors include the cleanliness of the passenger seat and the air quality in the car. Many passengers have high requirements for the cleanliness of the passenger seat, and cannot tolerate footprints, stains on the passenger seat mat, and debris on the foot mat. On the other hand, for passengers who are sensitive to odors, the air quality in the taxi is more important, and they cannot tolerate odors such as smoke, alcohol, and natural gas in the car. At present, the existing online car-hailing software fails to provide passengers with relevant judgment basis for car selection, forming the phenomenon of "car selection passengers, blind box experience".

[0003] The existing online car-hailing service software only provides the function of following up the passengers after getting off the car, that is, the passengers can evaluate the riding experience after getting off the car. The service software lacks the car selection reference service function based on the real-time monitoring data of the in-car environment and the comfort evaluation information. The environmental monitoring content in existing taxis and online car-hailing cars is mostly seen in the detection of in-car air quality or the identification of left-behind items. For example, the concentration detection of toxic gases such as carbon monoxide and nitrogen dioxide. Or the situation of passengers' left-behind items such as mobile phones and backpacks. The common in-car environment detection methods are as follows: With the STM32 microcontroller as the core, the low-power metal oxide gas sensor is used to realize the real-time monitoring of air quality. With a near-infrared camera as the monitoring device, a series of algorithms such as DHash, color histogram similarity, infrared, etc. are used to realize the detection of the cleanliness of the interior of the shared car and the items left by passengers.

[0004] There are few existing studies on in-vehicle environmental detection and comfort evaluation, and there are the following problems: ① The observation data angle is single and lacks the integration of multiple information. ② The multi-angle monitoring data is not effectively utilized, and there is a lack of environmental comfort evaluation methods based on multimodal data. Summary of the invention

[0005] The purpose of the present invention is to solve the existing technical problems. The purpose of this application is to provide a method for constructing a multimodal online car-hailing comfort evaluation network model optimized by a genetic algorithm. Its advanced features are: real-time monitoring of the odor, air quality and cleanliness in the car; converting the original sensor concentration detection data into a pseudo image, which is consistent with the on-board image feature information in terms of feature dimension and expression form, and effectively realizing the alignment of multimodal (on-board image, odor pseudo image) feature information; constructing an end-to-end multimodal online car-hailing comfort evaluation network, using the cleanliness feature extraction branch network and the odor pseudo image feature extraction branch network to obtain their respective features, using the multimodal information evaluation branch network to fuse features, and using a genetic algorithm to dynamically find the optimal representation dimension of the cleanliness image and the optimal fusion dimension of cleanliness-odor, thereby reducing the significant differences in the feature vectors of different modalities of cleanliness-odor, and then obtaining the best joint features and performing a five-star evaluation of the in-car environment.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for constructing a multimodal online car-hailing comfort evaluation network model optimized by a genetic algorithm comprises the following steps:

[0008] S1: Information collection: collect visual observation image data of passenger seats and odor concentration detection data in the vehicle;

[0009] S2: Construct an end-to-end visual-smell multimodal comfort evaluation network model:

[0010] The end-to-end visual-smell multimodal comfort evaluation network model consists of the following three parts: neatness feature extraction, odor pseudo-image feature extraction, and multimodal information evaluation;

[0011] S3: Optimize the key parameters of the multimodal evaluation neural network model built in S2 by using genetic algorithm combined with back propagation learning method:

[0012] S4: Use the trained “end-to-end multimodal evaluation network model” to conduct a multi-dimensional (cleanliness, air quality) quantitative evaluation of the in-vehicle environmental comfort.

[0013] Preferably, S3 comprises the following steps:

[0014] 3-1. Initialize the population, the population size is population←100:

[0015] Each individual contains four genes: neatness image feature dimension (d1), neatness-odor fusion feature dimension (d2), random discard probability (P d ) and the learning rate of the model (L r ),

[0016] The value range for each gene is as follows:

[0017]

[0018] 3-2. Design a fitness function, which uses the five-star classification accuracy of the in-car comfort as the size of the individual fitness, with maximizing the fitness function as the goal. The fitness function is expressed as:

[0019]

[0020] 3-3. Select, crossover, mutation, and set iteration and termination conditions:

[0021] Using tournament selection, individuals with better performance are selected as parents based on their fitness values;

[0022] Crossover, using the mixed crossover method, the genes of two individuals are mixed at a ratio of 0.5, and the parent generation is crossovered to generate new offspring individuals.

[0023] Mutation: Use Gaussian mutation with a mean of 0, a standard deviation of 0.2, and an independent mutation probability of 0.2 to perform mutation operations on the newly generated offspring individuals to prevent them from falling into the local optimum;

[0024] And terminate after iteration to select the final parameters.

[0025] 3-4. Based on the training data set, the cross loss function is used to measure the difference between the category probability distribution output by the model and the true label, and the network parameters of the "end-to-end visual-odor multimodal comfort evaluation network model" are obtained.

[0026] Preferably, the termination condition in 3-3 is that the algorithm terminates when the fitness of the optimal individual is no longer significantly improved or the number of iterations reaches a preset number. The final parameters obtained in 3-3 are expressed as:

[0027] x*=argmaxx i∈ populationF(xi).

[0028] Preferably, in S1, a vehicle-mounted camera is used to obtain visual observation image data of the passenger seat position, and an MQ series gas sensor and a single-chip microcomputer are used to design an in-vehicle odor detection device to obtain in-vehicle odor concentration detection data.

[0029] Preferably, the MQ series gas sensors are MQ-2 smoke sensor, MQ-3 alcohol sensor, MQ-7 carbon monoxide sensor and MQ-9 combustible gas sensor, and four odor sensors are used to obtain detection data of four common in-vehicle odor concentrations.

[0030] Preferably, the single chip microcomputer is an STM32 single chip, which is used for controlling the box to read the data of the MQ odor sensor.

[0031] Preferably, the specific steps of S2 are as follows:

[0032] 2-1. Construct the sub-network for extracting the cleanliness feature of the vehicle interior:

[0033] The vehicle interior cleanliness feature extraction subnetwork consists of two modules: a VGG-19 network feature extraction part and a global average pooling layer;

[0034] 2-2. Perform linear mapping on the concentration value obtained by the odor sensor and convert it into a pseudo image:

[0035] 2-3. Constructing a cleanliness-odor multimodal information evaluation subnetwork:

[0036] The subnetwork input is the 512-dimensional features output by the in-car cleanliness feature extraction module and the 64-dimensional features output by the in-car odor feature extraction module. The concatenate layer is used to realize the splicing and fusion of feature vectors of different modalities. The fused high-level features are further extracted through two fully connected layers, and the number of neurons in the two fully connected layers is 64 and 5 respectively. Finally, the softmax layer is used to realize the five-star comprehensive evaluation of the comfort of the in-car environment.

[0037] The present application also provides a multimodal online car-hailing comfort evaluation network model optimized by a genetic algorithm, which is constructed using the construction method described above.

[0038] Preferably, the network model includes: a vehicle-mounted camera, an MQ series odor sensor, a vehicle-mounted computing unit and a display terminal.

[0039] Compared with the existing technology, the network model constructed in this application is aimed at vision-odor multi-sensor fusion. The end-to-end multimodal convolutional neural network can complement the information advantages of vision and smell from multiple angles, extract effective comfort representation information, and more objectively and comprehensively realize multi-dimensional, real-time monitoring and evaluation of the comfort of the in-vehicle environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A flowchart of the image-air quality multimodal online car-hailing comfort evaluation proposed by the present invention;

[0041] Figure 2 Schematic diagram of the in-vehicle installation position of the vision-odor multi-sensor proposed by the present invention; in the figure, (a) is a schematic diagram of the sensor installation design; (b) is a schematic diagram of the in-vehicle environment installation;

[0042] Figure 3This is an example of the wiring diagram of the STM32 reading odor sensor MQ proposed by the present invention;

[0043] Figure 4 A schematic diagram of the structure of "an end-to-end multimodal online car-hailing comfort evaluation network optimized by a genetic algorithm" proposed in the present invention;

[0044] Figure 5 A schematic diagram of the odor pseudo image generation process proposed by the present invention;

[0045] Figure 6 It is a flow chart of the genetic optimization algorithm proposed by the present invention;

[0046] Figure 7 It is the accuracy iteration curve of the comfort evaluation network model based on genetic algorithm proposed by the present invention;

[0047] Figure 8 A schematic diagram showing the comparison of the accuracy of the in-car environment evaluation of the "multimodal online car-hailing comfort evaluation network" structure invented in this article and the single-modal evaluation network model based on odor pseudo-images and the single-modal evaluation network model based on vehicle-mounted images;

[0048] Fig. 9 This is a measured example 1 of a method for constructing a multi-modal online car-hailing comfort evaluation network model optimized by a genetic algorithm of the present invention;

[0049] Fig.10 This is a measured example 2 of a method for constructing a multi-modal online car-hailing comfort evaluation network model optimized by a genetic algorithm of the present invention;

[0050] Fig.11 This is a measured example 3 of a method for constructing a multi-modal online car-hailing comfort evaluation network model optimized by a genetic algorithm according to the present invention.

[0051] Fig.12 This is a measured example 4 of a method for constructing a multimodal online car-hailing comfort evaluation network model optimized by a genetic algorithm according to the present invention. DETAILED DESCRIPTION

[0052] The present invention is further described in detail below in conjunction with specific embodiments.

[0053] See also Figure 1 , a method for constructing a multi-modal online car-hailing comfort evaluation network model optimized by genetic algorithm, comprising the following steps:

[0054] S1: Information collection:

[0055] Use the vehicle-mounted camera to obtain visual observation image data of the passenger seat position; use the MQ series odor sensor and single-chip microcomputer to design an in-vehicle odor detection device to obtain in-vehicle odor concentration detection data;

[0056] In one embodiment, the specific steps are as follows:

[0057] 1-1. Camera installation:

[0058] See also Figure 2 ,The camera is installed on the longitudinal center axis of the car ceiling, close to the 1 / 3-1 / 2 of the front edge, that is, above the front seat in the car, and the visual observation image dataset of the passenger seat position in the car is collected. According to the cleanliness, it is divided into three categories: excellent, good, and poor;

[0059] 1-2. Selection and installation of odor monitoring equipment:

[0060] In one embodiment: MQ series gas sensors are selected as vehicle-mounted odor detectors to measure the concentration of common odors in the vehicle, including: MQ-2 smoke gas sensor, MQ-3 alcohol sensor, MQ-7 carbon monoxide sensor and MQ-9 combustible gas sensor, and four common odor concentration detection data in the vehicle are obtained through four odor sensors.

[0061] The odor monitoring device is installed on the side of the rear seat in the car; the on-board computing unit is fixed on the front panel in the car;

[0062] 1-3. Installation and selection of single chip microcomputer:

[0063] In one embodiment, see Figure 3 , select STM32 single chip to control and read the data of the above four MQ odor sensors. The specific method is as follows: STM32 uses USART1 for serial port transmission and reception. First enable GPIO and USART, configure the clock of serial port 1, and in the USART1 GPIO configuration, first configure USART1 Tx (PA.09) is used as a multiplexing function; among them, the serial port 1 is set to output through the PA9 pin, the GPIO mode is set to on-chip peripheral output, the pin speed is set to 50MHz and the PA port is initialized, and then, USART1Rx (PA.10) is configured to be floating, among which, the serial port 1 is set to input through the PA10 pin, the input mode is set to floating and the PA port is initialized; in the USART1 mode configuration, the serial port baud rate is set to 115200, the data length is 8 bits, 1 stop bit, no parity bit, hardware flow control bit, the receiving mode is set to full-duplex, and finally the serial port is initialized and enabled; using the ADC1 channel, PA0, PA1, PA2, PA3 are used as analog channel pins, and the corresponding voltage values ​​of MQ-2, MQ-3, MQ-7 and MQ-9 are obtained respectively, and then the odor concentration value is obtained according to the relationship between the concentration and the voltage value.

[0064] S2: Construct an end-to-end visual-smell multimodal comfort evaluation network model:

[0065] In one embodiment, see Figure 4 The “end-to-end visual-odor multimodal comfort evaluation network model” consists of the following three parts: neatness feature extraction, odor pseudo-image feature extraction, and multimodal information evaluation.

[0066] The specific construction method is as follows:

[0067] 2-1. Construct the sub-network for extracting the cleanliness feature of the vehicle interior:

[0068] The in-car cleanliness feature extraction subnetwork consists of two modules: a VGG-19 network feature extraction part and a global average pooling layer.

[0069] Among them, the VGG-19 network feature extraction part consists of five convolution blocks, each of which consists of continuous convolution layers and pooling layers. The first convolution block consists of two continuous convolution layers (3×3×64) and one maximum pooling layer (2×2); the second convolution block consists of two continuous convolution layers (3×3×128) and one maximum pooling layer (2×2); the third convolution block consists of four continuous convolution layers (3×3×256) and one maximum pooling layer (2×2); the fourth and fifth convolution blocks consist of four continuous convolution layers (3×3×512) and one maximum pooling layer (2×2), and output 7×7×512 feature maps.

[0070] The global average pooling layer is used to average all elements of each feature map, reduce the spatial dimension of the feature map to 1×1, and output a feature vector (F clean );

[0071] 2-2. Perform linear mapping on the concentration value obtained by the odor sensor and convert it into a pseudo image:

[0072] Specifically, in one embodiment, please refer to Figure 5 ,First, locate the size of the final pseudo image to be 32×32×3, and determine that the local rectangular area mapped to the image by each sensor is 16×16;

[0073] Secondly, a color space mapping function is designed to map the odor concentration detection data of each sensor to the RGB color space. The brightness values ​​of the R channel, the G channel, and the B channel are shown in formulas (1), (2), and (3):

[0074]

[0075] G i =255-R i (2)

[0076] Bi =128-R i (3)

[0077] Where i represents the sensor index value (i∈[1,4]), x i Represents the gas concentration value detected by the i-th sensor.

[0078] at the same time,

[0079] According to the brightness values ​​of the three RGB channels, the pseudo-color brightness values ​​corresponding to each odor sensor are constructed, as shown in formula (4):

[0080] S i =[R i ,G i ,B i ] T (4)

[0081] In the formula, s i Represents the RGB pseudo-color brightness vector corresponding to the i-th odor sensor.

[0082] According to the pseudo color brightness vector formula (4), the odor detection data of the four sensors are traversed to obtain the pseudo color brightness vector of each sensor;

[0083] Finally, the pseudo-color pixel values ​​of each sensor are copied and arranged to form a 16×16×3 local rectangular block, and the pseudo-color brightness values ​​of the four odor sensor detection data after RGB space mapping are combined into a 32×32×3 RGB pseudo-color image in the order of upper left, upper right, lower left, and lower right.

[0084] The main features of the pseudo image are extracted by combining basic convolution (3×3×32), pooling (2×2), and convolution (3×3×64) to obtain a 16×16×64 feature map. The feature map is converted into a feature vector of length 64 using a global average pooling layer (F odor ).

[0085] 2-3. Constructing a cleanliness-odor multimodal information evaluation subnetwork:

[0086] The subnetwork input is the 512-dimensional features output by the in-car cleanliness feature extraction module and the 64-dimensional features output by the in-car odor feature extraction module. The concatenate layer is used to realize the splicing and fusion of feature vectors of different modalities. The fused high-level features are further extracted through two fully connected layers, and the number of neurons in the two fully connected layers is 64 and 5 respectively. Finally, the softmax layer is used to realize the five-star comprehensive evaluation of the comfort of the in-car environment.

[0087] S3: Optimize the key parameters of the multimodal evaluation neural network model built in S2 by using genetic algorithm combined with back propagation learning method:

[0088] The optimal representation dimension of the cleanliness image and the optimal fusion dimension after splicing with the odor feature vector are dynamically found through genetic algorithm and back propagation learning method.

[0089] At the same time, in one embodiment, the genetic algorithm is also used to optimize other key hyperparameters of the model, including the dropout rate and the learning rate, so as to comprehensively improve the performance of the model. The genetic algorithm process is as follows: Figure 6 shown.

[0090] Specifically, in one embodiment, S3 comprises the following steps:

[0091] 3-1: Initialize the population, the population size is population←100.

[0092] Each individual contains four genes: neatness image feature dimension (d1), neatness-odor fusion feature dimension (d2), random discard probability (P d ) and the learning rate of the model (L r ), the individual is represented as:

[0093] x i =[d1,d2,P d ,L r ] (5)

[0094] The value range for each gene is as follows:

[0095]

[0096] 3-2. Design a fitness function, which uses the five-star classification accuracy of the in-car comfort as the size of the individual fitness, with maximizing the fitness function as the goal. The fitness function is expressed as:

[0097]

[0098] In the formula, x i represents an individual, Accuracy(x i ) represents the classification accuracy after model training using the parameters of the individual, N represents the total number of samples, Represents the model's predicted value for the jth sample, y j represents the true value of the jth sample, Represents the indicator function.

[0099] The results are as follows Figure 7 As shown, Figure 7The accuracy iteration curve of the comfort evaluation network model based on genetic algorithm in this application is shown.

[0100] 3-3. Select, crossover, mutation, and set iteration and termination conditions:

[0101] Specifically, tournament selection is adopted to select individuals with better performance as parents according to their fitness values.

[0102] Crossover, using the mixed crossover method, the genes of two individuals are mixed at a ratio of 0.5, and the parent generation is crossovered to generate new offspring individuals. As shown in the following formula:

[0103]

[0104] In the formula, P1 and P2 represent two parent individuals, and C1 and C2 represent newly generated offspring individuals.

[0105] Mutation, using Gaussian mutation with a mean of 0, a standard deviation of 0.2, and an independent mutation probability of 0.2, performs mutation operations on the newly generated offspring individuals to prevent them from falling into the local optimum. As shown in the following formula:

[0106] C i '=C i +N(0,0.2) (9)

[0107] Where N(0,0.2) represents a Gaussian distribution with a mean of 0 and a standard deviation of 0.2.

[0108] In one embodiment, selection, crossover and mutation are performed iteratively to optimize the average fitness of the population generation by generation, and the termination condition is that the algorithm terminates when the fitness of the optimal individual is no longer significantly improved or the number of iterations reaches a preset number.

[0109] Finally, the parameters are selected. After several iterations, the individual with the highest fitness is finally selected. This individual is the optimal parameter required, which is expressed as:

[0110]

[0111] 3-4. Based on the training data set, the cross loss function is used to measure the difference between the category probability distribution output by the model and the true label, and the network parameters of the "end-to-end visual-odor multimodal comfort evaluation network model" are obtained.

[0112] S4: Use the trained “end-to-end multimodal evaluation network model” to conduct a multi-dimensional (cleanliness, air quality) quantitative evaluation of the in-vehicle environmental comfort.

[0113] Specifically, in one embodiment, the following steps are included:

[0114] 4-1. First, input the cleanliness image to be evaluated and the odor sensor detection data into the trained end-to-end multimodal convolutional neural network;

[0115] 4-2. Input the neatness image and obtain the neatness feature vector F through the neatness feature extraction branch network based on the vehicle image. clean After the odor concentration detection data is converted into a pseudo image, the odor feature vector F is obtained based on the odor pseudo image feature extraction branch network. odor ;

[0116] 4-3. The optimal representation vector d1 of the cleanliness image is dynamically obtained through a genetic algorithm, and then the two features are fused through a vector concatenation layer to obtain a joint feature, and the optimal feature vector d2 of cleanliness-odor fusion is dynamically obtained through a genetic algorithm;

[0117] 4-4. The joint feature d2 is used to achieve a five-star comprehensive evaluation of the taxi environment comfort through two fully connected layers and a softmax layer.

[0118] The present application also provides a network model for evaluating the comfort of a multimodal online ride-hailing vehicle optimized by a genetic algorithm; the network model is constructed using the above-mentioned construction method, and includes: an on-board camera, an MQ series odor sensor, an on-board computing unit, and a display terminal. In one embodiment, the display terminal is a passenger's mobile phone, and the cleanliness and odor of the online ride-hailing vehicle are displayed through the passenger's mobile phone.

[0119] Figure 9-12 The actual usage status diagrams of the above-mentioned network models are given respectively. The network model of a multimodal online car-hailing comfort evaluation optimized by a genetic algorithm provided in this application uses a vehicle-mounted camera to obtain visual observation image data of the passenger seat position; uses the MQ series odor sensor and a single-chip microcomputer to design an in-vehicle odor detection device to obtain four common in-vehicle odor concentration detection data. The network model of this application is aimed at visual-odor multi-sensor fusion. The end-to-end multimodal convolutional neural network can complement the information advantages of vision and odor from multiple angles, extract effective comfort representation information, and more objectively and comprehensively realize multi-dimensional, real-time monitoring and evaluation of the comfort of the in-vehicle environment.

Claims

1. A method for constructing a multi-modal online car-hailing comfort evaluation network model optimized by genetic algorithm, characterized by: The following steps are involved: S1: Information collection: collect visual observation image data of passenger seats and odor concentration detection data in the vehicle; S2: Construct an end-to-end visual-smell multimodal comfort evaluation network model: The end-to-end visual-smell multimodal comfort evaluation network model consists of the following three parts: neatness feature extraction, odor pseudo-image feature extraction, and multimodal information evaluation; S3: Using genetic algorithm combined with back propagation learning method, the key parameters of the multimodal evaluation neural network model built in S2 are optimized: S4: Use the trained "end-to-end multimodal evaluation network model" to conduct a multi-dimensional (cleanliness, air quality) quantitative evaluation of the comfort level of the in-vehicle environment.

2. The method for constructing a multi-modal online car-hailing comfort evaluation network model optimized by genetic algorithm according to claim 1, characterized in that: The S3 comprises the following steps: 3-1. Initialize the population, the population size is population←100: Each individual contains four genes: neatness image feature dimension (d1), neatness-odor fusion feature dimension (d2), random discard probability (P d ) and the learning rate of the model (L r ), The value range for each gene is as follows: 3-2. Design a fitness function, which uses the five-star classification accuracy of the in-car comfort as the size of the individual fitness, with maximizing the fitness function as the goal. The fitness function is expressed as: 3-3. Select, crossover, mutation, and set iteration and termination conditions: Using tournament selection, individuals with better performance are selected as parents based on their fitness values; Crossover, using the mixed crossover method, the genes of the two individuals are mixed at a ratio of 0.5, and the parent generation is crossovered to generate new offspring individuals. Mutation: Use Gaussian mutation with a mean of 0, a standard deviation of 0.2, and an independent mutation probability of 0.2 to perform mutation operations on the newly generated offspring individuals to prevent them from falling into the local optimum; And terminate the selection of final parameters after iteration; 3-4. Based on the training data set, the cross loss function is used to measure the difference between the category probability distribution output by the model and the true label, and the network parameters of the "end-to-end visual-odor multimodal comfort evaluation network model" are obtained.

3. The method for constructing a multi-modal online car-hailing comfort evaluation network model optimized by genetic algorithm according to claim 1, characterized in that: The termination condition in 3-3 is that the algorithm terminates when the fitness of the optimal individual is no longer significantly improved or the number of iterations reaches a preset number. The final parameters obtained in 3-3 are expressed as:

4. The method for constructing a multi-modal online car-hailing comfort evaluation network model optimized by genetic algorithm according to claim 1, characterized in that: In the S1, the vehicle-mounted camera is used to obtain visual observation image data of the passenger seat position, and the MQ series gas sensor and the single-chip microcomputer are used to design an in-vehicle odor detection device to obtain in-vehicle odor concentration detection data.

5. The method for constructing a multi-modal online car-hailing comfort evaluation network model optimized by genetic algorithm according to claim 4, characterized in that: The MQ series gas sensors are MQ-2 smoke sensor, MQ-3 alcohol sensor, MQ-7 carbon monoxide sensor and MQ-9 combustible gas sensor. Four odor sensors are used to obtain detection data of four common in-vehicle odor concentrations.

6. The method for constructing a multi-modal online car-hailing comfort evaluation network model optimized by genetic algorithm according to claim 4, characterized in that: The single chip microcomputer is an STM32 single chip, which is used for controlling the box to read the data of the MQ odor sensor.

7. The method for constructing a multi-modal online car-hailing comfort evaluation network model optimized by genetic algorithm according to claim 1, characterized in that: The specific steps of S2 are as follows: 2-1. Construct the sub-network for extracting the cleanliness feature of the vehicle interior: The vehicle interior cleanliness feature extraction subnetwork consists of two modules: a VGG-19 network feature extraction part and a global average pooling layer; 2-2. Perform linear mapping on the concentration value obtained by the odor sensor and convert it into a pseudo image: 2-3. Constructing a cleanliness-odor multimodal information evaluation subnetwork: The subnetwork input is the 512-dimensional features output by the in-car cleanliness feature extraction module and the 64-dimensional features output by the in-car odor feature extraction module. The concatenate layer is used to realize the splicing and fusion of feature vectors of different modalities. The fused high-level features are further extracted through two fully connected layers, and the number of neurons in the two fully connected layers is 64 and 5 respectively. Finally, the softmax layer is used to realize the five-star comprehensive evaluation of the comfort of the in-car environment.

8. A genetic algorithm optimized multi-modal online car-hailing comfort evaluation network model, characterized by: The method is constructed using any one of claims 1 to 7.

9. The genetic algorithm optimized multi-modal online car-hailing comfort evaluation network model according to claim 8, characterized in that: The network model includes: a vehicle-mounted camera, an MQ series odor sensor, a vehicle-mounted computing unit and a display terminal.

Images