Scene classification model training method and device, computer equipment and storage medium

Abstract

The invention discloses a scene classification model training method and device, computer equipment and a storage medium, and relates to the technical field of intelligent cabin interaction, and the method comprises the steps: carrying out the preprocessing of historical video data, and obtaining a sample image set; inputting the sample image set into an initial scene classification model for training to obtain a historical classification feature vector corresponding to each sample image; determining a first loss corresponding to each sample image according to the historical classification feature vector corresponding to each sample image, the scene dimension label corresponding to each sample image and the cross entropy loss function with label smoothness, and adjusting the first loss corresponding to each sample image according to a preset adjustment method to obtain a first loss corresponding to each sample image; and obtaining the target loss, and training the initial scene classification model based on the target loss to obtain a target scene classification model. Therefore, the video scene can be accurately identified through the target scene classification model when the user watches the video.

Description

Technical Field

[0001] The present application relates to the field of intelligent cockpit interaction technology, and in particular to a scene classification model training method, device, computer equipment and storage medium. Background Art

[0002] With the development of vehicle technology, in-car viewing has become increasingly popular, and users can now watch movies and TV shows in their cars. Currently, the main way users watch movies and TV shows is through screen projection. Since the vehicle cannot recognize the video scene in real time, it cannot provide a corresponding cabin environment based on the video scene, and cannot provide users with a more immersive viewing experience. Summary of the invention

[0003] Based on this, a scene classification model training method, device, computer equipment and storage medium are provided to solve the problem in the prior art that the vehicle cannot recognize the video scene in real time, and thus cannot provide a corresponding cabin environment according to the video scene, and cannot provide users with a more immersive viewing experience.

[0004] In a first aspect, the present application provides a scene classification model training method, the method comprising:

[0005] Preprocess the historical video data to obtain a sample image set; wherein the sample image set includes scene dimension labels corresponding to the sample images, the scene dimension labels include environment category dimension labels and cold and warm dimension labels, the environment category dimension labels are associated with fragrance release instructions, and the cold and warm dimension labels are associated with air conditioning cold and warm wind instructions;

[0006] Inputting the sample image set into an initial scene classification model for training to obtain a historical classification feature vector corresponding to each of the sample images;

[0007] Determine a first loss corresponding to each of the sample images according to a historical classification feature vector corresponding to each of the sample images, a scene dimension label corresponding to each of the sample images, and a cross entropy loss function with label smoothing; wherein the first loss includes losses corresponding to different scene dimensions;

[0008] The first loss corresponding to each of the sample images is adjusted according to a preset adjustment method to obtain a target loss, and the initial scene classification model is trained based on the target loss to obtain a target scene classification model.

[0009] Through the above method, the initial scene classification model is trained based on historical video data to obtain the target scene classification model, and the loss is adjusted according to the preset adjustment method, which can improve the discrimination ability and accuracy of the target scene classification model. Therefore, the target scene classification model can be used to accurately identify the video scene when the user watches the video, and provide the corresponding cabin environment according to the video scene, providing the user with a more immersive viewing experience.

[0010] In one embodiment, the inputting the sample image set into the initial scene classification model for training to obtain the historical classification feature vector corresponding to each of the sample images includes:

[0011] Performing convolution processing on each of the sample images to obtain a multi-scale local feature map corresponding to each of the sample images, and performing feature extraction on each of the sample images based on a self-attention mechanism to obtain a global feature map corresponding to each of the sample images;

[0012] Fusing the multi-scale local feature map corresponding to each of the sample images with the global feature map corresponding to each of the sample images to obtain a fused feature map corresponding to each of the sample images;

[0013] Performing upsampling processing on each of the fused feature maps respectively to obtain each target fused feature map;

[0014] Each of the target fusion feature maps is input into the fully connected layer in the initial scene classification model, and the historical classification feature vector corresponding to each of the sample images is output.

[0015] Through the above method, the local features and global features of the sample image learned by CNN and Transformer are fused to obtain a fused feature map, and training is performed based on the fused feature map, which can effectively improve the accuracy and generalization ability of the target scene classification model.

[0016] In one embodiment, adjusting the first loss corresponding to each of the sample images according to a preset adjustment method to obtain a target loss includes:

[0017] Based on the first loss corresponding to each of the sample images, generating a loss vector corresponding to each of the sample images;

[0018] According to a preset selection method, K loss components corresponding to different scene dimensions are selected from the loss vectors corresponding to each of the sample images; wherein K is an integer greater than 0;

[0019] According to the preset adjustment matrix, adaptive weight adjustment is performed on the K loss components corresponding to different scene dimensions to obtain K target loss components corresponding to different scene dimensions;

[0020] Based on the K target loss components corresponding to different scene dimensions, generate the second losses corresponding to different scene dimensions;

[0021] Loss balancing is performed on the second losses corresponding to different scene dimensions to obtain the target loss.

[0022] Through the above method, K loss components corresponding to different scene dimensions are selected as the difficult samples corresponding to different scene dimensions, so as to adaptively adjust the weights of these difficult samples, which can improve the model's ability to distinguish difficult samples, and introduce a loss balancing strategy to improve the problem of unbalanced loss calculation in each scene dimension.

[0023] In one embodiment, the adaptive weight adjustment is performed on the K loss components corresponding to different scene dimensions according to the preset adjustment matrix to obtain the K target loss components corresponding to different scene dimensions, including:

[0024] Generate loss contribution weights corresponding to the K loss components according to a preset adjustment matrix;

[0025] Normalizing the loss contribution weights corresponding to the K loss components to obtain target loss contribution weights corresponding to the K loss components;

[0026] Based on the target loss contribution weights of each of the K loss components, the K loss components are weighted to obtain K target loss components.

[0027] Through the above method, based on the preset adjustment matrix, the weights of the K target loss components corresponding to different scene dimensions are adjusted to improve the discrimination ability of the model.

[0028] In one embodiment, performing loss balancing processing on the second losses corresponding to different scene dimensions to obtain the target loss includes:

[0029] According to the preset balance weights corresponding to the different scene dimensions, the second losses corresponding to the different scene dimensions are weighted to obtain the third losses corresponding to the different scene dimensions;

[0030] The target loss is obtained by calculating the average value of the third losses corresponding to different scene dimensions.

[0031] Through the above method, based on the preset balancing weights corresponding to different scene dimensions, the second losses corresponding to different scene dimensions are balanced, thereby reducing the impact of loss calculation caused by sample imbalance, which can help the model better learn the true distribution of data and improve the generalization ability of the model.

[0032] In a second aspect, the present application provides a method for adjusting a cabin environment, which adopts the target scene classification model in the first aspect, and the method comprises:

[0033] Preprocess the video data to be identified to obtain a target image set;

[0034] Inputting the target image set into the target scene classification model to obtain a target classification feature vector corresponding to each target image in the target image set;

[0035] Determine the target instruction corresponding to each of the target classification feature vectors;

[0036] Based on the target instructions corresponding to each of the target classification feature vectors, the vehicle cabin environment is adjusted.

[0037] Through the above method, the scene dimensions corresponding to the video data to be identified are accurately identified based on the target scene classification model, and the corresponding target instructions are determined according to the identification results, so as to adjust the cabin environment according to the target instructions to provide users with a more immersive viewing experience.

[0038] In one embodiment, determining the target instruction corresponding to each of the target classification feature vectors includes:

[0039] When the scene dimension corresponding to the target classification feature vector is an environment category dimension, determining that the target instruction is a fragrance release instruction corresponding to the environment category dimension;

[0040] When the scene dimension corresponding to the target classification feature vector is a cooling and heating dimension, the target instruction is determined to be an air conditioning cooling and heating instruction corresponding to the cooling and heating dimension.

[0041] Through the above method, the release of fragrance or the cooling and heating of air conditioners are controlled based on target instructions, providing users with a more immersive viewing experience.

[0042] In a third aspect, the present application provides a scene classification model training device, the device comprising:

[0043] A first preprocessing module is used to preprocess the historical video data to obtain a sample image set; wherein the sample image set includes a scene dimension label corresponding to the sample image, and the scene dimension label includes an environment category dimension label and a cold and warm dimension label, the environment category dimension label is associated with a fragrance release instruction, and the cold and warm dimension label is associated with an air conditioning cold and warm wind instruction;

[0044] A first training module is used to input the sample image set into an initial scene classification model for training to obtain a historical classification feature vector corresponding to each of the sample images;

[0045] A first determination module is used to determine a first loss corresponding to each of the sample images according to a historical classification feature vector corresponding to each of the sample images, a scene dimension label corresponding to each of the sample images, and a cross entropy loss function with label smoothing; wherein the first loss includes losses corresponding to different scene dimensions;

[0046] The second training module is used to adjust the first loss corresponding to each of the sample images according to a preset adjustment method to obtain a target loss, and train the initial scene classification model based on the target loss to obtain a target scene classification model.

[0047] In one embodiment, the first training module is specifically used for:

[0048] Performing convolution processing on each of the sample images to obtain a multi-scale local feature map corresponding to each of the sample images, and performing feature extraction on each of the sample images based on a self-attention mechanism to obtain a global feature map corresponding to each of the sample images;

[0049] Fusing the multi-scale local feature map corresponding to each of the sample images with the global feature map corresponding to each of the sample images to obtain a fused feature map corresponding to each of the sample images;

[0050] Performing upsampling processing on each of the fused feature maps respectively to obtain each target fused feature map;

[0051] Each of the target fusion feature maps is input into the fully connected layer in the initial scene classification model, and the historical classification feature vector corresponding to each of the sample images is output.

[0052] In one embodiment, the second training module is specifically used for:

[0053] Based on the first loss corresponding to each of the sample images, generating a loss vector corresponding to each of the sample images;

[0054] According to a preset selection method, K loss components corresponding to different scene dimensions are selected from the loss vectors corresponding to each of the sample images; wherein K is an integer greater than 0;

[0055] According to the preset adjustment matrix, adaptive weight adjustment is performed on the K loss components corresponding to different scene dimensions to obtain K target loss components corresponding to different scene dimensions;

[0056] Based on the K target loss components corresponding to different scene dimensions, generate the second losses corresponding to different scene dimensions;

[0057] Loss balancing is performed on the second losses corresponding to different scene dimensions to obtain the target loss.

[0058] In one embodiment, the second training module is further used for:

[0059] Generate loss contribution weights corresponding to the K loss components according to a preset adjustment matrix;

[0060] Normalizing the loss contribution weights corresponding to the K loss components to obtain target loss contribution weights corresponding to the K loss components;

[0061] Based on the target loss contribution weights of each of the K loss components, the K loss components are weighted to obtain K target loss components.

[0062] In one embodiment, the second training module is further used for:

[0063] According to the preset balance weights corresponding to the different scene dimensions, the second losses corresponding to the different scene dimensions are weighted to obtain the third losses corresponding to the different scene dimensions;

[0064] The target loss is obtained by calculating the average value of the third losses corresponding to different scene dimensions.

[0065] In a fourth aspect, the present application provides a cabin environment adjustment device, the device comprising:

[0066] A second preprocessing module is used to preprocess the video data to be identified to obtain a target image set;

[0067] A classification module, used for inputting the target image set into the target scene classification model to obtain a target classification feature vector corresponding to each target image in the target image set;

[0068] A second determination module is used to determine the target instruction corresponding to each of the target classification feature vectors;

[0069] The adjustment module is used to adjust the vehicle cabin environment based on the target instructions corresponding to each of the target classification feature vectors.

[0070] In one embodiment, the second determining module is specifically configured to:

[0071] When the scene dimension corresponding to the target classification feature vector is an environment category dimension, determining that the target instruction is a fragrance release instruction corresponding to the environment category dimension;

[0072] When the scene dimension corresponding to the target classification feature vector is a cooling and heating dimension, the target instruction is determined to be an air conditioning cooling and heating instruction corresponding to the cooling and heating dimension.

[0073] In a fifth aspect, the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the scene classification model training method of the first aspect and the cabin environment adjustment method of the second aspect when executing the computer program.

[0074] In a sixth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the scene classification model training method of the first aspect and the cabin environment adjustment method of the second aspect are implemented.

[0075] The above-mentioned scene classification model training method, device, computer equipment and storage medium perform model training on the initial scene classification model based on historical video data to obtain the target scene classification model, and perform loss adjustment according to a preset adjustment method, which can improve the discrimination ability and accuracy of the target scene classification model. Therefore, the target scene classification model can be used to accurately identify the video scene when the user watches the video, and provide a corresponding cabin environment according to the video scene, providing the user with a more immersive viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 A schematic diagram of a flow chart of a scene classification model training method in one embodiment;

[0077] Figure 2 A schematic diagram of a flow chart of a model training method in one embodiment;

[0078] Figure 3 is a structural block diagram of an initial scene classification model in one embodiment;

[0079] Figure 4 A schematic diagram of a flow chart of a model training method in another embodiment;

[0080] Figure 5is a structural block diagram of an initial scene classification model in another embodiment;

[0081] Figure 6 A schematic diagram of a flow chart of a loss adjustment method in one embodiment;

[0082] Figure 7 is a flowchart of a method for adjusting a cabin environment in one embodiment;

[0083] Figure 8 is a structural block diagram of a scene classification model training device in one embodiment;

[0084] Fig. 9 is a structural block diagram of a cabin environment adjustment device in one embodiment;

[0085] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0086] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The specific operating methods in the method embodiments can also be applied to device embodiments or system embodiments. It should be noted that in the description of the present application, "multiple" is understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, and A and B exist, and B exists alone. A is connected to B, which can represent: A is directly connected to B and A is connected to B through C. In addition, in the description of the present application, words such as "first" and "second" are only used to distinguish the purpose of description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0087] The method and device described in the embodiment of the present application are based on the same technical concept. Since the principles of the method and device to solve the problem are similar, the embodiments of the device and method can refer to each other, and the repeated parts will not be repeated.

[0088] Embodiment 1

[0089] Figure 1 FIG. 1 is a flow chart of a scene classification model training method in one embodiment, and the flow chart can be executed by a scene classification model training device. Figure 1 As shown, the process includes the following steps:

[0090] S101, pre-processing historical video data to obtain a sample image set.

[0091] Specifically, the historical video data is processed by frame extraction to obtain multiple historical images, and each historical image is scaled and adjusted respectively, and random noise, mosaic data enhancement, color tremor ColorJitter operation and normalization are added to obtain multiple pre-processed historical images. Among them, proportional scaling can avoid image distortion or distortion, which is particularly important for tasks such as object detection and image classification, because distorted images may cause the model to misunderstand the image content and affect the training effect. Adding random noise can improve the robustness of the algorithm. Mosaic data enhancement can increase the diversity of pictures, enrich background information, and avoid overfitting. The ColorJitter operation can randomly change the brightness, contrast, saturation and hue of the image, increase the diversity of the data set, and thus improve the robustness and generalization ability of the deep learning model. The normalization operation can improve the stability and convergence speed of model training, and prevent some eigenvalues ​​from being too large or too small to dominate or ignore the learning process of the model.

[0092] Then, the scene dimension labels corresponding to each preprocessed historical image are obtained respectively, and a sample image set is generated based on each preprocessed historical image and the scene dimension labels corresponding to each preprocessed historical image, wherein the scene dimension labels include environment category dimension labels and cold and warm dimension labels, the environment category dimension labels are associated with fragrance release instructions, and the cold and warm dimension labels are associated with air conditioning cold and warm wind instructions, the scene dimension labels can be obtained based on expert experience, and the specific acquisition method depends on the situation and is not limited here.

[0093] Exemplarily, the environment category dimension tags include water environment tags, flower environment tags, and forest-steppe environment tags. The water environment tag is associated with a first fragrance release instruction, which instructs the fragrance control unit to release fragrance related to the water environment; the flower environment tag is associated with a second fragrance release instruction, which instructs the fragrance control unit to release fragrance related to the flower environment; and the forest-steppe environment tag is associated with a third fragrance release instruction, which instructs the fragrance control unit to release fragrance related to the forest-steppe environment.

[0094] The cold and warm dimension labels include cold scene labels and warm scene labels, and the cold scene labels are associated with the air conditioner cold wind instructions, which instruct the air conditioner to blow cold air; the warm scene labels are associated with the air conditioner warm wind instructions, which instruct the air conditioner to blow warm air.

[0095] In another embodiment, negative sample images may also be collected, and the negative sample images are proportionally scaled, and random noise, mosaic data enhancement, color jitter ColorJitter operation, and normalization are added to the sample image set. Negative sample images refer to images other than water environment, flower environment, forest-steppe environment, cold scene, and warm scene. For negative sample images, this embodiment samples them according to a set probability (e.g., 0.3) in the data loader dataloader.

[0096] Through the above method, the diversity of the sample image set can be improved, so that when the model is trained based on the sample image set, the accuracy and generalization ability of the model can be improved.

[0097] S101, inputting the sample image set into the initial scene classification model for training, and obtaining the historical classification feature vector corresponding to each sample image;

[0098] Among them, the initial scene classification model is a deep learning model, including: a deep learning model based on a combination of a convolutional neural network (CNN)-Transformer and a deep learning model based on Mobilenetv4 small. Of course, the initial scene classification model can also be other deep learning models or neural network models. Those skilled in the art can use any model to perform model training based on this application.

[0099] S102, determining a first loss corresponding to each sample image according to a historical classification feature vector corresponding to each sample image, a scene dimension label corresponding to each sample image, and a cross entropy loss function with label smoothing;

[0100] The first loss includes the losses corresponding to different scene dimensions. The cross entropy loss function with label smoothing can make the model less dependent on the true value and alleviate the overfitting of the model.

[0101] S103, adjusting the first loss corresponding to each sample image according to a preset adjustment method to obtain a target loss, and training the initial scene classification model based on the target loss to obtain a target scene classification model;

[0102] S104, training the initial scene classification model based on the target loss to obtain a target scene classification model.

[0103] Specifically, when the target loss is greater than a preset threshold, the target loss is back-propagated to the initial scene classification model, the parameters of the initial scene classification model are adjusted, and the above training process is repeated until the target loss is less than the preset threshold, thereby obtaining the target scene classification model. The preset threshold depends on the situation and is not limited here.

[0104] Further, in conjunction with the accompanying drawings, the input of the sample image set into the initial scene classification model for training in S101 to obtain the historical classification feature vector corresponding to each sample image is described in detail:

[0105] Figure 2 FIG. 1 is a flow chart of a model training method in an embodiment. Figure 2 As shown, the process specifically includes the following steps:

[0106] S201, performing convolution processing on each sample image to obtain a local feature map corresponding to each sample image, and performing feature extraction on each sample image based on a self-attention mechanism to obtain a global feature map corresponding to each sample image;

[0107] S202, fusing the local feature map corresponding to each sample image with the global feature map corresponding to each sample image to obtain a fused feature map corresponding to each sample image;

[0108] S203, performing upsampling processing on each fused feature map respectively to obtain each target fused feature map;

[0109] S204, input each target fusion feature map into the fully connected layer in the initial scene classification model, and output the historical classification feature vector corresponding to each sample image.

[0110] For example, Figure 3 As shown, it is a structural block diagram of an initial scene classification model in an embodiment, including a CNN encoder 301 with Hierarchy-Aware Pixel-Excitation (HAPE), an efficient Transformer encoder 302, a fusion module 303, a decoder 304 and a fully connected layer 305. Taking a sample image I1 with a size of 480*288*3 as an input as an example, the model training process is specifically as follows:

[0111] First, I1 is input into the CNN encoder 301 with HAPE, and I1 is convolved through the CNN stem to generate a first local feature map with a size of 240*144*C1; the first local feature map is convolved through the downsampling DOWN module with HAPE to generate a second local feature map with a size of 120*72*C2; the second local feature map is convolved through the DOWN HAPE module to generate a third local feature map with a size of 60*36*C3; the third local feature map is convolved through the upsampling Upsample module with HAPE to generate a multi-scale local feature map F with a size of 120*72*C4, where C1, C2, C3, and C4 represent different numbers of channels.

[0112] At the same time, I1 is input into the Transformer encoder 302, and after feature extraction processing by Transformer stem and patch embedding PatchEmbedding, it enters the efficient Transformer to generate the global feature map T∈R^(N×D) corresponding to I1, where N represents the number of tags, D represents the dimension of each tag, and P represents the tag size:

[0113]

[0114] D=C t ×P 2 (2)

[0115] In formula (1), H t =288 / 16,W t =480 / 16.

[0116] Furthermore, based on the fusion module 303, F and T are subjected to correlation weighted fusion processing, and two different types of context features F and T are combined to obtain a fused feature map corresponding to I1. The fused feature map is upsampled based on the upsample Upsample module with HAPE in the decoder 304 to obtain a target fused feature map with a size of 240*144*C5, where C5 represents the number of channels.

[0117] Finally, the target fusion feature map is input into the fully connected layer 305, and the historical classification feature vector corresponding to I1 is output, wherein the historical classification feature vector corresponds to the recognition result of the model for I1, including the recognized scene dimensions, such as the environmental category dimension and the cold and warm dimension.

[0118] Through the above method, the local and global context features of the sample image learned by CNN and Transformer are fused to obtain a fused feature map, and training is performed based on the fused feature map, which can effectively improve the accuracy and generalization ability of the target scene classification model.

[0119] Figure 4 FIG. 1 is a flow chart of a model training method in another embodiment. Figure 4 As shown, the process specifically includes the following steps:

[0120] S401, extracting features from each sample image to obtain a multi-scale feature map corresponding to each sample image;

[0121] S402, performing fusion processing on the multi-scale feature maps corresponding to each sample image to obtain a first target feature map corresponding to each sample image;

[0122] S403, performing downsampling processing on each first target feature map respectively to obtain each second target feature map;

[0123] S404, input each second target feature map into the fully connected layer in the initial scene classification model to obtain the historical classification feature vector corresponding to each sample image.

[0124] For example, Figure 5 , which is a structural block diagram of an initial scene classification model in another embodiment, includes a feature extraction module 501, a feature fusion module 502, a downsampling module 503 and a fully connected layer 504. Taking a 224*224 sample image I2 as an example, the model training process is specifically as follows:

[0125] First, I2 is input into the feature extraction module 501 to obtain a multi-scale feature map, wherein the feature extraction module 501 includes an inverted neck structure (Universal Inverted Bottleneck, UIB), specifically including multiple feature extraction convolutional (Convolutional, Conv) layers and multiple feature extraction attention (Mobile MQA, MMQA) modules connected in series, wherein the feature extraction convolutional layer includes a deep convolutional layer (Conv layer) and an inverted neck convolutional layer (UIB layer), and each feature extraction attention module matches a UIB layer to form a UIB+MMQA layer.

[0126] Specifically, I2 is input into the first Conv layer to generate a first scale feature map of 112*112; the first scale feature map is input into the second Conv layer to generate a second scale feature map of 56*56; the second scale feature map is input into the third Conv layer to generate a third scale feature map of 28*28; the third scale feature map is input into the first UIB+MMQA layer to generate a fourth scale feature map of 14*14; the fourth scale feature map is input into the second UIB+MMQA layer to generate a fifth scale feature map of 7*7. Thus, each Conv layer and each UIB+MMQA layer of the feature extraction module 501 outputs a scale feature map to obtain a multi-scale feature map.

[0127] The feature extraction module of the present application adopts a combination of deep convolutional layers and inverted neck convolutional layers to extract feature data of sample images from multiple scales, thereby obtaining a multi-scale feature map, realizing parameter sharing across different instantiations, and making the model more efficient.

[0128] In one embodiment, in order to improve the generalization ability and robustness of the model, instance normalization (IN) may be added to the inverted bottleneck structure of the feature extraction module, so as to normalize the features of the sample images to the same range.

[0129] Then, the multi-scale feature map is input into the feature fusion module 502 for feature fusion, wherein the feature fusion module 502 includes a plurality of transposed convolution (DeConv) layers connected in series. The multi-scales are respectively input into the plurality of transposed convolution layers, and the multi-scale feature maps are fused to obtain the first target feature map corresponding to I2.

[0130] Specifically, the feature fusion module 502 is a feature pyramid structure, which inputs the fifth-scale feature map into the first DeConv layer, and the fourth-scale feature map into the first DeConv layer to generate a first fusion map of 14*14; the first fusion map is input into the second DeConv layer, and the third-scale feature map is input into the second DeConv layer to generate a second fusion map of 28*28; the second fusion map is input into the third DeConv layer, and the second-scale feature map is input into the third DeConv layer to generate a third fusion map of 56*56, that is, I2 corresponds to a first target feature map of 56*56.

[0131] Furthermore, the first target feature map is input into the downsampling module 503, and the first target feature map is subjected to the maximum pooling operation to generate a 28*28 second target feature map. Finally, the second target feature map is input into the fully connected layer 504, and the historical classification feature vector corresponding to I2 is output, wherein the historical classification feature vector corresponds to the recognition result of the model for I2, including the recognized scene dimension, such as the environment category dimension and the cold and warm dimension.

[0132] Through the above method, based on the MobileNet V4 architecture, by introducing IN, multi-scale feature fusion, MobileMQA and UIB search blocks, the accuracy, generalization ability and reasoning speed of the model are improved, and it is suitable for a variety of hardware platforms.

[0133] Further, in conjunction with the accompanying drawings, the first loss corresponding to each sample image is adjusted according to the preset adjustment method in S103 to obtain the target loss, which is described in detail:

[0134] Figure 6 This is a flow chart of a loss adjustment method in an embodiment. In an embodiment of the present application, a method for online mining of difficult samples is provided: during the training process, an online mining method for scene dimensions that are difficult to distinguish is established, and the scene dimensions that are difficult to distinguish are regarded as difficult samples. Thus, the loss contribution weights of these difficult samples are adjusted by presetting an adjustment matrix, thereby indirectly improving the model's ability to discriminate difficult samples. The preset adjustment matrix dynamically adjusts the weight of each loss component according to the current model learning state and error situation to ensure that the model can learn and optimize better. For samples that are easy to distinguish, a regular penalty term is added to alleviate overfitting during the loss calculation process. Figure 6 As shown, the process specifically includes the following steps:

[0135] S601, generating a loss vector corresponding to each sample image based on a first loss corresponding to each sample image;

[0136] The dimension of the loss vector is the scene dimension. For example, the scene dimension includes five types: water scene, flower scene, forest-grassland scene, cold scene, and warm scene, and the dimension of the loss vector is 5.

[0137] S602, selecting K loss components corresponding to different scene dimensions from the loss vector corresponding to each sample image according to a preset selection method;

[0138] Wherein, K is an integer greater than 0, and the specific value depends on the situation and is not limited here.

[0139] Specifically, in the loss vector corresponding to each sample image, the loss components belonging to the same scene dimension are sorted in descending order, and the K loss components with the largest values ​​are selected according to the sorting to obtain K loss components corresponding to different scene dimensions.

[0140] S603, according to a preset adjustment matrix, adaptively weighting the K loss components corresponding to the different scene dimensions to obtain the K target loss components corresponding to the different scene dimensions;

[0141] Specifically, according to the preset adjustment matrix, the loss contribution weights corresponding to the K loss components are generated, and the loss contribution weights corresponding to the K loss components are normalized through the softmax layer to obtain the target loss contribution weights corresponding to the K loss components, thereby realizing the normalization of the loss contribution weights. Then, based on the target loss contribution weights of the K loss components, the K loss components are weighted to obtain the K target loss components.

[0142] S604, generating second losses corresponding to different scene dimensions based on the K target loss components corresponding to different scene dimensions;

[0143] Specifically, the K loss components belonging to the same scene dimension and the remaining loss components other than the K loss components in the loss vector corresponding to each sample image are averaged to obtain the average loss corresponding to different scene dimensions. Then, based on the average loss and the regularization penalty term, the second loss corresponding to different scene dimensions is generated. For example, the average loss is added to the regularization penalty term to obtain the second loss, thereby alleviating model overfitting by adding the regularization penalty term.

[0144] S605, performing loss balancing processing on the second losses corresponding to different scene dimensions to obtain a target loss.

[0145] This application provides a loss balancing strategy: setting corresponding preset balancing weights according to the recognition difficulty of each scene dimension can help the model better learn the true distribution of the data, thereby improving the generalization ability of the model.

[0146] Specifically, according to the preset balance weights corresponding to different scene dimensions, the second losses corresponding to different scene dimensions are weighted to obtain the third losses corresponding to different scene dimensions. Then, the average value of the third losses corresponding to different scene dimensions is calculated to obtain the target loss.

[0147] It should be noted that the above preset balance weight is less than 1, and the sum of the preset balance weights corresponding to different scene dimensions is equal to 1. The specific value depends on the situation and is not limited here.

[0148] In the above-mentioned scene classification model training method, the initial scene classification model is trained based on historical video data to obtain the target scene classification model. The performance and reliability of the model are improved through online mining of difficult samples and loss balance strategy. Therefore, the target scene classification model can be used to accurately identify video scenes when users watch movies and TV shows, and control the release of fragrance and the cooling and heating of air conditioners according to the identified video scenes, providing users with a more immersive viewing experience.

[0149] Embodiment 2

[0150] Figure 7 FIG. 1 is a flow chart of a method for adjusting a cabin environment in an embodiment. The method adopts the target scene classification model in the first embodiment. The process can be executed by a cabin environment adjustment device. Figure 7 As shown, the process includes the following steps:

[0151] S701, preprocessing the video data to be recognized to obtain a target image set;

[0152] The preprocessing method is consistent with the preprocessing method in the first embodiment, and will not be described again here.

[0153] S702, inputting the target image set into the target scene classification model to obtain a target classification feature vector corresponding to each target image in the target image set;

[0154] Among them, the target classification feature vector corresponds to the recognition result of the target scene classification model on the target image, including the recognized scene dimensions, such as the environment category dimension and the cold and warm dimension. The environment category dimension includes water environment, flower environment, forest and grassland environment, and the cold and warm dimension includes cold scene and warm scene.

[0155] S703, determining the target instruction corresponding to each target classification feature vector;

[0156] Among them, the target command is a fragrance release command and / or an air conditioning cooling or heating command.

[0157] Specifically, when the scene dimension corresponding to the target classification feature vector is the environment category dimension, the target instruction is determined to be the fragrance release instruction corresponding to the environment category dimension. To elaborate, when the scene dimension corresponding to the target classification feature vector is the water environment, the target instruction is determined to be the first fragrance release instruction corresponding to the water environment, and the first fragrance release instruction instructs the fragrance control unit to release the fragrance related to the water environment; when the scene dimension corresponding to the target classification feature vector is the flower environment, the target instruction is determined to be the second fragrance release instruction corresponding to the flower environment, and the second fragrance release instruction instructs the fragrance control unit to release the fragrance related to the flower environment; when the scene dimension corresponding to the target classification feature vector is the forest-steppe environment, the target instruction is determined to be the third fragrance release instruction corresponding to the forest-steppe environment, and the third fragrance release instruction instructs the fragrance control unit to release the fragrance related to the forest-steppe environment.

[0158] When the scene dimension corresponding to the target classification feature vector is the cold and warm dimension, the target instruction is determined to be the air conditioning cold and warm wind instruction corresponding to the cold and warm dimension. To elaborate, when the scene dimension corresponding to the target classification feature vector is a cold scene, the target instruction is determined to be the air conditioning cold wind instruction corresponding to the cold scene, and the air conditioning cold wind instruction instructs the air conditioning device to blow cold air; when the scene dimension corresponding to the target classification feature vector is a warm scene, the target instruction is determined to be the air conditioning warm wind instruction corresponding to the warm scene, and the air conditioning warm wind instruction instructs the air conditioning device to blow warm air.

[0159] S704, adjusting the vehicle cabin environment based on the target instructions corresponding to each target classification feature vector.

[0160] Specifically, when the target instruction is a fragrance release instruction, the fragrance release instruction is sent to the fragrance control unit to enable the fragrance control unit to release the first fragrance, the second fragrance, and the third fragrance; when the target instruction is an air conditioning cold and warm wind instruction, the air conditioning cold and warm wind instruction is sent to the air conditioner to enable the air conditioner to turn on the cold wind and warm wind, thereby providing users with a more immersive viewing experience.

[0161] In the above-mentioned cabin environment adjustment method, the scene dimensions corresponding to the video data to be identified are accurately identified based on the target scene classification model, and the corresponding target instructions are determined according to the identification results, so as to adjust the cabin environment according to the target instructions to provide users with a more immersive viewing experience.

[0162] It should be understood that although Figure 1-7 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-7 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0163] In one embodiment, Figure 8 As shown, a scene classification model training device is provided, including: a first preprocessing module 801, a first training module 802, a first determination module 803 and a second training module 804, wherein:

[0164] The first preprocessing module 801 is used to preprocess the historical video data to obtain a sample image set; wherein the sample image set includes a scene dimension label corresponding to the sample image, and the scene dimension label includes an environment category dimension label and a cold and warm dimension label. The environment category dimension label is associated with a fragrance release instruction, and the cold and warm dimension label is associated with an air conditioning cold and warm wind instruction;

[0165] The first training module 802 is used to input the sample image set into the initial scene classification model for training, and obtain the historical classification feature vector corresponding to each sample image;

[0166] A first determination module 803 is used to determine a first loss corresponding to each sample image according to a historical classification feature vector corresponding to each sample image, a scene dimension label corresponding to each sample image, and a cross entropy loss function with label smoothing; wherein the first loss includes losses corresponding to different scene dimensions;

[0167] The second training module 804 is used to adjust the first loss corresponding to each sample image according to a preset adjustment method to obtain a target loss, and train the initial scene classification model based on the target loss to obtain a target scene classification model.

[0168] In one embodiment, the first training module 802 is specifically used for:

[0169] Convolution processing is performed on each sample image to obtain a multi-scale local feature map corresponding to each sample image, and feature extraction is performed on each sample image based on the self-attention mechanism to obtain a global feature map corresponding to each sample image;

[0170] The multi-scale local feature map corresponding to each sample image is fused with the global feature map corresponding to each sample image to obtain a fused feature map corresponding to each sample image;

[0171] Upsample each fusion feature map respectively to obtain each target fusion feature map;

[0172] Each target fusion feature map is input into the fully connected layer in the initial scene classification model, and the historical classification feature vector corresponding to each sample image is output.

[0173] In one embodiment, the second training module 804 is specifically used for:

[0174] Based on the first loss corresponding to each sample image, a loss vector corresponding to each sample image is generated;

[0175] According to a preset selection method, K loss components corresponding to different scene dimensions are selected from the loss vector corresponding to each sample image; wherein K is an integer greater than 0;

[0176] According to the preset adjustment matrix, adaptive weight adjustment is performed on the K loss components corresponding to different scene dimensions to obtain K target loss components corresponding to different scene dimensions;

[0177] Based on the K target loss components corresponding to different scene dimensions, generate the second losses corresponding to different scene dimensions;

[0178] The second losses corresponding to different scene dimensions are balanced to obtain the target loss.

[0179] In one embodiment, the second training module 804 is further configured to:

[0180] Generate the loss contribution weights corresponding to the K loss components according to the preset adjustment matrix;

[0181] Normalize the loss contribution weights corresponding to the K loss components to obtain the target loss contribution weights corresponding to the K loss components;

[0182] Based on the target loss contribution weights of each of the K loss components, the K loss components are weighted to obtain K target loss components.

[0183] In one embodiment, the second training module 804 is further configured to:

[0184] According to the preset balance weights corresponding to the different scene dimensions, the second losses corresponding to the different scene dimensions are weighted to obtain the third losses corresponding to the different scene dimensions;

[0185] The average value of the third losses corresponding to different scene dimensions is calculated to obtain the target loss.

[0186] For the specific limitations of the scene classification model training device, please refer to the limitations of the scene classification model training method above, which will not be repeated here. Each module in the above-mentioned scene classification model training device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0187] In one embodiment, Fig. 9As shown, a cockpit environment adjustment device is provided, comprising: a first preprocessing module 801, a first training module 802, a first determination module 803 and a second training module 804, a second preprocessing module 901, a classification module 902, a second determination module 903 and an adjustment module 904, wherein:

[0188] The second preprocessing module 901 is used to preprocess the video data to be identified to obtain a target image set;

[0189] Classification module 902, used for inputting the target image set into the target scene classification model to obtain the target classification feature vector corresponding to each target image in the target image set;

[0190] The second determination module 903 is used to determine the target instruction corresponding to each target classification feature vector;

[0191] The adjustment module 904 is used to adjust the vehicle cabin environment based on the target instructions corresponding to each target classification feature vector.

[0192] In one embodiment, the second determining module 903 is specifically configured to:

[0193] When the scene dimension corresponding to the target classification feature vector is the environment category dimension, determining the target instruction to be the fragrance release instruction corresponding to the environment category dimension;

[0194] When the scene dimension corresponding to the target classification feature vector is the cooling and heating dimension, the target instruction is determined to be the air conditioning cooling and heating instruction corresponding to the cooling and heating dimension.

[0195] For the specific definition of the cabin environment adjustment device, please refer to the definition of the cabin environment adjustment method above, which will not be repeated here. Each module in the above-mentioned cabin environment adjustment device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0196] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store scene classification model training data and cabin environment adjustment data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a scene classification model training method and a cabin environment adjustment method are implemented.

[0197] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0198] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0199] Preprocess the historical video data to obtain a sample image set; wherein the sample image set includes scene dimension labels corresponding to the sample images, the scene dimension labels include environment category dimension labels and cold and warm dimension labels, the environment category dimension labels are associated with fragrance release instructions, and the cold and warm dimension labels are associated with air conditioning cold and warm wind instructions;

[0200] The sample image set is input into the initial scene classification model for training, and the historical classification feature vector corresponding to each sample image is obtained;

[0201] Determine the first loss corresponding to each sample image according to the historical classification feature vector corresponding to each sample image, the scene dimension label corresponding to each sample image, and the cross entropy loss function with label smoothing; wherein the first loss includes the losses corresponding to different scene dimensions;

[0202] The first loss corresponding to each sample image is adjusted according to a preset adjustment method to obtain a target loss, and the initial scene classification model is trained based on the target loss to obtain a target scene classification model.

[0203] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0204] Convolution processing is performed on each sample image to obtain a multi-scale local feature map corresponding to each sample image, and feature extraction is performed on each sample image based on the self-attention mechanism to obtain a global feature map corresponding to each sample image;

[0205] The multi-scale local feature map corresponding to each sample image is fused with the global feature map corresponding to each sample image to obtain a fused feature map corresponding to each sample image;

[0206] Upsample each fusion feature map respectively to obtain each target fusion feature map;

[0207] Each target fusion feature map is input into the fully connected layer in the initial scene classification model, and the historical classification feature vector corresponding to each sample image is output.

[0208] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0209] Based on the first loss corresponding to each sample image, a loss vector corresponding to each sample image is generated;

[0210] According to a preset selection method, K loss components corresponding to different scene dimensions are selected from the loss vector corresponding to each sample image; wherein K is an integer greater than 0;

[0211] According to the preset adjustment matrix, adaptive weight adjustment is performed on the K loss components corresponding to different scene dimensions to obtain K target loss components corresponding to different scene dimensions;

[0212] Based on the K target loss components corresponding to different scene dimensions, generate the second losses corresponding to different scene dimensions;

[0213] The second losses corresponding to different scene dimensions are balanced to obtain the target loss.

[0214] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0215] Generate the loss contribution weights corresponding to the K loss components according to the preset adjustment matrix;

[0216] Normalize the loss contribution weights corresponding to the K loss components to obtain the target loss contribution weights corresponding to the K loss components;

[0217] Based on the target loss contribution weights of each of the K loss components, the K loss components are weighted to obtain K target loss components.

[0218] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0219] According to the preset balance weights corresponding to the different scene dimensions, the second losses corresponding to the different scene dimensions are weighted to obtain the third losses corresponding to the different scene dimensions;

[0220] The average value of the third losses corresponding to different scene dimensions is calculated to obtain the target loss.

[0221] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0222] Preprocess the video data to be identified to obtain a target image set;

[0223] Input the target image set into the target scene classification model to obtain the target classification feature vector corresponding to each target image in the target image set;

[0224] Determine the target instruction corresponding to each target classification feature vector;

[0225] The vehicle cabin environment is adjusted based on the target instructions corresponding to each target classification feature vector.

[0226] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0227] When the scene dimension corresponding to the target classification feature vector is the environment category dimension, determining the target instruction to be the fragrance release instruction corresponding to the environment category dimension;

[0228] When the scene dimension corresponding to the target classification feature vector is the cooling and heating dimension, the target instruction is determined to be the air conditioning cooling and heating instruction corresponding to the cooling and heating dimension.

[0229] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0230] Preprocess the historical video data to obtain a sample image set; wherein the sample image set includes scene dimension labels corresponding to the sample images, the scene dimension labels include environment category dimension labels and cold and warm dimension labels, the environment category dimension labels are associated with fragrance release instructions, and the cold and warm dimension labels are associated with air conditioning cold and warm wind instructions;

[0231] The sample image set is input into the initial scene classification model for training, and the historical classification feature vector corresponding to each sample image is obtained;

[0232] Determine the first loss corresponding to each sample image according to the historical classification feature vector corresponding to each sample image, the scene dimension label corresponding to each sample image, and the cross entropy loss function with label smoothing; wherein the first loss includes the losses corresponding to different scene dimensions;

[0233] The first loss corresponding to each sample image is adjusted according to a preset adjustment method to obtain a target loss, and the initial scene classification model is trained based on the target loss to obtain a target scene classification model.

[0234] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0235] Convolution processing is performed on each sample image to obtain a multi-scale local feature map corresponding to each sample image, and feature extraction is performed on each sample image based on the self-attention mechanism to obtain a global feature map corresponding to each sample image;

[0236] The multi-scale local feature map corresponding to each sample image is fused with the global feature map corresponding to each sample image to obtain a fused feature map corresponding to each sample image;

[0237] Upsample each fusion feature map respectively to obtain each target fusion feature map;

[0238] Each target fusion feature map is input into the fully connected layer in the initial scene classification model, and the historical classification feature vector corresponding to each sample image is output.

[0239] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0240] Based on the first loss corresponding to each sample image, a loss vector corresponding to each sample image is generated;

[0241] According to a preset selection method, K loss components corresponding to different scene dimensions are selected from the loss vector corresponding to each sample image; wherein K is an integer greater than 0;

[0242] According to the preset adjustment matrix, adaptive weight adjustment is performed on the K loss components corresponding to different scene dimensions to obtain K target loss components corresponding to different scene dimensions;

[0243] Based on the K target loss components corresponding to different scene dimensions, generate the second losses corresponding to different scene dimensions;

[0244] The second losses corresponding to different scene dimensions are balanced to obtain the target loss.

[0245] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0246] Generate the loss contribution weights corresponding to the K loss components according to the preset adjustment matrix;

[0247] Normalize the loss contribution weights corresponding to the K loss components to obtain the target loss contribution weights corresponding to the K loss components;

[0248] Based on the target loss contribution weights of each of the K loss components, the K loss components are weighted to obtain K target loss components.

[0249] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0250] According to the preset balance weights corresponding to the different scene dimensions, the second losses corresponding to the different scene dimensions are weighted to obtain the third losses corresponding to the different scene dimensions;

[0251] The average value of the third losses corresponding to different scene dimensions is calculated to obtain the target loss.

[0252] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0253] Preprocess the video data to be identified to obtain a target image set;

[0254] Input the target image set into the target scene classification model to obtain the target classification feature vector corresponding to each target image in the target image set;

[0255] Determine the target instruction corresponding to each target classification feature vector;

[0256] The vehicle cabin environment is adjusted based on the target instructions corresponding to each target classification feature vector.

[0257] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0258] When the scene dimension corresponding to the target classification feature vector is the environment category dimension, determining the target instruction to be the fragrance release instruction corresponding to the environment category dimension;

[0259] When the scene dimension corresponding to the target classification feature vector is the cooling and heating dimension, the target instruction is determined to be the air conditioning cooling and heating instruction corresponding to the cooling and heating dimension.

[0260] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0261] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0262] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A scene classification model training method, characterized in that: The method comprises: Preprocess the historical video data to obtain a sample image set; wherein the sample image set includes scene dimension labels corresponding to the sample images, the scene dimension labels include environment category dimension labels and cold and warm dimension labels, the environment category dimension labels are associated with fragrance release instructions, and the cold and warm dimension labels are associated with air conditioning cold and warm wind instructions; Inputting the sample image set into an initial scene classification model for training to obtain a historical classification feature vector corresponding to each of the sample images; Determine a first loss corresponding to each of the sample images according to a historical classification feature vector corresponding to each of the sample images, a scene dimension label corresponding to each of the sample images, and a cross entropy loss function with label smoothing; wherein the first loss includes losses corresponding to different scene dimensions; The first loss corresponding to each of the sample images is adjusted according to a preset adjustment method to obtain a target loss, and the initial scene classification model is trained based on the target loss to obtain a target scene classification model.

2. The method according to claim 1, characterized in that The step of inputting the sample image set into the initial scene classification model for training to obtain the historical classification feature vector corresponding to each sample image comprises: Performing convolution processing on each of the sample images to obtain a multi-scale local feature map corresponding to each of the sample images, and performing feature extraction on each of the sample images based on a self-attention mechanism to obtain a global feature map corresponding to each of the sample images; Fusing the multi-scale local feature map corresponding to each of the sample images with the global feature map corresponding to each of the sample images to obtain a fused feature map corresponding to each of the sample images; Performing upsampling processing on each of the fused feature maps respectively to obtain each target fused feature map; Each of the target fusion feature maps is input into the fully connected layer in the initial scene classification model, and the historical classification feature vector corresponding to each of the sample images is output.

3. The method according to claim 1, characterized in that The step of adjusting the first loss corresponding to each of the sample images according to a preset adjustment method to obtain a target loss includes: Based on the first loss corresponding to each of the sample images, generating a loss vector corresponding to each of the sample images; According to a preset selection method, K loss components corresponding to different scene dimensions are selected from the loss vectors corresponding to each of the sample images; wherein K is an integer greater than 0; According to the preset adjustment matrix, adaptive weight adjustment is performed on the K loss components corresponding to different scene dimensions to obtain K target loss components corresponding to different scene dimensions; Based on the K target loss components corresponding to different scene dimensions, generate the second losses corresponding to different scene dimensions; Loss balancing is performed on the second losses corresponding to different scene dimensions to obtain the target loss.

4. The method according to claim 3, characterized in that According to the preset adjustment matrix, adaptive weight adjustment is performed on the K loss components corresponding to different scene dimensions to obtain K target loss components corresponding to different scene dimensions, including: Generate loss contribution weights corresponding to the K loss components according to a preset adjustment matrix; Normalizing the loss contribution weights corresponding to the K loss components to obtain target loss contribution weights corresponding to the K loss components; Based on the target loss contribution weights of each of the K loss components, the K loss components are weighted to obtain K target loss components.

5. The method according to claim 3, characterized in that: The loss balancing process is performed on the second losses corresponding to different scene dimensions to obtain the target loss, including: According to the preset balance weights corresponding to the different scene dimensions, the second losses corresponding to the different scene dimensions are weighted to obtain the third losses corresponding to the different scene dimensions; The target loss is obtained by calculating the average value of the third losses corresponding to different scene dimensions.

6. A method for adjusting a cabin environment, using the target scene classification model according to any one of claims 1 to 5, characterized in that: The method comprises: Preprocess the video data to be identified to obtain a target image set; Inputting the target image set into the target scene classification model to obtain a target classification feature vector corresponding to each target image in the target image set; Determine the target instruction corresponding to each of the target classification feature vectors; Based on the target instructions corresponding to each of the target classification feature vectors, the vehicle cabin environment is adjusted.

7. The method according to claim 6, characterized in that The step of determining the target instruction corresponding to each of the target classification feature vectors comprises: When the scene dimension corresponding to the target classification feature vector is an environment category dimension, determining that the target instruction is a fragrance release instruction corresponding to the environment category dimension; When the scene dimension corresponding to the target classification feature vector is a cooling and heating dimension, the target instruction is determined to be an air conditioning cooling and heating instruction corresponding to the cooling and heating dimension.

8. A scene classification model training device, characterized in that: The device comprises: A first preprocessing module is used to preprocess the historical video data to obtain a sample image set; wherein the sample image set includes a scene dimension label corresponding to the sample image, and the scene dimension label includes an environment category dimension label and a cold and warm dimension label, the environment category dimension label is associated with a fragrance release instruction, and the cold and warm dimension label is associated with an air conditioning cold and warm wind instruction; A first training module is used to input the sample image set into an initial scene classification model for training to obtain a historical classification feature vector corresponding to each of the sample images; A first determination module is used to determine a first loss corresponding to each of the sample images according to a historical classification feature vector corresponding to each of the sample images, a scene dimension label corresponding to each of the sample images, and a cross entropy loss function with label smoothing; wherein the first loss includes losses corresponding to different scene dimensions; The second training module is used to adjust the first loss corresponding to each of the sample images according to a preset adjustment method to obtain a target loss, and train the initial scene classification model based on the target loss to obtain a target scene classification model.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images