Vehicle-mounted video image classification method and device, storage medium and electronic device

Abstract

The application discloses a vehicle-mounted video image classification method and device, a storage medium and an electronic device, and the technical key points are as follows: a sample video file is read, and a video frame image is extracted; image preprocessing is performed on the video frame image, wherein the image preprocessing comprises removing image noise through Gaussian blurring, converting an RGB image into a gray-scale image through image graying, and identifying actual edges in the image through edge detection; dimension reduction processing is performed on the extracted image features using an ILTSA algorithm, the feature dimension is reduced, and redundant features are removed; a label distribution learning algorithm is used to learn the mapping relationship between vehicle-mounted video image features and image labels, a vehicle-mounted video image classification model is constructed and trained; and after the vehicle-mounted video image classification model is trained, a target video file is input to predict the vehicle-mounted video image category, so that the technical effect of efficiently and accurately classifying vehicle-mounted video images is achieved.

Description

Technical Field

[0001] This invention relates to the field of image classification, and more particularly to a method, apparatus, storage medium, and electronic device for classifying in-vehicle video images. Background Technology

[0002] With rapid economic development, people's quality of life is constantly improving. Motor vehicles, as a modern means of transportation reflecting this quality of life, provide immense convenience. The frequency of driving and vehicle use is gradually increasing, leading to negative social phenomena such as frequent road traffic accidents. To reduce the number of road traffic accidents and ultimately solve this major problem, countries are vigorously developing Intelligent Transportation Systems (ITS). Autonomous driving assistance systems are an important component of ITS. Autonomous driving is bringing about a new revolution in transportation systems. Therefore, researching and developing vehicle detection technology has a significant and far-reaching impact on maintaining smooth traffic flow and preventing traffic accidents. For the various complex scenarios that occur during autonomous driving, if information from onboard video can be comprehensively mined and fed back to the driver or the autonomous driving controller with key information such as scene, weather, lane markings, or the presence of pedestrians, and this information is used to determine whether the road ahead is safe to drive, it will provide a strong guarantee for driver safety and autonomous driving.

[0003] Traditional single-label classification methods struggle to accurately describe all the information in a video. In recent years, multi-label classification techniques have been widely used to learn semantically rich objects. However, existing multi-label learning methods all assume that each label is equally important. In practice, this assumption is often difficult to hold. For example, among the labels in a driving video, the label "Is there a pedestrian ahead?" should be the most important, while the label "Trees on both sides of the road" is not very relevant to driving.

[0004] Label distribution learning, as a novel learning paradigm, is a generalization of multi-label learning. It transforms the logical labels of traditional multi-label learning into a label distribution, reflecting the different importance levels of various labels. This is more conducive to uncovering the intrinsic relationships between labels, leading to more accurate classification results. Label distribution learning can analyze different labels in vehicle video images, extracting key information while reducing the influence of confounding factors, thus improving road condition assessment.

[0005] Therefore, label distribution learning has significant application value in the field of vehicle video image classification, addressing the problems of low efficiency, low accuracy, and poor robustness of existing technologies. Summary of the Invention

[0006] Exemplary embodiments of this application provide a vehicle-mounted video image classification method, apparatus, storage medium, and electronic device to at least achieve the technical effect of efficient and accurate vehicle-mounted video image classification by enhancing traditional logical labels through label distribution learning techniques based on pre-training using a large-scale open driving video dataset.

[0007] This application provides an exemplary embodiment of a vehicle-mounted video image classification method, the method comprising the following steps:

[0008] Read the sample video file and extract the video frame images;

[0009] The video frame image is preprocessed, wherein the image preprocessing includes Gaussian blur to remove image noise, image grayscale conversion to convert RGB image to grayscale image, and edge detection to identify the actual edges in the image;

[0010] The extracted image features are then subjected to dimensionality reduction using the ILTSA algorithm to reduce feature dimensions and remove redundant features.

[0011] The fusion label distribution learning algorithm learns the mapping relationship between vehicle video image features and image labels, and constructs and trains a vehicle video image classification model;

[0012] After the vehicle-mounted video image classification model is trained, the target video file is input to predict the vehicle-mounted video image category.

[0013] In another aspect of this application, an in-vehicle video image classification device is also provided, the device comprising:

[0014] The image acquisition module is configured to read sample video files and extract video frame images;

[0015] The image preprocessing module is configured to perform image preprocessing on the video frame image, wherein the image preprocessing includes Gaussian blur to remove image noise, image grayscale conversion to convert RGB image to grayscale image, and edge detection to identify the actual edges in the image;

[0016] The dimension reduction module is configured to use the ILTSA algorithm to reduce the dimension of the extracted image features, thereby reducing the feature dimensions and removing redundant features.

[0017] The training module is configured to use a fusion label distribution learning algorithm to learn the mapping relationship between vehicle video image features and image labels, and to build and train a vehicle video image classification model.

[0018] The classification module is configured to predict the category of the vehicle video image by inputting a target video file after the vehicle video image classification model has been trained.

[0019] In another aspect of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps of any of the above method embodiments at runtime.

[0020] In another aspect of this application, an electronic device is also proposed, comprising a memory and a processor, characterized in that the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0021] Beneficial effects:

[0022] The technical solution of this invention focuses on solving the problems of low efficiency, low accuracy, and poor robustness in the prior art. It enhances traditional logical labels through label distribution learning technology, thereby achieving efficient and accurate vehicle video image classification; and effectively realizes vehicle video image classification even under conditions of label ambiguity. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0024] Figure 2 This is a schematic diagram of the process of the present invention;

[0025] Figure 3 This is a comparison chart of the effects of the embodiments of the present invention under the Chebyshev distance index;

[0026] Figure 4 This is a comparison chart of the effects of the embodiments of the present invention under the Clark distance index;

[0027] Figure 5 This is a comparison chart of the effects of the embodiments of the present invention under the Canberra index;

[0028] Figure 6 This is a comparison chart of the effects of the embodiments of the present invention under the cosine distance index;

[0029] Figure 7 This is a comparison chart of the effects of the embodiments of the present invention under the intersection index;

[0030] Figure 8 This is a schematic diagram of an optional electronic device structure according to this application. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0032] In one embodiment, such as Figures 1 to 2 As shown, this invention proposes a vehicle-mounted video image classification method, characterized in that the method includes:

[0033] S1, Read the sample video file and extract the video frame images;

[0034] S2, perform image preprocessing on the video frame image, wherein the image preprocessing includes Gaussian blur to remove image noise, image grayscale conversion to convert RGB image to grayscale image, and edge detection to identify the actual edges in the image;

[0035] S3. The extracted image features are processed using the ILTSA algorithm to reduce the feature dimension and remove redundant features.

[0036] S4, using a fusion label distribution learning algorithm to learn the mapping relationship between vehicle video image features and image labels, and to build and train a vehicle video image classification model;

[0037] S5. After the vehicle-mounted video image classification model has been trained, the target video file is input to predict the vehicle-mounted video image category.

[0038] In one embodiment, the image preprocessing in step S2 includes removing image noise using Gaussian blur, where the Gaussian kernel size is set to 3, the standard deviation of the Gaussian function is set to 1.5, and the Gaussian function used is...

[0039]

[0040] For a 3×3 pixel, the Gaussian blur process is as follows:

[0041] Assuming the center point's coordinates are (0,0), then the coordinates of the remaining points are:

[0042]

[0043] The weight matrix obtained by substituting the coordinates of each pixel into the Gaussian function to calculate the weight of each pixel is as follows:

[0044]

[0045] The value of the Gaussian blur is obtained by multiplying each point by its own weight.

[0046]

[0047] Finally, the nine values ​​are summed to obtain the Gaussian blur value of the center point. This process is repeated for all points in the RGB channels to obtain the Gaussian blurred image.

[0048] In one embodiment, the step of converting the RGB image to a grayscale image in step S2 above specifically includes the following:

[0049] The following formula is used for grayscale processing:

[0050]

[0051] Where X, Y, and Z are preset empirical values ​​[0.299, 0.587, 0.114] to preserve the brightness information of the image.

[0052] In one embodiment, the step of identifying actual edges in the image by edge detection in step S2 above specifically includes the following:

[0053] The gradient is calculated using a Sobel filter. The filter along the horizontal direction is denoted as H(x), and the filter along the vertical direction is denoted as V(x). The definitions of the two filters are as follows:

[0054]

[0055] For a given image, the formulas for calculating orientation and gradient are as follows:

[0056]

[0057]

[0058] Among them, G X and G Y These represent the gradients of the image along the horizontal and vertical directions, respectively. The calculation process is as follows: Indicates the convolution operation;

[0059] The maximum gradient value at each pixel is retained by comparing the gradient of each pixel with the gradient values ​​of pixels in the positive and negative directions of its gradient direction. If the gradient of a pixel is the maximum, it is retained; otherwise, it is set to 0. The calculation formula is as follows:

[0060]

[0061] Double thresholds are used to determine image edges. Specifically, an upper threshold (maxval) and a lower threshold (minval) are set. For a pixel in the image, if it is greater than the upper threshold, it is considered to be an edge; if it is less than the lower threshold, it is considered not to be an edge. For a pixel between the upper and lower thresholds, if it is adjacent to a pixel that is definitely an edge, it is considered to be an edge; otherwise, it is not considered to be an edge.

[0062] In one embodiment, the step S3 above, which involves using the ILTSA algorithm to reduce the dimensionality of the extracted image features, thereby reducing feature dimensions and removing redundant features, specifically includes the following:

[0063] The preprocessed m-dimensional feature matrix X = (x1, x2, ..., x...) N ), based on Euclidean distance for each point x i Determine the k nearest neighbors and form a matrix.

[0064] Iterative construction of covariance matrix C N , representing the incremental tangent space for each data point, is given by the following formula:

[0065]

[0066] For the newly added sample point x new The covariance matrix is ​​updated using the following formula.

[0067]

[0068] M new =αM N +(1-α)x new e T

[0069] For the covariance matrix C new Perform eigenvalue decomposition; based on the covariance matrix C new The first d largest eigenvalues ​​λ i and eigenvector v i Construct matrices A and B, and calculate as follows:

[0070] A = (y1, y2, ..., y3) d+1 )

[0071] B = AA T

[0072] in

[0073] Calculate the eigenvectors u1, u2, ..., u3 corresponding to the first d smallest eigenvalues ​​of matrix B. d

[0074] Calculate the newly added sample point x new The local coordinates are given by the following formula:

[0075]

[0076] x new Let λ represent the newly added sample point, and λ and u represent the eigenvalue and eigenvector, respectively. This represents the mean of adjacent points.

[0077] The eigenvectors corresponding to the first d smallest eigenvalues ​​of B constitute Let R i =IW i W i T Calculate R i R i T The eigenvectors corresponding to the first d smallest eigenvalues ​​are the features obtained by dimensionality reduction using the ILTSA algorithm.

[0078] In one embodiment, the step S4, in which the fusion label distribution learning algorithm learns the mapping relationship between vehicle video image features and image labels, and constructs and trains the vehicle video image classification model, specifically includes the following:

[0079] Videos were selected from the BDD100K open driving video dataset, and video frame images were extracted using OpenCV. The images were then preprocessed, and feature matrices were obtained through dimensionality reduction of the image features.

[0080] K-means clustering is performed on the feature matrix Z. After obtaining the categories through clustering, the features and categories are integrated into a dataset vehicle for training a labeled distribution learning model.

[0081] Dataset augmentation is performed using a label propagation algorithm;

[0082] Based on image features and label distribution information, a maximum entropy model is trained to predict the label distribution of unknown examples. The model parameters are:

[0083]

[0084] in, Let p(y) represent the true labeled distribution. j |x i ;θ) represents the label distribution predicted by the model, θ * This represents the model parameters obtained through training.

[0085] In one embodiment, the label augmentation of the dataset using the label propagation algorithm in step S4 specifically includes the following:

[0086] First, construct a similarity matrix A = [a] based on the features. ij ] N×N The calculation method is as follows:

[0087]

[0088] Calculate the mark propagation matrix Where D is a diagonal matrix.

[0089] A descriptive matrix F is constructed by combining the descriptive values ​​of all tags for each example. F is then iteratively updated using the label propagation algorithm, as shown in the following formula:

[0090] F (t) =αPF (t-1) +(1-α)Y

[0091] Where α is the balance parameter and Y is the logical label.

[0092] After iteration, the label distribution of the examples is obtained through normalization.

[0093]

[0094] in, Represents the predicted label distribution. This indicates the descriptive power of label j for sample i.

[0095] In this embodiment, five indicators are used to evaluate the model. As an example, the proportion of the training set is 80% during the model training process. To avoid experimental errors caused by random sampling, the experimental results are the average of ten five-fold cross-validations.

[0096] Based on the above scheme, in order to verify the effectiveness of the method of the present invention, this embodiment compares CLP-IIS with other label distribution learning methods on a dataset. The specific experimental results are as follows: Figures 3-8 As shown, the smaller the value of the first four evaluation indicators, the better the effect; the larger the value of the last evaluation indicator, the better the effect.

[0097] According to another aspect of the embodiments of this application, an electronic device for a multimodal large-scale language model training method based on adversarial distillation technology is also provided. This electronic device can be, but is not limited to, applied in a server. Figure 6 As shown, the electronic device includes a memory 902 and a processor 904. The memory 902 stores a computer program, and the processor 904 is configured to execute the steps of any of the above method embodiments via the computer program.

[0098] Optionally, in this embodiment, the electronic device may be located in at least one of a plurality of network devices in a computer network.

[0099] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0100] S1, Read the sample video file and extract the video frame images;

[0101] S2, perform image preprocessing on the video frame image, wherein the image preprocessing includes Gaussian blur to remove image noise, image grayscale conversion to convert RGB image to grayscale image, and edge detection to identify the actual edges in the image;

[0102] S3. The extracted image features are processed using the ILTSA algorithm to reduce the feature dimension and remove redundant features.

[0103] S4, using a fusion label distribution learning algorithm to learn the mapping relationship between vehicle video image features and image labels, and to build and train a vehicle video image classification model;

[0104] S5. After the vehicle-mounted video image classification model has been trained, the target video file is input to predict the vehicle-mounted video image category.

[0105] Alternatively, as those skilled in the art will understand, Figure 8 The structure shown is for illustrative purposes only; the electronic device may also be a mobile internet device (MID), a PAD, or other terminal device. Figure 8 This does not limit the structure of the aforementioned electronic device. For example, the electronic device may also include components that are more... Figure 8 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 8 The different configurations shown.

[0106] The memory 902 may include high-speed random access memory (RAM) and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 902 may further include memory remotely located relative to the processor 904, which can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks (LANs), mobile communication networks, and combinations thereof. As an example, such as... Figure 8 As shown, the memory 902 may include, but is not limited to, an image preprocessing module, etc., which will not be described in detail in this example.

[0107] Optionally, the transmission device 906 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 906 includes a Network Interface Controller (NIC), which can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 906 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0108] In addition, the aforementioned electronic device also includes: a display 908 for displaying the current simulation results; and a connection bus 910 for connecting the various module components in the aforementioned electronic device.

[0109] Embodiments of this application also provide a computer-readable storage medium storing a computer program configured to execute the steps in any of the above method embodiments when run.

[0110] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0111] S1, Read the sample video file and extract the video frame images;

[0112] S2, perform image preprocessing on the video frame image, wherein the image preprocessing includes Gaussian blur to remove image noise, image grayscale conversion to convert RGB image to grayscale image, and edge detection to identify the actual edges in the image;

[0113] S3. The extracted image features are processed using the ILTSA algorithm to reduce the feature dimension and remove redundant features.

[0114] S4, using a fusion label distribution learning algorithm to learn the mapping relationship between vehicle video image features and image labels, and to build and train a vehicle video image classification model;

[0115] S5. After the vehicle-mounted video image classification model has been trained, the target video file is input to predict the vehicle-mounted video image category.

[0116] Optionally, the storage medium is also configured to store a computer program for performing the steps included in the methods of the above embodiments, which will not be described again in this embodiment.

[0117] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0118] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0119] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0120] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0122] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0123] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0124] The above description is merely a preferred embodiment of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

[0125] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications based on these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of this application.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0127] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for classifying vehicle-mounted video images, characterized in that, The method includes: Read the sample video file and extract the video frame images; The video frame image is preprocessed, wherein the image preprocessing includes Gaussian blur to remove image noise, image grayscale conversion to convert RGB image to grayscale image, and edge detection to identify the actual edges in the image; The preprocessed image features are then subjected to dimensionality reduction using the ILTSA algorithm to reduce feature dimensions and remove redundant features; specifically, this includes: The m-dimensional feature matrix obtained through preprocessing For each point, the Euclidean distance is used. Determine the k nearest neighbors and form a matrix. , Iterative construction of covariance matrix , representing the incremental tangent space for each data point, is given by the following formula: For newly added sample points The covariance matrix is ​​updated using the following formula. For covariance matrix Perform eigenvalue decomposition; based on the covariance matrix The top d largest eigenvalues and eigenvectors Construct matrices A and B, and calculate as follows: Calculate the eigenvectors corresponding to the first d smallest eigenvalues ​​of matrix B. Calculate newly added sample points The local coordinates are given by the following formula: This indicates newly added sample points. and These represent eigenvalues ​​and eigenvectors, respectively. This represents the mean of adjacent points; The eigenvectors corresponding to the first d smallest eigenvalues ​​of B constitute ,make ,calculate The eigenvectors corresponding to the first d smallest eigenvalues ​​are the features obtained by dimensionality reduction using the ILTSA algorithm. ; A fusion-label distribution learning algorithm is used to learn the mapping relationship between in-vehicle video image features and image labels, and to construct and train an in-vehicle video image classification model; specifically, this includes: Videos were selected from the BDD100K open driving video dataset, and video frame images were extracted using OpenCV. The images were then preprocessed, and feature matrices were obtained through dimensionality reduction of the image features. For the characteristic matrix Perform K-means clustering, obtain the categories through clustering, and then integrate the features and categories as a dataset vehicle for training the label distribution learning model; Dataset augmentation is performed using a label propagation algorithm; Based on image features and label distribution information, a maximum entropy model is trained to predict the label distribution of unknown examples. The model parameters are: in, Represents the true labeled distribution. This represents the label distribution predicted by the model. This represents the model parameters obtained through training; After the vehicle-mounted video image classification model is trained, the target video file is input to predict the vehicle-mounted video image category.

2. The vehicle-mounted video image classification method according to claim 1, characterized in that, The step of converting an RGB image to a grayscale image specifically includes the following: The following formula is used for grayscale processing: in , , The preset empirical values ​​[0.299, 0.587, 0.114] are used to preserve the brightness information of the image.

3. The vehicle-mounted video image classification method according to claim 2, characterized in that, The step of detecting and identifying actual edges in the image specifically includes the following: The gradient is calculated using a Sobel filter; the filter along the horizontal direction is... The filter along the vertical direction is The two filters are defined as follows: For a given image, the formulas for calculating orientation and gradient are as follows: in, and These represent the gradients of the image along the horizontal and vertical directions, respectively. The calculation process is as follows: , Indicates the convolution operation; The maximum gradient value at each pixel is retained by comparing the gradient of each pixel with the gradient values ​​of pixels in the positive and negative directions of its gradient direction. If the gradient of a pixel is the maximum, it is retained; otherwise, it is set to 0. The calculation formula is as follows: ， Setting a dual threshold determines image edges; specifically, it involves setting an upper bound for the threshold. and lower bound of threshold For a pixel in an image, if it is greater than the upper threshold, it is considered to be an edge; if it is less than the lower threshold, it is considered not to be an edge. For a pixel between the upper and lower thresholds, if it is adjacent to a pixel that is definitely an edge, it is considered to be an edge; otherwise, it is not considered to be an edge.

4. The vehicle-mounted video image classification method according to claim 3, characterized in that, The pair of feature matrices The steps for performing K-means clustering are as follows: K centers are randomly selected, denoted as Kcenter. K is set to 10 Define loss function Set the maximum number of iterations to 500, and repeat the following process until... Convergence or reaching the maximum number of iterations For each sample Assign it to the nearest center Recalculate the class center for each class. in, Indicates a sample, This represents the center point of the cluster.

5. The vehicle-mounted video image classification method according to claim 4, characterized in that, The labeling enhancement of the dataset using the label propagation algorithm specifically includes the following: First, construct a similarity matrix based on the features. The calculation method is as follows: Calculate the mark propagation matrix Where D is a diagonal matrix, Construct a descriptive matrix from the descriptive values ​​of all tags for the examples. Iterative updates using the mark-propagation algorithm The formula is as follows: in, It is a balance parameter. It is a logical marker; After iteration, the label distribution of the examples is obtained through normalization. in, Represents the predicted label distribution. Indicates a marker For the sample The descriptiveness.

6. A vehicle-mounted video image classification device, characterized in that, The device includes: The image acquisition module is configured to read sample video files and extract video frame images; The image preprocessing module is configured to perform image preprocessing on the video frame image, wherein the image preprocessing includes Gaussian blur to remove image noise, image grayscale conversion to convert RGB image to grayscale image, and edge detection to identify the actual edges in the image; The dimensionality reduction module is configured to use the ILTSA algorithm to reduce the dimensionality of the preprocessed image features, thereby reducing the feature dimension and removing redundant features. Specifically, this includes: processing the preprocessed m-dimensional feature matrix... For each point, the Euclidean distance is used. Determine the k nearest neighbors and form a matrix. , Iterative construction of covariance matrix , representing the incremental tangent space for each data point, is given by the following formula: For newly added sample points The covariance matrix is ​​updated using the following formula. For covariance matrix Perform eigenvalue decomposition; based on the covariance matrix The top d largest eigenvalues and eigenvectors Construct matrices A and B, and calculate as follows: Calculate the eigenvectors corresponding to the first d smallest eigenvalues ​​of matrix B. Calculate newly added sample points The local coordinates are given by the following formula: This indicates newly added sample points. and These represent eigenvalues ​​and eigenvectors, respectively. This represents the mean of adjacent points; The eigenvectors corresponding to the first d smallest eigenvalues ​​of B constitute ,make ,calculate The eigenvectors corresponding to the first d smallest eigenvalues ​​are the features obtained by dimensionality reduction using the ILTSA algorithm. ; The training module is configured to use a fusion-label distribution learning algorithm to learn the mapping relationship between in-vehicle video image features and image labels, and to build and train an in-vehicle video image classification model; specifically, it includes: Videos were selected from the BDD100K open driving video dataset, and video frame images were extracted using OpenCV. The images were then preprocessed, and feature matrices were obtained through dimensionality reduction of the image features. For the characteristic matrix Perform K-means clustering, obtain the categories through clustering, and then integrate the features and categories as a dataset vehicle for training the label distribution learning model; Dataset augmentation is performed using a label propagation algorithm; Based on image features and label distribution information, a maximum entropy model is trained to predict the label distribution of unknown examples. The model parameters are: in, Represents the true labeled distribution. This represents the label distribution predicted by the model. This represents the model parameters obtained through training; The classification module is configured to input a target video file to predict the category of the vehicle video image after the vehicle video image classification model has been trained.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the method described in any one of claims 1 to 5 when it is run.

8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method as described in any one of claims 1 to 5.

Images