Automatic driving obstacle recognition method, device, electronic device and storage medium

Abstract

The present application discloses an obstacle recognition method, device, electronic device and storage medium for autonomous driving. The method includes: based on visual recognition, collecting image data and annotating the data; generating image samples based on a visual simulation generative adversarial network algorithm to expand image data; inputting the expanded data into a feature extraction model to train the feature extraction model; inputting the feature extracted data into a feature dimension reduction model to train the feature dimension reduction model; inputting the dimension reduction data into a classifier to train the classifier; performing visual obstacle recognition based on the trained feature extraction model, data dimension reduction model and classifier model. Through the above scheme, the image generated by the generative adversarial network and the feature enhancement loss function are generated to improve the recognition ability of the model on unseen obstacle images.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving, and in particular to an autonomous driving obstacle recognition method, an autonomous driving obstacle recognition device, an electronic device, a storage medium and a vehicle. Background Art

[0002] Traditional obstacle recognition systems mainly rely on rule-based algorithms and limited sensor data, which often have limitations when dealing with complex and dynamic driving environments. In particular, under changing road conditions, different lighting and weather conditions, traditional technologies have difficulty accurately and timely identifying various obstacles, such as pedestrians, animals, vehicles, and static obstacles, which limits the widespread deployment and safety performance of autonomous vehicles. In addition, existing obstacle recognition systems usually face the problem of insufficient data, and manually annotating large amounts of high-quality image data is time-consuming and expensive. In addition, these systems often lack effective data expansion methods and find it difficult to simulate the complexity of the real world, resulting in weak generalization of the model in real scenarios. At the same time, traditional techniques such as parameter optimization methods such as gradient descent often lead to low training efficiency and unstable model performance due to initial value dependence or gradient problems (vanishing or exploding).

[0003] The Chinese invention patent with application number CN202410362956.X proposes an airport clearance obstacle measurement method and device based on image recognition, which involves the field of airport clearance patrol technology. It combines the environmental feature point database and image recognition algorithm to enable dual cameras to perform high-precision measurements of long-distance obstacles, thereby meeting the airport clearance patrol needs.

[0004] The Chinese invention patent with application number CN202410374077.9 proposes a method for identifying obstacles in unmanned ships. The laser radar point cloud data of the area where the unmanned ship is located is obtained through a laser radar, and the laser radar point cloud data is transformed into a radar point cloud in the coordinate system where the unmanned ship is located. The unmanned ship is controlled to operate autonomously according to the obstacle point cloud.

[0005] The Chinese invention patent with application number CN202410078558.5 proposes a method and device for detecting dynamic obstacles in automatic parking. The method obtains dynamic obstacle information, the current vehicle's driving direction and turning radius; determines the driving state according to the turning radius and the current vehicle's driving direction; classifies scenes based on the driving state, and determines whether there is an intrusion risk of the dynamic obstacle to the current vehicle based on the scene classification results, the coordinates of the current vehicle, and the coordinates of the dynamic obstacle.

[0006] Publicly available comparative documents cannot solve the following problems in the current field of autonomous driving:

[0007] 1. Data augmentation often relies on simple image transformation or limited sample replication, lacking the ability to widely cover the diversity of the real world;

[0008] 2. Neural network training usually relies on gradient descent algorithms, which may fall into local optimality. For non-convex optimization problems, they are particularly susceptible to the impact of initial values ​​and gradient disappearance / explosion;

[0009] 3. When processing high-dimensional data, it may lead to a large consumption of computing resources and easy overfitting;

[0010] 4. In classification tasks, especially in the case of limited samples or unbalanced data sets, the performance of neural networks is limited, and the training process is time-consuming;

[0011] Design the autonomous driving obstacle recognition solution for this application. Summary of the invention

[0012] The purpose of the present invention is to provide an autonomous driving obstacle recognition method, an autonomous driving obstacle recognition device, an electronic device, a storage medium and a vehicle, which at least solve the problems of data expansion, susceptibility to the influence of initial values ​​and gradient disappearance / explosion, large consumption of computing resources and easy overfitting in processing high-dimensional data, and limited performance improvement of neural networks in classification tasks and long training process time in the current autonomous driving field.

[0013] The present invention provides the following scheme:

[0014] According to one aspect of the present invention, there is provided an automatic driving obstacle recognition method, characterized in that the automatic driving obstacle recognition method comprises:

[0015] Step S1, based on visual recognition, collecting image data and annotating the data;

[0016] Step S2, generating image samples based on a visual simulation generative adversarial network algorithm to expand image data;

[0017] Step S3, inputting the expanded data into a feature extraction model to train the feature extraction model;

[0018] Step S4, inputting the feature-extracted data into a feature dimensionality reduction model to train the feature dimensionality reduction model;

[0019] Step S5, inputting the dimension-reduced data into a classifier to train the classifier;

[0020] Step S6, performing vision-based obstacle recognition according to the trained feature extraction model, data dimension reduction model and classifier model.

[0021] Furthermore, the step S1 includes:

[0022] The collecting of image data and labeling of the data includes collecting image data from a vehicle-mounted camera;

[0023] Marking obstacles in the image data according to the image data of the vehicle-mounted camera;

[0024] The marking of obstacles in the image data includes marking the categories of the obstacles.

[0025] Further, the step S2 includes that the visual simulation generative adversarial network algorithm includes a generator and a discriminator;

[0026] Step S201, initializing the parameters of the generator and the discriminator;

[0027] Step S202, the generator selects samples from the image data with obstacles marked, for training the discriminator to identify the authenticity of obstacles in the image data;

[0028] Step S203, the discriminator receives the image data from the vehicle-mounted camera and the image sample data generated by the generative adversarial network algorithm, and performs discriminator training;

[0029] Step S204, after the discriminator is trained, the generator updates parameters according to the feedback from the discriminator;

[0030] Step S205, obtaining a feature enhancement loss function, inputting the current update parameters and continuously iterating;

[0031] After each parameter update iteration, the parameters of the generator are adjusted according to the result of the feature enhancement loss function.

[0032] Further, the step S3 includes obtaining an adaptive vortex and optimizing a neural network;

[0033] Step S301, randomly generate a group of vortices in the parameter space, each vortex represents a possible parameter configuration;

[0034] Step S302, calculating the performance of the parameter configuration represented by each vortex on the training set, and sorting the vortices according to the performance;

[0035] Step S303, dynamically updating the vortexes, dynamically adjusting the positions and intensities of the vortices according to the rankings of the vortices and the distances between them, and simulating the interaction and evolution of the vortices;

[0036] Step S304, simulating the merging and splitting phenomena of vortices in nature to promote the diversity of the solution space and explore new parameter regions;

[0037] Step S305, sorting the vortices according to the performance, selecting the vortex with the best performance, performing local search for fine adjustment of parameters, and simulating the local absorption and energy concentration phenomenon of the vortex in the fluid;

[0038] Step S306, performing vortex attenuation to simulate the dissipation process of vortex energy in nature, gradually eliminating vortices with poor performance, and releasing resources for newly generated or well-performing vortices;

[0039] Step S307, repeat the above steps S301 to S306 until the preset stop iteration condition is met, and the training of the feature extraction model is completed.

[0040] Further, the step S4 includes:

[0041] Step S401, randomly initializing the weights of the encoder and decoder according to a preset distribution;

[0042] Step S402, in a forward propagation manner, the encoder converts the input data into a low-dimensional feature representation, and the data represented as the low-dimensional feature is passed to the decoder to attempt to reconstruct the original data;

[0043] Step S403, calculating the total loss based on the mean square error of L2 regularization to avoid overfitting and ensure that the model can learn effective data compression, including obtaining a loss function;

[0044] Step S404, in the process of back propagation, updating the network weights by using a gradient descent algorithm according to the loss function;

[0045] Step S405, adaptively adjusting the learning rate according to the training progress, and using an exponential decay strategy to optimize the learning process;

[0046] Step S406, repeat the above steps S401 to S405 until the preset stop iteration condition is met, and the training of the feature dimensionality reduction model is completed.

[0047] Furthermore, the step S5 comprises:

[0048] Step S501, randomly initializing the weights and biases of the hidden layer of the extreme learning machine;

[0049] Step S502, setting the initial temperature parameters to control the annealing rate during the training process;

[0050] Step S503, the input data passes through the hidden layer and is processed using the Sigmoid function nonlinear activation function to generate the output of the intermediate layer;

[0051] Step S504, using a deterministic annealing strategy to optimize the solution of the output weights, in each annealing step, adjusting the solution process according to the current temperature, gradually reducing the temperature, and optimizing the weight of the output layer;

[0052] Step S505, dynamically adjusting the activation state of the hidden layer neurons according to the set participation threshold, to further optimize the output of the network;

[0053] Step S506: when the temperature drops to a preset threshold, the training process of the classifier model ends.

[0054] Further, the step S6 includes:

[0055] Step S601, collecting image data in real time as input data;

[0056] Step S602, inputting the input data into a feature extraction model to extract key features;

[0057] Step S603, extracting the feature vector of the key feature and sending it to the data dimensionality reduction model to reduce the data dimension;

[0058] Step S604, the feature vector after dimension reduction is sent to the classifier model;

[0059] Step S605: According to steps S601 to S604, the classifier outputs the recognition result of the marked obstacle category.

[0060] According to two aspects of the present invention, there is provided an automatic driving obstacle recognition device, the automatic driving obstacle recognition device comprising:

[0061] A data acquisition module is used to collect image data and annotate the data based on visual recognition;

[0062] Data expansion module, used to generate image samples and expand data based on the generative adversarial network algorithm of visual simulation;

[0063] A feature extraction module is used to input the expanded data into a feature extraction model to train the feature extraction model;

[0064] The feature dimension reduction module is used to input the feature extracted data into the feature dimension reduction model to train the feature dimension reduction model;

[0065] The classification training module is used to input the reduced-dimensional data into the classifier for classifier training.

[0066] The obstacle recognition module is used to perform vision-based obstacle recognition based on the trained feature extraction model, data dimension reduction model and classifier model.

[0067] According to three aspects of the present invention, there is provided an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0068] A computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the autonomous driving obstacle identification method.

[0069] According to four aspects of the present invention, a computer-readable storage medium is provided, comprising: a computer program executable by an electronic device is stored therein, and when the computer program runs on the electronic device, the electronic device executes the steps of the autonomous driving obstacle identification method.

[0070] According to five aspects of the present invention, there is provided a vehicle, comprising:

[0071] An electronic device for implementing the steps of the autonomous driving obstacle recognition method;

[0072] A processor, the processor running a program, and executing the steps of the autonomous driving obstacle recognition method from data output from the electronic device when the program is running;

[0073] A storage medium is used to store a program, which, when running, executes the steps of the autonomous driving obstacle recognition method for data output from the electronic device.

[0074] Through the above scheme, the following beneficial technical effects are obtained:

[0075] This application improves the model's recognition ability on unseen obstacle images by generating images generated by a generative adversarial network and a feature enhancement loss function.

[0076] This application optimizes parameter search, and the adaptive vortex optimization algorithm uses a non-gradient dependent parameter optimization method to dynamically adjust parameters through the simulation of natural vortices to improve the solution quality.

[0077] This application uses an autoencoder to help the model reduce dimensions and computational complexity during feature extraction while retaining important information.

[0078] This application achieves faster weight solution and higher classification accuracy through an improved extreme learning machine algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a flowchart of an automatic driving obstacle recognition method provided by one or more embodiments of the present invention.

[0080] Figure 2It is a structural diagram of an automatic driving obstacle recognition device provided by one or more embodiments of the present invention.

[0081] Figure 3 It is a schematic diagram of a feature extraction model of a specific embodiment of the present invention.

[0082] Figure 4 It is a schematic diagram of a feature dimensionality reduction model of a specific embodiment of the present invention.

[0083] Figure 5 It is a structural block diagram of an electronic device of an automatic driving obstacle recognition method provided by one or more embodiments of the present invention. DETAILED DESCRIPTION

[0084] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0085] Figure 1 It is a flowchart of an automatic driving obstacle recognition method provided by one or more embodiments of the present invention.

[0086] like Figure 1 The autonomous driving obstacle recognition method shown includes:

[0087] Step S1, based on visual recognition, collecting image data and annotating the data;

[0088] Step S2, generating image samples based on a visual simulation generative adversarial network algorithm to expand image data;

[0089] Step S3, inputting the expanded data into a feature extraction model to train the feature extraction model;

[0090] Step S4, inputting the feature-extracted data into a feature dimensionality reduction model to train the feature dimensionality reduction model;

[0091] Step S5, inputting the dimension-reduced data into a classifier to train the classifier;

[0092] Step S6, performing vision-based obstacle recognition according to the trained feature extraction model, data dimension reduction model and classifier model.

[0093] Specifically, in step S1, based on visual recognition, image data is collected and the data is labeled.

[0094] In one embodiment, the data is derived from an image acquisition device, such as a vehicle-mounted camera, to capture images of various obstacles in the driving environment. The acquired data is stored in the form of images, in high-resolution JPEG format. The data is annotated manually, and the annotated categories include pedestrians, vehicles, animals, and static obstacles.

[0095] Step S2, generating image samples based on a visual simulation generative adversarial network algorithm to expand the image data.

[0096] It is understandable that the collection, acquisition, labeling and preprocessing of training data are time-consuming and labor-intensive, and insufficient training samples can easily lead to poor generalization ability of the model and affect the accuracy of the model.

[0097] In one embodiment, a visual simulation-based generative adversarial network algorithm is used to generate samples, thereby achieving data expansion. The visual simulation-based generative adversarial network algorithm consists of two parts: a generator (G) and a discriminator (D). The generator is responsible for generating visually credible image data, and the discriminator's task is to distinguish between generated images and real images.

[0098] The loss function of the generative adversarial network adopted not only considers the overall authenticity of the image, but also strengthens the learning of key features, such as the shape and size of obstacles, which helps the generator to simulate key features more accurately and improve the practicality of the generated images.

[0099] Step S3, inputting the expanded data into the feature extraction model to train the feature extraction model.

[0100] In one embodiment, the expanded data is input into a feature extraction model to train the feature extraction model.

[0101] A 3-layer fully connected neural network is used for feature extraction. Some solutions use neural networks for feature extraction. In some neural network structures, problems such as gradient vanishing, gradient exploding, or falling into local optimal solutions may occur, affecting the stability of training and the performance of the model.

[0102] In one embodiment, a neural network algorithm based on adaptive vortex optimization optimizes the parameters (weight w and bias b) in the neural network based on the vortex phenomenon by simulating the formation, evolution and dissipation process of the vortex. In fluid dynamics, vortices can increase the chaos and complexity of the fluid, thereby guiding the search process in the parameter space towards a better solution. By simulating this natural process, the neural network of adaptive vortex optimization aims to find the parameter configuration that can minimize the neural network loss function without relying on the gradient-based neural network parameter optimization method.

[0103] Step S4, inputting the feature-extracted data into the feature dimensionality reduction model to train the feature dimensionality reduction model.

[0104] In one embodiment, the feature extracted data is input into a feature dimensionality reduction model for training the feature dimensionality reduction model, and an autoencoder algorithm is used for feature dimensionality reduction, and the autoencoder algorithm includes two parts: an encoder and a decoder. The encoder is responsible for converting high-dimensional feature data into low-dimensional feature representations, and the decoder attempts to reconstruct the original data from these low-dimensional representations.

[0105] In one embodiment, the total loss is calculated using mean square error based on L2 regularization to avoid overfitting while ensuring that the model can learn effective data compression.

[0106] Step S5, inputting the dimension-reduced data into the classifier to train the classifier.

[0107] In one embodiment, the reduced-dimensional data is input into a classifier for classifier training, and an extreme learning machine classification algorithm based on deterministic annealing is used. The extreme learning machine is a single-layer feedforward neural network, which is characterized in that the weights and biases of the hidden layer do not need to be learned through iteration, but only need to remain unchanged after random generation, and the output weights are quickly calculated by analytical methods. In this embodiment, the traditional extreme learning machine is structurally improved, and a deterministic annealing process is used to optimize the solution of the output weights, thereby improving the classification performance.

[0108] Step S6, performing vision-based obstacle recognition according to the trained feature extraction model, data dimension reduction model and classifier model.

[0109] In one embodiment, obstacle recognition is performed using a trained feature extraction model, a data dimension reduction model, and a classifier model.

[0110] In one embodiment, the input data is image data collected in real time. The input image data is firstly used to extract key features from the original image through a feature extraction model. Furthermore, the extracted feature vector is then sent to a data dimension reduction model to reduce the dimension of the data. Furthermore, the feature vector after dimension reduction is sent to a classifier model.

[0111] In this embodiment, the output categories of the classifier include pedestrians, vehicles, animals, and static obstacles.

[0112] In this embodiment, step S1 includes:

[0113] Collecting image data and annotating the data includes collecting image data from a vehicle-mounted camera;

[0114] According to the image data of the vehicle-mounted camera, obstacles in the image data are marked;

[0115] Labeling obstacles in the image data includes labeling categories of the obstacles.

[0116] Specifically, in one embodiment, the vehicle-mounted camera has multiple lenses arranged in the direction in which the vehicle can travel. Based on the set recognition features, manual annotation is first performed to initialize the categories.

[0117] In this embodiment, step S2 includes that the visual simulation generative adversarial network algorithm includes a generator and a discriminator;

[0118] Step S201, initializing the parameters of the generator and the discriminator;

[0119] Step S202, the generator selects samples from the image data with obstacles marked, for training the discriminator to identify the authenticity of obstacles in the image data;

[0120] Step S203, the discriminator receives the image data from the vehicle-mounted camera and the image sample data generated by the generative adversarial network algorithm, and performs discriminator training;

[0121] Step S204, after the discriminator is trained, the generator updates parameters according to the feedback from the discriminator;

[0122] Step S205, obtaining a feature enhancement loss function, inputting current update parameters and continuously iterating;

[0123] After each parameter update iteration, the parameters of the generator are adjusted according to the result of the feature enhancement loss function.

[0124] Specifically, in one embodiment, step S201, the generator and the discriminator Parameters and Initialization is performed respectively. In one embodiment, initialization is implemented in a standard normal distribution manner, which can be expressed as:

[0125] ;

[0126] ;

[0127] In the formula, is the parameter of the generator; are the parameters of the discriminator; Represents a normal distribution with mean 0 and variance the identity matrix.

[0128] In one embodiment, step S202, select samples from the labeled obstacle image , the samples are real obstacle images, which are used to train the discriminator to identify true and false images. Indicates A real image sample.

[0129] In this embodiment, the generator generates a fake image by sampling from the noise distribution according to the current parameters.

[0130] In this embodiment, the generator Based on input noise , through the network parameters Generate fake images , which can be expressed as:

[0131] ;

[0132] ;

[0133] In the formula, represents the generated fake image; represents a generator function, is the noise distribution The noise vector sampled from .

[0134] In one embodiment, step S203, the discriminator Receive real image and generate images , and by optimizing its parameters To improve the accuracy of classification, it can be expressed as:

[0135] ;

[0136] ;

[0137] In the formula, is the loss function of the discriminator; Represents the discriminator outputs the real image is the true probability; is the learning rate; is the loss function about Preferably, Set to 0.005.

[0138] In this embodiment, the discriminator outputs The calculation method can be expressed as:

[0139] ;

[0140] In the formula, is the sigmoid activation function; is the weight matrix of the discriminator; is the input image The feature representation is obtained through a series of convolutional layers; is the bias term.

[0141] In this embodiment, The calculation method can be expressed as:

[0142] ;

[0143] In the formula, is the ReLU nonlinear activation function; is the feature extraction weight; is the feature extraction bias.

[0144] In one embodiment, step S204, after the discriminator is trained, the generator According to the discriminator Feedback update parameters , the present invention adopts the feature enhancement loss function , which can be expressed as:

[0145]

[0146]

[0147] In the formula, is the loss function of the generator; It is a feature enhancement loss, emphasizing the generation of images and real images Similarity in key features; is the weight factor of feature enhancement; is the learning rate; is the loss function about Preferably, Set to 0.01.

[0148] In this embodiment, the feature enhancement loss function It can strengthen the similarity of key features, and the calculation method can be expressed as:

[0149]

[0150] In the formula, It is a feature extraction function used to extract high-level features of an image; represents the Euclidean distance.

[0151] In this embodiment, is defined by a series of pre-trained convolutional network layers and can be expressed as:

[0152]

[0153] In the formula, It is a global average pooling layer function, which is used to reduce the feature dimension and retain important information; and are the weights and biases of the feature extraction network, respectively.

[0154] In one embodiment, in step S205, after each iteration, the parameters of the generator are adjusted according to the result of the feature enhancement loss function, so that the generated image pays more attention to the simulation of the key features of the obstacle in the next round of training.

[0155] Repeat the above steps until the preset stop iteration condition is met, which means that the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations. Preferably, the preset maximum number of iterations is set to 1000 times.

[0156] In this embodiment, step S3 includes obtaining an adaptive vortex and optimizing a neural network;

[0157] Step S301, randomly generate a group of vortices in the parameter space, each vortex represents a possible parameter configuration;

[0158] Step S302, calculating the performance of the parameter configuration represented by each vortex on the training set, and sorting the vortices according to the performance;

[0159] Step S303, dynamically updating the vortexes, dynamically adjusting the positions and intensities of the vortices according to the rankings of the vortices and the distances between them, and simulating the interaction and evolution of the vortices;

[0160] Step S304, simulating the merging and splitting phenomena of vortices in nature to promote the diversity of the solution space and explore new parameter regions;

[0161] Step S305, sorting the vortices according to their performance, selecting the vortex with the best performance, performing local search for fine-tuning parameters, and simulating the local absorption and energy concentration phenomenon of the vortex in the fluid;

[0162] Step S306, performing vortex attenuation to simulate the dissipation process of vortex energy in nature, gradually eliminating vortices with poor performance, and releasing resources for newly generated or well-performing vortices;

[0163] Step S307, repeat the above steps S301 to S306 until the preset stop iteration condition is met, and the training of the feature extraction model is completed.

[0164] Specifically, Figure 3It is a schematic diagram of a feature extraction model of a specific embodiment of the present invention.

[0165] like Figure 3 As shown, a feature extraction model is established according to the training process of the neural network algorithm of adaptive vortex optimization.

[0166] In one embodiment, step S301 randomly generates a set of vortices in the parameter space, each vortex represents a possible parameter configuration (including weight w and bias b).

[0167] In this embodiment, the neural network parameter vector is defined as ,in is the vortex index.

[0168] In this embodiment, during the initialization process, the parameter vector of each vortex It is randomly generated in the parameter space, conforming to the uniform distribution, and the total number of initial vortices is set to , then the initialization process can be expressed as:

[0169] ;

[0170] In the formula, represents uniform distribution, and are the lower and upper bounds of the parameter space, which are preset manually. and Set to 0.0001 and 0.5 respectively.

[0171] In one embodiment, step S302 calculates the performance of the parameter configuration represented by each vortex on the training set, and sorts the vortices according to the performance. Specifically, the performance evaluation uses the loss function To complete, after the evaluation is completed, the vortex is sorted according to the value of the loss function:

[0172] ;

[0173] In this embodiment, the loss function is the cross entropy loss.

[0174] In one embodiment, step S303 is to perform vortex dynamic update, dynamically adjust the position and strength of the vortex according to the ranking of the vortex and the distance between them, and simulate the interaction and evolution of the vortex. Specifically, in the vortex dynamic update stage, the interaction of the vortex is simulated to adjust its position in the parameter space, and the vortex is set. and The distance between , then The update amount of vortex It can be expressed as:

[0175] ;

[0176] In the formula, is a hyperparameter that adjusts the distance effect, is a distance-based conditioning function, is the adaptive swirl factor, is a regulatory factor based on environmental feedback. Preferably, Set to 0.3.

[0177] In one embodiment, the distance-based adjustment function The calculation method uses the exponential decay model to reflect the characteristics of the interaction force between vortices weakening with increasing distance, which can be expressed as:

[0178]

[0179] In the formula, and is a hyperparameter that controls the magnitude and decay rate of the force. is set to 0.5, is set to 0.1.

[0180] In this embodiment, Represents the vortex and vortex The distance between them is calculated based on the Euclidean distance of the parameter vectors represented by the two vortices and can be expressed as:

[0181] ;

[0182] In the formula, is the dimension of the parameter vector, and Vortex and vortex In the Dimension parameter value.

[0183] In this embodiment, the adaptive swirl factor The information entropy of the vortex distribution is calculated using adaptive vortex guided learning. for:

[0184] ;

[0185] In the formula, It is The proportion of a vortex in all vortices is calculated by the inverse of the vortex loss function and can be expressed as:

[0186] ;

[0187] In this embodiment, the adaptive vortex factor is adjusted according to the change of information entropy. The search strategy for adjusting the vortex can be expressed as:

[0188] ;

[0189] In the formula, is a preset maximum adjustment intensity. It is the maximum information entropy value that can be achieved theoretically, indicating a completely uniform distribution state. is the momentum adjustment factor, For vortex In recent The average momentum vector over iterations.

[0190] In this embodiment, the momentum adjustment factor and the average momentum vector It is obtained through time series momentum adjustment. The time series momentum adjustment aims to adjust the exploration trend of the vortex by analyzing its historical movement trend in the parameter space, considering the movement direction and speed of the vortex in the historical iteration, calculating its momentum, and then using this momentum information to guide future parameter updates. Specifically, the vortex is calculated In recent The average momentum vector in iterations It can be expressed as:

[0191] ;

[0192] In the formula, It's a vortex In the The parameter vector after iterations, It is The parameter vector after iterations.

[0193] In this embodiment, the momentum adjustment factor is defined as It is used to measure the size and consistency of momentum and can be expressed as:

[0194] ;

[0195] In the formula, yes and The angle between them is used to measure the consistency between the latest moving direction and the average momentum direction. The smaller the angle, the more consistent the momentum.

[0196] In this embodiment, the calculation method of the adjustment factor based on environmental feedback can be expressed as:

[0197] ;

[0198] In the formula, It is a Sigmoid mapping function that maps environmental feedback to the (0, 1) interval. is the environmental feedback function, which is used to evaluate the vortex The characteristics of the surrounding environment can be expressed as:

[0199] ;

[0200] In the formula, It's a vortex The Hessian matrix of the loss function at location (used to evaluate local convexity), is the magnitude of the gradient (used to assess steepness in parameter space).

[0201] In one embodiment, step S304 simulates the merging and splitting of vortices in nature to promote the diversity of solution space and explore new parameter regions. Specifically, the merging threshold is set and split threshold , when the distance between the two vortices is less than When the strength of a vortex exceeds When , it will split into two vortices. The merge operation can be expressed as:

[0202] ;

[0203] In the formula, and It's a vortex and vortex The corresponding loss function value.

[0204] And, the split operation can be expressed as:

[0205] ;

[0206] In the formula, Based on the vortex A vector of random perturbations of the parameter vector, used to generate the parameters of the two new vortices.

[0207] In this embodiment, the random perturbation vector Based on the local gradient information of the current vortex position, the randomness and exploration of the search process are increased. The calculation method can be expressed as:

[0208] ;

[0209] In the formula, and is a hyperparameter that controls the gradient direction and random perturbation intensity, Represents the vortex position The gradient information calculated at The mean is 0 and the covariance matrix is Preferably, is set to 1.0, is set to 0.1.

[0210] In one embodiment, in step S305, the best performing vortex is selected and local search is performed to fine-tune its parameters, simulating the local absorption and energy concentration phenomenon of the vortex in the fluid. For the best performing vortex, local search is performed to further fine-tune its parameters, which is achieved through a domain-based search strategy.

[0211] In this embodiment, in the domain-based search strategy, the local search method can be expressed as:

[0212] ;

[0213] In the formula, is the learning rate, The mean is 0. is a Gaussian distribution with covariance matrix, used to generate random perturbations of the parameters.

[0214] In this embodiment, the covariance matrix According to the historical moving direction and speed of the vortex in the parameter space, it is dynamically adjusted to adapt to different search stages and characteristics. Its update strategy can be expressed as:

[0215] ;

[0216] In the formula, is the adjusted covariance matrix, is the covariance matrix before adjustment, is a learning rate parameter used to control the influence of historical information on the current search. Preferably, is set to 0.2.

[0217] In one embodiment, step S306 performs vortex attenuation, simulating the dissipation process of vortex energy in nature, gradually eliminating vortices with poor performance, and releasing resources to newly generated or better performing vortices. Specifically, the attenuation process can be achieved by adjusting the life value of the vortex. The vortex with a life value below a certain threshold will be eliminated. The life value is , the decay rule is:

[0218] ;

[0219] In the formula, is the health value after decay, is the amount of reduction in health value. Preferably, Set to 0.0001.

[0220] In one embodiment, step S307, the above steps are repeated until a preset stop iteration condition is met, indicating that the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations, preferably, the preset maximum number of iterations is set to 1000 times.

[0221] In this embodiment, step S4 includes:

[0222] Step S401, randomly initializing the weights of the encoder and decoder according to a preset distribution;

[0223] Step S402, in a forward propagation manner, the encoder converts the input data into a low-dimensional feature representation, and the data represented as the low-dimensional feature is passed to the decoder to attempt to reconstruct the original data;

[0224] Step S403, calculating the total loss based on the mean square error of L2 regularization to avoid overfitting and ensure that the model can learn effective data compression, including obtaining a loss function;

[0225] Step S404, in the process of back propagation, updating the network weights by using a gradient descent algorithm according to the loss function;

[0226] Step S405, adaptively adjusting the learning rate according to the training progress, and using an exponential decay strategy to optimize the learning process;

[0227] Step S406, repeat the above steps S401 to S405 until the preset stop iteration condition is met, and the training of the feature dimensionality reduction model is completed.

[0228] Specifically, Figure 4 It is a schematic diagram of a feature dimensionality reduction model of a specific embodiment of the present invention.

[0229] like Figure 4 As shown, the feature dimension reduction model training is based on the training process of the adopted autoencoder algorithm.

[0230] In one embodiment, step S401 randomly initializes the weights of the encoder and the decoder according to a preset distribution.

[0231] In this embodiment, the weight of the autoencoder and bias Using Gaussian distribution random initialization, it can be expressed as:

[0232] ;

[0233] ;

[0234] In the formula, For the The weight of the layer; For the Bias of the layer; The mean is 0 and the variance is Normal distribution of is the standard deviation. Preferably, Set to 0.1.

[0235] In one embodiment, step S402, input data by forward propagation The encoder converts it into a low-dimensional feature representation, which is then passed to the decoder to try to reconstruct the original data. Specifically, the output of the encoder part and the decoder output The calculation method can be expressed as:

[0236] ;

[0237] ;

[0238] In the formula, is the input data; is the output of the encoder, i.e., the feature representation; is the decoder reconstructed output; and are the weights and biases of the encoder respectively; and are the weight and bias of the decoder respectively; is the Sigmoid activation function.

[0239] In one embodiment, step S403 uses the mean square error based on L2 regularization to calculate the total loss to avoid overfitting while ensuring that the model can learn effective data compression. The loss function The calculation method can be expressed as:

[0240] ;

[0241] In the formula, is the reconstruction error, calculated using the Euclidean distance; is the regularization parameter; and is the L2 regularization term of the weights.

[0242] In this embodiment, the reconstruction error term The calculation method can be expressed as:

[0243] ;

[0244] In the formula, is the input data vector No. elements; is the reconstructed data vector No. elements; is the number of elements in the data vector.

[0245] In this embodiment, the regularization term and Expressed as the sum of squares of all encoder and decoder weights respectively, it can be expressed as:

[0246] ;

[0247] ;

[0248] In the formula, is the encoder weight matrix Middle Line Elements of a column; is the decoder weight matrix Middle Line Elements of a column.

[0249] In one embodiment, in step S404, during the back propagation process, the network weights are updated by a gradient descent algorithm according to the loss function, which can be expressed as:

[0250] ;

[0251] ;

[0252] In the formula, is the learning rate; and are the gradients of the loss function with respect to the encoder and decoder weights, respectively. Preferably, The adaptive adjustment method is used and the initial learning rate is set to 0.01.

[0253] In one embodiment, step S405 adaptively adjusts the learning rate according to the training progress, and uses an exponential decay strategy to optimize the learning process. The calculation method can be expressed as:

[0254] ;

[0255] In the formula, is the base of natural logarithms; is the decay rate; is the number of iterations.

[0256] Furthermore, the decay rate is the rate of change related to time, and the calculation method can be expressed as:

[0257] ;

[0258] In the formula, is the initial learning rate, preferably, Set to 0.01. Is to reach the specified learning rate The number of iterations required.

[0259] In one embodiment, step S406, the above steps are iterated repeatedly until a preset stop iteration condition is met, which means that the model training is completed.

[0260] In one embodiment, the preset condition for stopping iteration is reaching a preset maximum number of iterations. Preferably, the preset maximum number of iterations is set to 1000 times.

[0261] In this embodiment, step S5 includes:

[0262] Step S501, randomly initializing the weights and biases of the hidden layer of the extreme learning machine;

[0263] Step S502, setting the initial temperature parameters to control the annealing rate during the training process;

[0264] Step S503, the input data passes through the hidden layer and is processed using the Sigmoid function nonlinear activation function to generate the output of the intermediate layer;

[0265] Step S504, using a deterministic annealing strategy to optimize the solution of the output weights, in each annealing step, adjusting the solution process according to the current temperature, gradually reducing the temperature, and optimizing the weight of the output layer;

[0266] Step S505, dynamically adjusting the activation state of the hidden layer neurons according to the set participation threshold, to further optimize the output of the network;

[0267] Step S506: when the temperature drops to a preset threshold, the training process of the classifier model ends.

[0268] Specifically, in one embodiment, step S501 randomly initializes the weights and biases of the hidden layer of the extreme learning machine. The initialization method of the hidden layer weights and biases can be expressed as:

[0269] ;

[0270] ;

[0271] In the formula, is the hidden layer weight; is the hidden layer bias; Indicates that the weights are initialized to a mean of 0 and a variance of Normal distribution of Indicates that the bias is initialized to an interval Preferably, Set to 0.001, Set to 0.5.

[0272] In one embodiment, step S502, an initial temperature parameter is set to control the annealing rate during the training process. Specifically, the temperature parameter is set as:

[0273] ;

[0274] In the formula, is the current temperature; is the initial temperature; is the annealing rate; is the current training round.

[0275] In this embodiment, the annealing rate The calculation method can be expressed as:

[0276] ;

[0277] In the formula, is the minimum temperature threshold; is the predetermined maximum number of training rounds. Preferably, Set to 0.005, Set to 10000.

[0278] In one embodiment, in step S503, the input data passes through the hidden layer and is processed using the Sigmoid function nonlinear activation function to generate the output of the intermediate layer, which can be expressed as:

[0279] ;

[0280] In the formula, is the hidden layer output; is the input feature vector; is the Sigmoid activation function.

[0281] In one embodiment, step S504 uses a deterministic annealing strategy to optimize the solution of the output weight. In each annealing step, the solution process is adjusted according to the current temperature, the temperature is gradually reduced, and the weight of the output layer is optimized, which can be expressed as:

[0282] ;

[0283] In the formula, is the output layer weight; is the transpose of the hidden layer output; Output for the target; is the regularization parameter, which changes with temperature change; is the identity matrix.

[0284] In this embodiment, the regularization parameter According to the dynamic adjustment of the annealing process, the calculation method can be expressed as:

[0285] ;

[0286] In the formula, is an additional tuning parameter that helps balance the regularization strength. Preferably, Set to 5.

[0287] In one embodiment, step S505 dynamically adjusts the activation state of the hidden layer neurons according to the set participation threshold to further optimize the output of the network, which can be expressed as:

[0288] ;

[0289] In the formula, is the adjusted hidden layer output; represents the Hadamard product; is the activation threshold, controlling the participation of the neuron. Preferably, Set to 0.035.

[0290] In one embodiment, in step S506, when the temperature drops to a preset threshold When , the training process ends. Preferably, Set to 0.005.

[0291] In this embodiment, step S6 includes:

[0292] Step S601, collecting image data in real time as input data;

[0293] Step S602, inputting the input data into a feature extraction model to extract key features;

[0294] Step S603, extracting the feature vector of the key feature and sending it to the data dimensionality reduction model to reduce the data dimension;

[0295] Step S604, the feature vector after dimension reduction is sent to the classifier model;

[0296] Step S605: According to steps S601 to S604, the classifier outputs the recognition result of the marked obstacle category.

[0297] Figure 2 It is a structural diagram of an automatic driving obstacle recognition device provided by one or more embodiments of the present invention.

[0298] like Figure 2 The autonomous driving obstacle recognition device shown includes: a data collection module, a data expansion module, a feature extraction module, a feature dimension reduction module, a classification training module, and an obstacle recognition module;

[0299] A data acquisition module is used to collect image data and annotate the data based on visual recognition;

[0300] Data expansion module, used to generate image samples and expand data based on the generative adversarial network algorithm of visual simulation;

[0301] A feature extraction module is used to input the expanded data into a feature extraction model to train the feature extraction model;

[0302] The feature dimension reduction module is used to input the feature extracted data into the feature dimension reduction model to train the feature dimension reduction model;

[0303] The classification training module is used to input the reduced-dimensional data into the classifier for classifier training.

[0304] The obstacle recognition module is used to perform vision-based obstacle recognition based on the trained feature extraction model, data dimension reduction model and classifier model.

[0305] It is worth noting that although the system only discloses a data collection module, a data expansion module, a feature extraction module, a feature dimension reduction module, a classification training module, and an obstacle identification module, it does not mean that the device is limited to the above-mentioned basic functional modules. Rather, what the present invention wants to express is that, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with the prior art to form an infinite number of embodiments or technical solutions. In other words, the system is open rather than closed. Just because the present embodiment only discloses individual basic functional modules, it cannot be considered that the scope of protection of the claims of the present invention is limited to the above-mentioned basic functional modules.

[0306] Through the above scheme, the following beneficial technical effects are obtained:

[0307] This application improves the model's recognition ability on unseen obstacle images by generating images generated by a generative adversarial network and a feature enhancement loss function.

[0308] This application optimizes parameter search, and the adaptive vortex optimization algorithm uses a non-gradient dependent parameter optimization method to dynamically adjust parameters through the simulation of natural vortices to improve the solution quality.

[0309] This application uses an autoencoder to help the model reduce dimensions and computational complexity during feature extraction while retaining important information.

[0310] This application achieves faster weight solution and higher classification accuracy through an improved extreme learning machine algorithm.

[0311] Figure 5 It is a structural block diagram of an electronic device of an automatic driving obstacle recognition method provided by one or more embodiments of the present invention.

[0312] like Figure 5 As shown, the present application provides an electronic device, including: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0313] A computer program is stored in the memory. When the computer program is executed by the processor, the processor performs the steps of an automatic driving obstacle recognition method.

[0314] The present application also provides a computer-readable storage medium, which stores a computer program that can be executed by an electronic device. When the computer program runs on the electronic device, the electronic device executes the steps of an autonomous driving obstacle identification method.

[0315] The present application also provides a vehicle, comprising:

[0316] An electronic device for implementing the steps of the method for identifying obstacles in an autonomous driving vehicle;

[0317] A processor, wherein the processor runs a program, and when the program runs, the data output by the electronic device executes the steps of the automatic driving obstacle recognition method;

[0318] A storage medium is used to store a program, which, when running, executes the steps of an autonomous driving obstacle recognition method for data output from an electronic device.

[0319] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0320] The electronic device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory. The operating system can be any one or more computer operating systems that implement electronic device control through a process, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a Windows operating system. In addition, in an embodiment of the present invention, the electronic device can be a handheld device such as a smart phone or a tablet computer, or can be an electronic device such as a desktop computer or a portable computer, which is not particularly limited in the embodiment of the present invention.

[0321] The execution subject of the electronic device control in the embodiment of the present invention may be an electronic device, or a functional module in the electronic device that can call and execute a program. The electronic device may obtain the firmware corresponding to the storage medium. The firmware corresponding to the storage medium is provided by the supplier. The firmware corresponding to different storage media may be the same or different, which is not limited here. After the electronic device obtains the firmware corresponding to the storage medium, the firmware corresponding to the storage medium may be written into the storage medium, specifically, the firmware corresponding to the storage medium may be burned into the storage medium. The process of burning the firmware into the storage medium may be implemented using existing technology, which will not be described in detail in the embodiment of the present invention.

[0322] The electronic device may also obtain a reset command corresponding to the storage medium. The reset command corresponding to the storage medium is provided by the supplier. The reset commands corresponding to different storage media may be the same or different, and are not limited here.

[0323] At this time, the storage medium of the electronic device is a storage medium in which the corresponding firmware is written, and the electronic device can respond to the reset command corresponding to the storage medium in the storage medium in which the corresponding firmware is written, so that the electronic device resets the storage medium in which the corresponding firmware is written according to the reset command corresponding to the storage medium. The process of resetting the storage medium according to the reset command can be implemented by the existing technology and will not be described in detail in the embodiments of the present invention.

[0324] For the convenience of description, the above devices are described in terms of functions and are divided into various units and modules. Of course, when implementing the present application, the functions of each unit and module can be implemented in the same or multiple software and / or hardware.

[0325] Those skilled in the art will appreciate that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined.

[0326] For the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0327] It can be known from the description of the above implementation modes that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute the methods described in the various implementation modes of the present application or certain parts of the implementation modes.

[0328] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying obstacles in an autonomous driving system, characterized in that: The automatic driving obstacle recognition method comprises: Step S1, based on visual recognition, collecting image data and annotating the data; Step S2, generating image samples based on a visual simulation generative adversarial network algorithm to expand image data; Step S3, inputting the expanded data into a feature extraction model to train the feature extraction model; Step S4, inputting the feature-extracted data into a feature dimensionality reduction model to train the feature dimensionality reduction model; Step S5, inputting the dimension-reduced data into a classifier to train the classifier; Step S6, performing visual-based obstacle recognition according to the trained feature extraction model, data dimension reduction model and classifier model; Wherein, the step S2 includes that the visual simulation generative adversarial network algorithm includes a generator and a discriminator; Step S201, initializing the parameters of the generator and the discriminator; Step S202, the generator selects samples from the image data with obstacles marked, for training the discriminator to identify the authenticity of obstacles in the image data; Step S203, the discriminator receives the image data from the vehicle-mounted camera and the image sample data generated by the generative adversarial network algorithm, and performs discriminator training; Step S204, after the discriminator is trained, the generator updates parameters according to the feedback from the discriminator; Step S205, obtaining a feature enhancement loss function, inputting the current update parameters and continuously iterating; After each parameter update iteration, the parameters of the generator are adjusted according to the result of the feature enhancement loss function; Wherein, step S203 includes, the discriminator D receives the real image x and the generated image And by optimizing its parameter θ d To improve the accuracy of classification, it is expressed as: Where, L D is the loss function of the discriminator; D(x;θ d ) represents the probability that the discriminator outputs the real image x as real; α d is the learning rate; is the loss function with respect to θ d The gradient of It also includes the discriminator output D(x; θ d ) is calculated as: D(x;θ d )=Sig(W d ·f(x)+b d ); Where Sig() is the sigmoid activation function; W d is the weight matrix of the discriminator; f(x) is the feature representation of the input image x, obtained through a series of convolutional layers; b d is the bias term; Also included: The calculation method of f(x) is expressed as: f(x)=ReLU(W f ·x+b f ): Where ReLU() is the ReLU nonlinear activation function; W f is the feature extraction weight; b f is the feature extraction bias; Among them, step S204 includes, after the discriminator is trained, the generator G updates the parameter θ according to the feedback of the discriminator D g , using feature enhancement loss function L G , expressed as: Where, L G is the loss function of the generator; It is a feature enhancement loss, emphasizing the generation of images The similarity between the key features of the real image x and λ f is the weight factor of feature enhancement; α g is the learning rate; is the loss function with respect to θ g The gradient of Also includes, feature enhancement loss function The similarity of key features is strengthened and the calculation method is expressed as: Where F(x) is a feature extraction function used to extract high-level features of the image; ∥·∥2 represents the Euclidean distance; Wherein, the step S201 further includes: The parameters θ of the generator G and the discriminator D g and θ d They are initialized separately, and the initialization is implemented using the standard normal distribution, which is expressed as: In the formula, θ g is the parameter of the generator; θ d are the parameters of the discriminator; Represents a normal distribution with a mean of 0 and a variance of the identity matrix; Wherein, the step S202 further includes: Select samples from labeled obstacle images The sample is a real obstacle image, which is used to train the discriminator to identify true and false images. (i) represents the i-th real image sample; The generator generates fake images by sampling from the noise distribution according to the current parameters; The generator G is based on the input noise z and the network parameters θ g Generate fake images It is expressed as: z~p(z); In the formula, represents the generated fake image; G(z;θ g ) represents the generator function, z is the noise vector sampled from the noise distribution p(z); Wherein, the step S204 further includes: F(x) is defined by a series of pre-trained convolutional network layers and is expressed as: F(x)=GlobalAvgPool(ReLU(W F ·x+b F )) Where GlobalAvgPool() is the global average pooling layer function, which is used to reduce the feature dimension and retain important information; W F and b F are the weights and biases of the feature extraction network, respectively.

2. The automatic driving obstacle recognition method according to claim 1, characterized in that: The step S1 comprises: The collecting of image data and labeling of the data includes collecting image data from a vehicle-mounted camera; Marking obstacles in the image data according to the image data of the vehicle-mounted camera; The marking of obstacles in the image data includes marking the categories of the obstacles.

3. The automatic driving obstacle recognition method according to claim 1, characterized in that: The step S3 includes obtaining an adaptive vortex and optimizing a neural network; Step S301, randomly generate a group of vortices in the parameter space, each vortex represents a possible parameter configuration; Step S302, calculating the performance of the parameter configuration represented by each vortex on the training set, and sorting the vortices according to the performance; Step S303, dynamically updating the vortexes, dynamically adjusting the positions and intensities of the vortices according to the rankings of the vortices and the distances between them, and simulating the interaction and evolution of the vortices; Step S304, simulating the merging and splitting phenomena of vortices in nature to promote the diversity of the solution space and explore new parameter regions; Step S305, sorting the vortices according to the performance, selecting the vortex with the best performance, performing local search for fine adjustment of parameters, and simulating the local absorption and energy concentration phenomenon of the vortex in the fluid; Step S306, performing vortex attenuation to simulate the dissipation process of vortex energy in nature, gradually eliminating vortices with poor performance, and releasing resources for newly generated or well-performing vortices; Step S307, repeat the above steps S301 to S306 until the preset stop iteration condition is met, and the training of the feature extraction model is completed.

4. The automatic driving obstacle recognition method according to claim 1, characterized in that: The step S4 comprises: Step S401, randomly initializing the weights of the encoder and decoder according to a preset distribution; Step S402, in a forward propagation manner, the encoder converts the input data into a low-dimensional feature representation, and the data represented as the low-dimensional feature is passed to the decoder to attempt to reconstruct the original data; Step S403, calculating the total loss based on the mean square error of L2 regularization to avoid overfitting and ensure that the model can learn effective data compression, including obtaining a loss function; Step S404, in the process of back propagation, updating the network weights by using a gradient descent algorithm according to the loss function; Step S405, adaptively adjusting the learning rate according to the training progress, and using an exponential decay strategy to optimize the learning process; Step S406, repeat the above steps S401 to S405 until the preset stop iteration condition is met, and the training of the feature dimensionality reduction model is completed.

5. The automatic driving obstacle recognition method according to claim 1, characterized in that: The step S5 comprises: Step S501, randomly initializing the weights and biases of the hidden layer of the extreme learning machine; Step S502, setting the initial temperature parameters to control the annealing rate during the training process; Step S503, the input data passes through the hidden layer and is processed using the Sigmoid function nonlinear activation function to generate the output of the intermediate layer; Step S504, using a deterministic annealing strategy to optimize the solution of the output weights, in each annealing step, adjusting the solution process according to the current temperature, gradually reducing the temperature, and optimizing the weight of the output layer; Step S505, dynamically adjusting the activation state of the hidden layer neurons according to the set participation threshold, to further optimize the output of the network; Step S506: when the temperature drops to a preset threshold, the training process of the classifier model ends.

6. The automatic driving obstacle recognition method according to claim 1, characterized in that: The step S6 comprises: Step S601, collecting image data in real time as input data; Step S602, inputting the input data into a feature extraction model to extract key features; Step S603, extracting the feature vector of the key feature and sending it to the data dimensionality reduction model to reduce the data dimension; Step S604, the feature vector after dimension reduction is sent to the classifier model; Step S605: According to steps S601 to S604, the classifier outputs the recognition result of the marked obstacle category.

7. The automatic driving obstacle recognition method according to claim 1, characterized in that: The step S205 further includes: After each iteration, the parameters of the generator are adjusted according to the result of the feature enhancement loss function, and the generated images focus on simulating the key features of obstacles in the next round of training; Repeat the iteration until the preset stop iteration condition is met and the model training is completed; The preset stopping condition is when a preset maximum number of iterations is reached.

8. The automatic driving obstacle recognition method according to claim 1, characterized in that: The step S203 further includes: α d Set to 0.

005.

9. The automatic driving obstacle recognition method according to claim 1, characterized in that: The step S204 further includes: α g Set to 0.

01.

10. An automatic driving obstacle recognition device, characterized in that: The automatic driving obstacle recognition device comprises: A data acquisition module is used to collect image data and annotate the data based on visual recognition; Data expansion module, used to generate image samples and expand data based on the generative adversarial network algorithm of visual simulation; A feature extraction module is used to input the expanded data into a feature extraction model to train the feature extraction model; The feature dimension reduction module is used to input the feature extracted data into the feature dimension reduction model to train the feature dimension reduction model; The classification training module is used to input the reduced-dimensional data into the classifier for classifier training. The obstacle recognition module is used to perform visual obstacle recognition based on the trained feature extraction model, data dimension reduction model and classifier model; Wherein, the step of generating image samples based on a visual simulation generative adversarial network algorithm and expanding data includes: the visual simulation generative adversarial network algorithm includes a generator and a discriminator; Further comprising: step S201, initializing parameters of the generator and the discriminator; Step S202, the generator selects samples from the image data with obstacles marked, for training the discriminator to identify the authenticity of obstacles in the image data; Step S203, the discriminator receives the image data from the vehicle-mounted camera and the image sample data generated by the generative adversarial network algorithm, and performs discriminator training; Step S204, after the discriminator is trained, the generator updates parameters according to the feedback from the discriminator; Step S205, obtaining a feature enhancement loss function, inputting the current update parameters and continuously iterating; After each parameter update iteration, the parameters of the generator are adjusted according to the result of the feature enhancement loss function; Wherein, step S203 includes, the discriminator D receives the real image x and the generated image And by optimizing its parameter θ d To improve the accuracy of classification, it is expressed as: Where, L D is the loss function of the discriminator; D(x;θ d ) represents the probability that the discriminator outputs the real image x as real; α d is the learning rate; is the loss function with respect to θ d The gradient of It also includes the discriminator output D(x; θ d ) is calculated as: D(x;θ d )=Sig(W d ·f(x)+b d ); Where Sig() is the sigmoid activation function; W d is the weight matrix of the discriminator; f(x) is the feature representation of the input image x, obtained through a series of convolutional layers; b d is the bias term; Also included: The calculation method of f(x) is expressed as: f(x)=ReLU(W f ·x+b f ): Where ReLU() is the ReLU nonlinear activation function; W f is the feature extraction weight; b f is the feature extraction bias; Among them, step S204 includes, after the discriminator is trained, the generator G updates the parameter θ according to the feedback of the discriminator D g , using feature enhancement loss function L G , expressed as: Where, L G is the loss function of the generator; It is a feature enhancement loss, emphasizing the generation of images The similarity between the key features of the real image x and λ f is the weight factor of feature enhancement; α g is the learning rate; is the loss function with respect to θ g The gradient of Also includes, feature enhancement loss function It can strengthen the similarity of key features, and the calculation method is expressed as: Where F(x) is a feature extraction function used to extract high-level features of the image; ∥·∥2 represents the Euclidean distance; Wherein, the step S201 further includes: The parameters θ of the generator G and the discriminator D g and θ d They are initialized separately, and the initialization is implemented using the standard normal distribution, which is expressed as: In the formula, θ g is the parameter of the generator; θ d are the parameters of the discriminator; Represents a normal distribution with a mean of 0 and a variance of the identity matrix; Wherein, the step S202 further includes: Select samples from labeled obstacle images The sample is a real obstacle image, which is used to train the discriminator to identify true and false images. (i) represents the i-th real image sample; The generator generates fake images by sampling from the noise distribution according to the current parameters; The generator G is based on the input noise z and the network parameters θ g Generate fake images It is expressed as: z~p(z); In the formula, represents the generated fake image; G(z;θ g ) represents the generator function, z is the noise vector sampled from the noise distribution p(z); Wherein, the step S204 further includes: F(x) is defined by a series of pre-trained convolutional network layers and is expressed as: F(x)=GlobalAvgPool(ReLU(W F ·x+b F )) Where GlobalAvgPool() is the global average pooling layer function, which is used to reduce the feature dimension and retain important information; W F and b F are the weights and biases of the feature extraction network, respectively.

11. An electronic device, characterized in that: include: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; A computer program is stored in the memory. When the computer program is executed by the processor, the processor executes the steps of the autonomous driving obstacle identification method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: include: It stores a computer program that can be executed by an electronic device. When the computer program runs on the electronic device, the electronic device executes the steps of the autonomous driving obstacle identification method described in any one of claims 1 to 9.

13. A vehicle, characterized in that: include: An electronic device, used to implement the steps of the automatic driving obstacle recognition method according to any one of claims 1 to 9; A processor, wherein the processor runs a program, and when the program runs, data output from the electronic device executes the steps of the method for identifying obstacles in autonomous driving according to any one of claims 1 to 9; A storage medium for storing a program, which, when running, executes the steps of the autonomous driving obstacle recognition method according to any one of claims 1 to 9 for data output from an electronic device.

Images