A near-collision detection method, device, terminal device and storage medium for a vehicle

Abstract

The present application provides a near-collision detection method, apparatus, terminal device and storage medium for vehicles, and the method includes: obtaining an initial training model set and a training sample set; learning and training each initial training model in the initial training model set based on the training sample set to obtain a near-collision detection model set corresponding to the initial training model set; obtaining an image to be detected, and determining a target vehicle candidate near-collision detection model that best matches the image to be detected from the near-collision detection model set, and determining the target vehicle candidate model parameters; obtaining other vehicle candidate model parameters corresponding to other vehicle candidate near-collision detection models sent by other vehicles in the vehicle cluster; model aggregation of the target vehicle candidate model parameters and other vehicle candidate model parameters to obtain a federated near-collision detection model; using the federated near-collision detection model to perform near-collision detection on the image to be detected to obtain a near-collision detection result. The method can improve the accuracy of near-collision detection.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a vehicle near-collision detection method, apparatus, terminal device and storage medium. Background Art

[0002] The development of intelligent transportation systems is a direct response to the modern society's need to improve road safety, traffic efficiency and environmental friendliness. In intelligent transportation systems, vehicle near-collision detection is an important function, which specifically refers to the detection of potential collision risks through sensors, cameras, radars and other technical means when a traffic accident is about to occur, and timely warnings are issued to avoid accidents or reduce the severity of accidents.

[0003] Traditional near-collision detection methods usually rely on a single deep learning model, but different models may show different effectiveness in handling specific types of abnormal situations. Some models may perform well on certain abnormal patterns but poorly on other abnormal patterns. Therefore, the accuracy of near-collision detection based on current technical solutions is low.

[0004] Therefore, how to improve the accuracy of near-collision detection of vehicles is a technical problem that technical personnel in this field currently need to solve. Summary of the invention

[0005] The purpose of the present application is to provide a vehicle near-collision detection method, apparatus, terminal device and computer-readable storage medium, aiming to improve the accuracy of vehicle near-collision detection.

[0006] In a first aspect, the present application provides a vehicle near-collision detection method, which is applied to any target vehicle in a vehicle cluster; the method comprises:

[0007] Obtain an initial training model set and a training sample set;

[0008] Performing learning training on each initial training model in the initial training model set based on the training sample set to obtain a near collision detection model set corresponding to the initial training model set;

[0009] Acquire an image to be detected, and determine a target vehicle candidate near-collision detection model that best matches the image to be detected from the near-collision detection model set, and determine target vehicle candidate model parameters;

[0010] Obtaining other vehicle candidate model parameters corresponding to other vehicle candidate near collision detection models sent by other vehicles in the vehicle cluster;

[0011] Performing model aggregation on the target vehicle candidate model parameters and the other vehicle candidate model parameters to obtain a federated near collision detection model;

[0012] The federated near collision detection model is used to perform near collision detection on the image to be detected to obtain a near collision detection result.

[0013] In one embodiment, the step of performing model aggregation on the target vehicle candidate model parameters and the other vehicle candidate model parameters to obtain a federated near collision detection model includes:

[0014] Determining a target model group corresponding to the target vehicle;

[0015] If the number of models corresponding to the candidate model parameters in the target model group is greater than a preset value, model aggregation is performed according to each of the candidate model parameters in the target model group to obtain a local federated near-collision detection model; the candidate model parameters include the target vehicle candidate model parameters and the other vehicle candidate model parameters;

[0016] Otherwise, model aggregation is performed according to all the candidate model parameters to obtain a global federated near collision detection model.

[0017] In one embodiment, determining the target model group corresponding to the target vehicle includes:

[0018] Classifying and aggregating the candidate model parameters to obtain multiple model groups;

[0019] The model group corresponding to the target vehicle candidate model parameters is determined as the target model group.

[0020] In one embodiment, the candidate model parameters are classified and aggregated to obtain multiple model groups, including:

[0021] The candidate model parameters are classified and aggregated according to the optimization model algorithm to obtain multiple model groups.

[0022] In one embodiment, the acquiring of the image to be detected, determining the target vehicle candidate near-collision detection model that best matches the image to be detected from the near-collision detection model set, and determining the target vehicle candidate model parameters include:

[0023] Acquire the image to be detected;

[0024] Determine a similar sample image set corresponding to the image to be detected from the training sample set;

[0025] According to the loss values ​​respectively corresponding to each similar sample image in the similar sample image set and each near-collision detection model in the near-collision detection model set, a target vehicle candidate near-collision detection model that best matches the image to be detected is determined, and target vehicle candidate model parameters are determined.

[0026] In one embodiment, determining a similar sample image set corresponding to the image to be detected from the training sample set includes:

[0027] Determine the image similarity corresponding to the image to be detected and each sample image in the training sample set by using a similarity measurement network; the similarity measurement network is based on a deep neural network and is trained by using similarity training samples, the similarity training samples are images with label information, and the label information represents the similarity between the image and the target image; the image includes an original image and an enhanced image obtained by performing data enhancement processing based on the original image;

[0028] According to the similarities of each of the images, a similar sample image set corresponding to the image to be detected is determined from the training sample set based on a k-nearest neighbor algorithm.

[0029] In one embodiment, the step of performing model aggregation on the target vehicle candidate model parameters and the other vehicle candidate model parameters to obtain a federated near collision detection model includes:

[0030] Determine the amount of model training data corresponding to the target vehicle candidate model parameters and the other vehicle candidate model parameters, respectively, and determine the corresponding importance weights according to the amount of model training data;

[0031] Model aggregation is performed according to the target vehicle candidate model parameters, the other vehicle candidate model parameters and the importance weights to obtain a federated near-collision detection model.

[0032] In a second aspect, the present application also provides a vehicle near-collision detection device, which is applied to any target vehicle in a vehicle cluster; the device comprises:

[0033] A first acquisition module is used to acquire an initial training model set and a training sample set;

[0034] A training module, configured to perform learning training on each initial training model in the initial training model set based on the training sample set, so as to obtain a near collision detection model set corresponding to the initial training model set;

[0035] A screening module is used to obtain an image to be detected, and determine a target vehicle candidate near-collision detection model that best matches the image to be detected from the near-collision detection model set, and determine target vehicle candidate model parameters;

[0036] A second acquisition module is used to acquire other vehicle candidate model parameters corresponding to other vehicle candidate near collision detection models sent by other vehicles in the vehicle cluster;

[0037] an aggregation module, configured to perform model aggregation on the target vehicle candidate model parameters and the other vehicle candidate model parameters to obtain a federated near collision detection model;

[0038] The detection module is used to perform near collision detection on the image to be detected by using the federated near collision detection model to obtain a near collision detection result.

[0039] In a third aspect, the present application further provides a terminal device, wherein the terminal device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0040] In a fifth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0041] The embodiment of the present application provides a near-collision detection method for a vehicle, which is applied to any target vehicle in a vehicle cluster; the target vehicle first learns and trains each initial training model in the initial training model set based on the training sample set to obtain a near-collision detection model set corresponding to the initial training model set; then, an image to be detected is obtained, and a target vehicle candidate near-collision detection model that best matches the image to be detected is determined from the near-collision detection model set, the target vehicle candidate model parameters are determined, and other vehicle candidate model parameters corresponding to the candidate near-collision detection models of other vehicles sent by other vehicles in the vehicle cluster are obtained; a federal near-collision detection model is determined by model aggregation of the target vehicle candidate model parameters and the candidate model parameters of other vehicles; after each vehicle in the vehicle cluster has determined the candidate near-collision detection model that best matches its own image to be detected, the method performs model aggregation on the candidate model parameters corresponding to each candidate near-collision detection model to obtain a federal near-collision detection model, that is, the federal near-collision detection model can integrate the advantages of the most matching candidate near-collision detection models determined by each vehicle, so that near-collision detection can be performed more accurately on the image to be detected. Compared with using a single deep learning model for near-collision detection, the method can improve the accuracy of near-collision detection.

[0042] It can be understood that the vehicle near-collision detection device, terminal equipment and computer-readable storage medium provided in the embodiments of the present application have the same beneficial effects as the above-mentioned vehicle near-collision detection method, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A flowchart of a vehicle near-collision detection method provided in an embodiment of the present application;

[0045] Figure 2 In another embodiment of the present application, S300: acquiring an image to be detected, and determining a candidate near-collision detection model of a target vehicle that best matches the image to be detected from a near-collision detection model set;

[0046] Figure 3 A schematic structural diagram of a near-collision detection device for a vehicle provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0049] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0050] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0051] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0052] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0053] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. "Multiple" means "two or more".

[0054] A vehicle near-collision detection method provided in an embodiment of the present application can be executed by a terminal device of any target vehicle in a vehicle cluster when running a corresponding computer program.

[0055] Figure 1 This is a flow chart of a near-collision detection method for a vehicle provided in an embodiment of the present application. For ease of explanation, only the part related to the present embodiment is shown. The method provided in the present embodiment includes the following steps:

[0056] S100: Obtain an initial training model set and a training sample set.

[0057] Among them, the initial training model set includes multiple initial training models; the initial training model can be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory neural network (LSTM), a feed-forward neural network (FNNs), a variational autoencoder (VAE), a generative adversarial network (Visual Transformer), etc. This embodiment does not limit the specific type of the initial training model.

[0058] Specifically, you can choose a set of deep learning models As the initial training model set; the elements in the initial training model set represents the initial training model; Indicates the model number of the initial training model in the initial training model set.

[0059] Among them, the training sample set refers to the sample set used for model training, which can be expressed as ; Among them, the elements in the training sample set represents the training sample, Indicates the number of training samples in the training sample set.

[0060] Specifically, image samples can come from video streams, accident scene photos, etc. in the IoV environment; by setting label information for image samples, corresponding training samples are obtained.

[0061] It should be noted that for different types of label information, the model type of the near collision detection model trained is different; for example, if the label type is a classification label, the near collision detection model trained is a classification model, and if the label type is a collision probability, the near collision detection model trained is a prediction model. In actual applications, the type of label information can be determined according to actual needs, and this embodiment does not limit this.

[0062] S200: Performing learning training on each initial training model in the initial training model set based on the training sample set to obtain a near collision detection model set corresponding to the initial training model set.

[0063] In this step, for each initial training model, each training sample in the training sample set is used Initial training model Conduct learning and training to obtain the corresponding near-collision detection model.

[0064] After learning and training each initial training model in the initial training model set, a near collision detection model corresponding to each initial training model is obtained, and the near collision detection model set is determined.

[0065] After the near-collision detection model is trained, the model parameters corresponding to the near-collision detection model are determined; model parameters refer to the parameters used to describe the near-collision detection model, including the model type, model structure, and parameters such as the weights and biases that constitute the near-collision detection model; model parameters are adjusted during the training process to minimize the loss function of the model, thereby improving the prediction performance of the near-collision detection model. Among them, the weight determines the connection strength from one neuron to another in the near-collision detection model; the bias is a constant term added to the output of the near-collision detection model, which is used to adjust the output of the activation function so that the near-collision detection model can better fit the data.

[0066] S300: Acquire an image to be detected, determine a target vehicle candidate near-collision detection model that best matches the image to be detected from a near-collision detection model set, and determine target vehicle candidate model parameters.

[0067] The image to be detected refers to the road condition image collected by the target vehicle during driving; by performing near collision detection on the image to be detected, it is determined whether the target vehicle will have a near collision risk. The candidate near collision detection model of the target vehicle is the near collision detection model determined by the target vehicle from the near collision detection model set trained by itself.

[0068] Based on the near-collision detection model set determined in the previous step, this step, after determining the image to be detected, determines the target vehicle candidate near-collision detection model that best matches the image to be detected from the near-collision detection model set, and determines the model parameters corresponding to the target vehicle candidate near-collision detection model as the target vehicle candidate model parameters.

[0069] S400: Acquire other vehicle candidate model parameters corresponding to other vehicle candidate near collision detection models sent by other vehicles in the vehicle cluster.

[0070] For each other vehicle in the vehicle cluster, the above steps are performed respectively to determine the candidate near collision detection model of the other vehicle corresponding to itself, and determine the candidate model parameters of the other vehicle corresponding to the candidate near collision detection model of the other vehicle. In other words, the candidate near collision detection model of the other vehicle is the near collision detection model determined by the other vehicle from the near collision detection model set trained by itself; the candidate model parameters of the other vehicle are the model parameters corresponding to the candidate near collision detection model of the other vehicle.

[0071] The other vehicles send the determined candidate model parameters of the other vehicles to the target vehicle; for the target vehicle, the candidate model parameters of the other vehicles sent by the other vehicles in the vehicle cluster are obtained.

[0072] In a specific example, other vehicles in the vehicle cluster may send other vehicle candidate model parameters to the target vehicle based on vehicle-to-vehicle communication technology (V2V).

[0073] S500: Model aggregation is performed on the target vehicle candidate model parameters and other vehicle candidate model parameters to obtain a federated near collision detection model.

[0074] Among them, model aggregation refers to integrating the model parameters corresponding to the candidate near-collision detection models of each vehicle in the vehicle cluster to form a model that includes the model features of each candidate near-collision detection model; that is, model aggregation is performed on the candidate model parameters of the target vehicle and the candidate model parameters of other vehicles to obtain a federal near-collision detection model.

[0075] In this embodiment, model aggregation can be achieved by transmitting model parameters only, without transmitting the entire near-collision detection model or transmitting the training sample sets of other vehicles, so that the near-collision detection models of each vehicle in the comprehensive vehicle cluster can be aggregated to obtain a federal near-collision detection model; compared with each other vehicle sending a training sample set to the target vehicle, and training the target vehicle according to each training sample set to obtain the near-collision detection model finally used for near-collision detection, this method avoids transmitting original data and can ensure data privacy and security.

[0076] S600: Perform near collision detection on the image to be detected by using the federated near collision detection model to obtain a near collision detection result.

[0077] After the federated near collision detection model is determined, the image to be detected is input into the federated near collision detection model for near collision detection, and the near collision detection result is determined according to the output of the federated near collision detection model.

[0078] Specifically, the output type is determined according to the model type of the federal near-collision detection model; if the federal near-collision detection model is a judgment model, the near-collision detection result is the judgment result of "whether a near-collision danger occurs"; if the federal near-collision detection model is a prediction model, the near-collision detection result is the "probability of a near-collision danger occurring".

[0079] The embodiment of the present application provides a near-collision detection method for a vehicle, which is applied to any target vehicle in a vehicle cluster; the target vehicle first learns and trains each initial training model in the initial training model set based on the training sample set to obtain a near-collision detection model set corresponding to the initial training model set; then, an image to be detected is obtained, and a target vehicle candidate near-collision detection model that best matches the image to be detected is determined from the near-collision detection model set, the target vehicle candidate model parameters are determined, and other vehicle candidate model parameters corresponding to the candidate near-collision detection models of other vehicles sent by other vehicles in the vehicle cluster are obtained; a federal near-collision detection model is determined by model aggregation of the target vehicle candidate model parameters and the candidate model parameters of other vehicles; after each vehicle in the vehicle cluster has determined the candidate near-collision detection model that best matches its own image to be detected, the method performs model aggregation on the candidate model parameters corresponding to each candidate near-collision detection model to obtain a federal near-collision detection model, that is, the federal near-collision detection model can integrate the advantages of the most matching candidate near-collision detection models determined by each vehicle, so that near-collision detection can be performed more accurately on the image to be detected. Compared with using a single deep learning model for near-collision detection, the method can improve the accuracy of near-collision detection.

[0080] Based on the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, the target vehicle candidate model parameters and other vehicle candidate model parameters are model aggregated to obtain a federated near-collision detection model, including:

[0081] Determine a target model group corresponding to the target vehicle;

[0082] If the number of models corresponding to the candidate model parameters in the target model group is greater than a preset value, the model is aggregated according to the candidate model parameters in the target model group to obtain a local federated near-collision detection model; the candidate model parameters include the candidate model parameters of the target vehicle and the candidate model parameters of other vehicles;

[0083] Otherwise, model aggregation is performed based on all candidate model parameters to obtain a global federated near-collision detection model.

[0084] The target model group refers to the model group where the target vehicle is located. In this embodiment, each candidate model parameter is classified and aggregated to obtain multiple model groups; after the multiple model groups are determined, the target model group corresponding to the candidate model parameters of the target vehicle is determined, that is, the target model group where the candidate model parameters of the target vehicle corresponding to the target vehicle are located is determined.

[0085] Then determine whether the number of models corresponding to the candidate model parameters in the target model group is greater than the preset value; the preset value can be set according to actual needs, such as setting it to 1, indicating whether the target vehicle candidate model parameters corresponding to the target vehicle are in a single model group; if the number of models corresponding to the candidate model parameters in the target model group is greater than the preset value, it means that the target model group corresponding to the target vehicle also includes other candidate model parameters, so the model is aggregated according to each candidate model parameter in the target model group to obtain a local federal near-collision detection model. In other words, this step is to use the local federal near-collision detection model as the federal near-collision detection model when the number of models corresponding to the candidate model parameters in the target model group is greater than the preset value, and use the local federal near-collision detection model to perform near-collision detection on the image to be detected to obtain a near-collision detection result.

[0086] Otherwise, that is, the number of models corresponding to the candidate model parameters in the target model group is less than or equal to the preset value, indicating that the target model group corresponding to the target vehicle only includes the target vehicle candidate model parameters, and the target vehicle candidate model parameters corresponding to the target vehicle are in a single model group. Therefore, the model is aggregated according to all the candidate model parameters to obtain a global federated near-collision detection model. In other words, in this step, when the number of models corresponding to the candidate model parameters in the target model group is less than or equal to the preset value, all the candidate model parameters are aggregated to obtain a global federated near-collision detection model; the global federated near-collision detection model is used as the federated near-collision detection model, and the global federated near-collision detection model is used to perform near-collision detection on the image to be detected to obtain a near-collision detection result.

[0087] In this embodiment, local aggregation or global aggregation is performed based on the judgment result of whether the target vehicle candidate model parameters corresponding to the target vehicle are alone in a model group, and the corresponding local federal near-collision detection model or global federal near-collision detection model is used to perform near-collision detection on the image to be detected, which can determine the near-collision detection result more efficiently and accurately.

[0088] Based on the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, determining the target model group corresponding to the target vehicle includes:

[0089] Classify and aggregate the parameters of each candidate model to obtain multiple model groups;

[0090] The model group corresponding to the target vehicle candidate model parameters is determined as the target model group.

[0091] It can be understood that the purpose of classification aggregation is to classify candidate model parameters with similar characteristics into one category to form different model groups. The similarity of candidate model parameters in the same model group is high; the similarity of candidate model parameters in different model groups is low.

[0092] In this embodiment, the similarity between each candidate model parameter can be calculated by a similarity algorithm, and the candidate model parameters with a similarity greater than a preset similarity threshold are determined to belong to the same model group. After calculating each candidate model parameter separately, multiple model groups are determined. It is also possible to classify and aggregate each candidate model parameter according to the similarity of the candidate model parameters through cluster analysis, such as K-means, hierarchical clustering, etc., to obtain multiple model groups. In addition, it is also possible to classify and aggregate each candidate model parameter through an optimization model algorithm to obtain multiple model groups.

[0093] After a plurality of model groups are determined, a model group in which the target vehicle candidate model parameters corresponding to the target vehicle are located is determined, and the model group is determined as the target model group.

[0094] According to the method of this embodiment, by classifying and aggregating the candidate model parameters, multiple model groups are obtained, and the model group corresponding to the candidate model parameters of the target vehicle is determined as the target model group; when it is subsequently determined whether the number of models corresponding to the candidate model parameters in the target model group is greater than a preset value, the corresponding local aggregation or global aggregation operation is performed to determine the local federal near-collision detection model or the global federal near-collision detection model, which can more efficiently and accurately determine the near-collision detection results.

[0095] Based on the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, the parameters of each candidate model are classified and aggregated to obtain multiple model groups, including:

[0096] According to the optimization model algorithm, the parameters of each candidate model are classified and aggregated to obtain multiple model groups.

[0097] In a specific embodiment, after receiving the candidate model parameters of other vehicles respectively sent by other vehicles, the target vehicle determines a set of candidate model parameters corresponding to the vehicle cluster, which is expressed as ;in, u i Indicates and vehicles i corresponding candidate model parameters of the candidate near collision detection model; It represents the number of vehicles in the vehicle cluster, that is, the number of candidate models of the candidate near-collision detection model.

[0098] Determine the number of groups of model groups; assuming that the candidate near collision detection models are divided into Model groups, represented by ;in, Represents a model group;

[0099] Classification aggregation is achieved by optimizing the model:

[0100] Determine decision variables based on attribution identification x ij ; x ij is a binary variable, indicating the candidate model parameters corresponding to the candidate near collision detection model Belongs to the model group ;like x ij =1, indicating the candidate model parameters of the candidate near collision detection model Belongs to model group ;like x ij =0, indicating the candidate model parameters corresponding to the candidate near collision detection model Does not belong to the model group .

[0101] The constraint condition is determined as follows: each candidate near collision detection model belongs to only one model group, that is:

[0102] ;

[0103] The optimization goal is determined as follows: the optimization goal is to minimize the model similarity of candidate near-collision detection models in different model groups, and to maximize the model similarity of each candidate near-collision detection model in the same model group; the model similarity between candidate near-collision detection models is represented by the similarity between candidate model parameters;

[0104] Specifically, using Represents candidate model parameters and candidate model parameters The similarity between them means that the vehicles i and vehicles j The model similarity of the corresponding candidate near collision detection models.

[0105] The corresponding objective function is set as:

[0106] ;

[0107] After the optimized model is determined, the classification of each candidate near-collision detection model is continuously updated through an iterative clustering algorithm until the clustering results converge and the model group is determined.

[0108] This embodiment classifies and aggregates the candidate model parameters according to the optimization model algorithm, avoiding the calculation process of calculating the similarity between each candidate model parameter, that is, avoiding a large number of calculation processes, so that the model group can be determined more efficiently and accurately, thereby improving the efficiency and accuracy of vehicle near-collision detection.

[0109] Figure 2 In another embodiment of the present application, S300 is as follows: Based on the above embodiment, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, an image to be detected is obtained, and a target vehicle candidate near-collision detection model that best matches the image to be detected is determined from a near-collision detection model set, and target vehicle candidate model parameters are determined, including:

[0110] S310: Acquire an image to be detected;

[0111] S320: Determine a similar sample image set corresponding to the image to be detected from the training sample set;

[0112] S330: Determine a target vehicle candidate near-collision detection model that best matches the image to be detected, and determine target vehicle candidate model parameters, based on the loss values ​​corresponding to each similar sample image in the similar sample image set and each near-collision detection model in the near-collision detection model set.

[0113] In this embodiment, after obtaining the image to be detected, the image similarity between the image to be detected and each sample image in the training sample set is first determined. Specifically, the image features of the image to be detected are extracted, and the image similarity between the image to be detected and each sample image is calculated based on the image features. The image features can be the intermediate layer output of the near collision detection model or can be a custom feature, which is not limited in this embodiment.

[0114] In a specific example, the image similarity between the image to be detected and the sample image can be calculated according to a similarity algorithm; the similarity algorithm includes Euclidean distance and cosine similarity; in addition, considering the complexity of data drift and image data distribution, the image similarity between the image to be detected and each sample image can be calculated through a similarity metric network.

[0115] Determine the number of similar images, and determine a similar image set from the training sample set according to the number of similar images, that is, the similar sample image set includes similar sample images of a preset number of similar images; the similar sample images are sample images similar to the image to be detected. For example, if the number of similar images is 5, then determine the 5 training samples most similar to the image to be detected from the training sample set as the similar image set.

[0116] After determining the similar image set, determine the loss values ​​corresponding to each similar sample image in the similar image set and each near collision detection model in each near collision detection model set; for example, for the similar sample image (Sample image corresponding to the training sample), which is used in the initial training model After training and finally determining the near collision detection model, the similar sample image Initial training model The loss value is expressed as .

[0117] It should be noted that the larger the loss value, the worse the data quality of the similar sample images for training the near collision detection model, and the near collision detection model has a poorer processing effect on the similar sample images; the smaller the loss value, the better the data quality of the similar sample images for training the near collision detection model, and the near collision detection model has a better processing effect on the similar sample images.

[0118] Therefore, for each similar sample image, it has a corresponding loss value with each near-collision detection model. The near-collision detection model with the lowest loss value is determined as the target vehicle candidate near-collision detection model, and the target vehicle candidate near-collision detection model is the near-collision detection model that best matches the image to be detected; then, the target vehicle candidate model parameters are determined based on the model parameters of the target vehicle candidate near-collision detection model.

[0119] According to the method of this embodiment, the candidate near-collision detection model of the target vehicle that best matches the image to be detected can be determined efficiently and accurately, thereby improving the accuracy and efficiency of near-collision detection of the vehicle.

[0120] In a specific embodiment, determining a similar sample image set corresponding to the image to be detected from the training sample set includes:

[0121] A similarity measurement network is used to determine the image similarity corresponding to the image to be detected and each sample image in the training sample set; the similarity measurement network is based on a deep neural network and is trained using similarity training samples, and the similarity training samples are images with label information, and the label information represents the similarity between the image and the target image; the image includes an original image and an enhanced image obtained by performing data enhancement processing based on the original image;

[0122] According to the similarity of each image, a similar sample image set corresponding to the image to be detected is determined from the training sample set based on the k-nearest neighbor algorithm.

[0123] Specifically, similarity training samples are determined in advance, and a similarity metric network (Similarity Metric Network) is trained based on a deep neural network using the similarity training samples.

[0124] The similarity training samples are images with label information, the label information indicates the similarity between the image and the target image, and the images include original images and enhanced images obtained by performing data enhancement processing based on the original images.

[0125] The similarity measurement network is a fully connected neural network designed to evaluate the similarity between two images. The optimization of the similarity measurement network is performed through back propagation, which involves evaluating the proxy loss of the similarity network. In order to measure the image similarity between the image to be tested and the sample image, a fully connected neural network with a single hidden layer is used. The input includes the feature vector of the image to be tested. F new and the feature vector of the sample image F Ii . Similarity measurement function S(Fnew,F Ii ) It can be expressed as:

[0126] ;

[0127] Among them, C represents the coefficient of the similarity measurement function, is an activation function, usually a sigmoid function, which is used to constrain the output to be between 0 and 1. The feature vector representing the image to be detected F new and the feature vector of the sample image F Ii series connection.

[0128] In the process of training the similarity measurement network, the loss value in the training process is determined by smooth similarity loss learning. Smooth similarity loss learning is a loss function used in the deep learning training process. It aims to measure the difference between the similarity predicted by the model and the true similarity, and optimize the model parameters by minimizing this difference. Then, based on the calculated loss value, the gradient is calculated by the back propagation algorithm, and the model parameters are updated using an optimizer (such as SGD or Adam) to reduce the loss value and improve the image similarity prediction accuracy of the model.

[0129] After obtaining the image to be detected, a pre-trained similarity measurement network is obtained, the image to be detected and the sample image in the training sample set are input into the similarity measurement network, and the image similarity between the image to be detected and the sample image is output using the similarity measurement network.

[0130] After outputting the corresponding image similarity for each sample image in the training sample set, a similar sample image set corresponding to the image to be detected is determined from the training sample set based on the k-nearest neighbor algorithm (K-NN) according to the similarity of each image; k is a pre-set integer, indicating the number of sample images most similar to the image to be detected determined from the training sample set, that is, the number of similar images.

[0131] According to the method of this embodiment, a similar sample image set corresponding to the image to be detected can be determined from the training sample set efficiently and accurately, so that model aggregation can be performed more efficiently and accurately, thereby improving the efficiency and accuracy of vehicle near-collision detection.

[0132] Based on the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, the target vehicle candidate model parameters and other vehicle candidate model parameters are model aggregated to obtain a federated near-collision detection model, including:

[0133] Determine the amount of model training data corresponding to the target vehicle candidate model parameters and other vehicle candidate model parameters, and determine the corresponding importance weights according to the amount of model training data;

[0134] Model aggregation is performed according to the target vehicle candidate model parameters, other vehicle candidate model parameters and importance weights to obtain a federated near-collision detection model.

[0135] It is understandable that when training a near-collision detection model, the larger the amount of model training data, the higher the accuracy of the trained near-collision detection model is generally; that is, the larger the amount of data in the training sample set corresponding to the near-collision detection model, the higher the corresponding importance weight of the near-collision detection model.

[0136] In this embodiment, the amount of model training data corresponding to the target vehicle candidate model parameters and the other vehicle candidate model parameters is determined, and the corresponding importance weights are determined according to the model training data amount; then, model aggregation is performed according to the target vehicle candidate model parameters, the other vehicle candidate model parameters and the importance weights to obtain a federal near-collision detection model.

[0137] In a specific embodiment, if the number of models corresponding to the candidate model parameters in the target model group is greater than a preset value, model aggregation is performed according to the candidate model parameters in the target model group to obtain a local federated near collision detection model. Specifically, model aggregation is performed based on the following formula to obtain a local federated near collision detection model:

[0138] ;

[0139] in, Indicates the target model group All vehicles in iThe sum of the training sample sets of the corresponding near-collision detection model; Indicates vehicle i The corresponding training sample set size or importance weight of the near collision detection model; Indicates and vehicles i Corresponding candidate near-collision detection model parameters.

[0140] In another specific embodiment, if the number of models corresponding to the candidate model parameters in the target model group is less than or equal to a preset value, the model is aggregated according to all the candidate model parameters to obtain a global federated near collision detection model. Specifically, the model is aggregated based on the following formula to obtain a global federated near collision detection model:

[0141] ;

[0142] in, The model training data volume of the training sample set of the near collision detection model corresponding to all candidate model parameters in all model groups; Indicates vehicle i The importance weight of the training sample set of the corresponding near-collision detection model; Indicates and vehicles i Corresponding candidate near-collision detection model parameters.

[0143] According to the method of this embodiment, a more accurate local federal near collision detection model and a global federal near collision detection model can be determined, thereby improving the accuracy of near collision detection using the local federal near collision detection model or the global federal near collision detection model.

[0144] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0145] It should be noted that the information collection process (such as facial image collection process, fingerprint information collection process, etc.) / feature extraction process involved in this application is performed with the user's knowledge and permission, that is, the information collection process / feature extraction process complies with the requirements of laws and regulations and does not constitute an act that harms the public interest.

[0146] Figure 3 FIG. 1 is a schematic diagram of the structure of a near collision detection device for a vehicle provided in an embodiment of the present application. Figure 3 As shown, the vehicle near-collision detection device of this embodiment includes a first acquisition module, a training module, a screening module, a second acquisition module, an aggregation module and a detection module; wherein,

[0147] A first acquisition module 310 is used to acquire an initial training model set and a training sample set;

[0148] A training module 320, configured to perform learning training on each initial training model in the initial training model set based on the training sample set, to obtain a near collision detection model set corresponding to the initial training model set;

[0149] A screening module 330 is used to obtain an image to be detected, and determine a target vehicle candidate near-collision detection model that best matches the image to be detected from a near-collision detection model set, and determine target vehicle candidate model parameters;

[0150] The second acquisition module 340 is used to acquire other vehicle candidate model parameters corresponding to other vehicle candidate near collision detection models sent by other vehicles in the vehicle cluster;

[0151] Aggregation module 350, used for performing model aggregation on target vehicle candidate model parameters and other vehicle candidate model parameters to obtain a federated near collision detection model;

[0152] The detection module 360 ​​is used to perform near collision detection on the image to be detected by using the federated near collision detection model to obtain a near collision detection result.

[0153] A vehicle near-collision detection device provided in an embodiment of the present application has the same beneficial effects as the above-mentioned vehicle near-collision detection method.

[0154] In one embodiment, the aggregation module includes:

[0155] A target model group determination submodule is used to determine a target model group corresponding to a target vehicle;

[0156] A first model aggregation submodule is used to perform model aggregation according to each candidate model parameter in the target model group to obtain a local federated near-collision detection model if the number of models corresponding to the candidate model parameters in the target model group is greater than a preset value; the candidate model parameters include target vehicle candidate model parameters and other vehicle candidate model parameters;

[0157] The second model aggregation submodule is used to perform model aggregation according to all candidate model parameters to obtain a global federated near collision detection model if the number of models corresponding to the candidate model parameters in the target model group is less than or equal to a preset value.

[0158] In one embodiment, the target model group determination submodule includes:

[0159] A classification and aggregation unit, used to classify and aggregate the parameters of each candidate model to obtain multiple model groups;

[0160] The target determination unit is used to determine the model group corresponding to the target vehicle candidate model parameters as the target model group.

[0161] In one embodiment, the classification aggregation unit includes:

[0162] The classification and aggregation subunit is used to classify and aggregate the parameters of each candidate model according to the optimization model algorithm to obtain multiple model groups.

[0163] In one embodiment, the screening module includes:

[0164] The image acquisition submodule to be detected is used to acquire the image to be detected;

[0165] A similar sample image set determination submodule is used to determine a similar sample image set corresponding to the image to be detected from the training sample set;

[0166] The parameter determination submodule is used to determine the target vehicle candidate near-collision detection model that best matches the image to be detected, and determine the target vehicle candidate model parameters, based on the loss values ​​corresponding to each similar sample image in the similar sample image set and each near-collision detection model in the near-collision detection model set.

[0167] In one embodiment, the similar sample image set determination submodule includes:

[0168] An image similarity determination unit is used to determine the image similarities corresponding to the image to be detected and each sample image in the training sample set by using a similarity measurement network; the similarity measurement network is based on a deep neural network and is trained by using similarity training samples, and the similarity training samples are images with label information, and the label information represents the similarity between the image and the target image; the image includes an original image and an enhanced image obtained by performing data enhancement processing based on the original image;

[0169] The image set determination unit is used to determine a similar sample image set corresponding to the image to be detected from the training sample set based on the k-nearest neighbor algorithm according to the similarity of each image.

[0170] In one embodiment, the aggregation module includes:

[0171] A weight determination submodule is used to determine the amount of model training data corresponding to the target vehicle candidate model parameters and other vehicle candidate model parameters, and determine the corresponding importance weights according to the amount of model training data;

[0172] The third model aggregation submodule is used to perform model aggregation according to the target vehicle candidate model parameters, other vehicle candidate model parameters and importance weights to obtain a federal near-collision detection model.

[0173] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0174] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0175] Figure 4 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. Figure 4 As shown, the terminal device 400 of this embodiment includes a memory 401, a processor 402, and a computer program 403 stored in the memory 401 and executable on the processor 402; when the processor 402 executes the computer program 403, the steps in the above-mentioned near-collision detection method embodiments for each vehicle are implemented; or when the processor 402 executes the computer program 403, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0176] Exemplarily, the computer program 403 may be divided into one or more modules / units, one or more modules / units are stored in the memory 401, and are executed by the processor 402 to implement the method of the embodiment of the present application. One or more modules / units may be a series of computer program instruction segments that can perform specific functions, and the instruction segments are used to describe the execution process of the computer program 403 in the terminal device 400. For example, the computer program 403 may be divided into: a first acquisition module, a training module, a screening module, a second acquisition module, an aggregation module, and a detection module, and the specific functions of each module are as follows:

[0177] A first acquisition module is used to acquire an initial training model set and a training sample set;

[0178] A training module, used for training each initial training model in the initial training model set based on the training sample set to obtain a near collision detection model set corresponding to the initial training model set;

[0179] A screening module is used to obtain the image to be detected, and determine the target vehicle candidate near-collision detection model that best matches the image to be detected from the near-collision detection model set, and determine the target vehicle candidate model parameters;

[0180] A second acquisition module is used to acquire other vehicle candidate model parameters corresponding to other vehicle candidate near collision detection models sent by other vehicles in the vehicle cluster;

[0181] An aggregation module is used to perform model aggregation on the target vehicle candidate model parameters and other vehicle candidate model parameters to obtain a federated near collision detection model;

[0182] The detection module is used to perform near collision detection on the image to be detected by using the federated near collision detection model to obtain a near collision detection result.

[0183] In the application, the terminal device 400 can be a computing device such as a vehicle controller, a desktop computer, a notebook, a PDA, and a cloud server. The terminal device 400 can include but is not limited to a memory 401 and a processor 402. Those skilled in the art can understand that Figure 4 It is only an example of a terminal device and does not constitute a limitation of the terminal device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.; among which, the input and output devices may include cameras, audio acquisition / playback devices, display screens, etc.; the network access device may include a communication module for wireless communication with external devices.

[0184] In applications, the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0185] In applications, the memory can be an internal storage unit of a terminal device, such as a hard disk or memory of the terminal device; it can also be an external storage device of the terminal device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device; it can also include both the internal storage unit and the external storage device of the terminal device. The memory is used to store operating systems, applications, boot loaders, data, and other programs, such as program codes of computer programs. The memory can also be used to temporarily store data that has been output or is about to be output.

[0186] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0187] The present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, RandomAccess Memory), an electrical carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0188] A computer-readable storage medium provided in an embodiment of the present application has the same beneficial effects as the above-mentioned vehicle near-collision detection method.

[0189] Those of ordinary skill in the art will appreciate that the devices and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0190] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices, which can be electrical, mechanical or other forms.

[0191] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A vehicle near-collision detection method, characterized in that: Applied to any target vehicle in a vehicle cluster; the method comprises: Obtain an initial training model set and a training sample set; Performing learning training on each initial training model in the initial training model set based on the training sample set to obtain a near collision detection model set corresponding to the initial training model set; Acquire an image to be detected, and determine a target vehicle candidate near-collision detection model that best matches the image to be detected from the near-collision detection model set, and determine target vehicle candidate model parameters; the image to be detected is a road condition image collected by the target vehicle during driving; the acquisition of the image to be detected, and determining a target vehicle candidate near-collision detection model that best matches the image to be detected from the near-collision detection model set, and determining target vehicle candidate model parameters, include: acquiring the image to be detected; determining a similar sample image set corresponding to the image to be detected from the training sample set; determining a target vehicle candidate near-collision detection model that best matches the image to be detected according to loss values ​​corresponding to each similar sample image in the similar sample image set and each near-collision detection model in the near-collision detection model set, and determining target vehicle candidate model parameters; Obtaining other vehicle candidate model parameters corresponding to other vehicle candidate near collision detection models sent by other vehicles in the vehicle cluster; Performing model aggregation on the target vehicle candidate model parameters and the other vehicle candidate model parameters to obtain a federated near collision detection model; The federated near collision detection model is used to perform near collision detection on the image to be detected to obtain a near collision detection result.

2. The method according to claim 1, characterized in that The step of performing model aggregation on the target vehicle candidate model parameters and the other vehicle candidate model parameters to obtain a federated near collision detection model includes: Determining a target model group corresponding to the target vehicle; If the number of models corresponding to the candidate model parameters in the target model group is greater than a preset value, model aggregation is performed according to each of the candidate model parameters in the target model group to obtain a local federated near-collision detection model; the candidate model parameters include the target vehicle candidate model parameters and the other vehicle candidate model parameters; Otherwise, model aggregation is performed according to all the candidate model parameters to obtain a global federated near collision detection model.

3. The method according to claim 2, characterized in that The determining of the target model group corresponding to the target vehicle comprises: Classifying and aggregating the candidate model parameters to obtain multiple model groups; The model group corresponding to the target vehicle candidate model parameters is determined as the target model group.

4. The method according to claim 3, characterized in that The candidate model parameters are classified and aggregated to obtain multiple model groups, including: The candidate model parameters are classified and aggregated according to the optimization model algorithm to obtain multiple model groups.

5. The method according to claim 1, characterized in that The step of determining a similar sample image set corresponding to the image to be detected from the training sample set comprises: Determine the image similarity corresponding to the image to be detected and each sample image in the training sample set by using a similarity measurement network; the similarity measurement network is based on a deep neural network and is trained by using similarity training samples, the similarity training samples are images with label information, and the label information represents the similarity between the image and the target image; the image includes an original image and an enhanced image obtained by performing data enhancement processing based on the original image; According to the similarities of each of the images, a similar sample image set corresponding to the image to be detected is determined from the training sample set based on a k-nearest neighbor algorithm.

6. The method according to any one of claims 1 to 5, characterized in that: The step of performing model aggregation on the target vehicle candidate model parameters and the other vehicle candidate model parameters to obtain a federated near collision detection model includes: Determine the model training data amounts corresponding to the target vehicle candidate model parameters and the other vehicle candidate model parameters, respectively, and determine corresponding importance weights according to the model training data amounts; Model aggregation is performed according to the target vehicle candidate model parameters, the other vehicle candidate model parameters and the importance weights to obtain a federal near-collision detection model.

7. A near collision detection device for a vehicle, characterized in that: Applicable to any target vehicle in a vehicle cluster; the device comprises: A first acquisition module is used to acquire an initial training model set and a training sample set; A training module, configured to perform learning training on each initial training model in the initial training model set based on the training sample set, so as to obtain a near collision detection model set corresponding to the initial training model set; A screening module is used to obtain an image to be detected, and determine a target vehicle candidate near-collision detection model that best matches the image to be detected from the near-collision detection model set, and determine the target vehicle candidate model parameters; the image to be detected is a road condition image collected by the target vehicle during driving; the screening module includes: a submodule for obtaining an image to be detected, used to obtain an image to be detected; a submodule for determining a similar sample image set corresponding to the image to be detected from the training sample set; a submodule for determining a parameter for determining a target vehicle candidate near-collision detection model that best matches the image to be detected based on the loss values ​​corresponding to each similar sample image in the similar sample image set and each near-collision detection model in the near-collision detection model set, and determining the target vehicle candidate model parameters; a second acquisition module is used to obtain parameters of other vehicle candidate models corresponding to other vehicle candidate near-collision detection models sent by other vehicles in the vehicle cluster; an aggregation module, configured to perform model aggregation on the target vehicle candidate model parameters and the other vehicle candidate model parameters to obtain a federated near collision detection model; The detection module is used to perform near collision detection on the image to be detected by using the federated near collision detection model to obtain a near collision detection result.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images