Training methods, systems, equipment, and storage media for vehicle state recognition models

Abstract

This application discloses a training method, system, device, and storage medium for a vehicle state recognition model, belonging to the field of machine learning technology. It includes: a central server sending a first state recognition model to electronic devices corresponding to multiple car manufacturers; the electronic devices training the first state recognition model based on their local datasets and sending the first model parameters of the trained first state recognition model to the central server; the central server aggregating the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain second model parameters, generating a second state recognition model based on the second model parameters, and sending the second state recognition model to the electronic devices corresponding to the multiple car manufacturers; and the electronic devices fine-tuning the second state recognition model based on their local datasets to obtain a third state recognition model. This application can achieve state recognition model sharing while keeping the dataset confidential, breaking down the data barriers to training state recognition models.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a training method, system, device and storage medium for a vehicle state recognition model. Background Technology

[0002] Currently, new energy vehicles often deploy state recognition models. These models identify objects in front of the vehicle and make state judgments based on these objects. The new energy vehicle then makes decisions based on these state judgments. For example, if the state judgment indicates that there is an obstacle one meter in front of the new energy vehicle, the vehicle may decide to brake or change lanes to improve user safety. Summary of the Invention

[0003] This application provides a training method, system, device, and storage medium for a vehicle state recognition model, which enables the sharing of the state recognition model while maintaining the confidentiality of the dataset, breaking down the data barrier for training the state recognition model. The technical solution is as follows:

[0004] On the one hand, a method for training a vehicle state recognition model is provided, the method comprising:

[0005] The central server sends the first state recognition model to the electronic devices of multiple car manufacturers.

[0006] For any electronic device corresponding to any car manufacturer, the electronic device receives a first state recognition model sent by the central server, trains the first state recognition model based on the local dataset, and sends the first model parameters of the first state recognition model trained by the central server.

[0007] The central server receives first model parameters sent by electronic devices corresponding to the multiple car manufacturers, aggregates the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain second model parameters, generates a second state recognition model based on the second model parameters, and sends the second state recognition model to the electronic devices corresponding to the multiple car manufacturers.

[0008] For any electronic device corresponding to any car manufacturer, the electronic device receives the second state recognition model and fine-tunes the second state recognition model based on the local dataset to obtain a third state recognition model.

[0009] In one possible implementation, the central server aggregates the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain second model parameters, including:

[0010] The central server determines the average value of the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers, and obtains the second model parameters.

[0011] In another possible implementation, the central server aggregates the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain second model parameters, including:

[0012] The central server determines the number of local datasets on the electronic devices corresponding to the multiple car manufacturers; based on the number of local datasets on the electronic devices corresponding to the multiple car manufacturers, it determines the weights of the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers; based on the weights of the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers, it performs a weighted summation of the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain the second model parameters.

[0013] In another possible implementation, the central server generates a second state recognition model based on the second model parameters and sends the second state recognition model to the electronic devices corresponding to the multiple car manufacturers, including:

[0014] The central server updates the model parameters of the first state recognition model to the second model parameters and sends the updated first state recognition model to the electronic devices corresponding to the multiple car manufacturers.

[0015] For any electronic device corresponding to any car manufacturer, the electronic device receives the updated first state recognition model, retrains the updated first state recognition model based on the local dataset, and sends the third model parameters of the trained first state recognition model to the central server.

[0016] The central server receives the third model parameters sent by the electronic devices corresponding to the multiple car manufacturers, aggregates the third model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain the fourth model parameters, determines whether the number of training times has reached the preset number, and if the number of training times has reached the preset number, updates the model parameters of the first state recognition model to the fourth model parameters to obtain the second state recognition model, and sends the second state recognition model to the electronic devices corresponding to the multiple car manufacturers.

[0017] If the number of training iterations has not reached the preset number, the model parameters of the first state recognition model are updated to the fourth model parameters, and the updated first state recognition model is sent to the electronic devices corresponding to the multiple car manufacturers. This process is repeated until the number of training iterations reaches the preset number, at which point the second state recognition model is obtained, and the second state recognition model is sent to the electronic devices corresponding to the multiple car manufacturers.

[0018] In another possible implementation, the central server generates a second state recognition model based on the second model parameters, including:

[0019] The central server sends the second model parameters to the electronic devices corresponding to the multiple car manufacturers;

[0020] For any electronic device corresponding to any car manufacturer, the electronic device receives the second model parameters, updates the model parameters of the first state recognition model to the second model parameters, retrains the updated first state recognition model based on the local dataset, and sends the third model parameters of the trained first state recognition model to the central server.

[0021] The central server receives the third model parameters sent by the electronic devices corresponding to the multiple car manufacturers, aggregates the third model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain the fourth model parameters, and sends the fourth model parameters to the electronic devices corresponding to the multiple car manufacturers.

[0022] For any electronic device corresponding to any car manufacturer, the electronic device receives the fourth model parameters sent by the central server and determines whether the training count has reached a preset number. If the training count has reached the preset number, the model parameters of the first state recognition model are updated to the fourth model parameters to obtain the second state recognition model. If the training count has not reached the preset number, the model parameters of the first state recognition model are updated to the fourth model parameters, and then the updated first state recognition model is trained again based on the local dataset. This process is repeated until the training count reaches the preset number, at which point the second state recognition model is obtained.

[0023] In another possible implementation, the second state recognition module includes a fully connected layer and other network layers preceding the fully connected layer;

[0024] The electronic device fine-tunes the second state recognition model based on a local dataset to obtain a third state recognition model, including:

[0025] The electronic device fixes the model parameters of the other network layers and fine-tunes the model parameters of the fully connected layer based on the local dataset to obtain the third state recognition model.

[0026] On the other hand, a training system for a vehicle state recognition model is provided, the system comprising:

[0027] The central server is used to send the first state recognition model to the electronic devices of multiple car manufacturers.

[0028] For any electronic device corresponding to any car manufacturer, the electronic device is used to receive a first state recognition model sent by the central server, train the first state recognition model based on a local dataset, and send the first model parameters of the first state recognition model trained by the central server.

[0029] The central server is used to receive first model parameters sent by electronic devices corresponding to the multiple car manufacturers, aggregate the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain second model parameters, generate a second state recognition model based on the second model parameters, and send the second state recognition model to the electronic devices corresponding to the multiple car manufacturers.

[0030] For any electronic device corresponding to any car manufacturer, the electronic device is used to receive the second state recognition model and fine-tune the second state recognition model based on the local dataset to obtain a third state recognition model.

[0031] In one possible implementation, the central server is used to determine the average value of the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers, and obtain the second model parameters.

[0032] In another possible implementation, the central server is used to determine the number of local datasets of the electronic devices corresponding to the multiple car manufacturers; based on the number of local datasets of the electronic devices corresponding to the multiple car manufacturers, determine the weights of the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers; and based on the weights of the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers, perform a weighted summation of the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain the second model parameters.

[0033] In another possible implementation, the central server is used to update the model parameters of the first state recognition model to the second model parameters and send the updated first state recognition model to the electronic devices corresponding to the plurality of car manufacturers.

[0034] For any electronic device corresponding to any car manufacturer, the electronic device is used to receive the updated first state recognition model, retrain the updated first state recognition model based on the local dataset, and send the third model parameters of the trained first state recognition model to the central server.

[0035] The central server is used to receive third model parameters sent by electronic devices corresponding to the multiple car manufacturers, aggregate the third model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain fourth model parameters, determine whether the number of training times has reached a preset number, and if the number of training times has reached a preset number, update the model parameters of the first state recognition model to the fourth model parameters to obtain a second state recognition model, and send the second state recognition model to the electronic devices corresponding to the multiple car manufacturers.

[0036] The central server is used to update the model parameters of the first state recognition model to the fourth model parameters when the number of training iterations has not reached a preset number, and send the updated first state recognition model to the electronic devices corresponding to the multiple car manufacturers. This process is repeated until the number of training iterations reaches a preset number, at which point a second state recognition model is obtained, and the second state recognition model is sent to the electronic devices corresponding to the multiple car manufacturers.

[0037] In another possible implementation, the central server is used to send the second model parameters to the electronic devices corresponding to the multiple car manufacturers;

[0038] For any electronic device corresponding to any car manufacturer, the electronic device is used to receive the second model parameters, update the model parameters of the first state recognition model to the second model parameters, retrain the updated first state recognition model based on the local dataset, and send the third model parameters of the trained first state recognition model to the central server.

[0039] The central server is used to receive third model parameters sent by electronic devices corresponding to the multiple car manufacturers, aggregate the third model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain fourth model parameters, and send the fourth model parameters to the electronic devices corresponding to the multiple car manufacturers.

[0040] For any electronic device corresponding to any car manufacturer, the electronic device is used to receive the fourth model parameters sent by the central server and determine whether the training count has reached a preset number. If the training count has reached the preset number, the model parameters of the first state recognition model are updated to the fourth model parameters to obtain the second state recognition model. If the training count has not reached the preset number, the model parameters of the first state recognition model are updated to the fourth model parameters, and then the updated first state recognition model is trained again based on the local dataset. This process is repeated until the training count reaches the preset number, at which point the second state recognition model is obtained.

[0041] In another possible implementation, the second state recognition module includes a fully connected layer and other network layers preceding the fully connected layer;

[0042] The electronic device is used to fix the model parameters of the other network layers and fine-tune the model parameters of the fully connected layer based on the local dataset to obtain the third state recognition model.

[0043] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one line of program code, which is loaded and executed by the processor to implement the above-described method for training a vehicle state recognition model. The computer device includes a central server and electronic devices, wherein the central server and electronic devices execute the above-described method for training a vehicle state recognition model.

[0044] On the other hand, a computer-readable storage medium is provided, wherein at least one piece of program code is stored in the storage medium, the at least one piece of program code being loaded and executed by a processor to implement the above-described training method for the vehicle state recognition model.

[0045] On the other hand, a computer program product is provided, the product storing at least one piece of program code, the at least one piece of program code being executed by a processor to implement the above-described training method for the vehicle state recognition model.

[0046] In this embodiment, datasets from multiple automakers are stored locally at each automaker. Electronic devices corresponding to each automaker train a first state recognition model based on their local dataset, thus ensuring that each automaker's dataset is available but invisible to other automakers. Furthermore, since the central server can aggregate the model parameters from the training of the first state recognition model on the electronic devices of multiple automakers to update and obtain a second state recognition model, and synchronize the second state recognition model to the electronic devices of multiple automakers, state recognition model sharing is achieved. Therefore, this embodiment can achieve state recognition model sharing while maintaining dataset confidentiality, breaking down the data barriers to training state recognition models.

[0047] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this disclosure. Attached Figure Description

[0048] Figure 1 This is a schematic diagram illustrating the development process of a training method for a vehicle state recognition model, as shown in an exemplary embodiment of this application.

[0049] Figure 2 This is a flowchart illustrating a training method for a vehicle state recognition model, as shown in an exemplary embodiment of this application.

[0050] Figure 3This is a flowchart illustrating a training method for a vehicle state recognition model, as shown in an exemplary embodiment of this application.

[0051] Figure 4 This is a flowchart illustrating a training method for a vehicle state recognition model, as shown in an exemplary embodiment of this application.

[0052] Figure 5 This is a block diagram illustrating a training system for a vehicle state recognition model, as shown in an exemplary embodiment of this application.

[0053] Figure 6 This is a block diagram illustrating a central server in an exemplary embodiment of this application;

[0054] Figure 7 This is a block diagram illustrating an electronic device in an exemplary embodiment of this application. Detailed Implementation

[0055] To make the technical solution and advantages of this application clearer, the embodiments of this application will be described in further detail below.

[0056] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0057] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the questionnaire survey information, first image data, first voice data, and multiple vehicle system data involved in this application were all obtained with full authorization.

[0058] Please refer to Figure 1 This illustration shows a schematic diagram of the implementation environment for a vehicle state recognition model training method according to an exemplary embodiment of this application. The implementation environment includes a central server and multiple electronic devices. The central server and the multiple electronic devices perform functions such as data interaction between the state recognition model and model parameters. The multiple electronic devices can be electronic devices corresponding to multiple car manufacturers, with one electronic device corresponding to each car manufacturer.

[0059] The central server sends the first state recognition model to electronic devices corresponding to multiple car manufacturers (car manufacturers can also be referred to as users, with multiple car manufacturers being users 1 to users N, where N is an integer greater than 1). For any electronic device, the electronic device trains the first state recognition model (which can be called the local model) based on its local dataset and sends the first model parameters of the trained first state recognition model to the central server. The central server aggregates the first model parameters sent by multiple electronic devices to obtain the second model parameters (which can be called the aggregated local model parameters) and updates the first state recognition module based on the second model parameters. The above process is repeated until the training reaches a preset number of times to obtain the second state recognition model (which can be called the global model). The second state recognition model is then sent to multiple electronic devices. For any electronic device, the electronic device fine-tunes the second state recognition model based on its local dataset to obtain the third state recognition model.

[0060] Central servers can be deployed anywhere; for example, they can be deployed in official organizations such as the China Automotive Technology Research Center to ensure fairness. A central server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Electronic devices can be terminals or servers. When an electronic device is a terminal, it can be a smartphone, tablet, laptop, or smart wearable device (smartwatch or smart bracelet), but is not limited to these. When an electronic device is a server, it can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0061] Please refer to Figure 2 The diagram illustrates a flowchart of a training method for a vehicle state recognition model, as shown in an exemplary embodiment of this application. (Reference) Figure 2 The method includes:

[0062] Step 201: The central server sends the first state recognition model to the electronic devices of multiple car manufacturers.

[0063] The first state recognition model can be an initial state recognition model; or, the first state recognition model can be a state recognition model obtained by training the initial state recognition model on the central server. If the first state recognition model is obtained by training the initial state recognition model on the central server, the central server needs to train the first state recognition model. The process of the central server training the initial state recognition model to obtain the first state recognition model can be as follows: the central server obtains an open-source dataset, and trains the initial state recognition model based on the dataset to obtain the first state recognition model. The open-source dataset is a publicly available dataset, and it includes multiple first sample images and labels for the first sample images. The labels for the first sample images can be objects included in the first sample images, such as cars, pedestrians, trees, houses, pets, bicycles, etc. Multiple car manufacturers can be the car manufacturers for whom the state recognition model needs to be trained; or multiple car manufacturers can be a subset of car manufacturers identified from those for whom the state recognition model needs to be trained.

[0064] Step 202: For any electronic device corresponding to any car manufacturer, the electronic device receives the first state recognition model sent by the central server, trains the first state recognition model based on the local dataset, and sends the first model parameters of the first state recognition model trained by the central server.

[0065] The local dataset includes multiple second sample images and their labels. The labels for the second sample images can be objects included in the images, such as cars, pedestrians, trees, houses, pets, bicycles, etc. Different car manufacturers collect different datasets, and the corresponding electronic devices of these manufacturers train their first-state recognition models based on the local dataset. This eliminates the need to share the local dataset with other devices, preventing data leakage and thus protecting the security of the local dataset.

[0066] The steps for an electronic device to train a first-state recognition model based on a local dataset can be as follows: The electronic device inputs multiple second-sample images included in the local training set into the first-state recognition model, identifies objects included in the multiple second-sample images through the first-state recognition model, determines the loss value of the first-state recognition model based on the objects identified in the second-sample images and the objects labeled in the second-sample images for any given second-sample image, and adjusts the first-state recognition model based on the loss value of the first-state recognition model, thereby completing the sequential training process of the first-state recognition model.

[0067] The first set of parameters for the trained first-state recognition model includes training parameters, such as the learning rate, batch size, and regularization. Training parameters may also include an optimizer and a loss function. The learning rate controls the magnitude of model parameter updates; an excessively large learning rate may cause oscillations, while a too small learning rate may result in slow convergence. The batch size is the number of sample images used to update the first-state recognition model in each iteration, affecting the update frequency and computational efficiency. The optimizer can be either Adam or SGD; the optimizer is the algorithm for updating the first-state recognition model and affects convergence speed and stability. The loss function measures the difference between the model's predictions and the true labels. Regularization prevents overfitting, thereby improving the generalization ability of the first-state recognition model.

[0068] In one possible implementation, in addition to sending the first model parameters to the central server, the electronic device can also send the number of local datasets to the central server, so that the central server can determine the weight of the first model parameters sent by the electronic device based on the number of local datasets, and then perform federated aggregation of the first model parameters based on the weight.

[0069] Step 203: The central server receives the first model parameters sent by the electronic devices corresponding to multiple car manufacturers, aggregates the first model parameters sent by the electronic devices corresponding to multiple car manufacturers to obtain the second model parameters, generates the second state recognition model based on the second model parameters, and sends the second state recognition model to the electronic devices corresponding to multiple car manufacturers.

[0070] In the first implementation, the central server aggregates the first model parameters sent by electronic devices from multiple car manufacturers using a federated averaging algorithm. Correspondingly, the step of the central server aggregating the first model parameters sent by the electronic devices from multiple car manufacturers to obtain the second model parameters can be as follows: the central server determines the average value of the first model parameters sent by the electronic devices from multiple car manufacturers to obtain the second model parameters. For example, the first model parameters include training parameters, which include learning rate, batch size, and regularization. The step of the central server determining the average value of the first model parameters sent by the electronic devices from multiple car manufacturers to obtain the second model parameters can be as follows: the central server determines the average value of the learning rate sent by the electronic devices from multiple car manufacturers, the average value of the batch size sent by the electronic devices from multiple car manufacturers, and the average value of the regularization sent by the electronic devices from multiple car manufacturers. The average value of the learning rate, the average value of the batch size, and the average value of the regularization are then combined to form the second model parameters.

[0071] If the first model parameters also include an optimizer and a loss function, the central server determines the target optimizer by training the optimizer used in the first state recognition model based on the electronic devices corresponding to multiple car manufacturers; it determines the target loss function by training the first state recognition model based on the loss function used in the first state recognition model based on the electronic devices corresponding to multiple car manufacturers; and it combines the target optimizer, the target loss function, and the average of the learning rate, the average of the batch size, and the average of the regularization to form the second model parameters.

[0072] In one possible implementation, the target optimizer can be the optimizer most frequently used among the optimizers used to train the first-state recognition model for electronic devices from multiple car manufacturers. For example, if the multiple car manufacturers are car manufacturers 1-4, car manufacturers 1-3 use optimizer 1, and car manufacturer 2 uses optimizer 2, then the target optimizer can be optimizer 1. In another possible implementation, the target optimizer can be the best-performing optimizer among those used to train the first-state recognition model for electronic devices from multiple car manufacturers. In yet another possible implementation, the target optimizer can be the optimizer used by a specific car manufacturer among the multiple car manufacturers; the target car manufacturer could be the largest car manufacturer, the car manufacturer with the best reputation, or the car manufacturer with the strongest driver assistance features.

[0073] The target loss function can be the loss function most frequently used in training the first-state recognition model for electronic devices from multiple car manufacturers. For example, if car manufacturers 1-4 use loss function 1 for manufacturers 1-3 and loss function 2 for manufacturer 2, then the target loss function can be loss function 1. In another possible implementation, the target loss function can be the best-performing loss function among the optimizers used to train the first-state recognition model for electronic devices from multiple car manufacturers. In yet another possible implementation, the target loss function can be the optimizer used by a specific car manufacturer; the target car manufacturer could be the largest, the one with the best reputation, or the one with the strongest driver assistance features.

[0074] In the second implementation, the central server uses a federated weighted algorithm to aggregate the first model parameters sent by electronic devices corresponding to multiple car manufacturers, and the weights of the first model parameters are determined based on the number of datasets. Accordingly, the steps for the central server to aggregate the first model parameters sent by electronic devices corresponding to multiple car manufacturers to obtain the second model parameters can be as follows: the central server determines the number of datasets in the local datasets of the electronic devices corresponding to multiple car manufacturers; based on the number of datasets in the local datasets of the electronic devices corresponding to multiple car manufacturers, the central server determines the weights of the first model parameters sent by the electronic devices corresponding to multiple car manufacturers; based on the weights of the first model parameters sent by the electronic devices corresponding to multiple car manufacturers, the central server performs a weighted summation of the first model parameters sent by the electronic devices corresponding to multiple car manufacturers to obtain the second model parameters.

[0075] For example, the first model parameters include training parameters, which include learning rate, batch size, and regularization. The steps for the central server to obtain the second model parameters by weighting and summing the first model parameters sent by multiple electronic devices corresponding to different car manufacturers, based on the weights of the first model parameters sent by these devices, can be as follows: The central server weights and sums the learning rates sent by multiple electronic devices corresponding to different car manufacturers, based on the weights of the learning rates sent by these devices, to obtain a weighted learning rate value; it weights and sums the batch sizes sent by multiple electronic devices corresponding to different car manufacturers, based on the weights of the batch sizes sent by these devices, to obtain a weighted batch size value; it weights and sums the regularization values ​​sent by multiple electronic devices corresponding to different car manufacturers, based on the weights of the regularization values ​​sent by these devices, to obtain a weighted regularization value; and the weighted learning rate value, the weighted batch size value, and the weighted regularization value are combined to form the second model parameters.

[0076] The number of local datasets on the electronic device is positively correlated with the weight of the first model parameters sent by the electronic device. That is, the larger the number of local datasets on the electronic device, the higher the weight of the first model parameters sent by the electronic device, and the smaller the number of local datasets on the electronic device, the lower the weight of the first model parameters sent by the electronic device.

[0077] When the first model parameters also include an optimizer and a loss function, the central server determines the target optimizer based on the optimizer used to train the first state recognition model for electronic devices corresponding to multiple car manufacturers; it determines the target loss function based on the loss function used to train the first state recognition model for electronic devices corresponding to multiple car manufacturers; and it combines the target optimizer, the target loss function, and the learning rate weight, batch size weight, and regularization weight together to form the second model parameters.

[0078] In the third implementation, the central server uses a federated weighted algorithm to aggregate the first model parameters sent by electronic devices corresponding to multiple car manufacturers. Accordingly, the steps for the central server to aggregate the first model parameters sent by electronic devices corresponding to multiple car manufacturers to obtain the second model parameters can be as follows: the central server determines the scale of multiple car manufacturers, determines the weight of the first model parameters sent by electronic devices corresponding to multiple car manufacturers based on the scale of multiple car manufacturers, and performs a weighted summation of the first model parameters sent by electronic devices corresponding to multiple car manufacturers based on the weight of the first model parameters sent by electronic devices corresponding to multiple car manufacturers to obtain the second model parameters.

[0079] Among them, the size of the car company is positively correlated with the weight of the first model parameter sent by the electronic device corresponding to the car company. That is, the larger the size of the car company, the higher the weight of the first model parameter sent by the electronic device corresponding to the car company, and the smaller the size of the car company, the lower the weight of the first model parameter sent by the electronic device corresponding to the car company.

[0080] When the first model parameters also include an optimizer and a loss function, the central server determines the target optimizer based on the optimizer used to train the first state recognition model for electronic devices corresponding to multiple car manufacturers; it determines the target loss function based on the loss function used to train the first state recognition model for electronic devices corresponding to multiple car manufacturers; and it combines the target optimizer, the target loss function, and the learning rate weight, batch size weight, and regularization weight together to form the second model parameters.

[0081] In the fourth implementation, the central server uses a federated weighted algorithm to aggregate the first model parameters sent by electronic devices corresponding to multiple car manufacturers. Correspondingly, the steps for the central server to aggregate the first model parameters sent by electronic devices corresponding to multiple car manufacturers to obtain the second model parameters can be as follows: the central server determines the reputation of multiple car manufacturers, determines the weight of the first model parameters sent by electronic devices corresponding to multiple car manufacturers based on the reputation of multiple car manufacturers, and performs a weighted summation of the first model parameters sent by electronic devices corresponding to multiple car manufacturers based on the weight of the first model parameters sent by electronic devices corresponding to multiple car manufacturers to obtain the second model parameters.

[0082] Among them, the reputation of car companies is positively correlated with the weight of the first model parameters sent by the electronic devices corresponding to the car company. That is, the better the reputation of the car company, the higher the weight of the first model parameters sent by the electronic devices corresponding to the car company, and the worse the reputation of the car company, the lower the weight of the first model parameters sent by the electronic devices corresponding to the car company.

[0083] When the first model parameters also include an optimizer and a loss function, the central server determines the target optimizer based on the optimizer used to train the first state recognition model for electronic devices corresponding to multiple car manufacturers; it determines the target loss function based on the loss function used to train the first state recognition model for electronic devices corresponding to multiple car manufacturers; and it combines the target optimizer, the target loss function, and the learning rate weight, batch size weight, and regularization weight together to form the second model parameters.

[0084] The central server sets the model parameters of the first state recognition model to the parameters of the second model, thus obtaining the second state recognition model. For example, if the second model parameters include learning rate, batch size, and regularization, then the central server sets the learning rate of the first state recognition model to the learning rate included in the second model parameters, the batch size of the first state recognition model to the batch size included in the second model parameters, and the regularization of the first state recognition model to the regularization included in the second model parameters, thus obtaining the second state recognition model. If the second model parameters may also include an optimizer and a loss function, then the central server sets the optimizer of the first state recognition model to the optimizer included in the second model parameters and sets the loss function of the first state recognition model to the loss function included in the second model parameters, thus obtaining the second state recognition model.

[0085] Step 204: For any electronic device corresponding to any car manufacturer, the electronic device receives the second state recognition model and fine-tunes the second state recognition model based on the local dataset to obtain the third state recognition model.

[0086] In one possible implementation, when fine-tuning the second state recognition model, the electronic device only fine-tunes the fully connected layer in the second state recognition model; that is, the second state recognition module includes the fully connected layer and the other network layers preceding it. Correspondingly, the step of the electronic device fine-tuning the second state recognition model based on a local dataset to obtain the third state recognition model can be: the electronic device fixes the model parameters of the other network layers and fine-tunes the model parameters of the fully connected layer based on the local dataset to obtain the third state recognition model. In this embodiment, the electronic device fine-tunes the second state recognition model to obtain a personalized model suitable for the local dataset.

[0087] In another possible implementation, after obtaining the third-state recognition model, the electronic device can deploy the third-state recognition model into the new energy vehicle, enabling the new energy vehicle to perform intelligent driving based on the third-state recognition model. For example, the new energy vehicle uses the third-state recognition model to identify objects in front of it, makes a state judgment based on the objects, and then makes decisions based on the state judgment results; for example, if the state judgment result is that there is an obstacle one meter in front of the new energy vehicle, the new energy vehicle will make decisions such as braking or changing lanes to improve user safety.

[0088] In this embodiment, datasets from multiple automakers are stored locally at each automaker. Electronic devices corresponding to each automaker train a first state recognition model based on their local dataset, thus ensuring that each automaker's dataset is available but invisible to other automakers. Furthermore, since the central server can aggregate the model parameters from the training of the first state recognition model on the electronic devices of multiple automakers to update and obtain a second state recognition model, and synchronize the second state recognition model to the electronic devices of multiple automakers, state recognition model sharing is achieved. Therefore, this embodiment can achieve state recognition model sharing while maintaining dataset confidentiality, breaking down the data barriers to training state recognition models.

[0089] Please refer to Figure 3 This document illustrates a flowchart of a training method for a vehicle state recognition model according to an exemplary embodiment of this application. In this embodiment, the method is described using the example of a central server updating the state recognition model and sending the updated model to an electronic device for iterative training.

[0090] refer to Figure 3 The method includes:

[0091] Step 301: The central server sends the first state recognition model to the electronic devices corresponding to multiple car manufacturers.

[0092] In some embodiments, this step is the same as step 201, and will not be described again here.

[0093] Step 302: For any electronic device corresponding to any car manufacturer, the electronic device receives the first state recognition model sent by the central server, trains the first state recognition model based on the local dataset, and sends the first model parameters of the first state recognition model trained by the central server.

[0094] In some embodiments, this step is the same as step 202, and will not be described again here.

[0095] Step 303: The central server receives the first model parameters sent by electronic devices corresponding to multiple car manufacturers, aggregates the first model parameters sent by electronic devices corresponding to multiple car manufacturers, and obtains the second model parameters.

[0096] In some embodiments, the process of the central server aggregating the first model parameters sent by the electronic devices corresponding to multiple car manufacturers to obtain the second model parameters can be referred to step 203, and will not be repeated here.

[0097] Step 304: The central server updates the model parameters of the first state recognition model to the second model parameters and sends the updated first state recognition model to the electronic devices of multiple car manufacturers.

[0098] The central server sets the model parameters of the first-state recognition model to the second-state model parameters, resulting in an updated first-state recognition model. For example, if the second-state model parameters include learning rate, batch size, and regularization, the central server sets the learning rate of the first-state recognition model to the learning rate included in the second-state model parameters, the batch size of the first-state recognition model to the batch size included in the second-state model parameters, and the regularization of the first-state recognition model to the regularization included in the second-state model parameters, resulting in an updated first-state recognition model. If the second-state model parameters may also include an optimizer and a loss function, the central server sets the optimizer of the first-state recognition model to the optimizer included in the second-state model parameters and sets the loss function of the first-state recognition model to the loss function included in the second-state model parameters, resulting in an updated first-state recognition model.

[0099] Step 305: For any electronic device corresponding to any car manufacturer, the electronic device receives the updated first state recognition model, retrains the updated first state recognition model based on the local dataset, and sends the third model parameters of the trained first state recognition model to the central server.

[0100] In some embodiments, the process of the electronic device retraining the updated first state recognition model based on the local dataset is similar to the process of the electronic device training the first state recognition model based on the local dataset in step 201, and will not be described again here.

[0101] Step 306: The central server receives the third model parameters sent by the electronic devices corresponding to multiple car manufacturers, and aggregates the third model parameters sent by the electronic devices corresponding to multiple car manufacturers to obtain the fourth model parameters.

[0102] The process by which the central server aggregates the third model parameters sent by electronic devices corresponding to multiple car manufacturers to obtain the fourth model parameters is similar to the process by which the central server aggregates the first model parameters sent by electronic devices corresponding to multiple car manufacturers to obtain the second model parameters, and will not be described in detail here.

[0103] Step 307: The central server determines whether the training count has reached the preset number. If the training count has reached the preset number, the model parameters of the first state recognition model are updated to the fourth model parameters to obtain the second state recognition model, and the second state recognition model is sent to the electronic devices corresponding to multiple car manufacturers.

[0104] The steps by which the central server updates the model parameters of the first-state recognition model to the fourth-state recognition model to obtain the second-state recognition model can be as follows: The central server sets the model parameters of the first-state recognition model to the fourth-state recognition model, thus obtaining the second-state recognition model. For example, if the fourth-state recognition parameters include learning rate, batch size, and regularization, then the central server sets the learning rate of the first-state recognition model to the learning rate included in the fourth-state recognition parameters, sets the batch size of the first-state recognition model to the batch size included in the fourth-state recognition parameters, and sets the regularization of the first-state recognition model to the regularization included in the fourth-state recognition parameters, thus obtaining the second-state recognition model. If the fourth-state recognition parameters may also include an optimizer and a loss function, then the central server also sets the optimizer of the first-state recognition model to the optimizer included in the fourth-state recognition parameters and sets the loss function of the first-state recognition model to the loss function included in the fourth-state recognition parameters, thus obtaining the second-state recognition model.

[0105] The preset number of iterations is the number of training iterations, and the preset number of iterations can be set and changed as needed. In this embodiment, the preset number of iterations is not specifically limited.

[0106] Step 308: If the number of training iterations has not reached the preset number, the central server updates the model parameters of the first state recognition model to the fourth model parameters and sends the updated first state recognition model to the electronic devices corresponding to multiple car manufacturers. This process continues until the number of training iterations reaches the preset number, at which point the second state recognition model is obtained and sent to the electronic devices corresponding to multiple car manufacturers.

[0107] The central server sends updated first-state recognition models to electronic devices corresponding to multiple car manufacturers. Each electronic device corresponding to a car manufacturer receives the updated first-state recognition model, retrains it based on its local dataset, and sends the fifth model parameters of the updated first-state recognition model to the central server. The central server receives the fifth model parameters from the electronic devices of multiple car manufacturers, aggregates them to obtain sixth model parameters, and determines whether the training iterations have reached a preset number. If the training iterations have reached the preset number, the central server updates the model parameters of the first-state recognition model to the sixth model parameters to obtain a second-state recognition model, and sends the second-state recognition model to the electronic devices of multiple car manufacturers. If the training iterations have not reached the preset number, the central server updates the model parameters of the first-state recognition model to the sixth model parameters and sends the updated first-state recognition model to the electronic devices of multiple car manufacturers, and so on, until the training iterations reach the preset number, at which point the second-state recognition model is obtained and sent to the electronic devices of multiple car manufacturers.

[0108] Step 309: For any electronic device corresponding to any car manufacturer, the electronic device receives the second state recognition model and fine-tunes the second state recognition model based on the local dataset to obtain the third state recognition model.

[0109] In some embodiments, this step is the same as step 204, and will not be described again here.

[0110] In this embodiment, datasets from multiple automakers are stored locally at each automaker. Electronic devices corresponding to each automaker train a first state recognition model based on their local dataset, thus ensuring that each automaker's dataset is available but invisible to other automakers. Furthermore, since the central server can aggregate the model parameters from the training of the first state recognition model on the electronic devices of multiple automakers to update and obtain a second state recognition model, and synchronize the second state recognition model to the electronic devices of multiple automakers, state recognition model sharing is achieved. Therefore, this embodiment can achieve state recognition model sharing while maintaining dataset confidentiality, breaking down the data barriers to training state recognition models.

[0111] Please refer to Figure 4 This document illustrates a flowchart of a training method for a vehicle state recognition model according to an exemplary embodiment of this application. In this embodiment, the example described is a central server updating the state recognition model and sending the aggregated second model parameters to an electronic device, enabling the electronic device to iteratively train and update the first state recognition model based on the second model parameters. (See reference...) Figure 4 The method includes:

[0112] Step 401: The central server sends the first state recognition model to the electronic devices of multiple car manufacturers.

[0113] In some embodiments, this step is the same as step 201, and will not be described again here.

[0114] Step 402: For any electronic device corresponding to any car manufacturer, the electronic device receives the first state recognition model sent by the central server, trains the first state recognition model based on the local dataset, and sends the first model parameters of the first state recognition model trained by the central server.

[0115] In some embodiments, this step is the same as step 202, and will not be described again here.

[0116] Step 403: The central server receives the first model parameters sent by the electronic devices corresponding to multiple car manufacturers, aggregates the first model parameters sent by the electronic devices corresponding to multiple car manufacturers to obtain the second model parameters, and sends the second model parameters to the electronic devices corresponding to multiple car manufacturers.

[0117] In some embodiments, the process of the central server aggregating the first model parameters sent by the electronic devices corresponding to multiple car manufacturers to obtain the second model parameters can be referred to step 203, and will not be repeated here.

[0118] In this step, the central server sends the second model parameters to the electronic devices of multiple car manufacturers, instead of sending the updated first state model, thereby reducing the amount of information transmitted and improving the efficiency of information transmission.

[0119] Step 404: For any electronic device corresponding to any car manufacturer, the electronic device receives the second model parameters, updates the model parameters of the first state recognition model to the second model parameters, retrains the updated first state recognition model based on the local dataset, and sends the third model parameters of the trained first state recognition model to the central server.

[0120] The process by which the electronic device updates the model parameters of the first state recognition model to the second model parameters is similar to the process by which the central server updates the model parameters of the first state recognition model to the second model parameters, and will not be described in detail here.

[0121] Step 405: The central server receives the third model parameters sent by the electronic devices corresponding to multiple car manufacturers, aggregates the third model parameters sent by the electronic devices corresponding to multiple car manufacturers to obtain the fourth model parameters, and sends the fourth model parameters to the electronic devices corresponding to multiple car manufacturers.

[0122] The process by which the central server aggregates the third model parameters sent by electronic devices corresponding to multiple car manufacturers to obtain the fourth model parameters is similar to the process by which the central server aggregates the first model parameters sent by electronic devices corresponding to multiple car manufacturers to obtain the second model parameters, and will not be described in detail here.

[0123] Step 406: For any electronic device corresponding to any car manufacturer, the electronic device receives the fourth model parameter sent by the central server and determines whether the number of training times has reached the preset number; if the number of training times has reached the preset number, the model parameters of the first state recognition model are updated to the fourth model parameter to obtain the second state recognition model.

[0124] The steps for an electronic device to update the model parameters of a first-state recognition model to the fourth-state recognition parameters to obtain a second-state recognition model can be as follows: The electronic device sets the model parameters of the first-state recognition model to the fourth-state recognition parameters to obtain the second-state recognition model. For example, if the fourth-state recognition parameters include learning rate, batch size, and regularization, then the central server sets the learning rate of the first-state recognition model to the learning rate included in the fourth-state recognition parameters, sets the batch size of the first-state recognition model to the batch size included in the fourth-state recognition parameters, and sets the regularization of the first-state recognition model to the regularization included in the fourth-state recognition parameters to obtain the second-state recognition model. If the fourth-state recognition parameters may also include an optimizer and a loss function, then the central server also sets the optimizer of the first-state recognition model to the optimizer included in the fourth-state recognition parameters and sets the loss function of the first-state recognition model to the loss function included in the fourth-state recognition parameters to obtain the second-state recognition model.

[0125] The preset number of iterations is the number of training iterations, and the preset number of iterations can be set and changed as needed. In this embodiment, the preset number of iterations is not specifically limited.

[0126] Step 407: If the number of training iterations has not reached the preset number, the electronic device updates the model parameters of the first state recognition model to the fourth model parameters, and then retrains the updated first state recognition model based on the local dataset. This process continues until the preset number of training iterations is reached, at which point the second state recognition model is obtained.

[0127] The electronic device updates the model parameters of the first state recognition model to the fourth model parameters, then retrains the updated first state recognition model based on the local dataset, and sends the fifth model parameters of the updated first state recognition model to the central server. The central server receives the fifth model parameters sent by electronic devices corresponding to multiple car manufacturers, aggregates the fifth model parameters sent by electronic devices corresponding to multiple car manufacturers to obtain the sixth model parameters, and sends the sixth model parameters to electronic devices corresponding to multiple car manufacturers. For any electronic device corresponding to a car manufacturer, the electronic device receives the sixth model parameters and determines whether the training iterations have reached a preset number. If the training iterations have reached the preset number, the electronic device updates the model parameters of the first state recognition model to the sixth model parameters to obtain the second state recognition model. If the training iterations have not reached the preset number, the electronic device updates the model parameters of the first state recognition model to the sixth model parameters and sends the seventh model parameters of the updated first state recognition model to the central server, and so on, until the training iterations reach the preset number, at which point the second state recognition model is obtained.

[0128] Step 408: For any electronic device corresponding to any car manufacturer, the electronic device fine-tunes the second state recognition model based on the local dataset to obtain the third state recognition model.

[0129] In some embodiments, this step is the same as step 204, and will not be described again here.

[0130] In this embodiment, datasets from multiple automakers are stored locally at each automaker. Electronic devices corresponding to each automaker train a first state recognition model based on their local dataset, thus ensuring that each automaker's dataset is available but invisible to other automakers. Furthermore, since the central server can aggregate the model parameters from the training of the first state recognition model on the electronic devices of multiple automakers to update and obtain a second state recognition model, and synchronize the second state recognition model to the electronic devices of multiple automakers, state recognition model sharing is achieved. Therefore, this embodiment can achieve state recognition model sharing while maintaining dataset confidentiality, breaking down the data barriers to training state recognition models.

[0131] Please refer to Figure 5 This illustration shows a block diagram of a training system for a vehicle state recognition model, as illustrated in an exemplary embodiment of this application. The system includes:

[0132] The central server 501 is used to send the first state recognition model to the electronic devices 502 corresponding to multiple car manufacturers;

[0133] For any car manufacturer's corresponding electronic device 502, the electronic device 502 is used to receive the first state recognition model sent by the central server 501, train the first state recognition model based on the local dataset, and send the first model parameters of the first state recognition model trained by the central server 501.

[0134] The central server 501 is used to receive the first model parameters sent by the electronic devices 502 corresponding to multiple car manufacturers, aggregate the first model parameters sent by the electronic devices 502 corresponding to multiple car manufacturers to obtain the second model parameters, generate a second state recognition model based on the second model parameters, and send the second state recognition model to the electronic devices 502 corresponding to multiple car manufacturers.

[0135] For any car manufacturer's corresponding electronic device 502, the electronic device 502 is used to receive the second state recognition model and fine-tune the second state recognition model based on the local dataset to obtain the third state recognition model.

[0136] In another possible implementation, a central server 501 is used to determine the average value of the first model parameters sent by the electronic devices 502 corresponding to multiple car manufacturers, and to obtain the second model parameters.

[0137] In another possible implementation, a central server 501 is used to determine the number of local datasets of electronic devices 502 corresponding to multiple car manufacturers; based on the number of local datasets of electronic devices 502 corresponding to multiple car manufacturers, the weights of the first model parameters sent by the electronic devices 502 corresponding to multiple car manufacturers are determined; based on the weights of the first model parameters sent by the electronic devices 502 corresponding to multiple car manufacturers, the first model parameters sent by the electronic devices 502 corresponding to multiple car manufacturers are weighted and summed to obtain the second model parameters.

[0138] In another possible implementation, the central server 501 is used to update the model parameters of the first state recognition model to the second model parameters and send the updated first state recognition model to the electronic devices 502 corresponding to multiple car manufacturers.

[0139] For any car manufacturer's corresponding electronic device 502, the electronic device 502 is used to receive the updated first state recognition model, retrain the updated first state recognition model based on the local dataset, and send the third model parameters of the trained first state recognition model to the central server 501.

[0140] The central server 501 is used to receive the third model parameters sent by the electronic devices 502 corresponding to multiple car manufacturers, aggregate the third model parameters sent by the electronic devices 502 corresponding to multiple car manufacturers to obtain the fourth model parameters, determine whether the number of training times has reached the preset number of times, and if the number of training times has reached the preset number of times, update the model parameters of the first state recognition model to the fourth model parameters to obtain the second state recognition model, and send the second state recognition model to the electronic devices 502 corresponding to multiple car manufacturers.

[0141] The central server 501 is used to update the model parameters of the first state recognition model to the fourth model parameters when the number of training times has not reached the preset number, and send the updated first state recognition model to the electronic devices 502 corresponding to multiple car companies. This process continues until the number of training times reaches the preset number, at which point the second state recognition model is obtained and sent to the electronic devices 502 corresponding to multiple car companies.

[0142] In another possible implementation, the central server 501 is used to send the second model parameters to the electronic devices 502 corresponding to multiple car manufacturers;

[0143] For any car manufacturer's corresponding electronic device 502, the electronic device 502 is used to receive the second model parameters, update the model parameters of the first state recognition model to the second model parameters, retrain the updated first state recognition model based on the local dataset, and send the third model parameters of the trained first state recognition model to the central server 501.

[0144] The central server 501 is used to receive the third model parameters sent by the electronic devices 502 corresponding to multiple car manufacturers, aggregate the third model parameters sent by the electronic devices 502 corresponding to multiple car manufacturers to obtain the fourth model parameters, and send the fourth model parameters to the electronic devices 502 corresponding to multiple car manufacturers.

[0145] For any car manufacturer's corresponding electronic device 502, the electronic device 502 is used to receive the fourth model parameters sent by the central server 501 and determine whether the training count has reached the preset number of times. If the training count has reached the preset number of times, the model parameters of the first state recognition model are updated to the fourth model parameters to obtain the second state recognition model. If the training count has not reached the preset number of times, the model parameters of the first state recognition model are updated to the fourth model parameters, and then the updated first state recognition model is trained again based on the local dataset. This process is repeated until the training count reaches the preset number of times, at which point the second state recognition model is obtained.

[0146] In another possible implementation, the second state recognition module includes a fully connected layer and other network layers preceding the fully connected layer;

[0147] Electronic device 502 is used to fix the model parameters of other network layers and fine-tune the model parameters of the fully connected layer based on the local dataset to obtain the third state recognition model.

[0148] In this embodiment, datasets from multiple automakers are stored locally at each automaker. Electronic devices corresponding to each automaker train a first state recognition model based on their local dataset, thus ensuring that each automaker's dataset is available but invisible to other automakers. Furthermore, since the central server can aggregate the model parameters from the training of the first state recognition model on the electronic devices of multiple automakers to update and obtain a second state recognition model, and synchronize the second state recognition model to the electronic devices of multiple automakers, state recognition model sharing is achieved. Therefore, this embodiment can achieve state recognition model sharing while maintaining dataset confidentiality, breaking down the data barriers to training state recognition models.

[0149] It should be noted that the training system for the vehicle state recognition model provided in the above embodiments and the training method for the vehicle state recognition model belong to the same concept. For details of its implementation process, please refer to the method embodiments, which will not be repeated here.

[0150] The structural diagram of the central server can be found here. Figure 6 The central server 600 can vary considerably depending on its configuration or performance. It may include a central processing unit (CPU) 601 and a memory 602. The memory 602 stores at least one line of program code, which is loaded and executed by the processor 601 to perform the operations performed by the central server in the aforementioned vehicle-mounted display method. Of course, the central server 600 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The central server 600 may also include other components for implementing device functions, which will not be elaborated upon here.

[0151] In some embodiments, the electronic device may be a terminal or a server. When the electronic device is a server, its structural block diagram may also be referenced. Figure 6 When the electronic device is the terminal, please refer to... Figure 7 , Figure 7A structural block diagram of a terminal 700 provided in an exemplary embodiment of this application is shown. The terminal 700 may be a portable mobile terminal, such as a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The terminal 700 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0152] Typically, terminal 700 includes a processor 701 and a memory 702.

[0153] Processor 701 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 701 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 701 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 701 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 701 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0154] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 702 is used to store at least one piece of program code, which is executed by the processor 701 to implement the operations performed by the terminal in the in-vehicle display method provided in the method embodiments of this application.

[0155] In some embodiments, the terminal 700 may also optionally include a peripheral device interface 703 and at least one peripheral device. The processor 701, memory 702, and peripheral device interface 703 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 703 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 704, a display screen 705, a camera assembly 706, an audio circuit 707, and a power supply 708.

[0156] Peripheral device interface 703 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 701 and memory 702. In some embodiments, processor 701, memory 702 and peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 701, memory 702 and peripheral device interface 703 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0157] The radio frequency (RF) circuit 704 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 704 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 704 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 704 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0158] Display screen 705 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 705 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 701 for processing. In this case, display screen 705 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 705, disposed on the front panel of terminal 700; in other embodiments, there may be at least two display screens 705, disposed on different surfaces of terminal 700 or in a folded design; in other embodiments, display screen 705 may be a flexible display screen, disposed on a curved or folded surface of terminal 700. Furthermore, display screen 705 may be configured as a non-rectangular irregular shape, i.e., a non-rectangular screen. Display screen 705 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0159] The camera assembly 706 is used to acquire images or videos. Optionally, the camera assembly 706 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 706 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0160] The audio circuit 707 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 701 for processing, or input to the radio frequency circuit 704 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different part of the terminal 700. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert the electrical signals from the processor 701 or the radio frequency circuit 704 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 707 may also include a headphone jack.

[0161] Power supply 708 is used to power the various components in terminal 700. Power supply 708 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 708 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0162] In some embodiments, the terminal 700 further includes one or more sensors 709. The one or more sensors 709 include, but are not limited to: an accelerometer 710, a gyroscope 711, a pressure sensor 712, an optical sensor 713, and a proximity sensor 714.

[0163] Accelerometer 710 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by terminal 700. For example, accelerometer 710 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 701 can control display screen 705 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 710. Accelerometer 710 can also be used for games or for acquiring user motion data.

[0164] The gyroscope sensor 711 can detect the orientation and rotation angle of the terminal 700. The gyroscope sensor 711, in conjunction with the accelerometer sensor 710, can collect 3D motion data from the user on the terminal 700. Based on the data collected by the gyroscope sensor 711, the processor 701 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0165] The pressure sensor 712 can be disposed on the side bezel of the terminal 700 and / or on the lower layer of the display screen 705. When the pressure sensor 712 is disposed on the side bezel of the terminal 700, it can detect the user's grip signal on the terminal 700, and the processor 701 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 712. When the pressure sensor 712 is disposed on the lower layer of the display screen 705, the processor 701 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 705. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0166] An optical sensor 713 is used to collect ambient light intensity. In one embodiment, the processor 701 can control the display brightness of the display screen 705 based on the ambient light intensity collected by the optical sensor 713. Specifically, when the ambient light intensity is high, the display brightness of the display screen 705 is increased; when the ambient light intensity is low, the display brightness of the display screen 705 is decreased. In another embodiment, the processor 701 can also dynamically adjust the shooting parameters of the camera assembly 706 based on the ambient light intensity collected by the optical sensor 713.

[0167] The proximity sensor 714, also known as a distance sensor, is typically located on the front panel of the terminal 700. The proximity sensor 714 is used to detect the distance between the user and the front of the terminal 700. In one embodiment, when the proximity sensor 714 detects that the distance between the user and the front of the terminal 700 is gradually decreasing, the processor 701 controls the display screen 705 to switch from a screen-on state to a screen-off state; when the proximity sensor 714 detects that the distance between the user and the front of the terminal 700 is gradually increasing, the processor 701 controls the display screen 705 to switch from a screen-off state to a screen-on state.

[0168] Those skilled in the art will understand that Figure 7 The structure shown does not constitute a limitation on terminal 700, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0169] This application also provides a computer-readable storage medium storing at least one piece of program code, which is loaded and executed by a processor to implement the training method for the vehicle state recognition model described in any of the above implementations. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk, and optical data storage device.

[0170] This application also provides a computer program product that stores at least one piece of program code, which is loaded and executed by a processor to implement the training method for the vehicle state recognition model shown in the above embodiments.

[0171] In some embodiments, the computer program product involved in the present application can be deployed and executed on a computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed in multiple locations and interconnected through a communication network. Multiple computer devices distributed in multiple locations and interconnected through a communication network can form a blockchain system.

[0172] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0173] The above description is only for the purpose of enabling those skilled in the art to understand the technical solution of this application, and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A training method for a vehicle state recognition model, characterized in that, The method includes: The central server sends the first state recognition model to the electronic devices of multiple car manufacturers. For any electronic device corresponding to any car manufacturer, the electronic device receives a first state recognition model sent by the central server, trains the first state recognition model based on the local dataset, and sends the first model parameters of the first state recognition model trained by the central server. The central server receives first model parameters sent by electronic devices corresponding to the multiple car manufacturers, aggregates the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain second model parameters, generates a second state recognition model based on the second model parameters, and sends the second state recognition model to the electronic devices corresponding to the multiple car manufacturers. For any electronic device corresponding to any car manufacturer, the electronic device receives the second state recognition model and fine-tunes the second state recognition model based on the local dataset to obtain a third state recognition model; The central server generates a second state recognition model based on the second model parameters, including: The central server sends the second model parameters to the electronic devices corresponding to the multiple car manufacturers; For any electronic device corresponding to any car manufacturer, the electronic device receives the second model parameters, updates the model parameters of the first state recognition model to the second model parameters, retrains the updated first state recognition model based on the local dataset, and sends the third model parameters of the trained first state recognition model to the central server. The central server receives the third model parameters sent by the electronic devices corresponding to the multiple car manufacturers, aggregates the third model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain the fourth model parameters, and sends the fourth model parameters to the electronic devices corresponding to the multiple car manufacturers. For any electronic device corresponding to any car manufacturer, the electronic device receives the fourth model parameters sent by the central server and determines whether the training count has reached a preset number. If the training count has reached the preset number, the model parameters of the first state recognition model are updated to the fourth model parameters to obtain the second state recognition model. If the training count has not reached the preset number, the model parameters of the first state recognition model are updated to the fourth model parameters, and then the updated first state recognition model is trained again based on the local dataset. This process is repeated until the training count reaches the preset number, at which point the second state recognition model is obtained. The electronic device updates the model parameters of the first state recognition model to the fourth model parameters to obtain a second state recognition model, including: When the fourth model parameters include an optimizer and a loss function, the electronic device sets the optimizer of the first state recognition model to the optimizer included in the fourth model parameters and sets the loss function of the first state recognition model to the loss function included in the fourth model parameters to obtain the second state recognition model.

2. The method according to claim 1, characterized in that, The central server aggregates the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain the second model parameters, including: The central server determines the average value of the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers, and obtains the second model parameters.

3. The method according to claim 1, characterized in that, The central server aggregates the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain the second model parameters, including: The central server determines the number of local datasets on the electronic devices corresponding to the multiple car manufacturers; based on the number of local datasets on the electronic devices corresponding to the multiple car manufacturers, it determines the weights of the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers; based on the weights of the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers, it performs a weighted summation of the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain the second model parameters.

4. The method according to claim 1, characterized in that, The central server generates a second state recognition model based on the second model parameters, and sends the second state recognition model to the electronic devices corresponding to the multiple car manufacturers, including: The central server updates the model parameters of the first state recognition model to the second model parameters and sends the updated first state recognition model to the electronic devices corresponding to the multiple car manufacturers. For any electronic device corresponding to any car manufacturer, the electronic device receives the updated first state recognition model, retrains the updated first state recognition model based on the local dataset, and sends the third model parameters of the trained first state recognition model to the central server. The central server receives the third model parameters sent by the electronic devices corresponding to the multiple car manufacturers, aggregates the third model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain the fourth model parameters, determines whether the number of training times has reached the preset number, and if the number of training times has reached the preset number, updates the model parameters of the first state recognition model to the fourth model parameters to obtain the second state recognition model, and sends the second state recognition model to the electronic devices corresponding to the multiple car manufacturers. If the number of training iterations has not reached the preset number, the model parameters of the first state recognition model are updated to the fourth model parameters, and the updated first state recognition model is sent to the electronic devices corresponding to the multiple car manufacturers. This process is repeated until the number of training iterations reaches the preset number, at which point the second state recognition model is obtained, and the second state recognition model is sent to the electronic devices corresponding to the multiple car manufacturers.

5. The method according to claim 1, characterized in that, The second state recognition module includes a fully connected layer and other network layers preceding the fully connected layer; The electronic device fine-tunes the second state recognition model based on a local dataset to obtain a third state recognition model, including: The electronic device fixes the model parameters of the other network layers and fine-tunes the model parameters of the fully connected layer based on the local dataset to obtain the third state recognition model.

6. A training system for a vehicle state recognition model, characterized in that, The system includes: The central server is used to send the first state recognition model to the electronic devices of multiple car manufacturers. For any electronic device corresponding to any car manufacturer, the electronic device is used to receive a first state recognition model sent by the central server, train the first state recognition model based on a local dataset, and send the first model parameters of the first state recognition model trained by the central server. The central server is used to receive first model parameters sent by electronic devices corresponding to the multiple car manufacturers, aggregate the first model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain second model parameters, generate a second state recognition model based on the second model parameters, and send the second state recognition model to the electronic devices corresponding to the multiple car manufacturers. For any electronic device corresponding to any car manufacturer, the electronic device is used to receive the second state recognition model and fine-tune the second state recognition model based on the local dataset to obtain a third state recognition model; The central server is used to send the second model parameters to the electronic devices corresponding to the multiple car manufacturers; For any electronic device corresponding to any car manufacturer, the electronic device is used to receive the second model parameters, update the model parameters of the first state recognition model to the second model parameters, retrain the updated first state recognition model based on the local dataset, and send the third model parameters of the trained first state recognition model to the central server. The central server is used to receive third model parameters sent by electronic devices corresponding to the multiple car manufacturers, aggregate the third model parameters sent by the electronic devices corresponding to the multiple car manufacturers to obtain fourth model parameters, and send the fourth model parameters to the electronic devices corresponding to the multiple car manufacturers. For any electronic device corresponding to any car manufacturer, the electronic device is used to receive the fourth model parameters sent by the central server and determine whether the training count has reached a preset number of times; if the training count has reached the preset number of times, the model parameters of the first state recognition model are updated to the fourth model parameters to obtain the second state recognition model; if the training count has not reached the preset number of times, the model parameters of the first state recognition model are updated to the fourth model parameters, and then the updated first state recognition model is trained again based on the local dataset, and so on, until the training count reaches the preset number of times, at which point the second state recognition model is obtained; When the fourth model parameters include an optimizer and a loss function, the electronic device is configured to set the optimizer of the first state recognition model to the optimizer included in the fourth model parameters, and set the loss function of the first state recognition model to the loss function included in the fourth model parameters, thereby obtaining the second state recognition model.

7. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one piece of program code, which is loaded and executed by the processor to implement the training method for the vehicle state recognition model as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the training method for the vehicle state recognition model as described in any one of claims 1 to 5.

9. A computer program product, characterized in that, The product stores at least one piece of program code, which is executed by a processor to implement the training method for the vehicle state recognition model as described in any one of claims 1 to 5.

Images