Fine-tuning method for vehicle classification model, vehicle classification method, device and equipment

By determining the parameter contribution of the candidate network layer in the vehicle classification model and dynamically adjusting the network layer, the problem of low fine-tuning efficiency of pre-trained models in the prior art is solved, and efficient fine-tuning and accuracy improvement of vehicle classification model is achieved.

CN116091824BActive Publication Date: 2024-11-29BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211731537.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-30
Publication Date
2024-11-29
Estimated Expiration
2042-12-30

AI Technical Summary

Technical Problem

The existing vehicle classification models are computationally expensive when fine-tuning pre-training models, are inefficient and have high resource requirements, and lack efficient model fine-tuning methods.

Method used

By determining the parameter contribution of candidate network layers, dynamically adjusting the network layer in the pre-trained model that performs gradient updates to achieve efficient model fine-tuning. The specific steps include: respectively determining the parameter contribution degree of network parameters in at least two candidate network layers, determining the target network layer based on the contribution degree, and updating the parameter of the target network layer during training.

Benefits of technology

It improves vehicle classification accuracy, achieves efficient model fine-tuning, reduces the demand for resources, and improves the fine-grained classification effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116091824B_ABST
    Figure CN116091824B_ABST
Patent Text Reader

Abstract

The present disclosure provides a fine-tuning method for a vehicle classification model, a vehicle classification method, an apparatus and a device, which relate to the field of artificial intelligence technology, especially to the field of image processing and deep learning technology, and can be used in vehicle identification scenarios. The specific implementation scheme is: respectively determine the parameter contribution of network parameters in at least two candidate network layers; the at least two candidate network layers both belong to the pre-classification model; according to the parameter contribution of the network parameters in the at least two candidate network layers, determine the target network layer from the at least two candidate network layers; train the pre-classification model according to the vehicle training data, and update the parameters of the target network layer during the training process to obtain a vehicle classification model. Through the above scheme, the accuracy of vehicle classification can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the field of image processing and deep learning technology, and can be used in vehicle recognition scenarios. Background Art

[0002] With the development of artificial intelligence technology, in the actual implementation of vehicle classification models in vehicle classification and recognition scenarios, it is often necessary to fine-tune the pre-trained model on general data (such as Imagenet dataset, etc.) for a second time on vehicle data. However, when the pre-trained model has a large amount of computation, fine-tuning the entire network is inefficient and requires high training resources. Therefore, an efficient model fine-tuning method is urgently needed. Summary of the invention

[0003] The present disclosure provides a vehicle classification model fine-tuning method, a vehicle classification method, a device and an apparatus.

[0004] According to one aspect of the present disclosure, a method for fine-tuning a vehicle classification model is provided, the method comprising:

[0005] Respectively determining parameter contributions of network parameters in at least two candidate network layers; the at least two candidate network layers both belong to the pre-classification model;

[0006] Determining a target network layer from the at least two candidate network layers according to parameter contributions of network parameters in the at least two candidate network layers;

[0007] The pre-classification model is trained according to the vehicle training data, and the parameters of the target network layer are updated during the training process to obtain a vehicle classification model.

[0008] According to another aspect of the present disclosure, a vehicle classification method is provided, the method comprising:

[0009] Obtain a vehicle image to be classified;

[0010] A vehicle classification model is used to classify the vehicle image to be classified to obtain a vehicle classification result; wherein the vehicle classification model is trained according to the fine-tuning method of the vehicle classification model described in any embodiment of the present disclosure.

[0011] According to another aspect of the present disclosure, a fine-tuning device for a vehicle classification model is provided, the device comprising:

[0012] A parameter contribution determination module, used to respectively determine the parameter contributions of network parameters in at least two candidate network layers; the at least two candidate network layers both belong to the pre-classification model;

[0013] A target network layer determination module, configured to determine a target network layer from the at least two candidate network layers according to parameter contributions of network parameters in the at least two candidate network layers;

[0014] The vehicle classification model determination module is used to train the pre-classification model according to the vehicle training data, and update the parameters of the target network layer during the training process to obtain the vehicle classification model.

[0015] According to another aspect of the present disclosure, there is provided a vehicle classification device, the device comprising:

[0016] The module for acquiring images to be classified is used to acquire vehicle images to be classified;

[0017] The vehicle classification result determination module is used to classify the vehicle image to be classified using a vehicle classification model to obtain a vehicle classification result; wherein the vehicle classification model is trained according to the fine-tuning method of the vehicle classification model described in any embodiment of the present disclosure.

[0018] According to another aspect of the present disclosure, there is provided an electronic device, the electronic device comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the fine-tuning method of the vehicle classification model or the vehicle classification method described in any embodiment of the present disclosure.

[0022] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the vehicle classification model fine-tuning method or vehicle classification method described in any embodiment of the present disclosure.

[0023] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the vehicle classification model fine-tuning method or vehicle classification method described in any embodiment of the present disclosure.

[0024] According to the technology disclosed in the present invention, it is possible to improve the accuracy of vehicle classification.

[0025] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0027] Figure 1 is a flow chart of a method for fine-tuning a vehicle classification model provided according to an embodiment of the present disclosure;

[0028] Figure 2 is a flow chart of another method for fine-tuning a vehicle classification model provided according to an embodiment of the present disclosure;

[0029] Figure 3 is a flowchart of another method for fine-tuning a vehicle classification model provided according to an embodiment of the present disclosure;

[0030] Figure 4 is a flow chart of a vehicle classification method provided according to an embodiment of the present disclosure;

[0031] Figure 5 is a structural schematic diagram of a fine-tuning device for a vehicle classification model provided according to an embodiment of the present disclosure;

[0032] Figure 6 is a structural schematic diagram of a vehicle classification device provided according to an embodiment of the present disclosure;

[0033] Figure 7 The present invention is a block diagram of an electronic device for implementing the vehicle classification model fine-tuning method or vehicle classification method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0035] It should be noted that the terms "candidate", "target", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] In addition, it should be noted that the collection, storage, use, processing, transmission, provision and disclosure of vehicle training data involved in the technical solution of the present invention are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0037] Figure 1 1 is a flow chart of a fine-tuning method for a vehicle classification model provided according to an embodiment of the present disclosure. This embodiment is applicable to the case of how to classify vehicles. The method can be executed by a fine-tuning device for a vehicle classification model, which can be implemented in software and / or hardware and can be integrated into an electronic device that carries the fine-tuning function of the vehicle classification model, such as a server. Figure 1 As shown, the fine-tuning method of the vehicle classification model of this embodiment may include:

[0038] S101, respectively determine parameter contributions of network parameters in at least two candidate network layers.

[0039] In this embodiment, at least two candidate network layers belong to pre-classification models; optionally, the candidate network layers may be convolutional layers, pooling layers, fully connected layers, etc.; wherein the pre-classification model may be a deep learning model pre-trained using general data (such as the Imagenet dataset, etc.).

[0040] The so-called network parameters refer to the parameters in each candidate network layer in the pre-classification model, such as network weights, etc. The so-called parameter contribution refers to the importance of the network parameters to the optimization of the classification loss of the pre-classification model during training; for each network parameter, the greater the parameter contribution of the network parameter, the greater the contribution of the network parameter to the optimization of the classification loss, and the smaller the contribution of the network parameter, the smaller the contribution of the network parameter to the optimization of the classification loss.

[0041] Specifically, the pre-classification model is trained once using vehicle training data to obtain network parameters in at least two candidate network layers in the pre-classification model. For each network parameter in each candidate network layer, a parameter contribution of the network parameter can be determined based on a preset parameter contribution determination function.

[0042] S102: Determine a target network layer from at least two candidate network layers according to parameter contributions of network parameters in at least two candidate network layers.

[0043] In this embodiment, the target network layer refers to the network layer in the pre-classification model that needs to update parameters in the subsequent iterative training process. It should be noted that the number of the target network layer is one or more.

[0044] Specifically, based on the parameter contribution of the network parameters, the network parameters in at least two candidate network layers can be sorted from largest to smallest, and then a preset number of network parameters ranked first are determined, and the candidate network layers to which these network parameters belong are used as target network layers.

[0045] S103, training the pre-classification model according to the vehicle training data, and updating the parameters of the target network layer during the training process to obtain a vehicle classification model.

[0046] In this embodiment, the vehicle training data refers to vehicle image data used for model fine-tuning.

[0047] Specifically, after determining the target network layer, the vehicle training data can be used to continue iterative training of the pre-classification model. In each iterative training process, the network parameters of the target network layer are updated, that is, the network parameters of the target network layer are gradient updated, and the network parameters of other candidate network layers in the pre-classification model except the target network layer are kept unchanged, that is, the parameters are fixed. Iterative training sets rounds to determine whether the training stop conditions are met. If the training stop conditions are met, the iterative training is stopped, and the corresponding pre-classification model when the training is stopped is used as the final vehicle classification model.

[0048] If the training stop condition is not met, the target network layer is re-determined from at least two candidate network layers according to the parameter contribution of the network parameters in at least two candidate network layers. Specifically, after the set rounds of iterative training, if the training stop condition is not met, continue to return to execute S101-103, re-determine the target network layer, and then use the vehicle training data to perform the set rounds of iterative training to determine whether the iterative stop condition is met. If it is met, determine the vehicle classification model; if not, continue to return to execute S101-S103 until the iterative stop condition is met to obtain the vehicle classification model.

[0049] The training stop condition refers to the condition for judging whether the model has converged, which may include the classification loss being stable within a set range, or the number of iterations reaching a threshold, etc. The set range and the threshold may be set by those skilled in the art according to actual conditions.

[0050] The technical solution provided by the embodiment of the present disclosure determines the parameter contribution of the network parameters in at least two candidate network layers respectively, and then determines the target network layer from the at least two candidate network layers according to the parameter contribution of the network parameters in the at least two candidate network layers, and then trains the pre-classification model according to the vehicle training data, and updates the parameters of the target network layer during the training process to obtain a vehicle classification model. Compared with the solution of updating the fixed network gradient in the existing fine-tuning technology, the above technical solution can dynamically adjust the network layer in the pre-training model that is gradient updated based on the network parameter contribution during the fine-tuning of the pre-training model, and can achieve efficient model fine-tuning, thereby obtaining a vehicle classification model with higher classification accuracy and improving the classification effect of vehicles at a more fine-grained level.

[0051] Figure 2 1 is a flow chart of another method for fine-tuning a vehicle classification model provided according to an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes "respectively determining the parameter contribution of network parameters in at least two candidate network layers" and provides an optional real-time solution. Figure 2 As shown, the fine-tuning method of the vehicle classification model of this embodiment may include:

[0052] S201, training a pre-classification model according to batch vehicle data of at least one training batch, to obtain classification losses and network parameters corresponding to at least two candidate network layers in at least one training batch.

[0053] In this embodiment, the batch vehicle data is determined from the vehicle training data. Optionally, the vehicle training data can be randomly divided into at least one batch vehicle data. The so-called training batch refers to a training performed using the batch vehicle data in the vehicle training data in an iterative round of training. It should be noted that each training round includes at least one training batch; for example, the vehicle training data contains 1,000 vehicle image data (i.e., samples). If the training batch contains 10 vehicle image data, then 100 training batches (iterations), i.e., one round of training (epoch), are required to train the 1,000 vehicle image data. In other words, to complete one round of training for all vehicle training data, 100 training batches of training are required.

[0054] Specifically, in a round of training, for each training batch, the batch vehicle data of the training batch is used to train the pre-classification model to obtain the classification loss and network parameters corresponding to each candidate network layer in the training batch.

[0055] S202, determining parameter contributions of network parameters in at least two candidate network layers respectively according to the classification loss and the network parameters.

[0056] In an optional manner, for each network parameter in each candidate network layer, the classification loss corresponding to the candidate network layer and the network parameter may be derived, and the obtained result may be used as the parameter contribution of the network parameter.

[0057] S203: Determine a target network layer from at least two candidate network layers according to parameter contributions of network parameters in at least two candidate network layers.

[0058] S204, training the pre-classification model according to the vehicle training data, and updating the parameters of the target network layer during the training process to obtain a vehicle classification model.

[0059] The technical solution provided by the embodiment of the present disclosure is to train the pre-classification model according to the batch vehicle data of at least one training batch, obtain the classification loss and network parameters corresponding to at least two candidate network layers in at least one training batch, and determine the parameter contribution of the network parameters in at least two candidate network layers according to the classification loss and network parameters, and then determine the target network layer from the at least two candidate network layers according to the parameter contribution of the network parameters in at least two candidate network layers, and then train the pre-classification model according to the vehicle training data, and update the parameters of the target network layer during the training process to obtain a vehicle classification model. The above technical solution determines the parameter contribution of the network parameters through the classification loss and the network parameters, and can reasonably and accurately obtain the parameter contribution, so that the target network layer can be flexibly determined to achieve dynamic and efficient model fine-tuning.

[0060] On the basis of the above embodiments, as an optional method of the present disclosure, respectively determining the parameter contribution of network parameters in at least two candidate network layers according to the classification loss and the network parameters can also be: determining the network parameter gradient according to the classification loss and the network parameters; and respectively determining the parameter contribution of network parameters in at least two candidate network layers according to the network parameter gradient and the network parameters.

[0061] Specifically, for each network parameter in each candidate network layer, the partial derivative between the classification loss corresponding to the candidate network layer and the network parameter can be calculated, and the obtained result can be used as the network parameter gradient. Then, the network parameter gradient of the network parameter and the network parameter can be subjected to dot product processing, and the obtained dot product result can be used as the parameter contribution of the network parameter. The dot product can be a Hadamard dot product.

[0062] It can be understood that further determining the parameter contribution by means of the network parameter gradient provides an optional method for determining the parameter contribution, thereby laying a foundation for the dynamic determination of the target network layer.

[0063] Figure 3 This is a flow chart of another method for fine-tuning a vehicle classification model provided according to an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes "determining a target network layer from at least two candidate network layers according to the parameter contribution of network parameters in at least two candidate network layers" and provides an optional implementation scheme. Figure 3 As shown, the fine-tuning method of the vehicle classification model of this embodiment may include:

[0064] S301: Determine parameter contributions of network parameters in at least two candidate network layers respectively.

[0065] Among them, at least two candidate network layers belong to the pre-classification model.

[0066] S302: Determine a target network contribution of a candidate network layer according to parameter contributions of network parameters in at least two candidate network layers.

[0067] In this embodiment, the target network contribution refers to the importance of the candidate network layer to the optimization of the current training classification loss.

[0068] In an optional manner, for each candidate network layer, the parameter contribution of each network parameter in the candidate network layer may be summed up, and the summed result may be used as the target network contribution of the candidate network layer. Furthermore, the parameter contribution of each network parameter of the candidate network layer may be averaged, and the average may be used as the target network contribution of the candidate network layer.

[0069] S303: Determine a target network layer from at least two candidate network layers according to the target network contribution.

[0070] Specifically, the target contributions may be ranked from large to small, and a set number of candidate network layers ranked high may be selected as target network layers.

[0071] S304, training the pre-classification model according to the vehicle training data, and updating the parameters of the target network layer during the training process to obtain a vehicle classification model.

[0072] The technical solution provided by the embodiment of the present disclosure determines the parameter contribution of the network parameters in at least two candidate network layers respectively, and then determines the target network contribution of the candidate network layer according to the parameter contribution of the network parameters in at least two candidate network layers, and determines the target network layer from at least two candidate network layers according to the target network contribution, and then trains the pre-classification model according to the vehicle training data, and updates the parameters of the target network layer during the training process to obtain a vehicle classification model. The above technical solution selects the target network layer based on the target network contribution of the candidate network layer, which can avoid the influence of selecting only individual network parameters in a candidate network layer as the target network layer because of their high parameter contribution, thereby improving the accuracy of determining the target network layer, and then realizing efficient fine-tuning training of the model.

[0073] On the basis of the above embodiments, as an optional method of the present disclosure, determining the target network contribution of the candidate network layer according to the parameter contribution of the network parameters in at least two candidate network layers can also be: determining the candidate network contribution of the candidate network layer in each training batch according to the parameter contribution of the network parameters of the candidate network layer in the training batch; determining the target network contribution of the candidate network layer according to the candidate network contribution of the candidate network layer in at least one training batch.

[0074] Among them, the candidate network contribution refers to the importance of the candidate network layer for optimizing the classification loss in each training batch.

[0075] Specifically, in each training batch, for each candidate network layer, the parameter contribution of each network parameter in the candidate network layer is summed, and the summed result is used as the candidate network contribution of the candidate network layer in the training batch. Furthermore, the parameter contribution of each network parameter in the candidate network layer can be averaged, and the average is used as the candidate network contribution of the candidate network layer in the training batch. Thus, the candidate network contribution of the candidate network layer in each training batch can be obtained. Afterwards, the candidate network contribution of the candidate network layer in each training batch is averaged, and the average is used as the target network contribution of the candidate network layer, that is, the candidate network contribution of the candidate network layer in each training batch is summed, and the summed result is divided by the total number of training batches to obtain the target network contribution of the candidate network layer.

[0076] It can be understood that by determining the target network contribution of the candidate network layer by counting the mean of the network parameter contribution of the candidate network layer in each training batch, it is possible to avoid the situation where a candidate network layer has a higher candidate network contribution in a certain batch of training while having lower candidate network contributions in other batches of training, thereby ensuring the reliability of the network contribution of the candidate network layer, thereby laying a solid foundation for accelerating model fine-tuning training.

[0077] Figure 4 1 is a flow chart of a vehicle classification method provided according to an embodiment of the present disclosure. This embodiment is applicable to the case of how to classify vehicles. The method can be executed by a vehicle classification device, which can be implemented in software and / or hardware and can be integrated into an electronic device that carries the vehicle classification function, such as a server. Figure 4 As shown, the vehicle classification method of this embodiment may include:

[0078] S401, obtaining a vehicle image to be classified.

[0079] In this embodiment, the vehicle image to be classified refers to an image that needs to be classified and identified as a vehicle.

[0080] Specifically, the vehicle images to be classified may be obtained through various channels; for example, the vehicle images to be classified may be obtained from the Internet; or, in another example, the vehicle images to be classified may be obtained in real time based on an image acquisition device, wherein the image acquisition device may be a camera, etc.

[0081] S402, using a vehicle classification model to classify the vehicle image to be classified to obtain a vehicle classification result.

[0082] In this embodiment, the vehicle classification model is a deep learning model for classifying vehicle images, and can be trained according to the fine-tuning method of the vehicle classification model provided in any of the above embodiments.

[0083] The vehicle classification result refers to the classification and recognition result of the vehicle image to be classified.

[0084] Specifically, the vehicle image to be classified may be input into a vehicle classification model, and after being processed by the model, a vehicle classification result of the vehicle image to be classified may be obtained.

[0085] The technical solution provided by the embodiment of the present disclosure obtains a vehicle image to be classified, and then uses a vehicle classification model to classify the vehicle image to be classified to obtain a vehicle classification result. The above technical solution uses the vehicle classification model obtained by model fine-tuning to perform vehicle classification, which can improve the vehicle classification effect.

[0086] Figure 51 is a schematic diagram of a fine-tuning device for a vehicle classification model according to an embodiment of the present disclosure. This embodiment is applicable to the case of how to classify vehicles. The device can be implemented in software and / or hardware, and can be integrated into an electronic device that carries the fine-tuning function of the vehicle classification model, such as a server. Figure 5 As shown, the fine-tuning device 500 of the vehicle classification model of this embodiment may include:

[0087] A parameter contribution determination module 501 is used to respectively determine the parameter contributions of network parameters in at least two candidate network layers; at least two candidate network layers both belong to the pre-classification model;

[0088] A target network layer determination module 502, configured to determine a target network layer from at least two candidate network layers according to parameter contributions of network parameters in at least two candidate network layers;

[0089] The vehicle classification model determination module 503 is used to train the pre-classification model according to the vehicle training data, and update the parameters of the target network layer during the training process to obtain the vehicle classification model.

[0090] The technical solution provided by the embodiment of the present disclosure determines the parameter contribution of the network parameters in at least two candidate network layers respectively, and then determines the target network layer from the at least two candidate network layers according to the parameter contribution of the network parameters in the at least two candidate network layers, and then trains the pre-classification model according to the vehicle training data, and updates the parameters of the target network layer during the training process to obtain a vehicle classification model. Compared with the solution of updating the fixed network gradient in the existing fine-tuning technology, the above technical solution can dynamically adjust the network layer in the pre-training model that is gradient updated based on the network parameter contribution during the fine-tuning of the pre-training model, and can achieve efficient model fine-tuning, thereby obtaining a vehicle classification model with higher classification accuracy and improving the classification effect of vehicles at a more fine-grained level.

[0091] Furthermore, the parameter contribution determination module 501 includes:

[0092] A network parameter determination unit, used to train the pre-classification model according to batch vehicle data of at least one training batch, respectively, to obtain classification losses and network parameters corresponding to at least two candidate network layers in at least one training batch; the batch vehicle data is determined from the vehicle training data;

[0093] The parameter contribution determination unit is used to determine the parameter contributions of the network parameters in at least two candidate network layers respectively according to the classification loss and the network parameters.

[0094] Furthermore, the parameter contribution determination unit is specifically used for:

[0095] Determine the network parameter gradient based on the classification loss and network parameters;

[0096] According to the network parameter gradient and the network parameter, parameter contributions of the network parameters in at least two candidate network layers are determined respectively.

[0097] Further, the target network layer determination module 502 specifically includes:

[0098] a target network contribution determination unit, configured to determine a target network contribution of a candidate network layer according to parameter contributions of network parameters in at least two candidate network layers;

[0099] The target network layer determining unit is used to determine the target network layer from at least two candidate network layers according to the target network contribution.

[0100] Furthermore, the target network contribution determination unit is specifically used to:

[0101] Determine the candidate network contribution of the candidate network layer in the training batch according to the parameter contribution of the network parameters of the candidate network layer in each training batch;

[0102] A target network contribution of the candidate network layer is determined according to the candidate network contribution of the candidate network layer in at least one training batch.

[0103] Furthermore, the device also includes:

[0104] The target network layer updating module is used to re-determine the target network layer from at least two candidate network layers according to the parameter contribution of the network parameters in at least two candidate network layers if the training stop condition is not met.

[0105] Figure 6 1 is a schematic diagram of a vehicle classification device according to an embodiment of the present disclosure. This embodiment is applicable to the case of how to classify vehicles. The device can be implemented in software and / or hardware, and can be integrated into an electronic device that carries the vehicle classification function, such as a server. Figure 6 As shown, the vehicle classification device 600 of this embodiment may include:

[0106] The to-be-classified image acquisition module 601 is used to acquire the to-be-classified vehicle image;

[0107] The vehicle classification result determination module 602 is used to classify the vehicle image to be classified using a vehicle classification model to obtain a vehicle classification result; wherein the vehicle classification model is trained according to the fine-tuning method of the vehicle classification model provided in any embodiment of the present disclosure.

[0108] The technical solution provided by the embodiment of the present disclosure obtains a vehicle image to be classified, and then uses a vehicle classification model to classify the vehicle image to be classified to obtain a vehicle classification result. The above technical solution uses the vehicle classification model obtained by model fine-tuning to perform vehicle classification, which can improve the vehicle classification effect.

[0109] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0110] Figure 7 The present invention is a block diagram of an electronic device for implementing the vehicle classification model fine-tuning method or vehicle classification method of the embodiment of the present disclosure. Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0111] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0112] Multiple components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0113] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as a fine-tuning method for a vehicle classification model or a vehicle classification method. For example, in some embodiments, a fine-tuning method for a vehicle classification model or a vehicle classification method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the fine-tuning method for the vehicle classification model or the vehicle classification method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute the fine-tuning method of the vehicle classification model or the vehicle classification method in any other appropriate manner (eg, by means of firmware).

[0114] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0115] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0116] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0117] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0118] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0119] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0120] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning technology, big data processing technology, knowledge graph technology, and other major directions.

[0121] Cloud computing refers to a technology system that uses network access to elastically scalable shared physical or virtual resource pools. Resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on demand and in a self-service manner. Cloud computing technology can provide efficient and powerful data processing capabilities for technical applications such as artificial intelligence and blockchain, as well as model training.

[0122] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0123] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A fine-tuning method for a vehicle classification model, comprising: Determining parameter contributions of network parameters in at least two candidate network layers respectively; The at least two candidate network layers both belong to the pre-classification model; the parameter contribution refers to the importance of the network parameters to the optimization of the classification loss of the pre-classification model during the training process; Sorting the network parameters in the at least two candidate network layers according to the parameter contributions of the network parameters in the at least two candidate network layers, determining a preset number of network parameters ranked first, and taking the candidate network layers to which they belong as the target network layers; The pre-classification model is trained according to the vehicle training data, and the parameters of the target network layer are updated during the training process to obtain a vehicle classification model.

2. The method according to claim 1, characterized in that The respectively determining parameter contributions of network parameters in at least two candidate network layers includes: The pre-classification model is trained according to batch vehicle data of at least one training batch, respectively, to obtain classification losses and network parameters corresponding to at least two candidate network layers in at least one training batch; the batch vehicle data is determined from the vehicle training data; According to the classification loss and the network parameters, parameter contributions of the network parameters in at least two candidate network layers are respectively determined.

3. The method according to claim 2, wherein: The step of determining parameter contributions of network parameters in at least two candidate network layers respectively according to the classification loss and the network parameters comprises: Determining a network parameter gradient according to the classification loss and the network parameter; According to the network parameter gradient and the network parameter, parameter contributions of the network parameters in at least two candidate network layers are respectively determined.

4. The method according to claim 2, wherein: The step of sorting the network parameters in the at least two candidate network layers according to the parameter contributions of the network parameters in the at least two candidate network layers, determining a preset number of network parameters ranked first, and taking the candidate network layers to which they belong as the target network layers includes: Determining a target network contribution of the candidate network layer according to parameter contributions of network parameters in the at least two candidate network layers; The target network contributions are sorted, a set number of target network contributions that are ranked top are selected, and the candidate network layers to which they belong are used as target network layers.

5. The method according to claim 4, wherein: The determining, according to the parameter contributions of the network parameters in the at least two candidate network layers, the target network contribution of the candidate network layer comprises: Determine the candidate network contribution of the candidate network layer in the training batch according to the parameter contribution of the network parameters of the candidate network layer in each training batch; A target network contribution of the candidate network layer is determined according to the candidate network contribution of the candidate network layer in at least one training batch.

6. The method according to claim 1, further comprising: If the training stop condition is not met, the target network layer is re-determined from the at least two candidate network layers according to the parameter contribution of the network parameters in the at least two candidate network layers.

7. A vehicle classification method, comprising: Obtain a vehicle image to be classified; A vehicle classification model is used to classify the vehicle image to be classified to obtain a vehicle classification result; wherein the vehicle classification model is trained according to the fine-tuning method of the vehicle classification model described in any one of claims 1-6.

8. A fine-tuning device for a vehicle classification model, comprising: A parameter contribution determination module, used to respectively determine the parameter contributions of network parameters in at least two candidate network layers; The at least two candidate network layers both belong to the pre-classification model; the parameter contribution refers to the importance of the network parameters to the optimization of the classification loss of the pre-classification model during the training process; a target network layer determination module, configured to sort the network parameters in the at least two candidate network layers according to the parameter contribution of the network parameters in the at least two candidate network layers, determine a preset number of network parameters ranked first, and use the candidate network layer to which they belong as the target network layer; The vehicle classification model determination module is used to train the pre-classification model according to the vehicle training data, and update the parameters of the target network layer during the training process to obtain the vehicle classification model.

9. The device according to claim 8, characterized in that The parameter contribution determination module includes: A network parameter determination unit, configured to train the pre-classification model according to batch vehicle data of at least one training batch, respectively, to obtain classification losses and network parameters corresponding to at least two candidate network layers in at least one training batch; the batch vehicle data is determined from the vehicle training data; A parameter contribution determination unit is used to determine the parameter contributions of network parameters in at least two candidate network layers respectively according to the classification loss and the network parameters.

10. The device according to claim 9, wherein: The parameter contribution determination unit is specifically used for: Determining a network parameter gradient according to the classification loss and the network parameter; According to the network parameter gradient and the network parameter, parameter contributions of the network parameters in at least two candidate network layers are respectively determined.

11. The device according to claim 9, wherein: The target network layer determination module specifically includes: a target network contribution determination unit, configured to determine a target network contribution of the candidate network layer according to parameter contributions of network parameters in the at least two candidate network layers; The target network layer determination unit is used to sort the target network contributions, select a set number of target network contributions that are ranked high, and use the candidate network layers to which they belong as target network layers.

12. The device according to claim 11, wherein The target network contribution determination unit is specifically used for: Determine the candidate network contribution of the candidate network layer in the training batch according to the parameter contribution of the network parameters of the candidate network layer in each training batch; A target network contribution of the candidate network layer is determined according to the candidate network contribution of the candidate network layer in at least one training batch.

13. The device according to claim 8, further comprising: The target network layer updating module is used to re-determine the target network layer from at least two candidate network layers according to the parameter contribution of the network parameters in the at least two candidate network layers if the training stop condition is not met.

14. A vehicle classification device, comprising: The module for acquiring images to be classified is used to acquire vehicle images to be classified; The vehicle classification result determination module is used to classify the vehicle image to be classified using a vehicle classification model to obtain a vehicle classification result; wherein the vehicle classification model is trained according to the fine-tuning method of the vehicle classification model according to any one of claims 1-6.

15. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the fine-tuning method of the vehicle classification model described in any one of claims 1-6, or the vehicle classification method described in claim 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable a computer to execute the fine-tuning method for a vehicle classification model according to any one of claims 1 to 6, or the vehicle classification method according to claim 7.

17. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the fine-tuning method of the vehicle classification model according to any one of claims 1 to 6, or the vehicle classification method according to claim 7.

Citation Information

Patent Citations

  • Vision-based multi-motion human body target tracking method in man-machine coexistence environment

    CN110363799A

  • Meta-learning-based target classification method and apparatus, device and storage medium

    WO2022134586A1