Vehicle re-identification model training method and device and vehicle re-identification method

By distilling the knowledge of multimodal large language model into the vehicle re-identification task and training model parameters in two stages, the problem of poor performance of vehicle re-identification model in the existing technology is solved, and accuracy is achieved in complex environments Vehicle identification and positioning.

CN118864802BActive Publication Date: 2024-12-24INST OF AUTOMATION CHINESE ACAD OF SCI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411345362.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-25
Publication Date
2024-12-24
Estimated Expiration
2044-09-25

AI Technical Summary

Technical Problem

In the prior art, the text-based vehicle re-identification model has poor performance and is difficult to effectively identify and locate target vehicles in a complex traffic monitoring system.

Method used

By distilling the knowledge of multimodal large language models into the vehicle re-identification task, high-quality vehicle images-description pair data sets are generated, and model parameters are trained in two stages, and global and local features are adjusted respectively to improve model performance.

Benefits of technology

It significantly improves the cross-modal retrieval performance of the vehicle re-identification model, solves the problems of data shortage and poor model performance, and realizes accurate vehicle recognition in complex environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118864802B_ABST
    Figure CN118864802B_ABST
Patent Text Reader

Abstract

The present application discloses a training method and device for a vehicle re-identification model and a vehicle re-identification method. The training method includes: obtaining pre-prompt words guided by preset attributes, preset sample learning prompt words, a first preset vehicle re-identification data set, and a second preset vehicle re-identification data set based on preset annotations; inputting the pre-prompt words guided by preset attributes, the preset sample learning prompt words, and the first preset vehicle re-identification data set into a preset multimodal large language model to generate a first vehicle image-description pair data set; using the first vehicle image-description pair data set to perform a first stage of training on the vehicle re-identification model to obtain a trained vehicle re-identification model; using the second preset vehicle re-identification data set to perform a second stage of training on the trained vehicle re-identification model to obtain a final vehicle re-identification model, and in each stage of training, the distance between the features corresponding to the vehicle image and the features corresponding to the corresponding description is minimized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure generally relates to the technical field of visual language model algorithms and the technical field of natural language processing, and more specifically, to a training method and device for a vehicle re-identification model and a vehicle re-identification method. Background Art

[0002] In the relevant field, vehicle re-identification means the technology of finding the same designated vehicle in images or video sequences taken by different cameras. This technology plays a key role in the construction of intelligent transportation systems and smart cities, especially in the distributed network camera environment, where it is particularly important for cross-camera tracking and identification of vehicles. For example, it has a strong application scenario in intelligent video surveillance systems, which can track the trajectory of the target and find the target's location. The goal of vehicle re-identification is to locate and supervise the target vehicle in complex traffic monitoring systems.

[0003] However, in many practical application scenarios, what can be obtained is not the image or video of the vehicle, but the description of the vehicle. Therefore, text-based vehicle re-identification technology is widely needed in security monitoring, intelligent retrieval and other fields. Therefore, an improved vehicle re-identification algorithm is needed. Summary of the invention

[0004] Embodiments of the present disclosure provide a method and device for training a vehicle re-identification model and a vehicle re-identification method, which implement modeling of a vehicle re-identification model by distilling the knowledge of a multimodal large language model (which may be referred to as a "multimodal large model") into a text-based vehicle re-identification task.

[0005] In one general aspect, a method for training a vehicle re-identification model is provided, the training method comprising: obtaining pre-prompt words guided by preset attributes, preset sample learning prompt words, a first preset vehicle re-identification data set, and a second preset vehicle re-identification data set based on preset annotations; inputting the pre-prompt words guided by preset attributes, the preset sample learning prompt words, and the first preset vehicle re-identification data set into a preset multimodal large language model to generate a first vehicle image-description pair data set; performing a first stage of training on the vehicle re-identification model using the first vehicle image-description pair data set to obtain a trained vehicle re-identification model; performing a second stage of training on the trained vehicle re-identification model using the second preset vehicle re-identification data set to obtain a final vehicle re-identification model, wherein, in the training process of each stage, the target cost for adjusting the model parameters is to minimize the distance between the feature corresponding to the vehicle image and the feature corresponding to the corresponding description.

[0006] Optionally, the model parameters trained during the first stage of training may include parameters related to global features but not parameters related to local features, and the model parameters trained during the second stage of training may include both parameters related to global features and parameters related to local features.

[0007] Optionally, the vehicle image data in the first vehicle image-description pair dataset may be different from the vehicle image data in the second preset vehicle re-identification dataset.

[0008] Optionally, the step of inputting the pre-prompt words guided by the preset attributes, the preset sample learning prompt words and the first preset vehicle re-identification data set into a preset multimodal large language model to generate a first vehicle image-description pair data set may include: for each sub-data set in the first preset vehicle re-identification data set, performing the following processing: inputting the pre-prompt words guided by the preset attributes, the preset sample learning prompt words and each sub-data set into the preset multimodal large language model to generate a corresponding vehicle image-description pair sub-data set; removing data that does not meet preset conditions from the corresponding vehicle image-description pair sub-data set to obtain an output result corresponding to each sub-data set, and combining the output results corresponding to each sub-data set as the first vehicle image-description pair data set.

[0009] Optionally, the vehicle re-identification model may be a neural network-based image-description model.

[0010] In another general aspect, a vehicle re-identification method is provided, the vehicle re-identification method comprising: obtaining vehicle description data to be identified; inputting the vehicle description data to be identified into a vehicle re-identification model to obtain corresponding vehicle image data, wherein the vehicle re-identification model is trained using the training method described above.

[0011] In another general aspect, a training device for a vehicle re-identification model is provided, the training device comprising: a data acquisition module, configured to: acquire pre-prompt words guided by preset attributes, preset sample learning prompt words, a first preset vehicle re-identification data set and a second preset vehicle re-identification data set based on preset annotations; a data generation module, configured to: input the pre-prompt words guided by preset attributes, the preset sample learning prompt words and the first preset vehicle re-identification data set into a preset multimodal large language model to generate a first vehicle image-description pair data set; a model training module, configured to: perform a first stage of training on the vehicle re-identification model using the first vehicle image-description pair data set to obtain a trained vehicle re-identification model, and perform a second stage of training on the trained vehicle re-identification model using the second preset vehicle re-identification data set to obtain a final vehicle re-identification model, wherein, in the training process of each stage, the target cost for adjusting the model parameters is to minimize the distance between the feature corresponding to the vehicle image and the feature corresponding to the corresponding description.

[0012] Optionally, the model parameters trained during the first stage of training may include parameters related to global features but not parameters related to local features, and the model parameters trained during the second stage of training may include both parameters related to global features and parameters related to local features.

[0013] Optionally, the vehicle image data in the first vehicle image-description pair dataset may be different from the vehicle image data in the second preset vehicle re-identification dataset.

[0014] Optionally, the data generation module inputs the pre-prompt words guided by the preset attributes, the preset sample learning prompt words and the first preset vehicle re-identification data set into a preset multimodal large language model, and the operation of generating the first vehicle image-description pair data set may include: for each sub-data set in the first preset vehicle re-identification data set, performing the following processing: inputting the pre-prompt words guided by the preset attributes, the preset sample learning prompt words and each sub-data set into the preset multimodal large language model to generate a corresponding vehicle image-description pair sub-data set; removing data that does not meet preset conditions from the corresponding vehicle image-description pair sub-data set to obtain an output result corresponding to each sub-data set, and combining the output results corresponding to each sub-data set as the first vehicle image-description pair data set.

[0015] Optionally, the vehicle re-identification model may be a neural network-based image-description model.

[0016] In another general aspect, a computer program product is provided, the computer program product comprising a computer program / instruction, which when executed by a processor implements the training method of the vehicle re-identification model as described above and the vehicle re-identification method as described above.

[0017] In another general aspect, a computer-readable storage medium is provided, which, when instructions in the computer-readable storage medium are executed by a processor of an electronic device / server, enables the electronic device / server to execute the vehicle re-identification model training method described above and the vehicle re-identification method described above.

[0018] In another general aspect, a computing device is provided, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein the computer executable instructions, when executed by the at least one processor, cause the at least one processor to execute the vehicle re-identification model training method and the vehicle re-identification method as described above.

[0019] According to the vehicle re-identification model training method and device and vehicle re-identification method of the embodiments of the present disclosure, the vehicle re-identification model is modeled by distilling the knowledge of the multimodal large model into the text-based vehicle re-identification task. In addition, the cross-modal retrieval performance of the vehicle re-identification model is improved by transferring the knowledge of the multimodal large model from the generated high-quality vehicle description to the training of the vehicle re-identification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other objects and features of the embodiments of the present disclosure will become more apparent through the following description in conjunction with the accompanying drawings showing the embodiments, in which:

[0021] Figure 1 is a flow chart showing a method for training a vehicle re-identification model according to an embodiment of the present disclosure;

[0022] Figure 2 is a flowchart illustrating an example of a method for training a vehicle re-identification model according to an embodiment of the present disclosure;

[0023] Figure 3 is a schematic diagram showing an exemplary attribute-guided pre-prompt word according to an embodiment of the present disclosure;

[0024] Figure 4 is a schematic diagram showing exemplary sample learning prompt words according to an embodiment of the present disclosure;

[0025] Figure 5 is a flow chart showing a vehicle re-identification method according to an embodiment of the present disclosure;

[0026] Figure 6 is a block diagram showing a training apparatus for a vehicle re-identification model according to an embodiment of the present disclosure;

[0027] Figure 7 is a block diagram illustrating a computing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The following specific embodiments are provided to help the reader obtain a comprehensive understanding of the methods, devices and / or systems described herein. However, after understanding the disclosure of the present application, various changes, modifications and equivalents of the methods, devices and / or systems described herein will be clear. For example, the order of operations described herein is only an example and is not limited to those orders set forth herein, but can be changed as will be clear after understanding the disclosure of the present application, except for operations that must occur in a specific order. In addition, for greater clarity and simplicity, the description of features known in the art may be omitted.

[0029] Reference will now be made in detail to the embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like parts throughout. The embodiments will be described below with reference to the drawings in order to explain the present disclosure.

[0030] In the related art, text-based vehicle re-identification aims to search for target vehicles that meet the characteristics based on text descriptions. Currently, research in this field is in its initial stage, and one of the main problems it faces is the poor performance of the model.

[0031] In order to solve the problems in the related art, the present disclosure proposes a training method for a vehicle re-identification model, which can distill the knowledge of a large multimodal model into a text-based vehicle re-identification task. By transferring the knowledge of the large multimodal model from the generated high-quality vehicle description to the training of the vehicle re-identification model, the multimodal retrieval performance of the vehicle re-identification model is improved.

[0032] Refer to the following Figures 1 to 7 A vehicle re-identification model training method and device and a vehicle re-identification method according to an embodiment of the present disclosure are described in detail.

[0033] Figure 1 is a flowchart illustrating a method 100 for training a vehicle re-identification model according to an embodiment of the present disclosure.

[0034] Reference Figure 1 According to an embodiment of the present disclosure, in step S101, a pre-prompt word guided by a preset attribute, a preset sample learning prompt word, a first preset vehicle re-identification data set, and a second preset vehicle re-identification data set based on preset annotations are obtained.

[0035] Here, according to the embodiments of the present disclosure, by setting preset attribute-guided pre-prompt words and preset sample learning prompt words, it is possible to guide the multimodal large model to generate high-quality fine-grained descriptions for large-scale vehicle images by using attribute-guided pre-prompts and optimized few-sample learning strategies.

[0036] According to an embodiment of the present disclosure, in step S102, preset attribute-guided pre-prompt words, preset sample learning prompt words and a first preset vehicle re-identification data set are input into a preset multimodal large language model to generate a first vehicle image-description pair data set.

[0037] Here, the vehicle image data in the first vehicle image-description pair dataset may be different from the vehicle image data in the second preset vehicle re-identification dataset.

[0038] According to an embodiment of the present disclosure, through the above-mentioned method of training for different vehicle images, full use is made of the data generated by the multimodal large model to implement knowledge distillation (for example, high-quality fine-grained descriptions of vehicle images), thereby solving the problem of poor model performance caused by data shortage in text-based vehicle re-identification tasks.

[0039] As an example, step S102 may further include: for each sub-data set in the first preset vehicle re-identification data set (here, the sizes of the sub-data sets may be the same or different), performing the following processes S1021 to S1022:

[0040] In processing S1021, the preset attribute-guided pre-prompt words, the preset sample learning prompt words and each sub-dataset are input into the preset multimodal large language model to generate a corresponding vehicle image-description pair sub-dataset.

[0041] In process S1022, data that does not meet a preset condition is removed from the corresponding vehicle image-description pair sub-datasets to obtain an output result corresponding to each sub-dataset.

[0042] For example, the preset condition may be a relevant condition to ensure the availability of the description data, and the data that does not meet the preset condition may include, for example but not limited to, unavailable data such as rejection reply data, garbled reply data, and error reply data.

[0043] Next, after obtaining the output results corresponding to all the sub-datasets, the output results corresponding to each sub-dataset are combined as the first vehicle image-description pair dataset.

[0044] According to an embodiment of the present disclosure, a method for distilling the knowledge of a large multimodal model into a text-based vehicle re-identification task is implemented through the above-mentioned approach.

[0045] According to an embodiment of the present disclosure, in step S103, a first stage of training is performed on a vehicle re-identification model using a first vehicle image-description pair dataset to obtain a trained vehicle re-identification model.

[0046] Here, the vehicle re-identification model may be an image-description model based on various neural networks, and the present disclosure does not impose any specific limitation on the neural network.

[0047] According to the embodiments of the present disclosure, the above method solves the problem that large multimodal models are not suitable for high-throughput scenario deployment on the end. This application successfully constructs a high-throughput text-image vehicle re-identification model on the end by distilling the image-text description knowledge generated by the large multimodal model into the neural network on the end.

[0048] According to an embodiment of the present disclosure, the model parameters trained in the first stage of training may include parameters related to global features but not parameters related to local features.

[0049] According to the embodiments of the present disclosure, since the data generated from the multimodal large model as the source domain and the data manually annotated or described are constructed in different ways, their data distributions are different. Therefore, by only training the source domain and global features in the first stage without considering the detailed feature information manually annotated or described, it is possible to prevent the model from overfitting on the source domain.

[0050] According to an embodiment of the present disclosure, in step S104, the trained vehicle re-identification model is trained in a second stage using a second preset vehicle re-identification data set to obtain a final vehicle re-identification model.

[0051] According to an embodiment of the present disclosure, the model parameters trained in the training process of the second stage may include both parameters related to global features and parameters related to local features.

[0052] Here, by training both global and local features in the second stage, all detailed partial descriptions of the vehicle image can be fully recognized, so that the trained model is more accurate.

[0053] According to an embodiment of the present disclosure, during the second stage of training, the target cost for adjusting the model parameters is to minimize the distance between the features corresponding to the vehicle image and the features corresponding to the corresponding description.

[0054] According to the embodiments of the present disclosure, by adopting the above-mentioned vehicle re-identification model training method, text-search-image vehicle re-identification modeling based on multimodal large model knowledge distillation is implemented. In this process, the knowledge of the multimodal large model is transferred to the end-side model training through the generated high-quality description, thereby improving the cross-modal retrieval performance of the model.

[0055] Below, refer to Figure 2 An example of a method for training a vehicle re-identification model according to the present disclosure is described. Figure 2 is a flowchart illustrating an example of a training method of a vehicle re-identification model according to an embodiment of the present disclosure.

[0056] Reference Figure 2 , in step S201, a pre-prompt word for attribute guidance is designed.

[0057] For example, Figure 3 As an example, the pre-prompt word can be based on the ordinary prompt word, adding attribute prompts (car model, color, angle, etc.) (for example, Figure 3 ) and property options (with reference ranges given after the property prompt) (e.g. Figure 3 (shown in bold black text in the ).

[0058] In step S202, an optimized few-sample learning prompt word is designed.

[0059] For example, Figure 4 As an example, the exemplary sample learning prompt words can be used to learn the prompt words for 5 representative vehicle pictures (including different models, angles, colors, etc.) (for example, Figure 4 As shown in the left column of vehicle images in , design corresponding vehicle descriptions (including fine-grained, discriminative feature descriptions) (for example, Figure 4 (shown in blue text in the ).

[0060] In step S203, the vehicle image data to be described is filtered from the preset vehicle re-identification dataset, the pre-prompt words obtained in step S201 and the optimized few-sample learning prompt words obtained in step S202 are integrated to obtain complete prompt words, and the API of a multimodal large model (such as but not limited to GPT4-V) is called, and the vehicle image data to be described and the complete prompt words are input to obtain a fine-grained description of the vehicle image.

[0061] In step S204, the fine-grained description of the vehicle image obtained in step S203 is cleaned up, that is, unusable outputs such as rejection replies, garbled replies, and error replies are deleted.

[0062] In step S205, it is determined whether all descriptions of all vehicle images are available. If not, the process returns to step S203 until all descriptions of all vehicle images are available.

[0063] Here, it can be understood that all vehicle images for which descriptions are to be generated are divided into several parts, and steps S203 and S204 are respectively performed for each part until all available descriptions of all vehicle images are obtained.

[0064] In addition, the large-scale, high-quality vehicle image-description pair dataset obtained here is used as part of the training data for subsequent model training.

[0065] In step S206, the first stage of training is performed on the neural network-based vehicle re-identification model using the vehicle image-description pair dataset generated above: the generated vehicle image-description pair dataset is used as the model training performed on the source domain.

[0066] It should be noted that this first stage only aligns global features to train common feature representations (e.g., common features related to vehicles), thereby preventing the model from overfitting on the source domain.

[0067] In step S207, based on the model trained in step S206, the model is trained in the second stage (ie, fine-tuned): a data set with preset annotations (eg, manually annotated) is used as the model training on the target domain.

[0068] Here, the preset annotated dataset is compared with the previous vehicle image-description pair dataset. Assuming that the latter includes image A and the corresponding text description (e.g., text description) generated by a multimodal large model, the former includes image B different from image A and the corresponding annotated text.

[0069] It should be noted that this second stage aligns both global features and local features (e.g., vehicle-related detail features) simultaneously, further enhancing the performance of the model on the target domain.

[0070] In this example, the cost function of the training model involved in steps S206 and S207 may be to minimize the distance between the features corresponding to the image and the features corresponding to the text description.

[0071] Here, although the text description exemplified in the present disclosure is in English, the present disclosure is not limited thereto, and may also be in Chinese or other available text formats.

[0072] Here, it should be noted that the above Figure 2 The description of each step is merely exemplary, and the steps in the method according to the present disclosure are not limited thereto.

[0073] Figure 5 is a flow chart illustrating a vehicle re-identification method 500 according to an embodiment of the present disclosure.

[0074] Reference Figure 5, in step S501, the vehicle description data to be identified is obtained.

[0075] In step S502, the vehicle description data to be identified is input into a vehicle re-identification model to obtain corresponding vehicle image data.

[0076] Here, the vehicle re-identification model is trained using the training method 100 described above.

[0077] Figure 6 is a block diagram illustrating a training apparatus 600 for a vehicle re-identification model according to an embodiment of the present disclosure.

[0078] Reference Figure 6 According to an embodiment of the present disclosure, a training device 600 for a vehicle re-identification model may include a data acquisition module 610 , a data generation module 620 and a model training module 630 .

[0079] According to an embodiment of the present disclosure, the data acquisition module 610 is configured to: acquire pre-prompt words guided by preset attributes, preset sample learning prompt words, a first preset vehicle re-identification data set, and a second preset vehicle re-identification data set based on preset annotations.

[0080] According to an embodiment of the present disclosure, the data generation module 620 is configured to: input the pre-prompt words guided by preset attributes, the preset sample learning prompt words and the first preset vehicle re-identification data set into the preset multimodal large language model to generate a first vehicle image-description pair data set.

[0081] As an example, the vehicle image data in the first vehicle image-description pair dataset may be different from the vehicle image data in the second preset vehicle re-identification dataset.

[0082] As an example, the operation of the data generation module to generate the first vehicle image-description pair dataset may include: for each sub-dataset in the first preset vehicle re-identification dataset, performing the following processes (1) and (2):

[0083] (1) Input the preset attribute-guided pre-prompt words, the preset sample learning prompt words, and each sub-dataset into a preset multimodal large language model to generate a corresponding vehicle image-description pair sub-dataset.

[0084] (2) Removing data that does not meet a preset condition from the corresponding vehicle image-description pair sub-datasets to obtain an output result corresponding to each sub-dataset, and combining the output results corresponding to each sub-dataset to form a first vehicle image-description pair data set.

[0085] According to an embodiment of the present disclosure, the model training module 630 is configured to: perform a first stage of training on a vehicle re-identification model using a first vehicle image-description pair data set to obtain a trained vehicle re-identification model, and perform a second stage of training on the trained vehicle re-identification model using a second preset vehicle re-identification data set to obtain a final vehicle re-identification model.

[0086] Here, during each stage of training, the objective cost used to adjust the model parameters is to minimize the distance between the features corresponding to the vehicle image and the features corresponding to the corresponding description.

[0087] As an example, in the first stage of training, global features may be trained without training local features, and in the second stage of training, both global features and local features may be trained.

[0088] In addition, the vehicle re-identification model can be a neural network-based image-description model.

[0089] It should be noted that the operations performed on the above-mentioned structural frames can be referred to in Figure 1 The related contents described are similar and will not be repeated here.

[0090] Figure 7 is a block diagram illustrating a computing device 700 according to an embodiment of the present disclosure.

[0091] Reference Figure 7 According to an embodiment of the present disclosure, the computing device 700 may include a processor 710 and a memory 720. The processor 710 may include (but not limited to) a central processing unit (CPU), a digital signal processor (DSP), a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a microprocessor, an application-specific integrated circuit (ASIC), etc. The memory 720 may store computer-executable instructions to be executed by the processor 710. The memory 720 includes a high-speed random access memory and / or a non-volatile computer-readable storage medium. When the processor 710 executes the computer-executable instructions stored in the memory 720, the training method of the vehicle re-identification model and the vehicle re-identification method as described above may be implemented.

[0092] The evaluation method according to the embodiment of the present disclosure may be written as a computer program / instruction to form a computer program product and stored on a computer-readable storage medium. When the computer program / instruction is executed by a processor, the training method of the vehicle re-identification model and the vehicle re-identification method as described above may be implemented. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device / server, the electronic device / server is enabled to execute the training method of the vehicle re-identification model and the vehicle re-identification method as described above. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. In one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0093] According to the vehicle re-identification model training method and device and vehicle re-identification method of the embodiments of the present disclosure, the vehicle re-identification model is modeled by distilling the knowledge of a large multimodal model into a text-based vehicle re-identification task.

[0094] On the other hand, according to the vehicle re-identification model training method and device and the vehicle re-identification method of the embodiments of the present disclosure, the cross-modal retrieval performance of the vehicle re-identification model is improved by transferring the knowledge of the multimodal large model from the generated high-quality vehicle description to the training of the vehicle re-identification model.

[0095] On the other hand, according to the vehicle re-identification model training method and device and the vehicle re-identification method of the embodiments of the present disclosure, by making full use of the excellent performance of the multimodal large model in many multimodal downstream tasks (that is, the rich knowledge obtained by training on massive image and text data), the generated large-scale, high-quality vehicle image-description pair dataset is applied to the text-based vehicle re-identification task, thereby greatly improving the performance of the vehicle re-identification model.

[0096] Although some embodiments of the present disclosure have been disclosed and described, it will be appreciated by those skilled in the art that modifications and variations may be made to the embodiments without departing from the concept and spirit of the present disclosure, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for training a vehicle re-identification model, characterized in that: The training method comprises: Acquire a pre-prompt word guided by a preset attribute, a preset sample learning prompt word, a first preset vehicle re-identification data set, and a second preset vehicle re-identification data set based on a preset annotation; Inputting the preset attribute-guided pre-prompt words, the preset sample learning prompt words, and the first preset vehicle re-identification data set into a preset multimodal large language model to generate a first vehicle image-description pair data set; Performing a first phase of training on a vehicle re-identification model using the first vehicle image-description pair dataset to obtain a trained vehicle re-identification model; The trained vehicle re-identification model is trained in the second stage using the second preset vehicle re-identification data set to obtain a final vehicle re-identification model. In each stage of training, the target cost for adjusting the model parameters is to minimize the distance between the features corresponding to the vehicle image and the features corresponding to the corresponding description. The step of inputting the preset attribute-guided pre-prompt words, the preset sample learning prompt words and the first preset vehicle re-identification data set into a preset multimodal large language model to generate a first vehicle image-description pair data set includes: For each sub-data set in the first preset vehicle re-identification data set, the following processing is performed: Inputting the preset attribute-guided pre-prompt words, the preset sample learning prompt words and each sub-dataset into the preset multimodal large language model to generate a corresponding vehicle image-description pair sub-dataset; Data that does not meet preset conditions are removed from the corresponding vehicle image-description pair sub-datasets to obtain output results corresponding to each sub-dataset, and the output results corresponding to each sub-dataset are combined as the first vehicle image-description pair data set.

2. The training method according to claim 1, characterized in that: The model parameters trained in the first stage of training include parameters related to global features but not parameters related to local features, and the model parameters trained in the second stage of training include both parameters related to global features and parameters related to local features.

3. The training method according to claim 1, characterized in that: The vehicle image data in the first vehicle image-description pair dataset is different from the vehicle image data in the second preset vehicle re-identification dataset.

4. The training method according to claim 1, characterized in that: The vehicle re-identification model is an image-description model based on a neural network.

5. A vehicle re-identification method, characterized in that: The vehicle re-identification method comprises: Obtaining vehicle description data to be identified; Input the vehicle description data to be identified into the vehicle re-identification model to obtain the corresponding vehicle image data, The vehicle re-identification model is trained using the training method described in any one of claims 1 to 4.

6. A training device for a vehicle re-identification model, characterized in that: The training device comprises: A data acquisition module is configured to: acquire a pre-prompt word guided by a preset attribute, a preset sample learning prompt word, a first preset vehicle re-identification data set, and a second preset vehicle re-identification data set based on a preset annotation; A data generation module is configured to: input the preset attribute-guided pre-prompt words, the preset sample learning prompt words and the first preset vehicle re-identification data set into a preset multimodal large language model to generate a first vehicle image-description pair data set; The model training module is configured to: perform a first-stage training on a vehicle re-identification model using the first vehicle image-description pair data set to obtain a trained vehicle re-identification model, and perform a second-stage training on the trained vehicle re-identification model using the second preset vehicle re-identification data set to obtain a final vehicle re-identification model, In each stage of training, the target cost for adjusting the model parameters is to minimize the distance between the features corresponding to the vehicle image and the features corresponding to the corresponding description. The step of inputting the preset attribute-guided pre-prompt words, the preset sample learning prompt words and the first preset vehicle re-identification data set into a preset multimodal large language model to generate a first vehicle image-description pair data set includes: For each sub-data set in the first preset vehicle re-identification data set, the following processing is performed: Inputting the preset attribute-guided pre-prompt words, the preset sample learning prompt words and each sub-dataset into the preset multimodal large language model to generate a corresponding vehicle image-description pair sub-dataset; Data that does not meet preset conditions are removed from the corresponding vehicle image-description pair sub-datasets to obtain output results corresponding to each sub-dataset, and the output results corresponding to each sub-dataset are combined as the first vehicle image-description pair data set.

7. A computer program product, characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the training method of the vehicle re-identification model according to any one of claims 1 to 4 and the vehicle re-identification method according to claim 5 are implemented.

8. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the vehicle re-identification model training method according to any one of claims 1 to 4 and the vehicle re-identification method according to claim 5.

9. A computing device, characterized in that The computing device includes: at least one processor; and at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to execute the vehicle re-identification model training method according to any one of claims 1 to 4 and the vehicle re-identification method according to claim 5.