Vehicle retrieval method, device, equipment, medium and program product

Abstract

The embodiment of the invention discloses a vehicle retrieval method, device and equipment, a medium and a program product. Comprising the steps of obtaining to-be-retrieved vehicle information; the to-be-retrieved vehicle information at least comprises character type vehicle information and a to-be-retrieved vehicle picture set; inputting the to-be-retrieved vehicle information into a pre-trained vehicle retrieval model, and determining a vehicle retrieval result according to a model output result of the vehicle retrieval model; wherein the vehicle retrieval model is a cross-modal pre-training model obtained by training based on a multi-source vehicle feature training sample set; wherein the multi-source vehicle features corresponding to the multi-source vehicle feature training sample set at least comprise at least one of vehicle appearance features, license plate features and in-vehicle article features. The method improves the precision of targeted retrieval of the vehicle, can adapt to the demand of combination of natural language retrieval and image retrieval of a user, and can better adapt to scenes such as a current intelligent traffic system, security and protection monitoring and an intelligent city.

Description

Technical Field

[0001] This invention relates to the field of image retrieval technology, and in particular to a vehicle retrieval method, apparatus, device, medium, and program product. Background Technology

[0002] With the rapid development of intelligent transportation systems, security monitoring, and smart cities, the application scope of image-based vehicle retrieval technology is gradually expanding. This technology aims to quickly and accurately locate target vehicles from massive video or image libraries based on text descriptions or input query images.

[0003] However, current vehicle retrieval and location methods often rely on single-modal features, such as vehicle appearance features or license plate recognition. These methods depend on manually designed features or visual features extracted by deep neural networks. However, relying solely on appearance features makes query results susceptible to factors like lighting, angle, and occlusion, and fails to distinguish between vehicles with similar appearances. Conversely, relying solely on license plate information cannot achieve accurate vehicle retrieval in scenarios where the license plate is obscured, damaged, or at excessive distance.

[0004] Meanwhile, the significant differences between visual features and human natural language descriptions make it difficult to support flexible natural language queries. To address this, a cross-modal pre-trained model (CLIP) was introduced. This model learns through comparative studies on massive image-text pairs to achieve accurate vehicle retrieval. However, the general CLIP model is trained only on natural images, lacking sufficient understanding of fine-grained vehicle features and specialized components. Furthermore, the training data is typically generalized, failing to extract effective vehicle-specific information. This limits the accuracy of the general CLIP model in vehicle retrieval tasks and fails to meet the practical application needs of current intelligent transportation systems, security monitoring, and smart cities. Summary of the Invention

[0005] This invention provides a vehicle retrieval method, device, equipment, medium, and program product. Based on a vehicle retrieval model that fully considers vehicle appearance features, license plate features, and features of items inside the vehicle, it has cross-modal retrieval capabilities, enabling the retrieval of target vehicles from massive video and image libraries. This improves the accuracy of targeted vehicle retrieval and can adapt to users' needs for combining natural language retrieval and image retrieval, making it better suited for current intelligent transportation systems, security monitoring, and smart city scenarios.

[0006] In a first aspect, embodiments of the present invention provide a vehicle retrieval method, including:

[0007] Obtain the vehicle information to be searched; the vehicle information to be searched should include at least textual vehicle information and a set of images of the vehicle to be searched.

[0008] The vehicle information to be retrieved is input into the pre-trained vehicle retrieval model, and the vehicle retrieval result is determined based on the model output of the vehicle retrieval model.

[0009] Among them, the vehicle retrieval model is a cross-modal pre-trained model trained based on a multi-source vehicle feature training sample set;

[0010] The multi-source vehicle features corresponding to the multi-source vehicle feature training sample set include at least one of the following: vehicle appearance features, license plate features, and in-vehicle item features.

[0011] Secondly, embodiments of the present invention provide a vehicle retrieval device, comprising:

[0012] The information acquisition module is used to acquire information about the vehicle to be retrieved; the information about the vehicle to be retrieved includes at least textual vehicle information and a set of images of the vehicle to be retrieved.

[0013] The retrieval result determination module is used to input the vehicle information to be retrieved into the pre-trained vehicle retrieval model and determine the vehicle retrieval result based on the model output of the vehicle retrieval model.

[0014] Among them, the vehicle retrieval model is a cross-modal pre-trained model trained based on a multi-source vehicle feature training sample set;

[0015] The multi-source vehicle features corresponding to the multi-source vehicle feature training sample set include at least one of the following: vehicle appearance features, license plate features, and in-vehicle item features.

[0016] Thirdly, embodiments of the present invention also provide a vehicle retrieval device, comprising:

[0017] At least one processor; and a memory communicatively connected to the at least one processor;

[0018] The memory stores a computer program that can be executed by at least one processor, which enables the at least one processor to implement the vehicle retrieval method of any embodiment of the present invention.

[0019] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the vehicle retrieval method of any embodiment of the present invention.

[0020] Fifthly, embodiments of the present invention also provide a computer program product, including a computer program, which, when executed by a processor, is used to perform the vehicle retrieval method of any embodiment of the present invention.

[0021] This invention provides a vehicle retrieval method, apparatus, device, medium, and program product. The method involves acquiring vehicle information to be retrieved, which includes at least textual vehicle information and a set of vehicle images. This information is then input into a pre-trained vehicle retrieval model, and the vehicle retrieval result is determined based on the model's output. The vehicle retrieval model is a cross-modal pre-trained model trained using a multi-source vehicle feature training sample set. The multi-source vehicle features corresponding to the multi-source vehicle feature training sample set include at least one of vehicle appearance features, license plate features, and in-vehicle item features. By adopting the above technical solution, based on the existing CLIP model, a multi-source vehicle feature training sample set containing vehicle appearance features, license plate features, and in-vehicle item features is introduced for training to adapt to vehicle retrieval needs. This enables the trained vehicle retrieval model to effectively utilize vehicle-specific and structured multi-source information. When users input vehicle information, including textual vehicle information and a set of images of the vehicle to be searched, the system can use a vehicle retrieval model that is more adapted to the vehicle retrieval scenario. Based on the input vehicle information, it performs cross-modal semantic alignment and extracts multi-source vehicle features to complete the accurate retrieval of the target vehicle. This improves the accuracy of targeted vehicle retrieval and can meet the user's need for a combination of natural language retrieval and image retrieval. It is also better adapted to current intelligent transportation systems, security monitoring, and smart city scenarios.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a vehicle retrieval method provided in Embodiment 1 of the present invention;

[0025] Figure 2 This is a flowchart of a vehicle retrieval method provided in Embodiment 2 of the present invention;

[0026] Figure 3 This is a schematic diagram of the structure of a vehicle retrieval device provided in Embodiment 3 of the present invention;

[0027] Figure 4 This is a schematic diagram of the structure of a vehicle retrieval device provided in Embodiment 4 of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] Example 1

[0031] Figure 1 This is a flowchart illustrating a vehicle retrieval method according to Embodiment 1 of the present invention. This embodiment is applicable to situations where a target vehicle can be quickly and accurately located from a massive video or image database, achieving precise vehicle retrieval. The method can be executed by a vehicle retrieval device, which can be implemented using software and / or hardware, and can be configured within a vehicle retrieval equipment. Optionally, the vehicle retrieval device can be a laptop, desktop computer, or smart tablet, etc., and this embodiment of the present invention does not impose any limitations on this.

[0032] like Figure 1 As shown in the figure, a vehicle retrieval method provided by an embodiment of the present invention specifically includes the following steps:

[0033] S101. Obtain information about the vehicle to be searched.

[0034] The vehicle information to be retrieved includes at least textual vehicle information and a collection of images of the vehicle to be retrieved.

[0035] In this embodiment, the vehicle information to be retrieved can be specifically understood as information input by the user according to actual needs, describing the vehicle-related features that need to be retrieved from a massive video or image library. Text-based vehicle information can be specifically understood as text-based information input by the user, describing the features of the vehicle to be retrieved in natural language. The set of vehicle images to be retrieved can be specifically understood as the range of images to be retrieved for the vehicle, given according to the actual situation. Optionally, the vehicle information to be retrieved may also include image-based vehicle information, which can be specifically understood as vehicle images of the target vehicle to be retrieved, input by the user and requiring retrieval of similar vehicles from a massive video or image library.

[0036] Specifically, when a user needs to search for a vehicle, they can use the component in the vehicle retrieval device that receives user input to provide descriptive information about the relevant features of the vehicle to be retrieved. At this time, the vehicle retrieval device can obtain various types of vehicle retrieval-related information input by the user. Simultaneously, the vehicle retrieval device also needs to obtain the scope of the vehicle retrieval. In this embodiment of the invention, the scope of the vehicle retrieval can be represented by a set of images of the vehicles to be retrieved. This set of images can be directly input or stored in a corresponding database or other storage location. Here, the storage address is provided. Then, the vehicle retrieval device can integrate the information obtained above as the information of the vehicle to be retrieved.

[0037] It is understandable that when a user performs a vehicle search, they may already have an image of the vehicle they want to search for. In this case, the user can directly import the image as image-based vehicle information in the vehicle information to be searched. Alternatively, the user may not have a specific image of the vehicle they want to search for. In this case, the user can describe the characteristics of the vehicle they want to search for, and the information generated through natural language description can be used as text-based vehicle information in the vehicle information to be searched. At the same time, even if the user has already imported an image of the vehicle they want to search for, they can further supplement the description of the characteristics of the vehicle they want to search for through text. Therefore, the vehicle information to be searched obtained by the vehicle search device may contain only one of text-based vehicle information and image-based vehicle information, or it may contain both of the above. This embodiment of the invention does not impose any restrictions on this.

[0038] S102. Input the vehicle information to be retrieved into the pre-trained vehicle retrieval model, and determine the vehicle retrieval result based on the model output of the vehicle retrieval model.

[0039] The vehicle retrieval model is a cross-modal pre-trained model trained based on a multi-source vehicle feature training sample set.

[0040] The multi-source vehicle features corresponding to the multi-source vehicle feature training sample set include at least one of the following: vehicle appearance features, license plate features, and in-vehicle item features.

[0041] In this embodiment, the CLIP model can be specifically understood as a multimodal model based on a neural network with a dual-encoder architecture capable of cross-modal semantic alignment of images and text. The vehicle retrieval model can be specifically understood as a multimodal model trained on the CLIP model (trained with scene-free training samples) using training samples incorporating vehicle features from different sources, thus meeting the requirements of vehicle retrieval scenarios. The multi-source vehicle feature training samples can be specifically understood as training samples generated from a combination of vehicle features and vehicle images from various sources, used to train the CLIP model (trained with scene-free samples) for vehicle retrieval scenarios.

[0042] In this embodiment, vehicle appearance features can be specifically understood as features used to characterize the vehicle's shape. For example, vehicle appearance features may include information such as the vehicle's color, model, and brand (logo). In some optional cases, information related to vehicle peripherals, such as roof racks, may be added. This embodiment of the invention does not impose any limitations on this.

[0043] In this embodiment, license plate features can be specifically understood as features used to characterize the text related to a vehicle's license plate. It is understood that license plate features may include Chinese character features, English character features, and numerical features.

[0044] In this embodiment, the features of in-vehicle items can be specifically understood as features extracted from in-vehicle items that can be observed through the vehicle windows. For example, the features of in-vehicle items may include target recognition results obtained by target recognition of in-vehicle items, such as child safety seats, trunks, handbags, safety ornaments, etc., and this embodiment of the invention does not impose any limitations on this.

[0045] In this embodiment, the vehicle retrieval result can be specifically understood as an image retrieved from the set of images of vehicles to be retrieved, containing images of the target vehicle that match the description information of the vehicle to be retrieved in the obtained information of the vehicle to be retrieved, or a target vehicle that matches the description information of the vehicle to be retrieved directly extracted from the set of images of vehicles to be retrieved.

[0046] Specifically, the vehicle information to be retrieved is input into a pre-trained vehicle retrieval model. Through the corresponding encoder contained in the vehicle retrieval model, the vehicle information in the vehicle information to be retrieved is extracted. The extracted features are then subjected to cross-modal semantic alignment and similarity calculation. The vehicle image in the set of vehicle images to be retrieved with a similarity match exceeding a preset threshold or the best similarity match result is taken as the model output result. The vehicle image in the model output result is directly taken as the vehicle retrieval result, or the vehicle contained in the vehicle image in the model output result is taken as the vehicle retrieval result.

[0047] It is understandable that if the vehicle information to be retrieved contains both text-based and image-based vehicle information, it means that the vehicle retrieval model can simultaneously utilize the visual features of the images and the semantic features of the text as retrieval conditions. Since the vehicle retrieval model can achieve cross-modal alignment, the extracted image features and text features are in the same semantic space. Therefore, the two can be fused or jointly calculated to generate a hybrid query vector. Then, the similarity of this hybrid query vector with the image features in the image set of the vehicle to be retrieved is compared, and the model output result is determined based on the similarity matching situation in a similar manner as described above.

[0048] The technical solution of this embodiment involves acquiring vehicle information to be retrieved; this vehicle information includes at least textual vehicle information and a set of images of the vehicle to be retrieved; inputting the vehicle information into a pre-trained vehicle retrieval model, and determining the vehicle retrieval result based on the model's output; wherein the vehicle retrieval model is a cross-modal pre-trained model trained based on a multi-source vehicle feature training sample set; wherein the multi-source vehicle features corresponding to the multi-source vehicle feature training sample set include at least one of vehicle appearance features, license plate features, and in-vehicle item features. By adopting the above technical solution, based on the existing CLIP model, a multi-source vehicle feature training sample set including vehicle appearance features, license plate features, and in-vehicle item features is introduced to adapt the model to vehicle retrieval requirements, enabling the trained vehicle retrieval model to effectively utilize vehicle-specific and structured multi-source information. When users input vehicle information, including textual vehicle information and a set of images of the vehicle to be searched, the system can use a vehicle retrieval model that is more adapted to the vehicle retrieval scenario. Based on the input vehicle information, it performs cross-modal semantic alignment and extracts multi-source vehicle features to complete the accurate retrieval of the target vehicle. This improves the accuracy of targeted vehicle retrieval and can meet the user's need for a combination of natural language retrieval and image retrieval. It is also better adapted to current intelligent transportation systems, security monitoring, and smart city scenarios.

[0049] Example 2

[0050] Figure 2 This is a flowchart of a vehicle retrieval method provided in Embodiment 2 of the present invention. The technical solution of the present invention is further optimized based on the above-mentioned optional technical solutions. Before obtaining the vehicle information to be retrieved, multiple sample vehicle images are obtained. Based on the vehicle appearance features, license plate features, and in-vehicle item features obtained by feature extraction from the multiple sample vehicle images, at least one of them is combined with the sample vehicle images to form a multi-source vehicle feature training sample. Since the vehicle appearance features, license plate features, and in-vehicle item features can be combined in any way before being combined with the sample vehicle images, that is, multiple multi-source vehicle feature training samples can be generated for the same sample vehicle image. This allows the multi-source vehicle feature training sample set constructed based on multiple multi-source vehicle feature training samples to have richer combinations of vehicle features. After training the CLIP model using Chinese pre-training with the multi-source vehicle feature training sample set, a vehicle retrieval model that better meets the needs of the vehicle retrieval scenario can be obtained. Furthermore, after obtaining the vehicle information to be retrieved, the trained vehicle retrieval model can be used to perform cross-modal semantic alignment of the vehicle information to be retrieved, and the multi-source vehicle features extracted from it can be used to complete the accurate retrieval of the target vehicle, thereby improving the accuracy of targeted vehicle retrieval. It can also meet the user's needs for a combination of natural language retrieval and image retrieval, and can be better adapted to current intelligent transportation systems, security monitoring and smart city scenarios.

[0051] like Figure 2 As shown, the vehicle retrieval method provided in Embodiment 2 of the present invention specifically includes the following steps:

[0052] S201. Obtain a set of sample vehicle images.

[0053] In this embodiment, the sample vehicle image set can be understood as a collection of images of different appearances, license plates, models, and interior items, obtained in advance according to the actual situation, which can be used as a vehicle-containing image set to meet the needs of the CLIP model for vehicle retrieval scenarios.

[0054] Specifically, when training a CLIP model that has not been trained for specific scenarios to adapt it to vehicle retrieval scenarios, it is necessary to first acquire vehicle image samples that meet the requirements of the vehicle retrieval scenario. At this time, images of vehicles with different appearances, license plates, models, and interior items can be acquired according to actual needs. At the same time, for the same vehicle, images from different angles, under different lighting conditions, and under different occlusion environments can be acquired. All acquired images are used as sample vehicle images to form a sample vehicle image set that can be adapted to the training requirements of vehicle retrieval scenarios.

[0055] S202. For each sample vehicle image in the sample vehicle image set, extract vehicle appearance features, license plate features, and in-vehicle item features from the sample vehicle image, and combine at least one of the extracted vehicle appearance features, license plate features, and in-vehicle item features with the sample vehicle image to generate at least one multi-source vehicle feature training sample.

[0056] Specifically, to provide training samples containing more possible scenarios for vehicle retrieval, multiple multi-source vehicle feature training samples can be generated for each sample vehicle image when generating multi-source vehicle feature training samples based on a set of sample vehicle images. This means that for each sample vehicle image, multiple features, including at least vehicle appearance features, license plate features, and interior item features, can be extracted. These extracted features can be directly combined with the sample vehicle image, or randomly combined and then combined with the sample vehicle image. The resulting combination is then defined as a multi-source vehicle feature training sample. This achieves full utilization of the features contained in each sample vehicle image, enhancing the richness of the generated multi-source vehicle feature training samples.

[0057] Optionally, vehicle exterior features, license plate features, and interior item features can be extracted from the sample vehicle images. This can be achieved through several methods:

[0058] The sample vehicle images and pre-constructed appearance feature extraction prompts are used as input to a large language model. The large language model automatically describes the vehicle appearance in the sample vehicle images and generates vehicle appearance features represented by a preset vehicle appearance template.

[0059] The license plate text information is extracted from the sample vehicle images using a pre-trained license plate recognition model, and then the license plate text information is substituted into a preset license plate information template to determine the license plate features.

[0060] The pre-trained object detection model extracts information about items inside the vehicle from sample vehicle images. The information about items inside the vehicle and pre-constructed item feature extraction prompts are used as input to a large language model. The large language model automatically describes the items inside the vehicle based on the information about items inside the vehicle, generating features of items inside the vehicle represented by a preset item template.

[0061] In this embodiment, the pre-constructed appearance feature extraction prompts and pre-constructed item feature extraction prompts can be understood as being pre-constructed according to actual conditions, used to guide the large language model to automatically generate text descriptions based on the input image or text information using pre-given language templates. It is understood that the appearance feature extraction prompts and item feature extraction prompts can be set and adjusted according to actual needs, and this embodiment of the invention does not limit the specific content of the prompts. The preset vehicle appearance template can be understood as being pre-defined according to actual conditions, used to standardize the representation of vehicle appearance descriptions. The preset license plate information template can be understood as being pre-defined according to actual conditions, used to standardize the representation of vehicle license plate information descriptions. The preset in-vehicle item template can be understood as being pre-defined according to actual conditions, used to standardize the representation of in-vehicle item descriptions.

[0062] Specifically, in the process of multi-source vehicle feature extraction from sample vehicle images, different feature extraction methods are required for the vehicle appearance features, license plate features, and interior item features to be extracted. The extraction methods for different vehicle features are explained below:

[0063] 1) Regarding vehicle appearance features, since this embodiment of the invention requires training the CLIP model in a vehicle retrieval scenario using vehicle appearance features, the extracted vehicle appearance features can be represented in text description form to facilitate subsequent training of the CLIP model. Because the large language model has good text description information generation capabilities, sample vehicle images and pre-constructed appearance feature extraction prompts can be used as input to the large language model. The large language model uses these appearance feature extraction prompts to automatically describe the appearance of vehicles present in the sample vehicle images based on a preset vehicle appearance template, generating vehicle appearance features represented by the preset vehicle appearance template. For example, assuming the preset vehicle appearance template is "a {color} {brand} {model}", the vehicle appearance features output by the large language model can be represented as "a {white} {Volkswagen} {SUV}". Optionally, descriptions such as "has roof rack" can be added to the preset vehicle appearance template; this embodiment of the invention does not limit this.

[0064] 2) Regarding license plate features, since large language models have lower accuracy in recognizing text-and-number combinations like license plates compared to specially trained license plate recognition models, to improve the accuracy of the generated multi-source vehicle feature training samples, when extracting license plate features from sample vehicle images, a pre-trained license plate recognition model can be used to extract the license plate text information from the sample vehicle images. This extracted license plate text information is then substituted into the required filling area of ​​a preset license plate information template to generate the corresponding license plate features. For example, the preset license plate information template can be "license plate number is {license plate text}". Here, after the license plate text is recognized by the license plate recognition model, it is simply substituted into the corresponding position in the preset license plate information template.

[0065] 3) Regarding the features of items inside the vehicle, since the vehicle images contained in the sample vehicle images may be unclear due to reflections and viewing angles, directly describing the items inside the vehicle using a generalized large language model would have low accuracy. To improve the accuracy of the generated multi-source vehicle feature training samples, when extracting features of items inside the vehicle images, a pre-trained target detection model can be used to identify objects inside the vehicle images, thereby extracting the information of items inside the vehicle. Since this embodiment of the invention requires training the CLIP model in a vehicle retrieval scenario using the features of items inside the vehicle, the extracted information needs to be represented in text description form so that it can be used as input for training the CLIP model. Because the large language model has good text description generation capabilities, the information of items inside the vehicle and pre-constructed item feature extraction prompts can be used as input to the large language model. The large language model uses the item feature extraction prompts to automatically describe the information of items inside the vehicle based on a preset item template, generating the features of items inside the vehicle represented by the preset item template. For example, assuming the preset template for in-vehicle items is "there is a list of items in the car", the in-vehicle item features output by the large language model based on the input in-vehicle item information can be represented as "there is a child safety seat and a trunk in the car". Optionally, the in-vehicle item information input to the large language model can be the text information of the in-vehicle items extracted by the object detection model; it can also be the image information of the part of the in-vehicle items located by picking the target of interest in the sample vehicle image through the object detection model. Then, the large language model can extract prompt words and image information based on the pre-constructed item features to complete the automated text description. This embodiment of the invention does not limit this.

[0066] Optionally, at least one of the extracted vehicle appearance features, license plate features, and interior item features can be combined with the sample vehicle image to generate at least one multi-source vehicle feature training sample. This can be achieved in the following way:

[0067] Combine vehicle appearance features, license plate features, and in-vehicle item features according to a preset probability to determine at least one multi-source vehicle combination feature;

[0068] Perform feature enrichment processing on each multi-source vehicle combination feature, and combine each processed multi-source vehicle combination feature with a sample vehicle image to generate a multi-source vehicle feature training sample.

[0069] In this embodiment, the preset probability can be specifically understood as a probability value preset according to the actual situation for enriching vehicle feature combinations. Exemplarily, the preset probability of a combination including all features simultaneously can be 60%, the preset probability of randomly combining two types of features can be 30%, and the preset probability of a combination including only one feature can be 10%, etc. Based on the above preset probabilities, the construction of multi-source vehicle combination features can enrich the coverage range of multi-source vehicle feature training samples and enhance the vehicle retrieval accuracy of the trained vehicle retrieval model in the case of missing some features.

[0070] Specifically, combine vehicle appearance features, license plate features, and in-vehicle item features according to the preset probabilities corresponding to different combinations of feature quantities, and various types of multi-source vehicle combination features can be obtained. Furthermore, in order to make the generated multi-source vehicle feature training samples more in line with the situation of the user's natural input statements, feature enrichment processing can be performed on each multi-source vehicle combination feature, and each processed multi-source vehicle combination feature can be combined with a sample vehicle image respectively to obtain multiple multi-source vehicle feature training samples.

[0071] Continuing with the above example, for the multi-source vehicle combination feature including all features simultaneously, the logical order of different features can be adjusted during combination to make the generated sentence more in line with the natural word order. For example, the generated multi-source vehicle combination feature can be expressed as "A black Toyota sedan with a license plate number of Beijing A12345 and a safety seat and a handbag in the car."; for the multi-source vehicle combination feature randomly combining two types of features, the generated multi-source vehicle combination feature can be expressed as "A white BMW SUV with a peace charm hanging on the interior rearview mirror." (missing license plate feature), or "The license plate number is Guangdong B98765 and there is a parking phone placed on the front windshield." (missing vehicle appearance feature); for the multi-source vehicle combination feature including only one type of feature, the generated multi-source vehicle combination feature can be expressed as "A red sports car." or "There is a pet dog in the car", etc. The embodiments of the present invention do not limit this.

[0072] Optionally, the feature enrichment processing includes at least one of the following:

[0073] Synonym replacement processing;

[0074] Sentence pattern transformation processing;

[0075] Controllable noise injection processing.

[0076] In this embodiment, the synonym replacement can be specifically understood as a process of replacing some words in the generated multi-source vehicle combined features with a thesaurus. This process can avoid overfitting of fixed words in the vehicle retrieval model during training. Exemplarily, the synonym replacement can be "a vehicle -> a car", "there is in the vehicle -> there is placed in the vehicle", and "there is in the vehicle -> it can be seen in the vehicle", etc. The embodiments of the present invention are not limited thereto.

[0077] In this embodiment, the sentence pattern transformation processing can be specifically understood as expressing the same information included in the generated multi-source vehicle combined features in different sentence patterns. Exemplarily, assuming the original sentence is "The license plate number is Jing A12345.", the expressions obtained after sentence pattern transformation can be "Its license is Jing A12345" or "The license plate of this vehicle is Jing A12345", etc. The embodiments of the present invention are not limited thereto.

[0078] In this embodiment, the controllable noise injection processing can be specifically understood as a process of randomly deleting individual words or introducing common misspellings with an extremely low probability to the generated multi-source vehicle combined features based on the possible errors in the actual input of the user. Exemplarily, "safety seat" can be miswritten as "safety chair" to enhance the anti-interference ability of the vehicle retrieval model obtained by training.

[0079] It can be understood that when performing feature enrichment processing on the multi-source vehicle combined features, one or multiple of the three processing methods provided in the above feature enrichment processing can be selected for processing, or no processing can be performed. The embodiments of the present invention are not limited thereto.

[0080] S203. Construct a multi-source vehicle feature training sample set according to each multi-source vehicle feature training sample.

[0081] Specifically, the set composed of each multi-source vehicle feature training sample is determined as the multi-source vehicle feature training sample set.

[0082] S204. Use the cross-modal pre-training model pre-trained in Chinese as the initial vehicle retrieval model, and train the initial vehicle retrieval model through the multi-source vehicle feature training sample set to determine the vehicle retrieval model.

[0083] Specifically, since the vehicle retrieval model trained in this embodiment of the invention is a CLIP model that is trained for a second time for vehicle retrieval scenarios, the CLIP model pre-trained with a large Chinese corpus can be used as the basis for training the vehicle retrieval model, that is, as the initial vehicle retrieval model. The initial vehicle retrieval model is trained using a multi-source vehicle feature training sample set, and the fine-tuned initial vehicle retrieval model is determined as the vehicle retrieval model.

[0084] Optionally, the initial vehicle retrieval model can be trained using a multi-source vehicle feature training sample set to determine the vehicle retrieval model. This can be achieved in the following ways:

[0085] The image encoder of the initial vehicle retrieval model is frozen, and the text encoder and projection layer of the initial vehicle retrieval model are trained using a multi-source vehicle feature training sample set to determine the intermediate vehicle retrieval model.

[0086] Unfreeze the image encoder of the intermediate vehicle retrieval model, train the intermediate vehicle retrieval model using a multi-source vehicle feature training sample set, and determine the vehicle retrieval model.

[0087] Specifically, to reduce the convergence difficulty of training the vehicle retrieval model, a phased approach can be adopted to train the initial vehicle retrieval model. First, the image encoder in the initial vehicle retrieval model is frozen, and the text encoder and projection layer are trained only using a multi-source vehicle feature training sample set. Then, the image encoder in the initial vehicle retrieval model is unfrozen, and the image encoder and the previously trained text encoder and projection layer are trained in a second phase using a multi-source vehicle feature training sample set to obtain the vehicle retrieval model.

[0088] Furthermore, considering the possibility of excessively high similarity among multi-source vehicle feature training samples input in the same batch during vehicle retrieval model training, this embodiment of the invention can also employ the following method to improve training accuracy, specifically including the following steps:

[0089] 1) The input to the loss function of the vehicle retrieval model is the training samples of multi-source vehicle features from the same batch of inputs. The samples are deduplicated based on the similarity of the text descriptions to determine the effective samples.

[0090] 2) The vehicle retrieval model is trained based on the effective samples and the loss function of the vehicle retrieval model.

[0091] Specifically, during the training of the vehicle retrieval model, the similarity of vehicle text descriptions in text-based vehicle information within the same batch of multi-source vehicle feature training samples can be calculated. If the similarity exceeds a preset similarity threshold, only one feature corresponding to each vehicle text description with a similarity exceeding the preset similarity threshold can be retained as a valid sample to avoid interference from similar samples on the optimization of the vehicle retrieval model. Then, each valid sample can be used as the input to the loss function of the vehicle retrieval model, and the training of the vehicle retrieval model can be completed based on the loss function of the vehicle retrieval model.

[0092] In this embodiment of the invention, the loss function of the CLIP model pre-trained in Chinese is adjusted and used as the loss function of the vehicle retrieval model to avoid interference from similar samples on model optimization; wherein, the loss function of the vehicle retrieval model is:

[0093]

[0094] in, The number of valid samples after deduplication based on text description similarity is calculated for training samples of multi-source vehicle features input to the initial vehicle retrieval model in the same batch. The original loss function is used for the cross-modal pre-trained model that uses Chinese pre-training. The image features of the i-th valid sample; Let be the text features of the i-th valid sample.

[0095] Specifically, during the training process, the vehicle retrieval model extracts and aligns features from N multi-source vehicle feature training samples input from the same batch. In this process, this embodiment of the invention adds a comparison process for the similarity between different multi-source vehicle feature training samples. For multiple multi-source vehicle feature training samples with similarity exceeding a set threshold, one sample can be randomly selected or the multi-source vehicle feature training sample with the earlier input order can be retained to participate in subsequent loss calculations. That is, N' effective samples can be obtained in the end, and the training of the vehicle retrieval model can be completed based on N' effective samples with low similarity, reducing the interference of similar samples on the training of the vehicle retrieval model.

[0096] S205. Obtain information about the vehicle to be searched.

[0097] The vehicle information to be retrieved includes at least one of text-based vehicle information and image-based vehicle information.

[0098] S206. Input the vehicle information to be retrieved into the pre-trained vehicle retrieval model, and determine the vehicle retrieval result based on the model output.

[0099] The technical solution of this embodiment, before obtaining the vehicle information to be retrieved, acquires multiple sample vehicle images. Based on the vehicle appearance features, license plate features, and in-vehicle item features obtained by feature extraction from the multiple sample vehicle images, at least one of these features is combined with the sample vehicle images to form a multi-source vehicle feature training sample. Since the vehicle appearance features, license plate features, and in-vehicle item features can be combined in any way before being combined with the sample vehicle images, multiple multi-source vehicle feature training samples can be generated for the same sample vehicle image. This allows the multi-source vehicle feature training sample set constructed based on multiple multi-source vehicle feature training samples to have richer combinations of vehicle features. After training the CLIP model using Chinese pre-training with the multi-source vehicle feature training sample set, a vehicle retrieval model that better meets the needs of the vehicle retrieval scenario can be obtained. Furthermore, after obtaining the vehicle information to be retrieved, the trained vehicle retrieval model can be used to perform cross-modal semantic alignment of the vehicle information to be retrieved, and the multi-source vehicle features extracted from it can be used to complete the accurate retrieval of the target vehicle, thereby improving the accuracy of targeted vehicle retrieval. It can also meet the user's needs for a combination of natural language retrieval and image retrieval, and can be better adapted to current intelligent transportation systems, security monitoring and smart city scenarios.

[0100] Example 3

[0101] Figure 3 This is a schematic diagram of the structure of a vehicle retrieval device provided in Embodiment 3 of the present invention, as shown below. Figure 3 As shown, the vehicle retrieval device includes an information acquisition module 31 and a retrieval result determination module 32.

[0102] The information acquisition module 31 is used to acquire vehicle information to be retrieved; the vehicle information to be retrieved includes at least text-based vehicle information and a set of images of the vehicle to be retrieved; the retrieval result determination module 32 is used to input the vehicle information to be retrieved into a pre-trained vehicle retrieval model and determine the vehicle retrieval result based on the model output of the vehicle retrieval model; wherein the vehicle retrieval model is a cross-modal pre-trained model trained based on a multi-source vehicle feature training sample set; wherein the multi-source vehicle features corresponding to the multi-source vehicle feature training sample set include at least one of the following: vehicle appearance features, license plate features, and in-vehicle item features.

[0103] The technical solution of this invention, based on the existing CLIP model, introduces a multi-source vehicle feature training sample set including vehicle appearance features, license plate features, and in-vehicle item features for training to adapt to vehicle retrieval needs. This enables the trained vehicle retrieval model to effectively utilize vehicle-specific and structured multi-source information. When a user inputs vehicle information including textual vehicle information and a set of images of the vehicle to be retrieved, the vehicle retrieval model, more adapted to the vehicle retrieval scenario, performs cross-modal semantic alignment based on the input vehicle information and extracts multi-source vehicle features to complete accurate retrieval of the target vehicle. This improves the accuracy of targeted vehicle retrieval and can adapt to the user's need for a combination of natural language retrieval and image retrieval, making it better suited for current intelligent transportation systems, security monitoring, and smart city scenarios.

[0104] Optionally, the vehicle retrieval device further includes: a retrieval model training module, used for:

[0105] Before obtaining the information of the vehicle to be retrieved, a set of sample vehicle images is obtained;

[0106] For each sample vehicle image in the sample vehicle image set, vehicle appearance features, license plate features, and interior item features are extracted from the sample vehicle image. At least one of the extracted vehicle appearance features, license plate features, and interior item features is combined with the sample vehicle image to generate at least one multi-source vehicle feature training sample.

[0107] Construct a multi-source vehicle feature training sample set based on the multi-source vehicle feature training samples;

[0108] Using a cross-modal pre-trained model with Chinese pre-training as the initial vehicle retrieval model, the initial vehicle retrieval model is trained with a multi-source vehicle feature training sample set to determine the vehicle retrieval model.

[0109] Optionally, vehicle exterior features, license plate features, and interior item features are extracted from the sample vehicle images, including:

[0110] The sample vehicle images and pre-constructed appearance feature extraction prompts are used as input to a large language model. The large language model automatically describes the vehicle appearance in the sample vehicle images and generates vehicle appearance features represented by a preset vehicle appearance template.

[0111] The license plate text information is extracted from the sample vehicle images using a pre-trained license plate recognition model, and then the license plate text information is substituted into a preset license plate information template to determine the license plate features.

[0112] The pre-trained object detection model extracts information about items inside the vehicle from sample vehicle images. The information about items inside the vehicle and pre-constructed item feature extraction prompts are used as input to a large language model. The large language model automatically describes the items inside the vehicle based on the information about items inside the vehicle, generating features of items inside the vehicle represented by a preset item template.

[0113] Optionally, at least one of the extracted vehicle exterior features, license plate features, and interior item features is combined with the sample vehicle image to generate at least one multi-source vehicle feature training sample, including:

[0114] The vehicle's exterior features, license plate features, and interior item features are combined according to a preset probability to determine at least one multi-source vehicle combination feature.

[0115] Feature enrichment processing is performed on the combined features of each multi-source vehicle, and the processed combined features of each multi-source vehicle are combined with the sample vehicle images to generate multi-source vehicle feature training samples.

[0116] Feature enrichment processing includes at least one of the following:

[0117] Synonym replacement processing;

[0118] Sentence transformation processing;

[0119] Controllable noise injection processing.

[0120] Optionally, the initial vehicle retrieval model is trained using a multi-source vehicle feature training sample set to determine the vehicle retrieval model, including:

[0121] The image encoder of the initial vehicle retrieval model is frozen, and the text encoder and projection layer of the initial vehicle retrieval model are trained using a multi-source vehicle feature training sample set to determine the intermediate vehicle retrieval model.

[0122] Unfreeze the image encoder of the intermediate vehicle retrieval model, train the intermediate vehicle retrieval model using a multi-source vehicle feature training sample set, and determine the vehicle retrieval model.

[0123] Optionally, training the vehicle retrieval model may also include:

[0124] The input to the loss function of the vehicle retrieval model is the training samples of multi-source vehicle features from the same batch of inputs. The samples are deduplicated based on the similarity of the text descriptions to determine the effective samples.

[0125] The vehicle retrieval model is trained based on valid samples and the loss function of the vehicle retrieval model.

[0126] Optionally, the loss function for the vehicle retrieval model is:

[0127]

[0128] in, The number of valid samples after deduplication based on text description similarity is calculated for training samples of multi-source vehicle features input to the initial vehicle retrieval model in the same batch. The original loss function is used for the cross-modal pre-trained model that uses Chinese pre-training. The image features of the i-th valid sample; Let be the text features of the i-th valid sample.

[0129] The vehicle retrieval device provided in this embodiment of the invention can execute the vehicle retrieval method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0130] Example 4

[0131] Figure 4 This is a schematic diagram of a vehicle retrieval device according to Embodiment 4 of the present invention. The vehicle retrieval device 40 can represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, communication base stations, and other suitable computers. The vehicle retrieval device 40 can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0132] like Figure 4 As shown, the vehicle retrieval device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42 or a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the ROM 42 or loaded from storage unit 48 into the RAM 43. The RAM 43 can also store various programs and data required for the operation of the vehicle retrieval device 40. The processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0133] Multiple components in the vehicle retrieval device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, optical disk, etc.; and a communication unit 49, such as a network card, modem, wireless transceiver, etc. The communication unit 49 allows the vehicle retrieval device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0134] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as vehicle retrieval methods.

[0135] In some embodiments, the vehicle retrieval method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on the vehicle retrieval device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the vehicle retrieval method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the vehicle retrieval method by any other suitable means (e.g., by means of firmware).

[0136] Optionally, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the vehicle retrieval method as provided in any embodiment of the present invention.

[0137] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0139] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0140] To provide interaction with a user, the systems and techniques described herein can be implemented on a vehicle retrieval device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the vehicle retrieval device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including voice input, speech input, or tactile input).

[0141] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0142] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0143] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0144] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A vehicle retrieval method, characterized in that, include: Retrieve information about the vehicle to be searched; The vehicle information to be retrieved includes at least textual vehicle information and a set of images of the vehicle to be retrieved. The vehicle information to be retrieved is input into a pre-trained vehicle retrieval model, and the vehicle retrieval result is determined based on the model output of the vehicle retrieval model. The vehicle retrieval model is a cross-modal pre-trained model trained based on a multi-source vehicle feature training sample set. The multi-source vehicle features corresponding to the multi-source vehicle feature training sample set include at least one of the following: vehicle appearance features, license plate features, and in-vehicle item features.

2. The vehicle retrieval method according to claim 1, characterized in that, Before obtaining the vehicle information to be retrieved, the following is also included: Obtain a set of sample vehicle images; For each sample vehicle image in the sample vehicle image set, vehicle appearance features, license plate features, and interior item features are extracted from the sample vehicle image. At least one of the extracted vehicle appearance features, license plate features, and interior item features is combined with the sample vehicle image to generate at least one multi-source vehicle feature training sample. Construct a multi-source vehicle feature training sample set based on the multi-source vehicle feature training samples described above; Using a cross-modal pre-trained model with Chinese pre-training as the initial vehicle retrieval model, the initial vehicle retrieval model is trained using the multi-source vehicle feature training sample set to determine the vehicle retrieval model.

3. The vehicle retrieval method according to claim 2, characterized in that, The extraction of vehicle exterior features, license plate features, and interior item features from the sample vehicle images includes: The sample vehicle image and pre-constructed appearance feature extraction prompts are used as input to a large language model. The large language model automatically describes the vehicle appearance in the sample vehicle image, generating vehicle appearance features represented by a preset vehicle appearance template. The license plate text information of the sample vehicle images is extracted by a pre-trained license plate recognition model, and the license plate text information is substituted into a preset license plate information template to determine the license plate features. The sample vehicle images are processed by a pre-trained object detection model to extract information about items inside the vehicle. The information about items inside the vehicle and pre-constructed item feature extraction prompts are used as input to a large language model. The large language model then performs automated text description based on the information about items inside the vehicle, generating features of items inside the vehicle represented by a preset item template.

4. The vehicle retrieval method according to claim 2, characterized in that, The step of combining at least one of the extracted vehicle appearance features, license plate features, and interior item features with the sample vehicle image to generate at least one multi-source vehicle feature training sample includes: The vehicle appearance features, license plate features, and interior item features are combined according to a preset probability to determine at least one multi-source vehicle combination feature; The multi-source vehicle combination features are enriched by processing, and the processed multi-source vehicle combination features are combined with the sample vehicle images to generate multi-source vehicle feature training samples.

5. The vehicle retrieval method according to claim 2, characterized in that, The step of training the initial vehicle retrieval model using the multi-source vehicle feature training sample set to determine the vehicle retrieval model includes: Freeze the image encoder of the initial vehicle retrieval model, and train the text encoder and projection layer of the initial vehicle retrieval model using the multi-source vehicle feature training sample set to determine the intermediate vehicle retrieval model; Unfreeze the image encoder of the intermediate vehicle retrieval model, train the intermediate vehicle retrieval model using the multi-source vehicle feature training sample set, and determine the vehicle retrieval model.

6. The vehicle retrieval method according to any one of claims 2-5, characterized in that, The training of the vehicle retrieval model also includes: The input to the loss function of the vehicle retrieval model is a training sample of multi-source vehicle features from the same batch of inputs. The sample is deduplicated based on the similarity of the text descriptions to determine the effective samples. The vehicle retrieval model is trained based on the valid samples and the loss function of the vehicle retrieval model.

7. A vehicle retrieval device, characterized in that, include: The information acquisition module is used to acquire information about the vehicle to be retrieved. The vehicle information to be retrieved includes at least textual vehicle information and a set of images of the vehicle to be retrieved. The retrieval result determination module is used to input the vehicle information to be retrieved into a pre-trained vehicle retrieval model and determine the vehicle retrieval result based on the model output of the vehicle retrieval model. The vehicle retrieval model is a cross-modal pre-trained model trained based on a multi-source vehicle feature training sample set. The multi-source vehicle features corresponding to the multi-source vehicle feature training sample set include at least one of the following: vehicle appearance features, license plate features, and in-vehicle item features.

8. A vehicle retrieval device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the vehicle retrieval method according to any one of claims 1-6.

9. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the vehicle retrieval method as described in any one of claims 1-6.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the vehicle retrieval method as described in any one of claims 1-6.

Images