Artificial intelligence-based vehicle search method, device and electronic equipment

Abstract

The disclosure provides an artificial intelligence-based vehicle search method and device and electronic equipment, relates to the technical field of artificial intelligence, in particular to the technical fields of natural language processing, intelligent transportation, computer vision, intelligent search, deep learning, image recognition and video analysis, and can be applied in smart city, city governance and public security emergency scenarios. The specific implementation scheme is as follows: determining license plate information and a feature vector of a to-be-searched vehicle picture; determining a first recall result based on the feature vector and a second recall result based on the license plate information; determining a picture recall result according to the first recall result and the second recall result; and determining a picture search result of the to-be-searched vehicle picture according to the picture recall result. In this way, the picture search result of the to-be-searched vehicle picture is determined by combining the license plate information and the feature vector of the to-be-searched vehicle picture, thereby expanding the number of pictures in the picture search result and improving the search efficiency and search accuracy.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to the fields of natural language processing, intelligent transportation, computer vision, intelligent search, deep learning, image recognition, and video analysis. In particular, it relates to an artificial intelligence-based vehicle search method, device, and electronic device, which can be applied in smart cities, urban governance, and public security emergency scenarios. Background Technology

[0002] Currently, vehicle search methods in related technologies mainly rely on vehicle images, searching for relevant vehicle images from a vehicle image capture database to facilitate subsequent processing. However, in these image-based searches, a large number of highly similar but different vehicles exist, resulting in poor search performance. Summary of the Invention

[0003] This disclosure provides an artificial intelligence-based vehicle search method, apparatus, and electronic device.

[0004] According to one aspect of this disclosure, an artificial intelligence-based vehicle search method is provided, the method comprising: determining license plate information and feature vector of a vehicle image to be searched; determining a first recall result based on the feature vector and a second recall result based on the license plate information; determining an image recall result based on the first recall result and the second recall result; and determining an image search result for the vehicle image to be searched based on the image recall result.

[0005] According to another aspect of this disclosure, an artificial intelligence-based vehicle search device is provided. The device includes: a first determining module, configured to determine license plate information and feature vector of a vehicle image to be searched; a second determining module, configured to determine a first recall result based on the feature vector and a second recall result based on the license plate information; a third determining module, configured to determine an image recall result based on the first recall result and the second recall result; and a fourth determining module, configured to determine an image search result of the vehicle image to be searched based on the image recall result.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the artificial intelligence-based vehicle search method proposed above in this disclosure.

[0007] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to execute the artificial intelligence-based vehicle search method proposed above in this disclosure.

[0008] According to another aspect of this disclosure, a computer program is provided, comprising a computer program that, when executed by a processor, implements the steps of the artificial intelligence-based vehicle search method proposed above.

[0009] One embodiment disclosed above has the following advantages or beneficial effects:

[0010] The process involves determining the license plate information and feature vector of the vehicle image to be searched; determining a first recall result based on the feature vector and a second recall result based on the license plate information; determining the image recall result based on the first and second recall results; and determining the image search result for the vehicle image to be searched based on the image recall result. Therefore, by combining the license plate information and feature vector of the vehicle image, the second recall result based on the license plate information and the first recall result based on the feature vector are fused and deduplicated to obtain the search results for the vehicle image, improving search efficiency and accuracy.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0013] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0014] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0015] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure;

[0016] Figure 4 This is a flowchart of an artificial intelligence-based vehicle search method according to an embodiment of the present disclosure;

[0017] Figure 5 This is an application diagram illustrating the artificial intelligence-based vehicle search method used to implement the embodiments of this disclosure;

[0018] Figure 6 This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0019] Figure 7 This is a block diagram of an electronic device used to implement the artificial intelligence-based vehicle search method according to embodiments of the present disclosure. Detailed Implementation

[0020] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0021] Currently, vehicle search methods in related technologies mainly rely on vehicle images, searching for relevant vehicle images from a vehicle image capture database to facilitate subsequent processing. However, in these image-based searches, a large number of highly similar but different vehicles exist, resulting in poor search performance.

[0022] To this end, this disclosure determines the license plate information and feature vector of the vehicle image to be searched; determines a first recall result based on the feature vector and a second recall result based on the license plate information; determines an image recall result based on the first and second recall results; and determines the image search results for the vehicle image to be searched based on the image recall results. Thus, by combining the license plate information and feature vector of the vehicle image to be searched, and fusing and deduplicating the second recall result based on the license plate information and the first recall result based on the feature vector, the image search results for the vehicle image to be searched are obtained, thereby expanding the number of images in the image search results and improving search efficiency and accuracy.

[0023] The following description, with reference to the accompanying drawings, outlines an artificial intelligence-based vehicle search method, apparatus, and electronic device according to embodiments of the present disclosure.

[0024] Figure 1 The diagram is based on the first embodiment of this disclosure. It should be noted that the AI-based vehicle search method of this disclosure can be applied to an AI-based vehicle search device, which can be configured in an electronic device so that the electronic device can perform AI-based vehicle search functions.

[0025] The electronic device can be any device with computing capabilities, such as a personal computer (PC), mobile terminal, server, etc. Mobile terminals can be, for example, in-vehicle devices, mobile phones, tablets, personal digital assistants, wearable devices, and other hardware devices with various operating systems, touchscreens, and / or displays. The following explanation uses an electronic device as the executing entity.

[0026] like Figure 1 As shown, this AI-based vehicle search method may include the following steps:

[0027] Step 101: Determine the license plate information and feature vector of the vehicle image to be searched.

[0028] In this embodiment of the disclosure, the process by which the electronic device determines the license plate information of a vehicle image to be searched can be, for example, by inputting the vehicle image to be searched into a license plate information extraction model and obtaining the license plate information output by the model. Another example is determining the license plate region in the vehicle image to be searched, performing optical character recognition (OCR) on the license plate region, and obtaining the recognition result, wherein the recognition result includes the license plate information.

[0029] In this embodiment of the disclosure, the process by which the electronic device determines the feature vector of the vehicle image to be searched can, for example, involve inputting the vehicle image to be searched into a feature vector extraction model and obtaining the feature vector output by the feature vector extraction model. The feature vector extraction model can be, for example, a deep neural network.

[0030] Step 102: Determine the first recall result based on the feature vector and the second recall result based on the license plate information.

[0031] In this embodiment of the disclosure, the electronic device performing step 102 may, for example, involve querying an image database based on feature vectors to obtain a second candidate vehicle image from the image database; wherein the similarity between the feature vector of the second candidate image and the feature vector of the vehicle image to be searched is greater than or equal to a second similarity threshold; determining a first recall result based on the second candidate vehicle image; querying an image database based on license plate information to obtain a third candidate vehicle image from the image database; wherein the edit distance between the license plate information of the third candidate vehicle image and the license plate information of the image to be searched is less than or equal to a first distance threshold; and determining a second recall result based on the third candidate vehicle image, thereby improving the accuracy of the first and second recall results.

[0032] The process of determining the first recall result for electronic devices can be, for example, sorting the second candidate vehicle images in descending order based on similarity to obtain a sorting result; and taking a certain number of second candidate vehicle images ranked first in the sorting result as the first recall result.

[0033] The process by which the electronic device determines the second recall result can be, for example, sorting the third candidate vehicle images in descending order based on the edit distance to obtain a sorting result; and taking a certain number of the third candidate vehicle images that are ranked first in the sorting result as the second recall result.

[0034] The second similarity threshold, the first distance threshold, the number of second candidate vehicle images in the first recall result, and the number of third candidate vehicle images in the second recall result can be preset according to search requirements, without specific limitations.

[0035] Step 103: Determine the image recall result based on the first recall result and the second recall result.

[0036] In this embodiment of the disclosure, the process of the electronic device performing step 103 may, for example, be to merge the first recall result and the second recall result and perform deduplication to obtain the image recall result.

[0037] Step 104: Based on the image recall results, determine the image search results for the vehicle image to be searched.

[0038] In this embodiment, the electronic device performing step 104 may, for example, involve determining a score between each candidate vehicle image in the image retrieval results and the vehicle image to be searched; sorting the candidate vehicle images in the image retrieval results in descending order based on the score to obtain a sorting result; and generating an image search result based on a preset number of candidate vehicle images that rank first in the sorting result. Therefore, sorting based on the score of each candidate vehicle image can result in a higher accuracy rate. The preset number can be pre-set according to search requirements and is not specifically limited.

[0039] In an exemplary embodiment, for each candidate vehicle image in the image recall results, the electronic device determines the score between the candidate vehicle image and the vehicle image to be searched, which can be as follows: for each candidate vehicle image in the image recall results, determine the vehicle type and facial feature vector of the candidate vehicle image; determine the vehicle type and facial feature vector of the vehicle image to be searched; determine the type similarity between the vehicle type of the candidate vehicle image and the vehicle type of the vehicle image to be searched; determine the facial feature similarity between the facial feature vector of the candidate vehicle image and the facial feature vector of the vehicle image to be searched; and perform a weighted sum of the similarity, type similarity, and facial feature similarity between the feature vector of the vehicle image to be searched and the feature vector of the candidate vehicle image to obtain the score.

[0040] The vehicle type, such as sedan, truck, bus, trailer, incomplete vehicle, motorcycle, etc., can be set according to actual needs. The process by which the electronic device determines the facial feature vector can be, for example, identifying the facial region in the image of the vehicle to be searched; performing feature extraction processing on the facial region to obtain its feature vector; and using the feature vector of the facial region as the facial feature vector of the image of the vehicle to be searched.

[0041] The process of electronic devices weightedly summing the similarity, type similarity, and facial feature similarity between the feature vectors of the vehicle image to be searched and the feature vectors of candidate vehicle images to obtain a score can be as follows: Based on the feature vectors of the vehicle image to be searched, the facial feature vectors, and the vehicle type, and combined with an attention mechanism model, the weights of the feature vectors, the facial feature vectors, and the vehicle type are determined; the weights are then used to weight and sum the above three similarities to obtain a score. This allows for the combination of multiple similarities and weights to determine the score, improving the accuracy of the score and thus improving the accuracy of image search results.

[0042] In this embodiment of the disclosure, to further improve the accuracy of search results, a negative sample list can be set for the vehicle image to be searched. The negative sample list includes multiple negative sample vehicle images, which cannot be included in the image search results for the vehicle image to be searched. Correspondingly, for each candidate vehicle image in the image search results, if the score between the candidate vehicle image and all negative sample vehicle images in the negative sample list is greater than or equal to a certain score threshold, then the candidate vehicle image is removed from the image search results; or, if the score between the candidate vehicle image and one or more negative sample vehicle images in the negative sample list is greater than or equal to a certain score threshold, then the candidate vehicle image is removed from the image search results.

[0043] The AI-based vehicle search method of this disclosure determines the license plate information and feature vector of a vehicle image to be searched; determines a first recall result based on the feature vector and a second recall result based on the license plate information; determines an image recall result based on the first and second recall results; and determines the image search results for the vehicle image to be searched based on the image recall results. Thus, by combining the license plate information and feature vector of the vehicle image to be searched, and fusing and deduplicating the second recall result based on the license plate information and the first recall result based on the feature vector, the image search results for the vehicle image to be searched are obtained, thereby expanding the number of images in the image search results and improving search efficiency and accuracy.

[0044] To further improve the accuracy of license plate information in the images of vehicles to be searched, when no license plate information is found, the license plate information of the candidate vehicle images in the first recall results can be combined to determine the license plate information of the vehicle to be searched. For example... Figure 2 As shown, Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure. Figure 2 The illustrated embodiment may include the following steps:

[0045] Step 201: Perform vector extraction processing on the image of the vehicle to be searched to obtain feature vectors.

[0046] In this embodiment of the disclosure, the electronic device can input an image of the vehicle to be searched into a feature vector extraction model and obtain the feature vector output by the feature vector extraction model. The feature vector extraction model may be, for example, a deep neural network.

[0047] Step 202: Extract license plate information from the image of the vehicle to be searched.

[0048] In one embodiment of this disclosure, the electronic device can input a vehicle image to be searched into a license plate information extraction model to obtain the license plate information output by the model. In another embodiment, the electronic device can determine the license plate region in the vehicle image to be searched, perform OCR recognition on the license plate region, and obtain the recognition result, wherein the recognition result includes the license plate information.

[0049] Step 203: If no license plate information is obtained during license plate information extraction, determine the license plate information of the vehicle image to be searched based on the license plate information of the candidate vehicle images in the first recall result.

[0050] In this embodiment of the disclosure, the electronic device performing step 203 may, for example, select a first candidate vehicle image from the first recall result, wherein the similarity between the feature vector of the first candidate vehicle image and the feature vector of the vehicle image to be searched is greater than or equal to a first similarity threshold; determine the number of images corresponding to each license plate information based on the license plate information of the multiple first candidate vehicle images; when there is a first number of images among the number of images, use the license plate information corresponding to the first number of images as the license plate information of the vehicle image to be searched, wherein the first number of images is the maximum value among the multiple number of images, and the minimum difference between the first number of images and the other number of images among the multiple number of images is greater than or equal to a preset difference threshold.

[0051] Among these, the minimum difference between the number of the first image and the number of other images among the multiple image counts is greater than or equal to a preset difference threshold. This can occur in two ways: either all the license plate information of the first candidate vehicle images is the same, or the license plate information of the first candidate vehicle images contains the most images with the first license plate information, and very few images with other license plate information. The first license plate information refers to the license plate information corresponding to the first number of images.

[0052] In this case, when there is a first number of images in the image count, the license plate information corresponding to the first number of images is used as the license plate information of the vehicle image to be searched. This can further improve the accuracy of the obtained license plate information when the license plate information of the vehicle image to be searched is determined.

[0053] Step 204: Determine the first recall result based on feature vectors and the second recall result based on license plate information.

[0054] Step 205: Determine the image recall result based on the first recall result and the second recall result.

[0055] Step 206: Based on the image recall results, determine the image search results for the vehicle image to be searched.

[0056] It should be noted that for detailed descriptions of steps 204, 205, and 206, please refer to the descriptions in other embodiments of this disclosure, and they will not be described in detail here.

[0057] In summary, the AI-based vehicle search method of this disclosure involves: extracting feature vectors from images of vehicles to be searched; extracting license plate information from images of vehicles to be searched; when no license plate information is obtained during license plate extraction, determining the license plate information of the vehicle to be searched based on the license plate information of candidate vehicle images in the first recall result; determining the first recall result based on the feature vector and the second recall result based on the license plate information; determining the image recall result based on the first and second recall results; and determining the image search results for the vehicle to be searched based on the image recall results. Thus, by combining the license plate information and feature vectors of the vehicle to be searched, and fusing and deduplicating the second recall result based on the license plate information and the first recall result based on the feature vector, the image search results for the vehicle to be searched are obtained, thereby expanding the number of images in the image search results and improving search efficiency and accuracy.

[0058] To further improve the accuracy of the determined image recall results and expand the number of images in the image recall results, candidate vehicle images can be selected from the recall results of the images of the vehicle to be searched, and then a search can be performed to obtain the recall results of the candidate vehicle images. The two recall results are then merged to determine the image recall result of the image to be searched. Figure 3 As shown, Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure. Figure 3 The illustrated embodiment may include the following steps:

[0059] Step 301: Determine the license plate information and feature vector of the vehicle image to be searched.

[0060] Step 302: Determine the first recall result based on the feature vector and the second recall result based on the license plate information.

[0061] Step 303: Merge and deduplicate the first recall result and the second recall result to obtain the recall result of the vehicle image to be searched.

[0062] Step 304: Select candidate vehicle images to be searched from the recall results of the vehicle images to be searched.

[0063] In this embodiment of the disclosure, the electronic device can select a fourth candidate vehicle image from the recall results of the vehicle images to be searched, wherein the edit distance between the license plate information of the fourth candidate vehicle image and the license plate information of the vehicle image to be searched is less than or equal to a second distance threshold, and the similarity between the feature vector of the fourth candidate vehicle image and the feature vector of the vehicle image to be searched is greater than or equal to a third similarity threshold; by selecting the candidate vehicle image to be searched from the fourth candidate vehicle image, the accuracy of the candidate vehicle images is ensured.

[0064] It should be noted that, to further improve the accuracy of the selected candidate vehicle images, in one example, multiple candidate vehicle images from the fourth candidate vehicle image, taken within the same shooting time period as the image to be searched, can be obtained. The image to be searched can then be selected from these multiple candidate vehicle images. Within the same shooting time period, the same vehicle can be photographed multiple times, resulting in multiple images of the same vehicle. Therefore, the candidate vehicle image selected from multiple candidate vehicle images within the same shooting time period has a high probability of being a re-photographed image of the vehicle corresponding to the image to be searched.

[0065] In another example, the images of the same day and hour in the fourth candidate vehicle images are divided into a sampling interval and the number of images in each sampling interval is calculated. The vehicle images with the highest similarity are selected from each sampling interval until the number reaches a preset number. The preset number of selected vehicle images are then used as candidate vehicle images to be searched.

[0066] In this embodiment of the disclosure, when no fourth candidate vehicle image is selected, the electronic device can select a fifth candidate vehicle image from the recall results of the vehicle images to be searched, wherein the similarity between the feature vector of the fifth candidate vehicle image and the feature vector of the vehicle image to be searched is greater than or equal to a third similarity threshold; selecting the candidate vehicle image to be searched from the fifth candidate vehicle image ensures the accuracy of the candidate vehicle images.

[0067] It should be noted that the number of candidate vehicle images to be searched can be one or more. Furthermore, if a positive sample list is provided, vehicle images from the positive sample list can be used as candidate vehicle images for the search.

[0068] Step 305: Obtain the recall results of the candidate vehicle images to be searched.

[0069] Step 306: Determine the image recall result based on the recall results of the vehicle images to be searched and the candidate vehicle images to be searched.

[0070] In this embodiment of the disclosure, the electronic device performing step 306 may, for example, involve merging and deduplicating the recall results of the vehicle image to be searched and the recall results of the candidate vehicle images to be searched, to obtain a processed recall result; removing the sixth candidate vehicle image from the processed recall result to obtain an image recall result; wherein the edit distance between the license plate information of the sixth candidate vehicle image and the license plate information of the vehicle image to be searched is greater than or equal to a third distance threshold. This increases the number of images in the image recall result.

[0071] Step 307: Based on the image recall results, determine the image search results for the vehicle image to be searched.

[0072] It should be noted that for detailed descriptions of steps 301, 302, and 307, please refer to the descriptions in other embodiments of this disclosure, and they will not be described in detail here.

[0073] The AI-based vehicle search method of this disclosure determines the license plate information and feature vector of a vehicle image to be searched; determines a first recall result based on the feature vector and a second recall result based on the license plate information; merges and deduplicates the first and second recall results to obtain a recall result of the vehicle image to be searched; selects candidate vehicle images to be searched from the recall result of the vehicle image to be searched; obtains the recall result of the candidate vehicle images to be searched; determines the image recall result based on the recall result of the vehicle image to be searched and the recall result of the candidate vehicle images to be searched; and determines the image search result of the vehicle image to be searched based on the image recall result. Therefore, by combining the license plate information and feature vector of the vehicle image to be searched, and fusing and deduplicated the second recall result based on the license plate information and the first recall result based on the feature vector, the image search result of the vehicle image to be searched is obtained, thereby expanding the number of images in the image search result and improving search efficiency and accuracy.

[0074] For example, Figure 4 This is a flowchart of an artificial intelligence-based vehicle search method according to an embodiment of the present disclosure, such as... Figure 4As shown: First, input the query for the vehicle image to be searched. Then, perform the first recall based on the query. If the query is for a motor vehicle, the first recall result based on feature vectors and the second recall result based on license plate information can be merged and deduplicated to obtain the first recall result. Select candidate vehicle images from the first recall result as the processed recall result. Merge and deduplicate the first recall result based on feature vectors and the second recall result based on license plate information for the candidate vehicle images to be searched to obtain the second recall result. Merge and deduplicate the first and second recall results to obtain the image recall result, and perform image-to-image discrimination scoring on each candidate vehicle image in the image recall result. When a negative sample list exists, perform image-to-image discrimination scoring on the negative sample vehicle images in the negative sample list and the candidate vehicle images, and remove the candidate vehicle images with lower scores in the image recall result. Sort the candidate vehicle images in the image recall result from high to low scores, and select a preset number of candidate vehicle images at the top of the sorted list. These preset number of candidate vehicle images are the image search results for the vehicle image to be searched.

[0075] For example, Figure 5 This is an application diagram illustrating the artificial intelligence-based vehicle search method used to implement the embodiments of this disclosure. Figure 5 It includes: image database 501, data analysis module (DA) 502, basic search module (BS) 503, and access control module (AC) 504.

[0076] The image database 501 stores a large number of vehicle images, along with their license plate information and feature vectors. DA502 parses the images of vehicles to be searched and determines the positive and negative sample lists. AC504 includes a central control unit 5041 and a strategy operator. The central control unit obtains the images of vehicles to be searched, the positive sample list, and the negative sample list from DA502. The central control unit calls the strategy operator to obtain the license plate information and feature vectors of the images of vehicles to be searched. The BS module is then called to perform the following processing: based on the license plate information and feature vectors of the images of vehicles to be searched, obtains the image recall results, sorts the candidate vehicle images in the image recall results, and filters some of the candidate vehicle images in the image recall results.

[0077] To achieve the above embodiments, this disclosure also proposes an artificial intelligence-based vehicle search device.

[0078] like Figure 6 As shown, Figure 6The diagram is based on the fourth embodiment of the present disclosure. The artificial intelligence-based vehicle search device 600 includes: a first determining module 610, a second determining module 620, a third determining module 630, and a fourth determining module 640.

[0079] The first determining module 610 is used to determine the license plate information and feature vector of the vehicle image to be searched.

[0080] The second determining module 620 is used to determine a first recall result based on the feature vector and a second recall result based on the license plate information.

[0081] The third determining module 630 is used to determine the image recall result based on the first recall result and the second recall result.

[0082] The fourth determining module 640 is used to determine the image search results of the vehicle image to be searched based on the image recall results.

[0083] As one possible implementation of this disclosure, the first determining module 610 is specifically used to: perform vector extraction processing on the image of the vehicle to be searched to obtain the feature vector; extract license plate information from the image of the vehicle to be searched to obtain the license plate information; and when the license plate information is not obtained during the license plate information extraction, determine the license plate information of the image of the vehicle to be searched based on the license plate information of the candidate vehicle images in the first recall result.

[0084] As one possible implementation of this disclosure, the first determining module 610 is specifically configured to: select a first candidate vehicle image from the first recall result, wherein the similarity between the feature vector of the first candidate vehicle image and the feature vector of the vehicle image to be searched is greater than or equal to a first similarity threshold; determine the number of images corresponding to each license plate information based on the license plate information of the plurality of first candidate vehicle images; when there is a first number of images among the number of images, use the license plate information corresponding to the first number of images as the license plate information of the vehicle image to be searched, wherein the first number of images is the maximum value among the plurality of image numbers, and the minimum difference between the first number of images and the other number of images among the plurality of image numbers is greater than or equal to a preset difference threshold.

[0085] As one possible implementation of this disclosure, the second determining module 620 is specifically configured to: query an image database based on the feature vector to obtain a second candidate vehicle image from the image database; wherein the similarity between the feature vector of the second candidate image and the feature vector of the vehicle image to be searched is greater than or equal to a second similarity threshold; determine the first recall result based on the second candidate vehicle image; query the image database based on the license plate information to obtain a third candidate vehicle image from the image database; wherein the edit distance between the license plate information of the third candidate vehicle image and the license plate information of the image to be searched is less than or equal to a first distance threshold; and determine the second recall result based on the third candidate vehicle image.

[0086] As one possible implementation of this disclosure, the third determining module 630 includes: a processing unit, a selection unit, an acquisition unit, and a determining unit; the processing unit is used to merge and deduplicate the first recall result and the second recall result to obtain the recall result of the vehicle image to be searched; the selection unit is used to select a candidate vehicle image to be searched from the recall result of the vehicle image to be searched; the acquisition unit is used to acquire the recall result of the candidate vehicle image to be searched; and the determining unit is used to determine the image recall result based on the recall result of the vehicle image to be searched and the recall result of the candidate vehicle image to be searched.

[0087] As one possible implementation of this disclosure, the selection unit is specifically used to: select a fourth candidate vehicle image from the recall results of the vehicle image to be searched, wherein the edit distance between the license plate information of the fourth candidate vehicle image and the license plate information of the vehicle image to be searched is less than or equal to a second distance threshold, and the similarity between the feature vector of the fourth candidate vehicle image and the feature vector of the vehicle image to be searched is greater than or equal to a third similarity threshold; and select the candidate vehicle image to be searched from the fourth candidate vehicle image.

[0088] As one possible implementation of this disclosure, the selection unit is further configured to, when the fourth candidate vehicle image is not selected, select a fifth candidate vehicle image from the recall results of the vehicle images to be searched, wherein the similarity between the feature vector of the fifth candidate vehicle image and the feature vector of the vehicle image to be searched is greater than or equal to the third similarity threshold; and select the candidate vehicle image to be searched from the fifth candidate vehicle image.

[0089] As one possible implementation of this disclosure, the determining unit is specifically configured to: merge and deduplicate the recall results of the vehicle image to be searched and the recall results of the candidate vehicle images to be searched to obtain a processed recall result; remove the sixth candidate vehicle image from the processed recall result to obtain the image recall result; wherein the edit distance between the license plate information of the sixth candidate vehicle image and the license plate information of the vehicle image to be searched is greater than or equal to a third distance threshold.

[0090] As one possible implementation of this disclosure, the fourth determining module 640 is specifically configured to: determine a score between each candidate vehicle image and the vehicle image to be searched for each candidate vehicle image in the image recall result; sort the candidate vehicle images in the image recall result in descending order according to the score to obtain a sorting result; and generate the image search result based on a preset number of candidate vehicle images that are ranked first in the sorting result.

[0091] As one possible implementation of this disclosure, the fourth determining module 640 is specifically configured to: for each candidate vehicle image in the image recall result, determine the vehicle type and facial feature vector of the candidate vehicle image; determine the vehicle type and facial feature vector of the vehicle image to be searched; determine the type similarity between the vehicle type of the candidate vehicle image and the vehicle type of the vehicle image to be searched; determine the facial feature similarity between the facial feature vector of the candidate vehicle image and the facial feature vector of the vehicle image to be searched; and perform a weighted summation of the similarity between the feature vector of the vehicle image to be searched and the feature vector of the candidate vehicle image, the type similarity, and the facial feature similarity to obtain the scoring value.

[0092] It should be noted that the above explanation of the AI-based vehicle search method also applies to the AI-based vehicle search device in this embodiment, and this embodiment will not repeat the above.

[0093] The AI-based vehicle search device of this disclosure determines the license plate information and feature vector of a vehicle image to be searched; determines a first recall result based on the feature vector and a second recall result based on the license plate information; determines an image recall result based on the first and second recall results; and determines the image search results for the vehicle image to be searched based on the image recall results. Thus, by combining the license plate information and feature vector of the vehicle image to be searched, and fusing and deduplicating the second recall result based on the license plate information and the first recall result based on the feature vector, the image search results for the vehicle image to be searched are obtained, thereby expanding the number of images in the image search results and improving search efficiency and accuracy.

[0094] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision, and disclosure of users' personal information are all carried out with the consent of the users, and all comply with the provisions of relevant laws and regulations, and do not violate public order and good morals.

[0095] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0096] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0097] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0098] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0099] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0100] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as an AI-based vehicle search method. For example, in some embodiments, the AI-based vehicle search method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the AI-based vehicle search method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the AI-based vehicle search method by any other suitable means (e.g., by means of firmware).

[0101] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0102] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0103] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0104] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0105] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0106] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0107] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0108] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An artificial intelligence-based vehicle search method, comprising: Determine the license plate information and feature vector of the vehicle image to be searched; Determine a first recall result based on the feature vector and a second recall result based on the license plate information; Based on the first recall result and the second recall result, the image recall result is determined; Based on the image recall results, determine the image search results for the vehicle image to be searched; The step of determining the image recall result based on the first recall result and the second recall result includes: The first recall result and the second recall result are merged and deduplicated to obtain the recall result of the vehicle image to be searched; Select candidate vehicle images from the recall results of the images to be searched; Obtain the recall results of the candidate vehicle images to be searched; Based on the recall results of the images of the vehicles to be searched and the recall results of the images of the candidate vehicles to be searched, the image recall results are determined. The step of selecting candidate vehicle images from the recall results of the vehicle images to be searched includes: A fourth candidate vehicle image is selected from the recall results of the vehicle image to be searched, wherein the edit distance between the license plate information of the fourth candidate vehicle image and the license plate information of the vehicle image to be searched is less than or equal to a second distance threshold, and the similarity between the feature vector of the fourth candidate vehicle image and the feature vector of the vehicle image to be searched is greater than or equal to a third similarity threshold. Select the candidate vehicle image to be searched from the fourth candidate vehicle image; The step of selecting the candidate vehicle image to be searched from the fourth candidate vehicle image includes: From the fourth candidate vehicle image, obtain multiple candidate vehicle images that are within the same shooting time period as the vehicle image to be searched, and select the candidate vehicle image to be searched from the multiple candidate vehicle images. The multiple candidate vehicle images include multiple images of the same vehicle obtained by taking multiple photos of the same vehicle.

2. The method according to claim 1, wherein, The process of determining the license plate information and feature vector of the vehicle image to be searched includes: The image of the vehicle to be searched is subjected to vector extraction processing to obtain the feature vector; The license plate information is extracted from the image of the vehicle to be searched to obtain the license plate information; If the license plate information is not obtained during the license plate information extraction process, the license plate information of the vehicle image to be searched is determined based on the license plate information of the candidate vehicle images in the first recall result.

3. The method according to claim 2, wherein, The step of determining the license plate information of the vehicle image to be searched based on the license plate information of the candidate vehicle images in the first recall result includes: A first candidate vehicle image is selected from the first recall result, wherein the similarity between the feature vector of the first candidate vehicle image and the feature vector of the vehicle image to be searched is greater than or equal to a first similarity threshold. Based on the license plate information of multiple first candidate vehicle images, determine the number of images corresponding to each license plate information; When a first number of images exists in the number of images, the license plate information corresponding to the first number of images is used as the license plate information of the vehicle image to be searched. Here, the first number of images is the maximum value among the multiple number of images, and the minimum difference between the first number of images and the other number of images among the multiple number of images is greater than or equal to a preset difference threshold.

4. The method according to claim 1, wherein, Determining the first recall result based on the feature vector and the second recall result based on the license plate information includes: Based on the feature vector, the image database is queried to obtain a second candidate vehicle image from the image database; wherein the similarity between the feature vector of the second candidate image and the feature vector of the vehicle image to be searched is greater than or equal to a second similarity threshold. The first recall result is determined based on the second candidate vehicle image; Based on the license plate information, the image database is queried to obtain a third candidate vehicle image from the image database; wherein the edit distance between the license plate information of the third candidate vehicle image and the license plate information of the image to be searched is less than or equal to a first distance threshold. The second recall result is determined based on the third candidate vehicle image.

5. The method according to claim 1, wherein, The step of selecting candidate vehicle images from the recall results of the vehicle images to be searched also includes: If the fourth candidate vehicle image is not selected, a fifth candidate vehicle image is selected from the recall results of the vehicle images to be searched, wherein the similarity between the feature vector of the fifth candidate vehicle image and the feature vector of the vehicle image to be searched is greater than or equal to the third similarity threshold. Select the candidate vehicle image to be searched from the fifth candidate vehicle image.

6. The method according to claim 1, wherein, The step of determining the image recall result based on the recall result of the image of the vehicle to be searched and the recall result of the image of the candidate vehicle to be searched includes: The recall results of the vehicle images to be searched and the recall results of the candidate vehicle images to be searched are merged and deduplicated to obtain the processed recall results. The sixth candidate vehicle image in the processed recall result is removed to obtain the image recall result; wherein, the edit distance between the license plate information of the sixth candidate vehicle image and the license plate information of the vehicle image to be searched is greater than or equal to a third distance threshold.

7. The method according to claim 1, wherein, The step of determining the image search results for the vehicle image to be searched based on the image recall results includes: For each candidate vehicle image in the image recall results, a score is determined between the candidate vehicle image and the vehicle image to be searched. Based on the scoring values, the candidate vehicle images in the image recall results are sorted in descending order to obtain the sorting results; The image search results are generated based on a preset number of candidate vehicle images that appear first in the sorting results.

8. The method according to claim 7, wherein, For each candidate vehicle image in the image recall results, determining the score between the candidate vehicle image and the vehicle image to be searched includes: For each candidate vehicle image in the image recall results, determine the vehicle type and facial feature vector of the candidate vehicle image; Determine the vehicle type and facial feature vector of the vehicle image to be searched; Determine the type similarity between the vehicle type of the candidate vehicle image and the vehicle type of the vehicle image to be searched; Determine the facial feature similarity between the facial feature vector of the candidate vehicle image and the facial feature vector of the vehicle image to be searched; The score is obtained by weighted summing the similarity between the feature vector of the vehicle image to be searched and the feature vector of the candidate vehicle image, the type similarity, and the facial feature similarity.

9. An artificial intelligence-based vehicle search device, comprising: The first determining module is used to determine the license plate information and feature vector of the vehicle image to be searched; The second determining module is used to determine a first recall result based on the feature vector and a second recall result based on the license plate information; The third determining module is used to determine the image recall result based on the first recall result and the second recall result; The fourth determining module is used to determine the image search results for the vehicle image to be searched based on the image recall results. The third determining module includes: a processing unit, a selection unit, an acquisition unit, and a determining unit; The processing unit is used to merge and deduplicate the first recall result and the second recall result to obtain the recall result of the vehicle image to be searched. The selection unit is used to select candidate vehicle images to be searched from the recall results of the vehicle images to be searched. The acquisition unit is used to acquire the recall results of the candidate vehicle images to be searched; The determining unit is used to determine the image recall result based on the recall result of the vehicle image to be searched and the recall result of the candidate vehicle image to be searched. The selection unit is specifically used to select a fourth candidate vehicle image from the recall results of the vehicle image to be searched, wherein the edit distance between the license plate information of the fourth candidate vehicle image and the license plate information of the vehicle image to be searched is less than or equal to a second distance threshold, and the similarity between the feature vector of the fourth candidate vehicle image and the feature vector of the vehicle image to be searched is greater than or equal to a third similarity threshold. Select the candidate vehicle image to be searched from the fourth candidate vehicle image; The selection unit is specifically used to obtain multiple candidate vehicle images from the fourth candidate vehicle images that are within the same shooting time period as the vehicle image to be searched, and to select the candidate vehicle image to be searched from the multiple candidate vehicle images. The multiple candidate vehicle images include multiple images of the same vehicle obtained by taking multiple photos of the same vehicle.

10. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

11. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

12. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-8.

Images