Vehicle image retrieval method and device based on multilayer pattern structure, electronic equipment, storage medium and computer product

Abstract

The invention relates to the technical field of data processing, and provides a vehicle image retrieval method and device based on a multi-layer pattern structure, electronic equipment, a storage medium and a computer product, and the method comprises the steps: obtaining a to-be-retrieved vehicle image; determining a vehicle characteristic value of the to-be-retrieved vehicle image; based on the vehicle feature value of the to-be-retrieved vehicle image, performing retrieval in a multi-level retrieval graph to obtain at least one target vehicle feature value; the multi-level retrieval graph is constructed by adopting a neighbor algorithm based on vehicle characteristic values of vehicle passing images; and determining the vehicle passing image corresponding to each target vehicle feature value as a target vehicle image. Compared with the prior art, all data does not need to be compared during retrieval, only the data of the neighbor nodes need to be compared, and the vehicle retrieval efficiency is greatly improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a vehicle image retrieval method, device, electronic device, storage medium and computer product based on a multi-layer graph structure. Background Art

[0002] Image search for vehicles is a means of intelligence analysis for suspicious vehicles, aiming to perform similarity retrieval based on the features of the suspected vehicles. First, the images of the vehicles are annotated and trained to form rich feature values, which are then stored. When searching for the target, the feature values ​​are compared to find similar result sets.

[0003] As the construction of smart cities progresses, the construction of vehicle checkpoints has also gradually increased. At present, the average daily amount of vehicle checkpoint capture data is about 50 million in large cities, and more than 10 million in small and medium-sized cities. When searching for massive amounts of vehicle images, the top priority is retrieval efficiency.

[0004] However, the traditional vehicle image feature retrieval and analysis method is based on the 1:N vehicle image feature. In this way, the efficiency of pulling image feature value data from the disk and then comparing it in billions of data is extremely low. As a result, the current vehicle retrieval is inefficient. Summary of the invention

[0005] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a vehicle image retrieval method, device, electronic device, storage medium and computer product based on a multi-layer graph structure, which is used to solve the problem that the traditional vehicle image feature retrieval and analysis method is based on the 1:N of vehicle image features, resulting in extremely low efficiency, and improve the vehicle retrieval efficiency.

[0006] According to the first aspect of the present application, a vehicle image retrieval method based on a multi-layer graph structure includes: Obtaining the vehicle image to be retrieved; Determining a vehicle feature value of the vehicle image to be retrieved; Based on the vehicle feature value of the vehicle image to be retrieved, a search is performed in a multi-level search graph to obtain at least one target vehicle feature value; the multi-level search graph is constructed based on the vehicle feature value of the vehicle image by using a nearest neighbor algorithm; The passing vehicle image corresponding to each target vehicle feature value is determined as the target vehicle image.

[0007] According to one embodiment of the present application, the multi-level search graph is constructed in the following manner: Acquire multiple images of vehicles passing by; Determine the vehicle feature value of each vehicle passing image respectively; Start traversing from the first layer, randomly select a vehicle feature value from the vehicle feature values ​​of each vehicle passing image as a feature value node and write it into the comparison set; For each feature value node in the comparison set, determine the nearest neighbor result set from the vehicle feature values ​​of each passing vehicle image; Writing each of the neighbor result sets into the current level based on random decision making; If the current number of levels does not reach the preset level threshold, enter the next level of the current level, write each neighbor result set of the current level into the comparison set, return to execute the step of determining the neighbor result set from the vehicle feature values ​​of each passing vehicle image for each feature value node in the comparison set; write each neighbor result set into the current level based on random decision-making until the current number of levels reaches the preset level threshold.

[0008] According to an embodiment of the present application, when determining a neighbor result set from the vehicle feature values ​​of each passing vehicle image for each feature value node in the comparison set, the following steps are performed for each feature value node in the comparison set: Determine the similarity between the current feature value node and the vehicle feature value of each passing vehicle image; Adding vehicle feature values ​​of a first preset number of vehicle images having a similarity greater than a preset similarity threshold to a nearest neighbor node set; Determining vehicle feature values ​​of a second preset number of vehicle passing images from the nearest neighbor node set as target vehicle feature values; An edge relationship is constructed for each of the target vehicle feature values ​​to form a neighbor result set of the current feature value node.

[0009] According to an embodiment of the present application, the vehicle feature value based on the vehicle image to be retrieved is searched in a multi-level search graph to obtain at least target vehicle feature values, including: Taking the vehicle feature value of the vehicle image to be retrieved as a query point; In the first level of the multi-level retrieval graph, vehicle feature values ​​are randomly selected as candidate nodes and added to the candidate set; Traversing the neighbor result set of each candidate node in the candidate set; In the case where the search result set is not full, each node in each of the neighbor result sets is added to the search result set; the search result set is initially empty and the maximum number is a third preset number; When the search result set is full, the target node in each of the neighbor result sets is added to the search result set; the distance between the target node and the query point is less than the maximum distance between the query point and each node in the search result set; Eliminate the node with the largest distance from the query point among the nodes in the search result set; If the current level is not the last level in the multi-level search graph, jump to the next level of the current level; Obtain a neighbor result set of each node in the search result set at the current level; Add the nodes in the neighbor result set of each node in the retrieval result set that have not been traversed in the neighbor result set of the current level to the candidate set, and execute the step of traversing the neighbor result set of each candidate node in the candidate set until the level after execution is the last level in the multi-level retrieval graph, and determine each vehicle feature value in the retrieval result set as the target vehicle feature value.

[0010] According to one embodiment of the present application, determining the vehicle feature value of the vehicle image to be retrieved includes: Inputting the vehicle image to be retrieved into a vehicle feature extraction model to obtain a floating-point vehicle feature value output by the vehicle feature extraction model; the vehicle feature extraction model is used to extract the vehicle feature value based on the input image; The floating-point vehicle feature value is mapped to a signed integer data type to obtain the vehicle feature value of the vehicle image to be retrieved.

[0011] According to one embodiment of the present application, after determining the passing vehicle image corresponding to each target vehicle feature value as the target vehicle image, the method further includes: Dequantize each target vehicle characteristic value to obtain the corresponding target floating-point vehicle characteristic value; The similarities between the floating-point vehicle feature value of the to-be-retrieved vehicle image and each target floating-point vehicle feature value are respectively determined.

[0012] According to the second aspect of the present application, a vehicle image retrieval device based on a multi-layer graph structure includes: An acquisition module, used for acquiring a vehicle image to be retrieved; A first determining module, used to determine the vehicle feature value of the vehicle image to be retrieved; A retrieval module, configured to search in a multi-level retrieval graph based on the vehicle feature values ​​of the vehicle image to be retrieved, and obtain at least one target vehicle feature value; the multi-level retrieval graph is constructed based on the vehicle feature values ​​of the vehicle image by using a nearest neighbor algorithm; The second determination module is used to determine the passing vehicle image corresponding to each target vehicle characteristic value as the target vehicle image.

[0013] According to an electronic device of an embodiment of the third aspect of the present application, the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the vehicle image retrieval method based on a multi-layer graph structure as described above is implemented.

[0014] According to the storage medium of the fourth aspect embodiment of the present application, the storage medium is a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements any of the above-mentioned vehicle image retrieval methods based on a multi-layer graph structure.

[0015] A computer program product according to an embodiment of the fifth aspect of the present application includes a computer program, which, when executed by a processor, implements any of the above-mentioned vehicle image retrieval methods based on a multi-layer graph structure.

[0016] The above one or more technical solutions in the embodiments of the present application have at least the following technical effects: Based on the vehicle feature values ​​of the passing vehicle images, a nearest neighbor algorithm is used to construct a multi-level retrieval graph, so that after obtaining the vehicle image to be retrieved and determining the vehicle feature values ​​of the vehicle image to be retrieved, a search is performed in the multi-level retrieval graph based on the vehicle feature values ​​of the vehicle image to be retrieved, and at least one target vehicle feature value can be obtained. Then, the passing vehicle images corresponding to each target vehicle feature value can be used as the target vehicle images. Since it is not necessary to compare all the data during the retrieval, only the data of the nearest node needs to be compared, thereby greatly improving the vehicle retrieval efficiency.

[0017] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flowchart of a vehicle image retrieval method based on a multi-layer graph structure provided in an embodiment of the present application.

[0020] Figure 2 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0021] The following is a further detailed description of the implementation of the present application in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but cannot be used to limit the scope of the present application.

[0022] In the description of the embodiments of the present application, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the embodiments of the present application. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0023] In the description of the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0024] In the embodiments of the present application, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0025] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0026] The present application proposes a vehicle image retrieval method, device, electronic device, storage medium and computer product based on a multi-layer graph structure.

[0027] Figure 1 FIG. 1 is a flow chart of a vehicle image retrieval method based on a multi-layer graph structure provided in an embodiment of the present application. Figure 1 As shown, the vehicle image retrieval method based on the multi-layer graph structure includes: Step 110, obtaining the vehicle image to be retrieved.

[0028] Step 120: Determine the vehicle feature value of the vehicle image to be retrieved.

[0029] Step 130, based on the vehicle feature values ​​of the vehicle image to be retrieved, search in a multi-level search graph to obtain at least one target vehicle feature value; the multi-level search graph is constructed based on the vehicle feature values ​​of the passing vehicle image using a nearest neighbor algorithm.

[0030] Step 140: determine the passing vehicle image corresponding to each target vehicle feature value as the target vehicle image.

[0031] It should be noted that the execution subject of the vehicle image retrieval method based on the multi-layer graph structure provided in the embodiment of the present application can be a server, a computer device, etc. The computer device can be, for example, a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a receiver, a netbook or a personal digital assistant (PDA), etc. It should be noted that the data required to be obtained in this application are all legally obtained.

[0032] The server or computer device of the present application may be provided with or connected to a vehicle image retrieval device based on a multi-layer graph structure, thereby controlling the vehicle image retrieval device based on a multi-layer graph structure to execute the vehicle image retrieval method based on a multi-layer graph structure of the present application.

[0033] This application can be used in the scene of image search for checkpoint vehicles. It is particularly suitable for image retrieval application scenarios where the single image feature value is large and the amount of vehicle data reaches billions.

[0034] Specifically, the present application obtains an image (or referred to as a picture) containing a vehicle that needs to be searched by image as a vehicle image to be retrieved.

[0035] Furthermore, feature value extraction can be performed on the vehicle image to be retrieved to obtain a floating-point vehicle feature value of the vehicle image to be retrieved, and then the floating-point vehicle feature value is quantized to obtain an integer vehicle feature value. The floating-point vehicle feature value can be a 256-bit floating-point array consisting of floating-point feature values ​​corresponding to features such as lights, wheels, license plates, vehicle classification, brand, model, body color, window status, door status, body stickers, roof luggage racks, occlusion annotations, angles, etc.

[0036] It should be noted that the present application can also obtain vehicle passing images (hereinafter also referred to as vehicle passing pictures) taken from a specified vehicle checkpoint for vehicles passing through the checkpoint within a specified time. For example: according to the search requirements, all vehicle passing images of checkpoints A, B, and C in time period D among 10 vehicle checkpoints are obtained.

[0037] Furthermore, the integer vehicle feature value of each passing vehicle image can be obtained through feature value extraction and quantization.

[0038] Furthermore, a multi-level retrieval graph can be constructed using a nearest neighbor algorithm according to the vehicle feature values ​​of each vehicle passing image. In the multi-level retrieval graph, each layer can be regarded as a separate graph, and these graphs are connected to each other in some way.

[0039] Furthermore, according to the vehicle feature values ​​of the vehicle image to be retrieved, a neighbor search can be performed in a multi-level search graph to implement distributed vehicle image retrieval analysis, and finally at least one target vehicle feature value can be obtained.

[0040] Furthermore, the passing vehicle images corresponding to the characteristic values ​​of the target vehicles may be respectively determined as the target vehicle images.

[0041] According to the vehicle image retrieval method based on a multi-layer graph structure in an embodiment of the present application, a multi-level retrieval graph is constructed using a nearest neighbor algorithm based on the vehicle feature values ​​of the passing vehicle image. After obtaining the vehicle image to be retrieved and determining the vehicle feature values ​​of the vehicle image to be retrieved, a search is performed in the multi-level retrieval graph based on the vehicle feature values ​​of the vehicle image to be retrieved, and at least one target vehicle feature value can be obtained. Then, the passing vehicle image corresponding to each target vehicle feature value can be used as the target vehicle image. Since there is no need to compare all data during retrieval, only the neighboring node data needs to be compared, thereby greatly improving the vehicle retrieval efficiency.

[0042] Based on the above embodiment, determining the vehicle feature value of the vehicle image to be retrieved includes: Inputting the vehicle image to be retrieved into the vehicle feature extraction model to obtain the floating-point vehicle feature value output by the vehicle feature extraction model; the vehicle feature extraction model is used to extract the vehicle feature value based on the input image; The floating-point vehicle feature value is mapped to a signed integer data type to obtain the vehicle feature value of the vehicle image to be retrieved.

[0043] Specifically, the present application can pre-train the model based on the sample annotated vehicle images to obtain a vehicle feature extraction model that can extract the vehicle feature values ​​in the image based on the input image. The sample annotated vehicle images contain vehicles, and each vehicle may include annotation boxes and their annotation information such as lights, wheels, license plates, vehicle classification, brand, model, body color, window status, door status, body stickers, roof racks, occlusion annotations, angles, etc. Therefore, the vehicle feature extraction model can extract rich floating-point feature values ​​from the input image.

[0044] Therefore, the present application can input the vehicle image to be retrieved into the vehicle feature extraction model to obtain the floating-point vehicle feature value output by the vehicle feature extraction model.

[0045] Furthermore, the floating-point vehicle characteristic value in the present application may specifically be a float32 characteristic value, which is expected to occupy 32 bits (4 bytes).

[0046] Therefore, the present application can map the floating-point vehicle feature value into the int8 range, thereby mapping the floating-point vehicle feature value into a signed integer data type, and after completing the mapping, the vehicle feature value of the vehicle image to be retrieved is obtained.

[0047] Specifically, the floating-point vehicle characteristic value can be mapped to a signed integer data type by linear quantization, which can be achieved more specifically by the following formula: ; in, Represents the original float32 feature vector value; Represents the quantized int8 value; min_val and max_val represent the minimum and maximum values ​​of floating-point numbers respectively; round() means rounding the result value to the nearest integer.

[0048] This application can convert 32-bit Float values ​​into 8-bit int values ​​by quantizing the eigenvalues. First, it can reduce memory usage: compared with 32-bit floating-point numbers (4 bytes), 8-bit integers (1 byte) can significantly reduce memory usage, especially when processing large-scale vector data.

[0049] Secondly, it can speed up calculations: modern central processing units (CPUs) and graphics processing units (GPUs) have good support for int8 operations, which can accelerate matrix multiplication and other linear algebra operations, thereby improving retrieval speed.

[0050] Finally, the bandwidth requirement is reduced during cluster query data interaction: during network transmission or disk reading, the transmission bandwidth requirement for int8 data is lower, further improving the overall performance.

[0051] Based on the above embodiment, after determining the passing vehicle image corresponding to each target vehicle feature value as the target vehicle image, the method further includes: Dequantize each target vehicle characteristic value to obtain the corresponding target floating-point vehicle characteristic value; The similarities between the floating-point vehicle feature values ​​of the vehicle image to be retrieved and each target floating-point vehicle feature value are determined respectively.

[0052] Specifically, after determining the passing vehicle image corresponding to each target vehicle characteristic value as the target vehicle image, the present application can also perform dequantization processing on each target vehicle characteristic value respectively, converting the integer characteristic value into the original floating point type, thereby obtaining the corresponding target floating point vehicle characteristic value.

[0053] Furthermore, the similarity between the floating-point vehicle feature value of the vehicle image to be retrieved and each target floating-point vehicle feature value can be determined respectively by using the cosine function.

[0054] During the search process, this application restores some precision through inverse quantization to ensure the accuracy of vehicle retrieval results.

[0055] Based on the above embodiment, the multi-level search graph is constructed in the following manner: Acquire multiple images of vehicles passing by; Determine the vehicle feature value of each vehicle passing image respectively; Start traversing from the first layer, randomly select a vehicle feature value from the vehicle feature values ​​of each vehicle passing image as a feature value node and write it into the comparison set; For each feature value node in the comparison set, determine the nearest neighbor result set from the vehicle feature values ​​of each passing vehicle image; Write each neighbor result set to the current level based on random decision making; If the current number of levels does not reach the preset level threshold, enter the next level of the current level, write each neighbor result set of the current level into the comparison set, return to execute for each feature value node in the comparison set, and determine the neighbor result set from the vehicle feature values ​​of each passing vehicle image; write each neighbor result set into the current level based on random decision-making until the current number of levels reaches the preset level threshold.

[0056] Furthermore, for each feature value node in the comparison set, when determining the nearest neighbor result set from the vehicle feature values ​​of each passing vehicle image, the following steps are performed for each feature value node in the comparison set: Determine the similarity between the current feature value node and the vehicle feature value of each passing vehicle image; Adding vehicle feature values ​​of a first preset number of vehicle images having a similarity greater than a preset similarity threshold to a nearest neighbor node set; Determine vehicle feature values ​​of a second preset number of vehicle passing images from the nearest neighbor node set as target vehicle feature values; Build edge relationships for each target vehicle eigenvalue and form a neighbor result set of the current eigenvalue node.

[0057] Specifically, the present application can continuously receive vehicle passing images from each vehicle checkpoint, extract feature values ​​from the vehicle passing images through a pre-trained vehicle feature extraction model, and form a 256-dimensional floating-point array of vehicle feature values ​​F.

[0058] The vehicle characteristic value F is subjected to characteristic value quantization processing to form a characteristic value I, thereby obtaining the vehicle characteristic value of each passing vehicle image.

[0059] Furthermore, starting from the first layer (e.g., the i-th layer, i=0, 1, 2, ..., n, where n is the maximum number of layers in the multi-level retrieval graph), a vehicle feature value is randomly selected from the vehicle feature values ​​of all passing vehicle images as a feature value node and written into the comparison set C.

[0060] Get the vehicle eigenvalue corresponding to each eigenvalue node in the comparison set C.

[0061] The cosine function is used to calculate the one-to-one similarity between each feature value node in the comparison set C and the vehicle feature values ​​of all vehicle passing images, obtain the vehicle feature values ​​of the first preset number of vehicle passing images whose similarity is greater than the preset similarity threshold, and form the nearest neighbor node set (or the closest node set) c_Search of the corresponding feature value node in the comparison set C. Among them, the preset similarity threshold can be a similarity value set according to the actual scene and retrieval requirements; the first preset number and the subsequent second preset number and third preset number are all numerical values ​​set according to actual requirements, and can be adjusted according to actual requirements. In addition, the first preset number is greater than the second preset number, and the first preset number and the third preset number can be the same or different.

[0062] Furthermore, for the nearest neighbor node set c_Search of each eigenvalue node, each node in the nearest neighbor node set c_Search may be sorted according to similarity, and vehicle eigenvalues ​​of a second preset number (first m) of nodes are obtained as target vehicle eigenvalues ​​of the corresponding eigenvalue node.

[0063] Furthermore, edge relationships are constructed for each target vehicle eigenvalue corresponding to each eigenvalue node to form a neighbor result set M of the corresponding eigenvalue node.

[0064] Furthermore, a random decision (for example, 50%) is made as to whether the neighbor result set M of each eigenvalue node is to be written into the level. If the decision is to write into the level, the neighbor result set M of each eigenvalue node is added to the graph relationship of the level.

[0065] Further, to determine whether the current level is the last level of the multi-level search graph, it can be determined whether the current level number reaches a preset level threshold, where the preset level threshold is the maximum level number of the multi-level search graph set according to actual needs.

[0066] If it is not the last layer, enter the graph structure of the next layer, obtain the neighbor result sets M at the current layer, write them into the comparison set C, and return to execute the above steps from obtaining the vehicle feature values ​​corresponding to each feature value node in the comparison set C to determining whether the current layer is the last layer of the multi-level retrieval graph. If it is not the last layer, enter the graph structure of the next layer, obtain the neighbor result sets M at the current layer, and write them into the comparison set C, until the layer after execution is the last layer, and end the writing process.

[0067] The present application constructs a multi-level search graph based on the nearest neighbor algorithm, so that when searching, there is no need to compare all data, only the nearest neighbor nodes need to be compared, thereby greatly improving the search efficiency.

[0068] Based on the above embodiment, based on the vehicle feature values ​​of the vehicle image to be retrieved, a search is performed in a multi-level search graph to obtain at least target vehicle feature values, including: The vehicle feature value of the vehicle image to be retrieved is used as a query point; In the first level of the multi-level retrieval graph, vehicle feature values ​​are randomly selected as candidate nodes and added to the candidate set; Traverse the neighbor result set of each candidate node in the candidate set; When the search result set is not full, each node in each neighbor result set is added to the search result set; the search result set is initially empty and the maximum number is a third preset number; When the search result set is full, the target node in each neighbor result set is added to the search result set; the distance between the target node and the query point is less than the maximum distance between the query point and each node in the search result set; Eliminate the node with the largest distance from the query point among the nodes in the search result set; If the current level is not the last level in the multi-level search graph, jump to the next level of the current level; Get the neighbor result set of each node in the search result set at the current level; The nodes in the retrieval result set that have not been traversed in the neighbor result set of the current level are added to the candidate set, and the step of traversing the neighbor result set of each candidate node in the candidate set is executed until the level after execution is the last level in the multi-level retrieval graph, and the feature values ​​of each vehicle in the retrieval result set are determined as the target vehicle feature values.

[0069] Specifically, the vehicle feature value of the vehicle image to be retrieved is used as the query point Q, and in the first level of the multi-level retrieval graph, the vehicle feature value is randomly selected as the candidate node C and added to the candidate set D.

[0070] Furthermore, all neighbor result sets M of the candidate node Ci in the candidate set D are traversed.

[0071] Determine whether the distance between each node C(i,j) in the neighbor result set M and the query point Q is greater than the maximum distance between the query point Q and each node in the retrieval result set R.

[0072] Determine whether the search result set R is full. If the search result set is not full, add node C(i,j) to the search result set R. If the search result set is full, delete the node farthest from the query point in the search result set R, and then add node C(i,j) to the search result set R, or add node C(i,j) to the search result set R and then delete the node farthest from the query point in the search result set R.

[0073] Further, it is determined whether the candidate set D has been traversed completely. If not, the step of returning to traverse all neighbor result sets M of the candidate node Ci in the candidate set D is started.

[0074] If the traversal is completed, it is determined whether the current level has reached the last level of the multi-level retrieval graph.

[0075] If it has not reached the last layer, it will jump to the next layer and obtain the neighbor result set M of each node in this layer according to the search result set R.

[0076] Furthermore, it is determined whether the newly added node C(i,j) (each node in the neighbor result set M of each node obtained in this layer) has been traversed. If not, the node C(i,j) is added to the candidate set D, and the step of traversing all neighbor result sets M of the candidate node Ci in the candidate set D is returned to start execution until the level after execution is the last level in the multi-level retrieval graph.

[0077] If the last layer has been traversed, each vehicle feature value in the retrieval result set is determined as the target vehicle feature value.

[0078] Since the present application constructs a multi-level retrieval graph based on the nearest neighbor algorithm, it is not necessary to compare all data during retrieval, only the nearest neighbor nodes need to be compared, thereby greatly improving the retrieval efficiency.

[0079] The vehicle image retrieval device based on a multi-layer graph structure provided by the present application is described below. The vehicle image retrieval device based on a multi-layer graph structure described below and the vehicle image retrieval method based on a multi-layer graph structure described above can be referenced to each other.

[0080] Furthermore, the present application also provides a vehicle image retrieval device based on a multi-layer graph structure.

[0081] The vehicle image retrieval device based on the multi-layer graph structure comprises: An acquisition module, used for acquiring a vehicle image to be retrieved; A first determining module, used to determine the vehicle feature value of the vehicle image to be retrieved; A retrieval module, configured to search in a multi-level retrieval graph based on the vehicle feature values ​​of the vehicle image to be retrieved, and obtain at least one target vehicle feature value; the multi-level retrieval graph is constructed based on the vehicle feature values ​​of the vehicle image by using a nearest neighbor algorithm; The second determination module is used to determine the passing vehicle image corresponding to each target vehicle characteristic value as the target vehicle image.

[0082] The vehicle image retrieval device based on a multi-layer graph structure of the present application uses a nearest neighbor algorithm to construct a multi-level retrieval graph based on the vehicle feature values ​​of the passing vehicle images, so that after obtaining the vehicle image to be retrieved and determining the vehicle feature values ​​of the vehicle image to be retrieved, a search is performed in the multi-level retrieval graph based on the vehicle feature values ​​of the vehicle image to be retrieved, and at least one target vehicle feature value can be obtained. Then, the passing vehicle images corresponding to each target vehicle feature value can be used as the target vehicle images. Since there is no need to compare all data during retrieval, only the neighboring node data needs to be compared, which greatly improves the vehicle retrieval efficiency.

[0083] In one embodiment, the first determining module is specifically configured to: Inputting the vehicle image to be retrieved into a vehicle feature extraction model to obtain a floating-point vehicle feature value output by the vehicle feature extraction model; the vehicle feature extraction model is used to extract the vehicle feature value based on the input image; The floating-point vehicle feature value is mapped to a signed integer data type to obtain the vehicle feature value of the vehicle image to be retrieved.

[0084] In one embodiment, the retrieval module is specifically used to: Taking the vehicle feature value of the vehicle image to be retrieved as a query point; In the first level of the multi-level retrieval graph, vehicle feature values ​​are randomly selected as candidate nodes and added to the candidate set; Traversing the neighbor result set of each candidate node in the candidate set; In the case where the search result set is not full, each node in each of the neighbor result sets is added to the search result set; the search result set is initially empty and the maximum number is a third preset number; When the search result set is full, the target node in each of the neighbor result sets is added to the search result set; the distance between the target node and the query point is less than the maximum distance between the query point and each node in the search result set; Eliminate the node with the largest distance from the query point among the nodes in the search result set; If the current level is not the last level in the multi-level search graph, jump to the next level of the current level; Obtain a neighbor result set of each node in the search result set at the current level; Add the nodes in the neighbor result set of each node in the retrieval result set that have not been traversed in the neighbor result set of the current level to the candidate set, and execute the step of traversing the neighbor result set of each candidate node in the candidate set until the level after execution is the last level in the multi-level retrieval graph, and determine each vehicle feature value in the retrieval result set as the target vehicle feature value.

[0085] In one embodiment, the second determining module is further configured to: Dequantize each target vehicle characteristic value to obtain the corresponding target floating-point vehicle characteristic value; The similarities between the floating-point vehicle feature value of the to-be-retrieved vehicle image and each target floating-point vehicle feature value are respectively determined.

[0086] Figure 2 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 2 As shown, the electronic device may include: a processor 210, a communications interface 220, a memory 230 and a communication bus 240, wherein the processor 210, the communications interface 220 and the memory 230 communicate with each other via the communication bus 240. The processor 210 may call the logic instructions in the memory 230 to execute the following method: obtaining a vehicle image to be retrieved; Determining a vehicle feature value of the vehicle image to be retrieved; Based on the vehicle feature value of the vehicle image to be retrieved, a search is performed in a multi-level search graph to obtain at least one target vehicle feature value; the multi-level search graph is constructed based on the vehicle feature value of the vehicle image by using a nearest neighbor algorithm; The passing vehicle image corresponding to each target vehicle feature value is determined as the target vehicle image.

[0087] In addition, the logic instructions in the above-mentioned memory 230 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0088] In another aspect, an embodiment of the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method provided by the above embodiments is implemented, for example, including: obtaining a vehicle image to be retrieved; Determining a vehicle feature value of the vehicle image to be retrieved; Based on the vehicle feature value of the vehicle image to be retrieved, a search is performed in a multi-level search graph to obtain at least one target vehicle feature value; the multi-level search graph is constructed based on the vehicle feature value of the vehicle image by using a nearest neighbor algorithm; The passing vehicle image corresponding to each target vehicle feature value is determined as the target vehicle image.

[0089] In another aspect, an embodiment of the present application further provides a computer program product having a computer program stored thereon, and when the computer program is executed by a processor, the method provided by the above embodiments is implemented, for example, including: obtaining a vehicle image to be retrieved; Determining a vehicle feature value of the vehicle image to be retrieved; Based on the vehicle feature value of the vehicle image to be retrieved, a search is performed in a multi-level search graph to obtain at least one target vehicle feature value; the multi-level search graph is constructed based on the vehicle feature value of the vehicle image by using a nearest neighbor algorithm; The passing vehicle image corresponding to each target vehicle feature value is determined as the target vehicle image.

[0090] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0091] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0092] Finally, it should be noted that the above implementation modes are only used to illustrate the present application, rather than to limit the present application. Although the present application is described in detail with reference to the embodiments, a person skilled in the art should understand that various combinations, modifications or equivalent substitutions of the technical solutions of the present application do not depart from the spirit and scope of the technical solutions of the present application.

Claims

1. A vehicle image retrieval method based on a multi-layer graph structure, characterized in that: include: Obtaining the vehicle image to be retrieved; Determining a vehicle feature value of the vehicle image to be retrieved; Based on the vehicle feature value of the vehicle image to be retrieved, searching in a multi-level search graph to obtain at least one target vehicle feature value; The multi-level retrieval graph is constructed based on the vehicle feature values ​​of the vehicle passing image using a nearest neighbor algorithm; The passing vehicle image corresponding to each target vehicle feature value is determined as the target vehicle image.

2. The vehicle image retrieval method based on a multi-layer graph structure according to claim 1, characterized in that: The multi-level search graph is constructed in the following manner: Acquire multiple images of vehicles passing by; Determine the vehicle feature value of each vehicle passing image respectively; Start traversing from the first layer, randomly select a vehicle feature value from the vehicle feature values ​​of each vehicle passing image as a feature value node and write it into the comparison set; For each feature value node in the comparison set, determine the nearest neighbor result set from the vehicle feature values ​​of each passing vehicle image; Writing each of the neighbor result sets into the current level based on random decision making; If the current level number does not reach the preset level threshold, enter the next level of the current level, write each neighbor result set of the current level into the comparison set, return to execute the above method for each feature value node in the comparison set, and determine the neighbor result set from the vehicle feature values ​​of each passing vehicle image respectively; The step of writing each of the neighbor result sets into the current level based on random decision making until the current level number reaches the preset level threshold.

3. The vehicle image retrieval method based on a multi-layer graph structure according to claim 2, characterized in that: When determining the nearest neighbor result set from the vehicle feature values ​​of each passing vehicle image for each feature value node in the comparison set, the following steps are performed for each feature value node in the comparison set: Determine the similarity between the current feature value node and the vehicle feature value of each passing vehicle image; Adding vehicle feature values ​​of a first preset number of vehicle images having a similarity greater than a preset similarity threshold to a nearest neighbor node set; Determining vehicle feature values ​​of a second preset number of vehicle passing images from the nearest neighbor node set as target vehicle feature values; An edge relationship is constructed for each of the target vehicle feature values ​​to form a neighbor result set of the current feature value node.

4. The vehicle image retrieval method based on a multi-layer graph structure according to claim 1, characterized in that: The vehicle feature value based on the vehicle image to be retrieved is searched in a multi-level search graph to obtain at least target vehicle feature values, including: Taking the vehicle feature value of the vehicle image to be retrieved as a query point; In the first level of the multi-level retrieval graph, vehicle feature values ​​are randomly selected as candidate nodes and added to the candidate set; Traversing the neighbor result set of each candidate node in the candidate set; In the case where the search result set is not full, each node in each of the neighbor result sets is added to the search result set; the search result set is initially empty and the maximum number is a third preset number; When the search result set is full, the target node in each of the neighbor result sets is added to the search result set; the distance between the target node and the query point is less than the maximum distance between the query point and each node in the search result set; Eliminate the node with the largest distance from the query point among the nodes in the search result set; If the current level is not the last level in the multi-level search graph, jump to the next level of the current level; Obtain a neighbor result set of each node in the search result set at the current level; Add the nodes in the neighbor result set of each node in the retrieval result set that have not been traversed in the neighbor result set of the current level to the candidate set, and execute the step of traversing the neighbor result set of each candidate node in the candidate set until the level after execution is the last level in the multi-level retrieval graph, and determine each vehicle feature value in the retrieval result set as the target vehicle feature value.

5. The vehicle image retrieval method based on a multi-layer graph structure according to claim 1, characterized in that: The determining of the vehicle feature value of the vehicle image to be retrieved includes: Inputting the vehicle image to be retrieved into a vehicle feature extraction model to obtain a floating-point vehicle feature value output by the vehicle feature extraction model; the vehicle feature extraction model is used to extract the vehicle feature value based on the input image; The floating-point vehicle feature value is mapped to a signed integer data type to obtain the vehicle feature value of the vehicle image to be retrieved.

6. The vehicle image retrieval method based on a multi-layer graph structure according to claim 5, characterized in that: After determining the passing vehicle image corresponding to each target vehicle feature value as the target vehicle image, the method further includes: Dequantize each target vehicle characteristic value to obtain the corresponding target floating-point vehicle characteristic value; The similarities between the floating-point vehicle feature value of the to-be-retrieved vehicle image and each target floating-point vehicle feature value are respectively determined.

7. A vehicle image retrieval device based on a multi-layer graph structure, characterized in that: include: An acquisition module, used for acquiring a vehicle image to be retrieved; A first determining module, used to determine the vehicle feature value of the vehicle image to be retrieved; A retrieval module, configured to search in a multi-level retrieval graph based on the vehicle feature value of the vehicle image to be retrieved, and obtain at least one target vehicle feature value; The multi-level retrieval graph is constructed based on the vehicle feature values ​​of the vehicle passing image using a nearest neighbor algorithm; The second determination module is used to determine the passing vehicle image corresponding to each target vehicle characteristic value as the target vehicle image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the vehicle image retrieval method based on a multi-layer graph structure as described in any one of claims 1 to 6 is implemented.

9. A storage medium, the storage medium being a non-transitory computer-readable storage medium, on which a computer program is stored, characterized in that: When the computer program is executed by a processor, the vehicle image retrieval method based on a multi-layer graph structure as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the vehicle image retrieval method based on a multi-layer graph structure described in any one of claims 1 to 6 is implemented.

Images