A vehicle image retrieval model training method, device, equipment and medium

Abstract

The application provides a vehicle image retrieval model training method, device and equipment and a medium. The training method comprises: acquiring image data sets of vehicles; performing normalization processing on the image data sets to generate training data sets; constructing an initial vehicle image retrieval model based on a local attention mechanism, and configuring an optimizer and a loss function of the initial vehicle image retrieval model; taking the training data sets as input variables of the initial vehicle image retrieval model, training and optimizing the initial vehicle image retrieval model, and acquiring a target vehicle image retrieval model. The application can meet the data retrieval of large-scale vehicle images and ensure high retrieval efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a vehicle image retrieval model training method and device, equipment and medium. BACKGROUND

[0002] With the continuous expansion of highway freight transportation scale, the image retrieval application of vehicles gradually expands from conventional image retrieval to image retrieval in a big data scenario. With the gradual increase of the image retrieval library, the existing technology has the problems of slow retrieval speed and low retrieval efficiency for large-scale vehicle image retrieval. Therefore, there is room for improvement. SUMMARY

[0003] In view of the above shortcomings of the prior art, the present application provides a vehicle image retrieval model training method, device, equipment and medium to solve the above technical problems.

[0004] The present application provides a lane line recognition model training method, comprising:

[0005] obtaining an image data set of a vehicle;

[0006] normalizing the image data set to generate a training data set;

[0007] constructing an initial vehicle image retrieval model based on a local attention mechanism, and configuring an optimizer and a loss function of the initial vehicle image retrieval model; and

[0008] training and optimizing the initial vehicle image retrieval model by taking the training data set as an input variable of the initial vehicle image retrieval model, to obtain a target vehicle image retrieval model.

[0009] In an embodiment of the present application, the step of obtaining an image data set of a vehicle comprises:

[0010] obtaining vehicle images of multiple scenes;

[0011] respectively performing label annotation processing on the vehicle images to generate corresponding image data sets, wherein the image data sets comprise multiple image data; and

[0012] dividing all the image data into a training image set, a verification image set and a test image set according to a preset ratio.

[0013] In an embodiment of the present application, the step of configuring the optimizer and the loss function of the initial vehicle image retrieval model comprises:

[0014] setting the optimizer of the initial vehicle image retrieval model as a stochastic gradient descent optimizer; and

[0015] The loss function of the initial vehicle image retrieval model is calculated according to a cross-entropy classification loss function.

[0016] In an embodiment of the present application, the loss function Loss of the initial vehicle image retrieval model is calculated according to a cross-entropy classification loss function. final The following formula can be satisfied:

[0017]

[0018] wherein k represents a local feature vector, J(V I ) represents a global feature vector, represents a local feature weight coefficient generated based on the local feature vector and the global feature vector, M(R) represents a feature vector after local region pooling, Ω represents all local regions after multi-scale sampling, and R represents a certain region in the local regions.

[0019] In an embodiment of the present application, the step of training and optimizing the initial vehicle image retrieval model by taking the training data set as an input variable of the initial vehicle image retrieval model comprises:

[0020] performing feature extraction processing on the training data set to generate a local feature vector of each local image;

[0021] performing hash processing on the local feature vector to generate a hash code of each local image; and

[0022] generating a hash code of an overall image based on the hash code of each local image.

[0023] In an embodiment of the present application, the step of performing feature extraction processing on the training data set to generate a local feature vector of each local image comprises:

[0024] performing overall feature extraction processing on the training data set to generate an overall feature of the vehicle image;

[0025] performing multi-scale sampling and same-scale transformation processing on the vehicle image to generate multi-scale features; and

[0026] performing maximum pooling processing and post-processing on the multi-scale features to generate a feature vector corresponding to each local image.

[0027] In an embodiment of the present application, the step of generating a hash code of an overall image based on the hash code of each local image comprises:

[0028] calculating a weight coefficient of each local image; and

[0029] generate a hash code of the overall image based on the weight coefficient and the hash code of the local image.

[0030] The application further provides a training device of a vehicle image retrieval model, comprising:

[0031] An image acquisition module is configured to acquire an image dataset of a vehicle, wherein the image dataset comprises a training image set, a verification image set and a test image set;

[0032] A data processing module is configured to perform normalization processing on the image dataset to generate a training dataset;

[0033] A model construction module is configured to construct an initial vehicle image retrieval model based on a local attention mechanism, and configure an optimizer and a loss function of the initial vehicle image retrieval model; and

[0034] A model training module is configured to take the training dataset as an input variable of the initial vehicle image retrieval model, train and optimize the initial vehicle image retrieval model, and obtain a target vehicle image retrieval model.

[0035] The application further provides a computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the training method of the vehicle image retrieval model according to any one of the above.

[0036] The application further provides a computer readable storage medium storing a computer program, wherein the computer program is executable on a processor to implement the steps of the training method of the vehicle image retrieval model according to any one of the above.

[0037] In summary, the training method, device, equipment and medium of the vehicle image retrieval model have the following beneficial effects: the application can retrieve similar vehicle pictures by analyzing and comparing the retrieval vehicle pictures and the pictures in the vehicle gallery, thereby achieving the retrieval of vehicle images, and the application can guarantee high retrieval efficiency.

[0038] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application. It is to be understood that the drawings are designed solely for purposes of illustration to be used in conjunction with the description in

[0040] Figure 1 A flowchart showing a training method of a vehicle image retrieval model provided by the present application.

[0041] Figure 2 A flowchart showing a specific embodiment of step S100 in the method. Figure 1

[0042] A flowchart showing a specific embodiment of step S300 in the method. Figure 3 Figure 1 A flowchart showing a specific embodiment of step S400 in the method.

[0043] Figure 4 Figure 1 A flowchart showing a specific embodiment of step S410 in the method.

[0044] Figure 5 A flowchart showing a specific embodiment of step S430 in the method. Figure 4

[0045] A structural diagram showing multi-scale sampling in an embodiment of the present application. Figure 6

[0046] A structural diagram showing local feature hashing in an embodiment of the present application. Figure 7

[0047] A flowchart showing a specific embodiment of step S430 in the method. Figure 8 Figure 4 A schematic diagram of a training device of a vehicle image retrieval model provided by the present application.

[0048] Figure 9 A schematic diagram of a computer device provided by the present application.

[0049] Figure 10 A schematic diagram of another computer device provided by the present application.

[0050] DETAILED DESCRIPTION Figure 11

[0051] ​​​​Other advantages and novel features of the present application will become apparent from the following detailed description of the application when considered in conjunction with the drawings. The application can be put into practice by different embodiments, and the details of the application can be modified in different ways without departing from the spirit and scope of the application. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application, as claimed.

[0052] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings:

[0053] In the following description, numerous specific details are discussed in order to provide a thorough understanding of the embodiments of the application. It will be apparent, however, to one of ordinary skill in the art that the embodiments of the application can be practiced without these specific details. In other instances, well-known structures and devices are not described in exhaustive detail in order to avoid obscuring the embodiments of the application.

[0054] It should be noted that the "searching image by image" mainly uses the content-based image retrieval technology, and the image feature extraction is the key of CBIR (Content-based image retrieval). In order to better express the rich semantic information of the image, the global high-dimensional features are generally used to represent, such as GIST, SIFT feature or the fusion of multiple features. If the sequential search is performed in the content-based image retrieval method, there is a problem of low efficiency when the data volume is very large. The solution is to use an index structure to improve the retrieval efficiency, such as K-D tree, B tree, R tree and other tree structures. However, the retrieval efficiency of these tree index structures will decrease rapidly with the increase of feature dimension, and the retrieval efficiency is even lower than the linear retrieval. The present application introduces a semantic hash related algorithm, that is, a hash-based image retrieval. Based on the approximate concept, more attention is paid to the efficiency of retrieval in large-scale data retrieval, and the accuracy of retrieval is not required too high, so as to realize the demand of most users.

[0055] Referring to Figure 1 as shown, Figure 1A flowchart of a training method of a vehicle image retrieval model provided by the present application is shown. The present application provides a training method of a vehicle image retrieval model, which can retrieve similar vehicle pictures by analyzing and comparing the retrieval vehicle pictures and the pictures in the vehicle picture library in the video monitoring vehicle image library, so as to realize the retrieval of vehicle images. The present application can meet large-scale data retrieval and can guarantee high retrieval efficiency. The training method can include the following steps:

[0056] Step S100, acquiring image data set of vehicle;

[0057] Step S200, normalizing the image data set to generate training data set;

[0058] Step S300, constructing an initial vehicle image retrieval model based on a local attention mechanism, and configuring an optimizer and a loss function of the initial vehicle image retrieval model;

[0059] Step S400, taking the training data set as an input variable of the initial vehicle image retrieval model, training and optimizing the initial vehicle image retrieval model, and acquiring a target vehicle image retrieval model.

[0060] Please refer to Figure 2 shown, Figure 2 shown as Figure 1 a specific embodiment of step S100. In an embodiment of the present application, when step S100 is performed, steps S110 to S130 can be included, which are described in detail as follows:

[0061] Step S110, acquiring vehicle images of multiple scenes;

[0062] Step S120, respectively performing label annotation processing on the vehicle images to generate corresponding image data sets, wherein the image data sets include multiple image data;

[0063] Step S130, dividing all the image data into training image set, verification image set and test image set according to a preset proportion.

[0064] In an embodiment of the present application, when steps S110 to S130 are performed. Specifically, first, a plurality of intelligent traffic camera devices can be used to collect vehicle images, for example, through cameras installed on road bridges, traffic lights or mobile monitoring vehicles, to respectively take pictures of passing vehicles, thereby generating a plurality of vehicle images of multiple scenes. Then, a picture labeling tool can be used to label the acquired vehicle images of multiple scenes to generate corresponding image data. In this embodiment, the picture labeling tool can use the open source image labeling tool Labelme. The image data can include vehicle images and corresponding label images. The plurality of image data can be integrated to generate an image data set.

[0065] Further, since the number of image data is multiple, it needs to be allocated, and divided into a training image set, a verification image set and a test image set according to a preset ratio. For example, the preset ratio can be 4:1:1, i.e. the ratio of the number of image data in the training image set, the number of image data in the verification image set and the number of image data in the test image set can be 4:1:1. The specific size of the preset ratio can not be limited, as long as it can meet the training of the vehicle image retrieval model. Among them, the m vehicle images in the training image set can be represented as T = {T1, T2, …, Tm}, the corresponding m label images can be represented as TL = {TL1, TL2, …, TLm}, where T represents the i-th vehicle image in the training image set, TL represents the label image of the i-th vehicle image in the training image set, i < m. The n vehicle images in the verification image set can be represented as V = {V1, V2, …, Vn}, the corresponding n label images can be represented as VL = {VL1, VL2, …, VLn}, where V represents the i-th vehicle image in the verification image set, VL represents the label image of the i-th vehicle image in the verification image set, i < n. The test image set can include a plurality of vehicle images for testing. i m i m i i i n i n i i

[0066] ​​​​​​​​​​​​In an embodiment of the present application, when step S200 is performed, the image data set is normalized to generate a training data set. Specifically, normalization processing is performed on all image data in the image data set, so that each vehicle image and its corresponding label image is unified to a preset size. In this embodiment, the preset size can be set to 800x800, which is conducive to the convergence of the model.

[0067] In an embodiment of the present application, when step S300 is performed, an initial vehicle image retrieval model is constructed based on a local attention mechanism, and an optimizer and a loss function of the initial vehicle image retrieval model are configured. Specifically, based on the local attention mechanism, an encoder-decoder of the initial vehicle image retrieval model is constructed. Using a local attention mechanism with a fixed window size, the model retrieval efficiency can be improved while meeting the accuracy requirement.

[0068] Referring to Figure 3 , Figure 3 is shown as Figure 1 a flowchart of a specific implementation of step S300. In an embodiment of the present application, when step S300 is performed, steps S310 to S320 can be included, which are described in detail as follows:

[0069] Step S310, setting the optimizer of the initial vehicle image retrieval model as a stochastic gradient descent optimizer;

[0070] Step S320, calculating the loss function of the initial vehicle image retrieval model according to a cross-entropy classification loss function.

[0071] In an embodiment of the present application, when steps S310 to S320 are performed. Specifically, the optimizer of the initial vehicle image retrieval model can be set as a stochastic gradient descent (SGD) optimizer, so as to control the update step, iteratively update the network parameters, make the network prediction result closer to the true value, and achieve the effect of training the network. At the same time, the final loss function can be calculated according to the cross-entropy classification loss function. In this embodiment, the loss function can be obtained according to the weight coefficient of the local region generated by the local attention module. The loss function Loss final can satisfy the following formula:

[0072]

[0073] wherein k represents a local feature vector, J(V I ) represents a global feature vector, represents a local feature weight coefficient generated based on the local feature vector and the global feature vector, M(R) represents a feature vector after local region pooling, and represents all local regions after multi-scale sampling, and R represents a certain region in all local regions.

[0074] Further, after the optimizer and loss function for setting the initial vehicle image retrieval model are completed, the constructed initial vehicle image retrieval model can be pre-trained using a preset data set to improve the model convergence speed. In the embodiment, the preset data set can adopt the ImageNet data set.

[0075] In an embodiment of the present application, when step S400 is performed, the training data set is used as an input variable of the initial vehicle image retrieval model to train and optimize the initial vehicle image retrieval model to obtain the target vehicle image retrieval model. Specifically, it is first noted that a model training strategy needs to be formulated before the model training is performed. In an embodiment of the present application, the model training strategy can be formulated according to a preset proportion of 4:1 of the training image set and the verification image set. Specifically, it can be set that starting from the 100th epoch, the model precision is calculated on the training image set after every 10 epochs, and the model precision is retained. Wherein, the epoch represents a process of training once using all samples in the training image set. Then, the model precision is calculated every 10 epochs, and compared with the previous model precision. If the precision of the latter model exceeds that of the previous model, the previous model is replaced, otherwise the previous model is retained. The training data set is used as an input variable of the initial vehicle image retrieval model, and the initial vehicle image retrieval model is trained and optimized according to the above model training strategy, so that the model is suitable for processing of vehicle images. In this embodiment, the model is trained based on the training image set, and the model precision is verified based on the verification image set, the optimal weight of the model is saved, and the precision of the training image set and the verification set is recorded, so as to facilitate parameter adjustment, and finally the target vehicle image retrieval model meeting the preset precision requirement is obtained.

[0076] Referring to Figure 4 as shown in Figure 4 is shown as Figure 1 a flowchart of a specific embodiment of step S400. In an embodiment of the present application, when step S400 is performed, step S400 can include steps S410 to S430, which are described in detail as follows:

[0077] Step S410, feature extraction processing is performed on the training data set to generate a local feature vector of each local image;

[0078] Step S420, the local feature vector is subjected to hash processing to generate a hash code of each local image;

[0079] Step S430, based on the hash code of each local image, a hash code of the whole image is generated.

[0080] Referring toFigure 5 as shown, Figure 5 as shown Figure 4 A flowchart of an embodiment of step S410 is shown. In an embodiment of the present application, when step S410 is performed, steps S411-S413 can be included and are described in detail as follows:

[0081] Step S411, overall feature extraction processing is performed on the training data set to generate overall features of the vehicle image.

[0082] Step S412, multi-scale sampling and same-scale transformation processing is performed on the vehicle image to generate multi-scale features.

[0083] Step S413, maximum pooling processing and post-processing are performed on the multi-scale features to generate a feature vector corresponding to each local image.

[0084] In an embodiment of the present application, when step S411 is performed, overall feature extraction processing is performed on the training data set to generate overall features of the vehicle image. Specifically, the deep convolutional network can be used to perform overall feature extraction processing on the training data set to extract overall features of each vehicle image.

[0085] Referring to Figure 6 as shown, Figure 6 A structure diagram of multi-scale sampling in an embodiment of the present application is shown. In an embodiment of the present application, when step S412 is performed, multi-scale sampling and same-scale transformation processing is performed on the vehicle image to generate multi-scale features. Specifically, based on the initial vehicle image retrieval model, the R-MAC (Regional Maximum Activation of Convolutions) is used to perform multi-scale sampling on the vehicle image 601 to extract multi-scale features 602, and same-scale transformation is performed to map to the deep features extracted by the convolutional neural network. In this embodiment, the R-MAC can be used to perform multi-scale sampling for 55 times. The sampling overlap rate can be set to 0.4, and the sampling area size can satisfy the following formula:

[0086] R s = 2min(W, H) / (s + 1)

[0087] where W and H represent the width and height of the feature map, respectively, and s represents the scale ratio of the sampling.

[0088] Furthermore, the coordinates sampled by R-MAC are transformed at the same scale, and the features of the sampled local region can be obtained from the transformed coordinates. In this embodiment, the same-scale transformation coefficient can be set to 0.03125. The global feature size extracted by the convolutional neural network can be set to 25×25. Scale normalization is performed on the above image dataset, and the size of the image dataset can then satisfy the following formula:

[0089] Ω=C s (V I )

[0090] Among them, V I C represents the convolutional feature map of a vehicle image sample. s It is a local region sampler that can obtain multi-scale features of vehicle image samples after performing same-scale transformation. Feature maps at the same scale can have different receptive fields. The fused feature map 603 after superposition has the features of multiple receptive fields and is more representative.

[0091] Please see Figure 7 As shown, Figure 7 This is a schematic diagram of the structure of local feature hashing in one embodiment of the present invention. In one embodiment of the present invention, when step S413 is executed, max pooling and post-processing are performed on the multi-scale features to generate feature vectors corresponding to each local image. Specifically, pooling and post-processing are performed on the multi-scale features through pooling layer 701. In this embodiment, after obtaining the multi-scale features of the vehicle image samples, the multi-scale sampled features are first pooled to make the local feature sizes of different scales consistent, so as to facilitate subsequent processing. In this embodiment, max pooling is used for data processing.

[0092] Furthermore, after max pooling, the data needs to be post-processed to facilitate the subsequent generation of the corresponding hash code. In this post-processing, the local feature vector R generated after post-processing of local features is... I The following formula can be satisfied:

[0093] R I =P(M(R))

[0094] Here, M(R) represents the max pooling process applied to the local features, and P(M(R)) represents the post-processing of the data after the max pooling process. The purpose of the post-processing is to obtain the local feature vector.

[0095] Please see Figure 7As shown, in one embodiment of the present invention, when step S420 is executed, the local feature vectors are hashed to generate hash codes for each local image. Specifically, after obtaining the local feature vectors through post-processing, the hash layer 702 is used to hash the local feature vectors to obtain the hash code corresponding to the local feature. The hashing of the local feature vectors can satisfy the following formula:

[0096] R H =H(R) I )

[0097] Among them, H(R) I This indicates that the local feature vector generated after post-processing will be hashed through hash layer 702.

[0098] Please see Figure 8 As shown, Figure 8 Displayed as Figure 4 A flowchart illustrating a specific implementation of step S430 is shown below. In one embodiment of the present invention, when step S430 is executed, it may include steps S431 to S432, as detailed below:

[0099] Step S431: Calculate the weight coefficients of each local image;

[0100] Step S432: Generate the hash code of the overall image based on the weight coefficients and the hash code of the local image.

[0101] In one embodiment of the present invention, when steps S413 to S432 are executed, the weight coefficients of each local image can be calculated using the context-aware module, and the hash code of the overall image can be obtained using the weight coefficients and the hash codes of the local images. Specifically, the context-aware module estimates and analyzes each local region to obtain the weight coefficients of each local region, uses the weight coefficients and the hash codes of the local images to obtain the hash code of the overall image, and finally uses the hash code for accurate image retrieval.

[0102] In one embodiment of the present invention, when step S431 is executed, the weight coefficients of each local image are calculated. Specifically, the context-aware module estimates and analyzes each local region to obtain the weight coefficients of each local region. First, a context-aware global feature vector can be calculated, and simultaneously, the region attention weight coefficients are calculated conditionally, wherein the region attention weight coefficients can satisfy the following formula:

[0103]

[0104] Where k represents the local feature vector, J(V) I) represents a global feature vector, and in order to conditionally consider the regional attention weight, the local feature and the global feature are jointly considered as the input of Φ().

[0105] In an embodiment of the present application, when step S432 is performed, that is, the hash code of the overall image is generated based on the weight coefficient and the hash code of the local image. Specifically, the hash code of the overall image is generated based on the hash code of each local image using the context-aware module, that is, the globally context-aware hash code is obtained, and the hash code of the overall image can satisfy the following formula:

[0106]

[0107] wherein, represents the regional attention weight coefficient generated based on the local feature and the global feature, H(P(M(R))) represents the hash code generated based on the local feature, Ω represents all the local regions after multi-scale sampling of the R-MAC, and |Ω| represents the number of multi-scale sampled regions.

[0108] Further, after generating the hash code of the overall image, retrieval can be performed based on the vehicle target retrieval library. First, the generated hash code is compared with the hash code of the vehicle image in the vehicle target retrieval library; then, the relevant vehicle pictures are arranged according to the Hamming distance sorting.

[0109] Please refer to Figure 9 The present application also provides a training device of a vehicle image retrieval model, which corresponds to the training method in the above embodiment. The training device can include an image acquisition module 901, a data processing module 902, a model construction module 903, and a model training module 904. The functions of each module are described in detail as follows:

[0110] The image acquisition module 901 can be used to acquire the image data set of the vehicle, wherein the image data set includes a training image set, a verification image set, and a test image set. Further, the image acquisition module 901 can be specifically used to acquire vehicle images of multiple scenes; label mark the vehicle images respectively to generate corresponding image data sets, wherein the image data set includes multiple image data; and divide all image data into a training image set, a verification image set, and a test image set according to a preset ratio.

[0111] The data processing module 902 can be used to normalize the image data set to generate a training data set. Specifically, the data processing module 902 can normalize all image data in the image data set, so that each vehicle image and its corresponding label image are unified to a preset size.

[0112] The model construction module 903 can be configured to construct an initial vehicle image retrieval model based on a local attention mechanism, and configure an optimizer and a loss function of the initial vehicle image retrieval model. Further, the model construction module 903 can be specifically configured to set the optimizer of the initial vehicle image retrieval model as a stochastic gradient descent optimizer, and calculate the loss function of the initial vehicle image retrieval model according to a cross-entropy classification loss function.

[0113] The model training module 904 can be configured to take a training data set as an input variable of the initial vehicle image retrieval model, train and optimize the initial vehicle image retrieval model, and obtain a target vehicle image retrieval model. Further, the model training module 904 can be specifically configured to perform feature extraction processing on the training data set to generate a local feature vector of each local image, perform hashing processing on the local feature vector to generate a hash code of each local image, and generate a hash code of an overall image based on the hash code of each local image.

[0114] The specific limitations of the training device can be referred to the limitations of the training method described above, which will not be repeated here. Each module in the above training device can be realized by software, hardware and their combinations in whole or in part. Each module described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to call and execute the operations corresponding to each module by the processor.

[0115] Please refer to Figure 10 As shown in the figure, the present application also provides a computer device, which can be a server. The computer device includes a processor, a memory, a network interface and a database connected by a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external client through the network connection. The computer program is executed by the processor to realize the functions or steps of the vehicle image retrieval model training method.

[0116] Please refer to Figure 11As shown, the present application also provides another computer device, which can be a client. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is used to communicate with the external server through the network connection. The computer program is executed by the processor to realize the functions or steps of the vehicle image retrieval model training method.

[0117] In an embodiment of the present application, a computer device is provided, including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to realize the following steps:

[0118] Obtaining an image data set of a vehicle, wherein the image data set includes a training image set, a verification image set and a test image set;

[0119] Normalizing the image data set to generate a training data set;

[0120] Constructing an initial vehicle image retrieval model based on a local attention mechanism, and configuring an optimizer and a loss function of the initial vehicle image retrieval model;

[0121] Taking the training data set as an input variable of the initial vehicle image retrieval model, training and optimizing the initial vehicle image retrieval model to obtain a target vehicle image retrieval model.

[0122] In an embodiment of the present application, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to realize the following steps:

[0123] Obtaining an image data set of a vehicle, wherein the image data set includes a training image set, a verification image set and a test image set;

[0124] Normalizing the image data set to generate a training data set;

[0125] Constructing an initial vehicle image retrieval model based on a local attention mechanism, and configuring an optimizer and a loss function of the initial vehicle image retrieval model;

[0126] Taking the training data set as an input variable of the initial vehicle image retrieval model, training and optimizing the initial vehicle image retrieval model to obtain a target vehicle image retrieval model.

[0127] It should be noted that the functions or steps that can be implemented by the computer readable storage medium or the computer device described above can be referred to the foregoing method embodiments, and to avoid repetition, they will not be described here.

[0128] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of the method. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0129] In summary, the present application provides a vehicle image retrieval model training method, device, equipment and medium, which can be applied to the field of image processing. The present application uses semantic hashing, feature fusion, attention mechanism and other technical means, which can be applied to vehicle image retrieval, and improves the retrieval speed and accuracy of vehicle images, solves the problem that the conventional vehicle image retrieval algorithm has poor effect when applied to large-scale vehicle image library. The present application is based on a convolutional neural network based on deep learning, and automatically samples different regions of different scales of images according to the input of multiple scene vehicle images; then the context perception module is used to realize the fusion of features of different scale regions, and more representative vehicle image features are obtained. The present application uses cross-entropy classification loss function to make the training effect of the model more stable and better. At the same time, the query expansion technology is used to realize the enhancement operation during query, so as to carry out more accurate retrieval and achieve higher retrieval accuracy.

[0130] In the description of the specification, the description with reference to the terms "the embodiment", "an example", "a specific example" and the like means that the specific feature, structure, material or the like described in connection with the embodiment or example is included in at least one embodiment or example of the present application. Illustrative expressions of the above terms in the specification do not necessarily refer to the same embodiment or example. Also, the specific feature, structure, material or the like described can be combined in an appropriate manner in any one or more embodiments or examples.

[0131] The embodiments of the application disclosed above are only used for helping to explain the present application. The embodiments do not describe all the details and limit the present application to the specific embodiments. Obviously, according to the content of the specification, many modifications and changes can be made. The specification selects and specifically describes these embodiments in order to better explain the principles and practical application of the present application, so that the persons skilled in the art can well understand and utilize the present application. The present application is limited by the claims and the entire scope and equivalents thereof.

Claims

1. A training method for a vehicle image retrieval model, characterized in that, The method comprises the following steps: obtaining an image dataset of a vehicle; normalizing the image dataset to generate a training dataset; constructing an initial vehicle image retrieval model based on a local attention mechanism, and configuring an optimizer and a loss function of the initial vehicle image retrieval model; and training and optimizing the initial vehicle image retrieval model by taking the training dataset as an input variable of the initial vehicle image retrieval model to obtain a target vehicle image retrieval model; wherein the step of configuring the optimizer and the loss function of the initial vehicle image retrieval model comprises: setting the optimizer of the initial vehicle image retrieval model as a stochastic gradient descent optimizer; and calculating the loss function of the initial vehicle image retrieval model according to a cross-entropy classification loss function; The loss function Loss of the initial vehicle image retrieval model final The following equation can be satisfied: where k represents a local feature vector, J(V I ) represents a global feature vector, represents a local feature weight coefficient generated based on the local feature vector and the global feature vector, M(R) represents a feature vector after local region pooling, Ω represents all local regions after multi-scale sampling, and R represents a certain region in the all local regions. the step of training and optimizing the initial vehicle image retrieval model by taking the training dataset as an input variable of the initial vehicle image retrieval model comprises: performing feature extraction processing on the training dataset to generate a local feature vector of each local image; performing hashing processing on the local feature vector to generate a hash code of each local image; and generating a hash code of an overall image based on the hash codes of the local images. 2.The method of claim 1, wherein, The step of obtaining the image dataset of the vehicle comprises: obtaining vehicle images of multiple scenes; performing label annotation processing on the vehicle images respectively to generate corresponding image datasets, wherein the image datasets comprise multiple image data; and dividing all the image data into a training image set, a verification image set and a test image set according to a preset ratio. 3.The method of claim 1, wherein, The step of performing feature extraction processing on the training dataset to generate a local feature vector of each local image comprises: performing overall feature extraction processing on the training dataset to generate an overall feature of the vehicle image; performing multi-scale sampling and same-scale transformation processing on the vehicle image to generate multi-scale features; and performing maximum pooling processing and post-processing on the multi-scale features to generate a feature vector corresponding to each local image. 4.The method of claim 1, wherein, The step of generating a hash code of an overall image based on hash codes of the local images comprises: calculating weight coefficients of the local images; and generating the hash code of the overall image based on the weight coefficients and the hash codes of the local images. 5.A device for training a vehicle image retrieval model, characterized in that, The method comprises the following steps: an image acquisition module is configured to obtain an image dataset of a vehicle, wherein the image dataset comprises a training image set, a verification image set and a test image set; a data processing module is configured to perform normalization processing on the image dataset to generate a training dataset; a model construction module is configured to construct an initial vehicle image retrieval model based on a local attention mechanism, and configure an optimizer and a loss function of the initial vehicle image retrieval model; and a model training module is configured to train and optimize the initial vehicle image retrieval model by taking the training dataset as an input variable of the initial vehicle image retrieval model to obtain a target vehicle image retrieval model. The model construction module is further configured to set an optimizer of the initial vehicle image retrieval model as a stochastic gradient descent optimizer; and calculate a loss function of the initial vehicle image retrieval model according to a cross-entropy classification loss function; The loss function Loss of the initial vehicle image retrieval model final The following equation can be satisfied: where k represents a local feature vector, J(V I ) represents a global feature vector, represents a local feature weight coefficient generated based on the local feature vector and the global feature vector, M(R) represents a feature vector after local region pooling, Ω represents all local regions after multi-scale sampling, and R represents a certain region in the all local regions. The model training module is further configured to perform feature extraction processing on the training data set to generate a local feature vector of each local image; perform hash processing on the local feature vector to generate a hash code of each local image; and generate a hash code of the whole image based on the hash code of each local image.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the vehicle image retrieval model training method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: receiving a request for a resource from a client; determining whether the client is authorized to access the resource; and if the client is authorized to access the resource, providing the resource to the client. The computer program is executed by the processor to implement the steps of the vehicle image retrieval model training method according to any one of claims 1 to 4.

Images