Vehicle matching positioning method and device, storage medium and electronic equipment

Abstract

The application discloses a vehicle matching positioning method and device, a storage medium and electronic equipment. The method comprises the following steps: performing first identification on a plurality of continuous images collected based on a predetermined time interval to obtain and intercept license plate images and vehicle images corresponding to the plurality of continuous images respectively; performing second identification on the license plate images of the plurality of continuous images to obtain license plate recognition results; in the case that the license plate recognition result corresponding to any one of the continuous images indicates that license plate recognition fails, performing third identification based on the vehicle images corresponding to the plurality of continuous images to determine the feature similarity between the vehicles included in two adjacent images in the plurality of continuous images; and determining the positioning result of a target vehicle in the plurality of continuous images based on the feature similarity between the vehicles included in the two adjacent images. The application solves the technical problem of abnormal vehicle matching positioning recognition and poor accuracy caused by the license plate recognition for vehicle matching positioning in the related art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of vehicle positioning and identification, in particular to a vehicle matching positioning method and device, a storage medium and an electronic device. BACKGROUND

[0002] When performing vehicle AI (Artificial Intelligence) abnormal driving behavior detection, a set of vehicle abnormal behavior information is often received, such as Figure 1 As shown in the figure, the target vehicle may have a large deformation in multiple consecutive images, resulting in poor license plate clarity and low recognition accuracy. There are certain difficulties in vehicle positioning, which leads to the inability to perform subsequent abnormal driving behavior judgment. It is difficult to identify the vehicle license plate using license plate recognition technology alone (such as a small vehicle target, vehicle license plate obstruction, turning, etc.). The vehicle re-identification algorithm is also difficult to effectively identify the target vehicle and the template vehicle picture with large changes.

[0003] At present, there is no effective solution to the above problems. SUMMARY

[0004] The embodiments of the present application provide a vehicle matching positioning method, device, storage medium and electronic device to at least solve the technical problem of abnormal vehicle matching positioning identification and poor accuracy caused by using license plate recognition for vehicle matching positioning in the related art.

[0005] According to an aspect of an embodiment of the present application, a vehicle matching positioning method is provided, comprising: performing first identification on multiple consecutive images collected based on a predetermined time interval to obtain and intercept license plate images including license plate information corresponding to the multiple consecutive images respectively, and vehicle images corresponding to the multiple consecutive images respectively, wherein a first image in the multiple consecutive images includes a target vehicle; performing second identification on the license plate images corresponding to the multiple consecutive images to obtain license plate recognition results corresponding to the multiple consecutive images respectively; in the case that the license plate recognition result corresponding to any one of the consecutive images indicates that license plate recognition fails, performing third identification based on the vehicle images corresponding to the multiple consecutive images to determine the feature similarity between the vehicles included in two adjacent images in the multiple consecutive images; and determining the positioning result of the target vehicle in the multiple consecutive images based on the feature similarity between the vehicles included in the two adjacent images.

[0006] According to another aspect of the embodiments of the present application, a vehicle matching positioning apparatus is also provided, which comprises: a first identification module configured to perform first identification on a plurality of continuous images acquired based on a predetermined time interval, to obtain and intercept license plate images including license plate information corresponding to the plurality of continuous images respectively, and vehicle images corresponding to the plurality of continuous images respectively, wherein the first image in the plurality of continuous images includes a target vehicle; a second identification module configured to perform second identification on the license plate images corresponding to the plurality of continuous images respectively, to obtain license plate recognition results corresponding to the plurality of continuous images respectively; a third identification module configured to, in a case that the license plate recognition result corresponding to any one of the continuous images in the plurality of continuous images indicates that license plate recognition fails, perform third identification based on the vehicle images corresponding to the plurality of continuous images, to determine a feature similarity between vehicles included in two adjacent images in the plurality of continuous images; and a determination module configured to determine a positioning result of the target vehicle in the plurality of continuous images based on the feature similarity between the vehicles included in the two adjacent images.

[0007] According to another aspect of the embodiments of the present application, a nonvolatile storage medium is also provided, which is characterized in that the nonvolatile storage medium stores a plurality of instructions, and the instructions are adapted to be loaded and executed by a processor to implement any one of the vehicle matching positioning methods.

[0008] According to another aspect of the embodiments of the present application, an electronic device is also provided, which is characterized in that the electronic device comprises one or more processors and a memory, and the memory is configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement any one of the vehicle matching positioning methods.

[0009] In the embodiment of the present application, by performing first identification on a plurality of continuous images collected based on a predetermined time interval, a plurality of license plate images respectively corresponding to the plurality of continuous images and including license plate information, and a plurality of vehicle images respectively corresponding to the plurality of continuous images are obtained and intercepted, wherein the first image in the plurality of continuous images includes a target vehicle; second identification is performed on the plurality of license plate images respectively corresponding to the plurality of continuous images to obtain license plate recognition results respectively corresponding to the plurality of continuous images; in the case that the license plate recognition result corresponding to any one of the plurality of continuous images indicates that license plate recognition fails, third identification is performed on the plurality of vehicle images respectively corresponding to the plurality of continuous images to determine the feature similarity between the vehicles included in two adjacent images in the plurality of continuous images; based on the feature similarity between the vehicles included in the two adjacent images, a positioning result of the target vehicle in the plurality of continuous images is determined, achieving comprehensive vehicle tracking and positioning through license plate recognition and vehicle recognition, i.e. in the case that vehicle tracking and positioning cannot be achieved through license plate recognition, vehicle tracking and positioning is achieved through vehicle feature matching, thereby achieving the technical effect of improving vehicle matching and positioning recognition accuracy, and further solving the technical problem of vehicle matching and positioning recognition abnormalities and poor accuracy caused by license plate recognition in related technologies. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings, which are included to provide a further understanding of the present application and are incorporated in and constitute a part of this application, illustrate embodiments of the present application and serve to explain the present application. In the drawings:

[0011] Figure 1 is a schematic diagram of a vehicle matching and positioning method according to the prior art;

[0012] Figure 2 is a schematic diagram of a vehicle matching and positioning method according to an embodiment of the present application;

[0013] Figure 3 is a schematic diagram of an improved Resnet50 residual network structure according to an embodiment of the present application;

[0014] Figure 4 is a schematic diagram of a target re-identification model structure according to an embodiment of the present application;

[0015] Figure 5 is a schematic diagram of a vehicle matching and positioning method according to an embodiment of the present application;

[0016] Figure 6 is a schematic diagram of a vehicle matching and positioning device according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should be within the scope of protection of the present application.

[0018] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0019] First, for the convenience of understanding the embodiments of the present application, the following will explain some terms or nouns involved in the present application:

[0020] Person re-identification (ReID), also known as person re-identification, is a technology that uses computer vision technology to determine whether a specific person exists in an image or video sequence.

[0021] LPRNET (License Plate Recognition) is a license plate recognition model with the following characteristics: 1) supports end-to-end training of indefinite length license plates; 2) does not use recurrent neural networks; 3) robust, can handle various special scenarios.

[0022] YoLOv5 is an object detection algorithm that redefines object detection as a regression problem. A single convolutional neural network is applied to the entire image, dividing the image into grids and predicting class probabilities and bounding boxes for each grid. YoLOv7 is a multi-object detection algorithm, which is an upgraded version of YoLOv5, and has better performance than YoLOv5.

[0023] OCR (Optical Character Recognition), for printed characters, uses optical methods to convert the text in paper documents into black and white bitmap image files, and through recognition software, the text in the image is converted into text format for further editing and processing by word processing software.

[0024] At present, the vehicle positioning method in the related art mainly includes two kinds, one is to use target detection technology (Yolov5) to identify the region of the license plate, to identify the license plate small graph using license plate recognition technology (such as LPRNET, OCR recognition technology) for license plate recognition, to identify all the license plates (or equally spaced license plates), and to perform vehicle matching positioning according to the read license plate. The other is to use target detection technology (Yolov5) to identify the vehicle region, to use a re-identification algorithm (ReID) to extract the vehicle features of the identified vehicle, and to compare the feature similarity of the remaining pictures or frames according to the vehicle features of the first picture. According to the threshold value of the similarity and the similarity ranking, the position of the vehicle in the video or picture is determined.

[0025] However, in actual application, for example, when performing vehicle AI abnormal driving behavior detection, a group of vehicle abnormal behavior information is often received, such as Figure 1 As shown in the figure, the target vehicle may have a large deformation in a plurality of continuous images, resulting in poor license plate clarity and low recognition accuracy, and there is a certain difficulty in vehicle positioning, which leads to the failure of subsequent abnormal driving behavior judgment, and it is difficult to identify the license plate of the vehicle (such as too small vehicle target, vehicle license plate shielding, turning, etc.) using only license plate recognition technology, and the vehicle re-identification algorithm is also difficult to effectively identify the target vehicle and the template vehicle picture with large changes.

[0026] According to the vehicle matching positioning method embodiment of the present application, it is necessary to point out that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0027] Figure 2 is a flowchart of the vehicle matching positioning method according to the embodiment of the present application, as shown in the figure, the method includes the following steps: Figure 2

[0028] ​Step S202, first identification is performed on the plurality of continuous images collected based on the predetermined time interval, to obtain and intercept the license plate image including the license plate information corresponding to each of the plurality of continuous images, and the vehicle image corresponding to each of the plurality of continuous images, wherein the first image in the plurality of continuous images includes the target vehicle.

[0029] Optionally, the first image in the plurality of continuous images includes at least the target vehicle, and can also include other vehicles in addition to the target vehicle; and the other images in the plurality of continuous images in addition to the first image include at least the target vehicle, and other vehicles in addition to the target vehicle. The vehicle features can include, but are not limited to, the overall vehicle features, the license plate frame, the wheel frame, the vehicle shape frame, etc. The first image can be understood as a close-up image of the target vehicle, and the license plate information of the target vehicle in the close-up image can be accurately identified.

[0030] In an optional embodiment, the first identification performed on the plurality of continuous images collected based on the predetermined time interval to obtain and intercept the license plate image including the license plate information corresponding to each of the plurality of continuous images, and the vehicle image corresponding to each of the plurality of continuous images: a YoLOv7 multi-target detection algorithm is used to perform the first identification on the plurality of continuous images to obtain and intercept the license plate image including the license plate information corresponding to each of the plurality of continuous images, and the vehicle image corresponding to each of the plurality of continuous images.

[0031] It should be noted that, considering that there are many small targets in vehicles, the YOLOv7 algorithm with better recognition effect for small targets is used, all the plurality of continuous images in which the target vehicle may exist abnormal behavior are loaded, and a pre-trained YOLOv7 model is used to identify the target vehicle and the corresponding license plate, to obtain the license plate image including the license plate information of the target vehicle, and the vehicle image including the vehicle features (such as overall vehicle features, license plate frame, wheel frame, vehicle shape frame, etc.) of the target vehicle.

[0032] Step S204, second identification is performed on the license plate image corresponding to each of the plurality of continuous images to obtain the license plate recognition result corresponding to each of the plurality of continuous images.

[0033] Optionally, a pre-trained LPRNET license plate recognition model is used to perform the second identification on the license plate image corresponding to each of the plurality of continuous images to obtain the license plate recognition result corresponding to each of the plurality of continuous images.

[0034] Step S206, in the case that the license plate recognition result corresponding to any one of the plurality of continuous images indicates that the license plate recognition fails, performing third recognition on the vehicle images corresponding to the plurality of continuous images respectively to determine the feature similarity between the vehicles included in the adjacent two images of the plurality of continuous images.

[0035] Optionally, in the case that the target vehicle has an abnormal driving behavior, a plurality of continuous images corresponding to the target vehicle are acquired, and in the case that the license plate positioning of the target vehicle cannot be achieved through the license plate recognition results corresponding to the plurality of continuous images respectively, the accurate positioning result of the target vehicle in the plurality of continuous images cannot be accurately recognized, and thus the accurate recognition of the abnormal driving behavior of the target vehicle cannot be achieved. Based on this, the positioning and recognition of the target vehicle are further performed based on the feature similarity between the vehicles included in the adjacent two images, so as to improve the positioning and matching accuracy of the target vehicle.

[0036] In an optional embodiment, the third recognition performed on the vehicle images corresponding to the plurality of continuous images respectively to determine the feature similarity between the vehicles included in the adjacent two images of the plurality of continuous images includes: based on the vehicle images corresponding to the plurality of continuous images respectively, a pre-trained target pedestrian re-identification model is used to determine the feature vectors of the vehicles included in the plurality of continuous images respectively; from the feature vectors of the vehicles included in the plurality of continuous images respectively, the feature vectors of the vehicles included in the adjacent two images respectively are determined; and according to the feature vectors of the vehicles included in the adjacent two images respectively, the feature similarity between the vehicles included in the adjacent two images is determined.

[0037] Optionally, the initial pedestrian re-identification model includes an improved ResNet deep residual network (i.e., ResNet network), which can be but is not limited to an improved Resnet50 residual network or a Resnet34 residual network.

[0038] Optionally, the feature similarity between the vehicles included in the adjacent two images is determined according to the feature vectors of the vehicles included in the adjacent two images respectively by the following manner:

[0039]

[0040] wherein A0=(a1, a2, a3, a4...a512) represents the feature vector of one vehicle included in the former one of the adjacent two images, B0=(b1, b2, b3, b4...b512) represents the feature vector of one vehicle included in the latter one of the adjacent two images, and Sim(A0, B0)=cos represents the feature similarity.

[0041] In an optional embodiment, the method further comprises: inputting the pre-acquired training image dataset into the pre-constructed initial pedestrian re-identification model for training to obtain a first pedestrian re-identification model, wherein the training image dataset includes multiple groups of images of multiple vehicles, the initial pedestrian re-identification model includes an improved ResNet network, the improved ResNet network is obtained by deleting the last fully connected layer in the original ResNet network and replacing it with a combination of a first fully connected layer and a second fully connected layer, the number of network layers of the first fully connected layer is less than the number of network layers corresponding to the second last network layer in the original ResNet network, and the output result of the second fully connected layer corresponds to the number of vehicle categories; and removing the second fully connected layer in the first pedestrian re-identification model to obtain the target pedestrian re-identification model.

[0042] Optionally, the improved ResNet network can be but is not limited to an improved Resnet50 residual network, a Resnet34 residual network, and the improved residual network is taken as an improved Resnet50 residual network as an example, wherein the improved Resnet50 residual network is obtained by modifying the last several layers of a traditional Resnet50 residual network, and the fully connected layer in the traditional Resnet50 residual network is (2048, num_class), wherein 2048 is the output dimension of the previous network layer of the current network layer, and num_class is used to indicate that the output result of the fully connected layer is the number of classified vehicle categories. The improved Resnet50 residual network modifies the fully connected layer in the traditional Resnet50 residual network, Figure 3 is a schematic diagram of an optional improved Resnet50 residual network structure according to an embodiment of the application, as shown in Figure 3 the fully connected layer (2048, num_class) in the traditional Resnet50 residual network is replaced by two network layers, i.e., (2048, 512) first and (512, num_class) second.

[0043] Optionally, the target pedestrian re-identification model is trained based on an improved Resnet50 residual network. The backbone feature extraction network is obtained by using the improved Resnet50 to perform feature extraction based on the training image data. The improved Resnet50 is expanded into a fully connected layer of (2048, 512), and then (512, num_class) is connected. A classification model (i.e., the first pedestrian re-identification model) is trained. When the first pedestrian re-identification model reaches a preset training condition, i.e., when the first pedestrian re-identification model has a good accuracy on the classification network, the last layer (512, num_class) of the first pedestrian re-identification model is removed, and the feature extraction feature layer of the model to (2048, 512) is retained as the actual use feature extraction network, to obtain the target pedestrian re-identification model, as shown in Figure 4 is an optional target re-identification model structure diagram according to an embodiment of the application, as shown in Figure 4 After a picture is input into the target pedestrian re-identification model, a 512-dimensional feature is output. The similarity is calculated based on the obtained feature.

[0044] In step S208, the positioning result of the target vehicle in the plurality of continuous images is determined based on the feature similarity between the vehicles included in the adjacent two images.

[0045] In an optional embodiment, the positioning result of the target vehicle in the plurality of continuous images is determined based on the feature similarity between the vehicles included in the adjacent two images, by the following method: taking the target vehicle as the vehicle determined in the first image, starting from the first image, and determining the positioning result of the target vehicle in the latter image based on the feature similarity between the vehicle determined in the former image in the adjacent two images and the vehicle included in the latter image in the adjacent two images.

[0046] It can be understood that the positioning result of the target vehicle included in the first image is that the vehicle determined in the first image is the target vehicle. The similarity between the target vehicle determined in the first image and the vehicle included in the second image is determined, and the vehicle corresponding to the maximum similarity is determined as the vehicle in the second image. The determination of the vehicles in the remaining images is based on the similarity between the vehicle determined in the previous image and the vehicle included in the current image. That is, based on the principle that the similarity between the vehicles in the adjacent two images is the maximum, the positioning result of the target vehicle in each image is determined in sequence, thereby realizing the continuous and accurate positioning of the target vehicle.

[0047] In an optional embodiment, when the first image, the second image, the third image and the fourth image are included in the plurality of continuous images, the determining the positioning result of the target vehicle in the plurality of continuous images based on the feature similarity between the vehicle determined in the previous one of the two adjacent images and the vehicle included in the latter one of the two adjacent images comprises: taking the target vehicle as the vehicle determined in the first image, determining the vehicle included in the second image that has the greatest feature similarity with the vehicle determined in the first image, taking the positioning result of the vehicle determined in the second image as the positioning result of the target vehicle in the second image; determining the vehicle included in the third image that has the greatest feature similarity with the vehicle determined in the second image, taking the positioning result of the vehicle determined in the third image as the positioning result of the target vehicle in the third image; and determining the vehicle included in the fourth image that has the greatest feature similarity with the vehicle determined in the third image, taking the positioning result of the vehicle determined in the fourth image as the positioning result of the target vehicle in the fourth image.

[0048] In the above manner, when the license plate information of the target vehicle cannot be completely recognized in the plurality of continuous images, the pre-trained target pedestrian re-identification model (i.e., the pre-trained ReID model) is used to calculate the feature vectors of the vehicle sub-images corresponding to the plurality of continuous images (i.e., the first, second, third and fourth images), and the similarity between the vehicle features of the target vehicle in the first image and the vehicle features included in the second image is calculated. The vehicle feature with the highest similarity score in the second image is the vehicle feature of the target vehicle in the second image, i.e., the vehicle determined in the second image. Similarly, the similarity between the vehicle determined in the second image and the vehicle included in the third image is calculated to determine the vehicle determined in the third image, and the similarity between the vehicle determined in the third image and the vehicle included in the fourth image is calculated to determine the vehicle determined in the fourth image, thereby achieving accurate matching and positioning of the target vehicle in each image.

[0049] In an optional embodiment, the method further comprises: when the license plate recognition results corresponding to the plurality of continuous images all indicate that the license plate recognition is successful, determining the positioning result of the target vehicle in the plurality of continuous images according to the license plate recognition results corresponding to the plurality of continuous images.

[0050] By the above mode, the pre-trained LPRNET model is used for license plate recognition on the identified multiple continuous images corresponding to the license plate image. If the license plate information of the target vehicle included in each of the multiple continuous images can be recognized, the target vehicle is positioned in the multiple continuous images based on the license plate information of the target vehicle, the target vehicle is matched and positioned, and further analysis is not required. If the license plate information of the target vehicle included in each of the multiple continuous images cannot be effectively recognized, it indicates that the continuous and accurate positioning of the target vehicle cannot be achieved based on the recognized license plate information. At this time, the positioning results of the target vehicle in the multiple continuous images are determined through similarity analysis, which can not only achieve the continuous and accurate positioning of the target vehicle, but also improve the matching and positioning efficiency of the target vehicle in the multiple continuous images.

[0051] Through the above steps S202 to S208, the license plate recognition and vehicle recognition are combined to track and position the vehicle. In the case where the vehicle cannot be tracked and positioned by the license plate, the vehicle is tracked and positioned by the vehicle feature matching, thereby achieving the technical effect of improving the accuracy of vehicle matching and positioning identification, and solving the technical problems of vehicle matching and positioning identification abnormality and poor accuracy caused by using license plate recognition for vehicle matching and positioning in the related art.

[0052] Based on the above embodiments and optional embodiments, the present application provides an optional implementation, Figure 5 is a flowchart of an optional vehicle matching and positioning method according to an embodiment of the present application, as Figure 5 shown, the method comprises:

[0053] Step S1, considering that there are many small targets in vehicles, a YOLOv7 algorithm with better recognition effect for small targets is used, and the YOLOv7 algorithm is trained to recognize vehicle targets and license plate targets, to obtain vehicle target small images and license plate target small images.

[0054] Step S2, according to the vehicle target small images and the license plate target small images, a license plate image training set and a vehicle feature image training set are respectively made, a license plate recognition model is trained on a license plate recognition dataset using an LPRNET character recognition network model, and a vehicle feature model is trained on a vehicle feature dataset using a pedestrian re-identification model (ReID, with a feature dimension of 512). The training of the three models (YOLOv7 model, LPRNET model, and ReID model) is completed.

[0055] Step S3, load all possible images of the target vehicle that may have abnormal behavior (i.e. the first, second, third and fourth images), use the pre-trained YOLOv7 model to identify the target vehicle and the corresponding license plate, and obtain the license plate image including the license plate information corresponding to each of the multiple consecutive images, and the vehicle image.

[0056] Step S4, using the pre-trained LPRNET model to identify the license plate of the license plate image (i.e. the license plate sub-image) corresponding to each of the multiple consecutive images, if the license plate information of the target vehicle included in each of the multiple consecutive images can be identified, the positioning result in the multiple consecutive images based on the license plate information of the target vehicle is used to realize the positioning matching of the target vehicle, i.e. the matching of the vehicle target is completed, and further analysis is not required.

[0057] Step S5, if the license plate information of the target vehicle cannot be completely identified in the multiple consecutive images, the pre-trained ReID model is used to calculate the feature vector of the vehicle sub-image in the first, second, third and fourth images, and the target vehicle in the first image is taken as the vehicle determined in the first image. The similarity between the vehicle determined in the first image and the vehicle included in the second image is calculated, and the vehicle determined in the second image is the vehicle with the highest similarity score in the second image. Similarly, the similarity between the vehicle determined in the second image and the vehicle included in the third image is calculated to obtain the vehicle determined in the third image, and the similarity between the vehicle determined in the third image and the vehicle included in the fourth image is calculated to obtain the vehicle determined in the fourth image. The positioning of the vehicle determined in the four images is the positioning result of the target vehicle.

[0058] The embodiment of the application can achieve the following technical effects: 1) when detecting the target, the small target is often difficult to identify, and the latest YOLOv7 released by the target detection is applied to the identification of the vehicle, which has better identification effect for small target identification; 2) when positioning the vehicle, the license plate detection is not used to complete the detection, and the ReID feature extraction algorithm is used to detect when the license plate detection cannot detect the vehicle, which reduces the vehicle missed detection; 3) in the process of ReID feature detection, in order to further ensure the accuracy of identification, the extraction dimension of the feature is set to 512, which can obtain more accurate features and further improve the accuracy; 4) in the process of feature matching, the similarity calculation is performed using the adjacent images, the template is dynamically updated, and the detection accuracy is improved.

[0059] A vehicle matching positioning apparatus is also provided in the embodiments, which is configured to implement the above-mentioned embodiments and preferred embodiments, and details of which have been described above. As used below, the term "module" or "apparatus" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

[0060] According to the embodiments of the present application, there is also provided an apparatus embodiment for implementing the above-mentioned vehicle matching positioning method, Figure 6 is a structural schematic diagram of a vehicle matching positioning apparatus according to the embodiments of the present application, as Figure 6 shown, the vehicle matching positioning apparatus comprises a first identification module 600, a second identification module 602, a third identification module 604, and a determination module 606, wherein:

[0061] The first identification module 600 is configured to perform first identification on a plurality of continuous images collected based on a predetermined time interval, to obtain and intercept license plate images including license plate information corresponding to the plurality of continuous images respectively, and vehicle images corresponding to the plurality of continuous images respectively, wherein the first image in the plurality of continuous images includes a target vehicle;

[0062] The second identification module 602 is connected to the first identification module 600 and is configured to perform second identification on the license plate images corresponding to the plurality of continuous images respectively, to obtain license plate recognition results corresponding to the plurality of continuous images respectively;

[0063] The third identification module 604 is connected to the second identification module 602 and is configured to, in a case where the license plate recognition result corresponding to any one of the plurality of continuous images indicates that license plate recognition fails, perform third identification based on the vehicle images corresponding to the plurality of continuous images, to determine a feature similarity between vehicles included in adjacent two images of the plurality of continuous images;

[0064] The determination module 606 is connected to the third identification module 604 and is configured to determine a positioning result of the target vehicle in the plurality of continuous images based on the feature similarity between the vehicles included in the adjacent two images.

[0065] In the embodiment of the present application, by setting the first identification module 600, the first identification is performed on the plurality of continuous images collected based on the predetermined time interval, the license plate images including the license plate information corresponding to the plurality of continuous images are obtained and intercepted, and the vehicle images corresponding to the plurality of continuous images are obtained, wherein the first image in the plurality of continuous images includes the target vehicle; the second identification module 602 is connected to the first identification module 600, and is used for performing the second identification on the license plate images corresponding to the plurality of continuous images to obtain the license plate recognition results corresponding to the plurality of continuous images; the third identification module 604 is connected to the second identification module 602, and is used for performing the third identification on the vehicle images corresponding to the plurality of continuous images to determine the feature similarity between the vehicles included in the adjacent two images in the plurality of continuous images in the case that the license plate recognition result corresponding to any one of the continuous images in the plurality of continuous images indicates that the license plate recognition fails; and the determination module 606 is connected to the third identification module 604, and is used for determining the positioning result of the target vehicle in the plurality of continuous images based on the feature similarity between the vehicles included in the adjacent two images, so as to achieve the tracking and positioning of the vehicle through the comprehensive license plate recognition and vehicle recognition, that is, the tracking and positioning of the vehicle through the vehicle feature matching in the case that the tracking and positioning of the vehicle through the license plate cannot be achieved, thereby achieving the technical effect of improving the vehicle matching and positioning recognition accuracy, and further solving the technical problems of vehicle matching and positioning recognition abnormalities and poor accuracy caused by the license plate recognition for vehicle matching and positioning in the related art.

[0066] It should be noted that each of the above modules can be implemented by software or hardware. For example, for the latter, the modules can be located in the same processor or in different processors in any combination.

[0067] It should be noted that the first identification module 600, the second identification module 602, the third identification module 604, and the determination module 606 correspond to steps S202 to S208 in the embodiment, and the modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above embodiment. It should be noted that the modules can run in a computer terminal as part of the device.

[0068] It should be noted that the optional or preferred implementation of the present embodiment can refer to the related description in the embodiment, which will not be repeated here.

[0069] The vehicle matching positioning apparatus can further include a processor and a memory, and the first identification module 600, the second identification module 602, the third identification module 604, and the determination module 606 are stored in the memory as program modules, and the processor executes the program modules stored in the memory to realize the corresponding functions.

[0070] The processor includes a core, and the core calls the corresponding program modules from the memory, and the core can be one or more. The memory can include a non-permanent memory in a computer readable medium, a random access memory (RAM), and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory includes at least one memory chip.

[0071] According to the embodiments of the present application, an embodiment of a non-volatile storage medium is further provided. Optionally, in the embodiment, the non-volatile storage medium includes a stored program, and when the program runs, the non-volatile storage medium controls a device where the non-volatile storage medium is located to perform any of the vehicle matching positioning methods.

[0072] Optionally, in the embodiment, the non-volatile storage medium can be located in any of computer terminals in a computer terminal group in a computer network, or in any of mobile terminals in a mobile terminal group, and the non-volatile storage medium includes a stored program.

[0073] Optionally, when the program runs, the non-volatile storage medium controls a device where the non-volatile storage medium is located to perform the following functions: performing first identification on a plurality of continuous images collected based on a predetermined time interval to obtain and intercept vehicle license plate images including license plate information corresponding to the plurality of continuous images respectively, and vehicle images corresponding to the plurality of continuous images respectively, and the first image in the plurality of continuous images includes a target vehicle; performing second identification on the vehicle license plate images corresponding to the plurality of continuous images to obtain license plate recognition results corresponding to the plurality of continuous images respectively; in a case where license plate recognition results corresponding to any one of the continuous images indicate that license plate recognition fails, performing third identification based on the vehicle images corresponding to the plurality of continuous images to determine feature similarity between vehicles included in adjacent two images in the plurality of continuous images; and determining a positioning result of the target vehicle in the plurality of continuous images based on the feature similarity between the vehicles included in the adjacent two images.

[0074] According to the embodiments of the present application, an embodiment of a processor is further provided. Optionally, in the embodiment, the processor is used to run a program, and when the program runs, the processor performs any of the vehicle matching positioning methods.

[0075] According to the embodiment of the present application, an embodiment of a computer program product is also provided, which is adapted to execute the steps of the vehicle matching positioning method when executed on a data processing device.

[0076] Optionally, the computer program product is adapted to execute the steps of the method when executed on a data processing device, the method comprising: performing first identification on a plurality of continuous images collected based on a predetermined time interval to obtain and extract license plate images including license plate information corresponding to the plurality of continuous images respectively, and vehicle images corresponding to the plurality of continuous images respectively, wherein a first image in the plurality of continuous images includes a target vehicle; performing second identification on the license plate images corresponding to the plurality of continuous images respectively to obtain license plate recognition results corresponding to the plurality of continuous images respectively; in a case where a license plate recognition result corresponding to any one of the continuous images in the plurality of continuous images indicates a license plate recognition failure, performing third identification based on the vehicle images corresponding to the plurality of continuous images respectively to determine a feature similarity between vehicles included in two adjacent images in the plurality of continuous images; and determining a positioning result of the target vehicle in the plurality of continuous images based on the feature similarity between the vehicles included in the two adjacent images.

[0077] An electronic device is provided, which includes a processor, a memory, and a program stored in the memory and executable on the processor, and the processor implements the following steps when executing the program: performing first identification on a plurality of continuous images collected based on a predetermined time interval to obtain and extract license plate images including license plate information corresponding to the plurality of continuous images respectively, and vehicle images corresponding to the plurality of continuous images respectively, wherein a first image in the plurality of continuous images includes a target vehicle; performing second identification on the license plate images corresponding to the plurality of continuous images respectively to obtain license plate recognition results corresponding to the plurality of continuous images respectively; in a case where a license plate recognition result corresponding to any one of the continuous images in the plurality of continuous images indicates a license plate recognition failure, performing third identification based on the vehicle images corresponding to the plurality of continuous images respectively to determine a feature similarity between vehicles included in two adjacent images in the plurality of continuous images; and determining a positioning result of the target vehicle in the plurality of continuous images based on the feature similarity between the vehicles included in the two adjacent images.

[0078] The above embodiments of the present application are merely for description, and do not represent advantages or disadvantages of the embodiments.

[0079] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0080] In several embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other manners. For example, the described device embodiments are merely illustrative, and the division of modules can be different from the above. For example, one or more modules or components can be combined or integrated into another system, or some features can be ignored or not implemented. In addition, the coupling or direct coupling or communication connection between the modules can be implemented through some interfaces, and the indirect coupling or communication connection between the modules can be implemented in electrical or other forms.

[0081] The modules described above as separated components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, i.e., can be located in one place, or can be distributed to multiple modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0082] In addition, the functional modules in each embodiment of the present application can be integrated into a processing module, or each module can exist physically, or two or more modules can be integrated into one module. The integrated module can be realized in the form of hardware or in the form of a software functional module.

[0083] The integrated module, if realized in the form of a software functional module and sold or used as an independent product, can be stored in a computer-readable nonvolatile storage medium. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a nonvolatile storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The aforementioned nonvolatile storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0084] The above is only a preferred embodiment of the present application, and it should be pointed out that for ordinary skilled persons in the art, without departing from the principles of the present application, several improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A vehicle matching positioning method, characterized by, The method comprises the following steps: performing first identification on a plurality of continuous images collected based on a predetermined time interval to obtain and intercept license plate images including license plate information corresponding to the plurality of continuous images respectively, and vehicle images corresponding to the plurality of continuous images respectively, wherein a first image in the plurality of continuous images includes a target vehicle; performing second identification on the license plate images corresponding to the plurality of continuous images respectively to obtain license plate recognition results corresponding to the plurality of continuous images respectively; in a case where the license plate recognition result corresponding to any one of the continuous images indicates that license plate recognition fails, performing third identification based on the vehicle images corresponding to the plurality of continuous images to determine a feature similarity between vehicles included in adjacent two images in the plurality of continuous images; determining a positioning result of the target vehicle in the plurality of continuous images based on the feature similarity between the vehicles included in the adjacent two images; wherein the method further comprises: determining the positioning result of the target vehicle in the plurality of continuous images based on the feature similarity between the vehicles included in the adjacent two images by the following manner: taking the target vehicle as the vehicle determined in the first image, starting from the first image, and determining the positioning result of the target vehicle in a subsequent image in the adjacent two images based on the feature similarity between the vehicle determined in a previous image in the adjacent two images and the vehicle included in the subsequent image in the adjacent two images.

2. The method of claim 1, wherein, The third identification based on the vehicle images corresponding to the plurality of continuous images to determine the feature similarity between the vehicles included in the adjacent two images in the plurality of continuous images comprises: based on the vehicle images corresponding to the plurality of continuous images, using a pre-trained target pedestrian re-identification model to determine feature vectors of the vehicles included in the plurality of continuous images respectively; from the feature vectors of the vehicles included in the plurality of continuous images respectively, determining feature vectors of the vehicles included in the adjacent two images respectively; determining the feature similarity between the vehicles included in the adjacent two images according to the feature vectors of the vehicles included in the adjacent two images respectively.

3. The method of claim 2, wherein, The method further comprises: inputting a pre-acquired training image dataset into a pre-constructed initial pedestrian re-identification model for training to obtain a first pedestrian re-identification model, wherein the training image dataset includes a plurality of groups of images of a plurality of vehicles, the initial pedestrian re-identification model includes an improved ResNet network, and the improved ResNet network is obtained by deleting a last fully connected layer in an original ResNet network and replacing it with a combination of a first fully connected layer and a second fully connected layer, wherein the number of network layers of the first fully connected layer is less than the number of network layers corresponding to a penultimate network layer in the original ResNet network, and the output result corresponding to the second fully connected layer is a vehicle category number; removing the second fully connected layer in the first pedestrian re-identification model to obtain the target pedestrian re-identification model.

4. The method of claim 3, wherein, In the case that the plurality of continuous images include a first image, a second image, a third image and a fourth image, the determining of the positioning result of the target vehicle in the plurality of continuous images based on the feature similarity between the vehicle determined in the previous one of the two adjacent images and the vehicle included in the latter one of the two adjacent images includes: taking the target vehicle as the vehicle determined in the first image, determining the vehicle included in the second image that has the greatest feature similarity with the vehicle determined in the first image, and taking the positioning result of the vehicle determined in the second image as the positioning result of the target vehicle in the second image; determining the vehicle included in the third image that has the greatest feature similarity with the vehicle determined in the second image, and taking the positioning result of the vehicle determined in the third image as the positioning result of the target vehicle in the third image; determining the vehicle included in the fourth image that has the greatest feature similarity with the vehicle determined in the third image, and taking the positioning result of the vehicle determined in the fourth image as the positioning result of the target vehicle in the fourth image.

5. The method of claim 1, wherein, The method further includes: in the case that the license plate recognition results corresponding to the plurality of continuous images respectively all indicate that the license plate recognition is successful, determining the positioning result of the target vehicle in the plurality of continuous images according to the license plate recognition results corresponding to the plurality of continuous images respectively.

6. The method according to any one of claims 1 to 5, characterized in that, The first identification of the plurality of continuous images collected based on a predetermined time interval to obtain and intercept the vehicle image and the license plate image including license plate information corresponding to the plurality of continuous images respectively includes: adopting a YoLOv7 multi-target detection algorithm to perform the first identification on the plurality of continuous images to obtain and intercept the vehicle image and the license plate image including license plate information corresponding to the plurality of continuous images respectively.

7. A vehicle matching positioning apparatus characterized by comprising: It includes: a first identification module, configured to perform first identification on a plurality of continuous images collected based on a predetermined time interval to obtain and intercept the vehicle image and the license plate image including license plate information corresponding to the plurality of continuous images respectively, wherein a first image in the plurality of continuous images includes a target vehicle; a second identification module, configured to perform second identification on the license plate image corresponding to the plurality of continuous images respectively to obtain a license plate recognition result corresponding to the plurality of continuous images respectively; a third identification module, configured to, in the case that the license plate recognition result corresponding to any one of the plurality of continuous images indicates that the license plate recognition fails, perform third identification based on the vehicle image corresponding to the plurality of continuous images to determine the feature similarity between the vehicles included in two adjacent images in the plurality of continuous images; a determining module, configured to determine the positioning result of the target vehicle in the plurality of continuous images based on the feature similarity between the vehicles included in the two adjacent images. The apparatus is further configured to determine the positioning result of the target vehicle in the plurality of continuous images based on feature similarity between vehicles included in the two adjacent images, by taking the target vehicle as the vehicle determined in the first image, and starting from the first image, determining the positioning result of the target vehicle in a subsequent image of the two adjacent images based on feature similarity between the vehicle determined in a previous image of the two adjacent images and the vehicles included in the subsequent image.

8. A non-volatile storage medium, comprising: The non-volatile storage medium stores a plurality of instructions adapted to be loaded and executed by the processor to implement the vehicle matching positioning method according to any one of claims 1 to 6.

9. An electronic device, comprising: The apparatus includes one or more processors and a memory, and the memory is configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the vehicle matching positioning method according to any one of claims 1 to 6.

Images