Target vehicle tracking method and device, storage medium, and electronic device

Abstract

The embodiment of the present invention provides a method and device for tracking a target vehicle, a storage medium, and an electronic device, the method comprising: using the acquired N-th frame image to identify the target vehicle, wherein N is a natural number greater than or equal to 1; when the license plate information of the target vehicle is not identified from the N-th frame image, using the acquired M-th frame image to identify the license plate information of the target vehicle, wherein the exposure of the M-th frame image is less than the exposure of the N-th frame image, and M is a natural number greater than or equal to 1; tracking the target vehicle based on the characteristic information of the target vehicle. Through the present invention, the problem of identifying the vehicle license plate in the related art is solved, and the effect of accurately identifying the license plate is achieved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of communications, and in particular, to a method and device for tracking a target vehicle, a storage medium, and an electronic device. Background Art

[0002] In the road traffic monitoring industry, cameras are often used to capture the passing records or violations of motor vehicles, non-motor vehicles, pedestrians and other targets on the road, and to capture pictures or associate videos for evidence of violations. In order for the camera to capture clear images of vehicles and other targets, it is necessary to install strobe lights and burst lights to provide sufficient brightness for taking pictures. However, in some scenarios, there are requirements for light pollution control. If fill lights are not installed or the installation points are relatively remote, the ambient brightness is too low, and the photos taken by the camera are dark. There are many types of license plates on the market, including red background with white letters, gray background with black letters, reflective, non-reflective, etc., so the overall ambient illumination is low, resulting in some reflective license plates being overexposed when the lights are turned on, or the lights are too bright, resulting in serious backlighting and unclear license plates, resulting in missed vehicles, which is not conducive to intelligent traffic road monitoring. Summary of the invention

[0003] The embodiments of the present invention provide a method and device for tracking a target vehicle, a storage medium, and an electronic device, so as to at least solve the problem of identifying a vehicle license plate in the related art.

[0004] According to one embodiment of the present invention, a method for tracking a target vehicle is provided, comprising: identifying the target vehicle using an acquired N-th frame image, wherein N is a natural number greater than or equal to 1; in the case where the license plate information of the target vehicle is not identified from the N-th frame image, identifying the license plate information of the target vehicle using an acquired M-th frame image, wherein the exposure of the M-th frame image is less than the exposure of the N-th frame image, and M is a natural number greater than or equal to 1; and tracking the target vehicle based on the characteristic information of the target vehicle.

[0005] According to another embodiment of the present invention, a target vehicle tracking device is provided, including: a first recognition module, used to identify the target vehicle using the acquired N-th frame image, wherein N is a natural number greater than or equal to 1; a second recognition module, used to identify the license plate information of the target vehicle using the acquired M-th frame image when the license plate information of the target vehicle is not identified from the N-th frame image, wherein the exposure of the M-th frame image is less than the exposure of the N-th frame image, and M is a natural number greater than or equal to 1; a first tracking module, used to track the target vehicle based on the characteristic information of the target vehicle.

[0006] In an exemplary embodiment, the above-mentioned first recognition module includes: a first acquisition unit, used to acquire the above-mentioned N-th frame image in the first target area with a first exposure through a camera device; a first extraction unit, used to extract the license plate information of the above-mentioned target vehicle from the above-mentioned N-th frame image to identify the above-mentioned target vehicle.

[0007] In an exemplary embodiment, the second recognition module includes: a second acquisition unit, used to acquire the M-th frame image with a second exposure through a camera device in the first target area of ​​the N-th frame image when the license plate information of the target vehicle is not recognized from the N-th frame image, wherein the second exposure is less than the first exposure of the N-th frame image; a first mapping unit, used to map the vehicle area in the N-th frame image to the vehicle area in the M-th frame image, so as to match the license plate position of the target vehicle in the M-th frame image; and a second extraction unit, used to extract the license plate information from the license plate position.

[0008] In an exemplary embodiment, the apparatus further includes: a first matching module, configured to match the Nth image frame and the Mth image frame to determine positions of components other than the license plate of the target vehicle in the Mth image frame.

[0009] In an exemplary embodiment, the above-mentioned device also includes: a first determination module, which is used to determine a first ratio between the first license plate and the first headlight from the above-mentioned Nth frame image before using the acquired Mth frame image to identify the license plate information of the above-mentioned target vehicle when the license plate information of the above-mentioned target vehicle is not identified from the above-mentioned Nth frame image; a second determination module, which is used to determine a second ratio between the second license plate and the second headlight from the above-mentioned Mth frame image; and a second matching module, which is used to match the above-mentioned first ratio and the second ratio to determine whether the vehicle in the above-mentioned Nth frame image and the vehicle in the above-mentioned Mth frame image are the above-mentioned target vehicle.

[0010] In an exemplary embodiment, the device further includes: a first trigger module, for, if the license plate information of the target vehicle is not identified from the Nth frame image, after identifying the license plate information of the target vehicle using the acquired Mth frame image, and upon determining that the target vehicle has traveled to a preset area, triggering the shooting of the target vehicle to obtain the Kth frame image, wherein the exposure of the Kth frame image is greater than that of the Nth frame image; a first extraction module, for extracting the window coordinate information of the target vehicle from the Kth frame image; and a third determination module, for determining the window area based on the window coordinate information.

[0011] In an exemplary embodiment, the above-mentioned device also includes: a third recognition module, which is used to perform face recognition on the object in the above-mentioned window area after determining the above-mentioned window area based on the above-mentioned window coordinate information; and a fourth determination module, which is used to determine the face area and face attribute information of the above-mentioned object from the above-mentioned face recognition.

[0012] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0013] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0014] According to the present invention, the target vehicle is identified by using the acquired N-th frame image, wherein N is a natural number greater than or equal to 1; when the license plate information of the target vehicle is not identified from the N-th frame image, the license plate information of the target vehicle is identified by using the acquired M-th frame image, wherein the exposure of the M-th frame image is less than the exposure of the N-th frame image, and M is a natural number greater than or equal to 1; and the target vehicle is tracked based on the characteristic information of the target vehicle. The purpose of tracking the target vehicle by identifying the license plate of the vehicle by using the bright frame image and the dark frame image is achieved. Therefore, the problem of identifying the license plate of the vehicle in the related art can be solved, and the effect of accurately identifying the license plate can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a hardware structure block diagram of a mobile terminal of a target vehicle tracking method according to an embodiment of the present invention;

[0016] Figure 2 is a flow chart of a target vehicle tracking method according to an embodiment of the present invention;

[0017] Figure 3 is a schematic diagram of a vehicle capture method based on bright and dark frames according to an embodiment of the present invention;

[0018] Figure 4 is a schematic diagram of a vehicle capture device based on bright and dark frames according to an embodiment of the present invention;

[0019] Figure 5 is a schematic diagram of a bright and dark frame working sequence according to an embodiment of the present invention;

[0020] Figure 6 is a schematic diagram of a bright and dark frame vehicle area mapping and matching method according to an embodiment of the present invention;

[0021] Figure 7 is a vehicle capture flow chart based on bright and dark frames according to an embodiment of the present invention;

[0022] Figure 8 is a schematic diagram of a bright frame according to an embodiment of the present invention (I);

[0023] Fig. 9 is a schematic diagram of a dark frame according to an embodiment of the present invention (I);

[0024] Fig.10 is a schematic diagram of a bright frame according to an embodiment of the present invention (II);

[0025] Fig.11 is a schematic diagram of a dark frame according to an embodiment of the present invention (II);

[0026] Fig.12 4 is a structural block diagram of a target vehicle tracking device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0029] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a target vehicle tracking method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0030] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the target vehicle tracking method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0031] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0032] In this embodiment, a method for tracking a target vehicle is provided. Figure 2 is a flow chart of a target vehicle tracking method according to an embodiment of the present invention, such as Figure 2 As shown, the process includes the following steps:

[0033] Step S202, identifying a target vehicle using the acquired Nth frame image, where N is a natural number greater than or equal to 1;

[0034] Step S204, when the license plate information of the target vehicle is not recognized from the Nth frame image, the license plate information of the target vehicle is recognized using the acquired Mth frame image, wherein the exposure of the Mth frame image is less than the exposure of the Nth frame image, and M is a natural number greater than or equal to 1;

[0035] Step S206: Track the target vehicle based on the characteristic information of the target vehicle.

[0036] This embodiment includes but is not limited to being applied to the scene of license plate recognition, where the Nth frame image is an image with a higher exposure than the Mth frame image. For example, a camera device uses an image sensor to collect two channels of images, and uses high-speed devices such as FPGAs to control three exposure parameters to achieve an image frame sequence with different exposure outputs in three channels. The image with a larger shutter value has a brighter brightness, and the image with a smaller shutter value has a darker brightness, i.e., a bright frame image and a dark frame image.

[0037] In this embodiment, the target vehicle is identified through the Nth frame image, and the definition of the target vehicle in the Nth frame image is relatively high. If the license plate cannot be identified due to excessive reflection brightness of the license plate, that is, the license plate cannot be displayed in the Nth frame image, the license plate is identified from the vehicle area of ​​the Mth frame image corresponding to the Nth frame image.

[0038] In this embodiment, other parts of the target vehicle may also be identified through the Mth frame image, and the driver's face recognition may also be performed.

[0039] The execution subject of the above steps may be a terminal, etc., but is not limited thereto.

[0040] Through the above steps, the target vehicle is identified by using the acquired N-th frame image, where N is a natural number greater than or equal to 1; when the license plate information of the target vehicle is not identified from the N-th frame image, the license plate information of the target vehicle is identified by using the acquired M-th frame image, where the exposure of the M-th frame image is less than the exposure of the N-th frame image, and M is a natural number greater than or equal to 1; the target vehicle is tracked based on the characteristic information of the target vehicle. The purpose of tracking the target vehicle by identifying the vehicle license plate using the bright frame image and the dark frame image is achieved. Therefore, the problem of identifying the vehicle license plate in the related art can be solved, and the effect of accurately identifying the license plate can be achieved.

[0041] In an exemplary embodiment, identifying a target vehicle using the acquired Nth frame image includes:

[0042] S1, acquiring an Nth frame of image in a first target area with a first exposure level by a camera device;

[0043] S2, extracting the license plate information of the target vehicle from the Nth frame image to identify the target vehicle.

[0044] In this embodiment, the frame sequence of the Nth frame image maintains a certain frame rate requirement, such as 12.5 (Phase Alternation Line by line (PAL) system) or 15 frames (NTSC system) to achieve the purpose of continuous target tracking.

[0045] In an exemplary embodiment, when the license plate information of the target vehicle is not recognized from the Nth frame image, the license plate information of the target vehicle is recognized by using the acquired Mth frame image, including:

[0046] S1, when the license plate information of the target vehicle is not recognized from the Nth frame image, in the first target area where the Nth frame image is obtained, the Mth frame image is obtained by a camera device at a second exposure level, wherein the second exposure level is less than the first exposure level of the Nth frame image;

[0047] S2, mapping the vehicle area in the Nth frame image to the vehicle area in the Mth frame image, so as to match the license plate position of the target vehicle in the Mth frame image;

[0048] S3, extracting the license plate information from the license plate position.

[0049] In this embodiment, the Nth frame image and the Mth frame image may be acquired simultaneously by a camera device.

[0050] In an exemplary embodiment, the method further comprises:

[0051] S1, matching the Nth frame image and the Mth frame image, and determining the positions of other parts of the target vehicle except the license plate in the Mth frame image.

[0052] In this embodiment, the Mth frame image is mainly used to obtain detailed attribute information of the target vehicle, such as feature extraction of license plate, logo, car model, body color, window, driver's face and other information.

[0053] In an exemplary embodiment, when the license plate information of the target vehicle is not recognized from the Nth frame image, before using the acquired Mth frame image to recognize the license plate information of the target vehicle, the method further includes:

[0054] S1, determining a first ratio between a first license plate and a first headlight from an Nth frame image;

[0055] S2, determining a second ratio between the second license plate and the second headlight from the Mth frame image;

[0056] S3, matching the first ratio and the second ratio to determine whether the vehicle in the Nth frame image and the vehicle in the Mth frame image are the target vehicle.

[0057] This embodiment can improve the matching accuracy by identifying the headlights or reflective tapes in the same position of the vehicle body frame area in the Mth frame image, and judging whether the license plate in the Mth frame image belongs to the vehicle in the same position area in the Nth frame image by comparing the ratio feature similarity (i.e., the comparison between the first ratio and the second ratio) of the distance between the two or more headlight areas and the block coordinate points where the license plate area is located exceeds the set threshold, thereby further improving the accuracy of the target vehicle area prediction.

[0058] In an exemplary embodiment, when the license plate information of the target vehicle is not recognized from the Nth frame image, after the license plate information of the target vehicle is recognized using the acquired Mth frame image, the method further includes:

[0059] S1, when it is determined that the target vehicle travels into the preset area, triggering the shooting of the target vehicle to obtain the Kth frame image, wherein the exposure of the Kth frame image is greater than the exposure of the Nth frame image;

[0060] S2, extracting the window coordinate information of the target vehicle from the K-th frame image;

[0061] S3, determining the vehicle window area based on the vehicle window coordinate information.

[0062] In this embodiment, the Kth frame image is an image frame captured at the moment when the flash light is triggered, and its brightness is the highest enough to illuminate targets such as faces in the car.

[0063] In an exemplary embodiment, after determining the vehicle window area based on the vehicle window coordinate information, the method further includes:

[0064] S1, face recognition of objects in the window area;

[0065] S2, determining the face area and face attribute information of the object from face recognition.

[0066] In this embodiment, the driver's face area and face attributes can be acquired by performing face recognition on the driver in the vehicle window area.

[0067] The present invention will be described below in conjunction with specific embodiments:

[0068] This embodiment takes bright frame T, dark frame S, and flash frame M (corresponding to the Nth frame image, the Mth frame image, and the Kth frame image in the above) as examples to provide a method for capturing vehicles with a camera, so as to solve the problem that the brightness of some license plates is too bright or too dark under low illumination, which makes it difficult to adjust the image to an appropriate level to ensure the shooting quality, resulting in a low license plate recognition rate. A vehicle capturing device with low implementation complexity is also provided, which can ensure stable tracking and extract more detailed features, thereby improving the vehicle capture rate.

[0069] The terms in this embodiment are explained as follows:

[0070] Fill light: includes strobe light and burst light to provide lighting for capturing vehicles.

[0071] FPGA: A high-speed digital processing device.

[0072] Sesor: An image sensor device.

[0073] Main recognition: The module is divided into main recognition according to its functions, and mainly performs target tracking based on neural networks.

[0074] Secondary identification: A software module that is functionally divided into auxiliary identification modules, mainly used for target attribute feature extraction.

[0075] Trigger: Notify FPGA to operate the flashing light through I / O and other control signals.

[0076] The specific plan is as follows:

[0077] The camera uses an image sensor to collect two channels of images, and uses high-speed devices such as FPGA to control three exposure parameters to achieve the output image frame sequence of three channels with different exposures. The image with a larger shutter value (bright frame T) is brighter, and the image with a smaller shutter value is darker (dark frame S). The camera is connected to an infrared fill light (providing a certain brightness of ambient light) and an infrared exposure flash light (providing a sufficient brightness of exposure flash). The image frame that has been flashed has the highest brightness (flash frame M). Figure 3 shown.

[0078] The vehicle target in the bright frame is clear and is sent to the main recognition of the intelligent algorithm for vehicle target tracking. When the main recognition algorithm tracks the vehicle target and triggers the capture event set by the user, the vehicle image is captured. Then it is determined whether the main recognition algorithm recognizes the license plate. If the license plate cannot be recognized due to excessive reflective brightness of the license plate, that is, the main recognition result shows that there is no license plate, the image of the vehicle area in the dark frame corresponding to the bright frame is sent to the secondary recognition algorithm for license plate and face recognition again, and the recognized license plate is used as the license plate of the captured vehicle, and the recognized facial attributes are output as the driver's facial information; when the secondary recognition of the bright frame T cannot recognize the facial information due to insufficient brightness, the window area in the flash frame M that is highlighted in the flash frame triggered by the bright frame target tracking is used for secondary recognition to obtain facial information to make up for the defect that the ordinary bright frame is not enough to capture the face in the car clearly.

[0079] This embodiment utilizes the different brightness of reflected images of different parts of the vehicle (body, license plate, face on the window, etc.) under three shutters, and uses three operations: primary recognition and tracking of targets, secondary recognition and attribute extraction, and line-touching trigger snapshot and flash, combined with different processing logics to comprehensively improve the accuracy of simultaneous recognition of vehicle feature attributes.

[0080] like Figure 4 FIG. 1 is a schematic diagram of a system device for capturing a vehicle in a low-light scene in this embodiment, which includes the following units:

[0081] The image acquisition unit controls the output acquisition frames of an image sensor Sensor through FPGA, and exposes the acquisition frames to images under three different shutter values. According to the exposure brightness, the frames are divided into three image frames with different exposures: light frame T, dark frame S, and flash frame M. Among them, the T frame sequence of the light frame channel maintains a certain frame rate requirement, such as 12.5 (PAL system) or 15 frames (NTSC system) to meet the requirements of the intelligent algorithm to achieve continuous target tracking; the dark frame S channel is a frame channel with a smaller exposure shutter synchronized with the light frame; the M frame is an image frame captured at the moment of whether the flash light is triggered, and its maximum brightness is enough to illuminate targets such as faces in the car. The T frame sequence and the S frame sequence will be continuously generated and sent to the algorithm unit, and the M frame will be generated and sent to the algorithm unit at a specific time. The three image frames are sent in chronological order and are marked with their respective channels or frame types, such as Figure 5 In order to speed up the logic operation processing, T frames, S frames, and M frames are stored and transmitted in a way of up-down, left-right, etc. splicing combination or frame-related index association, that is, after obtaining a T (or M) frame, the matching S frame can be quickly queried.

[0082] Algorithm processing unit, algorithm processing is divided into two modules: main recognition and secondary recognition according to the functional complexity and priority. The main recognition uses deep learning neural network to achieve target tracking, such as using Kalman filter and CSK tracking algorithm (Circulant Structure of Tracking-by-detection with Kernels) to perform target tracking (Object Detection, referred to as OD). After the main recognition is performed, a small number of license plate recognitions will be performed to maintain the real-time performance of OD. The main recognition continuously tracks vehicle targets, the network model is small, and the execution speed is fast to achieve a high frame rate analysis capability to meet the capture requirements of urban or highway speeds (the camera installation guarantees a certain shooting depth of field, and the frame rate reaches 12.5 frames according to the speed requirement of 80km / h on urban roads, and 25 frames at a speed of 120km / h on highways can ensure stable target tracking). The secondary recognition module mainly realizes the extraction of detailed attributes of the tracked target, including feature extraction of license plate, logo, car series, body color, window, driver's face and other information (Object Attribute, referred to as OA). Secondary recognition can identify richer information, has higher recognition accuracy, and takes longer time. In order not to affect the continuity and real-time performance of target tracking, the device uses a deep neural network coprocessor to perform stronger computing power and maintain concurrent processing of primary and auxiliary recognition.

[0083] The capture unit is triggered. When the vehicle target tracked by the main recognition od reaches the detection area set by the user, the capture unit is controlled to work. The capture unit is triggered to package the target information and the user-defined capture information snapInfo together and send them to the FPGA image processing unit to generate a capture signal and immediately change the exposure value and mark the next acquisition frame as a flash frame M. The M frame will be interspersed in the sequence of bright frames T and dark frames S in chronological order and also sent to the algorithm processing unit. For the M frames that appear in the next few frames, the recognition logic unit parses the snapInfo in the frame information to find the target ID and other trigger information that triggers the capture for binding the vehicle, indicating that the frame belongs to the captured picture of the vehicle.

[0084] The recognition logic unit sends the input T frame sequence to OD for processing. Since the T frame brightness is high, OD can easily track the vehicle target. However, when the license plate is severely reflected or the headlights are strongly reflected and the backlight is severe, the license plate cannot be clearly photographed (such as Figure 8 , Fig.10 As shown), the primary recognition cannot track the license plate. At this time, the secondary recognition module is called in the vehicle body area in the S frame that matches the T frame to perform secondary recognition of the license plate (as shown Fig. 9 , Fig.11As shown). When the vehicle drives to a special area (generally the area with the best flashing light brightness), the control triggers the capture unit to generate a capture signal to notify the FPGA image acquisition unit, and generates data including the vehicle tracking ID and the capture information snapInfo to be inserted into the frame information of the next frame M. When the recognition logic module receives the M frame, it performs a secondary recognition of the vehicle window in the M frame to obtain the window coordinates, and then performs a secondary recognition of the window area to obtain the driver's face area and face attributes.

[0085] Among them, the prediction of the vehicle area in the dark frame is found by matching and mapping the vehicle area in the bright frame, such as Figure 6 As shown, the enclosed area formed by the lane line is taken as a fixed range, and the vehicle area R1 in the bright frame T is projected to R2 in the S frame, and the range of the R2 area is guaranteed not to exceed the range of the lane, so as to prevent the regional error caused by different speeds from causing the range to cross the boundary and cause position matching errors. At the same time, in order to further improve the accuracy of vehicle area prediction, the lights or reflective tapes in the body frame area at the same position of the S frame are identified, and the proportional feature similarity of the distance between the coordinate points of the block where the two or more light areas and the license plate area are located exceeds the set threshold to determine whether the license plate in the dark frame S belongs to the vehicle in the same position area in the bright frame, thereby improving the matching accuracy.

[0086] Output unit,The output unit integrates important attribute information such as vehicle body, license plate, and face, and encodes T frames, S frames, and M frames into captured images, user-defined captured images, or synthetic images.

[0087] like Figure 7 As shown, the process of the vehicle capture solution in the low-light scene of this embodiment includes the following steps:

[0088] S701, the camera collects image frame sequences under 3 exposure shutters (including 1 image frame under the triggering flash light);

[0089] S702, sending the bright frame to the main recognition for target tracking and license plate recognition of the vehicle in the front position.

[0090] S703, determining whether the vehicle enters the capture area;

[0091] S704, when the vehicle enters the capture area (the area where the vehicle position is better and the license plate is clearer), capture is performed. Determine whether the bright frame T recognizes the vehicle;

[0092] S705, when the license plate is not recognized in the captured bright frame (divided into several situations such as no license plate, serious license plate reflection, and serious license plate backlight), the area of ​​the vehicle in the bright frame is mapped to the area in the matching dark frame, and the license plate is secondary recognized in the designated area.

[0093] S706, determining whether the vehicle has entered the exposure capture area;

[0094] S707, when the vehicle enters the flashing area, the flashing lights are triggered, and the high-definition window area captured in the flashing frame after the current frame is recognized again to extract facial features.

[0095] S708, determining whether the frame mark of the flash frame is an M frame;

[0096] S709, when the frame mark of the flash frame is M frame, send the flash frame M to secondary recognition for face attribute extraction to obtain face coordinates and face features. If there is no license plate, secondary recognition of the license plate is performed; the corresponding dark frame in the flash frame can also recognize the license plate.

[0097] S710, outputting the vehicle body, license plate, window, face and other information of the vehicle target in the bright frame, dark frame or flashing frame together with the captured image.

[0098] In summary, this embodiment uses bright frames, dark frames, and flash frames under three types of exposure shutters to track and identify vehicle targets, effectively solving the problem of license plate reflection being too severe to be identified when taking photos with infrared fill light in low-light scenes. In the case of poor lighting including severe reflection or insufficient lighting, the accuracy of identifying key vehicle information including vehicle body, license plate, face, and other key information is guaranteed. Compared with single-channel exposure image processing that cannot take into account the image quality requirements of vehicle parts or the complex processing of automatic exposure algorithms, this method is simpler to implement.

[0099] This embodiment divides the algorithm into primary recognition and secondary recognition. The primary recognition runs a small model detection algorithm to prioritize target tracking. When the target is tracked, it triggers capture and performs detailed feature attribute extraction on the captured image through secondary recognition, which reduces the performance consumption of the primary recognition and improves the algorithm analysis frame rate to meet the capture of higher speed targets.

[0100] This embodiment uses the coordinates of the headlights and reflective tapes extracted based on the image processing of the headlights and reflective tape areas, and establishes a connection relationship graph together with the license plate coordinates. By comparing the similarity of the connection relationship graph, it is determined whether the license plate recognized in the dark frame belongs to the target vehicle, with higher accuracy.

[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0102] In the present embodiment, a tracking device for a target vehicle is also provided, and the device is used to implement the above-mentioned embodiment and preferred implementation mode, and the descriptions have been made no further. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0103] Fig.12 is a structural block diagram of a target vehicle tracking device according to an embodiment of the present invention, such as Fig.12 As shown, the device comprises:

[0104] A first recognition module 1202 is used to recognize a target vehicle using the acquired N-th frame image, where N is a natural number greater than or equal to 1;

[0105] The second recognition module 1204 is used to recognize the license plate information of the target vehicle by using the acquired M-th frame image when the license plate information of the target vehicle is not recognized from the N-th frame image, wherein the exposure of the M-th frame image is less than the exposure of the N-th frame image, and M is a natural number greater than or equal to 1;

[0106] The first tracking module 1206 is used to track the target vehicle based on the characteristic information of the target vehicle.

[0107] In an exemplary embodiment, the first identification module includes:

[0108] A first acquisition unit, configured to acquire the Nth frame of image in the first target area with a first exposure level by using a camera device;

[0109] The first extraction unit is used to extract the license plate information of the target vehicle from the Nth frame image to identify the target vehicle.

[0110] In an exemplary embodiment, the second identification module includes:

[0111] A second acquisition unit is used to acquire the M-th frame image at a second exposure level in the first target area of ​​the N-th frame image by using a camera device when the license plate information of the target vehicle is not recognized from the N-th frame image, wherein the second exposure level is smaller than the first exposure level of the N-th frame image;

[0112] A first mapping unit, used for mapping the vehicle area in the Nth frame image to the vehicle area in the Mth frame image, so as to match the license plate position of the target vehicle in the Mth frame image;

[0113] The second extraction unit is used to extract the license plate information from the license plate position.

[0114] In an exemplary embodiment, the above device further comprises:

[0115] The first matching module is used to match the Nth frame image with the Mth frame image to determine the positions of other components of the target vehicle except the license plate in the Mth frame image.

[0116] In an exemplary embodiment, the above device further comprises:

[0117] A first determination module is used to determine a first ratio between the first license plate and the first headlight from the Nth frame image before using the acquired Mth frame image to identify the license plate information of the target vehicle when the license plate information of the target vehicle is not identified from the Nth frame image;

[0118] A second determining module, used to determine a second ratio between the second license plate and the second headlight from the Mth frame image;

[0119] The second matching module is used to match the first ratio and the second ratio to determine whether the vehicle in the Nth frame image and the vehicle in the Mth frame image are the target vehicle.

[0120] In an exemplary embodiment, the apparatus further comprises:

[0121] A first trigger module is used for, when the license plate information of the target vehicle is not recognized from the Nth frame image, after using the acquired Mth frame image to recognize the license plate information of the target vehicle, and when it is determined that the target vehicle has traveled to a preset area, triggering the shooting of the target vehicle to obtain a Kth frame image, wherein the exposure of the Kth frame image is greater than the exposure of the Nth frame image;

[0122] A first extraction module is used to extract the window coordinate information of the target vehicle from the Kth frame image;

[0123] The third determining module is used to determine the vehicle window area based on the vehicle window coordinate information.

[0124] In an exemplary embodiment, the apparatus further comprises:

[0125] A third recognition module, configured to perform face recognition on an object in the vehicle window area after determining the vehicle window area based on the vehicle window coordinate information;

[0126] The fourth determination module is used to determine the face area and face attribute information of the above object from the above face recognition.

[0127] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0128] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0129] In this embodiment, the computer-readable storage medium may be configured to store a computer program for executing the above steps.

[0130] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0131] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0132] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0133] In an exemplary embodiment, the processor may be configured to execute the above steps through a computer program.

[0134] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0135] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0136] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for tracking a target vehicle, characterized in that: include: Using the acquired N-th frame image to identify the target vehicle, wherein N is a natural number greater than or equal to 1; In the case that the license plate information of the target vehicle is not recognized from the Nth frame image, the license plate information of the target vehicle is recognized using the acquired Mth frame image, wherein the exposure of the Mth frame image is less than the exposure of the Nth frame image, and M is a natural number greater than or equal to 1; Tracking the target vehicle based on the characteristic information of the target vehicle; Among them, tracking the target vehicle based on the characteristic information of the target vehicle includes: identifying the target vehicle through a main recognition module, and continuously tracking the target vehicle based on the license plate information of the target vehicle, wherein the main recognition module recognizes the target vehicle based on the license plate information of the target vehicle, and the main recognition module is a module obtained based on a deep learning application network; when the target vehicle is in a designated area, the detailed attributes of the target vehicle are extracted through a secondary recognition module, and the target vehicle is tracked using the detailed attributes.

2. The method according to claim 1, characterized in that The target vehicle is identified by using the acquired N-th frame image, including: Acquire the Nth frame of image in the first target area with a first exposure level by a camera device; The license plate information of the target vehicle is extracted from the Nth frame image to identify the target vehicle.

3. The method according to claim 1, characterized in that In the case that the license plate information of the target vehicle is not recognized from the Nth frame image, recognizing the license plate information of the target vehicle by using the acquired Mth frame image includes: In the case that the license plate information of the target vehicle is not recognized from the Nth frame image, in the first target area where the Nth frame image is obtained, the Mth frame image is obtained by a camera device at a second exposure level, wherein the second exposure level is less than the first exposure level of the Nth frame image; Mapping the vehicle area in the Nth frame image to the vehicle area in the Mth frame image to match the license plate position of the target vehicle in the Mth frame image; The license plate information is extracted from the license plate position.

4. The method according to claim 3, characterized in that The method further comprises: The Nth frame image and the Mth frame image are matched to determine the positions of other components of the target vehicle except the license plate in the Mth frame image.

5. The method according to claim 1, characterized in that In the case that the license plate information of the target vehicle is not recognized from the Nth frame image, before using the acquired Mth frame image to recognize the license plate information of the target vehicle, the method further includes: Determine a first ratio between the first license plate and the first headlight from the Nth frame image; Determine a second ratio between the second license plate and the second headlight from the Mth frame image; The first ratio and the second ratio are matched to determine whether the vehicle in the Nth frame image and the vehicle in the Mth frame image are the target vehicle.

6. The method according to claim 1, characterized in that In the case that the license plate information of the target vehicle is not recognized from the Nth frame image, after recognizing the license plate information of the target vehicle using the acquired Mth frame image, the method further includes: When it is determined that the target vehicle has traveled into a preset area, triggering shooting of the target vehicle to obtain a K-th frame image, wherein the exposure of the K-th frame image is greater than the exposure of the N-th frame image; Extracting the window coordinate information of the target vehicle from the Kth frame image; A vehicle window area is determined based on the vehicle window coordinate information.

7. The method according to claim 6, characterized in that After determining the vehicle window area based on the vehicle window coordinate information, the method further includes: Performing face recognition on objects in the vehicle window area; The face region and face attribute information of the object are determined from the face recognition.

8. A target vehicle tracking device, characterized in that: include: A first recognition module, used to recognize a target vehicle by using the acquired N-th frame image, wherein N is a natural number greater than or equal to 1; a second recognition module, configured to recognize the license plate information of the target vehicle by using the acquired M-th frame image when the license plate information of the target vehicle is not recognized from the N-th frame image, wherein the exposure of the M-th frame image is less than the exposure of the N-th frame image, and M is a natural number greater than or equal to 1; A first tracking module, used for tracking the target vehicle based on the characteristic information of the target vehicle; Among them, the first tracking module is also used to identify the target vehicle through the main recognition module, and continuously track the target vehicle based on the license plate information of the target vehicle, wherein the main recognition module recognizes the target vehicle based on the license plate information of the target vehicle, and the main recognition module is a module obtained based on the deep learning application network; when the target vehicle is in the designated area, the detailed attributes of the target vehicle are extracted through the secondary recognition module, and the target vehicle is tracked using the detailed attributes.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the method described in any one of claims 1 to 7 when executed by a processor.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 7.

Images