License plate selection method, device and equipment in vehicle snapshot event and storage medium

Abstract

The application relates to a license plate selection method and device in a vehicle snapshot event, computer equipment and a storage medium. The method comprises the following steps: obtaining a correlation degree coefficient of a vehicle and a license plate in each frame image based on a dense optical flow algorithm; determining a license plate optimization coefficient of the license plate in each frame image based on optical flow consistency of a front frame license plate region and a current frame license plate region in each frame image; determining a license plate selection parameter of the license plate in each frame image based on the correlation degree coefficient and the license plate optimization coefficient; determining a corresponding frame image based on the license plate selection parameter, and taking the license plate in the frame image as a target license plate. The method can solve the problem that the continuity of the license plate in front and back frames disappears in a large-angle, blurred and easily-shielded scene, so that the license plate correlation judgment basis is lost, and the reliability and accuracy of the optimized license plate are improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, computer equipment, and storage medium for selecting license plates in vehicle capture events. Background Technology

[0002] In recent years, smart parking has become increasingly popular in order to solve the urban parking problem. To maintain the daily management and operation of parking lots with minimal manpower, smart parking typically combines video surveillance and intelligent algorithms. Especially in the daily management and charging of parking lots, it relies primarily on vehicles and their corresponding license plates as a chain of evidence to achieve automated charging. Therefore, the selection of license plates, as a key component of the parking algorithm, directly affects the algorithm's performance indicators; its stability and accuracy directly impact the quality of the parking product and the user experience.

[0003] Currently, license plates in parking scenarios are often partially obscured, partially blurred, or at large angles, which can easily lead to low reliability of license plate frame character detection results or extreme inconsistencies between the reliability of license plate frame and character detection results, thus reducing the reliability and accuracy of license plate detection and recognition. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, device, computer equipment, and computer-readable storage medium for vehicle capture events that can solve the problem of reduced accuracy in license plate selection due to large angles, partial obstruction, and blurriness of license plates.

[0005] Firstly, this application provides a method for selecting license plates in vehicle capture events, the method comprising:

[0006] The correlation coefficient between vehicles and license plates in each frame of the image is obtained based on the dense optical flow algorithm; and

[0007] Based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, the license plate optimization coefficient of the license plate in each frame image is determined.

[0008] Based on the correlation coefficient and license plate selection coefficient, the license plate selection parameters in each frame image are determined;

[0009] Based on the license plate selection parameters, the image of the corresponding frame is determined, and the license plate in the image of that frame is used as the target license plate.

[0010] In one embodiment, obtaining the correlation coefficient between vehicles and license plates in each frame of the image based on the dense optical flow algorithm includes:

[0011] Based on the dense optical flow algorithm, the first optical flow vector corresponding to the license plate region of the previous frame in the current frame and the second optical flow vector corresponding to the license plate region of the previous frame in the vehicle selection area of ​​the vehicle in the previous frame are obtained.

[0012] Based on a deep learning algorithm, the first target confidence score of the vehicle selection region in the previous frame is obtained;

[0013] Based on the first optical flow vector, the second optical flow vector, and the first target confidence, the correlation coefficient between the vehicle and the license plate in each frame image is obtained.

[0014] In one embodiment, the previous frame vehicle selection area includes one of the front, rear, and body of the vehicle, and obtaining the previous frame vehicle selection area of ​​the vehicle includes:

[0015] The vehicle detection frame is divided into multiple regions, and the intersection of each region is the center point of the vehicle detection frame.

[0016] When the license plate area of ​​the previous frame falls within the area of ​​the vehicle detection frame, the license plate area of ​​the previous frame is shifted to the center point, and the shifted license plate area of ​​the previous frame becomes the vehicle selection area of ​​the previous frame.

[0017] In one embodiment, determining the license plate preference coefficient in each frame image based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame includes:

[0018] Based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, the optical flow coordinates of the license plate region of the previous frame in the current frame and the dense optical flow point set corresponding to the optical flow coordinates are obtained.

[0019] Based on deep learning algorithms, the second target confidence of the license plate region of the current frame and the set of region pixels corresponding to the license plate region of the current frame are obtained in each frame.

[0020] Based on the dense optical flow point set, the regional pixel point set, and the second target confidence level, the license plate optimization coefficient in each frame image is determined.

[0021] In one embodiment, before determining the license plate preference coefficient in each frame image based on the dense optical flow point set, the region pixel point set, and the second target confidence, the method further includes:

[0022] Based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, a first number of optical flow coordinates in the license plate region of the previous frame and a second number of optical flow coordinates in the license plate region of the current frame are obtained;

[0023] Based on the positional relationship between the license plate area in the previous frame and the license plate area in the current frame, the displacement values ​​of the center point of the license plate area in the previous frame and the center point of the license plate area in the current frame are obtained;

[0024] Based on the first quantity, the second quantity, and the displacement value, obtain the frame association weight between the license plate region of the previous frame and the license plate region of the current frame in each frame;

[0025] Based on the dense optical flow point set, the regional pixel point set, the second target confidence, and the frame association weight, the license plate selection coefficient in each frame image is determined.

[0026] In one embodiment, determining the license plate selection parameters for each frame image based on the correlation coefficient and the license plate preference coefficient includes:

[0027] The correlation coefficient is multiplied by the license plate selection coefficient to determine the license plate selection parameters in each frame of the image.

[0028] In one embodiment, determining the image of the corresponding frame based on each of the license plate selection parameters, and using the license plate in that frame image as the target license plate, includes:

[0029] Based on each of the license plate selection parameters, the image of the frame corresponding to the maximum value of the license plate selection parameter is obtained, and the license plate in that frame image is used as the target license plate.

[0030] Secondly, this application also provides a license plate selection device for vehicle capture events, the device comprising:

[0031] The first acquisition module is used to obtain the correlation coefficient between vehicles and license plates in each frame of the image based on the dense optical flow algorithm; and

[0032] The second determining module is used to determine the license plate optimization coefficient in each frame image based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image.

[0033] The joint calculation module is used to determine the license plate selection parameters in each frame image based on the correlation coefficient and the license plate selection coefficient.

[0034] The license plate selection module is used to determine the image of the corresponding frame based on the license plate selection parameters, and to use the license plate in the image of that frame as the target license plate.

[0035] Thirdly, this application also provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the content of the first aspect above.

[0036] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the contents of the first aspect described above.

[0037] The aforementioned method, device, computer equipment, and storage medium for selecting license plates in vehicle capture events obtain the correlation coefficient between vehicles and license plates in each frame image based on a dense optical flow algorithm; and determine the license plate selection coefficient in each frame image based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame. Based on the correlation coefficient and the license plate selection coefficient, the license plate selection parameters in each frame image are determined; based on the license plate selection parameters, the corresponding frame image is determined, and the license plate in that frame image is used as the target license plate. This achieves license plate selection in scenarios with large-angle, blurred, and easily obscured license plates, and improves the reliability and accuracy of the selected license plates. Attached Figure Description

[0038] Figure 1 This is an application environment diagram of a license plate selection method in a vehicle capture event in one embodiment.

[0039] Figure 2 This is a flowchart illustrating a method for selecting license plates in a vehicle capture event, as shown in one embodiment.

[0040] Figure 3 This is a flowchart illustrating S202 in one embodiment;

[0041] Figure 4 This is a schematic diagram of the process of obtaining the vehicle selection area in the previous frame in S202 of one embodiment;

[0042] Figure 5 This is a structural diagram of the vehicle selection area obtained in one embodiment;

[0043] Figure 6 This is a flowchart illustrating S204 in one embodiment;

[0044] Figure 7 This is a flowchart illustrating the calculation of frame association weights in one embodiment;

[0045] Figure 8 This is a structural block diagram of a license plate selection device in a vehicle capture event according to one embodiment;

[0046] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] The license plate selection method in vehicle capture events provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. Data storage system 106 can store vehicle capture data that server 104 needs to process. Data storage system 106 can be integrated on server 104 or placed on the cloud or other network servers. On server 104, the correlation coefficient between vehicles and license plates in each frame image is obtained based on a dense optical flow algorithm; and the license plate selection coefficient in each frame image is determined based on the optical flow consistency between the license plate area of ​​the previous frame and the license plate area of ​​the current frame image; based on the correlation coefficient and the license plate selection coefficient, the license plate selection parameters in each frame image are determined; based on the license plate selection parameters, the image of the corresponding frame is determined, and the license plate in that frame image is used as the target license plate. This result can be displayed on terminal 102. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, and tablets. Server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0049] In one embodiment, such as Figure 2 As shown, a method for selecting license plates in vehicle capture events is provided, and this method is applied to... Figure 1 Taking the application environment in [the document] as an example, the following steps are included:

[0050] S202, based on the dense optical flow algorithm, obtains the correlation coefficient between vehicles and license plates in each frame of the image.

[0051] Dense optical flow (DIF) is an image registration method that performs point-by-point matching. It calculates the offset of all points in an image, thus forming a dense optical flow field. Therefore, DIF can maintain the consistency of optical flow between frames in the unobstructed license plate area.

[0052] Optionally, based on a dense optical flow algorithm, the first optical flow vector of the license plate region in the preceding and following frames and the second optical flow vector of the license plate region in each frame on the vehicle selection region are obtained. The confidence level of the vehicle selection region is then obtained through a license plate detection deep learning model, thereby yielding the correlation coefficient between the vehicle and the license plate in each frame image. The vehicle selection region includes one of the front, rear, or body of the vehicle.

[0053] Optionally, the higher the correlation coefficient, the more relevant the license plate and the vehicle are, and the higher the credibility of the license plate.

[0054] S204. Based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, determine the license plate optimization coefficient in each frame image.

[0055] Optionally, based on the optical flow consistency between the license plate region in the previous frame and the license plate region in the current frame in each frame image, the optical flow coordinates of the license plate region in the previous frame and the dense optical flow point set corresponding to the optical flow coordinates are obtained in the current frame. The confidence level of the license plate region in the current frame and the corresponding region pixel point set are obtained through a license plate detection deep learning model, and the license plate optimization coefficient of the license plate in each frame image is determined.

[0056] Optionally, the higher the license plate preference coefficient, the higher the credibility of the license plate.

[0057] S206, Based on the correlation coefficient and the license plate selection coefficient, determine the license plate selection parameters in each frame image.

[0058] Optionally, the correlation coefficient is multiplied by the license plate preference coefficient to determine the license plate selection parameters in each frame image.

[0059] S208, based on the license plate selection parameters, determine the image of the corresponding frame, and use the license plate in the image of that frame as the target license plate.

[0060] Optionally, when capturing vehicle images, license plates at large angles, blurred license plates, or obscured license plates are often captured. In these abnormal scenarios, the corresponding license plate selection parameters are small. Therefore, based on each of the license plate selection parameters, the image of the frame corresponding to the maximum value of the license plate selection parameters is obtained, and the license plate in that frame image is used as the target license plate.

[0061] In the above-mentioned method for selecting license plates in vehicle capture events, the correlation coefficient between vehicles and license plates in each frame image is obtained based on the dense optical flow algorithm; and the license plate selection coefficient in each frame image is determined based on the optical flow consistency between the license plate area of ​​the previous frame and the license plate area of ​​the current frame in each frame image; based on the correlation coefficient and the license plate selection coefficient, the license plate selection parameters in each frame image are determined; based on the license plate selection parameters, the image of the corresponding frame is determined, and the license plate in that frame image is used as the target license plate. This solves the problem that the continuity of the license plate disappears between frames in scenes with large angles, blurriness, and easy occlusion, thus losing the basis for license plate correlation judgment, and improves the reliability and accuracy of selecting the license plate.

[0062] In one embodiment, such as Figure 3 As shown, the correlation coefficient between vehicles and license plates in each frame of the image obtained based on the dense optical flow algorithm includes:

[0063] S302, based on the dense optical flow algorithm, obtain the first optical flow vector corresponding to the license plate region of the previous frame in the current frame and the second optical flow vector corresponding to the license plate region of the previous frame in the vehicle selection area of ​​the vehicle in the previous frame.

[0064] S304, Based on a deep learning algorithm, obtain the first target confidence score of the selected area of ​​the vehicle in the previous frame.

[0065] The confidence level of the first objective ranges from 0 to 1.

[0066] S306, based on the first optical flow vector, the second optical flow vector, and the first target confidence, obtain the correlation coefficient between the vehicle and the license plate in each frame image.

[0067] Optionally, the first and second optical flow vectors are unfolded into one-dimensional data, and the second optical flow vector is transposed. The cosine similarity between the first and second optical flow vectors is then calculated. To avoid the optical flow vector value being 0, a correlation parameter is introduced into the cosine similarity. This correlation parameter is a number greater than 0 and much less than 1, with the correlation coefficient between the vehicle and license plate in each frame ranging from 0 to 1.

[0068] In the above embodiments, the first optical flow vector and the second optical flow vector are obtained by the dense optical flow algorithm, and the first target confidence is obtained by the deep learning algorithm, thereby obtaining the correlation coefficient between the vehicle and the license plate in each frame image. This increases the constraints on license plate selection, enhances reliability, and improves the accuracy of license plate selection.

[0069] In one embodiment, such as Figure 4 As shown, obtaining the vehicle selection region of the previous frame includes:

[0070] S402, the vehicle detection frame of the vehicle is divided into multiple regions.

[0071] The intersection of each of the aforementioned regions is the center point of the vehicle detection frame.

[0072] Optionally, the vehicle selection area mentioned in the above embodiments includes one of the front, rear, and body of the vehicle. When the vehicle selection area is the front of the vehicle, the vehicle detection frame is the front detection frame of the vehicle. The vehicle detection frame is divided into multiple areas, and the specific position of the license plate area in the previous frame within the vehicle detection frame is determined.

[0073] S404, when the previous frame license plate area falls within the area of ​​the vehicle detection frame, the previous frame license plate area is shifted to the center point, and the shifted previous frame license plate area becomes the previous frame vehicle selection area of ​​the vehicle.

[0074] Optionally, such as Figure 5As shown, this is a method for obtaining the license plate area of ​​the previous frame of a vehicle detection box. In the figure, the vehicle detection box is divided into four regions: upper left, lower left, upper right, and lower right. The intersection of the four regions is the center point of the vehicle detection box.

[0075] Optionally, when the license plate area of ​​the previous frame falls within a single area of ​​the vehicle detection frame, such as Figure 5 As shown in Figure (a), the corner point closest to the center point of the license plate area in the previous frame is determined first, and the vehicle is moved to the center point of the vehicle detection frame and aligned with it. The license plate area in the previous frame after alignment is the vehicle selection area.

[0076] Optionally, when the license plate area of ​​the previous frame falls within two areas of the vehicle detection frame, such as Figure 5 As shown in Figure (b), the edge closest to the center point of the license plate area in the previous frame is determined first, the midpoint of the edge is obtained, and the edge is translated to the center point of the vehicle detection box with the midpoint as the starting position. The license plate area in the previous frame after alignment is the vehicle selection area.

[0077] Optionally, when the license plate area of ​​the previous frame falls within two or more areas of the vehicle detection frame, such as Figure 5 As shown in Figure (c), the center point of the license plate area in the previous frame is obtained and the vehicle detection box is translated to align with the center point of the vehicle detection box, and the aligned license plate area in the previous frame is the vehicle selection area.

[0078] In the above embodiments, by dividing the vehicle detection frame of the vehicle into multiple regions, and the intersection of each region is the center point of the vehicle detection frame, the position of the license plate region of the previous frame falling within the region of the vehicle detection frame is determined to select the vehicle selection region of the previous frame. In the two-dimensional category, a smaller region is used to reduce the probability of overlap to enhance the reliability of license plate association, improve the standard for selecting the second optical flow vector, and thus improve the accuracy of the association coefficient.

[0079] In one embodiment, such as Figure 6 As shown, the process of determining the license plate preference coefficient in each frame of the image based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame includes:

[0080] S602, based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, obtain the optical flow coordinates of the license plate region of the previous frame in the current frame and the dense optical flow point set corresponding to the optical flow coordinates.

[0081] Optionally, based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, the optical flow coordinates of the license plate region of the previous frame in the current frame can be obtained from the first optical flow vector, and the optical flow coordinates can be aggregated into a dense optical flow point set.

[0082] S604, based on a deep learning algorithm, obtain the second target confidence of the license plate region of the current frame and the set of region pixels corresponding to the license plate region of the current frame in each frame.

[0083] S606, Based on the dense optical flow point set, the regional pixel point set, and the second target confidence, determine the license plate optimization coefficient in each frame image.

[0084] Optionally, the JS divergence of the dense optical flow point set and the regional pixel point set is calculated. Based on the JS divergence of each frame and the second target confidence, the license plate selection coefficient in each frame image is determined. The license plate selection coefficient ranges from 0 to 1.

[0085] In the above embodiments, the license plate selection coefficient in each frame image is determined by using dense optical flow point set, regional pixel point set, and second target confidence. This utilizes pixel-level correlation criteria, increases the constraints on the license plate selection coefficient, improves the accuracy of the license plate selection coefficient, and thus improves the reliability of the license plate selection in the entire scheme.

[0086] In one embodiment, such as Figure 7 As shown, before determining the license plate selection coefficient in each frame image based on the dense optical flow point set, the region pixel point set, and the second target confidence, considering that there may be static frames between consecutive frames in a vehicle capture event, it is also necessary to calculate the frame association weight between consecutive frames, including the following steps:

[0087] S702, based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, obtain the first number of optical flow coordinates in the license plate region of the previous frame and the second number of optical flow coordinates in the license plate region of the current frame.

[0088] S704, based on the positional relationship between the license plate area of ​​the previous frame and the license plate area of ​​the current frame, obtain the displacement values ​​of the center point of the license plate area of ​​the previous frame and the center point of the license plate area of ​​the current frame.

[0089] S706, based on the first quantity, the second quantity, and the displacement value, obtain the frame association weight between the license plate region of the previous frame and the license plate region of the current frame in each frame.

[0090] S708, based on the dense optical flow point set, the regional pixel point set, the second target confidence and the frame association weight, determine the license plate selection coefficient in each frame image.

[0091] In the above embodiments, considering the case where the preceding and following frames are static, a frame association weight is added when calculating the license plate selection coefficient, which further improves the reliability of the license plate selection coefficient.

[0092] In one example embodiment, a method for selecting license plates in a vehicle capture event is provided, specifically including the following steps:

[0093] Step 1: Obtain the correlation coefficient between vehicles and license plates in each frame of the image based on the dense optical flow algorithm.

[0094] S1.1, Calculate the first optical flow vector v corresponding to the license plate region of the previous frame in the current frame. p And the second optical flow vector v corresponding to the license plate area in the previous frame on the vehicle selection area in the previous frame. c And record the first target confidence score (cof) of the vehicle selection area in the previous frame. c ,cof c ∈(0,1).

[0095] S1.2, the first optical flow vector v p Second optical flow vector v c Unfold into one-dimensional data, and then use the second optical flow vector v c Transpose, calculate v p v c The cosine similarity is:

[0096]

[0097] S1.3, a relevant parameter ε = 0.001 is introduced into the cosine similarity, and the cosine similarity is normalized to obtain the correlation coefficient α between vehicles and license plates in each frame of the image:

[0098]

[0099] The larger the correlation coefficient α, the more relevant the license plate and the vehicle are, and the higher the credibility of the license plate.

[0100] Step 2: Based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, determine the license plate optimization coefficient of the license plate in each frame image.

[0101] S2.1, based on the first optical flow vector v of the license plate region in the previous frame in the current frame. p Calculate the optical flow coordinates of the current frame and the dense optical flow point set p corresponding to the optical flow coordinates. f .

[0102] S2.2, using a license plate detection deep learning model, obtain the second target confidence score (cof) for the license plate region in the current frame. p and the set of region pixels p of the license plate area in the current frame p .

[0103] S2.3, Calculate the dense optical flow point set p f and region pixel set p pThe JS divergence, combined with the second objective confidence cof p The theoretical license plate selection coefficient ω is calculated as follows:

[0104] ω=cof p *(1-JSD(p p ||p f )), ω∈(0,1).

[0105] S2.4, Considering the case where the preceding and following frames are stationary, count the first number M of optical flow coordinates falling within the license plate region of the preceding frame. n The second quantity M of optical flow coordinates falling in the license plate region of the current frame n+1 Based on the positional relationship between the license plate regions in the previous frame and the current frame, the displacement values ​​d of the center points of the license plate regions in the previous and current frames are obtained, and the frame association weight g is calculated.

[0106]

[0107] S2.5, let ω be the theoretical license plate selection coefficient of the previous frame. n The theoretical license plate selection coefficient for the current frame is ω. n+1 Combining the frame association weights, the license plate selection coefficient β is obtained:

[0108] β=g*ω n +(1-g)*ω n+1

[0109] The larger the β value, the higher the credibility of the license plate.

[0110] Step 3: Based on the correlation coefficient α and the license plate selection coefficient β, determine the license plate selection parameter λ for each frame image:

[0111] λ=αβ

[0112] Step 4: Select parameter λ based on each license plate, determine the image of the frame corresponding to the maximum value of λ, and use the license plate in that frame image as the target license plate.

[0113] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0114] Based on the same inventive concept, this application also provides a license plate selection device for implementing the license plate selection method in the vehicle capture event described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the license plate selection device in the vehicle capture event provided below can be found in the limitations of the license plate selection method in the vehicle capture event described above, and will not be repeated here.

[0115] In one embodiment, such as Figure 8 As shown, a license plate selection device for vehicle capture events is provided, comprising: a first acquisition module 82, a second determination module 84, a joint calculation module 86, and a license plate selection module 88, wherein:

[0116] The first acquisition module 82 is used to acquire the correlation coefficient between vehicles and license plates in each frame of the image based on the dense optical flow algorithm; and

[0117] The second determining module 84 is used to determine the license plate optimization coefficient in each frame image based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image.

[0118] The joint calculation module 86 is used to determine the license plate selection parameters in each frame image based on the correlation coefficient and the license plate selection coefficient.

[0119] The license plate selection module 88 is used to determine the image of the corresponding frame based on the license plate selection parameters, and to use the license plate in the image of that frame as the target license plate.

[0120] In one embodiment, the first acquisition module 82 includes:

[0121] Based on the dense optical flow algorithm, the first optical flow vector corresponding to the license plate region of the previous frame in the current frame and the second optical flow vector corresponding to the license plate region of the previous frame in the vehicle selection area of ​​the vehicle in the previous frame are obtained.

[0122] Based on a deep learning algorithm, the first target confidence score of the vehicle selection region in the previous frame is obtained;

[0123] Based on the first optical flow vector, the second optical flow vector, and the first target confidence, the correlation coefficient between the vehicle and the license plate in each frame image is obtained.

[0124] In one embodiment, the first acquisition module 82 includes:

[0125] The vehicle detection frame is divided into multiple regions, and the intersection of each region is the center point of the vehicle detection frame.

[0126] When the license plate area of ​​the previous frame falls within the area of ​​the vehicle detection frame, the license plate area of ​​the previous frame is shifted to the center point, and the shifted license plate area of ​​the previous frame becomes the vehicle selection area of ​​the previous frame.

[0127] In one embodiment, the second determining module 84 includes:

[0128] Based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, the optical flow coordinates of the license plate region of the previous frame in the current frame and the dense optical flow point set corresponding to the optical flow coordinates are obtained.

[0129] Based on deep learning algorithms, the second target confidence of the license plate region of the current frame and the set of region pixels corresponding to the license plate region of the current frame are obtained in each frame.

[0130] Based on the dense optical flow point set, the regional pixel point set, and the second target confidence level, the license plate optimization coefficient in each frame image is determined.

[0131] In one embodiment, the second determining module 84 includes:

[0132] Based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, a first number of optical flow coordinates in the license plate region of the previous frame and a second number of optical flow coordinates in the license plate region of the current frame are obtained;

[0133] Based on the positional relationship between the license plate area in the previous frame and the license plate area in the current frame, the displacement values ​​of the center point of the license plate area in the previous frame and the center point of the license plate area in the current frame are obtained;

[0134] Based on the first quantity, the second quantity, and the displacement value, obtain the frame association weight between the license plate region of the previous frame and the license plate region of the current frame in each frame;

[0135] Based on the dense optical flow point set, the regional pixel point set, the second target confidence, and the frame association weight, the license plate selection coefficient in each frame image is determined.

[0136] In one embodiment, the joint calculation module 86 includes: multiplying the correlation coefficient with the license plate selection coefficient to determine the license plate selection parameters in each frame image.

[0137] In one embodiment, the license plate selection module 88 includes: acquiring an image of the frame corresponding to the maximum value of each of the license plate selection parameters, and using the license plate in the frame image as the target license plate.

[0138] The various modules in the license plate selection device used in the aforementioned vehicle capture incident can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0139] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores vehicle capture data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a method for selecting license plates during vehicle capture events.

[0140] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0141] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0142] The correlation coefficient between vehicles and license plates in each frame of the image is obtained based on the dense optical flow algorithm; and

[0143] Based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, the license plate optimization coefficient of the license plate in each frame image is determined.

[0144] Based on the correlation coefficient and license plate selection coefficient, the license plate selection parameters in each frame image are determined;

[0145] Based on the license plate selection parameters, the image of the corresponding frame is determined, and the license plate in the image of that frame is used as the target license plate.

[0146] In one embodiment, the processor further performs the following steps when executing the computer program:

[0147] Based on the dense optical flow algorithm, the first optical flow vector corresponding to the license plate region of the previous frame in the current frame and the second optical flow vector corresponding to the license plate region of the previous frame in the vehicle selection area of ​​the vehicle in the previous frame are obtained.

[0148] Based on a deep learning algorithm, the first target confidence score of the vehicle selection region in the previous frame is obtained;

[0149] Based on the first optical flow vector, the second optical flow vector, and the first target confidence, the correlation coefficient between the vehicle and the license plate in each frame image is obtained.

[0150] In one embodiment, the processor further performs the following steps when executing the computer program:

[0151] The vehicle detection frame is divided into multiple regions, and the intersection of each region is the center point of the vehicle detection frame.

[0152] When the license plate area of ​​the previous frame falls within the area of ​​the vehicle detection frame, the license plate area of ​​the previous frame is shifted to the center point, and the shifted license plate area of ​​the previous frame becomes the vehicle selection area of ​​the previous frame.

[0153] In one embodiment, when the processor executes the computer program, it further implements the following steps: based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, it obtains the optical flow coordinates of the license plate region of the previous frame in the current frame and the dense optical flow point set corresponding to the optical flow coordinates.

[0154] Based on deep learning algorithms, the second target confidence of the license plate region of the current frame and the set of region pixels corresponding to the license plate region of the current frame are obtained in each frame.

[0155] Based on the dense optical flow point set, the regional pixel point set, and the second target confidence level, the license plate optimization coefficient in each frame image is determined.

[0156] In one embodiment, the processor further performs the following steps when executing the computer program:

[0157] Based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, a first number of optical flow coordinates in the license plate region of the previous frame and a second number of optical flow coordinates in the license plate region of the current frame are obtained;

[0158] Based on the positional relationship between the license plate area in the previous frame and the license plate area in the current frame, the displacement values ​​of the center point of the license plate area in the previous frame and the center point of the license plate area in the current frame are obtained;

[0159] Based on the first quantity, the second quantity, and the displacement value, obtain the frame association weight between the license plate region of the previous frame and the license plate region of the current frame in each frame;

[0160] Based on the dense optical flow point set, the regional pixel point set, the second target confidence, and the frame association weight, the license plate selection coefficient in each frame image is determined.

[0161] In one embodiment, when the processor executes the computer program, it further performs the following steps: multiplying the correlation coefficient with the license plate selection coefficient to determine the license plate selection parameters in each frame image.

[0162] In one embodiment, when the processor executes the computer program, it further performs the following steps: based on each of the license plate selection parameters, it obtains an image of the frame corresponding to the maximum value of the license plate selection parameters, and uses the license plate in the frame image as the target license plate.

[0163] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0164] The correlation coefficient between vehicles and license plates in each frame of the image is obtained based on the dense optical flow algorithm; and

[0165] Based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, the license plate optimization coefficient of the license plate in each frame image is determined.

[0166] Based on the correlation coefficient and license plate selection coefficient, the license plate selection parameters in each frame image are determined;

[0167] Based on the license plate selection parameters, the image of the corresponding frame is determined, and the license plate in the image of that frame is used as the target license plate.

[0168] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0169] Based on the dense optical flow algorithm, the first optical flow vector corresponding to the license plate region of the previous frame in the current frame and the second optical flow vector corresponding to the license plate region of the previous frame in the vehicle selection area of ​​the vehicle in the previous frame are obtained.

[0170] Based on a deep learning algorithm, the first target confidence score of the vehicle selection region in the previous frame is obtained;

[0171] Based on the first optical flow vector, the second optical flow vector, and the first target confidence, the correlation coefficient between the vehicle and the license plate in each frame image is obtained.

[0172] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0173] The vehicle detection frame is divided into multiple regions, and the intersection of each region is the center point of the vehicle detection frame.

[0174] When the license plate area of ​​the previous frame falls within the area of ​​the vehicle detection frame, the license plate area of ​​the previous frame is shifted to the center point, and the shifted license plate area of ​​the previous frame becomes the vehicle selection area of ​​the previous frame.

[0175] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, it obtains the optical flow coordinates of the license plate region of the previous frame in the current frame and the dense optical flow point set corresponding to the optical flow coordinates;

[0176] Based on deep learning algorithms, the second target confidence of the license plate region of the current frame and the set of region pixels corresponding to the license plate region of the current frame are obtained in each frame.

[0177] Based on the dense optical flow point set, the regional pixel point set, and the second target confidence level, the license plate optimization coefficient in each frame image is determined.

[0178] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0179] Based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, a first number of optical flow coordinates in the license plate region of the previous frame and a second number of optical flow coordinates in the license plate region of the current frame are obtained;

[0180] Based on the positional relationship between the license plate area in the previous frame and the license plate area in the current frame, the displacement values ​​of the center point of the license plate area in the previous frame and the center point of the license plate area in the current frame are obtained;

[0181] Based on the first quantity, the second quantity, and the displacement value, obtain the frame association weight between the license plate region of the previous frame and the license plate region of the current frame in each frame;

[0182] Based on the dense optical flow point set, the regional pixel point set, the second target confidence, and the frame association weight, the license plate selection coefficient in each frame image is determined.

[0183] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: multiplying the correlation coefficient by the license plate selection coefficient to determine the license plate selection parameters in each frame image.

[0184] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: based on each of the license plate selection parameters, obtains an image of the frame corresponding to the maximum value of the license plate selection parameters, and uses the license plate in the frame image as the target license plate.

[0185] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0186] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0187] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0188] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for selecting license plates in vehicle capture events, characterized in that, The method includes: The correlation coefficient between vehicles and license plates in each frame of the image is obtained based on the dense optical flow algorithm; and Based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, the license plate optimization coefficient of the license plate in each frame image is determined. Based on the correlation coefficient and license plate selection coefficient, the license plate selection parameters in each frame image are determined; Based on the license plate selection parameters, the image of the corresponding frame is determined, and the license plate in the image of that frame is used as the target license plate. The method of determining the license plate optimization coefficient in each frame of the image based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame includes: Based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, the optical flow coordinates of the license plate region of the previous frame in the current frame and the dense optical flow point set corresponding to the optical flow coordinates are obtained. Based on deep learning algorithms, the second target confidence of the license plate region of the current frame and the set of region pixels corresponding to the license plate region of the current frame are obtained in each frame. Based on the dense optical flow point set, the regional pixel point set, and the second target confidence level, the license plate optimization coefficient in each frame image is determined.

2. The method for selecting license plates in vehicle capture events according to claim 1, characterized in that, The correlation coefficient between vehicles and license plates in each frame of the image obtained based on the dense optical flow algorithm includes: Based on the dense optical flow algorithm, the first optical flow vector corresponding to the license plate region of the previous frame in the current frame and the second optical flow vector corresponding to the license plate region of the previous frame in the vehicle selection area of ​​the vehicle in the previous frame are obtained. Based on a deep learning algorithm, the first target confidence score of the vehicle selection region in the previous frame is obtained; Based on the first optical flow vector, the second optical flow vector, and the first target confidence, the correlation coefficient between the vehicle and the license plate in each frame image is obtained.

3. The method for selecting license plates in vehicle capture events according to claim 2, characterized in that, The previous frame vehicle selection area includes one of the front, rear, and body of the vehicle. Obtaining the previous frame vehicle selection area includes: The vehicle detection frame is divided into multiple regions, and the intersection of each region is the center point of the vehicle detection frame. When the license plate area of ​​the previous frame falls within the area of ​​the vehicle detection frame, the license plate area of ​​the previous frame is shifted to the center point, and the shifted license plate area of ​​the previous frame becomes the vehicle selection area of ​​the previous frame.

4. The method for selecting license plates in vehicle capture events according to claim 1, characterized in that, Before determining the license plate selection coefficient in each frame image based on the dense optical flow point set, the region pixel point set, and the second target confidence score, the process further includes: Based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image, a first number of optical flow coordinates in the license plate region of the previous frame and a second number of optical flow coordinates in the license plate region of the current frame are obtained; Based on the positional relationship between the license plate area in the previous frame and the license plate area in the current frame, the displacement values ​​of the center point of the license plate area in the previous frame and the center point of the license plate area in the current frame are obtained; Based on the first quantity, the second quantity, and the displacement value, obtain the frame association weight between the license plate region of the previous frame and the license plate region of the current frame in each frame; Based on the dense optical flow point set, the regional pixel point set, the second target confidence, and the frame association weight, the license plate selection coefficient in each frame image is determined.

5. The method for selecting license plates in vehicle capture events according to claim 1, characterized in that, The method for determining the license plate selection parameters in each frame image based on the correlation coefficient and the license plate selection coefficient includes: The correlation coefficient is multiplied by the license plate selection coefficient to determine the license plate selection parameters in each frame of the image.

6. The method for selecting license plates in vehicle capture events according to claim 1, characterized in that, The step of determining the image of the corresponding frame based on each of the license plate selection parameters, and using the license plate in that frame image as the target license plate, includes: Based on each of the license plate selection parameters, the image of the frame corresponding to the maximum value of the license plate selection parameter is obtained, and the license plate in that frame image is used as the target license plate.

7. A license plate selection device for vehicle capture events, characterized in that, The device includes: The first acquisition module is used to obtain the correlation coefficient between vehicles and license plates in each frame of the image based on the dense optical flow algorithm; and The second determining module is used to determine the license plate optimization coefficient in each frame image based on the optical flow consistency between the license plate region of the previous frame and the license plate region of the current frame in each frame image. The joint calculation module is used to determine the license plate selection parameters in each frame image based on the correlation coefficient and the license plate selection coefficient. The license plate selection module is used to determine the image of the corresponding frame based on the license plate selection parameters, and to use the license plate in the image of that frame as the target license plate; The second determining module includes: based on the optical flow consistency between the license plate region in the previous frame and the license plate region in the current frame in each frame image, obtaining the optical flow coordinates of the license plate region in the previous frame in the current frame and the dense optical flow point set corresponding to the optical flow coordinates; based on a deep learning algorithm, obtaining the second target confidence of the license plate region in the current frame in each frame and the regional pixel point set corresponding to the license plate region in the current frame; and based on the dense optical flow point set, the regional pixel point set, and the second target confidence, determining the license plate optimization coefficient of the license plate in each frame image.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images