License plate recognition method and device, equipment, storage medium and program product

Abstract

The invention relates to a license plate recognition method, device and equipment, a storage medium and a program product, and relates to the field of intelligent traffic systems. The method comprises the following steps: acquiring video data collected for at least one parking space; for each parking space, determining a state type of the parking space based on the video data, and obtaining a historical video clip corresponding to the parking space from the video data according to a state starting time point of the state type; based on the historical video clip, extracting target feature information of a target vehicle which enables the parking space to belong to the state type; if the target vehicle is determined to be located in the parking space based on the historical video clip, taking the target feature information of the target vehicle as the initial feature of single target tracking; and performing multi-feature matching on all the vehicles in the historical video clip based on the initial features to realize single-target tracking of the target vehicle in the parking space so as to determine the license plate number of the target vehicle. By adopting the method, the accuracy of license plate recognition can be improved under the condition of relatively low computing power requirements.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation systems, and in particular, to a license plate recognition method, device, equipment, storage medium, and program product. Background Art

[0002] License plate recognition technology in a parking scenario belongs to the field of intelligent transportation systems, and it is the product of the cross-integration of multiple disciplines such as computer vision technology, pattern recognition technology, communication technology, and automatic control technology. License plate recognition technology specifically involves using an image acquisition device (such as a high-definition camera) to obtain vehicle images in a parking scenario, processing the images through computer vision algorithms, extracting the license plate area and recognizing the license plate number, and then transmitting the recognition result to a management system through communication technology for operations such as parking timing, charging, and vehicle management.

[0003] In traditional license plate recognition methods, the monitoring video for parking spaces is mainly collected through a camera, and then the license plate numbers of the vehicles located in the parking spaces are recognized in real time based on the monitoring video, which belongs to license plate recognition under real-time video. However, when recognizing the license plate numbers of vehicles located in parking spaces in real time based on the monitoring video, the accuracy of license plate recognition is relatively low, and the computing power requirement for the processor is relatively high. Summary of the Invention

[0004] Based on this, it is necessary to provide a license plate recognition method, device, equipment, storage medium, and program product that can improve the accuracy of license plate recognition under low computing power requirements for the above technical problems.

[0005] In a first aspect, the present application provides a license plate recognition method, and the method includes:

[0006] Obtain video data collected for at least one parking space;

[0007] For each of the parking spaces, determine the status type of the parking space based on the video data, and obtain the historical video segment corresponding to the parking space from the video data according to the start time point of the status of the status type;

[0008] Extract the target feature information of the target vehicle that makes the parking space belong to the status type based on the historical video segment;

[0009] If it is determined based on the historical video segment that the target vehicle is located within the parking space, use the target feature information of the target vehicle as the initial feature for single-object tracking;

[0010] Perform multi-feature matching on all vehicles in the historical video segment based on the initial feature to implement single-object tracking of the target vehicle in the parking space, so as to determine the license plate number of the target vehicle.

[0011] In one embodiment, obtaining a historical video segment corresponding to the parking space from the video data according to the state start time point of the state type includes:

[0012] If the state type of the parking space is vehicle driving in, obtain a video segment corresponding to a preset time period before the state start time point of the state type from the video data as the historical video segment corresponding to the parking space;

[0013] If the state type of the parking space is vehicle driving out or vehicle parked stably, obtain a video segment corresponding to a preset time period after the state start time point of the state type from the video data as the historical video segment corresponding to the parking space.

[0014] In one embodiment, the method further includes the step of determining whether the target vehicle is located within the parking space based on the historical video segment. Determining whether the target vehicle is located within the parking space based on the historical video segment includes:

[0015] For each historical video frame in the historical video segment, determine the parking space area of the parking space and the vehicle area of the target vehicle in the historical video frame;

[0016] Determine whether the target vehicle is located within the parking space according to the positional relationship between the parking space area and the vehicle area.

[0017] In one embodiment, determining whether the target vehicle is located within the parking space according to the positional relationship between the parking space area and the vehicle area includes:

[0018] If the center point of the vehicle area is located within the parking space area, determine that the target vehicle is located within the parking space; otherwise, determine that the target vehicle is located outside the parking space.

[0019] In one embodiment, determining whether the target vehicle is located within the parking space according to the positional relationship between the parking space area and the vehicle area includes:

[0020] Determine the overlapping area and the union area of the parking space area and the vehicle area;

[0021] Based on the ratio of the overlapping area to the union area, determine the overlapping degree of the parking space area and the vehicle area;

[0022] If the overlapping degree is greater than a preset threshold, determine that the target vehicle is located within the parking space; otherwise, determine that the target vehicle is located outside the parking space.

[0023] In one embodiment, it is applied to a hyper-converged edge intelligent agent including multiple cameras; performing multi-feature matching on all vehicles in the historical video segment based on the initial features to achieve single-object tracking of the target vehicle in the parking space, so as to determine the license plate number of the target vehicle, including:

[0024] For each vehicle in each historical video frame in the historical video segment, if the vehicle in the historical video frame is located in the central region of the field of view of the target camera that collected the historical video segment, perform feature matching on the vehicle in the historical video frame based on the initial features to obtain the matching score of the vehicle in the historical video frame;

[0025] If the vehicle in the historical video frame is located in the edge region of the field of view of the target camera that collected the historical video segment, and the driving direction of the vehicle in the historical video frame is the field of view region of the remaining cameras, obtain the historical video segment collected by the remaining cameras, and perform feature matching on the vehicle in the relevant historical video frames based on the initial features to obtain the matching score of the vehicle in the relevant historical video frames; the relevant historical video frames are the video frames with the same collection time as the historical video frame in the historical video segment collected by the remaining cameras;

[0026] For each target historical video frame, determine the vehicles with matching scores greater than the preset score in the target historical video frame as the matched vehicles, and update the license plate count for the matched vehicles; the target historical video frame is the historical video frame or the relevant historical video frame;

[0027] Determine the license plate number of the vehicle with the largest license plate count among the matched vehicles as the license plate number of the target vehicle.

[0028] In a second aspect, the present application provides a license plate recognition device, and the device includes:

[0029] An acquisition module, configured to acquire video data collected for at least one parking space;

[0030] A determination module, configured to determine the status type of each parking space based on the video data;

[0031] The acquisition module is further configured to obtain the historical video segment corresponding to the parking space from the video data according to the start time point of the status of the status type;

[0032] An extraction module, configured to extract the target feature information of the target vehicle that makes the parking space belong to the status type based on the historical video segment;

[0033] The determining module is further configured to, if it is determined based on the historical video segment that the target vehicle is located within the parking space, use the target feature information of the target vehicle as the initial feature for single-object tracking; perform multi-feature matching on all vehicles in the historical video segment based on the initial feature to implement single-object tracking of the target vehicle in the parking space, so as to determine the license plate number of the target vehicle.

[0034] In a third aspect, the present application provides a computer device, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the method embodiments of the present application are implemented.

[0035] In a fourth aspect, the present application provides a computer-readable storage medium, storing a computer program, and when the computer program is executed by a processor, the steps in the method embodiments of the present application are implemented.

[0036] In a fifth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps in the method embodiments of the present application are implemented.

[0037] The above license plate recognition method, device, equipment, storage medium, and program product obtain video data collected for at least one parking space; for each parking space, determine the state type of the parking space based on the video data, and according to the start time point of the state of the state type, obtain the historical video segment corresponding to the parking space from the video data; based on the historical video segment, extract the target feature information of the target vehicle that makes the parking space belong to the state type; if it is determined based on the historical video segment that the target vehicle is located within the parking space, use the target feature information of the target vehicle as the initial feature for single-object tracking; perform multi-feature matching on all vehicles in the historical video segment based on the initial feature to implement single-object tracking of the target vehicle in the parking space, so as to determine the license plate number of the target vehicle. Compared with the traditional license plate recognition method, the present application obtains the historical video segment corresponding to the parking space from the video data according to the start time point of the state of the state type of the parking space, and performs single-object tracking on all vehicles in the historical video segment by playing back the historical video segment to identify the license plate number of the target vehicle that makes the parking space belong to the state type, which can improve the accuracy of license plate recognition under the condition of lower computing power requirements. Description of the Drawings

[0038] Figure 1 It is a schematic flowchart of the license plate recognition method in an embodiment;

[0039] Figure 2 It is a schematic flowchart of the license plate recognition method in another embodiment;

[0040] Figure 3The structural block diagram of a license plate recognition device in an embodiment;

[0041] Figure 4 The internal structure diagram of a computer device in an embodiment;

[0042] Figure 5 The internal structure diagram of a computer device in another embodiment. Detailed implementation manners

[0043] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] In one embodiment, as Figure 1 shown, a license plate recognition method is provided. In this embodiment, the method is applied to a computer device as an example, and includes the following steps:

[0045] Step 102, obtaining video data collected for at least one parking space.

[0046] Specifically, the field of view of the camera includes at least one parking space. The camera can perform real-time video collection for at least one parking space, obtain corresponding video data, and send the collected video data to the computer device. The computer device can receive the video data collected by the camera for at least one parking space area within its field of view.

[0047] In one embodiment, the computer device can be a hyper-converged edge intelligent agent including multiple cameras, that is, the license plate recognition method of the present application can be applied to a hyper-converged edge intelligent agent including multiple cameras. The hyper-converged edge intelligent agent can be installed on a roadside street lamp pole for license plate recognition of roadside parked vehicles in a roadside parking scenario.

[0048] In one embodiment, the camera can be a low-position camera. A low-position camera refers to a camera installed in a low-position area of a target object. A low-position area refers to an area in the target object with a height lower than a preset height. The target object can include roadside lamp poles and buildings, etc. It can be understood that compared with traditional high-position cameras, the installation position of the low-position camera in the present application is lower, and the installation and maintenance costs are lower.

[0049] Step 104, for each parking space, determining the status type of the parking space based on the video data, and obtaining the historical video segment corresponding to the parking space from the video data according to the start time point of the status of the status type.

[0050] Among them, the status types of the parking spaces include vehicle entry, vehicle exit, or vehicle parked stably. Vehicle entry means the vehicle enters the parking space. Vehicle exit means the vehicle leaves the parking space. Vehicle parked stably means the vehicle stays stably in the parking space. The status start time point refers to the time point when the vehicle starts to enter the corresponding status of the corresponding status type.

[0051] In one embodiment, for each parking space, the computer device can analyze each video frame in the video data through a vehicle detection algorithm to determine the status type of the parking space. The computer device can extract the corresponding historical video segment of the parking space from the video data according to the status start time point of the status type. It can be understood that the historical video segment is a sub-segment in the video data.

[0052] Step 106: Based on the historical video segment, extract the target feature information of the target vehicle that makes the parking space belong to the status type.

[0053] Among them, the target feature information of the target vehicle includes the vehicle feature information and license plate feature information of the target vehicle. The vehicle feature information includes at least one of the coordinate information of the vehicle, vehicle feature value, color, brand, and model of the vehicle. The license plate feature information includes at least one of the license plate number, license plate color, license plate type, and coordinate information of the license plate.

[0054] In one embodiment, the computer device can obtain the coordinate information of the vehicle and the license plate through a target detection algorithm, obtain the vehicle feature value of the vehicle through a feature extraction method, obtain the structured information of the vehicle and the license plate through a multi-attribute recognition algorithm, that is, including the above-mentioned color, brand, model, and license plate type, and obtain the license plate number through an OCR (Optical Character Recognition) recognition algorithm.

[0055] Step 108: If it is determined based on the historical video segment that the target vehicle is located within the parking space, then use the target feature information of the target vehicle as the initial feature for single-object tracking.

[0056] In one embodiment, the computer device can determine whether the target vehicle is located within the parking space based on the historical video segment. If it is determined based on the historical video segment that the target vehicle is located within the parking space, then the computer device can use the target feature information of the target vehicle as the initial feature for single-object tracking. Among them, single-object tracking means tracking only one vehicle at the same time point.

[0057] Step 110: Perform multi-feature matching on all vehicles in the historical video segment based on the initial feature to achieve single-object tracking of the target vehicle in the parking space, so as to determine the license plate number of the target vehicle.

[0058] In one embodiment, for each vehicle among all the vehicles included in the historical video clip, the computer device can extract the feature information of the vehicle based on the historical video clip. The computer device can compare the feature information of the vehicle with the initial features to perform single-object tracking on the vehicle. It can be understood that by performing single-object tracking on all the vehicles included in the historical video clip respectively, the license plate number of the target vehicle can be determined. It can be understood that the target vehicle is one of all the vehicles included in the historical video clip, and by replaying the historical video clip for single-object tracking, the license plate number of the target vehicle can be accurately identified.

[0059] In the above license plate recognition method, video data collected for at least one parking space is obtained; for each parking space, the state type of the parking space is determined based on the video data, and according to the start time point of the state of the state type, the historical video clip corresponding to the parking space is obtained from the video data; based on the historical video clip, the target feature information of the target vehicle that makes the parking space belong to the state type is extracted; if it is determined based on the historical video clip that the target vehicle is within the parking space, the target feature information of the target vehicle is used as the initial feature for single-object tracking; multi-feature matching is performed on all the vehicles in the historical video clip based on the initial feature to achieve single-object tracking of the target vehicle in the parking space, so as to determine the license plate number of the target vehicle. Compared with the traditional license plate recognition method, in this application, by obtaining the historical video clip corresponding to the parking space from the video data according to the start time point of the state of the state type of the parking space, and performing single-object tracking on all the vehicles in the historical video clip respectively by replaying the historical video clip to identify the license plate number of the target vehicle that makes the parking space belong to the state type, the accuracy of license plate recognition can be improved under the condition of relatively low computing power requirements.

[0060] In one embodiment, obtaining the historical video clip corresponding to the parking space from the video data according to the start time point of the state of the state type includes: if the state type of the parking space is vehicle entry, obtaining the video clip corresponding to the preset time period before the start time point of the state of the state type from the video data as the historical video clip corresponding to the parking space; if the state type of the parking space is vehicle exit or vehicle parked stably, obtaining the video clip corresponding to the preset time period after the start time point of the state of the state type from the video data as the historical video clip corresponding to the parking space.

[0061] Specifically, the status types of the parking space include vehicle entry, vehicle exit, and vehicle parked stably. If the status type of the parking space is vehicle entry, the computer device can obtain, from the video data, the video clip corresponding to the preset time period before the status start time point of the status type as the historical video clip corresponding to the parking space. If the status type of the parking space is vehicle exit or vehicle parked stably, the computer device can obtain, from the video data, the video clip corresponding to the preset time period after the status start time point of the status type as the historical video clip corresponding to the parking space.

[0062] For example, if the status type of the parking space is vehicle entry, the computer device can obtain, from the video data, the video clip of the 1 minute before the status start time point of the status type, and use the video clip of this 1 minute during the process of the vehicle entering the parking space as the historical video clip corresponding to the parking space. If the status type of the parking space is vehicle exit or vehicle parked stably, the computer device can obtain, from the video data, the video clip of the 1 minute after the status start time point of the status type, and use the video clip of this 1 minute during the process of the vehicle exiting the parking space, or the video clip of this 1 minute after the vehicle is stably parked in the parking space as the historical video clip corresponding to the parking space.

[0063] In the above embodiments, in the case of vehicle entry, by using the video clip corresponding to the preset time period before the status start time point in the video data as the historical video clip corresponding to the parking space, the license plate of the vehicle can be recognized during the vehicle entry process by playing back the historical video clip, further improving the license plate recognition accuracy. In the case of vehicle exit, by using the video clip corresponding to the preset time period after the status start time point in the video data as the historical video clip corresponding to the parking space, the license plate of the vehicle can be recognized during the vehicle exit process by playing back the historical video clip, further improving the license plate recognition accuracy. In the case of vehicle parked stably, by using the video clip corresponding to the preset time period after the status start time point in the video data as the historical video clip corresponding to the parking space, the license plate of the vehicle located in the parking space can be recognized within a certain time interval by playing back the historical video clip. It can be understood that, for example, within a certain time interval, the vehicle located in front of the parking space may drive away and the obscured license plate may be photographed, further improving the license plate recognition accuracy.

[0064] In one embodiment, the method further includes the step of determining whether the target vehicle is located within the parking space based on the historical video clip. Determining whether the target vehicle is located within the parking space based on the historical video clip includes: for each historical video frame in the historical video clip, determining the parking space area of the parking space and the vehicle area of the target vehicle in the historical video frame; and determining whether the target vehicle is located within the parking space according to the positional relationship between the parking space area and the vehicle area.

[0065] In one embodiment, the parking space area can be a polygon Region of Interest (ROI) pre-set for the parking space. The vehicle area can be a rectangular frame area of the vehicle identified through a vehicle detection algorithm (e.g., a deep learning algorithm of yoloV8).

[0066] In the above embodiment, determining whether the target vehicle is located within the parking space based on the positional relationship between the parking space area and the vehicle area in each historical video frame can improve the accuracy of judging whether the target vehicle is located within the parking space.

[0067] In one embodiment, determining whether the target vehicle is located within the parking space based on the positional relationship between the parking space area and the vehicle area includes: if the center point of the vehicle area is located within the parking space area, it is determined that the target vehicle is located within the parking space; otherwise, it is determined that the target vehicle is located outside the parking space.

[0068] Specifically, if the center point of the vehicle area is located within the parking space area, the computer device can determine that the target vehicle is located within the parking space. If the center point of the vehicle area is located outside the parking space area, the computer device can determine that the target vehicle is located outside the parking space.

[0069] In the above embodiment, by judging whether the center point of the vehicle area is located within the parking space area to determine whether the target vehicle is located within the parking space, the accuracy of judging whether the target vehicle is located within the parking space can be further improved.

[0070] In one embodiment, determining whether the target vehicle is located within the parking space based on the positional relationship between the parking space area and the vehicle area includes: determining the overlapping area and the union area between the parking space area and the vehicle area; based on the ratio of the overlapping area to the union area, determining the overlapping degree between the parking space area and the vehicle area; if the overlapping degree is greater than a preset threshold, it is determined that the target vehicle is located within the parking space; otherwise, it is determined that the target vehicle is located outside the parking space.

[0071] Among them, the overlapping area (Area of Overiap) is the area of the intersection part between the parking space area and the vehicle area, and the union area (Area of Union) is the total area after the parking space area and the vehicle area are combined.

[0072] In one embodiment, the computer device can calculate the overlapping area and the union area between the parking space area and the vehicle area, calculate the ratio of the overlapping area to the union area, and directly use the ratio of the overlapping area to the union area as the overlapping degree between the parking space area and the vehicle area. If the overlapping degree is greater than the preset threshold, the computer device can determine that the target vehicle is located within the parking space; if the overlapping degree is less than or equal to the preset threshold, the computer device can determine that the target vehicle is located outside the parking space.

[0073] In the above embodiments, by determining whether the overlap degree between the parking space area and the vehicle area is greater than a preset threshold to determine whether the target vehicle is within the parking space, the accuracy of determining whether the target vehicle is within the parking space can be further improved.

[0074] In one embodiment, as Figure 2 shown, it is applied to a hyper-converged edge intelligent agent including multiple cameras; multi-feature matching is performed on all vehicles in the historical video segment based on the initial features to achieve single-object tracking of the target vehicle in the parking space to determine the license plate number of the target vehicle, including:

[0075] Step 202, for each vehicle in each historical video frame in the historical video segment, if the vehicle in the historical video frame is located in the central region of the field of view of the target camera that collected the historical video segment, feature matching is performed on the vehicle in the historical video frame based on the initial features to obtain the matching score of the vehicle in the historical video frame;

[0076] Step 204, if the vehicle in the historical video frame is located in the edge region of the field of view of the target camera that collected the historical video segment, and the driving direction of the vehicle in the historical video frame is towards the field of view region of the remaining cameras, obtain the historical video segments collected by the remaining cameras, and perform feature matching on the vehicle in the relevant historical video frames based on the initial features to obtain the matching score of the vehicle in the relevant historical video frames; the relevant historical video frames are the video frames with the same collection time as the historical video frame in the historical video segments collected by the remaining cameras;

[0077] Step 206, for each target historical video frame, determine the vehicles with a matching score greater than the preset score in the target historical video frame as the matched vehicles, and update the license plate count for the matched vehicles; the target historical video frame is the historical video frame or the relevant historical video frame;

[0078] Step 208, determine the license plate number of the vehicle with the largest license plate count among the matched vehicles as the license plate number of the target vehicle.

[0079] In one embodiment, if the status type of the parking space is vehicle entry or vehicle parked stably, the historical video segment corresponding to the parking space is played in the forward order. If the status type of the parking space is vehicle exit, the historical video segment corresponding to the parking space is played in the reverse order to perform single-object tracking on all vehicles in the historical video segment respectively, so as to determine the license plate number of the target vehicle.

[0080] In one embodiment, initialize the matching score as score = 0, and the single-object tracking may include the following steps:

[0081] a. Vehicle feature information and license plate feature information of the current target vehicle, where the current target vehicle refers to each vehicle in each historical video frame of the historical video segment;

[0082] b. License plate number comparison: If the initial feature has a license plate number and the current target vehicle also has a license plate number, then compare the license plate number of the initial feature with the license plate number of the current target vehicle. If the number of identical license plate numbers is m, then score = score + m;

[0083] c. Feature value comparison: Calculate the similarity between the feature value of the initial feature and the feature value of the current target vehicle (which can be calculated using Euclidean distance or cosine distance), and normalize it to simi, with a value range of 0 - 10. Then score = score + simi;

[0084] d. Structured feature comparison: For vehicle color, license plate color, vehicle type, license plate type, vehicle brand, for each successfully matched feature, score = score + 1;

[0085] e. Bounding box comparison: Calculate the overlap IOU between the bounding box of the vehicle area of the initial feature and the bounding box of the vehicle area of the current target vehicle, and normalize it to 0 - 10. Then score = score + IOU;

[0086] f. Find the maximum score. If the score is greater than the preset score, it is determined as a successful match. Save the license plate number of the matched vehicle and compare it with the stored license plates. For the same license plate, add 1, that is, the license plate count + 1. Finally, determine the license plate number of the matched vehicle with the largest license plate count as the license plate number of the target vehicle.

[0087] In one embodiment, if the center point of the vehicle area of the vehicle in the historical video frame is within the field - of - view edge area of the target camera that captured the historical video segment, it is determined that the vehicle in the historical video frame is in the field - of - view edge area of the target camera that captured the historical video segment.

[0088] In one embodiment, if the direction of the center point of the vehicle area of the vehicle in the historical video frame is the field - of - view area of the remaining camera, it is determined that the driving direction of the vehicle in the historical video frame is the field - of - view area of the remaining camera.

[0089] In the above - mentioned embodiments, by playing back the historical video segment, single - target tracking, and single - target cross - domain tracking, the complexity of multi - target tracking in real - time video processing is simplified, the computing power requirement of the processor is reduced, and at the same time, the problem of low license plate accuracy caused by license plate occlusion is solved, and the license plate recognition accuracy is improved.

[0090] It should be understood that although the steps in the flowcharts of the above embodiments are shown in sequence, these steps are not necessarily executed in sequence. Unless there is a clear indication in this document, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0091] In one embodiment, as Figure 3 shown, a license plate recognition device 300 is provided, which specifically includes:

[0092] An acquisition module 302, configured to acquire video data collected for at least one parking space;

[0093] A determination module 304, configured to determine the status type of the parking space based on the video data for each parking space;

[0094] The acquisition module 302 is further configured to acquire a historical video segment corresponding to the parking space from the video data according to the start time point of the status of the status type;

[0095] An extraction module 306, configured to extract target feature information of a target vehicle that makes the parking space belong to the status type based on the historical video segment;

[0096] The determination module 304 is further configured to, if it is determined based on the historical video segment that the target vehicle is located within the parking space, use the target feature information of the target vehicle as the initial feature for single-object tracking; perform multi-feature matching on all vehicles in the historical video segment based on the initial feature to implement single-object tracking of the target vehicle in the parking space, so as to determine the license plate number of the target vehicle.

[0097] In one embodiment, the acquisition module 302 is further configured to, if the status type of the parking space is vehicle entry, acquire a video segment corresponding to a preset time period before the start time point of the status of the status type from the video data as the historical video segment corresponding to the parking space; if the status type of the parking space is vehicle exit or vehicle parked stably, acquire a video segment corresponding to a preset time period after the start time point of the status of the status type from the video data as the historical video segment corresponding to the parking space.

[0098] In one embodiment, the determination module 304 is further configured to, for each historical video frame in the historical video segment, determine the parking space area of the parking space and the vehicle area of the target vehicle in the historical video frame; and determine whether the target vehicle is located within the parking space according to the positional relationship between the parking space area and the vehicle area.

[0099] In one embodiment, the determination module 304 is further configured to determine that the target vehicle is located within the parking space if the center point of the vehicle area is located within the parking space area; otherwise, determine that the target vehicle is located outside the parking space.

[0100] In one embodiment, the determination module 304 is further configured to determine the overlapping area and the union area of the parking space area and the vehicle area; determine the overlapping degree of the parking space area and the vehicle area based on the ratio of the overlapping area to the union area; and determine that the target vehicle is located within the parking space if the overlapping degree is greater than a preset threshold; otherwise, determine that the target vehicle is located outside the parking space.

[0101] In one embodiment, it is applied to a hyper-converged edge intelligent agent including multiple cameras; the determination module 304 is further configured to, for each vehicle in each historical video frame in the historical video segment, if the vehicle in the historical video frame is located in the central region of the field of view of the target camera that captures the historical video segment, perform feature matching on the vehicle in the historical video frame based on the initial features to obtain the matching score of the vehicle in the historical video frame; if the vehicle in the historical video frame is located in the edge region of the field of view of the target camera that captures the historical video segment, and the driving direction of the vehicle in the historical video frame is towards the field of view region of the remaining cameras, obtain the historical video segment captured by the remaining cameras, and perform feature matching on the vehicle in the relevant historical video frame based on the initial features to obtain the matching score of the vehicle in the relevant historical video frame; the relevant historical video frame is the video frame with the same capture time as the historical video frame in the historical video segment captured by the remaining cameras; for each target historical video frame, determine the vehicle with a matching score greater than the preset score in the target historical video frame as the matched vehicle, and update the license plate count for the matched vehicle; the target historical video frame is the historical video frame or the relevant historical video frame; determine the license plate number of the vehicle with the largest license plate count among the matched vehicles as the license plate number of the target vehicle.

[0102] The above license plate recognition device obtains video data collected for at least one parking space; for each parking space, determines the status type of the parking space based on the video data, and obtains the corresponding historical video segment of the parking space from the video data according to the status start time point of the status type; based on the historical video segment, extracts the target feature information of the target vehicle that makes the parking space belong to the status type; if it is determined based on the historical video segment that the target vehicle is located within the parking space, uses the target feature information of the target vehicle as the initial feature for single-object tracking; performs multi-feature matching on all vehicles in the historical video segment based on the initial feature to implement single-object tracking of the target vehicle in the parking space, so as to determine the license plate number of the target vehicle. Compared with the traditional license plate recognition method, in this application, by obtaining the corresponding historical video segment of the parking space from the video data according to the status start time point of the status type of the parking space, and performing single-object tracking on all vehicles in the historical video segment respectively by playing back the historical video segment to identify the license plate number of the target vehicle that makes the parking space belong to the status type, the accuracy of license plate recognition can be improved under the condition of relatively low computing power requirements.

[0103] Each module in the above license plate recognition device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor in the computer device in hardware form or be independent of it, or can be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0104] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 4 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used for the processor to exchange information with external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a license plate recognition method.

[0105] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 5As shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program, when executed by the processor, implements a license plate recognition method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0106] Those skilled in the art can understand that Figure 4 and Figure 5 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0107] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0108] In one embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0109] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0110] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0111] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0112] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as these technical feature combinations do not conflict, they should be considered as within the scope described in this specification.

[0113] The above-described embodiments merely represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application patent should be subject to the appended claims.

Claims

1. A license plate recognition method, characterized in that, The method includes: Obtaining video data collected for at least one parking space; For each of the parking spaces, determining the status type of the parking space based on the video data, and obtaining the corresponding historical video segment of the parking space from the video data according to the status start time point of the status type; Based on the historical video segment, extracting target feature information of a target vehicle that makes the parking space belong to the status type; If it is determined based on the historical video segment that the target vehicle is within the parking space, using the target feature information of the target vehicle as the initial feature for single-object tracking; Performing multi-feature matching on all vehicles in the historical video segment based on the initial feature to implement single-object tracking of the target vehicle in the parking space, so as to determine the license plate number of the target vehicle.

2. The method according to claim 1, characterized in that, The obtaining the corresponding historical video segment of the parking space from the video data according to the status start time point of the status type includes: If the status type of the parking space is vehicle entering, obtaining the video segment corresponding to a preset time period before the status start time point of the status type from the video data as the corresponding historical video segment of the parking space; If the status type of the parking space is vehicle leaving or vehicle parked stably, obtaining the video segment corresponding to a preset time period after the status start time point of the status type from the video data as the corresponding historical video segment of the parking space.

3. The method according to claim 1, characterized in that The method further includes the step of determining whether the target vehicle is within the parking space based on the historical video segment. The determining whether the target vehicle is within the parking space based on the historical video segment includes: For each historical video frame in the historical video segment, determining the parking space area of the parking space and the vehicle area of the target vehicle in the historical video frame; Determining whether the target vehicle is within the parking space according to the positional relationship between the parking space area and the vehicle area.

4. The method according to claim 3, wherein The determining whether the target vehicle is within the parking space according to the positional relationship between the parking space area and the vehicle area includes: If the center point of the vehicle area is within the parking space area, determining that the target vehicle is within the parking space; otherwise, determining that the target vehicle is outside the parking space.

5. The method according to claim 3, characterized in that, The determining whether the target vehicle is within the parking space according to the positional relationship between the parking space area and the vehicle area includes: Determining the overlapping area and the union area of the parking space area and the vehicle area; Based on the ratio of the overlapping area to the union area, determining the overlapping degree of the parking space area and the vehicle area; If the overlapping degree is greater than a preset threshold, determining that the target vehicle is within the parking space; otherwise, determining that the target vehicle is outside the parking space.

6. The method according to any one of claims 1 to 5, characterized in that Applied to a hyper-converged edge intelligent agent including multiple cameras; the performing multi-feature matching on all vehicles in the historical video segment based on the initial feature to implement single-object tracking of the target vehicle in the parking space, so as to determine the license plate number of the target vehicle, includes: For each vehicle in each historical video frame of the historical video segment, if the vehicle in the historical video frame is located in the central region of the field of view of the target camera that captured the historical video segment, perform feature matching on the vehicle in the historical video frame based on the initial features to obtain the matching score of the vehicle in the historical video frame; If the vehicle in the historical video frame is located in the edge region of the field of view of the target camera that captured the historical video segment, and the driving direction of the vehicle in the historical video frame is towards the field of view region of the remaining cameras, obtain the historical video segment captured by the remaining cameras, and perform feature matching on the vehicle in the relevant historical video frames based on the initial features to obtain the matching score of the vehicle in the relevant historical video frames; the relevant historical video frames are the video frames in the historical video segment captured by the remaining cameras that have the same capture time as the historical video frame; For each target historical video frame, determine the vehicles with a matching score greater than the preset score in the target historical video frame as the matched vehicles, and update the license plate count for the matched vehicles; the target historical video frame is the historical video frame or the relevant historical video frame; Determine the license plate number of the vehicle with the largest license plate count among the matched vehicles as the license plate number of the target vehicle.

7. A license plate recognition device, characterized in that, The apparatus includes: An acquisition module, configured to acquire video data collected for at least one parking space; A determination module, configured to determine the status type of each parking space based on the video data; The acquisition module is further configured to obtain the historical video segment corresponding to the parking space from the video data according to the start time point of the status of the status type; An extraction module, configured to extract the target feature information of the target vehicle that makes the parking space belong to the status type based on the historical video segment; The determination module is further configured to, if it is determined based on the historical video segment that the target vehicle is located within the parking space, use the target feature information of the target vehicle as the initial feature for single-object tracking; perform multi-feature matching on all vehicles in the historical video segment based on the initial feature to implement single-object tracking of the target vehicle in the parking space, so as to determine the license plate number of the target vehicle.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images