Data processing system and method for video detection of parking lot

Abstract

The invention relates to the technical field of video data processing, and discloses a data processing system and method for parking lot video detection, and the system comprises an image processing module, an information extraction module, and an image matching module. Wherein the image processing module is used for detecting and extracting a target picture of a target vehicle from the monitoring video; the information extraction module is used for extracting license plate information of a target vehicle from the target picture and verifying the license plate information; if the license plate information fails to be extracted or the license plate information does not pass verification, triggering an image matching strategy; an image matching module responds to the image matching strategy and matches a reference picture for a target vehicle based on the target picture; the information extraction module re-extracts license plate information of the target vehicle based on the reference picture; the overall performance of the parking lot video detection system is improved, and the system is more intelligent and automatic.

Description

Technical Field

[0001] The present invention relates to the technical field of video data processing, and in particular to a data processing system and method for parking lot video detection. Background Art

[0002] In recent years, the number of cars has been increasing year by year, and the demand for parking lots has also increased. The development of computer vision, Internet of Things, big data and other technologies has led to the widespread application of intelligent parking management systems, especially the application of video surveillance technology, which not only improves the safety of parking lots, but also makes the parking process more convenient and efficient. However, in actual application, the existing parking lot video detection system still has many shortcomings.

[0003] First, one of the biggest challenges facing video surveillance systems is how to maintain stable performance in various complex environments. For example, factors such as changes in light, weather conditions (such as rain and snow), and poor lighting conditions at night can affect the effectiveness of video surveillance. Especially in an open environment such as a parking lot, vehicles frequently enter and exit, and the lighting conditions are unstable, which puts higher requirements on the video surveillance system. Secondly, traditional video surveillance systems often rely on a fixed-frequency image acquisition mode. Although this mode is simple and easy to implement, it has obvious defects in practical applications. In the absence of vehicle activity, maintaining a high frame rate acquisition not only wastes storage space, but also increases the cost of data processing. In the case of rapid vehicle movement or emergency, if the frame rate cannot be adjusted in time, it may lead to the loss of key information. Finally, as one of the core links in the intelligent parking management system, license plate recognition faces problems such as low recognition rate and high false alarm rate. Due to the different positions and angles of the license plates, as well as factors such as occlusion and reflection, the difficulty of license plate recognition increases. However, most existing license plate recognition technologies rely on a single algorithm or model, which is difficult to cope with complex and changeable actual scenarios.

[0004] At present, most intelligent parking management systems still require a certain degree of manual intervention. For example, when the license plate information cannot be read correctly, the staff needs to manually input the information, which not only reduces work efficiency but also increases operating costs. Especially during peak hours, the inefficiency of manual confirmation will seriously affect the user experience.

[0005] For example, the patent application with publication number CN117315564A discloses a parking lot management method, device, computer equipment and storage medium, and the method is implemented by: collecting video data to be analyzed of the target parking lot, wherein the video data to be analyzed includes at least one target vehicle; performing structured analysis on the video data to be analyzed to determine whether the target vehicle has abnormal traffic behavior; if so, determining the current number of vehicles entering and exiting the target parking lot based on the abnormal type of the abnormal traffic behavior; and determining the current number of remaining parking spaces and the number of remaining vehicles in the target parking lot based on the current number of vehicles entering and exiting the parking lot. Accurate control of the parking lot can be achieved based on the abnormal types of abnormal traffic behavior of the target vehicles, which can effectively avoid serious distortion of the number of remaining parking spaces displayed on the guide screen. At the same time, it can avoid some safety issues, facilitate the refined operation of the parking lot, and increase revenue and profits.

[0006] For example, the patent application with publication number CN118521941A discloses a vehicle entry and exit management method and device, and the method implementation includes: receiving video data from multiple video data acquisition units installed in multiple perspectives, and the multiple video data acquisition units are used to collect video data of multiple areas to be detected; fusing the video data of each area to be detected to obtain the video data to be detected; tracking the target vehicle in the video data to be detected to obtain the vehicle tracking result of the target vehicle; determining the behavior characteristics of the target vehicle based on the vehicle tracking result; and managing the entry and exit of the target vehicle based on the behavior characteristics. In the embodiment of this application, the panoramic video data of the area to be detected is obtained by fusing the video data of different areas to be detected, and the behavior characteristics of the vehicle are determined from the panoramic video data, so as to accurately control the entry and exit of the vehicle based on the behavior characteristics, and realize the refined management of the parking lot's vehicle entry and exit management.

[0007] The above patents all have the problem raised by the background technology: they rely on a fixed-frequency image acquisition mode, which not only wastes storage space but also increases the cost of data processing.

[0008] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgement or any form of suggestion that the information constitutes the prior art already known to ordinary technicians in this field. Summary of the invention

[0009] The technical problem to be solved by the present invention is to overcome the defects of the prior art, provide a data processing system and method for parking lot video detection, improve the overall performance of the parking lot video detection system, and make the system more intelligent and automated.

[0010] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0011] In one aspect, the present invention provides a data processing system for parking lot video detection, comprising an image processing module, an information extraction module, and an image matching module; wherein:

[0012] The image processing module is used to adaptively control the monitoring video of the parking lot and detect and extract the target image of the target vehicle from the monitoring video;

[0013] The information extraction module is used to extract the license plate information of the target vehicle from the target image and verify the license plate information; if the license plate information extraction fails or the license plate information fails to pass the verification, the image matching strategy is triggered;

[0014] The image matching module responds to the image matching strategy and matches a reference image for a target vehicle based on the target image; the information extraction module re-extracts the license plate information of the target vehicle based on the reference image.

[0015] As a preferred solution of the data processing system for parking lot video detection of the present invention, wherein: the image processing module includes a vehicle detection unit and a preprocessing unit; wherein:

[0016] The vehicle detection unit is used to detect a target vehicle from a surveillance video of the parking lot and extract a target image of the target vehicle;

[0017] The preprocessing unit is used to preprocess the target image, and the preprocessing includes filtering and noise reduction, image enhancement, shadow removal, and image cropping.

[0018] As a preferred solution of the data processing system for parking lot video detection of the present invention, the vehicle detection unit detects the target vehicle from the monitoring video of the parking lot in the following manner:

[0019] Extract surveillance images from surveillance videos of parking lots;

[0020] Input the extracted surveillance image into a pre-created background subtractor to perform background subtraction to obtain a foreground mask, and perform binarization processing on the foreground mask to obtain a motion foreground image;

[0021] De-noising the motion foreground image by morphological operation;

[0022] Perform contour detection on the denoised motion foreground image and mark the motion area;

[0023] The area of ​​the moving area is calculated, and it is determined whether the target vehicle appears in the monitoring image, specifically as follows: the vehicle detection unit is configured with a detection threshold of the moving area; when the area of ​​the moving area is greater than the detection threshold, the target vehicle appears in the monitoring image, otherwise, the target vehicle does not appear in the monitoring image.

[0024] As a preferred solution of the data processing system for parking lot video detection of the present invention, wherein: the image processing module further includes a monitoring adjustment unit, and the monitoring adjustment unit adaptively adjusts the real-time frame rate of the monitoring video based on the detection result of the target vehicle;

[0025] The monitoring and adjustment unit is configured with a frame rate control strategy as follows: If the target vehicle does not appear in the monitoring image within seconds, the real-time frame rate of the monitoring video is set to the first frame rate; if the target vehicle appears in the monitoring image, the real-time frame rate of the monitoring video is set to the second frame rate.

[0026] As a preferred solution of the data processing system for parking lot video detection of the present invention, the vehicle detection unit is further configured with an image extraction strategy, which is as follows: when the frame rate of the parking lot surveillance video is the first frame rate, Extract one frame of surveillance image from the surveillance video of the parking lot every second, and detect the target vehicle in the extracted surveillance image; is the preset first time interval; when the frame rate of the parking lot surveillance video is the second frame rate, then every Extract one frame of monitoring image from the monitoring video of the parking lot every second, detect the target vehicle from the extracted monitoring image, and if the target vehicle appears in the monitoring image, mark the monitoring image as a target image, and transmit the target image to the preprocessing unit; is a preset second time interval.

[0027] As a preferred solution of the data processing system for parking lot video detection of the present invention, the information extraction module includes a positioning unit, an identification unit, and a verification unit; wherein:

[0028] The positioning unit is used to locate the license plate area in the target image; if the license plate area is not successfully located, the extraction of the license plate information fails;

[0029] The recognition unit is used to recognize the license plate information of the target vehicle from the license plate area; if the license plate information cannot be successfully recognized, the extraction of the license plate information fails;

[0030] The verification unit is configured with an information verification strategy for verifying the license plate information, specifically as follows: extracting the license plate information from n consecutive target images of the same target vehicle, where n is a positive integer; if the license plate information extracted from the n target images are all consistent, the license plate information passes the verification; otherwise, the license plate information fails the verification.

[0031] As a preferred solution of the data processing system for parking lot video detection of the present invention, the image matching module includes a picture retrieval unit, a target detection unit, and a matching unit; wherein:

[0032] The image retrieval unit is used to retrieve surveillance images from surveillance videos at different locations in the parking lot;

[0033] The target detection unit is used to detect and segment the to-be-matched pictures containing vehicles from the surveillance images at different locations of the parking lot;

[0034] The matching unit is used to filter out reference pictures containing a target vehicle from the pictures to be matched containing vehicles.

[0035] As a preferred solution of the data processing system for parking lot video detection of the present invention, the method in which the matching unit determines whether any image to be matched contains the target vehicle is as follows:

[0036] S100: Marking feature points in the target image and the image to be matched respectively by using a feature detection algorithm;

[0037] S200: performing feature matching on the feature points in the target image and the feature points in the image to be matched to obtain matching point pairs;

[0038] S300: Perform consistency check on the matching point pairs to obtain consistent matching point pairs in the target image and the image to be matched;

[0039] S400: Counting the number of consistent matching point pairs in the target image and the image to be matched, and calculating the similarity score between the image to be matched and the target image;

[0040] S500: The matching unit is configured with a matching threshold. If the similarity score is higher than the matching threshold, the corresponding image to be matched contains the target vehicle.

[0041] As a preferred solution of the data processing system for parking lot video detection of the present invention, the method of performing consistency check on the matching point pairs and obtaining consistent matching point pairs in the target image and the image to be matched is as follows:

[0042] S301: Initialize a homography matrix H;

[0043] S302: Randomly select 4 pairs of matching points between the target image and the image to be matched as a sample set;

[0044] S303: Solving the homography matrix H based on the coordinates of the feature points of the matching point pairs in the sample set;

[0045] S304: Mapping feature points in any matching point pair in the target image to the image to be matched through the homography matrix H; if the Euclidean distance between the mapped feature point and the corresponding feature point in the image to be matched is less than a preset consistency threshold, the matching point pair is marked as a consistent matching point pair;

[0046] S305: Counting the number of consistent matching point pairs in the target image and the image to be matched;

[0047] S306: Repeat the iterations from S301 to S306 M times, and count the number of consistent matching point pairs in each iteration;

[0048] S307: Obtain consistent matching point pairs in the iteration with the largest number of consistent matching point pairs.

[0049] As a preferred solution of the data processing system for parking lot video detection of the present invention, the calculation formula of the similarity score is as follows:

[0050] ;

[0051] in, Represents the similarity score between the image to be matched and the target image; Indicates the horizontal coordinate of the feature point in the image to be matched in the i-th pair of consistent matching points; the value range of i is 1, 2, ..., N; where N represents the number of consistent matching point pairs in the target image and the image to be matched; Represents the ordinate of the feature point in the image to be matched in the i-th pair of consistent matching points; Represents the horizontal coordinate of the feature point in the target image in the i-th pair of consistent matching points; It represents the ordinate of the feature point in the target image in the i-th pair of consistent matching points; w represents the width of the target image; h represents the height of the target image.

[0052] As a preferred solution of the data processing system for parking lot video detection of the present invention, the matching unit sends no less than k reference images to the information extraction module; after receiving the reference image of the target vehicle, the information extraction module triggers the information re-verification strategy, which is as follows:

[0053] The positioning unit locates the license plate area in the target image;

[0054] The recognition unit recognizes the license plate information of the target vehicle from the license plate area;

[0055] The verification unit verifies the license plate information, counts the number of times each type of license plate information is extracted from k reference images, and takes the license plate information with the most occurrences as the final license plate information extraction result.

[0056] In a second aspect, the present invention provides a data processing method for parking lot video detection, comprising the following steps:

[0057] S1: Detect and extract the target image of the target vehicle from the parking lot surveillance video, and adaptively control the parking lot surveillance video;

[0058] S2: extracting the license plate information of the target vehicle from the target image, and verifying the license plate information;

[0059] S3: If the license plate information is successfully extracted and verified, return to S1; otherwise, execute S4;

[0060] S4: matching a reference image for a target vehicle based on the target image;

[0061] S5: Re-extract the license plate information of the target vehicle based on the reference image, and return to S1.

[0062] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0063] Through the adaptive frame rate adjustment mechanism, the system can reduce the frame rate when there is no vehicle activity, reduce unnecessary data volume, and save computing resources. When a vehicle appears, the frame rate is automatically increased to ensure that all the details of the vehicle's movements are captured, thereby achieving effective use of resources.

[0064] The background subtraction technology combined with binarization processing can effectively distinguish the moving area, and the target vehicle can be accurately detected even when the light conditions change greatly, which improves the robustness and reliability of the system. The information extraction module ensures the accurate extraction and verification of license plate information through the three-step method of positioning unit, recognition unit, and verification unit. In particular, the consistency verification of multiple consecutive pictures greatly improves the accuracy of license plate information.

[0065] When license plate information extraction fails, the system automatically starts the image matching strategy, retrieves surveillance images from other locations in the parking lot, and automatically re-extracts the target vehicle’s license plate information, reducing the need for manual intervention and improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0067] Figure 1A schematic diagram of the structure of a data processing system for parking lot video detection provided by the present invention;

[0068] Figure 2 A flow chart of a data processing method for parking lot video detection provided by the present invention;

[0069] Figure 3 A flow chart of a method for detecting a target vehicle from a surveillance video of a parking lot provided by the present invention. DETAILED DESCRIPTION

[0070] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0071] Example 1

[0072] This embodiment introduces a data processing system for parking lot video detection. Figure 1 The system includes an image processing module, an information extraction module, and an image matching module; wherein:

[0073] The image processing module is used to adaptively control the monitoring video of the parking lot and detect and extract the target image of the target vehicle from the monitoring video;

[0074] The image processing module includes a vehicle detection unit, a monitoring adjustment unit, and a preprocessing unit; wherein: the vehicle detection unit is used to detect a target vehicle from a monitoring video of a parking lot and extract a target image of the target vehicle;

[0075] Reference Figure 3 , the way to detect the target vehicle from the surveillance video of the parking lot is as follows:

[0076] Extract surveillance images from surveillance videos of parking lots;

[0077] The extracted surveillance image is input into a pre-created background subtractor for background subtraction to obtain a foreground mask, and the foreground mask is binarized to obtain a motion foreground image; some image processing tools, such as MOG2 in OpenCV, have the function of creating a background subtractor, which can remove the static background area in the surveillance image, and then highlight the non-background part, that is, the motion area, through binarization. The motion area is likely to be the target vehicle;

[0078] De-noising the motion foreground image by morphological operations; for example, removing small noise points in the motion foreground image by dilation and erosion operations;

[0079] Perform contour detection on the denoised motion foreground image and mark the motion area to determine the location and size of the motion area;

[0080] The area of ​​the moving area is calculated, and it is determined whether the target vehicle appears in the monitoring image, specifically as follows: the vehicle detection unit is configured with a detection threshold of the moving area; when the area of ​​the moving area is greater than the detection threshold, the target vehicle appears in the monitoring image, otherwise, the target vehicle does not appear in the monitoring image.

[0081] The target vehicle is detected by the above-mentioned background subtraction method, which has good robustness and can achieve accurate detection of the target vehicle under various lighting conditions.

[0082] The monitoring adjustment unit adaptively adjusts the real-time frame rate of the monitoring video based on the detection result of the target vehicle;

[0083] The monitoring and adjustment unit is configured with a frame rate control strategy as follows: If the target vehicle does not appear in the monitoring image within seconds, the real-time frame rate of the monitoring video is set to the first frame rate, such as 5 frames, where m is a positive integer; if the target vehicle appears in the monitoring image, the real-time frame rate of the monitoring video is set to the second frame rate, such as 30 frames.

[0084] The goal of frame rate adjustment is to reduce unnecessary data volume and save computing resources while meeting recognition accuracy. In the absence of vehicle activity, the frame rate can be reduced because it is not necessary to capture image information at a high frequency. When a vehicle is detected entering or leaving the parking lot, the frame rate is automatically increased to ensure that all the details of the vehicle's movements can be captured.

[0085] The vehicle detection unit is also configured with an image extraction strategy, which is as follows: when the frame rate of the surveillance video of the parking lot is the first frame rate, Extract one frame of surveillance image from the surveillance video of the parking lot every second, and detect the target vehicle in the extracted surveillance image; is a preset first time interval, for example, 1.5 seconds; when the frame rate of the parking lot surveillance video is the second frame rate, then every Extract one frame of monitoring image from the monitoring video of the parking lot every second, detect the target vehicle from the extracted monitoring image, and if the target vehicle appears in the monitoring image, mark the monitoring image as a target image, and transmit the target image to the preprocessing unit; is a preset second time interval, such as 0.5 seconds;

[0086] Based on the above-mentioned adaptive frame rate control, the frame rate of the surveillance video can be adjusted according to the actual application scenario, which not only ensures the timely detection of the target vehicle but also avoids data redundancy.

[0087] The preprocessing unit is used to preprocess the target image, and the preprocessing includes filtering noise reduction, image enhancement, shadow removal, and image cropping; the preprocessing unit is configured with an image noise reduction algorithm (such as non-local mean denoising, bilateral filtering, etc.), which can remove noise in the target image; the preprocessing unit is also configured with an image enhancement algorithm, such as histogram equalization, contrast enhancement, etc., which can further improve the image quality of the target image; in addition, the preprocessing unit is also configured with a shadow removal algorithm (such as Retinex algorithm), which can reduce the impact of ambient light changes on the clarity of the target image; finally, after the above preprocessing, the preprocessing unit crops the target image into a specified size through a preset cropping frame to facilitate subsequent information extraction and image matching.

[0088] The information extraction module is used to extract the license plate information of the target vehicle from the target image and verify the license plate information; if the license plate information extraction fails or the license plate information fails to pass the verification, the image matching strategy is triggered;

[0089] The information extraction module includes a positioning unit, an identification unit, and a verification unit; wherein:

[0090] The positioning unit is used to locate the license plate area in the target image; if the license plate area is not successfully located, the extraction of the license plate information fails; the positioning unit is configured with a positioning algorithm, such as a template matching-based method, which can accurately locate the position of the license plate, but the positioning may fail due to occlusion, insufficient light, etc.

[0091] The recognition unit is used to recognize the license plate information of the target vehicle from the license plate area; if the license plate information cannot be successfully recognized, the extraction of the license plate information fails; the recognition unit is configured with an OCR recognition algorithm, which can extract text information based on image recognition.

[0092] The verification unit is configured with an information verification strategy for verifying the license plate information, specifically as follows: extracting the license plate information from n consecutive target images of the same target vehicle, where n is a positive integer; if the license plate information extracted from the n target images are all consistent, the license plate information passes the verification; otherwise, the license plate information fails the verification.

[0093] The image matching module responds to the image matching strategy and matches a reference image for a target vehicle based on the target image; the information extraction module re-extracts the license plate information of the target vehicle based on the reference image.

[0094] The image matching module includes a picture retrieval unit, a target detection unit, and a matching unit; wherein:

[0095] The image retrieval unit is used to retrieve surveillance images from surveillance videos at different locations in the parking lot; the extraction of the target vehicle's license plate information is generally performed at the exit and entrance of the parking lot in order to calculate the parking time; if the target vehicle's license plate information cannot be effectively extracted from the surveillance video at the entrance and exit of the parking lot, manual confirmation is required, which is time-consuming and labor-intensive; therefore, the surveillance images of other locations (these locations may have captured clear images of the license plate) are retrieved by the image retrieval unit, and the target vehicle's license plate information is automatically extracted through subsequent processes, which can significantly improve the efficiency of license plate information confirmation.

[0096] The method of retrieving surveillance images from surveillance videos at other locations in the parking lot is as follows: assuming that the license plate information extraction of the target vehicle fails at the parking lot exit, select camera A at other locations in the parking lot, calculate the distance from camera A to the parking lot exit, and estimate the time point when the target vehicle passes camera A based on the average speed of the vehicle in the parking lot. Retrieving surveillance images taken by camera A before and after this time point can effectively find the target vehicle.

[0097] The target detection unit is used to detect and segment the images to be matched containing vehicles from the surveillance images at different locations in the parking lot; the target detection unit is configured with a pre-trained deep learning model, such as YOLOv4, SSD, Faster R-CNN, etc., which can efficiently detect vehicles from the surveillance images; then the vehicle area is segmented to obtain the images to be matched containing vehicles; wherein the size of the segmented images to be matched is consistent with the size of the cropping frame preset in the pre-processing unit, ensuring that the target image and the reference image are the same size. Downloading a pre-trained deep learning model for target detection from the Internet, and then training and fine-tuning the model using a data set of vehicle images under various angles and lighting conditions, can improve the model's vehicle recognition capabilities in complex environments and greatly save the cost of building and training the deep learning model.

[0098] The matching unit is used to filter out reference images containing the target vehicle from the images to be matched containing the vehicle; for any image to be matched, the method for determining whether it contains the target vehicle is as follows:

[0099] S100: Mark feature points in the target image and the image to be matched respectively through feature detection algorithms; the feature detection algorithms that can be used include SIFT (Scale Invariant Feature Transform), SURF (Speeded Robust Features), ORB (Oriented FAST and Rotated BRIEF), etc.

[0100] S200: Perform feature matching on the feature points in the target image and the feature points in the image to be matched to obtain matching point pairs; use the BFMatcher or FLANN Matcher algorithm to find corresponding points between the feature points of the two images.

[0101] S300: Perform consistency check on the matching point pairs to obtain consistent matching point pairs in the target image and the image to be matched; the method is as follows:

[0102] S301: Initialize a homography matrix H; for a planar scene, a homography matrix is ​​usually used to describe the relationship between two images. The homography matrix H is a 3x3 matrix that can map points on one plane to the vicinity of corresponding points on another plane.

[0103] S302: Randomly select 4 pairs of matching point pairs between the target image and the image to be matched as a sample set; in order to estimate the 3x3 homography matrix, theoretically at least four sets of matching point pairs are required.

[0104] S303: Solving the homography matrix H based on the coordinates of the feature points of the matching point pairs in the sample set; the homography matrix H can be determined by solving a linear equation group constructed by the coordinates of four sets of matching point pairs.

[0105] S304: Mapping feature points in any matching point pair in the target image to the image to be matched through the homography matrix H; if the Euclidean distance between the mapped feature point and the corresponding feature point in the image to be matched is less than a preset consistency threshold, the matching point pair is marked as a consistent matching point pair;

[0106] S305: Count the number of consistent matching point pairs in the target image and the image to be matched; the larger the number of consistent matching point pairs is, the more suitable the homography matrix H is for describing the geometric transformation relationship between the target image and the image to be matched.

[0107] S306: Repeat iteratively execute S301 to S306 M times, and count the number of consistent matching point pairs in each iteration; M is a positive integer, such as 1000;

[0108] S307: Obtain consistent matching point pairs in the iteration with the largest number of consistent matching point pairs.

[0109] S400: Count the number of consistent matching point pairs in the target image and the image to be matched, and calculate the similarity score between the image to be matched and the target image, the formula is as follows:

[0110] ;

[0111] in, Represents the similarity score between the image to be matched and the target image; Indicates the horizontal coordinate of the feature point in the image to be matched in the i-th pair of consistent matching points; the value range of i is 1, 2, ..., N; where N represents the number of consistent matching point pairs in the target image and the image to be matched; Represents the ordinate of the feature point in the image to be matched in the i-th pair of consistent matching points; Represents the horizontal coordinate of the feature point in the target image in the i-th pair of consistent matching points; represents the ordinate of the feature point in the target image in the i-th pair of consistent matching points; w represents the width of the target image; h represents the height of the target image;

[0112] S500: The matching unit is configured with a matching threshold. If the similarity score is higher than the matching threshold, the corresponding image to be matched contains the target vehicle.

[0113] The matching unit sends no less than k reference images to the information extraction module; after receiving the reference image of the target vehicle, the information extraction module triggers the information re-verification strategy, which is as follows:

[0114] The positioning unit locates the license plate area in the target image;

[0115] The recognition unit recognizes the license plate information of the target vehicle from the license plate area;

[0116] The verification unit verifies the license plate information, counts the number of times each type of license plate information is extracted from k reference images, and takes the license plate information with the most occurrences as the final license plate information extraction result.

[0117] Example 2

[0118] This embodiment is the second embodiment of the present invention; it is based on the same inventive concept as embodiment 1, Figure 2 This embodiment introduces a data processing method for parking lot video detection, including the following steps:

[0119] S1: Detect and extract the target image of the target vehicle from the surveillance video of the parking lot, and adaptively control the surveillance video of the parking lot; adaptively adjust the real-time frame rate of the surveillance video, reduce the frame rate when no vehicle is detected, and increase the frame rate when a vehicle is detected.

[0120] S2: extracting the license plate information of the target vehicle from the target image, and verifying the license plate information; verifying the consistency of the license plate information, if the license plate information in several consecutive images is consistent, the verification is passed.

[0121] S3: If the license plate information is successfully extracted and verified, return to S1; otherwise, execute S4; if it returns to S1, prepare to detect the next target vehicle.

[0122] S4: Matching a reference image for the target vehicle based on the target image; retrieving images that may contain clear license plate information from surveillance videos at different locations in the parking lot. Using a deep learning model to detect and segment the images to be matched that contain the vehicle; using a feature detection algorithm to mark feature points in the target image and the image to be matched, and perform feature matching; calculating a similarity score through consistency testing, and screening out reference images that contain the target vehicle.

[0123] S5: Re-extract the license plate information of the target vehicle based on the reference image and return to S1. Use the successfully matched reference image to re-execute the license plate information extraction and verification process, and return to S1 to continue processing the new surveillance video stream after completion.

[0124] The specific functions of the above modules are realized by referring to the relevant contents of the data processing system for parking lot video detection described in Example 1, which will not be elaborated here.

[0125] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the purpose and scope of protection of the present invention, which are all within the protection of the present invention.

Claims

1. A data processing system for parking lot video detection, characterized in that: It includes image processing module, information extraction module and image matching module; among which: The image processing module is used to adaptively control the monitoring video of the parking lot and detect and extract the target image of the target vehicle from the monitoring video; The information extraction module is used to extract the license plate information of the target vehicle from the target image and verify the license plate information; if the license plate information extraction fails or the license plate information fails to pass the verification, the image matching strategy is triggered; The image matching module responds to the image matching strategy and matches a reference image for a target vehicle based on the target image; the information extraction module re-extracts the license plate information of the target vehicle based on the reference image.

2. A data processing system for parking lot video detection as claimed in claim 1, characterized in that: The image processing module includes a vehicle detection unit and a preprocessing unit; wherein: The vehicle detection unit is used to detect a target vehicle from a surveillance video of the parking lot and extract a target image of the target vehicle; The preprocessing unit is used to preprocess the target image, and the preprocessing includes filtering and noise reduction, image enhancement, shadow removal, and image cropping.

3. A data processing system for parking lot video detection as claimed in claim 2, characterized in that: The vehicle detection unit detects the target vehicle from the surveillance video of the parking lot in the following manner: Extract surveillance images from surveillance videos of parking lots; Input the extracted surveillance image into a pre-created background subtractor to perform background subtraction to obtain a foreground mask, and perform binarization processing on the foreground mask to obtain a motion foreground image; De-noising the motion foreground image by morphological operation; Perform contour detection on the denoised motion foreground image and mark the motion area; The area of ​​the moving area is calculated, and it is determined whether the target vehicle appears in the monitoring image, specifically as follows: the vehicle detection unit is configured with a detection threshold of the moving area; when the area of ​​the moving area is greater than the detection threshold, the target vehicle appears in the monitoring image, otherwise, the target vehicle does not appear in the monitoring image.

4. A data processing system for parking lot video detection as claimed in claim 3, characterized in that: The image processing module also includes a monitoring adjustment unit, which adaptively adjusts the real-time frame rate of the monitoring video based on the detection result of the target vehicle; The monitoring and adjustment unit is configured with a frame rate control strategy as follows: If the target vehicle does not appear in the monitoring image within seconds, the real-time frame rate of the monitoring video is set to the first frame rate; if the target vehicle appears in the monitoring image, the real-time frame rate of the monitoring video is set to the second frame rate, and m is a positive integer.

5. A data processing system for parking lot video detection as claimed in claim 4, characterized in that: The vehicle detection unit is also configured with an image extraction strategy as follows: when the frame rate of the monitoring video of the parking lot is the first frame rate, Extract one frame of surveillance image from the surveillance video of the parking lot every second, and detect the target vehicle in the extracted surveillance image; is the preset first time interval; when the frame rate of the parking lot surveillance video is the second frame rate, then every Extract one frame of monitoring image from the monitoring video of the parking lot every second, detect the target vehicle from the extracted monitoring image, and if the target vehicle appears in the monitoring image, mark the monitoring image as a target image, and transmit the target image to the preprocessing unit; is a preset second time interval.

6. A data processing system for parking lot video detection as claimed in claim 5, characterized in that: The information extraction module includes a positioning unit, an identification unit, and a verification unit; wherein: The positioning unit is used to locate the license plate area in the target image; if the license plate area is not successfully located, the extraction of the license plate information fails; The recognition unit is used to recognize the license plate information of the target vehicle from the license plate area; if the license plate information cannot be successfully recognized, the extraction of the license plate information fails; The verification unit is configured with an information verification strategy for verifying the license plate information, specifically as follows: extracting the license plate information from n consecutive target images of the same target vehicle, where n is a positive integer; if the license plate information extracted from the n target images are all consistent, the license plate information passes the verification; otherwise, the license plate information fails the verification.

7. A data processing system for parking lot video detection as claimed in claim 6, characterized in that: The image matching module includes a picture retrieval unit, a target detection unit, and a matching unit; wherein: The image retrieval unit is used to retrieve surveillance images from surveillance videos at different locations in the parking lot; The target detection unit is used to detect and segment the to-be-matched pictures containing vehicles from the surveillance images at different locations of the parking lot; The matching unit is used to filter out reference pictures containing a target vehicle from the pictures to be matched containing vehicles.

8. A data processing system for parking lot video detection as claimed in claim 7, characterized in that: For any image to be matched, the matching unit determines whether it contains the target vehicle as follows: S100: Marking feature points in the target image and the image to be matched respectively by using a feature detection algorithm; S200: performing feature matching on feature points in the target image and feature points in the image to be matched to obtain matching point pairs; S300: Perform consistency check on the matching point pairs to obtain consistent matching point pairs in the target image and the image to be matched; S400: Counting the number of consistent matching point pairs in the target image and the image to be matched, and calculating the similarity score between the image to be matched and the target image; S500: The matching unit is configured with a matching threshold. If the similarity score is higher than the matching threshold, the corresponding image to be matched contains the target vehicle.

9. A data processing system for parking lot video detection as claimed in claim 8, characterized in that: The method of performing consistency check on the matching point pairs to obtain consistent matching point pairs in the target image and the image to be matched is as follows: S301: Initialize a homography matrix H; S302: Randomly select 4 pairs of matching points between the target image and the image to be matched as a sample set; S303: Solving the homography matrix H based on the coordinates of the feature points of the matching point pairs in the sample set; S304: Mapping feature points in any matching point pair in the target image to the image to be matched through the homography matrix H; If the Euclidean distance between the mapped feature point and the corresponding feature point in the image to be matched is less than the preset consistency threshold, the matching point pair is marked as a consistent matching point pair; S305: Counting the number of consistent matching point pairs in the target image and the image to be matched; S306: Repeat the iterations from S301 to S306 M times, and count the number of consistent matching point pairs in each iteration; S307: Obtain consistent matching point pairs in the iteration with the largest number of consistent matching point pairs.

10. A data processing system for parking lot video detection as claimed in claim 9, characterized in that: The calculation formula of the similarity score is as follows: ； in, Represents the similarity score between the image to be matched and the target image; Indicates the horizontal coordinate of the feature point in the image to be matched in the i-th pair of consistent matching points; the value range of i is 1, 2, ..., N; where N represents the number of consistent matching point pairs in the target image and the image to be matched; Represents the ordinate of the feature point in the image to be matched in the i-th pair of consistent matching points; Represents the horizontal coordinate of the feature point in the target image in the i-th pair of consistent matching points; It represents the ordinate of the feature point in the target image in the i-th pair of consistent matching points; w represents the width of the target image; h represents the height of the target image.

11. A data processing system for parking lot video detection as claimed in claim 10, characterized in that: The matching unit sends no less than k reference images to the information extraction module; after receiving the reference image of the target vehicle, the information extraction module triggers the information re-verification strategy, which is as follows: The positioning unit locates the license plate area in the target image; The recognition unit recognizes the license plate information of the target vehicle from the license plate area; The verification unit verifies the license plate information, counts the number of times each type of license plate information is extracted from k reference images, and takes the license plate information with the most occurrences as the final license plate information extraction result.

12. A data processing method for parking lot video detection, implemented based on the data processing system for parking lot video detection according to any one of claims 1 to 11, characterized in that: The following steps are involved: S1: Detect and extract the target image of the target vehicle from the surveillance video of the parking lot, and adaptively control the surveillance video of the parking lot; S2: extracting the license plate information of the target vehicle from the target image, and verifying the license plate information; S3: If the license plate information is successfully extracted and verified, return to S1; Otherwise, execute S4; S4: matching a reference image for a target vehicle based on the target image; S5: Re-extract the license plate information of the target vehicle based on the reference image, and return to S1.

Images