A data processing system and method for video detection in a parking lot

Abstract

The present invention relates to the technical field of video data processing, and discloses a data processing system and method for parking lot video detection. The system includes an image processing module, an information extraction module, and an image matching module. Specifically: The image processing module is used to detect and extract a target picture of a target vehicle from the monitoring video; the information extraction module is used to extract the license plate information of the target vehicle from the target picture and verify the license plate information; if the extraction of the license plate information fails or the license plate information fails to pass the verification, an image matching strategy is triggered; the image matching module responds to the image matching strategy and matches a reference picture for the target vehicle based on the target picture; the information extraction module re-extracts the license plate information of the target vehicle based on the reference picture. The present invention improves the overall performance of the parking lot video detection system, making the system more intelligent and automated.

Description

Technical Field

[0001] The present invention relates to the technical field of video data processing, and particularly relates to a data processing system and method for parking lot video detection. Background Art

[0002] In recent years, the number of automobiles in possession has been increasing year by year, and the demand for parking lots has also grown accordingly. The development of technologies such as computer vision, Internet of Things, and big data has enabled the wide application of intelligent parking lot management systems. In particular, the application of video surveillance technology not only improves the security of parking lots but also makes the parking process more convenient and efficient. However, in the actual application process, there are still many deficiencies in the existing parking lot video detection systems.

[0003] Firstly, one of the greatest challenges faced by video surveillance systems is how to maintain stable performance in various complex environments. For example, factors such as changes in light, weather conditions (such as rain and snow), and poor night lighting conditions will all affect the effect of video surveillance. Especially in an open environment like a parking lot, where vehicles come and go frequently and the lighting conditions are unstable, this poses higher requirements for video surveillance systems. Secondly, traditional video surveillance systems often rely on a fixed-frequency image acquisition mode. Although this mode is simple and easy to implement, it has obvious defects in actual applications. When there is no vehicle activity, maintaining high-frame-rate acquisition not only wastes storage space but also increases the cost of data processing. And in the case of rapid vehicle movement or emergencies, if the frame rate cannot be adjusted in time, it may lead to the loss of key information. Finally, license plate recognition, as one of the core links in the intelligent parking lot management system, faces problems such as low recognition rate and high false alarm rate. Due to different positions and angles of license plates, as well as factors such as occlusion and reflection, it increases the difficulty of license plate recognition. And most of the existing license plate recognition technologies rely on a single algorithm or model and are difficult to handle complex and changeable actual scenarios.

[0004] Currently, most intelligent parking lot management systems still require a certain degree of manual intervention. For example, when the license plate information cannot be read correctly, staff need to manually input the information, which not only reduces work efficiency but also increases operating costs. Especially during peak hours, the low efficiency of manual confirmation will seriously affect the user experience.

[0005] A patent application with publication number CN117315564A discloses a parking lot management method, device, computer device, and storage medium. The implementation of the method includes: collecting video data to be analyzed of a target parking lot, where the video data to be analyzed includes at least one target vehicle; performing structured analysis on the video data to be analyzed to determine whether the target vehicle has abnormal passing behavior; if so, based on the abnormal type of the abnormal passing behavior, determining the current number of incoming and outgoing vehicles in the target parking lot; and based on the current number of incoming and outgoing vehicles, determining the current remaining number of parking spaces and the remaining number of vehicles in the target parking lot. According to the abnormal types of different abnormal passing behaviors of the target vehicle, accurate control of the parking lot can be achieved, effectively avoiding serious distortion of the remaining number of parking spaces displayed on the guidance screen. At the same time, some safety problems can be avoided, facilitating the refined operation of the parking lot and increasing revenue and profit.

[0006] A patent application with publication number CN118521941A discloses a vehicle incoming and outgoing management method and device. The implementation of the method includes: receiving video data from multiple video data acquisition units installed from multiple perspectives, where the multiple video data acquisition units are used to collect video data of multiple areas to be detected; fusing the video data of each area to be detected to obtain video data to be detected; performing vehicle tracking on the target vehicle in the video data to be detected to obtain the vehicle tracking result of the target vehicle; determining the behavior characteristics of the target vehicle based on the vehicle tracking result; and performing incoming and outgoing management on the target vehicle based on the behavior characteristics. In the embodiments of this application, by fusing the video data of different areas to be detected, panoramic video data of the areas to be detected is obtained, and the behavior characteristics of the vehicle are determined from the panoramic video data, so as to accurately control the incoming and outgoing of the vehicle based on the behavior characteristics and achieve refined management of the vehicle incoming and outgoing management in the parking lot.

[0007] The above patents all have the problems raised in this background technology: relying on the image acquisition mode with a fixed frequency not only wastes storage space but also increases the cost of data processing.

[0008] The information disclosed in this background technology section is only intended to increase the understanding of the overall background of the present invention and should not be regarded as an admission or any form of implication that this information constitutes prior art already known to those of ordinary skill in the art. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a data processing system and method for parking lot video detection, so as to improve the overall performance of the parking lot video detection system and make the system more intelligent and automated.

[0010] To solve the above technical problems, the present invention provides the following technical solutions:

[0011] On the one hand, the present invention provides a data processing system for video detection in a parking lot, including an image processing module, an information extraction module, and an image matching module; wherein:

[0012] The image processing module is used to adaptively regulate the surveillance video of the parking lot, and detect and extract the target picture of the target vehicle from the surveillance video;

[0013] The information extraction module is used to extract the license plate information of the target vehicle from the target picture and verify the license plate information; if the extraction of the license plate information fails or the license plate information fails to pass the verification, an image matching strategy is triggered;

[0014] The image matching module responds to the image matching strategy and matches a reference picture for the target vehicle based on the target picture; the information extraction module re-extracts the license plate information of the target vehicle based on the reference picture.

[0015] As a preferred solution of the data processing system for video detection in a parking lot according to the present invention, wherein: the image processing module includes a vehicle detection unit and a preprocessing unit; wherein:

[0016] The vehicle detection unit is used to detect the target vehicle from the surveillance video of the parking lot and extract the target picture of the target vehicle;

[0017] The preprocessing unit is used to preprocess the target picture, and the preprocessing includes filtering and noise reduction, image enhancement, shadow removal, and picture cropping.

[0018] As a preferred solution of the data processing system for video detection in a parking lot according to the present invention, wherein: the vehicle detection unit detects the target vehicle from the surveillance video of the parking lot in the following manner:

[0019] Extract surveillance images from the surveillance video of the parking lot;

[0020] Input the extracted surveillance images into a pre-created background subtractor for background subtraction to obtain a foreground mask, and perform binary processing on the foreground mask to obtain a moving foreground image;

[0021] Denoise the moving foreground image through morphological operations;

[0022] Perform contour detection on the denoised moving foreground image to mark the moving area;

[0023] Calculate the area of the moving area and determine whether a target vehicle appears in the surveillance image, specifically as follows: the vehicle detection unit is configured with a detection threshold for the area of the moving area; when the area of the moving area is greater than the detection threshold, a target vehicle appears in the surveillance image, otherwise, no target vehicle appears in the surveillance image.

[0024] As a preferred solution of the data processing system for parking lot video detection according to the present invention, wherein: the image processing module further includes a monitoring adjustment unit, and the monitoring adjustment unit adaptively adjusts the real-time frame rate of the monitoring video based on the detection result of the target vehicle;

[0025] The monitoring adjustment unit is configured with a frame rate control strategy, specifically as follows: if no target vehicle appears in the monitoring images within consecutive seconds, the real-time frame rate of the monitoring video is set to the first frame rate; if a target vehicle appears in the monitoring image, the real-time frame rate of the monitoring video is set to the second frame rate.

[0026] As a preferred solution of the data processing system for parking lot video detection according to the present invention, wherein: the vehicle detection unit is further configured with an image extraction strategy, specifically as follows: when the frame rate of the monitoring video in the parking lot is the first frame rate, then every seconds, 1 frame of monitoring image is extracted from the monitoring video in the parking lot, and the extracted monitoring image is detected for the target vehicle; is a preset first time interval; when the frame rate of the monitoring video in the parking lot is the second frame rate, then every seconds, 1 frame of monitoring image is extracted from the monitoring video in the parking lot, and the extracted monitoring image is detected for the target vehicle, and if a target vehicle appears in the monitoring image, the monitoring image is marked as a target picture, and the target picture is transmitted to the preprocessing unit; is a preset second time interval.

[0027] As a preferred solution of the data processing system for parking lot video detection according to the present invention, wherein: the information extraction module includes a positioning unit, an identification unit, and a verification unit; wherein:

[0028] The positioning unit is used to locate the license plate area in the target picture; if the license plate area cannot be successfully located, the license plate information extraction fails;

[0029] The identification unit is used to identify the license plate information of the target vehicle from the license plate area; if the license plate information cannot be successfully identified, the license plate information extraction fails;

[0030] The verification unit is configured with an information verification strategy for verifying the license plate information, specifically as follows: license plate information is extracted from consecutive n target pictures of the same target vehicle, where n is a positive integer; if the license plate information extracted from the n target pictures is all the same, the license plate information passes the verification; otherwise, the license plate information fails to pass the verification.

[0031] As a preferred solution of the data processing system for parking lot video detection according to the present invention, wherein: the image matching module includes a picture retrieval unit, a target detection unit, and a matching unit; wherein:

[0032] The picture retrieval unit is used to retrieve surveillance images from surveillance videos at different positions in the parking lot;

[0033] The target detection unit is used to detect and segment the pictures to be matched containing vehicles from the surveillance images at different positions in the parking lot;

[0034] The matching unit is used to screen out the reference pictures containing the target vehicle from the pictures to be matched containing vehicles.

[0035] As a preferred solution of the data processing system for parking lot video detection according to the present invention, wherein: for any picture to be matched, the method for the matching unit to determine whether it contains the target vehicle is as follows:

[0036] S100: Mark feature points in the target picture and the picture to be matched respectively through a feature detection algorithm;

[0037] S200: Perform feature matching on the feature points in the target picture and the feature points in the picture to be matched to obtain matching point pairs;

[0038] S300: Perform consistency verification on the matching point pairs to obtain the consistent matching point pairs in the target picture and the picture to be matched;

[0039] S400: Count the number of consistent matching point pairs in the target picture and the picture to be matched, and calculate the similarity score between the picture to be matched and the target picture;

[0040] S500: The matching unit is configured with a matching threshold. If the similarity score is higher than the matching threshold, the corresponding picture to be matched contains the target vehicle.

[0041] As a preferred solution of the data processing system for parking lot video detection according to the present invention, wherein: the method for performing consistency verification on the matching point pairs to obtain the consistent matching point pairs in the target picture and the picture to be matched is as follows:

[0042] S301: Initialize a homography matrix H;

[0043] S302: Randomly select 4 pairs from the matching point pairs of the target picture and the picture to be matched as a sample set;

[0044] S303: Solve the homography matrix H based on the feature point coordinates of the matching point pairs in the sample set;

[0045] S304: Map the feature points in any pair of matching points in the target image to the image to be matched through the homography matrix H; if the Euclidean distance between the mapped feature points and the corresponding feature points in the image to be matched is less than the preset consistency threshold, the pair of matching points is marked as a consistent pair of matching points;

[0046] S305: Count the number of consistent pairs of matching points in the target image and the image to be matched;

[0047] S306: Repeat and execute S301 to S306 for M times, and count the number of consistent pairs of matching points in each iteration;

[0048] S307: Obtain the consistent pairs of matching points in the iteration with the largest number of consistent pairs of matching points.

[0049] As a preferred solution of the data processing system for parking lot video detection described in the present invention, wherein: the calculation formula of the similarity score is as follows:

[0050] ;

[0051] Wherein, represents the similarity score between the image to be matched and the target image; represents the abscissa of the feature point in the image to be matched in the i-th pair of consistent matching points; the value range of i is 1, 2,..., N; wherein, N represents the number of pairs of consistent matching points in the target image and the image to be matched; represents the ordinate of the feature point in the image to be matched in the i-th pair of consistent matching points; represents the abscissa of the feature point in the target image in the i-th pair of consistent matching points; represents the ordinate of the feature point in the target image in the i-th pair of consistent matching points; w represents the width of the target image; h represents the height of the target image.

[0052] As a preferred solution of the data processing system for parking lot video detection described in the present invention, wherein: the matching unit sends no less than k reference images to the information extraction module; after receiving the reference images of the target vehicle, the information extraction module triggers an information re-verification strategy, which is specifically as follows:

[0053] The positioning unit locates the license plate area in the target image;

[0054] The recognition unit recognizes the license plate information of the target vehicle from the license plate area;

[0055] The verification unit verifies the license plate information, counts the number of times of each license plate information extracted from k reference images, and takes the license plate information with the most occurrences as the final license plate information extraction result.

[0056] In a second aspect, the present invention provides a data processing method for video detection in a parking lot, comprising the following steps:

[0057] S1: Detect and extract target pictures of target vehicles from the surveillance videos of the parking lot, and adaptively regulate the surveillance videos of the parking lot;

[0058] S2: Extract license plate information of the target vehicle from the target pictures, and verify the license plate information;

[0059] S3: If the license plate information is successfully extracted and the license plate information passes the verification, return to S1; otherwise, execute S4;

[0060] S4: Match a reference picture for the target vehicle based on the target picture;

[0061] S5: Re-extract the license plate information of the target vehicle based on the reference picture, and return to S1.

[0062] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0063] Through the adaptive frame rate adjustment mechanism, the system can reduce the frame rate when there is no vehicle activity, reduce unnecessary data volume, and save computing resources. When a vehicle appears, the frame rate is automatically increased to ensure that all action details of the vehicle are captured, thereby realizing the effective utilization of resources.

[0064] By adopting background subtraction technology combined with binarization processing, the moving area can be effectively distinguished, and the target vehicle can be accurately detected even under large changes in light conditions, improving the robustness and reliability of the system. The information extraction module ensures the accurate extraction and verification of license plate information through a three-step method of a positioning unit, an identification unit, and a verification unit. Especially through the consistency verification of consecutive multiple pictures, the accuracy of license plate information is greatly improved.

[0065] When the license plate information extraction fails, the system automatically activates the image matching strategy, retrieves surveillance images from other locations in the parking lot, and automatically completes the re-extraction of the license plate information of the target vehicle, reducing the need for manual intervention and improving the efficiency. Description of the Drawings

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. Among them:

[0067] Figure 1Schematic diagram of a data processing system for parking lot video detection provided by the present invention;

[0068] Figure 2 Flowchart of a data processing method for parking lot video detection provided by the present invention;

[0069] Figure 3 Flowchart of the method for detecting target vehicles from the surveillance videos of the parking lot provided by the present invention. Detailed implementation manners

[0070] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. Without conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.

[0071] Embodiment 1

[0072] This embodiment introduces a data processing system for parking lot video detection. Referring to Figure 1 , the system includes an image processing module, an information extraction module, and an image matching module; where:

[0073] The image processing module is used to adaptively adjust the surveillance video of the parking lot, and detect and extract the target pictures of the target vehicles from the surveillance video;

[0074] The image processing module includes a vehicle detection unit, a monitoring adjustment unit, and a preprocessing unit; where: the vehicle detection unit is used to detect the target vehicles from the surveillance video of the parking lot and extract the target pictures of the target vehicles;

[0075] Referring to Figure 3 , the method for detecting target vehicles from the surveillance video of the parking lot is as follows:

[0076] Extract surveillance images from the surveillance video of the parking lot;

[0077] Input the extracted surveillance images into a pre-created background subtractor for background subtraction to obtain a foreground mask, and perform binary processing on the foreground mask to obtain a moving foreground image; some image processing tools, such as MOG2 in OpenCV, have the function of creating a background subtractor. Through the background subtractor, the static and unchanged background areas in the surveillance images can be removed, and then the non-background part, that is, the moving area, can be highlighted through binary processing. The moving area is very likely to be the target vehicle;

[0078] Denoise the moving foreground image through morphological operations; for example, remove small noise points in the moving foreground image through dilation and erosion operations;

[0079] Perform contour detection on the denoised moving foreground image, mark the moving area, and determine the position and size of the moving area;

[0080] Calculate the area of the moving area and determine whether a target vehicle appears in the monitored image as follows: The vehicle detection unit is configured with a detection threshold for the area of the moving area; when the area of the moving area is greater than the detection threshold, a target vehicle appears in the monitored image, otherwise, no target vehicle appears in the monitored image.

[0081] Detecting the target vehicle through the above background subtraction method has good robustness and can accurately detect the target vehicle under various light conditions.

[0082] The monitoring adjustment unit adaptively adjusts the real-time frame rate of the monitoring video based on the detection result of the target vehicle;

[0083] The monitoring adjustment unit is configured with a frame rate control strategy as follows: If no target vehicle appears in the monitored images within consecutive seconds, set the real-time frame rate of the monitoring video to the first frame rate, such as 5 frames, where m is a positive integer; if a target vehicle appears in the monitored image, set the real-time frame rate of the monitoring video to the second frame rate, such as 30 frames.

[0084] The goal of frame rate adjustment is to reduce unnecessary data volume and save computing resources on the premise of meeting the recognition accuracy. In the absence of vehicle activities, the frame rate can be reduced because there is no need to capture image information frequently at this time. When a vehicle is detected entering or leaving the parking lot, the frame rate is automatically increased to ensure that all action details of the vehicle can be captured.

[0085] The vehicle detection unit is also configured with an image extraction strategy as follows: When the frame rate of the monitoring video of the parking lot is the first frame rate, extract 1 frame of the monitoring image from the monitoring video of the parking lot every seconds and detect the target vehicle in the extracted monitoring image; is a preset first time interval, such as 1.5 seconds; when the frame rate of the monitoring video of the parking lot is the second frame rate, extract 1 frame of the monitoring image from the monitoring video of the parking lot every seconds and detect the target vehicle in the extracted monitoring image, and if a target vehicle appears in the monitored image, mark the monitored image as a target picture and transmit the target picture to the preprocessing unit; is a preset second time interval, such as 0.5 seconds;

[0086] Based on the above adaptive frame rate control, the frame rate of the monitoring video can be adjusted according to the actual application scenario, which not only ensures the timely detection of the target vehicle but also avoids data redundancy.

[0087] The preprocessing unit is used to preprocess the target picture, and the preprocessing includes filtering and noise reduction, image enhancement, shadow removal, and picture cropping; the preprocessing unit is configured with image noise reduction algorithms (such as non-local means denoising, bilateral filtering, etc.), which can remove the noise in the target picture; the preprocessing unit is also configured with image enhancement algorithms, such as histogram equalization, contrast enhancement, etc., which can further improve the image quality of the target image; in addition, the preprocessing unit is also configured with a shadow removal algorithm (such as the Retinex algorithm), which can reduce the influence of environmental light changes on the clarity of the target picture; finally, after the above preprocessing, the preprocessing unit crops the target picture into a specified size through a preset cropping frame for subsequent information extraction and image matching.

[0088] The information extraction module is used to extract the license plate information of the target vehicle from the target picture and verify the license plate information; if the extraction of the license plate information fails or the license plate information fails to pass the verification, the image matching strategy is triggered;

[0089] The information extraction module includes a positioning unit, an identification unit, and a verification unit; among them:

[0090] The positioning unit is used to locate the license plate area in the target picture; if the license plate area fails to be successfully located, the extraction of the license plate information fails; the positioning unit is configured with a positioning algorithm, such as a method based on template matching, which can accurately locate the position of the license plate, but may fail to locate due to occlusion, insufficient light, etc.

[0091] The identification unit is used to identify the license plate information of the target vehicle from the license plate area; if the license plate information fails to be successfully identified, the extraction of the license plate information fails; the identification unit is configured with an OCR recognition algorithm, which can extract text information based on image recognition.

[0092] The verification unit is configured with an information verification strategy for verifying the license plate information, specifically as follows: extract the license plate information from n consecutive target pictures of the same target vehicle, where n is a positive integer; if the license plate information extracted from the n target pictures is consistent, the license plate information passes the verification; otherwise, the license plate information fails to pass the verification.

[0093] The image matching module responds to the image matching strategy and matches a reference picture for the target vehicle based on the target picture; the information extraction module re-extracts the license plate information of the target vehicle based on the reference picture.

[0094] The image matching module includes a picture retrieval unit, a target detection unit, and a matching unit; among them:

[0095] The described picture retrieval unit is used to retrieve surveillance images from surveillance videos at different positions in the parking lot; the license plate information of the target vehicle is generally extracted at the exit and entrance of the parking lot to calculate the parking duration; if the license plate information of the target vehicle cannot be effectively extracted from the surveillance videos at the parking lot entrance and exit, manual confirmation is required, which is time-consuming and laborious; therefore, by using the picture retrieval unit to retrieve surveillance images from other positions (where clear pictures of the license plate may be captured) and automatically extracting the license plate information of the target vehicle through subsequent processes, the efficiency of license plate information confirmation can be significantly improved.

[0096] The method of retrieving surveillance images from surveillance videos at other positions in the parking lot is as follows: Assume that the license plate information extraction fails at the exit of the parking lot for the target vehicle. Select camera A at another position in the parking lot, calculate the distance between camera A and the parking lot exit, and estimate the time point when the target vehicle passes by camera A based on the average speed of the vehicle in the parking lot. Retrieve the surveillance images captured by camera A around this time point, which can effectively find the target vehicle.

[0097] The described target detection unit is used to detect and segment the pictures to be matched containing vehicles from surveillance images at different positions in the parking lot; the target detection unit is configured with pre-trained deep learning models, such as YOLOv4, SSD, Faster R-CNN, etc., which can efficiently detect vehicles from surveillance images; then, image segmentation is performed on the vehicle area to obtain the pictures to be matched containing vehicles; among them, the size of the segmented pictures to be matched is the same as the size of the preset cropping frame in the preprocessing unit, ensuring that the size of the target picture is the same as that of the reference picture. Downloading pre-trained deep learning models for target detection from the network and then training and fine-tuning the models using a dataset of vehicle pictures under various angles and lighting conditions can improve the vehicle recognition ability of the models in complex environments and greatly save the construction and training costs of deep learning models.

[0098] The described matching unit is used to screen out the reference pictures containing the target vehicle from the pictures to be matched containing vehicles; for any picture to be matched, the method of determining whether it contains the target vehicle is as follows:

[0099] S100: Mark feature points in the target picture and the picture to be matched respectively through feature detection algorithms; The feature detection algorithms that can be used include SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), etc.

[0100] S200: Perform feature matching between the feature points in the target image and the feature points in the image to be matched to obtain matching point pairs; Use the BFMatcher or FLANN Matcher algorithm to find the corresponding points between the feature points of the two images.

[0101] S300: Conduct a consistency check on the matching point pairs to obtain the consistent matching point pairs in the target image and the image to be matched; The method is as follows:

[0102] S301: Initialize a homography matrix H; For planar scenes, a homography matrix is usually used to describe the relationship between two images. The homography matrix H is a 3x3 matrix that can map points on one plane to the vicinity of the corresponding points on another plane.

[0103] S302: Randomly select 4 pairs from the matching point pairs of the target image and the image to be matched as the sample set; To estimate a 3x3 homography matrix, theoretically at least four sets of matching point pairs are required.

[0104] S303: Solve the homography matrix H based on the feature point coordinates of the matching point pairs in the sample set; The homography matrix H can be determined by solving the linear equations constructed from the coordinates of the four sets of matching point pairs.

[0105] S304: Map the feature points in any matching point pair in the target image to the image to be matched through the homography matrix H; If the Euclidean distance between the mapped feature points and the corresponding feature points in the image to be matched is less than the preset consistency threshold, the matching point pair is marked as a consistent matching point pair;

[0106] S305: Count the number of consistent matching point pairs in the target image and the image to be matched; The larger the number of consistent matching point pairs, the more suitable the homography matrix H is for describing the geometric transformation relationship between the target image and the image to be matched.

[0107] S306: Repeatedly iterate and execute S301 to S306 M times, and count the number of consistent matching point pairs in each iteration; M is a positive integer, such as 1000;

[0108] S307: Obtain the consistent matching point pairs in the iteration with the largest number of consistent matching point pairs.

[0109] S400: Count the number of consistent matching point pairs in the target image and the image to be matched, and calculate the similarity score between the image to be matched and the target image. The formula is as follows:

[0110] ;

[0111] Among them, represents the similarity score between the image to be matched and the target image; represents the abscissa of the feature point in the image to be matched in the i-th pair of consistent matching point pairs; the value range of i is 1, 2, ……, N; where N represents the number of consistent matching point pairs between the target image and the image to be matched; represents the ordinate of the feature point in the image to be matched in the i-th pair of consistent matching point pairs; represents the abscissa of the feature point in the target image in the i-th pair of consistent matching point pairs; represents the ordinate of the feature point in the target image in the i-th pair of consistent matching point pairs; w represents the width of the target image; h represents the height of the target image;

[0112] S500: The matching unit is configured with a matching threshold. If the similarity score is higher than the matching threshold, the corresponding image to be matched contains the target vehicle.

[0113] The matching unit sends no less than k reference images to the information extraction module; after receiving the reference images of the target vehicle, the information extraction module triggers an information re-verification strategy, which is as follows:

[0114] The positioning unit locates the license plate area in the target image;

[0115] The recognition unit recognizes the license plate information of the target vehicle from the license plate area;

[0116] The verification unit verifies the license plate information, counts the number of times each license plate information is extracted from k reference images, and takes the license plate information with the most occurrences as the final license plate information extraction result.

[0117] Embodiment 2

[0118] This embodiment is the second embodiment of the present invention; based on the same inventive concept as Embodiment 1, referring to Figure 2 , this embodiment introduces a data processing method for parking lot video detection, including the following steps:

[0119] S1: Detect and extract the target image of the target vehicle from the surveillance video of the parking lot, and adaptively adjust the surveillance video of the parking lot; adaptively adjust the real-time frame rate of the surveillance video, reduce the frame rate when no vehicle is detected, and increase the frame rate when a vehicle is detected.

[0120] S2: Extract the license plate information of the target vehicle from the target image, and verify the license plate information; verify the consistency of the license plate information. If the license plate information in several consecutive images is the same, it passes the verification.

[0121] S3: If the license plate information is successfully extracted and the license plate information passes the verification, return to S1; otherwise, execute S4; if returning to S1, prepare to detect the next target vehicle.

[0122] S4: Match a reference picture for the target vehicle based on the target picture; retrieve images that may contain clear license plate information from surveillance videos at different positions in the parking lot. Use a deep learning model to detect and segment the pictures to be matched that contain vehicles; use a feature detection algorithm to mark feature points in the target picture and the pictures to be matched, and perform feature matching; calculate a similarity score through consistency checking, and filter out the reference pictures that contain the target vehicle.

[0123] S5: Re-extract the license plate information of the target vehicle based on the reference picture, and return to S1. Use the successfully matched reference picture to re-execute the process of license plate information extraction and verification. After completion, return to S1 to continue processing the new surveillance video stream.

[0124] For the specific function implementation of each of the above modules, refer to the relevant content in the data processing system for parking lot video detection described in Embodiment 1, which will not be elaborated here.

[0125] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) that contain computer-usable program codes.

[0126] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit and scope of the present invention. These all fall within the protection scope of the present invention.

Claims

1. A data processing system for parking lot video detection, characterized in that: It includes image processing module, information extraction module and image matching module; among which: The image processing module is used to adaptively control the monitoring video of the parking lot and detect and extract the target image of the target vehicle from the monitoring video; The information extraction module is used to extract the license plate information of the target vehicle from the target image and verify the license plate information; if the license plate information extraction fails or the license plate information fails to pass the verification, the image matching strategy is triggered; The image matching module responds to the image matching strategy and matches a reference image for the target vehicle based on the target image; the information extraction module re-extracts the license plate information of the target vehicle based on the reference image; The image matching module includes a picture retrieval unit, a target detection unit, and a matching unit; wherein: The image retrieval unit is used to retrieve surveillance images from surveillance videos at different locations in the parking lot; The target detection unit is used to detect and segment the to-be-matched pictures containing vehicles from the surveillance images at different locations of the parking lot; The matching unit is used to filter out reference images containing the target vehicle from the images to be matched containing the vehicle; for any image to be matched, the matching unit determines whether it contains the target vehicle in the following manner: S100: Marking feature points in the target image and the image to be matched respectively by using a feature detection algorithm; S200: performing feature matching on the feature points in the target image and the feature points in the image to be matched to obtain matching point pairs; S300: Perform consistency check on the matching point pairs to obtain consistent matching point pairs in the target image and the image to be matched; S400: Count the number of consistent matching point pairs in the target image and the image to be matched, and calculate the similarity score between the image to be matched and the target image; the calculation formula of the similarity score is as follows: ； in, Represents the similarity score between the image to be matched and the target image; Indicates the horizontal coordinate of the feature point in the image to be matched in the i-th pair of consistent matching points; the value range of i is 1, 2, ..., N; where N represents the number of consistent matching point pairs in the target image and the image to be matched; Represents the ordinate of the feature point in the image to be matched in the i-th pair of consistent matching points; Represents the horizontal coordinate of the feature point in the target image in the i-th pair of consistent matching points; represents the ordinate of the feature point in the target image in the i-th pair of consistent matching points; w represents the width of the target image; h represents the height of the target image; S500: The matching unit is configured with a matching threshold. If the similarity score is higher than the matching threshold, the corresponding image to be matched contains the target vehicle. The method of performing consistency check on the matching point pairs to obtain consistent matching point pairs in the target image and the image to be matched is as follows: S301: Initialize a homography matrix H; S302: Randomly select 4 pairs of matching points between the target image and the image to be matched as a sample set; S303: Solving the homography matrix H based on the coordinates of the feature points of the matching point pairs in the sample set; S304: Mapping feature points in any matching point pair in the target image to the image to be matched through the homography matrix H; if the Euclidean distance between the mapped feature point and the corresponding feature point in the image to be matched is less than a preset consistency threshold, the matching point pair is marked as a consistent matching point pair; S305: Counting the number of consistent matching point pairs in the target image and the image to be matched; S306: Repeat the iterations from S301 to S306 M times, and count the number of consistent matching point pairs in each iteration; S307: Obtain consistent matching point pairs in the iteration with the largest number of consistent matching point pairs.

2. A data processing system for parking lot video detection as claimed in claim 1, characterized in that: The image processing module includes a vehicle detection unit and a preprocessing unit; wherein: The vehicle detection unit is used to detect a target vehicle from a surveillance video of the parking lot and extract a target image of the target vehicle; The preprocessing unit is used to preprocess the target image, and the preprocessing includes filtering and noise reduction, image enhancement, shadow removal, and image cropping.

3. A data processing system for parking lot video detection as claimed in claim 2, characterized in that: The vehicle detection unit detects the target vehicle from the surveillance video of the parking lot in the following manner: Extract surveillance images from surveillance videos of parking lots; Input the extracted surveillance image into a pre-created background subtractor to perform background subtraction to obtain a foreground mask, and perform binarization processing on the foreground mask to obtain a motion foreground image; De-noising the motion foreground image by morphological operation; Perform contour detection on the denoised motion foreground image and mark the motion area; The area of ​​the moving area is calculated, and it is determined whether the target vehicle appears in the monitoring image, specifically as follows: the vehicle detection unit is configured with a detection threshold of the moving area; when the area of ​​the moving area is greater than the detection threshold, the target vehicle appears in the monitoring image, otherwise, the target vehicle does not appear in the monitoring image.

4. A data processing system for parking lot video detection as claimed in claim 3, characterized in that: The image processing module also includes a monitoring adjustment unit, which adaptively adjusts the real-time frame rate of the monitoring video based on the detection result of the target vehicle; The monitoring and adjustment unit is configured with a frame rate control strategy as follows: If the target vehicle does not appear in the monitoring image within seconds, the real-time frame rate of the monitoring video is set to the first frame rate; if the target vehicle appears in the monitoring image, the real-time frame rate of the monitoring video is set to the second frame rate, and m is a positive integer.

5. A data processing system for parking lot video detection as claimed in claim 4, characterized in that: The vehicle detection unit is also configured with an image extraction strategy as follows: when the frame rate of the monitoring video of the parking lot is the first frame rate, Extract one frame of surveillance image from the surveillance video of the parking lot every second, and detect the target vehicle in the extracted surveillance image; is the preset first time interval; when the frame rate of the parking lot surveillance video is the second frame rate, then every Extract one frame of monitoring image from the monitoring video of the parking lot every second, detect the target vehicle from the extracted monitoring image, and if the target vehicle appears in the monitoring image, mark the monitoring image as a target image, and transmit the target image to the preprocessing unit; is a preset second time interval.

6. A data processing system for parking lot video detection as claimed in claim 5, characterized in that: The information extraction module includes a positioning unit, an identification unit, and a verification unit; wherein: The positioning unit is used to locate the license plate area in the target image; if the license plate area is not successfully located, the extraction of the license plate information fails; The recognition unit is used to recognize the license plate information of the target vehicle from the license plate area; if the license plate information cannot be successfully recognized, the extraction of the license plate information fails; The verification unit is configured with an information verification strategy for verifying the license plate information, specifically as follows: extracting the license plate information from n consecutive target images of the same target vehicle, where n is a positive integer; if the license plate information extracted from the n target images are all consistent, the license plate information passes the verification; otherwise, the license plate information fails the verification.

7. A data processing system for parking lot video detection as claimed in claim 6, characterized in that: The matching unit sends no less than k reference images to the information extraction module; after receiving the reference image of the target vehicle, the information extraction module triggers the information re-verification strategy, which is as follows: The positioning unit locates the license plate area in the target image; The recognition unit recognizes the license plate information of the target vehicle from the license plate area; The verification unit verifies the license plate information, counts the number of times each type of license plate information is extracted from k reference images, and takes the license plate information with the most occurrences as the final license plate information extraction result.

8. A data processing method for parking lot video detection, implemented based on the data processing system for parking lot video detection according to any one of claims 1 to 7, characterized in that: The following steps are involved: S1: Detect and extract the target image of the target vehicle from the surveillance video of the parking lot, and adaptively control the surveillance video of the parking lot; S2: extracting the license plate information of the target vehicle from the target image, and verifying the license plate information; S3: If the license plate information is successfully extracted and verified, return to S1; Otherwise, execute S4; S4: matching a reference image for a target vehicle based on the target image; S5: Re-extract the license plate information of the target vehicle based on the reference image, and return to S1.

Images