Method for detecting parking lot payment two-dimensional code based on vehicle-mounted surround view fisheye camera

Abstract

The application discloses a method for detecting parking lot payment two-dimensional codes based on a vehicle-mounted surround-view fisheye camera, which comprises the following steps: pre-shooting multiple parking lot two-dimensional code images by using the surround-view fisheye camera, counting the two-dimensional code appearing areas in the images shot by the surround-view fisheye camera, recording the positions where the two-dimensional codes often appear and can be clearly identified in the images, and then splitting the positions into multiple sliding windows; acquiring the images shot by the surround-view fisheye camera in real time, and extracting the pictures corresponding to the sliding windows in the images; performing distortion correction on the pictures corresponding to the sliding windows, and then performing two-dimensional code detection on the distortion-corrected pictures; if a two-dimensional code is detected, the detection of the two-dimensional code is stopped; otherwise, the above steps are repeated until a two-dimensional code is detected. The method can effectively avoid causing a garage jam, simultaneously selectively detect the payment two-dimensional code images acquired by the vehicle-mounted surround-view fisheye camera, effectively reduce the operation amount, improve the detection frame number to avoid lag, and improve the identification success rate.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of two-dimensional code reading and mobile payment, and particularly relates to a method for detecting a parking lot fee two-dimensional code based on a vehicle-mounted ring-view fisheye camera. BACKGROUND

[0002] The automobile industry is undergoing an intelligent revolution. Compared with traditional automobile design, many sensors and high-performance computing platforms are now installed in automobiles to meet the growing demand for intelligent functions. Among them, the ring-view fisheye camera and the Android-based vehicle information entertainment terminal controller have become the standard configuration of a large number of medium and high-end automobile brands. The ring-view fisheye camera is usually used to monitor the situation around the car and provide image information of the driver's visual blind area during parking to prevent accidental collisions with obstacles in the blind area. The Android-based vehicle information entertainment terminal can provide more personalized, intelligent, and expandable information and entertainment services for the driver.

[0003] At the same time, with the progress of computer vision technology and Internet technology, many parking lots have provided card-free, cashless, and unattended parking services. When the driver enters the parking lot, the camera at the entrance of the parking lot recognizes the license plate and starts timing. When leaving the parking lot, the driver scans the two-dimensional code provided by the parking lot to pay the fee. When the vehicle leaves the parking lot, the camera at the exit of the parking lot recognizes the license plate to confirm the payment of the vehicle, and if the fee has been cleared, the vehicle is released. This parking lot management mode has greatly optimized the parking experience compared to the past, but still has its shortcomings. One of them is that the position, size, and lighting conditions of the fee two-dimensional code vary from parking lot to parking lot. The driver needs to find the two-dimensional code first, open the mobile phone scanning function, stretch his hand out of the window, align the two-dimensional code for scanning, and then complete the payment on the mobile phone. This operation process is not simple enough, and operation errors can easily cause congestion in the garage. SUMMARY

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a method for detecting a parking lot fee two-dimensional code based on a vehicle-mounted fisheye camera, which can effectively avoid causing congestion in the garage, selectively detect the fee two-dimensional code image obtained by the vehicle-mounted ring-view fisheye camera, effectively reduce the amount of calculation, improve the detection frame rate to avoid lag, and improve the recognition success rate.

[0005] The technical solution of the present application is as follows:

[0006] The method for detecting a parking lot fee two-dimensional code based on a vehicle-mounted ring-view fisheye camera comprises the following steps:

[0007] S1: Pre-adopting the look-around fisheye camera to shoot multiple parking lot two-dimensional code images, counting the two-dimensional code appearing area in the image shot by the look-around fisheye camera, and recording the positions where the two-dimensional code often appears and can be clearly identified in the image, then splitting these positions into multiple sliding windows;

[0008] S2: Real-time acquisition of the image shot by the look-around fisheye camera, and extraction of the picture corresponding to the sliding window in the image;

[0009] S3: Distortion correction of the picture corresponding to the sliding window in S2, thereby obtaining the distortion-corrected picture, and then two-dimensional code detection of the distortion-corrected picture, if the two-dimensional code is detected, stopping the two-dimensional code detection, otherwise entering S4;

[0010] S4: Repeating steps S2-S3 until the two-dimensional code is detected.

[0011] Further, the look-around fisheye camera includes a front-view fisheye camera, a left-view fisheye camera, and a right-view fisheye camera.

[0012] Further, according to different vehicle speeds V, the detection is performed in the following cases, and as long as the two-dimensional code is detected during the detection, the detection is immediately stopped:

[0013] Case one: when V=0, the detection functions of the front-view fisheye camera, the left-view fisheye camera, and the right-view fisheye camera are started, and the front-view fisheye camera, the left-view fisheye camera, and the right-view fisheye camera perform at most N times of two-dimensional code detection respectively.

[0014] Case two: when 0

[0015] Case three: when 5 Km / h < V≤ 15 Km / h, the detection function of the front fisheye camera, the left fisheye camera and the right fisheye camera is turned on, it is judged whether the vehicle steering angle is greater than or equal to 30°, if yes, the two-dimensional code detection is carried out through the front fisheye camera; if no, the detection function of the front fisheye camera is turned off, and the two-dimensional code detection is carried out through the left fisheye camera and the right fisheye camera, if the ultrasonic radar shows that there is no obstacle in the set range on the left and right sides of the vehicle, the left fisheye camera or the right fisheye camera carries out the two-dimensional code detection at most M times respectively, if the ultrasonic radar shows that there is an obstacle in the corresponding side of the set range on the left and right sides of the vehicle, the two-dimensional code detection is carried out through the fisheye camera on the side with the obstacle, and the fisheye camera on the side without the obstacle carries out the two-dimensional code detection at most M times respectively.

[0016] Case four: when 15 Km / h < V≤ 25 Km / h, the detection function of the front fisheye camera is turned off, and the detection function of the left fisheye camera and the right fisheye camera is turned on, if the ultrasonic radar shows that there is no obstacle in the set range on the left and right sides of the vehicle, the left fisheye camera or the right fisheye camera carries out the two-dimensional code detection at most M times respectively, if the ultrasonic radar shows that there is an obstacle in the corresponding side of the set range on the left and right sides of the vehicle, the two-dimensional code detection is carried out through the fisheye camera on the side with the obstacle, and the fisheye camera on the side without the obstacle carries out the two-dimensional code detection at most M times respectively.

[0017] Case five: when V > 25 Km / h, the detection function of the front fisheye camera, the left fisheye camera and the right fisheye camera is turned off, and the two-dimensional code detection is not carried out.

[0018] Further, in case one, N≤3.

[0019] Further, if the ultrasonic radar shows that there is an obstacle within 3 m in front of the vehicle, the two-dimensional code detection is carried out through the front fisheye camera; if the ultrasonic radar shows that there is an obstacle within 2 m on the left and right sides of the vehicle, the two-dimensional code detection is carried out through the left fisheye camera and the right fisheye camera.

[0020] Further, M is equal to 2, and a waiting time is added between the two detections.

[0021] Further, the image captured by the front fisheye camera is split into a sliding window located on the left side of the image and a sliding window located on the right side of the image; the images captured by the left fisheye camera and the right fisheye camera are split into three sliding windows, two of which are arranged horizontally to form a horizontal sliding window combination, and the other is arranged below the horizontal sliding window combination to form a vertical sliding window combination.

[0022] Further, in case three, when the vehicle steering angle is greater than or equal to 30°, and the vehicle turns left, the sliding window on the right side of the image captured by the front fisheye camera is opened, and the sliding window on the left side of the image is closed; when the vehicle turns right, the sliding window on the left side of the image captured by the front fisheye camera is opened, and the sliding window on the right side of the image is closed.

[0023] Further, when 5 Km / h < V≤ 25 Km / h, the images captured by the left fisheye camera and the right fisheye camera are used for two-dimensional code detection, and the horizontal sliding window combination and the vertical sliding window combination are alternately used for two-dimensional code detection.

[0024] Compared with the prior art, the present application has the following beneficial effects:

[0025] 1. The vehicle-mounted surround-view fisheye camera is used instead of a mobile phone camera in the present application, so that the front fisheye camera, the left fisheye camera and the right fisheye camera can continuously collect the toll two-dimensional code in the environment during the driving process of the vehicle, and the recognized toll two-dimensional code can be displayed on the vehicle-mounted information entertainment terminal screen when the driver needs to pay, so as to provide the driver with scanning, so that the driver does not need to stop the vehicle in front of the two-dimensional code to complete the entire toll process, thereby avoiding congestion at the parking lot exit.

[0026] 2. The detection strategy is set in the present application, and the front fisheye camera, the left fisheye camera and the right fisheye camera are selectively used to capture images, and the sliding window is selectively used, so as to reduce the demand of two-dimensional code detection on the operation capacity, and to improve the detection frame number to avoid lag and improve the recognition success rate. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 - Process schematic diagram of the present application.

[0028] Figure 2 - Structure schematic diagram of the sliding window combination of the right fisheye camera and the left fisheye camera.

[0029] Figure 3 - Detection strategy of the front fisheye camera.

[0030] Figure 4 - Detection strategy of the right fisheye camera.

[0031] Figure 5 - Detection strategy of the left fisheye camera. DETAILED DESCRIPTION

[0032] The present application will be further described in detail below in combination with the drawings and specific embodiments.

[0033] The method for detecting parking lot toll two-dimensional code based on a vehicle-mounted surround-view fisheye camera comprises the following steps:

[0034] S1: Pre-adopting the look around fisheye camera to shoot multiple parking lot two-dimensional code images, counting the two-dimensional code appearing area in the image shot by the look around fisheye camera, and recording the positions where the two-dimensional code often appears and can be clearly identified in the image, and then splitting these positions into multiple sliding windows;

[0035] S2: After the vehicle enters the parking lot, the image shot by the look around fisheye camera is obtained in real time, and the picture corresponding to the sliding window is extracted;

[0036] S3: The picture corresponding to the sliding window in S2 is rectified, so as to obtain the rectified picture, and then the two-dimensional code detection is performed on the rectified picture, if the two-dimensional code is detected, the detection of the two-dimensional code is stopped, otherwise S4 is entered;

[0037] S4: Repeating steps S2-S3 until the two-dimensional code is detected.

[0038] In specific implementation, the look around fisheye camera includes a front fisheye camera, a left fisheye camera and a right fisheye camera.

[0039] In this way, the vehicle-mounted look around fisheye camera can replace the mobile phone camera. During the driving of the vehicle, the front fisheye camera, the left fisheye camera and the right fisheye camera continuously collect the toll two-dimensional code in the environment. When the driver needs to pay the toll, the recognized toll two-dimensional code can be displayed on the vehicle-mounted information entertainment terminal screen to provide the driver for scanning, so that the driver does not need to stop the vehicle in front of the two-dimensional code to complete the entire toll payment process, thereby avoiding congestion at the parking lot exit.

[0040] At the same time, since the field of view angle of the fisheye camera is generally about 180°, the photographed toll two-dimensional code has serious barrel distortion, and the two-dimensional code in the image shot by the fisheye camera cannot be recognized. Therefore, the image is first shot by the look around fisheye camera, then the image is split to obtain the sliding window, then the distortion of each sliding window is rectified, and then the rectified sliding window is recognized. If the sliding window contains the two-dimensional code, the two-dimensional code can be effectively detected.

[0041] The intrinsic matrix, distortion parameters and virtual intrinsic matrix of the look around fisheye camera are used to obtain the mapping matrix of the look around fisheye camera image to the de-distorted image. The de-distorted image corresponding to each sliding window is transformed by the mapping matrix, and the de-distorted image is directly obtained from the fisheye image by one-step transformation.

[0042] In specific implementation, according to different vehicle speeds V, the detection is performed in the following cases, and as long as the two-dimensional code is detected during the detection, the detection is immediately stopped:

[0043] Case one: when V=0, the detection function of the front fisheye camera, the left fisheye camera and the right fisheye camera is turned on, and the front fisheye camera, the left fisheye camera and the right fisheye camera can detect the two-dimensional code at most N times respectively.

[0044] Case two: when 0

[0045] In this way, when the corresponding side of the ultrasonic radar vehicle front, left and right set range has an obstacle, the obstacle here is generally a wall or column nearby, and a two-dimensional code is often pasted on it, so the corresponding fisheye camera is used for two-dimensional code detection, and when there is no obstacle, it means that the front is an open space and generally does not contain a two-dimensional code, so at most M times of two-dimensional code detection are performed, and here M can be set according to needs.

[0046] Case three: when 5 Km / h

[0047] Case four: when 15 Km / h

[0048] Case five: when V > 25 Km / h, the detection function of the front fisheye camera, the left fisheye camera and the right fisheye camera is closed, and the two-dimensional code detection is not performed.

[0049] In the implementation, N≤3 in case one. In the embodiment, when the vehicle is in the stop state, the maximum detection number is set to 3 times, if the two-dimensional code cannot be detected, it is indicated that the two-dimensional code does not exist in the collected image, and then the two-dimensional code detection is stopped, and the vehicle is started, which can effectively save the operation amount of the vehicle information entertainment terminal.

[0050] In the implementation, when the ultrasonic radar displays that there is an obstacle within 3 m in front of the vehicle, the two-dimensional code detection is performed through the front fisheye camera; when the ultrasonic radar displays that there is an obstacle within 2 m on the left and right sides of the vehicle, the two-dimensional code detection is performed through the left fisheye camera and the right fisheye camera.

[0051] Here, the detection ranges of the front fisheye camera, the left fisheye camera and the right fisheye camera are limited, the set range of the front fisheye camera is 3 m, and the set ranges of the left fisheye camera and the right fisheye camera are 2 m.

[0052] In the implementation, M is equal to 2, and a waiting time is added between the two detections.

[0053] When the ultrasonic radar does not detect the obstacle, it is indicated that the possibility of the existence of the two-dimensional code is small, and here, the waiting time is added, which can effectively reduce the detection frequency, thereby reducing the occupation of the operation capacity by the two-dimensional code detection.

[0054] In the implementation, the image captured by the front fisheye camera is split into a sliding window located on the left side of the image and a sliding window located on the right side of the image; the images captured by the left fisheye camera and the right fisheye camera are split into three sliding windows, two of which are horizontally arranged to form a horizontal sliding window combination, and the other is located below the horizontal sliding window combination to form a vertical sliding window combination.

[0055] The structure diagram of the sliding window combination of the right fisheye camera and the left fisheye camera is shown in Figure 2 Figure 1 1 and 2 sliding windows form a horizontal sliding window combination, and 3 forms a vertical sliding window combination. The purpose of 1 and 2 sliding windows is to detect the two-dimensional code with a height of about 1.4~1.8 m on the parking column, and the purpose of 3 sliding window is to detect the two-dimensional code with a relatively close distance or a shallow position.

[0056] ​In the third case, when the vehicle steering angle is greater than or equal to 30°, the left sliding window of the image captured by the front fisheye camera is opened, and the right sliding window is closed when the vehicle turns left; the left sliding window of the image captured by the front fisheye camera is opened, and the right sliding window is closed when the vehicle turns right.

[0057] In the third case, when the vehicle steering angle is greater than or equal to 30°, the left sliding window of the image captured by the front fisheye camera is opened, and the right sliding window is closed when the vehicle turns left; the left sliding window of the image captured by the front fisheye camera is opened, and the right sliding window is closed when the vehicle turns right.

[0058] The alternating use of the horizontal sliding window combination and the vertical sliding window combination for QR code detection at high vehicle speeds can improve the detection speed of a single picture.

[0059] Figure 3 For the detection strategy of the front fisheye camera, all the sliding window combinations in the figure are the left sliding window and the right sliding window. When the vehicle speed is 0, only three QR code detections are performed, and then the vehicle waits until it starts. When the vehicle speed is less than or equal to 5 km / h, all the sliding window combinations of the front fisheye camera are started, but if the ultrasonic radar does not detect an obstacle within a 3 m fan-shaped area in front, a certain waiting time is added between two detections. The significance of this strategy is that the likelihood of a QR code appearing when there is no obstacle in front is low, but at the same time, the possibility of false positives of the ultrasonic radar is considered, such as a thin rod on a sign that the ultrasonic radar may not detect, and there may be a QR code on it. Detection is still performed twice, but a waiting time is added, thereby effectively reducing the detection frequency to reduce the occupation of the QR code detection on the computing power. When the vehicle speed is greater than 5 km / h but less than or equal to 15 km / h, it generally means that the vehicle is in normal driving state, and the likelihood of a QR code appearing in front is low, so generally no detection is performed. However, if the vehicle turns, it means that there is a possibility of an obstacle in front, so when the vehicle steering angle is greater than or equal to 30°, QR code detection is started. At the same time, the turning direction is determined. If the vehicle turns left, it means that the right side of the image is closer to the obstacle, and the likelihood of a QR code appearing is greater, so the right sliding window of the image is used for detection. When the vehicle turns right, the same applies. When the vehicle speed is greater than 15 km / h, the likelihood of an obstacle with a pasted QR code appearing in front is low, so the front QR code detection is turned off.

[0060] Figure 4For the detection strategy of the right-view fisheye camera, all the sliding window combinations in the figure are combined into horizontal sliding window combinations and vertical sliding window combinations. When the vehicle speed is 0 km / h, the detection strategy is the same as that of the front-view fisheye camera. When the vehicle speed is less than or equal to 5 km / h, the detection strategy is the same as that of the front-view fisheye camera when the vehicle speed is less than or equal to 5 km / h, but the obstacle distance threshold is changed to 2 m. When the vehicle speed is greater than 5 km / h but less than or equal to 25 km / h, the two aforementioned sliding window subsets are alternately used for detection. When the vehicle speed is greater than 25 km / h, the detection function of the right-view fisheye camera is closed.

[0061] Figure 5 For the detection strategy of the left-view fisheye camera, all the sliding window combinations in the figure are combined into horizontal sliding window combinations and vertical sliding window combinations. When the vehicle speed is less than or equal to 15 km / h, the detection strategy is the same as that of the right-view fisheye camera. When the vehicle speed is greater than 15 km / h but less than or equal to 25 km / h, the vehicle is generally in a normal driving state. Parking lots are generally provided with two-way two-lane, and the vehicle passes on the right side in the normal driving state. The left-view fisheye camera is too far away from the two-dimensional code, and generally cannot detect the two-dimensional code, so while alternately using the two aforementioned sliding window subsets for detection, a waiting time is added between the two detections, the computing resources are allocated to the right-view fisheye camera, and the detection frame number of the right-view fisheye camera is improved. When the vehicle speed is greater than 25 km / h, the detection function of the left-view fisheye camera is closed.

[0062] Finally, it should be noted that the above embodiments of the present application are only examples for illustrating the present application, and are not intended to limit the embodiments of the present application. Based on the above description, those skilled in the art can make other different forms of changes and modifications. Here, all the embodiments cannot be exhausted. Any obvious changes or modifications derived from the technical solutions of the present application still fall within the protection scope of the present application.

Claims

1. A method for detecting a parking lot payment two-dimensional code based on a vehicle-mounted surround view fisheye camera, characterized in that, The method comprises the following steps: S1: Pre-adopting a surround-view fisheye camera to shoot images of multiple parking lot two-dimensional codes, counting the two-dimensional code appearing areas in the images shot by the surround-view fisheye camera, and recording the positions where the two-dimensional codes frequently appear and can be clearly identified in the images, and then splitting the positions into multiple sliding windows; S2: Real-time acquiring the images shot by the surround-view fisheye camera, and extracting the pictures corresponding to the sliding windows in the images; S3: Carrying out distortion correction on the pictures corresponding to the sliding windows in S2, so as to obtain the distortion-corrected pictures, and then carrying out two-dimensional code detection on the distortion-corrected pictures, if the two-dimensional code is detected, stopping the two-dimensional code detection, otherwise, entering S4; S4: Repeating steps S2-S3 until the two-dimensional code is detected; The surround-view fisheye camera comprises a front-view fisheye camera, a left-view fisheye camera and a right-view fisheye camera; According to different vehicle speeds V, the detection is carried out in the following cases, and as long as the two-dimensional code is detected during the detection, the detection is immediately stopped: Case one: when V=0, the detection functions of the front-view fisheye camera, the left-view fisheye camera and the right-view fisheye camera are started, and the front-view fisheye camera, the left-view fisheye camera and the right-view fisheye camera carry out at most N times of two-dimensional code detection respectively; Case two: when 0V≤5 Km / h, the detection functions of the front-view fisheye camera, the left-view fisheye camera and the right-view fisheye camera are started, if the ultrasonic radar shows that there is no obstacle in the set range in front of, left side and right side of the vehicle, the front-view fisheye camera, the left-view fisheye camera and the right-view fisheye camera carry out at most M times of two-dimensional code detection respectively, if the ultrasonic radar shows that there is an obstacle in the corresponding side in the set range in front of, left side and right side of the vehicle, the fisheye camera on the side with the obstacle carries out two-dimensional code detection, and the fisheye camera on the side without the obstacle carries out at most M times of two-dimensional code detection; Case three: when 5 Km / hV≤15 Km / h, the detection functions of the front-view fisheye camera, the left-view fisheye camera and the right-view fisheye camera are started, it is judged whether the vehicle turning angle is greater than or equal to 30°, if yes, the front-view fisheye camera carries out two-dimensional code detection, if not, the detection function of the front-view fisheye camera is closed, and the left-view fisheye camera and the right-view fisheye camera carry out two-dimensional code detection, if the ultrasonic radar shows that there is no obstacle in the set range in left side and right side of the vehicle, the left-view fisheye camera or the right-view fisheye camera carries out at most M times of two-dimensional code detection, if the ultrasonic radar shows that there is an obstacle in the corresponding side in the set range in left side and right side of the vehicle, the fisheye camera on the side with the obstacle carries out two-dimensional code detection, and the fisheye camera on the side without the obstacle carries out at most M times of two-dimensional code detection; Case four: when 15 Km / h < V≤25 Km / h, the detection function of the front fisheye camera is closed, the detection function of the left fisheye camera and the right fisheye camera is opened, if the ultrasonic radar shows that there is no obstacle in the set range on the left and right sides of the vehicle, then the left fisheye camera or the right fisheye camera is used for at most M times of two-dimensional code detection, if the ultrasonic radar shows that there is an obstacle in the corresponding side of the set range on the left and right sides of the vehicle, then the fisheye camera on the side with the obstacle is used for two-dimensional code detection, and the fisheye camera on the side without the obstacle is used for at most M times of two-dimensional code detection; Case five: when V>25 Km / h, the detection function of the front fisheye camera, the left fisheye camera and the right fisheye camera is closed, and no two-dimensional code detection is performed.

2. The method for detecting a parking lot payment two-dimensional code based on a vehicle-mounted surround view fisheye camera according to claim 1, characterized in that, In case one, N≤3.

3. The method for detecting parking lot payment two-dimensional code based on vehicle-mounted surround view fisheye camera according to claim 1, characterized in that, If the ultrasonic radar shows that there is an obstacle within 3 m in front of the vehicle, then the front fisheye camera is used for two-dimensional code detection; if the ultrasonic radar shows that there is an obstacle within 2 m on the left and right sides of the vehicle, then the left fisheye camera and the right fisheye camera are used for two-dimensional code detection.

4. The method for detecting parking lot payment two-dimensional code based on vehicle-mounted surround view fisheye camera according to claim 1, characterized in that, M is equal to 2, and a waiting time is added between the two detections.

5. The method for detecting parking lot payment two-dimensional code based on vehicle-mounted surround view fisheye camera according to claim 1, characterized in that, The image captured by the front fisheye camera is divided into a sliding window located on the left side of the image and a sliding window located on the right side of the image; the images captured by the left fisheye camera and the right fisheye camera are divided into three sliding windows, two of which are arranged horizontally to form a horizontal sliding window combination, and the other is arranged below the horizontal sliding window combination to form a vertical sliding window combination.

6. The method for detecting parking lot payment two-dimensional code based on vehicle-mounted surround view fisheye camera according to claim 5, characterized in that, In case three, when the turning angle of the vehicle is greater than or equal to 30°, the vehicle turns left, the sliding window located on the right side of the image of the front fisheye camera is opened, and the sliding window located on the left side of the image is closed; when the vehicle turns right, the sliding window located on the left side of the image of the front fisheye camera is opened, and the sliding window located on the right side of the image is closed.

7. The method for detecting parking lot payment two-dimensional code based on vehicle-mounted surround view fisheye camera according to claim 5, characterized in that, When 5 Km / h < V≤25 Km / h, the images of the left fisheye camera and the right fisheye camera are obtained for two-dimensional code detection, and the horizontal sliding window combination and the vertical sliding window combination are alternately used for two-dimensional code detection.

Images