Automatic parking stall line position sensing method and system

Abstract

The invention provides an automatic parking stall line position sensing method and system, and the method comprises the steps: obtaining internal and external parameters of each fisheye camera on a to-be-parked vehicle, carrying out the image preprocessing, and generating an overlook spliced image; inputting the image into a pre-trained parking space line detection model to obtain a parking space line detection result; selecting one of the parking space line detection results as a parking space line position detected at the current moment, and matching a parking space frame detection position of the parking space line detection result with an existing target frame filtering prediction position; when the position is successfully matched with the filtering prediction position of the single existing target frame, the position is determined as the position of a to-be-parked parking space line; when the detection position of each feature point of the parking space line position is successfully matched with a plurality of existing target frame filtering prediction positions, matching the detection position of each feature point of the parking space line position with each existing feature point filtering prediction position of the existing parking space line corresponding to the plurality of existing target frame filtering prediction positions; and determining that the position is a to-be-parked position line position. The position of the to-be-parked parking space line can be quickly and accurately obtained

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method and system for sensing the position of parking spaces for automatic parking. Background Art

[0002] Automatic parking is an advanced automotive technology that can help drivers park their vehicles in vacant parking spaces without manual intervention. Therefore, it is important for automatic parking to reliably provide the location of parking spaces and the location of surrounding obstacles, prevent dangerous events such as collisions, and automatically, safely and accurately park the vehicle in the parking space specified by the user. There are many types of diagonal parking spaces in parking spaces. In order to obtain a good detection effect of diagonal parking spaces, it is usually necessary to collect a large number of diagonal parking space samples in different scenarios for training, which requires a lot of time and cost. During the automatic parking process, the position of the selected parking space relative to the vehicle changes dynamically and there is a cumulative deviation in the relative position information. Even the parking space may be partially blocked, such as the vehicle body covering part of the parking space. In automatic parking based on visual detection, four-way fisheye image detection is required, but the distortion of the four-way fisheye image becomes greater as it moves toward the periphery, resulting in a lower target detection rate; at the same time, four-way fisheye image detection will also lead to a decrease in the detection frame rate. Therefore, how to accurately and stably obtain the relative vehicle position information of the parking space, and how to improve the target detection rate and detection frame rate have become urgent problems to be solved in the industry. Summary of the invention

[0003] In view of this, one or more embodiments of the present specification describe a method and system for sensing parking space lines for automatic parking, so as to eliminate or improve one or more defects existing in the prior art.

[0004] According to a first aspect, a method for sensing a parking space line position of an automatic parking system is provided, the method comprising the following steps:

[0005] Acquire internal reference data and external reference data of each fisheye camera on the vehicle to be parked, and perform image preprocessing on the original images taken by the corresponding fisheye cameras according to the internal reference data and the external reference data one by one to generate a top-view mosaic image around the vehicle to be parked;

[0006] Inputting the top-view spliced ​​image as an input image into a pre-trained parking space line detection model to perform parking space line detection, and obtaining a parking space line detection result, wherein the parking space line detection result includes a plurality of image coordinates corresponding to parking space line positions, wherein each image coordinate includes a plurality of feature points representing a corresponding single parking space line position and a parking space border detection position defined by the feature points;

[0007] A parking line position in the parking line detection result is selected as the parking line position detected at the current moment, until all the parking line positions in the parking line detection result are selected, wherein after each selection of the parking line position detected at the current moment, the selected parking line position detected at the current moment is matched one by one with all the existing parking lines in the existing parking line buffer pool, and during each match, the parking space frame detection position of the parking line position detected at the current moment is matched with the existing target frame filter prediction position of the existing parking line;

[0008] In the case where the parking space frame detection position successfully matches only a single existing target frame filter prediction position, determining the parking space line position detected at the current moment as the automatic parking waiting parking space line position;

[0009] In the case where the parking space frame detection position successfully matches the multiple existing target frame filter prediction positions, the image coordinates of each feature point of the parking space line position detected at the current moment are converted into global coordinates, and the detection position of each feature point of the parking space line position detected at the current moment is matched with each existing feature point filter prediction position of the existing parking space line corresponding to each of the multiple existing target frame filter prediction positions that have been matched successfully according to the global coordinates. In the case of successful matching, the parking space line position detected at the current moment is determined to be the automatic parking waiting parking space line position;

[0010] The pre-trained parking space line detection model is trained using a deep learning method using a data set, and the data set includes a top-down mosaic image around the vehicle to be parked.

[0011] In some embodiments, when the parking space border detection position only successfully matches a single existing target frame filter prediction position, before determining that the parking space line position detected at the current moment is the automatic parking waiting space line position, it also includes, within a predetermined continuous time period: judging whether the parking space border detection position successfully matches a single existing target frame filter prediction position multiple times; if multiple matches are successful, determining that the parking space line position detected at the current moment is the automatic parking waiting space line position.

[0012] In some embodiments, when the parking space border detection position is successfully matched with multiple existing target frame filter prediction positions, the image coordinates of each feature point of the parking line position detected at the current moment are converted into global coordinates, and based on the global coordinates, the detection position of each feature point of the parking line position detected at the current moment is matched with each existing feature point filter prediction position of the existing parking lines corresponding to each of the multiple existing target frame filter prediction positions that have been successfully matched. In the case of successful matching, before determining that the parking line position detected at the current moment is the automatic parking waiting line position, it also includes, within a predetermined continuous time period: judging whether the detection position of each feature point of the parking line position detected at the current moment and each existing feature point filter prediction position of the existing parking lines corresponding to each of the existing target frame filter prediction positions have been successfully matched multiple times; if multiple matches are successful, determining that the parking line position detected at the current moment is the automatic parking waiting line position.

[0013] In some embodiments, image preprocessing is performed on the original images taken by the corresponding fisheye camera according to the internal reference data and the external reference data one by one to generate a bird's-eye view mosaic image around the vehicle to be parked, including: performing distortion correction on the corresponding original images according to the internal reference data and the external reference data one by one to generate a corrected image, mapping the corrected image to the ground to generate a bird's-eye view; and stitching all the bird's-eye views with the ground as a reference system to generate the bird's-eye view mosaic image.

[0014] In some embodiments, all bird's-eye views are stitched together with the ground as a reference system to generate the overhead stitched image, including: adjusting the brightness information of the corresponding fisheye cameras according to the brightness information of the overlapping areas of adjacent images of all the bird's-eye views so that the brightness of all the bird's-eye views is consistent; and stitching all the bird's-eye views with adjusted brightness to generate the overhead stitched image.

[0015] In some embodiments, a number of feature points are marked in a predetermined order through the following steps: marking the entrance feature points according to the direction of the parking space line position, wherein, when the parking space line position is an obtuse parking space, the obtuse-angle side entrance point that forms an obtuse angle with the vehicle to be parked is marked as the first entrance feature point, and a perpendicular line is drawn from the first entrance feature point to the other side line, and the perpendicular point between the perpendicular line and the other side line is marked as the second entrance feature point; when the parking space line position is a vertical parking space or a horizontal parking space, the two inner corner points at the entrance point are marked as the first entrance feature point and the second entrance feature point respectively; after extending a fixed length along the two side lines of the parking space line position, the end point of the parking space line is marked as the end feature point; when the end point of the parking space line exceeds the image area, the boundary of the image area is used as the end point, and the end point is marked as the end feature point.

[0016] In some embodiments, the parking space border detection position of the parking space line position detected at the current moment is matched with the existing target frame filter prediction position of the existing parking space line, including: calculating the overlap between the parking space border detection position and the existing target frame filter prediction position; when the overlap is greater than a predetermined threshold, the parking space border detection position is successfully matched with the existing target frame filter prediction position; when the overlap is less than or equal to a predetermined threshold, the parking space border detection position is unmatched with the existing target frame filter prediction position.

[0017] In some embodiments, after the parking space border detection position fails to match the existing target frame filter prediction position, it also includes: adding the parking space border detection position of the parking line position detected at the current moment to the existing parking line cache pool, converting the image coordinates of the feature points of the parking line position detected at the current moment into global coordinates, and adding the feature points of the parking line position detected at the current moment that have been converted to global coordinates to the existing parking line cache pool.

[0018] In some embodiments, the detection position of each feature point of the parking line position detected at the current moment is matched with each existing feature point filtered prediction position of each existing parking line corresponding to each of the multiple existing target frame filtered prediction positions that have been successfully matched, including: selecting one of the existing parking lines corresponding to each of the multiple existing target frame filtered prediction positions that have been successfully matched as the current existing parking line to be matched, until all corresponding existing parking lines are selected, wherein after each selection of the current existing parking line to be matched, the detection position of each feature point of the parking line position detected at the current moment is matched one by one with each existing feature point filtered prediction position of the current existing parking line to be matched in the same predetermined order as the marked feature points through the following steps: calculating the detection position of the first feature point of the parking line position detected at the current moment and the filter prediction position of the existing parking line to be matched; The first corresponding distance of the first existing feature point filtering prediction position corresponding to the parking line; calculating the second corresponding distance between the detection position of the second feature point of the parking line position detected at the current moment and the corresponding second existing feature point filtering prediction position of the existing parking line to be matched; and so on, until calculating the Nth corresponding distance between the detection position of the Nth feature point of the parking line position detected at the current moment and the corresponding Nth existing feature point filtering prediction position of the existing parking line to be matched, where N is equal to the number of feature points; calculating the distance sum from the first corresponding distance to the Nth corresponding distance; selecting the smallest distance sum from all the calculated distance sums; the detection position of each feature point of the parking line position detected at the current moment corresponding to the smallest distance sum successfully matches the filtering prediction position of each existing feature point of the existing parking line.

[0019] In some embodiments, after the parking space border detection position is successfully matched with only a single existing target frame filter prediction position, the method further includes: weighting the parking space border detection position and the existing target frame filter prediction position to obtain the latest target frame prediction position; updating the existing target frame filter prediction position of the existing parking line in the existing parking line buffer pool according to the latest target frame prediction position; and after the detection position of each feature point of the parking space line position detected at the current moment corresponding to the minimum distance is successfully matched with each existing feature point filter prediction position of the existing parking line, the method further includes: weighting the parking space border detection position and the existing target frame filter prediction position to obtain the latest target frame prediction position; updating the existing target frame filter prediction position of the existing parking line in the existing parking line buffer pool according to the latest target frame prediction position; and The parking space border detection position of the set parking space line is weightedly summed with the existing target frame filter prediction position of the existing parking line to obtain the latest target frame prediction position; according to the latest target frame prediction position, the existing target frame filter prediction position of the existing parking line in the existing parking line buffer pool is updated; and the detection position of each feature point of the parking line position detected at the current moment is weightedly summed with the corresponding existing feature point filter prediction position of each existing parking line to obtain the latest detection position of each feature point; according to the latest detection position of each feature point, the existing feature point filter prediction position of the existing parking line in the existing parking line buffer pool is updated one by one.

[0020] According to the second aspect, there is provided an automatic parking space line position perception system, comprising a processor, a memory and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method described in the first aspect.

[0021] According to a third aspect, a computer-readable storage medium is provided, on which a computer program / instruction is stored, characterized in that when the computer program / instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0022] According to a fourth aspect, a computer program product is provided, comprising a computer program / instruction, wherein the computer program / instruction implements the steps of the method described in the first aspect when executed by a processor.

[0023] The automatic parking space line position sensing method and system of the present invention can accurately and stably obtain the position information of the parking space relative to the vehicle, thereby improving the target detection rate and the detection frame rate.

[0024] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0025] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0027] Figure 1 A flow chart of a method for sensing parking space line position of automatic parking according to an embodiment is shown;

[0028] Figure 2 shows an original image and a top-view stitched image taken by a fisheye camera according to an embodiment;

[0029] Figures 3a to 3b Effect diagrams of a bird's-eye view before and after brightness equalization adjustment according to an embodiment are respectively shown;

[0030] Figure 4 A flowchart of a parking space line detection model training according to an embodiment is shown;

[0031] Figures 5a to 5c The annotation effect diagrams of feature points in various parking space locations are shown respectively. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0033] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0034] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0035] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0036] Figure 1A flow chart of a method for sensing the position of parking spaces for automatic parking according to an embodiment is shown. The method can be executed by any device, equipment and system with computing and processing capabilities. As shown in the figure, the method for sensing the position of parking spaces for automatic parking includes at least the following steps.

[0037] First, in step S101, the internal reference data and external reference data of each fisheye camera on the vehicle to be parked are obtained, and the original images taken by the corresponding fisheye cameras are preprocessed according to the internal reference data and the external reference data to generate a bird's-eye view mosaic around the vehicle to be parked. It should be understood that the original images taken by the fisheye camera produce image distortion, and the error caused by the distortion can be reduced after preprocessing. In one embodiment, the corresponding original images can be distorted according to the internal reference data and the external reference data to generate a corrected image, and the corrected image can be mapped to the ground to generate a bird's-eye view; all the bird's-eye views are spliced ​​with the ground as the reference system to generate a bird's-eye view mosaic. For example, referring to Figure 2 , the left side is the original image taken by four fisheye cameras, and the right side is the top-down stitching image.

[0038] In one embodiment, before generating the top-view mosaic image, the brightness information of the fisheye cameras corresponding to each other is adjusted according to the brightness information of the overlapping areas of adjacent images of all the bird's-eye views, so that the brightness of all the bird's-eye views is consistent. Then, all the bird's-eye views with adjusted brightness are mosaicked to generate the top-view mosaic image. Figure 3a and Figure 3b The following is a comparison of the effects before and after the brightness balance adjustment of the bird's-eye view. Figure 3a This is a bird's-eye view before brightness equalization; Figure 3b This is a bird's-eye view after brightness is balanced. Figure 3a and Figure 3b From the effect comparison chart, we can see that by adjusting the brightness of the fisheye image, the brightness information around the parking space line can be kept as consistent as possible. This can reduce the intra-class difference, thereby improving the detection effect and reducing the data volume requirement.

[0039] After the top-view mosaic image is generated, the top-view mosaic image can be used as an input image to be input into a pre-trained parking line detection model to perform parking line detection, and obtain a parking line detection result, as shown in step S103. It should be understood that the pre-trained parking line detection model is trained using a data set using a deep learning method, and is used for real-time parking line detection after training. The data set includes a top-view mosaic image around the vehicle to be parked, and the deep learning method includes but is not limited to: yolov5.

[0040] In one embodiment, the parking space line detection result obtained includes a number of image coordinates corresponding to the parking space line positions, each image coordinate includes a number of feature points representing the corresponding single parking space line position and the parking space frame detection position defined by these feature points.

[0041] See also Figure 4 , shows a flow chart of parking line detection model training according to an embodiment. Step S401, obtain the internal reference data and external reference data of each fisheye camera. Then, step S403, perform image preprocessing on the original image taken by the corresponding fisheye camera according to the internal reference data and the external reference data one by one, and generate a bird's-eye view splicing map. When performing preprocessing, the method described above can be used to perform distortion correction on the image to generate a bird's-eye view, and perform brightness equalization on the bird's-eye view. Then, in step S405, feature points are annotated on the bird's-eye view splicing map, and these feature points are used to define a parking space border detection position. The specific annotation method is described in detail below. Step S407, train the bird's-eye view splicing map annotated with feature points and parking space border detection positions. The training method includes but is not limited to yolov5. In step S409, generate a parking line detection model. The parking line detection model generated after training is used for real-time parking line detection during automatic parking line position perception.

[0042] In one embodiment, a plurality of feature points are marked in a predetermined order by the following steps: the entrance feature points are marked according to the direction of the parking space line position. When the parking space line position is an oblique parking space, the obtuse side entrance point that is in an obtuse angle direction with the vehicle to be parked is marked as the first entrance feature point, and a perpendicular line is drawn from the first entrance feature point to the other side line, and the perpendicular point between the perpendicular line and the other side line is marked as the second entrance feature point. When the parking space line position is a vertical parking space or a horizontal parking space, the two inner corner points at the entrance point are marked as the first entrance feature point and the second entrance feature point respectively. The end point of the parking space line is marked as the end feature point after being extended by a fixed length along the two side lines of the parking space line position. When the end point of the parking space line exceeds the image area, the end point is marked as the end feature point with the boundary of the image area as the end point. It should be understood that there may be two end feature points corresponding to the entrance feature point. When the end point of the parking space line exceeds the image area, one end point of the parking space line may exceed the image area, or both end points of the parking space line may exceed the image area. Obviously, although only two entrance feature points are described here, those skilled in the art will appreciate that more than two entrance feature points may be marked, and similarly, more than two tail feature points may be marked.

[0043] See also Figures 5a to 5c , which respectively show the annotation effect diagrams of feature points under various parking space positions. Figure 5a The figure shows the effect of feature point annotation in the case of vertical parking space. Figure 5aAs shown, the parking space line position on the left side of the figure is marked with point 1 starting from the tail feature point, and the two inner corner points at the entrance point, i.e., the first entrance feature point and the second entrance feature point are marked with point 2 and point 3 in a counterclockwise direction, and the other tail feature point is marked with point 4. The border formed by the line segment connecting marked points 1 to 4 defines a parking space border detection position. Figure 5a The marking order of the parking space line position on the right side is the same as that on the left side, and it is also marked in a counterclockwise direction starting from the tail feature point. Figure 5b The effect diagram of feature point marking in the case of a horizontal parking space is shown. The marking steps are similar and will not be repeated here.

[0044] Figure 5c The following figure shows the effect of feature point annotation in the case of an oblique parking space. Figure 5c As shown on the left, the end point of the parking space line is marked as the end feature point after extending the side line at an obtuse angle to the vehicle to be parked by a fixed length, and the end feature point here is marked as the upper mark point 1. In counterclockwise order, the obtuse side entrance point at an obtuse angle to the vehicle to be parked is marked as the first entrance feature point, and the upper mark point 2 is marked. A perpendicular line is drawn from the first entrance feature point to the other side line, and the perpendicular point of the perpendicular line and the other side line is marked as the second entrance feature point, and the second entrance feature point is marked as the upper mark point 3, and the other end feature point is marked as the upper mark point 4. The frame enclosed by the line segment connected by the mark points 1 to 4 defines a parking space frame detection position.

[0045] It should be understood that Figures 5a to 5c The feature points in are marked in counterclockwise order, but obviously clockwise order or other specified order is also possible. When marking feature points, a predetermined order is used for marking, and then when matching feature points, the matching is also carried out in accordance with the predetermined order, which can speed up the matching and improve the matching efficiency. In the case of an oblique parking space, when marking feature points, a vertical line is drawn from the obtuse angle side to the opposite side of the obtuse angle with the vehicle to be parked, and the feature points at the entrance on the other side are automatically obtained, thereby making the oblique parking space and the vertical parking space have a high similarity, thereby greatly reducing the amount of data for the oblique parking space.

[0046] Next, the parking line detection results obtained are used to track and match the parking lines. First, large-scale matching, i.e., frame matching, is performed. That is, in step S105, a parking line position in the parking line detection results is selected as the parking line position detected at the current moment, and the parking space frame detection position of the parking line position detected at the current moment is matched with the existing target frame filter prediction position of the existing parking line. It should be understood that step S105 can be performed multiple times until all parking line positions in the parking line detection results are selected.

[0047] When performing frame matching, the degree of overlap between the parking space border detection position and the existing target frame filter prediction position is first calculated. When the degree of overlap is greater than a predetermined threshold, the parking space border detection position successfully matches the existing target frame filter prediction position. In one embodiment, the predetermined threshold is 95%. It should be understood that other values ​​can be selected for the predetermined threshold based on the matching accuracy. When the degree of overlap is less than or equal to a predetermined threshold, the parking space border detection position fails to match the existing target frame filter prediction position. Obviously, frame matching can quickly filter out a large number of irrelevant matches, greatly improving the matching speed.

[0048] Obviously, when the frame match is successful, it can be a single frame match or multiple frame matches. Step S107, when the parking space frame detection position only successfully matches a single existing target frame filter prediction position, determine that the parking space line position detected at the current moment is the automatic parking waiting parking space line position.

[0049] In one embodiment, when the parking space frame detection position only successfully matches a single existing target frame filter prediction position, before determining that the parking space line position detected at the current moment is the automatic parking waiting parking space line position, it also includes, within a predetermined continuous time period: judging whether the parking space frame detection position successfully matches a single existing target frame filter prediction position multiple times, and if multiple matches are successful, then determining that the parking space line position detected at the current moment is the automatic parking waiting parking space line position. It should be understood that by judging whether the frame is successfully matched multiple times within a predetermined continuous time period, unreliable parking spaces can be filtered out, thereby improving the accuracy of parking space line detection. In one embodiment, the predetermined continuous time period is 30 seconds. Obviously, based on the detection accuracy, other values ​​can be selected for the predetermined continuous time period.

[0050] In one embodiment, after the parking space border detection position successfully matches only a single existing target frame filter prediction position, the method further includes: performing a weighted summation of the parking space border detection position and the existing target frame filter prediction position to obtain the latest target frame prediction position; and updating the existing target frame filter prediction position of the existing parking space line in the existing parking space line buffer pool according to the latest target frame prediction position. By instantly updating the existing target frame filter prediction position of the existing parking space line in the buffer pool, the latest frame position can be always cached in the buffer pool.

[0051] Step S109, when the parking space border detection position is successfully matched with multiple existing target frame filter prediction positions, the detection position of each feature point of the parking space line position detected at the current moment is matched with each existing feature point filter prediction position of the existing parking space lines corresponding to each of the multiple existing target frame filter prediction positions that have been matched successfully. When the match is successful, the parking space line position detected at the current moment is determined to be the automatic parking waiting parking line position.

[0052] In one embodiment, when the parking space frame detection position matches the multiple existing target frame filter prediction positions successfully, the detection position of each feature point of the parking space line position detected at the current moment is matched with each existing feature point filter prediction position of each existing parking space line corresponding to each of the multiple existing target frame filter prediction positions that have been matched successfully. In the case of successful matching, before determining that the parking space line position detected at the current moment is the automatic parking waiting parking space line position, it also includes, within a predetermined continuous time period: judging whether the detection position of each feature point of the parking space line position detected at the current moment and each existing feature point filter prediction position of each existing parking space line corresponding to each of the existing target frame filter prediction positions are matched successfully multiple times, and if they are matched successfully multiple times, then determining that the parking space line position detected at the current moment is the automatic parking waiting parking space line position. It should be understood that by judging whether the feature points are matched successfully multiple times within a predetermined continuous time period, unreliable parking spaces can be filtered out, thereby improving the accuracy of parking space line detection. In one embodiment, the predetermined continuous time period is 30 seconds. Obviously, based on the detection accuracy, other values ​​can be selected for the predetermined continuous time period.

[0053] In one embodiment, the detection position of each feature point of the parking line position detected at the current moment is matched with each existing feature point filter prediction position of the existing parking lines corresponding to each of the multiple existing target frame filter prediction positions that have been successfully matched, including: selecting one of the existing parking lines corresponding to each of the multiple existing target frame filter prediction positions that have been successfully matched as the current existing parking line to be matched, until all corresponding existing parking lines are selected. After selecting the existing parking line to be matched each time, the detection positions of each feature point of the parking line position detected at the current moment are matched one by one with the filter prediction positions of each existing feature point of the existing parking line to be matched in the same predetermined order as the marked feature points through the following steps: calculating the first corresponding distance between the detection position of the first feature point of the parking line position detected at the current moment and the corresponding first existing feature point filter prediction position of the existing parking line to be matched; calculating the second corresponding distance between the detection position of the second feature point of the parking line position detected at the current moment and the corresponding second existing feature point filter prediction position of the existing parking line to be matched; and so on, until calculating the Nth corresponding distance between the detection position of the Nth feature point of the parking line position detected at the current moment and the corresponding Nth existing feature point filter prediction position of the existing parking line to be matched, where N is equal to the number of feature points. Calculate the distance sum for the first corresponding distance to the Nth corresponding distance, select the smallest distance sum from all the calculated distance sums, and the detection position of each feature point of the parking line position detected at the current moment corresponding to the smallest distance sum successfully matches the filter prediction positions of each existing feature point of the existing parking line.

[0054] In one embodiment, after the minimum distance and the detection position of each feature point of the parking line position detected at the current moment corresponding to the detection position are successfully matched with each existing feature point filter prediction position of the existing parking line, it also includes: weighted summing the parking space frame detection position of the parking line position detected at the current moment and the existing target frame filter prediction position of the existing parking line to obtain the latest target frame prediction position; updating the existing target frame filter prediction position of the existing parking line in the existing parking line buffer pool according to the latest target frame prediction position; and weighted summing the detection position of each feature point of the parking line position detected at the current moment with each existing feature point filter prediction position of the corresponding existing parking line one by one to obtain the latest detection position of each feature point; updating each existing feature point filter prediction position of the existing parking line in the existing parking line buffer pool one by one according to the latest detection position of each feature point. By instantly updating the existing target frame filter prediction position and the existing feature point filter prediction position of the existing parking line in the buffer pool, the latest frame position and feature point position can be always cached in the buffer pool.

[0055] In one embodiment, when the frame matching fails, the parking space border detection position of the parking space line position detected at the current moment is added to the existing parking space line cache pool, the image coordinates of the feature points of the parking space line position detected at the current moment are converted into global coordinates, and the feature points of the parking space line position detected at the current moment that have been converted into global coordinates are added to the existing parking space line cache pool.

[0056] From this, it can be seen that the automatic parking space line position perception method described above can accurately and stably obtain the position information of the parking space relative to the vehicle, thereby improving the target detection rate and detection frame rate.

[0057] Corresponding to the above method, the present invention also provides an automatic parking space line position perception system, including a processor, a memory and a computer program / instruction stored in the memory, the processor is used to execute the computer program / instruction, and when the computer program / instruction is executed, the system implements the steps of the above method

[0058] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described above are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0059] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, wherein the computer program / instruction implements the steps of the method described above when executed by a processor.

[0060] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0061] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0062] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0063] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for sensing the position of parking spaces for automatic parking, characterized in that: The method comprises: Acquire internal reference data and external reference data of each fisheye camera on the vehicle to be parked, and perform image preprocessing on the original images taken by the corresponding fisheye cameras according to the internal reference data and the external reference data one by one to generate a top-view mosaic image around the vehicle to be parked; Inputting the top-view spliced ​​image as an input image into a pre-trained parking space line detection model to perform parking space line detection, and obtaining a parking space line detection result, wherein the parking space line detection result includes a plurality of image coordinates corresponding to parking space line positions, wherein each image coordinate includes a plurality of feature points representing a corresponding single parking space line position and a parking space border detection position defined by the feature points; A parking line position in the parking line detection result is selected as the parking line position detected at the current moment, until all the parking line positions in the parking line detection result are selected, wherein after each selection of the parking line position detected at the current moment, the selected parking line position detected at the current moment is matched one by one with all the existing parking lines in the existing parking line buffer pool, and during each match, the parking space frame detection position of the parking line position detected at the current moment is matched with the existing target frame filter prediction position of the existing parking line; In the case where the parking space frame detection position successfully matches only a single existing target frame filter prediction position, determining the parking space line position detected at the current moment as the automatic parking waiting parking space line position; In the case where the parking space frame detection position successfully matches the multiple existing target frame filter prediction positions, the image coordinates of each feature point of the parking space line position detected at the current moment are converted into global coordinates, and the detection position of each feature point of the parking space line position detected at the current moment is matched with each existing feature point filter prediction position of the existing parking space line corresponding to each of the multiple existing target frame filter prediction positions that have been matched successfully according to the global coordinates. In the case of successful matching, the parking space line position detected at the current moment is determined to be the automatic parking waiting parking space line position; The pre-trained parking space line detection model is trained using a deep learning method using a data set, and the data set includes a top-down mosaic image around the vehicle to be parked.

2. The method according to claim 1, characterized in that Before determining that the parking space line position detected at the current moment is the automatic parking waiting parking space line position, the method further includes, within a predetermined continuous time period: Determine whether the parking space frame detection position is successfully matched with a single existing target frame filter prediction position multiple times, and if multiple matches are successful, determine that the parking space line position detected at the current moment is the automatic parking waiting parking space line position; or Determine whether the detection positions of each feature point of the parking space line position detected at the current moment and the existing feature point filtered prediction positions of the existing parking space lines corresponding to the existing target frame filtered prediction positions have been matched successfully multiple times. If multiple matches are successful, determine that the parking space line position detected at the current moment is the automatic parking waiting parking line position.

3. The method according to claim 2, characterized in that The method of performing image preprocessing on the original images taken by the corresponding fisheye camera according to the internal reference data and the external reference data one by one to generate a top-view mosaic image around the vehicle to be parked includes: Performing distortion correction on the corresponding original images according to the internal reference data and the external reference data one by one to generate a corrected image, and mapping the corrected image to the ground to generate a bird's-eye view; All the bird's-eye views are spliced ​​with the ground as a reference system to generate the top-view spliced ​​view.

4. The method according to claim 3, characterized in that The step of stitching all the bird's-eye views with the ground as a reference system to generate the top-view stitched view includes: According to the brightness information of the overlapping areas of adjacent images of all the bird's-eye views, the brightness information of the fisheye cameras corresponding to each other is adjusted so that the brightness of all the bird's-eye views is consistent; All the bird's-eye views after adjusting the picture brightness are stitched together to generate the top-view stitched view.

5. The method according to any one of claims 1 to 4, characterized in that The plurality of feature points are marked in a predetermined order by the following steps: The entrance feature points are marked according to the direction of the parking space line position, wherein, when the parking space line position is an oblique parking space, the obtuse-angle side entrance point that forms an obtuse angle with the vehicle to be parked is marked as the first entrance feature point, and a perpendicular line is drawn from the first entrance feature point to the other side line, and the perpendicular point between the perpendicular line and the other side line is marked as the second entrance feature point; when the parking space line position is a vertical parking space or a horizontal parking space, the two inner corner points at the entrance point are marked as the first entrance feature point and the second entrance feature point respectively; After extending a fixed length along the two side line directions of the parking space line position, the end point of the parking space line is marked as the end feature point; when the end point of the parking space line exceeds the image area, the boundary of the image area is used as the end point, and the end point is marked as the end feature point.

6. The method according to any one of claims 1 to 4, characterized in that The matching of the parking space frame detection position of the parking space line position detected at the current moment with the existing target frame filtering prediction position of the existing parking space line includes: Calculating the degree of overlap between the parking space frame detection position and the existing target frame filter prediction position; When the overlap is greater than a predetermined threshold, the parking space frame detection position successfully matches the existing target frame filter prediction position; When the overlap is less than or equal to a predetermined threshold, the parking space frame detection position fails to match the existing target frame filter prediction position.

7. The method according to claim 6, characterized in that The step of matching the detected position of each feature point of the parking space line position detected at the current moment with each existing feature point filter prediction position of the existing parking space line corresponding to each of the multiple successfully matched existing target frame filter prediction positions comprises: One of the existing parking lines corresponding to the multiple existing target frame filter prediction positions that have been successfully matched is selected as the current existing parking line to be matched, until all the corresponding existing parking lines are selected, wherein after the current existing parking line to be matched is selected each time, the detection positions of the feature points of the parking line position detected at the current moment are matched one by one with the filter prediction positions of the existing feature points of the current existing parking line to be matched in the same predetermined order as the marked feature points through the following steps: calculating the first corresponding distance between the detection position of the first feature point of the parking line position detected at the current moment and the first existing feature point filter prediction position corresponding to the existing parking line to be matched; calculating the second corresponding distance between the detection position of the second feature point of the parking line position detected at the current moment and the second existing feature point filter prediction position corresponding to the existing parking line to be matched; and so on, until the Nth corresponding distance between the detection position of the Nth feature point of the parking line position detected at the current moment and the Nth existing feature point filter prediction position corresponding to the existing parking line to be matched is calculated, where N is equal to the number of feature points; and calculating the distance sum from the first corresponding distance to the Nth corresponding distance; Select the smallest distance sum from all the calculated distance sums; The minimum distance and the corresponding detection position of each feature point of the parking space line position detected at the current moment are successfully matched with the filtered predicted positions of each existing feature point of the existing parking space line.

8. An automatic parking space line position sensing system, comprising a processor, a memory and a computer program / instruction stored in the memory, characterized in that: The processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images