Parking space detection method, terminal device and storage medium

Abstract

The present invention relates to a parking space detection method, a terminal device and a storage medium. The method comprises: collecting a panoramic view of the surroundings of a vehicle from a bird's-eye view perspective; detecting all parking space corner points, parking space separation lines and parking space entry lines in the panoramic view of the surroundings of the vehicle; matching the parking space corner points with the parking space separation lines, and setting the corner points and separation lines that are successfully matched and unsuccessfully matched to different confidence levels; judging whether the parking space entry line matches the parking space corner point or the parking space separation line by different distance thresholds according to different confidence levels; determining a parking space according to a parking space entry line that satisfies the requirement that both endpoints of the parking space entry line have parking space corner points or parking space separation lines that match it, and two parking space corner points and two parking space separation lines that match it; the present invention can ensure the stability of the corner points and greatly reduce the false detection rate.

Description

Technical Field

[0001] The present invention relates to the field of parking space detection, and in particular to a parking space detection method, terminal equipment and storage medium. Background Art

[0002] Autonomous driving technology is a technology that uses on-board sensors to perceive the surrounding environment and relies on technologies such as artificial intelligence, computer vision, radar ranging, and planning control to enable a computer system to drive the vehicle safely and automatically without human control.

[0003] Automatic parking is the application of autonomous driving technology in the specific driving condition of parking. Automatic parking systems usually use radar or image sensors to detect parking spaces around the vehicle body, including the shape, size and distance of the parking space, and sense the location of obstacles, and then automatically park the vehicle into the parking space. In actual automatic parking applications, the cost of using visual technology is low, and it can avoid the disadvantage of radar being unable to perceive parking spaces due to the lack of reference objects. Summary of the invention

[0004] In order to solve the above problems, the present invention proposes a parking space detection method, a terminal device and a storage medium.

[0005] The specific plan is as follows:

[0006] A parking space detection method comprises the following steps:

[0007] S1: Collects a panoramic view of the vehicle's surroundings from a bird's-eye view;

[0008] S2: Detecting all parking space corner points, parking space separation lines and parking space entry lines according to the panoramic image around the vehicle;

[0009] S3: Matching parking space corner points with parking space separation lines, grouping successfully matched parking space corner points and parking space separation lines, and setting the confidence levels of both to the first confidence level, and setting the confidence levels of parking space corner points that have no parking space separation lines to match them and parking space separation lines that have no parking space corner points to match them to the second confidence level;

[0010] S4: matching the parking space entry line with the parking space corner point or the parking space separation line, and if the confidence level of the parking space corner point or the parking space separation line is the first confidence level, determining whether the parking space entry line matches the parking space corner point or the parking space separation line according to the second distance threshold; if the confidence level of the parking space corner point or the parking space separation line is the second confidence level, determining whether the parking space entry line matches the parking space corner point or the parking space separation line according to a third distance threshold that is smaller than the second distance threshold;

[0011] S5: Determine whether there is a parking space entry line that satisfies: both end points of the parking space entry line have matching parking space corner points or parking space separation lines. If so, output the parking space determined by the parking space entry line and the two matching parking space corner points and two parking space separation lines.

[0012] Furthermore, a method for collecting a panoramic view of the vehicle's surroundings from a bird's-eye view perspective is as follows: collecting images through multiple cameras installed in multiple directions of the vehicle, and after splicing the collected multiple images, the spliced ​​images are transformed into a panoramic stitching image from a bird's-eye view perspective.

[0013] Furthermore, the multiple directions include at least four directions of the vehicle: front, rear, left, and right.

[0014] Furthermore, the parking space corner points, parking space separation lines and parking space entry lines are obtained through a neural network model; the network structure of the model includes a backbone network, an image segmentation branch and an object detection branch;

[0015] The panoramic image around the vehicle is input from the backbone network as an input image and downsampled;

[0016] After the image segmentation branch upsamples the image output by the backbone network to the same resolution as the input image, it obtains the corner point heat map of each pixel in the image containing the parking space corner point and the separation line probability map of each pixel in the image belonging to the parking space separation line through two channels;

[0017] The target detection branch detects the parking space entry line in the image according to the image output by the backbone network, and outputs the entry line midpoint probability map of each pixel in the image containing the midpoint of the parking space entry line, the offset dx of the midpoint of the parking space entry line relative to the upper left corner coordinate point of the pixel grid where it is located in the x-axis direction, the offset dy of the midpoint of the parking space entry line relative to the upper left corner coordinate point of the pixel grid where it is located in the y-axis direction, the length l of the parking space entry line, the cosine value cosθ of the angle between the parking space entry line and the x-axis direction, and the sine value sinθ of the angle between the parking space entry line and the x-axis direction respectively through 6 channels;

[0018] The parking space corner points are extracted according to the corner point heat map output by the image segmentation branch, the parking space separation lines are extracted according to the separation line probability map output by the image segmentation branch, and the parking space entry line is calculated according to the output of the target detection branch.

[0019] Furthermore, a method for extracting corner points of parking spaces based on a corner point heat map is as follows: searching in the corner point heat map for a rectangular frame range in which the coordinate value of the center point is the largest and greater than 0.5 and the coordinate values ​​of the surrounding points gradually decrease, and determining whether the coordinate value of the center point within the rectangular frame is greater than a first threshold value; if so, taking the center point as a corner point of the parking space.

[0020] Furthermore, a method for extracting parking space separation lines based on the separation line probability map is as follows: after binarizing the separation line probability map, a set of pixel points whose values ​​are 1 and are adjacent to each other is searched in the binarized map, and the points in the set are fitted with a least square straight line to obtain the parking space separation lines.

[0021] Furthermore, the method for calculating the parking space entry line according to the output of the target detection branch is as follows: extract the pixel corresponding to the midpoint of the entry line according to the midpoint probability map of the entry line output by the target detection branch, and calculate the coordinates of the two endpoints of the entry line according to the pixels corresponding to the midpoint of the entry line:

[0022] P e-p1 =(x+dx-l / 2×cosθ,y+dy-l / 2×sinθ)

[0023] P e-p2 =(x+dx+l / 2×cosθ,y+dy+l / 2×sinθ)

[0024] Among them, P e-p1 , P e-p2 They represent the two endpoints of the entry line, and x and y represent the x-axis coordinate and y-axis coordinate of the upper left corner of the pixel grid where the midpoint of the entry line is located.

[0025] Further, the method for determining whether the parking space corner point matches the parking space separation line successfully is: determining whether ||P c -L s-p1 ||2<T1or||P c -L s-p2 ||2<T1,P c Indicates the coordinates of the corner points of the parking space, L s-p1 and L s-p2 They respectively represent the coordinates of the two endpoints of the parking space separation line, ||.||2 represents the 2-norm, and T1 represents the first distance threshold. If yes, it is determined that the parking space corner point and the parking space separation line are matched successfully; otherwise, it is determined that the parking space corner point and the parking space separation line are matched unsuccessfully.

[0026] A parking space detection terminal device comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method in the embodiment of the present invention when executing the computer program.

[0027] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described above in an embodiment of the present invention are implemented.

[0028] The present invention adopts the above technical solution and adopts the high and low threshold strategy. For the combination of corner points and separation lines that successfully match, the matching threshold with the entry line is relaxed, and vice versa, a stricter matching threshold is set, which can improve the detection rate of parking space detection. The parking space is determined by combining the three parking space elements of corner points, separation lines and entry lines, rather than determining the parking space by inferring the corner point through the endpoints of the separation line and the entry line, which can ensure the stability of the corner point and greatly reduce the false detection rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Shown is a flow chart of Embodiment 1 of the present invention.

[0030] Figure 2 Shown is a schematic diagram of the network structure of the model in this embodiment.

[0031] Figure 3 Shown is a schematic diagram of the definition of various parameters in the target detection branch in this embodiment. DETAILED DESCRIPTION

[0032] To further illustrate various embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementations and advantages of the present invention.

[0033] The present invention will now be further described with reference to the accompanying drawings and specific implementation methods.

[0034] Embodiment 1:

[0035] The embodiment of the present invention provides a parking space detection method, such as Figure 1 As shown, the method comprises the following steps:

[0036] S1: Collect a panoramic view of the vehicle's surroundings from a bird's-eye view.

[0037] In this embodiment, the method for collecting the panoramic image of the vehicle surroundings from a bird's-eye view perspective is as follows: collect images through multiple cameras installed in multiple directions of the vehicle, and after splicing the collected multiple images, transform the spliced ​​images into a panoramic spliced ​​image from a bird's-eye view perspective. The multiple directions include at least four directions: the front, rear, left, and right of the vehicle. When splicing the multiple images, the splicing is performed according to the camera calibration parameters corresponding to the cameras.

[0038] S2: Detect all parking space corner points, parking space separation lines, and parking space entry lines according to the panoramic image around the vehicle.

[0039] In this embodiment, the parking space corner points, parking space separation lines and parking space entry lines are obtained through a neural network model. Figure 2 As shown, the network structure of the model in this embodiment includes a downsampling backbone network, an upsampling image segmentation branch and a target detection branch.

[0040] The panoramic image around the vehicle is input from the backbone network as an input image and downsampled (32 times downsampling is used in this embodiment).

[0041] After the image segmentation branch upsamples the image output by the backbone network to the same resolution as the input image, it obtains the corner point heat map of each pixel in the image containing the parking space corner point and the separation line probability map of each pixel in the image belonging to the parking space separation line through two channels.

[0042] Define the parking space corner point P c (x c ,y c ), then the values ​​of the pixels around the corner points in the heat map are Among them, h(x,y) represents the corner point P c (x c ,y c ) is the center and the variance is σ. x and y represent the coordinates of other points in the figure.

[0043] The target detection branch detects the parking space entry line in the image (32x downsampled image) based on the image output by the backbone network, referring to Figure 3 As shown, according to the detected parking space entry line, each pixel in the image is output through 6 channels respectively (each pixel in the output image represents a 32x32 grid in the input image), including the entry line midpoint probability map of the midpoint of the parking space entry line, the offset dx of the midpoint of the parking space entry line relative to the upper left corner coordinate point of the pixel grid where it is located in the x-axis direction, the offset dy of the midpoint of the parking space entry line relative to the upper left corner coordinate point of the pixel grid where it is located in the y-axis direction, the length of the parking space entry line l, the cosine value cosθ of the angle between the parking space entry line and the x-axis direction, and the sine value sinθ of the angle between the parking space entry line and the x-axis direction.

[0044] The parking space corner points are extracted according to the corner point heat map output by the image segmentation branch, the parking space separation lines are extracted according to the separation line probability map output by the image segmentation branch, and the parking space entry line is calculated according to the output of the target detection branch.

[0045] The specific method for extracting the corner points of the parking space in this embodiment is: searching the rectangular frame range in the corner point heat map where the coordinate value of the center point is the largest and greater than 0.5 and the coordinate values ​​of the surrounding points gradually decrease, and judging whether the coordinate value of the center point in the rectangular frame is greater than a first threshold (set to 0.8 in this embodiment). If so, the center point is used as the corner point of the parking space.

[0046] The specific method for extracting the parking space separation line is as follows: after binarizing the separation line probability map, find a set of pixel points whose value is 1 and are adjacent to each other in the binarized map, perform least squares straight line fitting on the points in the set, and obtain the parking space separation line L s and the dividing line L s The two endpoints L s-p1 and L s-p2 .

[0047] The calculation formula for binarization is: Where P l (x,y) represents the output value at the (x,y) position in the separation line probability map.

[0048] The method for calculating the parking space entry line according to the output of the target detection branch is as follows: extract the pixel corresponding to the midpoint of the entry line according to the midpoint probability map of the entry line output by the target detection branch, and calculate the coordinates of the two endpoints of the entry line according to the pixels corresponding to the midpoint of the entry line:

[0049] P e-p1 =(x+dx-l / 2×cosθ,y+dy-l / 2×sinθ)

[0050] P e-p2 =(x+dx+l / 2×cosθ,y+dy+l / 2×sinθ)

[0051] Among them, P e-p1 , P e-p2 They represent the two endpoints of the entry line, x and y represent the x-axis coordinate and y-axis coordinate of the upper left corner of the pixel grid where the midpoint of the entry line is located, and dx, dy, l, cosθ and sinθ are determined according to the output results of the other five channels of the target detection branch.

[0052] In this embodiment, the method for extracting the pixel corresponding to the midpoint of the entry line based on the midpoint probability map of the entry line is: determine whether the probability of each pixel containing the midpoint of the entry line is greater than a second threshold (set to 0.8 in this embodiment); if so, determine that the pixel is the pixel corresponding to the midpoint of the entry line.

[0053] S3: Match the parking space corner points with the parking space separation lines, group the successfully matched parking space corner points and parking space separation lines, and set the confidence levels of both to the first confidence level A (e.g., C(P c ,L s ,A)), the confidence levels of the parking space corner points that have no matching parking space separation lines and the parking space separation lines that have no matching parking space corner points are both set to the second confidence level B.

[0054] The method for judging whether the parking space corner point matches the parking space dividing line is as follows: judging whether ||P c-L s-p1 ||2<T1or||P c -L s-p2 ||2<T1,P c Indicates the coordinates of the corner points of the parking space, L s-p1 and L s-p2 They respectively represent the coordinates of the two endpoints of the parking space separation line, ||.||2 represents the L2 norm, and T1 represents the first distance threshold. If so, it is determined that the parking space corner point and the parking space separation line are matched successfully; otherwise, it is determined that the parking space corner point and the parking space separation line are matched unsuccessfully.

[0055] S4: Match the parking space entrance line with the parking space corner point or the parking space separation line. If the confidence level of the parking space corner point or the parking space separation line is the first confidence level A, determine whether the parking space entrance line matches the parking space corner point or the parking space separation line based on the second distance threshold T2; if the confidence level of the parking space corner point or the parking space separation line is the second confidence level B, determine whether the parking space entrance line matches the parking space corner point or the parking space separation line based on a third distance threshold T3 that is smaller than the second distance threshold T2.

[0056] The method for judging whether the parking space entrance line matches the parking space corner point or the parking space separation line according to the second distance threshold or the third distance threshold is similar to the method for judging whether the parking space corner point matches the parking space separation line successfully, that is: judging whether the Euclidean distance between any endpoint of the parking space entrance line and any endpoint of the parking space corner point or the parking space separation line is less than the second distance threshold or the third distance threshold. If so, it is judged that the match is successful; otherwise, it is judged that the match is unsuccessful.

[0057] S5: Determine whether there is a parking space entry line that satisfies: both end points of the parking space entry line have matching parking space corner points or parking space separation lines. If so, output the parking space determined by the parking space entry line and the two matching parking space corner points and two parking space separation lines.

[0058] In the embodiment of the present invention, a high and low threshold strategy is adopted. For a combination of corner points and separation lines that successfully matches, the matching threshold with the entry line is relaxed, and vice versa, a stricter matching threshold is set, which can improve the detection rate of parking space detection. In this embodiment, the parking space is determined by combining three parking space elements, namely, corner points, separation lines, and entry lines, rather than determining the parking space by inferring the corner points from the endpoints of the separation lines and the entry lines. This can ensure the stability of the corner points and greatly reduce the false detection rate.

[0059] Embodiment 2:

[0060] The present invention also provides a parking space detection terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the above method embodiment of the first embodiment of the present invention when executing the computer program.

[0061] Further, as an executable solution, the parking space detection terminal device may be a computing device such as a vehicle-mounted terminal, a cloud server, etc. The parking space detection terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art may understand that the composition structure of the above-mentioned parking space detection terminal device is only an example of a parking space detection terminal device, and does not constitute a limitation on the parking space detection terminal device. It may include more or less components than the above-mentioned components, or a combination of certain components, or different components. For example, the parking space detection terminal device may also include input and output devices, network access devices, buses, etc., which are not limited in the embodiments of the present invention.

[0062] Further, as an executable solution, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the parking space detection terminal device, and uses various interfaces and lines to connect various parts of the entire parking space detection terminal device.

[0063] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the parking space detection terminal device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0064] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method in the embodiment of the present invention are implemented.

[0065] If the module / unit integrated in the parking space detection terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory) and software distribution medium, etc.

[0066] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.

Claims

1. A parking space detection method, characterized in that: The following steps are involved: S1: Collects a panoramic view of the vehicle's surroundings from a bird's-eye view; S2: Detecting all parking space corner points, parking space separation lines and parking space entry lines according to the panoramic image around the vehicle; S3: Matching parking space corner points with parking space separation lines, grouping successfully matched parking space corner points and parking space separation lines, and setting the confidence levels of both to the first confidence level, and setting the confidence levels of parking space corner points that have no parking space separation lines to match them and parking space separation lines that have no parking space corner points to match them to the second confidence level; S4: matching the parking space entry line with the parking space corner point or the parking space separation line, and if the confidence level of the parking space corner point or the parking space separation line is the first confidence level, determining whether the parking space entry line matches the parking space corner point or the parking space separation line according to the second distance threshold; if the confidence level of the parking space corner point or the parking space separation line is the second confidence level, determining whether the parking space entry line matches the parking space corner point or the parking space separation line according to a third distance threshold that is smaller than the second distance threshold; S5: Determine whether there is a parking space entry line that satisfies: both end points of the parking space entry line have matching parking space corner points or parking space separation lines. If so, output the parking space determined by the parking space entry line and the two matching parking space corner points and two parking space separation lines.

2. The parking space detection method according to claim 1, characterized in that: The method for collecting the panoramic image of the vehicle surroundings from a bird's-eye view perspective is as follows: collecting images through multiple cameras installed in multiple directions of the vehicle, and after splicing the collected multiple images, the spliced ​​images are transformed into a panoramic stitching image from a bird's-eye view perspective.

3. The parking space detection method according to claim 2, characterized in that: The multiple directions include at least four directions of the vehicle: front, rear, left, and right.

4. The parking space detection method according to claim 1, characterized in that: The parking space corner points, parking space separation lines and parking space entry lines are obtained through a neural network model; the network structure of the model includes a backbone network, an image segmentation branch and a target detection branch; The panoramic image around the vehicle is input from the backbone network as an input image and downsampled; After the image segmentation branch upsamples the image output by the backbone network to the same resolution as the input image, it obtains the corner point heat map of each pixel in the image containing the parking space corner point and the separation line probability map of each pixel in the image belonging to the parking space separation line through two channels; The target detection branch detects the parking space entry line in the image according to the image output by the backbone network, and outputs the entry line midpoint probability map of each pixel in the image containing the midpoint of the parking space entry line, the offset dx of the midpoint of the parking space entry line relative to the upper left corner coordinate point of the pixel grid where it is located in the x-axis direction, the offset dy of the midpoint of the parking space entry line relative to the upper left corner coordinate point of the pixel grid where it is located in the y-axis direction, the length l of the parking space entry line, the cosine value cosθ of the angle between the parking space entry line and the x-axis direction, and the sine value sinθ of the angle between the parking space entry line and the x-axis direction respectively through 6 channels; The parking space corner points are extracted according to the corner point heat map output by the image segmentation branch, the parking space separation lines are extracted according to the separation line probability map output by the image segmentation branch, and the parking space entry line is calculated according to the output of the target detection branch.

5. The parking space detection method according to claim 4, characterized in that: The method for extracting the corner points of parking spaces according to the corner point heat map is as follows: searching the rectangular frame range in the corner point heat map where the coordinate value of the center point is the largest and greater than 0.5 and the coordinate values ​​of the surrounding points gradually decrease, and judging whether the coordinate value of the center point within the rectangular frame is greater than a first threshold value. If so, the center point is used as the corner point of the parking space.

6. The parking space detection method according to claim 4, characterized in that: The method for extracting parking space separation lines based on the separation line probability map is as follows: after binarizing the separation line probability map, a set of pixel points whose value is 1 and are adjacent to each other is searched in the binarized map, and the points in the set are fitted with the least squares straight line to obtain the parking space separation lines.

7. The parking space detection method according to claim 4, characterized in that: The method for calculating the parking space entry line according to the output of the target detection branch is as follows: extract the pixel corresponding to the midpoint of the entry line according to the midpoint probability map of the entry line output by the target detection branch, and calculate the coordinates of the two endpoints of the entry line according to the pixels corresponding to the midpoint of the entry line: P e-p1 =(x+dx-l / 2×cosθ,y+dy-l / 2×sinθ) P e-p2 =(x+dx+l / 2×cosθ,y+dy+l / 2×sinθ) Among them, P e-p1 , P e-p2 They represent the two endpoints of the entry line, and x and y represent the x-axis coordinate and y-axis coordinate of the upper left corner of the pixel grid where the midpoint of the entry line is located.

8. The parking space detection method according to claim 1, characterized in that: The method for judging whether the parking space corner point matches the parking space dividing line is as follows: judging whether ||P c -L s-p1 ||2<T1or||P c -L s-p2 ||2<T1,P c Indicates the coordinates of the corner points of the parking space, L s-p1 and L s-p2 They respectively represent the coordinates of the two endpoints of the parking space separation line, ||.||2 represents the 2-norm, and T1 represents the first distance threshold. If yes, it is determined that the parking space corner point and the parking space separation line are matched successfully; otherwise, it is determined that the parking space corner point and the parking space separation line are matched unsuccessfully.

9. A parking space detection terminal device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images