Vehicle systems and methods for enhancing parking space line display

Abstract

Vehicle systems and methods for enhancing parking space line display are provided. One vehicle system includes one or more cameras configured to capture one or more images of a vehicle exterior, a control module, and a display module. The control module is configured to: detect, based on the one or more captured images, a portion of a parking space line that defines at least a portion of a parking space; generate, based on the detected portion of the parking space line, an estimated polynomial representing the parking space line; and generate, based on the estimated polynomial, a virtual parking space line corresponding to at least one missing portion of the parking space line. The display module is configured to render a view of the virtual parking space line superimposed on the parking space line. Other example vehicle systems and methods are also disclosed.

Description

Technical Field

[0001] The information provided in this section is for the purpose of presenting the general background of this disclosure. To the extent described in this section, the work of the currently named inventors, and aspects of the description that may otherwise not conform to the prior art at the time of filing, are neither explicitly nor implicitly considered to be prior art of this disclosure.

[0002] This disclosure relates to vehicle systems and methods for enhancing the display of parking space lines. Background Technology

[0003] Vehicles include onboard cameras to provide information about the surrounding environment, which can be used to render views for display and / or for various operations of the vehicle. For example, a vehicle may rely on images captured by the cameras to render a bird's-eye view of the vehicle, a front view of the vehicle, a rear view of the vehicle, a side view of the vehicle, etc. Sometimes, the rendered view can provide lines representing the vehicle's expected trajectory, the front end of the vehicle, the rear end of the vehicle, etc., relative to, for example, external objects (e.g., parking space lines, curbs, walls, etc.). Summary of the Invention

[0004] A vehicle system for enhancing the display of at least one parking space line adjacent to a vehicle includes one or more cameras configured to capture one or more images of the exterior of the vehicle, a control module communicating with the one or more cameras, and a display module communicating with the control module. The control module is configured to: detect a portion of a parking space line defining at least a portion of a parking space based on the one or more captured images; generate an estimated polynomial representing the parking space line based on the detected portion; and generate a virtual parking space line corresponding to at least one missing portion of the parking space line based on the estimated polynomial. The display module is configured to render a view of the virtual parking space line superimposed on the parking space line.

[0005] Among other features, when a vehicle is in a parking space, the parking space line is a line that extends substantially parallel to the vehicle.

[0006] Among other features, the control module is configured to generate virtual parking space lines that extend to the horizontal line associated with the parking space.

[0007] Among the other features, the estimation polynomial is a first estimation polynomial, and the control module is configured to: detect features of vehicles adjacent to the parking space; generate a second estimation polynomial representing the features of adjacent vehicles; and generate virtual parking space lines based on the first and second estimation polynomials.

[0008] Among other features, the control module is configured to: apply weights to each of the first and second estimation polynomials; and generate virtual parking space lines based on the weighted first and second estimation polynomials.

[0009] Among other features, the estimation polynomial is a first estimation polynomial, and the control module is configured to: receive map data of an area adjacent to or surrounding the parking space; generate a second estimation polynomial representing the parking space line based on the map data; and generate a virtual parking space line based on the first and second estimation polynomials.

[0010] Among other features, the control module is configured to: apply weights to each of the first and second estimation polynomials; and generate virtual parking space lines based on the weighted first and second estimation polynomials.

[0011] Among other features, the control module is configured to: receive data representing one or more captured images in a first coordinate system; apply a transformation to the data to convert it to a second coordinate system; and detect portions of parking space lines and generate an estimated polynomial based on the transformed data.

[0012] Among other features, the control module is configured to: detect multiple lines that include portions of parking space lines; and apply filters to the multiple lines based on a defined set of colors to identify portions of parking space lines.

[0013] Among other features, the control module is configured to: detect multiple colors associated with portions of the parking space lines; select one of the detected colors based on the frequency of occurrence of each detected color; and generate virtual parking space lines using the selected color.

[0014] Among other features, the control module is configured to: detect the color associated with a portion of the parking space line; and generate a virtual parking space line using the detected color.

[0015] A vehicle system for enhancing the display of at least one parking space line adjacent to a vehicle includes one or more cameras configured to capture one or more images of the exterior of the vehicle, a control module communicating with the one or more cameras, and a display module communicating with the control module. The control module is configured to: detect a portion of a parking space line defining at least a portion of a parking space based on the one or more captured images, and generate a first estimated polynomial representing the parking space line based on the detected portion; detect features of vehicles adjacent to the parking space based on the one or more captured images, and generate a second estimated polynomial representing the features of the adjacent vehicles; receive map data of an area adjacent to or surrounding the parking space, and generate a third estimated polynomial representing the parking space line based on the map data; and generate a virtual parking space line corresponding to at least one missing portion of the parking space line based on the first, second, and third estimated polynomials. The display module is configured to render a view of the virtual parking space line overlaid on the parking space line.

[0016] Among other features, the control module is configured to: apply weights to each of the first, second, and third estimation polynomials; and generate virtual parking space lines based on the weighted first, second, and third estimation polynomials.

[0017] Among other features, the control module is configured to: detect the color associated with a portion of the parking space line; and generate a virtual parking space line using the detected color.

[0018] A method for enhancing the display of at least one parking space line adjacent to a vehicle includes: capturing one or more images of the exterior of the vehicle; detecting a portion of a parking space line defining at least a portion of the parking space based on the one or more captured images; generating an estimated polynomial representing the parking space line based on the detected portion; generating a virtual parking space line corresponding to at least one missing portion of the parking space line based on the estimated polynomial; and rendering a view of the virtual parking space line superimposed on the parking space line on a display module.

[0019] Among other features, when a vehicle is in a parking space, the parking space line is a line that extends substantially parallel to the vehicle, and generating a virtual parking space line includes generating a virtual parking space line that extends to the horizontal line associated with the parking space.

[0020] Among other features, the estimation polynomial is a first estimation polynomial, and the method further includes: detecting features of vehicles adjacent to the parking space; generating a second estimation polynomial representing features of adjacent vehicles, and generating virtual parking space lines includes generating virtual parking space lines based on the first and second estimation polynomials.

[0021] Among other features, the method further includes: receiving map data of an area adjacent to or surrounding a parking space; generating a third estimated polynomial representing a parking space line based on the map data; and generating a virtual parking space line includes generating a virtual parking space line based on a first estimated polynomial, a second estimated polynomial, and a third estimated polynomial.

[0022] Among other features, the method also includes applying weights to each of the first, second, and third estimation polynomials, and generating virtual parking lines includes generating virtual parking lines based on the weighted first, second, and third estimation polynomials.

[0023] Among other features, the method also includes: detecting multiple colors associated with portions of the parking space line; selecting one of the detected colors based on the frequency of occurrence of each detected color; and generating the virtual parking space line by generating the virtual parking space line using the selected color.

[0024] The following solutions are provided:

[0025] 1. A vehicle system for enhancing the display of at least one parking space line adjacent to a vehicle, the vehicle system comprising:

[0026] One or more cameras are configured to capture one or more images of the exterior of the vehicle;

[0027] A control module that communicates with one or more cameras is configured to:

[0028] Detect a portion of a parking space line that defines at least a portion of a parking space based on one or more captured images;

[0029] Generate an estimated polynomial representing the parking space lines based on the detected portion of the parking space lines; and

[0030] Based on the estimated polynomial, a virtual parking space line corresponding to at least one missing portion of the parking space line is generated; and

[0031] The display module communicates with the control module and is configured to render a view of virtual parking space lines overlaid on the parking space lines.

[0032] 2. The vehicle system according to Scheme 1, wherein when the vehicle is in a parking space, the parking space line is a line that is substantially parallel to the extension of the vehicle.

[0033] 3. The vehicle system according to Scheme 2, wherein the control module is configured to generate virtual parking space lines that extend to a horizontal line associated with the parking space.

[0034] 4. The vehicle system according to Scheme 2, wherein:

[0035] The estimating polynomial is the first estimating polynomial; and

[0036] The control module is configured to: detect the features of vehicles adjacent to the parking space; generate a second estimated polynomial representing the features of the adjacent vehicles; and generate virtual parking space lines based on the first and second estimated polynomials.

[0037] 5. The vehicle system according to Scheme 4, wherein the control module is configured as follows:

[0038] Apply the weights to each of the first and second estimating polynomials; and

[0039] Virtual parking space lines are generated based on a weighted first estimate polynomial and a weighted second estimate polynomial.

[0040] 6. The vehicle system according to Scheme 2, wherein:

[0041] The estimating polynomial is the first estimating polynomial; and

[0042] The control module is configured to: receive map data of an area adjacent to or surrounding a parking space; generate a second estimated polynomial representing the parking space line based on the map data; and generate a virtual parking space line based on the first and second estimated polynomials.

[0043] 7. The vehicle system according to Scheme 6, wherein the control module is configured as follows:

[0044] Apply the weights to each of the first and second estimating polynomials; and

[0045] Virtual parking space lines are generated based on a weighted first estimate polynomial and a weighted second estimate polynomial.

[0046] 8. The vehicle system according to Scheme 1, wherein the control module is configured as follows:

[0047] Receive data representing one or more captured images in a first coordinate system;

[0048] Apply a transformation to the data to convert the data to a second coordinate system; and

[0049] The system detects portions of the parking space lines and generates an estimated polynomial based on the transformed data.

[0050] 9. The vehicle system according to Scheme 1, wherein the control module is configured as follows:

[0051] The detection includes multiple lines, including parking space lines; and

[0052] Filters are applied to multiple lines based on a defined set of colors to identify portions of the parking space lines.

[0053] 10. The vehicle system according to Scheme 1, wherein the control module is configured as follows:

[0054] Detects multiple colors associated with portions of the parking space lines;

[0055] One of the detected colors is selected based on the frequency of occurrence of each detected color; and

[0056] Generate virtual parking space lines using the selected colors.

[0057] 11. The vehicle system according to Scheme 1, wherein the control module is configured as follows:

[0058] Detect the color associated with the portion of the parking space line; and

[0059] Virtual parking space lines are generated using the detected colors.

[0060] 12. A vehicle system for enhancing the display of at least one parking space line adjacent to a vehicle, the vehicle system comprising:

[0061] One or more cameras are configured to capture one or more images of the exterior of the vehicle;

[0062] A control module that communicates with one or more cameras is configured to:

[0063] Based on one or more captured images, a portion of the parking space line that defines at least a portion of the parking space is detected, and a first estimated polynomial representing the parking space line is generated based on the detected portion of the parking space line.

[0064] Detect features of vehicles adjacent to parking spaces based on one or more captured images, and generate a second estimated polynomial representing the features of adjacent vehicles;

[0065] Receive map data of the area adjacent to or surrounding the parking space, and generate a third estimated polynomial representing the parking space lines based on the map data; and

[0066] Based on the first, second, and third estimation polynomials, a virtual parking line corresponding to at least one missing portion of the parking line is generated; and

[0067] The display module communicates with the control module and is configured to render a view of virtual parking space lines overlaid on the parking space lines.

[0068] 13. The vehicle system according to claim 12, wherein the control module is configured as follows:

[0069] The weights are applied to each of the first, second, and third estimating polynomials; and

[0070] Virtual parking space lines are generated based on a weighted first estimation polynomial, a weighted second estimation polynomial, and a weighted third estimation polynomial.

[0071] 14. The vehicle system according to claim 13, wherein the control module is configured as follows:

[0072] Detect the color associated with the portion of the parking space line; and

[0073] Virtual parking space lines are generated using the detected colors.

[0074] 15. A method for enhancing the display of at least one parking space line adjacent to a vehicle, the method comprising:

[0075] Capture one or more images of the exterior of the vehicle;

[0076] Detect a portion of the parking space line that defines at least a portion of the parking space based on one or more captured images;

[0077] Based on the detection of parking space lines, a partially generated estimated polynomial representing the parking space lines is generated.

[0078] Virtual parking lines are generated based on the estimated polynomial, corresponding to at least one missing portion of the parking space line; and

[0079] Render a view on the display module where virtual parking space lines are overlaid on the parking space lines.

[0080] 16. The method according to scheme 15, wherein:

[0081] When a vehicle is in a parking space, the parking space line is a line that extends basically parallel to the vehicle.

[0082] The generation of virtual parking space lines includes generating virtual parking space lines that extend to the horizontal lines associated with the parking spaces.

[0083] 17. The method according to scheme 15, wherein:

[0084] The estimating polynomial is the first estimating polynomial; and

[0085] The method further includes: detecting features of vehicles adjacent to the parking space; and generating a second estimation polynomial representing the features of adjacent vehicles; and

[0086] Generating virtual parking space lines involves generating virtual parking space lines based on a first estimation polynomial and a second estimation polynomial.

[0087] 18. The method according to scheme 17, wherein:

[0088] The method further includes: receiving map data of an area adjacent to or surrounding the parking space; and generating a third estimation polynomial representing the parking space line based on the map data; and

[0089] Generating virtual parking space lines involves generating virtual parking space lines based on a first estimated polynomial, a second estimated polynomial, and a third estimated polynomial.

[0090] 19. The method according to scheme 18, wherein:

[0091] The method further includes applying weights to each of the first, second, and third estimating polynomials; and

[0092] Generating virtual parking space lines involves generating virtual parking space lines based on a weighted first estimation polynomial, a weighted second estimation polynomial, and a weighted third estimation polynomial.

[0093] 20. The method according to scheme 15, wherein:

[0094] The method further includes: detecting multiple colors associated with a portion of the parking space line; and selecting one of the detected colors based on the frequency of occurrence of each detected color; and

[0095] Generating virtual parking space lines includes generating virtual parking space lines using the selected colors.

[0096] Further applications of this disclosure will become apparent from the detailed description, claims, and accompanying drawings. The detailed description and specific examples are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0097] This disclosure will become more fully understood in light of the detailed description and accompanying drawings, in which:

[0098] Figure 1 This is a block diagram of a vehicle system for enhancing parking space line display according to the present disclosure;

[0099] Figure 2 The present disclosure shows example images of parking spaces and parking space lines;

[0100] Figure 3 It is based on the purpose of this disclosure. Figure 2 Example of a virtual parking space line;

[0101] Figure 4 This is another example image showing a parking space and detected parking space lines and vehicle features according to this disclosure;

[0102] Figure 5-6 It is based on this disclosure and has the following characteristics: Figure 1 Example image of map information received by the vehicle system;

[0103] Figure 7-8 This is an example display of a view with virtual parking space lines according to this disclosure; and

[0104] Figure 9-10 This is a flowchart of an example process for enhancing the display of parking space lines according to this disclosure.

[0105] In the accompanying drawings, reference numerals may be used repeatedly to identify similar and / or identical elements. Detailed Implementation

[0106] The vehicle includes onboard cameras to provide information about its surroundings, which can be used to render views for display. For example, the vehicle may rely on images captured by the cameras to render a bird's-eye view, a front view, a rear view, a side view, and so on. In such an example, the captured images may include various objects outside the vehicle, such as parking lines, other vehicles, curbs, etc. These objects are then compared with the displayed view. Figure 1 The following are provided. Some objects, such as parking space lines, are rarely in perfect condition. For example, parking space lines often wear (e.g., deteriorate over time) and / or become blurred (e.g., obscured by dust, snow, etc.). In other instances, parking space lines may be difficult to see in the view due to glare from, for example, sunlight or other light sources from the driver's surface and / or the display. Furthermore, insufficient light, shadows, etc., can also make parking space lines difficult to see. Due to the lack of visibility in the displayed view, vehicle drivers may find it difficult to park their vehicles in the desired spaces.

[0107] The vehicle system and method according to this disclosure utilize image processing techniques to identify remaining parking space lines and overlay virtual lines to assist the driver. For example, and as further explained herein, the vehicle system and method detect parking space lines in images captured by a vehicle camera, and if interference exists in the parking space lines, generate virtual parking space lines based on an estimated polynomial representing the detected parking space lines, at least for the interference. In various embodiments, the vehicle system and method may also rely on one or more other optional estimated polynomials representing other detected lines from the same vehicle camera and / or other sources to generate virtual parking space lines. The vehicle system and method then overlay the virtual parking space lines onto a view for display. By doing so, the driver is provided with complete and clear parking space lines, even under challenging conditions, and consequently, a more accurate and comprehensive representation of the parking space. This significantly improves the driver's ability to correctly align and maneuver the vehicle and enhances overall parking safety.

[0108] Now for reference Figure 1 A block diagram of an example vehicle system 100 for enhancing the display of parking space lines adjacent to vehicle 102 is presented. Figure 1 As shown, vehicle system 100 typically includes a control module 104, cameras 106, 108, 110, 112, a display module 114, a memory circuit 116, and one or more sensors 118. Figure 1 In the example, the memory circuit 116 can be located outside the control module 104 as shown, or inside the control module 104 if necessary. Although Figure 1The vehicle system 100 is shown as including specific dedicated modules, but it should be understood that one or more other modules may be employed if needed. For example, any combination of modules (e.g., control module 104 and display module 114) and / or their functions may be integrated into a single module or multiple different modules.

[0109] Figure 1 The vehicle system 100 can be used in any suitable vehicle, such as autonomous vehicles, semi-autonomous vehicles, etc. Furthermore, the vehicle system 100 is applicable to electric vehicles (e.g., pure electric vehicles, plug-in hybrid electric vehicles, etc.) and internal combustion engine (ICE) vehicles. Figure 1 In the example, vehicle system 100 is used in vehicle 102, which can be an autonomous vehicle, a semi-autonomous vehicle, etc.

[0110] Cameras 106, 108, 110, 112, display module 114, memory circuit 116, and sensor 118 communicate with control module 104. In this example, control module 104 can receive and / or transmit signals, data, etc., from and / or to each of cameras 106, 108, 110, 112, display module 114, memory circuit 116, and sensor 118. In this example, control module 104 can receive and / or transmit signals, data, etc., via a network (e.g., Controller Area Network (CAN)).

[0111] exist Figure 1 In the example, cameras 106, 108, 110, and 112 are positioned to capture one or more images of the exterior of vehicle 102. In various embodiments, camera 108 may be a front-end camera module, camera 112 may be a rear-end camera module, and cameras 106 and 110 may be side-mounted camera modules. Although Figure 1 The vehicle system 100 is shown as including four cameras located at specific positions relative to the vehicle 102, but it should be understood that in other embodiments, the vehicle system 100 may include more or fewer cameras, cameras located at different positions, etc.

[0112] In various embodiments, vehicle system 100 generates virtual parking space lines to assist the driver during parking maneuvers. For example, vehicle system 100 typically utilizes line detection and reconstruction algorithms to detect parking space lines based on captured images from one or more cameras 106, 108, 110, 112. Vehicle system 100 then generates virtual parking space lines corresponding to missing portions of the actual parking space lines. Based on the analysis performed by the line detection and reconstruction algorithms, vehicle system 100 uses display module 114 (e.g., a real-time rendering module, etc.) to overlay the virtual lines onto the missing portions of the parking space lines in the image actually displayed to the driver. In such examples, rendering is seamlessly integrated with the live video feed, providing enhanced display of the parking space lines. Furthermore, vehicle system 100 may include a user-friendly interface that allows the driver to configure display settings for the virtual lines, such as line thickness, color, transparency, etc. Additionally, in various embodiments, the interface may provide options to enable or disable enhanced display features, depending on the driver's preferences.

[0113] Figure 2 Depicting what can be Figure 1 An example of the line detection and reconstruction algorithm of the vehicle system 100 utilizing the captured image 200. For example, in Figure 2 In the image 200, parking spaces 202 and 204 are defined by parking space lines 206 and 208 and curbs 210 and 224. Specifically, parking space 202 is defined by parking space lines 206 and 208 and curb 210, and parking space 204 is defined by parking space lines 208 and curbs 210 and 224. In this example, parking space lines 206 and 208 and curbs 210 and 224 are difficult to see due to factors such as degradation, blurring, or glare.

[0114] Vehicle system 100 can detect parking space lines 206, 208 and curbs 210, 224, as further explained herein, and then generate virtual parking space lines corresponding to the missing portions or all of parking space lines 206, 208 and curbs 210, 224. In such an example, vehicle system 100 can rely on existing markers (e.g., curbs, line endpoints, etc.) to extrapolate equations to represent parking space lines 206, 208 and curbs 210, 224. For example only, markers 212, 214, 216, 218, 220, and 222 are used in... Figure 2 The middle is shown as X. In this example, marks 212 and 216 are the general endpoints of parking space lines 206 and 208, marks 214 and 218 are points along curb 210 corresponding to parking space lines 206 and 208, and marks 220 and 222 are points along curb 224.

[0115] Because parking lines 206, 208 and curbs 210, 224 are typically straight, vehicle system 100 can use interpolation between existing markings and then extrapolate equations extending beyond the existing markings for parking lines 206, 208 and curbs 210, 224, as further explained below. For example, Figure 3 Depicting the corresponding Figure 2 An example of the virtual lines 306, 308, 310, and 312 for parking space lines 206 and 208 and curb lines 210 and 224.

[0116] In various embodiments, Figure 1 The control module 104 can implement line detection and reconstruction algorithms for line detection. For example, after receiving a captured image, the control module 104 can detect parking space lines (e.g., ...) based on one or more captured images. Figure 2 At least a portion of the parking space line 206. In such an example, when the vehicle is in the parking space, the parking space line may be a line extending substantially parallel to the vehicle 102.

[0117] For example, Figure 4 An example of an image 400 is depicted, including a parking space 402 defined by parking space lines 404, 406 and a parking curb 408. In this example, parking space lines 404, 406 are vertical lines that typically extend parallel to the vehicle 102 when located within the parking space 402. Furthermore, in this example, the parking curb 408 serves as a horizontal line associated with the parking space 402, which typically separates the ground (e.g., parking space 402) from the area on the ground. As shown, the parking curb 408 (or sometimes a curb) is detected from the captured image by the control module 104 and is represented by a line 410 (with a dotted-dotted configuration) extending in a different direction (e.g., laterally, orthogonally, etc.) from the parking space lines 404, 406.

[0118] In various embodiments, control module 104 can detect parking space lines 404, 406, or at least a portion thereof. In such examples, control module 104 can implement an edge detection algorithm to detect lines 404, 406. For example, control module 104 can initially perform image distortion correction on images received from one or more cameras 106, 108, 110, 112, and then initiate edge detection techniques to detect edges of objects in the captured images. In some examples, control module 104 can implement the Canny edge detector method or another suitable method to detect edges of objects in the image. For example, in the Canny edge detector method, the resolution of edge detection decreases as the sigma value (e.g., image smoothness) increases. In some examples, parking line detection works best at high sigma values ​​to filter out high-resolution lines, such as asphalt cracks.

[0119] Then, the control module 104 can filter the edges to focus on the vertical edges. For example and refer to Figure 4 The control module 104 can filter out detected edges associated with parking stones 408 and retain detected edges associated with parking space lines 404, 406. For example, parking stones 408 (or curbs) extend perpendicularly rather than vertically. In this example, as parking space lines 404, 406 approach vehicle 102, they branch off from parking stones 408 (e.g., horizontal lines) and are typically approximately 8.5–9 feet apart. This standard specification helps the edge detection algorithm separate the signal from noise.

[0120] exist Figure 4 In the example, control module 104 detects the edges 412 and 414 of parking space lines 404 and 406, respectively. Edges 412 and 414 are represented by lines with a dashed configuration. Although Figure 4 The left edge of parking space lines 404 and 406 being detected is shown, but it should be understood that the right edge can also be detected, or alternatively.

[0121] Next, the control module 104 can apply a transformation to detect line targets in the captured image. For example, the control module 104 can implement the Hough Transform or another suitable transformation for this purpose. In the example using the Hough Transform, each point on the detected edge (e.g., x) from the filter... i y i The line (c, m) is transformed from image space to parameter space. When multiple lines are drawn in parameter space, an intersection will be generated. This intersection point can now be transformed back to image space and become a line, which in this case is the detected line.

[0122] In various embodiments, control module 104 may implement line detection if one or more conditions apply. For example, control module 104 may initiate line detection only when vehicle 102 is in a parking lot. In such an example, control module 104 may receive GPS data, etc., from one of sensors 118 and a known parking lot location stored in memory 116 (or at a location outside vehicle 102), and determine whether vehicle 102 is in a parking lot based on this information. Furthermore, control module 104 may initiate line detection only when the vehicle speed is less than or equal to a defined threshold (e.g., a calibration value), such as 4 km / h, 5 km / h, 6 km / h, etc. Control module 104 may make this determination based on vehicle speed data provided by one of sensors 118.

[0123] Then, in various embodiments, the control module 104 can generate an estimated polynomial representing each parking space line 404, 406 or its edges 412, 414. In such an example, the estimated polynomial is generated based on the detected portion of the corresponding parking space line 404, 406. For example, the lines can be represented by a polynomial equation with a certain curvature (if appropriate). In such an example, the radius of curvature of each line can be calculated when the equation of the curve is known. By way of example only, equation (1) below is a polynomial equation, and equation (2) is a radius of curvature equation that can be adopted.

[0124] Equation (1) f(y)=Ay 2 +By+c

[0125] Equation (2)

[0126] In other examples, control module 104 may utilize existing classical or deep learning-based techniques to determine the estimated polynomials corresponding to parking space lines (e.g., parking space lines 404, 406) in the captured image. For example, control module 104 may implement PolyLaneNet or other suitable machine learning (ML) techniques that perform lane detection on the received image and represent the polynomial of each lane sign in the image through, for example, deep polynomial regression output.

[0127] In various embodiments, the control module 104 can then generate virtual parking space lines corresponding to at least one parking space line based on the estimated polynomial. For example, and continuing to refer to... Figure 4 The parking space line 404 is difficult to see due to factors such as degradation, blurring, or glare. In this example, the control module 104 can then generate a virtual parking space line corresponding to the entire parking space line 404 or one or more missing portions of the parking space line 404 based on the estimated polynomial of the line 404.

[0128] Virtual parking space lines can extend along their corresponding parking space lines as needed. For example, the virtual parking space line generated for parking space line 404 can extend to the horizontal line associated with parking space 402. In other words, in Figure 4 In the example, the generated virtual parking space line can extend to parking stone 408 (or line 410). In other examples, the generated virtual parking space line can extend to near the horizontal line (e.g., parking stone 408) or to another suitable location.

[0129] In various embodiments, control module 104 may generate and rely on additional polynomials for generating the virtual lines. For example, control module 104 may utilize the orientation of other nearby parked vehicles to enhance the parking space line estimation. For example, control module 104 may detect features of vehicles adjacent to parking space 402, such as... Figure 4 One or two parked vehicles 416, 418. As an example, control module 104 can detect the lower edge of vehicle 416 between tires 420, 422, and then insert line 424 between tires 420, 422. Additionally, if needed, control module 104 can detect the lower edge of vehicle 418 between tires 426, 428, and then insert another line 430 between tires 426, 428. Control module 104 can then generate an estimated polynomial representing the detected features (e.g., the lower edges of vehicles 416, 418). In examples where multiple vehicles (e.g., vehicles 416, 418) exist in the same parking segment, control module 104 can average the estimates to increase robustness. Existing edge detection techniques (e.g., the Canny edge detector method, etc.), tire detection models, trained ML models, etc., can be used for such actions.

[0130] Furthermore, control module 104 can utilize externally received data to enhance parking space line estimation. For example, control module 104 may rely on map information (e.g., satellite data, etc.), which may or may not be based on crowdsourced data. For instance, control module 104 may receive map data of the area adjacent to or surrounding parking space 402. In such an example, control module 104 can use map matching to determine the attitude of a vehicle relative to nearby lane lines in the map. Control module 104 can then generate an estimation polynomial representing parking space lines 404, 406, and / or other parking space lines in the parking lot based on the map data.

[0131] For example, regarding satellite-based data, control module 104 can determine the orientation of a vehicle in a parking lot by applying a vehicle detection model that provides a bounding box around the vehicle. Control module 104 can then estimate a polynomial based on points along the longer end of the bounding box (e.g., a vertical line), since most vehicles have long sides and short rear ends. The position and orientation of the parking space line feature / polynomial can then be determined by control module 104 relative to a reference point. For example, in a GPS coordinate system, control module 104 can rely on latitude, longitude, and heading relative to north as reference points.

[0132] Figure 5-6 Example images 500 and 600 depict map information received by control module 104. For example, in Figure 5 In this module, control module 104 can receive map data (e.g., coordinates) of parking space lines 504, 506, and / or other lines, and then generate estimated polynomials representing lines 504, 506, and / or other lines. Figure 6In the image 600, the bounding box around the vehicle in the parking lot is shown. The control module 104 can rely on the bounding box to generate an estimated polynomial representing the parking space lines 604, 606, 608, 610, 612, 614 and / or other lines.

[0133] Then, once additional estimation polynomials have been generated for the detected features of adjacent vehicles and / or based on map data, control module 104 can rely on one or more of these estimation polynomials to generate virtual parking space lines. For example, control module 104 can generate virtual parking space lines corresponding to the entire parking space line 404 or one or more missing portions of parking space line 404 based on the estimation polynomials of line 404, line 424 and / or line 430 (or the lower edges of vehicles 416, 418) and / or estimation polynomials based on map data. In such examples, estimation polynomials can be combined to generate new estimation polynomials for a specific parking space line (e.g., parking space line 404), or the estimation polynomial of line 404 can be adjusted as needed based on other estimation polynomials for the detected vehicle features and / or based on map data. For example, equations (3), (4), and (5) below represent example functions for the initial estimation polynomials, detected vehicle features, and map data, respectively. Then, equation (6) below represents an example function that combines the estimated polynomials of equations (3), (4), and (5). Although equation (6) provides an example of combining estimated polynomials, it should be understood that other suitable functions can be implemented to combine estimated polynomials if needed.

[0134] Equation (3)f i (y)=Ay 2 +By+c

[0135] Equation (4)f c (y)=Ey 2 +Fy+g

[0136] Equation (5)f m (y)=Py 2 +Qy+r

[0137] Equation (6)f n (y)=f i (y)+f c (y)+f m (y)

[0138] In various embodiments, control module 104 can apply weights to any or all of the estimated polynomials. In such an example, each weight may be a calibration (e.g., defined) value determined based on ML techniques, etc. Furthermore, the weights may be changed or otherwise adjusted based on the location of vehicle 102 (e.g., in Florida vs. Michigan), time of year (e.g., winter vs. summer), time of day, weather, etc. For example, equation (7) below shows the three weight values ​​of the estimated polynomials of equations (3), (4), and (5) above, and equation (8) below represents an example function combining the weighted estimated polynomials of equations (3), (4), and (5).

[0139] Equation (7) 1=λ i +λ c +λ m

[0140] Equation (8)f n (y)=λ i *f i (y)+λ c *f c (y)+λ m *f m (y)

[0141] In various embodiments, control module 104 may transform data from one coordinate system to another. For example, features (e.g., coordinate points, lines, etc.) detected by control module 104 from captured images may need to be transformed by one or more transformations (e.g., one or more transformation matrices) into different views of vehicle 102 (e.g., how they look in the vehicle's camera view) to be shown to the driver. This may include, for example, features detected as associated with parking space lines 404, 406, features detected as associated with vehicles 416, 418, etc. In other examples, control module 104 may apply one or more transformations (e.g., one or more transformation matrices, such as GPS to camera view transformation) to map data of parking space line features (e.g., in a GPS / ENU framework) to transform the data into the coordinate system of vehicle 102. As explained above, control module 104 may then detect parking space lines and generate an estimation polynomial based on the transformed data. Equation (9) below provides an example of GPS to camera view transformation, and equation (10) below provides an example of transforming lane line features in the GPS / ENU framework to the camera view of the vehicle.

[0142] Equation (9) l,view T l,GPS

[0143] Equation (10) l,view Tl,map = l,view T l,vehicle * l,vehicle T l,map

[0144] Continue to refer to Figure 1 The display module 114 renders a view with virtual parking space lines generated by the control module 104. In such an example, the virtual parking space lines can be overlaid on the entire parking space line (e.g., parking space line 404, etc.) or a portion of the parking space line (e.g., only the missing portion). For example, the control module 104 may generate signals or data representing the virtual parking space lines and then transmit such signals or data to the display module 114, which can then render a view with the virtual parking space lines for the driver in real time.

[0145] For example, Figure 7-8 Depicting by Figure 1 The display module 114 provides examples of different views, such as 700 and 800. Figure 7 As shown, display 700 provides a bird's-eye view of vehicle 102 with two virtual parking space lines 704 and 706. Figure 8 In the middle, display 800 provides a front (or rear) view of vehicle 102 with two virtual parking space lines 804 and 706.

[0146] In various embodiments, the driver of vehicle 102 can be notified in real time whether vehicle 102 needs to shift gears to ensure that vehicle 102 is centered at the parking point. For example, Figure 7-8 One or two of the virtual parking space lines 704, 706, 804, and 806 can change color in real time to provide direction to the driver in maneuvering vehicle 102 into the center of the parking spot. In such an example, the virtual parking space line can turn green to notify the driver to turn toward the virtual line, turn red to notify the driver to turn away from the virtual line, and so on.

[0147] Furthermore, in some embodiments, the display module 114 may add center guide lines to help the driver properly center the vehicle 102 between the created virtual parking space lines. For example, in Figure 8 In the middle, display 800 provides a center guide line 808.

[0148] In various embodiments, Figure 1 The control module 104 detects parking space lines and / or generates virtual parking space lines based on color. For example, parking space lines are typically yellow or white. In other scenarios, parking space lines may be another color, such as green or blue, to indicate a specifically designated area. In such an example, the control module 104 may detect parking space lines (e.g., based on image processing techniques) Figure 4The color associated with at least a portion of the parking space line 404 is then used to generate a virtual parking space line with the same detection color.

[0149] In other embodiments, the control module 104 may filter out some detected parking space lines based on color. For example, when detecting parking space lines (e.g., ... Figure 4 After identifying multiple parking space lines (404, 406), the control module 104 can apply filters based on a defined set of colors. For example, the control module 104 can remove lines that are not yellow, white, green, blue, or other colors commonly used for parking space lines. This reduces the set of parking space lines that the control module 104 can identify.

[0150] Additionally, in some examples, virtual parking space lines are generated using selected colors. For instance, control module 104 can detect multiple hues and / or colors associated with detected parking space lines based on image processing techniques. Then, control module 104 can select one of the detected hues / colors based on the frequency of occurrence of each detected hue / color. For example, control module 104 can implement an "upward voting" scheme to count the occurrence of each detected hue / color and select the hue / color with the highest frequency. Then, control module 104 can use the selected color detection to generate virtual parking space lines.

[0151] Figure 9-10 It shows that it can be made by Figure 1 The vehicle system 100 employs example methods 900 and 1000 for enhancing the display of parking space lines adjacent to the vehicle 102. Although example methods 900 and 1000 are relative to components including control module 104, display module 114, etc. Figure 1 The vehicle system 100 is described in this way, and any one of the methods 900 and 1000 may be another suitable system and / or module that may be adopted.

[0152] like Figure 9 As shown, method 900 begins at 902, where control module 104 receives camera images captured by, for example, cameras 106, 108, 110, and 112. Method 900 then proceeds to 904, where control module 104 detects parking space lines or portions thereof. For example, and as explained above, control module 104 may implement an edge detection algorithm or another suitable method to detect parking space lines. Method 900 then proceeds to 906.

[0153] At 906, control module 104 determines whether a portion of the parking space line is missing or otherwise has low viewing quality. This can be achieved using image processing techniques. If yes, method 900 proceeds to 908. Otherwise, if no at 906, method 900 proceeds to 912.

[0154] At 908, control module 104 generates an estimated polynomial representing the parking space lines. For example, and as explained above, control module 104 can generate the estimated polynomial using conventional line equations (e.g., equation (1) above), classical or deep learning-based techniques, etc. Method 900 then proceeds to 910, where control module 104 generates virtual parking space lines based on the estimated polynomial. In such an example, the virtual parking space lines may correspond to missing portions or the entire parking space line, as explained above. Method 900 then proceeds to 912, where display module 114 renders a view with the virtual parking space lines for the driver in real time. Method 900 can then end, as... Figure 9 As shown.

[0155] Figure 10 Method 1000 is similar Figure 9 Method 900, but includes additional and / or alternative steps. For example, such as Figure 10 As shown, method 1000 begins with the explanation above. Figure 9 The process proceeds to steps 902, and then to steps 1004 and 1006. At 1004, control module 104 receives sensor data. Then, at 1006, control module 104 determines whether vehicle 102 is located in the parking lot. In various embodiments, control module 104 makes this determination based on GPS data, etc., received at 1004. If the result is negative at 1006, method 1000 can terminate, as... Figure 10 As shown. If yes in 1006, then method 1000 proceeds to 1008. In 1008, control module 104 determines whether the vehicle speed is less than or equal to a defined threshold, such as 4 km / h, 5 km / h, 6 km / h, etc. If no, method 1000 can end, as shown. Figure 10 As shown. If it is yes in 1006, then method 1000 proceeds to 1010.

[0156] In step 1010, control module 104 performs edge detection analysis on the captured image from step 902. Then, method 1000 proceeds to step 1012, where control module 104 applies one or more transforms to transform the data and identify lines in the image. In such an example, control module 104 may implement the Hough transform or another suitable transform, as explained above. Method 1000 then proceeds to step 1014.

[0157] At 1014, control module 104 can apply filters to identify desired parking space lines. For example, as explained above, control module 104 can filter to remove non-vertical lines and retain vertical lines. Furthermore, in some examples, control module 104 can filter lines based on color, as explained above. Method 1000 then proceeds to 1016.

[0158] At 1016, control module 104 generates one or more estimation polynomials. For example, and as explained above, control module 104 may generate estimation polynomials representing each identified or otherwise detected parking space line, estimation polynomials representing detected features of adjacent vehicles, estimation polynomials based on received map information, etc. Method 1000 then proceeds to... Figure 9 Methods 910 and 912, wherein control module 104 generates one or more virtual parking space lines based on an estimated polynomial, and display module 114 renders a view with the virtual parking space lines for the driver in real time. Method 1000 can then end, as... Figure 10 As shown.

[0159] The foregoing description is merely illustrative in nature and is by no means intended to limit this disclosure, its application, or its use. The broad teachings of this disclosure can be implemented in various forms. Therefore, while this disclosure includes specific examples, its true scope should not be so limited, as other modifications will become apparent upon examination of the drawings, specification, and appended claims. It should be understood that one or more steps in the method may be performed in a different order (or concurrently) without altering the principles of this disclosure. Furthermore, while each embodiment is described above as having certain features, any one or more of those features described with respect to any embodiment of this disclosure may be implemented in and / or combined with features of any other embodiment, even if such combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and substitutions of one or more embodiments for each other remain within the scope of this disclosure.

[0160] Spatial and functional relationships between components (e.g., between modules, circuit elements, semiconductor layers, etc.) are described using various terms, including “connected,” “joined,” “coupled,” “adjacent,” “next to,” “on top,” “above,” “below,” and “set.” Unless explicitly described as “direct,” when describing the relationship between the first and second components in the above disclosure, the relationship can be a direct relationship where no other intervening components exist between the first and second components, but it can also be an indirect relationship (spatially or functionally) between the first and second components. As used herein, the phrase at least one of A, B, and C should be interpreted as meaning the use of a non-exclusive logical OR (A or B or C) and should not be interpreted as meaning “at least one of A, at least one of B, and at least one of C.”

[0161] In a diagram, the direction of the arrow usually indicates the flow of information (such as data or instructions) that is of interest to the diagram. For example, when components A and B exchange various kinds of information, but the information transmitted from component A to component B is relevant to the diagram, the arrow can point from component A to component B. This unidirectional arrow does not mean that no other information is transmitted from component B to component A. Furthermore, for information sent from component A to component B, component B can send a request for or confirmation of receipt of that information to component A.

[0162] In this application, including the following limitations, the term "module" or "controller" may be replaced by the term "circuit". The term "module" may refer to or include a portion of the following: application-specific integrated circuit (ASIC); digital, analog, or mixed-signal analog / digital discrete circuit; digital, analog, or mixed-signal analog / digital integrated circuit; combinational logic circuit; field-programmable gate array (FPGA); processor circuitry (shared, dedicated, or grouped) that executes code; memory circuitry (shared, dedicated, or grouped) that stores code executed by the processor circuitry; other suitable hardware components that provide the described functionality; or some or all of the foregoing, such as in a system-on-a-chip.

[0163] A module may include one or more interface circuits. In some examples, the interface circuits may include wired or wireless interfaces that connect to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functionality of any given module disclosed herein can be distributed among multiple modules connected via the interface circuits. For example, multiple modules can allow for load balancing. In another example, a server (also referred to as a remote or cloud) module may perform some functions on behalf of a client module.

[0164] The term "code" as used above can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, data structures, and / or objects. The term "shared processor circuit" covers a single processor circuit that executes some or all of the code from multiple modules. The term "grouped processor circuit" covers a processor circuit that, in combination with additional processor circuits, executes some or all of the code from one or more modules. References to multiple processor circuits cover multiple processor circuits on a discrete die, multiple processor circuits on a single die, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above. The term "shared memory circuit" covers a single memory circuit that stores some or all of the code from multiple modules. The term "grouped memory circuit" covers a memory circuit that, in combination with additional memory, stores some or all of the code from one or more modules.

[0165] The term memory circuitry is a subset of the term computer-readable medium. As used herein, the term computer-readable medium does not cover transient electrical or electromagnetic signals propagating through a medium (e.g., on a carrier wave); therefore, the term computer-readable medium can be considered tangible and non-transitory. Non-limiting examples of non-transitory tangible computer-readable media are non-volatile memory circuitry (e.g., flash memory circuitry, erasable programmable read-only memory circuitry, or mask read-only memory circuitry), volatile memory circuitry (e.g., static random access memory circuitry or dynamic random access memory circuitry), magnetic storage media (e.g., analog or digital magnetic tape or hard disk drives), and optical storage media (e.g., CDs, DVDs, or Blu-ray discs).

[0166] The apparatus and methods described in this application can be implemented, in part or in whole, by a special-purpose computer created by configuring a general-purpose computer to perform one or more specific functions embodied in a computer program. The function blocks, flowchart components, and other elements described above serve as software specifications that can be translated into computer programs through the routine work of a skilled technician or programmer.

[0167] A computer program includes processor-executable instructions stored on at least one non-transitory tangible computer-readable medium. A computer program may also include or depend on stored data. A computer program may encompass a basic input / output system (BIOS) for interacting with the hardware of a special-purpose computer, device drivers for interacting with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0168] Computer programs may include: (i) descriptive text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation); (ii) assembly code; (iii) object code generated from source code by a compiler; (iv) source code executed by an interpreter; (v) source code compiled and executed by a real-time compiler, etc. As an example only, source code may come from languages ​​including C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, etc. Fortran, Perl, Pascal, Curl, OCaml, HTML5 (Hypertext Markup Language, Fifth Revision), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Visual Lua, MATLAB, SIMULINK and It is written using the syntax of the language.

Claims

1. A vehicle system for enhancing the display of at least one parking space line adjacent to a vehicle, the vehicle system comprising: One or more cameras are configured to capture one or more images of the exterior of the vehicle; A control module that communicates with one or more cameras is configured to: Detect a portion of a parking space line that defines at least a portion of a parking space based on one or more captured images; Generate an estimated polynomial representing the parking space line based on the detected portion of the parking space line; as well as A virtual parking space line corresponding to at least one missing part of the parking space line is generated based on the estimated polynomial. as well as The display module communicates with the control module and is configured to render a view of virtual parking space lines overlaid on the parking space lines.

2. The vehicle system of claim 1, wherein when the vehicle is in a parking space, the parking space line is a line substantially parallel to the extension of the vehicle.

3. The vehicle system of claim 2, wherein the control module is configured to generate virtual parking space lines that extend to a horizontal line associated with the parking space.

4. The vehicle system according to claim 2, wherein: The estimating polynomial is the first estimating polynomial; and The control module is configured to detect the characteristics of vehicles adjacent to the parking space; Generate a second estimated polynomial representing the characteristics of neighboring vehicles; and generate virtual parking space lines based on the first and second estimated polynomials.

5. The vehicle system of claim 4, wherein the control module is configured as follows: Apply the weights to each of the first and second estimating polynomials; and Virtual parking space lines are generated based on a weighted first estimate polynomial and a weighted second estimate polynomial.

6. The vehicle system according to claim 2, wherein: The estimating polynomial is the first estimating polynomial; and The control module is configured to receive map data of the area adjacent to or surrounding the parking space; A second estimated polynomial representing parking space lines is generated based on map data; And generate virtual parking space lines based on the first and second estimation polynomials.

7. The vehicle system of claim 6, wherein the control module is configured to: Apply the weights to each of the first and second estimating polynomials; and Virtual parking space lines are generated based on a weighted first estimate polynomial and a weighted second estimate polynomial.

8. The vehicle system of claim 1, wherein the control module is configured to: The detection includes multiple lines, including parking space lines; and Filters are applied to multiple lines based on a defined set of colors to identify portions of the parking space lines.

9. The vehicle system of claim 1, wherein the control module is configured to: Detects multiple colors associated with portions of the parking space lines; One of the detected colors is selected based on the frequency of occurrence of each detected color; and Generate virtual parking space lines using the selected colors.

10. A vehicle system for enhancing the display of at least one parking space line adjacent to a vehicle, the vehicle system comprising: One or more cameras are configured to capture one or more images of the exterior of the vehicle; A control module that communicates with one or more cameras is configured to: Based on one or more captured images, a portion of the parking space line that defines at least a portion of the parking space is detected, and a first estimated polynomial representing the parking space line is generated based on the detected portion of the parking space line. Detect features of vehicles adjacent to parking spaces based on one or more captured images, and generate a second estimated polynomial representing the features of adjacent vehicles; Receive map data of the area adjacent to or surrounding the parking space, and generate a third estimated polynomial representing the parking space line based on the map data; as well as A virtual parking space line corresponding to at least one missing part of the parking space line is generated based on the first estimation polynomial, the second estimation polynomial, and the third estimation polynomial. as well as The display module communicates with the control module and is configured to render a view of virtual parking space lines overlaid on the parking space lines.

Images