Optical object detection in front of a vehicle via the photoelectric effect of metal stripe lines

Abstract

Optical object detection in front of a vehicle via a photoelectric effect of a metallic stripe line is provided. A system and method for reliably determining a road lane includes an optical sensing device for sensing a metallic stripe line from a photoelectric effect. The position of a stripe line defining a traffic lane boundary is determined, and the position of the stripe line is displayed on a graphical user interface. The position can be used to provide: lane control to ensure that the vehicle maintains a proper orientation in a traffic lane, lane warning assistance, collision avoidance, parking control, and guidance for autonomous driving.

Description

[0001] Related Applications

[0002] This patent application claims the benefit of prior-filed co-pending U.S. Provisional Patent Application No. 62 / 937,563, filed on November 19, 2019, the entire contents of which are hereby incorporated by reference.

[0003] Background of the relevant application

[0004] The device relates to a vehicle front optical object detection system which senses photons emitted from metal striping based on the photoelectric effect.

[0005] It is known that a vehicle front video camera is used for object processing to detect an object in front of the vehicle. Methods include sensing texture, classifying objects, and determining structure from motion (SFM). In determining the presence of an object and the boundaries or lanes of a road, various sensed information is combined to obtain redundancy. Summary of the invention

[0006] One embodiment is a system for determining a lane of a road, the system comprising: an optical sensing device fixed to a vehicle for sensing a forward image relative to the vehicle, including sensing a metallic lane stripe line from a photoelectric effect of photoelectrons generated by light from a vehicle headlight or from light from another source; and an electronic control unit including an electronic processor and a memory. The electronic processor is configured to: determine the position of the metallic lane stripe line defining a traffic lane based on the position of the photoelectrons emitted from the metallic lane stripe line defining the traffic lane boundary, and provide at least one from a vehicle operation group consisting of: lane control to ensure that the vehicle maintains a proper orientation in the traffic lane, lane warning assistance, collision avoidance, parking control, and guidance for autonomous driving.

[0007] Another embodiment is a method for determining a lane of a road. The method includes: optically sensing a forward image relative to a vehicle, including sensing a metallic lane stripe line from a photoelectric effect of photoelectrons generated from light provided by a vehicle headlight or from other sources, and determining the position of a metallic lane stripe line defining a traffic lane based on the position of photoelectrons emitted from the metallic lane stripe line defining a traffic lane boundary. Finally, the method includes providing at least one from a vehicle operation group consisting of: lane control to ensure that the vehicle maintains a proper orientation in a traffic lane, lane warning assistance, collision avoidance, parking control, and guidance for autonomous driving.

[0008] Other aspects, features, and embodiments will become apparent by consideration of the detailed description and accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A block diagram of one embodiment of a vehicle front optical object detection system for assisting vehicle operation is illustrated.

[0010] Figure 2 A top view of a vehicle traveling along a road segment is illustrated.

[0011] Figure 3 A flow chart of an operational embodiment of a vehicle object detection system is illustrated.

[0012] Figure 4 The diagram shows Figure 3 The electronic control unit of an embodiment outputs a graphical user interface.

[0013] Figure 5 A flow chart of a lane warning and lane control embodiment is illustrated.

[0014] Figure 6 A flow chart of a parallel parking embodiment is illustrated. DETAILED DESCRIPTION

[0015] Before any embodiments are explained in detail, it will be understood that the present disclosure is not intended to be limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the following drawings. Embodiments are capable of other configurations and of being practiced or carried out in various ways.

[0016] Various embodiments may be implemented using a plurality of hardware and software-based devices and a plurality of differently structured components. In addition, embodiments may include hardware, software, and electronic components or modules, which, for purposes of discussion, may be illustrated and described as if most components were implemented only in hardware. However, one of ordinary skill in the art and based on a reading of this detailed description will recognize that, in at least one embodiment, the electronic-based aspects of the present invention may be implemented in software (e.g., stored on a non-transitory computer-readable medium) that may be executed by one or more electronic controllers. For example, the "unit," "control unit," and "controller" described in the specification may include one or more electronic controllers, one or more memories including non-transitory computer-readable media, one or more input / output interfaces, one or more application-specific integrated circuits (ASICs) and other circuits, and various connections or connectors (e.g., wires, printed traces, and buses) connecting the various components.

[0017] Figure 1A block diagram view of a vehicle front optical object detection system 40 for determining lanes and objects in front of the vehicle is shown. In one embodiment, the vehicle front optical object detection system 40 includes an electronic control unit 44. The electronic control unit 44 includes an electronic processor 50 and a memory 54. The memory 54 includes one or more memory modules, such as random access memory ("RAM") and electronically erasable programmable read-only memory ("EEPROM"). An input / output interface 58 transmits and receives information via a communication bus 60. The electronic processor 50 processes information by executing one or more applications or modules. The applications or modules may be stored in the memory 54 as instructions, etc. The electronic processor 50 also stores information generated by the applications in the memory.

[0018] Figure 1 The communication bus 60 shown in is a FlexRay automotive communication bus, a controller area network (CAN) bus, or other type of communication link between multiple control units, sensors, and other devices. In some embodiments, the communication bus 60 connects the electronic control unit 44 to a display 64 or other device (such as a graphical user interface) for displaying a road image taken from the front of the vehicle. In one embodiment, the display 64 and the user interface are combined in a touch screen. In another embodiment, the user interface includes a keyboard or other input device to receive input.

[0019] also, Figure 1 An optical sensing device is shown, which includes (one or more) optical sensors 70 such as (one or more) video cameras for optically sensing a forward image relative to the vehicle and providing video information to the electronic control unit 44 via the communication bus 60. In one embodiment, two spaced video cameras are provided as a stereo camera system. In one embodiment, the stereo video cameras are horizontally spaced from 25 cm to 50 cm apart. In some embodiments, other sensors and information 74, such as global positioning signal (GPS) information, vehicle speed and direction information are also provided to the electronic control unit 44 via the communication bus 60.

[0020] Figure 1 Also shown is an audio and / or tactile warning system 78 that includes speakers of the vehicle's interior stereo system or other speakers to provide audio warnings of inappropriate lane changes or other conditions. In some embodiments, the warning system 78 includes a tactile warning generated by vibrating the steering wheel.

[0021] Figure 1 Also shown are a brake system 80 for controlling brakes, a steering system 84, and an engine train system 88. In various embodiments, an electronic control unit 44 connected to the communication bus 60 provides control signals to the systems 80, 84, 88 via the communication bus.

[0022] Figure 2 A vehicle 100 is shown that includes (one or more) optical sensors 70 mounted or fixed to its front end. The vehicle 100 is on a section of road 110 that includes metal lane striping lines 112, 114 that define a right traffic lane and metal lane striping lines 116, 118 that define a left traffic lane adjacent thereto. In some embodiments, the metal lane striping lines include one or more of powders or particles of bronze, copper, aluminum, zinc, magnetic, or ferromagnetic materials. The metal lane striping lines 112, 114, 116, 118 emit photoelectrons in response to light directed toward the metal lane striping lines. The photoelectrons are sensed by the optical sensor 70 mounted on the vehicle 100. The light applied to the metal lane striping lines 112, 114, 116, 118 comes from the headlights of the vehicle 100 or from other light sources such as street lights, sunlight, moonlight, or other vehicle headlights. The metal lane striping lines may be formed of metallic paint, metal plates, metal strips, or other materials having a sufficient amount of metal particles to exhibit a photoelectric effect. Figure 2 In the embodiment, metal lane stripe lines 112, 114, 116, 118 are metal lane stripe lines for defining the boundaries of the driving lane. In another embodiment, the metal lane stripe line is centered in the lane, and the vehicle considers that the path defines or conforms to the driving path.

[0023] Figure 2 A vehicle curb is shown that includes metallic curb striping lines 122, 124. Thus, the metallic curb striping lines 122, 124 are provided on the top side and / or the inner side of the curb. In one embodiment, the curb is painted with a metallic paint. In one embodiment, the metallic curb striping lines 122, 124 have different metallic materials such that the wavelength or frequency of the photoelectric effect of the metallic curb striping lines 122, 124 is distinguishable from the wavelength or frequency of the photoelectric effect of the metallic lane striping lines 112, 114, 116, 118. Thus, the vehicle front optical object detection system 40 can distinguish the metallic lane striping lines from the metallic curb striping lines.

[0024] In one embodiment, metallic parking stripe lines 126, 128 are provided so that the vehicle front optical object detection system 40 can distinguish parking spaces for parallel parking. The metallic parking stripe lines 126, 128 are oriented laterally and have corner lines to define individual parking spaces.

[0025] operate

[0026] Figure 3 Shows Figure 1 The operation of the electronic control unit 44 is shown in a flow chart 200. Although Figure 1A single electronic control unit 44 is shown, but multiple control units and / or electronic processors may execute Figure 3 Although Figure 3 Multiple operations or steps are shown, but not all operations are necessarily required. In addition, multiple steps can occur substantially simultaneously or in parallel with multiple electronic processors.

[0027] At stereo step 204, left and right forward-directed video cameras 70 acquire left and right images. The images are ordered and combined to calculate the presence of objects and the relative distance from the vehicle to the objects.

[0028] exist Figure 3 At the structure from motion (SFM) step 208 shown in FIG. 1 , successive frames from a video camera are compared and a calculation of the presence and / or movement of an object is determined taking into account the movement of the vehicle or the speed of the vehicle.

[0029] exist Figure 3 At a texture step 212 shown in FIG. 1 , the texture of the sensed object is determined by the electronic control unit 44. At a classifier step 216, the object detected by the electronic control unit 44 is classified. Textures may also be identified, such as determining the presence of another vehicle in the field of view.

[0030] exist Figure 3 At the metal stripe line detection step 220 shown in , a lane or curb having a metal stripe line is detected by photoelectrons emitted therefrom by the photoelectric effect, which are then received by (one or more) optical sensors 70 or (one or more) video cameras. In some embodiments, the photoelectric effect causes the emission of photoelectrons at a specific wavelength. Metal stripe lines can be detected as lanes and curbs, such as in Figure 4 As shown in the graphical user interface 300 shown in .

[0031] exist Figure 3 At the probabilistic data fusion step 230 shown in , various data from the stereo step 204 or the SFM step 208 are combined, including texture, classified objects, and photoelectric detection of metal stripe lines. Other attributes such as vehicle speed, direction, and GPS coordinates are also utilized in some embodiments. The combined information results in video classification of objects and metal stripe lines for at least one of the vehicle operation group, which consists of the following: lane control to ensure that the vehicle maintains proper orientation in the traffic lane, lane warning assistance, collision avoidance, parking control, and guidance for autonomous driving.

[0032] Lane warning assist and control

[0033] In one embodiment, according to Figure 2The metal lane stripe lines 112, 114, 116, 118 provided on the road shown in FIG. define traffic lanes. The lanes are clearly defined and periodically updated by the electronic processor 50. The orientation of the vehicle 100 relative to the position of the metal lane stripe lines 112, 114, 116, 118 is also substantially continuously updated.

[0034] As in Figure 5 As shown in the flowchart 400 of FIG. 4 , the lane warning and lane control routine is executed by the electronic processor 50 as follows. In a first decision step 404, the electronic processor determines whether the vehicle turn signal is operating. If turn signaling is occurring, the electronic processor 50 does not provide any alerts or further sensing. The electronic processor 50 periodically returns to the decision step 404 to determine when the turn signal is in an off condition.

[0035] When the rotation signal is off and inactive, the electronic processor 50 proceeds to decision step 408. At decision step 408, it is determined whether the vehicle 100 is moving relative to Figure 2 The metal lane stripe lines 112, 114 are positioned by using the photoelectric effect. Figure 2 When the vehicle 100 shown in is operating or begins to move outside the appropriate lane, the electronic processor proceeds to step 412 and executes a program to automatically control the steering system 84 without input from the vehicle operator to maintain the vehicle in the traffic lane. The operator holding the vehicle steering wheel can feel the wheel turning to indicate that automatic adjustment in the direction of the vehicle 100 returning to the traffic lane is occurring. In addition, one or more of a tactile warning, an audible warning, and a visual warning are provided to the vehicle operator to alert the operator to corrective action. From providing warnings and steering control, the program returns to decision step 404 and repeats the steps. If the vehicle 100 remains outside the lane boundary of the intended or current traffic lane or begins to move outside of this, the electronic processor 50 again adjusts the steering system 84 to move the vehicle toward the appropriate traffic lane.

[0036] like Figure 5 As shown in , when the vehicle is not outside the lane boundary at decision step 408, the electronic processor 50 proceeds to decision step 416. At decision step 416, the electronic processor determines whether the vehicle in the current traffic lane is inappropriately close to or near the metal lane stripe lines 112, 114. If so, in one embodiment, the electronic processor vibrates the steering wheel of the vehicle to provide a tactile warning to the vehicle operator. In another embodiment, a visual warning is provided to the vehicle operator. In another embodiment, two or more of a tactile warning, an audible warning, and a visual warning on the display 64 or another instrument panel are provided to the vehicle operator.

[0037] Collision Avoidance

[0038] In the embodiment of collision avoidance, in addition to determining that the vehicle 100 is traveling in the appropriate traffic lane, the electronic processor 50 is configured to: determine the metallic paint position of the vehicle body of another vehicle in front of it, which is based on the position of the photoelectrons generated by the metallic paint of the other vehicle sensed from the optical sensor 70. Therefore, the electronic processor 50 is configured to determine the presence of the vehicle in the same traffic lane or adjacent traffic lane. In one embodiment, the electronic processor 50 is configured to provide a collision avoidance warning to the interior of the vehicle when another vehicle is detected in the same traffic lane and the other vehicle is too close to or is traveling toward the vehicle for the vehicle speed. The warning can be tactile, auditory and / or visual. In another embodiment, the electronic processor 50 is configured to automatically control the brakes of the vehicle braking system 80, and / or the steering system 84, in an attempt to avoid or minimize the possibility of a collision.

[0039] Parallel Parking

[0040] Figure 6 The flow chart 500 shown in FIG. 1 illustrates an automatic parallel parking mode. In this embodiment, a rearwardly and / or laterally directed optical sensor 70 (such as a video camera) is provided to detect the metal curb stripe lines 122, 124 provided for the curb, and additionally detect the metal parking stripe lines 126, 128 - which when provided are used to mark individual parking spaces, such as Figure 2 as shown in .

[0041] In the automatic parallel parking mode, the vehicle operator provides an input indicating the desire to parallel park to the display 64 or other input device at step 508 to initiate parallel parking. At decision step 516, the electronic processor 50 determines whether the vehicle is stopped and the optical sensor 70 detects the curb, which detects the metal curb stripe line and the metal parking stripe line and the presence of the parked vehicle through the photoelectric effect to determine whether a parking space of the appropriate size is available. If not, then at step 528, the electronic processor 50 provides an audible message and / or a visual message that the parking space is not available.

[0042] When the conditions are met, the electronic processor 50 proceeds to the parking step 532 and controls the steering system 84, the braking system 80 and the engine transmission system 88 to parallel park the vehicle adjacent to the curb defined by the metal stripe lines 122, 124. The parking step 532 includes a subroutine for operating the system using feedback from the optical sensor 70 to properly park the vehicle 100. When the metal parking stripe lines 126, 128 are also detected, the electronic processor is configured to park the vehicle 100 only in an empty parking space between the metal parking stripe lines. Therefore, the electronic processor 50 is configured to provide automatic parallel parking of the vehicle 100 adjacent to the curb or metal curb stripe lines when the vehicle is provided in the automatic parallel parking mode and the parking space is available. Thereafter, at step 540, the electronic processor 50 provides an indication to the display 64 and / or the warning system 78 that the vehicle 100 has completed parallel parking.

[0043] Autonomous driving or self-driving

[0044] In the autonomous or automatic driving mode, the electronic processor is configured to provide autonomous driving of the vehicle 100 as long as the presence of metallic lane stripe lines 112, 114 defining a traffic lane is detected. Figure 2 Two boundaries of metallic lane striping lines 112, 114 are shown - with the vehicle 100 in the traffic lane formed therebetween, but in another embodiment, a single metallic lane striping line may be provided in the center of the traffic lane, with the electronic processor 50 being configured to automatically follow the single metallic lane striping line through control of the braking system 80, the steering system 84, and the engine transmission system 88.

[0045] Various features, advantages, and embodiments are set forth in the following claims.

Claims

1. A vehicle system for determining a road lane, the system comprising: an optical sensing device secured to the vehicle, the optical sensing device comprising at least one video camera, the optical sensing device being adapted to sense a forward image relative to the vehicle, including sensing metallic lane stripe lines from a photoelectric effect of photoelectrons generated by light from vehicle headlights or light from other sources; as well as an electronic control unit comprising an electronic processor and a memory, Wherein the electronic processor is configured as: determining the position of the metal lane stripe lines defining the traffic lane based on the position of the photoelectrons emitted from the metal lane stripe lines defining the traffic lane boundary, determining a position of metallic paint from a vehicle body of another vehicle based on a position of photoelectrons generated from the metallic paint of the other vehicle, and At least one from the vehicle operations group consisting of lane control to ensure the vehicle maintains proper orientation in a traffic lane, lane warning assistance, collision avoidance, parking control, and guidance for autonomous driving is provided.

2. The vehicle system according to claim 1, wherein: The electronic processor is configured to provide an interior vehicle warning when the vehicle begins to move out of a current lane of traffic.

3. The vehicle system according to claim 1, wherein: The electronic processor is configured to provide automatic steering control when the vehicle begins to move out of the current lane of traffic in the absence of a turn signal.

4. The vehicle system according to claim 1, wherein: The electronic processor is configured to: Based on the orientation of photoelectrons generated from the metal curb stripe line that defines the curb boundary, the position of the metal stripe line that defines the curb is determined, wherein the photoelectric effect of the metal curb stripe line is distinguishable from the photoelectric effect of the metal lane stripe line, and the positions of the metal curb stripe line of the curb and the metal lane stripe line of the traffic lane are displayed on a graphical user interface.

5. The vehicle system according to claim 4, wherein: The electronic processor is configured to: determining a position of a metal parking stripe line based on a position of photoelectrons emitted from the metal parking stripe line defining the parking space, and providing automatic parallel parking of the vehicle adjacent to a curb when the vehicle is provided in an automatic parallel parking mode, Due to the different metal materials, the metal curb stripe lines and the metal lane stripe lines are distinguishable, so that the wavelength or frequency of the photoelectric effect of the metal curb stripe lines is different from the wavelength or frequency of the photoelectric effect of the metal lane stripe lines.

6. The system according to claim 1, wherein: The electronic processor is configured to provide autonomous driving of the vehicle whenever the presence of a metallic stripe line defining a lane of traffic is detected.

7. The system according to claim 1, wherein: The electronic processor is configured to provide a collision avoidance warning to an interior of the vehicle when the other vehicle is detected in the current lane of traffic.

8. The system according to claim 1, wherein: An electronic control unit including an electronic processor is connected to the communication bus for communication and control of brakes of the vehicle's braking system to minimize the possibility of a collision.

9. A method for determining a road lane, the method comprising: Optically sensing a forward image relative to the vehicle using at least one video camera, including sensing metallic lane stripe lines from a photoelectric effect of photoelectrons generated from light provided by vehicle headlights or from other sources; determining the position of the metal lane stripe lines defining the traffic lane based on the position of the photoelectrons emitted from the metal lane stripe lines defining the traffic lane boundary, determining a position of metallic paint from a vehicle body of another vehicle based on a position of photoelectrons emitted from the metallic paint of the other vehicle, and At least one from the vehicle operations group consisting of lane control to ensure the vehicle maintains proper orientation in a traffic lane, lane warning assistance, collision avoidance, parking control, and guidance for autonomous driving is provided.

10. The method of claim 9, comprising providing an interior vehicle warning when the vehicle begins to move out of a current lane of traffic.

11. The method of claim 9, comprising the step of providing automatic steering control when the vehicle begins to move out of the current lane of traffic without signaling.

12. The method according to claim 9, comprising the steps of: Based on the orientation of photoelectrons emitted from the metal curb stripe line defining the curb boundary, the position of the metal curb stripe line defining the curb is determined, wherein the photoelectric effect of the metal curb stripe line is distinguishable from the photoelectric effect of the metal lane stripe line, and the positions of the metal curb stripe line of the curb and the metal lane stripe line of the traffic lane are displayed on a graphical user interface.

13. The method according to claim 12, comprising: determining a position of a metal parking stripe line based on a position of photoelectrons emitted from the metal parking stripe line defining the parking space, and providing automatic parking of the vehicle adjacent to a curb when the vehicle is provided in an automatic parallel parking mode, Due to the different metal materials, the metal curb stripe lines and the metal lane stripe lines are distinguishable, so that the wavelength or frequency of the photoelectric effect of the metal curb stripe lines is different from the wavelength or frequency of the photoelectric effect of the metal lane stripe lines.

14. The method of claim 9, comprising providing automated driving of the vehicle whenever the presence of metallic lane striping lines defining a traffic lane is detected.

15. The method of claim 9, comprising providing a collision avoidance warning to an interior of the vehicle when the other vehicle is detected in a current lane of traffic.

16. The method according to claim 9, wherein: An electronic control unit including an electronic processor is connected to the communication bus for communication and control of the brakes of the vehicle braking system.

17. A vehicle system for determining a lane of a road, the system comprising: an optical sensing device affixed to the vehicle for sensing a forward image relative to the vehicle, including sensing metallic lane stripe lines from a photoelectric effect of photoelectrons generated by light from vehicle headlights or light from other sources; an electronic control unit comprising an electronic processor and a memory, Wherein the electronic processor is configured as: determining the position of the metal lane stripe lines defining the traffic lane based on the position of the photoelectrons emitted from the metal lane stripe lines defining the traffic lane boundary, determining a position of metallic paint from a vehicle body of another vehicle based on a position of photoelectrons emitted from the metallic paint of the other vehicle, determining a position of a metal curb stripe line based on a position of photoelectrons emitted from a metal curb stripe line defining a curb, wherein a photoelectric effect of the metal curb stripe line is distinguishable from a photoelectric effect of a metal lane stripe line, displaying on a graphical user interface metal lane stripe lines defining traffic lanes and metal curb stripe lines defining curb boundaries, and At least one from the vehicle operations group consisting of lane control to ensure the vehicle maintains proper orientation in a traffic lane, lane warning assistance, collision avoidance, parking control, and guidance for autonomous driving is provided.

18. The system of claim 17, wherein: The electronic processor is configured to: determining a position of a metal parking stripe line based on a position of photoelectrons emitted from the metal parking stripe line defining the parking space, and When the vehicle is provided in the automatic parallel parking mode, automatic parallel parking of the vehicle adjacent to a curb is provided.

19. The system of claim 18, wherein the metal curb stripe line is distinguishable from the metal lane stripe line due to different metal materials, so that a wavelength or frequency of the photoelectric effect of the metal curb stripe line is different from a wavelength or frequency of the photoelectric effect of the metal lane stripe line.

Images