Vehicle control device and method

Abstract

A vehicle control device and method are provided. The vehicle control device can determine first traffic lane marking information based on object recognition information received from a sensor of the vehicle, determine second traffic lane marking information based on map information received from a navigation system of the vehicle, and generate combined traffic lane marking information for the vehicle by comparing the first traffic lane marking information with the second traffic lane marking information. The combined traffic lane marking information can indicate at least one mismatched traffic lane marking for which the first traffic lane marking information and the second traffic lane marking information do not match. The vehicle control device can further control a driving operation of the vehicle based on the combined traffic lane marking information.

Description

[0001] Cross-references to related applications

[0002] This application claims priority and benefit to Korean Patent Application No. 10-2024-0182448, filed on December 10, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to a vehicle control device and method. Background Technology

[0004] Autonomous vehicle systems can use multiple sensors and high-definition (HD) maps to accurately identify roads and driving environments. Vehicles can compare traffic lines (e.g., road markings) detected by various sensors, such as cameras, lidar, and radar, with lane marking information stored in real-time on HD maps. This process allows for precise vehicle location identification, route determination, and ensures safe driving.

[0005] However, when a mismatch exists between sensor data and HD map information, current autonomous driving systems employ the method of immediately disabling or stopping the autonomous driving mode to ensure driving safety. While this simple response may initially seem reasonable in terms of increasing the stability of the autonomous driving system, it can lead to problems such as traffic flow disruptions and reduced marketability of autonomous vehicles. Summary of the Invention

[0006] This disclosure aims to provide a vehicle control device and method that can maintain stable driving even when lane line information identified by an autonomous vehicle does not match lane line information in an HD map.

[0007] This disclosure also aims to provide a vehicle control device and method that can diagnose the cause of a problem and generate an alternative route if lane line information does not match.

[0008] According to one or more exemplary embodiments of this disclosure, a vehicle control device may include a plurality of processors and a memory storing at least one instruction. The plurality of processors may include a first processor, a second processor, and a third processor. The at least one instruction may be configured, when executed by a first processor communicating with the memory, to cause the vehicle control device to: determine first traffic lane marking information based on object recognition information received from the vehicle's sensors; and determine second traffic lane marking information based on map information received from the vehicle's navigation system. The at least one instruction may be configured, when executed by a second processor communicating with the memory, to further cause the vehicle control device to generate combined traffic lane marking information for the vehicle by comparing the first traffic lane marking information with the second traffic lane marking information. The combined traffic lane marking information may indicate at least one mismatched traffic lane marking where the first traffic lane marking information and the second traffic lane marking information do not match. The at least one instruction may be configured, when executed by a third processor communicating with the memory, to further cause the vehicle control device to control the vehicle's driving operation based on the combined traffic lane marking information.

[0009] The at least one instruction can be configured, when executed by a second processor communicating with the memory, to further cause the vehicle control device to: determine at least one mismatched traffic lane marking by comparing at least one of the coordinates, slope, and angle of the first and second traffic lane marking information.

[0010] The at least one instruction can be configured, when executed by a third processor communicating with the memory, to cause the vehicle control device to control the vehicle's driving operation in such a way that, based on determining that the vehicle's destination would be unreachable by driving in a traffic lane associated with a mismatched traffic lane marking, the vehicle is controlled to avoid driving in a traffic lane associated with a mismatched traffic lane marking.

[0011] The at least one instruction can be configured, when executed by a third processor communicating with the memory, to cause the vehicle control device to control the vehicle's driving operation in such a way as to deactivate the vehicle's automatic driving operation or change the automatic driving level of the vehicle's automatic driving operation based on determining that the vehicle's destination is unreachable by avoiding driving in a traffic lane associated with a mismatched traffic lane marking.

[0012] The at least one instruction can be configured, when executed by a third processor communicating with the memory, to cause the vehicle control device to control the driving operation of the vehicle by comparing combined traffic lane marking information generated by the vehicle control device with second combined traffic lane marking information generated by the second vehicle.

[0013] The at least one instruction can be configured, when executed by a third processor communicating with the memory, to cause the vehicle control device to control the vehicle's driving operation by determining that the combined traffic lane marking information generated by the vehicle is inaccurate based on the fact that the mismatched traffic lane markings are not identified as mismatches in the second combined traffic lane marking information.

[0014] The at least one instruction can be configured, when executed by a third processor communicating with the memory, to cause the vehicle control device to control the vehicle's driving operation by: disabling the vehicle's automatic driving operation or changing the automatic driving level of the vehicle's automatic driving operation based on the determination that the combined traffic lane marking information generated by the vehicle is inaccurate.

[0015] The at least one instruction can be configured, when executed by a third processor communicating with the memory, to cause the vehicle control device to control the vehicle's driving operation by: identifying a mismatch in the second combined traffic lane marking information based on the mismatched traffic lane markings, and sending the combined traffic lane marking information generated by the vehicle and the second combined traffic lane marking information generated by the second vehicle to the server.

[0016] The at least one instruction can be configured, when executed by a third processor communicating with the memory, to cause the vehicle control device to control the vehicle's driving operation in such a way that, based on determining that the vehicle's destination would be unreachable by driving in a traffic lane associated with a mismatched traffic lane marking, the vehicle is controlled to avoid driving in a traffic lane associated with a mismatched traffic lane marking.

[0017] The at least one instruction can be configured, when executed by a third processor communicating with the memory, to cause the vehicle control device to control the vehicle's driving operation in such a way as to deactivate the vehicle's automatic driving operation or change the automatic driving level of the vehicle's automatic driving operation based on determining that the vehicle's destination is unreachable by avoiding driving in a traffic lane associated with a mismatched traffic lane marking.

[0018] According to one or more exemplary embodiments of this disclosure, a method performed by a vehicle's device may include: determining first traffic lane marking information based on object recognition information received from the vehicle's sensors; determining second traffic lane marking information based on map information received from the vehicle's navigation system; and generating combined traffic lane marking information for the vehicle by comparing the first traffic lane marking information with the second traffic lane marking information. The combined traffic lane marking information may indicate at least one mismatched traffic lane marking where the first and second traffic lane marking information do not match. The method may further include controlling the vehicle's driving operations based on the combined traffic lane marking information.

[0019] The method may further include: determining at least one mismatched traffic lane marking by comparing at least one of the coordinates of the first traffic lane marking information and the second traffic lane marking information, the slope of the traffic lane marking, and the angle.

[0020] Controlling vehicle driving operations may include: based on determining that driving in a traffic lane associated with a mismatched traffic lane marking would render the vehicle's destination unreachable, controlling the vehicle to avoid driving in a traffic lane associated with a mismatched traffic lane marking.

[0021] Controlling the vehicle's driving operations may include: deactivating the vehicle's autonomous driving operation or changing the autonomous driving level of the vehicle's autonomous driving operation based on determining that the vehicle's destination is unreachable by avoiding driving in traffic lanes associated with mismatched traffic lane markings.

[0022] Controlling the vehicle's driving operations may include comparing combined traffic lane marking information generated by the vehicle's equipment with second combined traffic lane marking information generated by a second vehicle.

[0023] Controlling vehicle driving operations may include: determining that the combined traffic lane marking information generated by the vehicle is inaccurate based on the fact that the mismatched traffic lane markings are not identified as mismatched in the second combined traffic lane marking information.

[0024] Controlling the vehicle's driving operations may include: disabling the vehicle's autonomous driving operation or changing the autonomous driving level of the vehicle's autonomous driving operation based on the determination that the combined traffic lane marking information generated by the vehicle is inaccurate.

[0025] Controlling the vehicle's driving operations may include: identifying mismatched traffic lane markings as mismatched in the second combined traffic lane marking information, and sending the combined traffic lane marking information generated by the vehicle and the second combined traffic lane marking information generated by the second vehicle to the server.

[0026] Controlling vehicle driving operations may include: based on determining that driving in a traffic lane associated with a mismatched traffic lane marking would render the vehicle's destination unreachable, controlling the vehicle to avoid driving in a traffic lane associated with a mismatched traffic lane marking.

[0027] Controlling the vehicle's driving operations may include: deactivating the vehicle's autonomous driving operation or changing the autonomous driving level of the vehicle's autonomous driving operation based on determining that the vehicle's destination is unreachable by avoiding driving in traffic lanes associated with mismatched traffic lane markings. Attached Figure Description

[0028] The above and other objects, features, and advantages of this disclosure will become more apparent to those skilled in the art from the detailed description of one or more exemplary embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0029] Figure 1 This is a diagram illustrating a vehicle that sends and receives data by communicating with other devices;

[0030] Figure 2 This is a diagram showing the modules that make up a vehicle;

[0031] Figure 3 It is a diagram used to describe the operation of vehicle control devices;

[0032] Figure 4 , Figure 5 and Figure 6 This is a conceptual diagram of the vehicle control system.

[0033] Figure 7 This is a flowchart of the method for controlling the vehicle. Detailed Implementation

[0034] In the following, one or more exemplary embodiments of this disclosure will be described in detail with reference to the accompanying drawings.

[0035] However, the technical concept of this disclosure is not limited to the exemplary embodiments described, but can be implemented in various different forms, and one or more components in the exemplary embodiments can be used by selective combination and substitution within the scope of the technical concept of this disclosure.

[0036] Furthermore, unless specifically defined and described, the terms (including technical and scientific terms) used in the exemplary embodiments of this disclosure may be interpreted as meaning commonly understood by one of ordinary skill in the art to which this disclosure pertains, and commonly used terms (such as terms defined in a dictionary) may be interpreted in light of the contextual meaning of the relevant art.

[0037] The terminology used in one or more exemplary embodiments of this disclosure is for the purpose of describing exemplary embodiments only and is not intended to limit this disclosure.

[0038] In this specification, the singular form may include the plural form unless the context clearly specifies otherwise. For the purposes of this application and claims, the exemplary phrases “at least one of A, B, or C” or “at least one of A, B, or C” are used, which means “at least one A, or at least one B, or at least one C, or any combination of at least one A, at least one B, and at least one C.” Furthermore, as used herein, exemplary phrases such as “at least one of A, B, or C,” “at least one of A, B, and C,” “at least one of A, B, or C,” etc., may mean each listed item or all possible combinations of listed items. For example, “at least one of A or B” may mean (1) at least one A; (2) at least one B; or (3) at least one A and at least one B.

[0039] Throughout this disclosure, references to components, units, or modules generally refer to items that are logically grouped together to perform a function or a group of related functions. The same reference numerals are generally intended to refer to the same or similar components. Components, units, and modules can be implemented in software, hardware, or a combination of software and hardware. The components, units, modules, and / or functions described above can be implemented and / or performed by one or more processors. For example, components, units, and / or modules can include processors, microprocessors, graphics processing units, logic circuits, application-specific circuits, application-specific integrated circuits, programmable array logic, field-programmable gate arrays, controllers, microcontrollers, and / or other suitable hardware. Components, units, and / or modules may also include, for example, software control modules implemented using processors or logic circuits. Components, units, and / or modules may include, for example, or otherwise have access to memory, such as one or more non-transitory computer-readable storage media, such as random access memory, read-only memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, flash memory / other memory devices, data registry, databases, and / or other suitable hardware. One or more storage media may include any or all of the tangible memory of a computer, processor, or other associated modules, such as various semiconductor memories, tape drives, disk drives, etc., which can provide non-transitory storage for software programming at any time.

[0040] Additionally, when describing components of exemplary embodiments of this disclosure, terms such as first, second, A, B, (a), (b), etc., may be used.

[0041] These terms are used only to distinguish the component from other components, and the nature, sequence, or order of the component is not limited by these terms.

[0042] Additionally, when a component is described as “linked,” “coupled,” or “connected” to another component, the component is not only directly linked, coupled, or connected to the other component, but also “linked,” “coupled,” or “connected” to the other component, where the other component is positioned between the component and the other component.

[0043] Furthermore, when a component is described as being formed or positioned "on (above) or below (below) another component," the term "on (above) or below (below)" includes not only when the two components are in direct contact with each other, but also when one or more other components are formed or positioned between the two components. Additionally, when a component is described as being "on (above) or below (below)," the description can include meanings based on the upward and downward directions of a component.

[0044] In the following description, one or more exemplary embodiments will be described in detail with reference to the accompanying drawings, but the same or corresponding components will be indicated by the same reference numerals, regardless of the reference numerals, and redundant descriptions will be omitted.

[0045] In the following text, reference will be made to Figure 1 and Figure 2 Describe the vehicle. Figure 1 This is a diagram illustrating a vehicle that sends and receives data by communicating with other devices;

[0046] refer to Figure 1 Vehicle 100 can be powered by either electricity or fossil fuels. In the case of electricity, vehicle 100 can be, for example, a battery-based vehicle powered solely by a high-voltage battery, or it can utilize a gas-based fuel cell unit as its energy source. Furthermore, the fuel cell unit can use various types of gases capable of generating electricity, and vehicle 100 can be filled with, for example, a liquefied gas. An example of such a gas could be hydrogen. However, the gas is not limited to this, and various gases can be used. In the case of fossil fuels, vehicle 100 can be powered by fuels such as gasoline, diesel, or liquefied petroleum gas (LPG), and can be equipped with an internal combustion engine that drives an actuation unit (also referred to as an actuator) 116 through the combustion of fuel. In providing driving power to the wheels of the drive unit (e.g., powertrain) 118, the engine can be included in an energy generation unit (also referred to as a generator, generator, energy generator, etc.) 110. As another example, vehicle 100 can selectively utilize energy from a fossil-based internal combustion engine and a battery to drive the actuation unit 116, and can be a hybrid vehicle.

[0047] Vehicle 100 can refer to a mobile device. Vehicle 100 can be a ground vehicle that travels on the ground and can be a typical bus, commercial vehicle, special purpose vehicle (PBV), etc. Vehicle 100 can be a four-wheeled vehicle, such as a bus, sport utility vehicle (SUV), or light truck, or it can be a vehicle with more than four wheels, such as a bus, large truck, container truck, heavy equipment vehicle, etc. Ground vehicle can be referred to as any vehicle, including vehicles that move underground and vehicles that move on land. Vehicle 100 can be a robot in a broad sense, such as a mobile device, and the robot can move using wheels, tracks, or other mobile modules. In this disclosure, ground mobile devices such as ground vehicles are primarily described; however, unless contradicted by this disclosure, this disclosure can also be applied to air mobile devices such as advanced air mobility (AAM) vehicles, aircraft, etc., and water mobile devices such as ships, submarines, etc.

[0048] Vehicle 100 can be controlled and driven by autonomous driving, and autonomous driving can be implemented as semi-autonomous or fully autonomous driving. Fully autonomous driving can be provided as autonomous movement, wherein even when driving conditions are uncertain, the processor 130 (first processing unit 131 to third processing unit 133) of vehicle 100 maintains full control without user intervention. Semi-autonomous driving can be provided as autonomous movement that requires driver intervention depending on specific driving conditions. Semi-autonomous driving can be implemented such that when the aforementioned conditions occur, the processor 130 transfers control to the user by disabling autonomous driving, thereby allowing the user to perform manual driving. According to the levels of autonomous driving defined by the Society of Automotive Engineers (SAE), semi-autonomous driving can correspond to levels 1 to 4, and fully autonomous driving can correspond to level 5.

[0049] According to the Society of Automotive Engineers (SAE), the automation levels of autonomous vehicles can be classified as follows: At Level 0, the SAE classification corresponds to "No Automated Operation," where the automated system temporarily handles emergency situations (e.g., automatic emergency braking) and / or only provides warnings (e.g., blind spot warning, lane departure warning, etc.) and expects the driver to operate the vehicle. At Level 1, the SAE classification corresponds to "Driver Assistance," where the system performs some driving functions (e.g., steering, acceleration, braking, lane centering, adaptive cruise control, etc.) when the driver operates the vehicle in normal operating conditions, and expects the driver to determine the system's operating status and / or timing, perform other driving functions, and handle (e.g., resolve) emergency situations. At Level 2, the SAE classification corresponds to "Partial Automation," where the system performs steering, acceleration, and / or braking under driver supervision, and expects the driver to determine the system's operating status and / or timing, perform other driving functions, and handle (e.g., resolve) emergency situations. At Level 3 of autonomous driving, the SAE classification standard can correspond to "conditional automation," where the system drives the vehicle under limited conditions (e.g., performing driving functions such as steering, acceleration, and / or braking), but transfers driving control to the driver when the desired conditions are not met, and expects the driver to determine the system's operating state and / or timing, and to take over control in emergency situations, but not to operate the vehicle in other ways (e.g., steering, acceleration, and / or braking). At Level 4 of autonomous driving, the SAE classification standard can correspond to "high automation," where the system performs all driving functions, and expects the driver to control the vehicle only in emergency situations. At Level 5 of autonomous driving, the SAE classification standard can correspond to "full automation," where the system performs all driving functions without any assistance from the driver, including in emergency situations, and the driver does not expect to perform any driving functions other than determining the system's operating state. While this disclosure can apply the SAE classification standard to autonomous driving classification, other classification methods and / or algorithms can be used in one or more configurations described herein. One or more features associated with autonomous driving control can be activated based on the configured autonomous driving control settings (e.g., based on at least one of the following: autonomous driving classification, selection of the vehicle's autonomous driving level, etc.).

[0050] Based on one or more features described herein (e.g., analyzing and comparing traffic marking information obtained from sensors and maps), vehicle operation can be controlled. Vehicle control can include various operational controls associated with the vehicle (e.g., automatic driving control, sensor control, braking control, braking time control, acceleration control, acceleration rate of change control, warning timing control, forward collision warning timing control, etc.).

[0051] One or more auxiliary devices (e.g., engine brakes, exhaust brakes, hydraulic reducers, electric reducers, regenerative brakes, etc.) can also be controlled, for example, based on one or more features described herein (e.g., analyzing and comparing traffic marking information obtained from sensors and maps). For example, based on one or more features described herein (e.g., analyzing and comparing traffic marking information obtained from sensors and maps), one or more communication devices (e.g., modems, network adapters, radio transceivers, antennas, etc.) capable of communicating via one or more wired or wireless communication protocols such as Ethernet, Wi-Fi, Near Field Communication (NFC), Bluetooth, Long Term Evolution (LTE), 5G New Radio (NR), Vehicle-to-Everything (V2X), etc.) can also be controlled.

[0052] One or more Minimum Risk Maneuvering (MRM) operations can also be controlled, for example, based on one or more features described herein (e.g., analyzing and comparing traffic marking information obtained from sensors and maps). Minimum Risk Maneuvering operations (e.g., minimum risk maneuver, minimum risk control) can be vehicle maneuvers designed to minimize (e.g., reduce) the risk of collisions with surrounding vehicles in order to achieve a reduced (e.g., minimum) risk state. Minimum Risk Maneuvering can be an operation activated during autonomous driving when the driver is unable to respond to an intervention request. During Minimum Risk Maneuvering, one or more processors in the vehicle can control the vehicle's driving operations for a set time period.

[0053] For example, multiple offset driving operations can be controlled based on one or more features described herein (e.g., analyzing and comparing traffic marking information obtained from sensors and maps). The drive control device can perform offset drive control. To perform offset driving, the drive control device can control the vehicle to travel within the lane by maintaining the lateral distance between the vehicle's center position and the lane center. For example, the drive control device can control the vehicle to remain within the lane but not in the center of the lane.

[0054] The drive control unit can identify a target lateral distance for bias drive control. For example, the target lateral distance may include an intentionally adjusted lateral distance that the vehicle can maintain from a reference point (such as the center of the lane or another vehicle) during maneuvers such as lane changes. This adjustment can be made to improve the vehicle's stability, safety, and / or performance under changing driving conditions. For example, during lane changes, the drive control system may bias the lateral distance to maintain a safer clearance from nearby vehicles, taking into account factors such as vehicle speed, road conditions, and / or the presence of obstacles.

[0055] One or more sensors (e.g., IMU sensor, camera, LIDAR, RADAR, blind spot monitoring sensor, lane departure warning sensor, parking sensor, light sensor, rain sensor, traction control sensor, anti-lock braking system sensor, tire pressure monitoring sensor, seat belt sensor, airbag sensor, fuel sensor, emission sensor, throttle position sensor, inverter, converter, motor controller, power distribution unit, high voltage wiring and connectors, auxiliary power module, charging interface, etc.) may also be used for control, for example, based on one or more features described herein (e.g., analyzing and comparing traffic marking information obtained from sensors and maps).

[0056] Operational control for autonomous driving of vehicles may include various driving controls of the vehicle via vehicle control devices (e.g., acceleration, deceleration, steering control, gear shifting control, braking system control, traction control, stability control, cruise control, lane keeping assist control, collision avoidance system control, emergency braking assist control, traffic marking recognition control, adaptive headlight control, driver alert control, and the field of autonomous driving operation design (ODD)).

[0057] The level of autonomous driving and / or activation / deactivation of autonomous driving can also be controlled, for example, based on one or more features described herein (e.g., analyzing and comparing traffic marking information obtained from sensors and maps). The driving control unit can perform autonomous driving level control (e.g., changing the level of autonomous driving, changing the required user attention, etc.) or deactivate autonomous driving operation. For example, by changing the required user attention, the driver may be required to place his / her hands on the drive wheels more frequently (e.g., at least once within a threshold time period, such as 5 seconds, 30 seconds, 1 minute, etc.). By changing the required user attention, the driver may be required to look forward more frequently (e.g., at least once within a threshold time period, such as 5 seconds, 30 seconds, 1 minute, etc.). By changing the level of autonomous driving, one or more video contents may not be displayed on the vehicle's displays.

[0058] Vehicle 100 can communicate with other devices 200 and 300 or another vehicle 400. These other devices may include, for example, a server 200 supporting various controls, status management, and driving functions of vehicle 100; an Intelligent Transportation System (ITS) device 300 for receiving information from the ITS; and various types of user devices. Server 200 may be, for example, an external device operated by the vehicle manufacturer or configured to serve autonomous driving, and can receive connection data from vehicle 100 or transmit data required for autonomous driving. In response to requests and data transmitted from vehicle 100 and user devices, server 200 can transmit various information and software modules to vehicle 100 for controlling vehicle 100 to support autonomous driving and various services of vehicle 100.

[0059] The ITS device 300 can be, for example, a roadside unit (RSU), and can exchange vehicle identification data, driving operation control and status data, environmental data around the vehicle, map data, etc., with the vehicle-to-infrastructure (V2I) communication of the vehicle 100 to assist the user in driving his or her own vehicle or support the autonomous driving of the vehicle 100. The vehicle 100 can also support manual or autonomous driving by exchanging the data listed above with other vehicles 400 through vehicle-to-vehicle (V2V) communication.

[0060] Vehicle 100 can communicate with other vehicles or other devices based on cellular communication, in-vehicle wireless access (WAVE) communication, dedicated short-range communication (DSRC), short-range communication, or other communication methods.

[0061] For example, vehicle 100 can use cellular communication networks (e.g., LTE or 5G), Wi-Fi communication networks, WAVE communication networks, etc., to communicate with server 200, ITS device 300, and other vehicles 400. As another example, DSRC or similar technologies used in vehicle 100 can be used for communication between vehicles. The communication methods between vehicle 100, server 200, ITS device 300, other vehicles 400, and user devices are not limited to the exemplary embodiments described herein.

[0062] Figure 2 This is a diagram showing the modules that make up a vehicle.

[0063] Vehicle 100 may include one or more sensors 102, an operating unit (also referred to as a user interface, control panel, instrument panel, dashboard, etc.) 106, a display 108, a load device (also referred to as a load or electrical load) 114, and a transmitting / receiving unit (also referred to as a communication interface, transceiver, etc.) 112.

[0064] One or more sensors 102 may be equipped with various types of detectors to detect various states and conditions that occur in the external environment, internal systems, user operations, and the vehicle's upper space.

[0065] Specifically, the first sensor 102 may be equipped with an externally oriented camera 102a, a lidar 102b, a radar 102c, etc., to identify dynamic and static objects existing outside the vehicle 100. When the vehicle 100 is in use, the camera 102a can identify the external object as an image, generate image data, and transmit the image data to the processor 130. The lidar 102b can generate point cloud data as identification data for the external object, and transmit the point cloud data to the processor 130 to generate 3D spatial information that identifies at least the shape of the external object. To determine the presence of the external object and its relative distance, speed, direction, etc., the radar 102c can emit radio waves of a specific frequency around the vehicle 100 and generate radar data from the radio waves reflected from the external object. In this disclosure, the sensor unit is shown to have a lidar 102b, but in other examples, the lidar 102b may not be installed.

[0066] The first sensor 102 can generate object recognition information based on sensing data. The object recognition information may include information about the existence of the object, information about the object's location, information about the distance between the vehicle 100 and the object, and information about the relative speed between the vehicle 100 and the object. External objects can be various objects related to the operation of the vehicle 100.

[0067] The second sensor unit 103 may be equipped with a positioning sensor 103a, a wheel sensor 103b, an attitude sensor 103c, etc., to confirm its own position, speed, driving posture, etc. The attitude sensor 103c may include a gyroscope sensor, an angular velocity sensor, an acceleration sensor, etc. The attitude sensor may be an inertial measurement unit (IMU) sensor and may be equipped with a 3-axis accelerometer and a 3-axis gyroscope. The attitude sensor 103c can measure the acceleration of the vehicle 100 in the direction of travel (x), the acceleration in the lateral direction (y), and the acceleration in the height direction (z), as well as yaw, pitch, and roll as the angular velocity of the vehicle.

[0068] The second sensor unit 103 can generate vehicle driving information based on the sensing data. Vehicle driving information can be generated based on data detected by various sensors installed inside the vehicle. For example, vehicle driving information may include vehicle attitude information, vehicle speed information, vehicle tilt information, vehicle weight information, vehicle direction information, vehicle battery information, vehicle fuel information, vehicle tire pressure information, vehicle steering information, vehicle interior temperature information, vehicle interior humidity information, pedal position information, vehicle engine temperature information, etc.

[0069] In addition, vehicle driving information may include route information. Route information may refer to information generated based on the destination input by the vehicle user through operation unit 106. Route information refers to information indicating the driving path from the vehicle's current location to the destination on the map, if a destination is set. If no destination is set, route information may refer to information including the road the vehicle is currently traveling on and the future driving route including the road.

[0070] The third sensor unit 105 may include a voice sensor 105a for collecting voice signals inside the vehicle, a vibration sensor 105b arranged around the occupants, and a camera 105c for capturing images inside the vehicle.

[0071] The voice sensor 105a may include at least one microphone disposed inside the vehicle and may collect speech and buzzing sounds expressed by occupants inside the vehicle to generate a voice signal (e.g., an electrical signal).

[0072] Vibration sensor 105b may include at least one accelerometer or gyroscope sensor located in a position accessible to the occupant's body, and may generate a vibration signal (e.g., an electrical signal) by measuring the vibrations generated when the steering wheel, console box, or dashboard inside the vehicle is touched or tapped.

[0073] The camera 105c can capture the interior of a vehicle and can be positioned to face the front of the occupant's upper body, thereby generating a video signal by capturing the occupant's movements.

[0074] The operating unit 106 can be configured as a module controlled by a user for driving. For example, the operating unit 106 can be a manually driven steering wheel, an automatic or manual transmission, an accelerator pedal, a brake pedal, etc. The operating unit 106 can also be provided with an interface for enabling or disabling autonomous driving modes and selecting detailed functions requested by the user to enable autonomous driving functions. To receive various requests related to autonomous driving, the operating unit 106 can be configured as, for example, a hard interface located at a predetermined location inside the vehicle 100, or a soft interface that can be touched on the display 108. Depending on the specifications of the autonomous vehicle, at least one of the steering wheel, transmission, and pedals may be omitted. As another example, the operating unit 106 can be provided with a module for receiving user control requests for the load device 114 in addition to driving control.

[0075] The display 108 can be used as a user interface. The display 108 can output and display, via the processor 130, the operating status and control status of the vehicle 100, route / traffic information, remaining energy information, driver requests, etc. Furthermore, the display 108 can be configured to detect driver input to receive driver requests instructing the processor 130.

[0076] The load device 114 can be mounted on the vehicle 100 and can be any electrical device independent of the drive power system, such as the wheel drive unit 118. The load device 114 can be an auxiliary device that receives electricity from the energy generation unit 110, and can be, for example, an air conditioning system, a lighting system, a seating system, various devices installed in the vehicle 100, etc. In this disclosure, a cooling / heating system may be further included, which cools or heats at least one of the battery, fuel cell, internal combustion engine, air conditioning system, and specific parts of the vehicle 100.

[0077] The transmitting / receiving unit 112 can support communication with the server 200, the ITS device 300, nearby vehicles 400, etc. The transmitting / receiving unit 112 may include modules for processing, for example, cellular communication, WAVE, DSRC communication, etc. In this disclosure, the transmitting / receiving unit 112 can send data generated or stored during operation to the server 200 and receive data and software modules sent from the server 200. The transmitting / receiving unit 112 can support communication with electronic devices carried by occupants within the vehicle 100. In this disclosure, the vehicle 100 can use the transmitting / receiving unit 112 to send data utilized in the method according to this disclosure to the outside and to receive data utilized in the method according to this disclosure from the outside.

[0078] For example, the transmitting / receiving unit 112 can receive traffic signal information from the traffic signal controller and send the traffic signal information to the processor 130. In addition, the transmitting / receiving unit 112 can receive control signals from the traffic flow signal controller and send the control signals to the processor 130.

[0079] In addition, the vehicle 100 may include an energy generation unit 110 and an actuation unit 116.

[0080] Energy generation unit 110 can generate and supply power and electricity for driving and non-driving power systems (such as actuation unit 116). For example, the non-driving power system may be one or more sensors 102, operating unit 106, display 108, load device 114, and transmitting / receiving unit 112, but is not limited thereto, and may include various components for sensing, interface, communication, and convenience functions, excluding components directly involved in driving operation. If the vehicle 100 is driven by electric energy, energy generation unit 110 may be configured as an externally charged battery or as a combination of a battery and a fuel cell. In the case of a battery and fuel cell combination, energy generation unit 110 may include a tank for storing materials (such as liquefied hydrogen) used to generate electricity from the fuel cell. If the vehicle 100 is driven by fossil energy, energy generation unit 110 may be configured as an internal combustion engine. Furthermore, if the vehicle 100 is a hybrid type, energy generation unit 110 may be configured as a combination of an internal combustion engine and a battery.

[0081] Actuation unit 116 may include at least one module that, upon a user request from operation unit 106, performs driving operations and executes at least one of longitudinal control (such as acceleration and deceleration) and lateral control (such as steering). To perform driving operations via manual or automatic driving control according to commands from processor 130, actuation unit 116 may include wheel drive unit 118 and mechanical components and electronic modules for performing driving operations within wheel drive unit 118. If vehicle 100 operates on electric power, actuation unit 116 may include components for transmitting requested driving operations to wheel drive unit 118. If vehicle 100 operates on fossil fuel power, actuation unit 116 may include a transmission and gear module for transmitting power from an internal combustion engine.

[0082] The wheel drive unit 118 may include: multiple wheels; a drive force generation module (e.g., an engine, motor, etc.) for generating drive force and applying or transmitting the drive force to the wheels; a braking module for decelerating the drive of the wheels; and a steering module for performing lateral control of the wheels. If the electric vehicle 100 is powered by electricity, the drive force generation module may be configured as a motor assembly that generates drive force based on electricity output from a battery. The braking module of the electric vehicle 100 may further have regenerative braking functionality.

[0083] The navigation unit (also known as the navigation system) 122 can provide navigation information. The navigation information may include at least one of the following: map information, set destination information, route information based on the set destination, information about various objects on the route, lane information, and current vehicle location information.

[0084] The navigation unit 122 can receive information from external devices through the sending / receiving unit 112 and update pre-stored information. The navigation unit 122 can be classified as a sub-component of the operation unit 106.

[0085] The vibrator 140 can be installed on the backrest and bottom of the seat and can output vibration signals independently. The vibrator 140 can be arranged to be embedded in the empty space of the backrest and bottom of the seat. Each vibrator 140 operates independently under the control of the processor 130 and can output a predetermined vibration signal.

[0086] Figure 3 This is a diagram used to illustrate the operation of vehicle control devices. (Reference) Figure 3 The vehicle control device may include a memory 210, a processor 220, and a transmit / receive unit 230. The memory 210 and processor 220 of the vehicle control device may have [specific features / equivalent to / other features]. Figure 2 The memory 120, processor, and transmission / reception unit have the same configuration.

[0087] The memory 210 can store applications and various types of data used to control the vehicle control device, and can load applications or read and record data upon request from the processor 220.

[0088] Processor 220 can perform overall control of the vehicle control unit. Processor 220 can be configured to execute applications and instructions stored in memory 210.

[0089] The processor 220 may include a first processing unit (also referred to as a first processor) 221, a second processing unit (also referred to as a second processor) 222 and a third processing unit (also referred to as a third processor) 223.

[0090] The first processing unit 221 can determine first traffic lane marking information (also referred to as first lane line information) through object recognition information detected by the sensor unit, and determine second traffic lane marking information through map information from the navigation unit. Traffic lane markings can also be referred to as road markings, pavement markings, traffic lines, lane lines, traffic lane lines, pavement markings, pavement lines, etc. Traffic lane marking information can also be referred to as road marking information, pavement marking information, traffic line information, traffic lane line information, pavement marking information, pavement line information, etc. Traffic lane marking information can include information about the location, direction, orientation, color, shape, type, grade (e.g., slope), angle, etc., of one or more traffic lane markings.

[0091] The first processing unit 221 can determine the first traffic lane marking information using at least one of the camera image information, lidar point cloud data, and radar data.

[0092] For example, the first processing unit 221 can use image information from the camera received from the first sensor unit to determine the color and shape of road traffic lane markings (e.g., dashed lines, solid lines).

[0093] For example, the first processing unit 221 can estimate traffic lane markings by using point cloud data of lidar received from the first sensor unit to identify road height differences, surface shapes, etc.

[0094] For example, the first processing unit 221 can use radar data received from the first sensor unit to determine distance information of an object or road structure.

[0095] The second processing unit 222 can analyze the map information received from the navigation unit to determine the second traffic lane marking information. The map information may be high-definition map (HD Map) information. A high-definition map can refer to a map or map data with a resolution higher than a certain threshold (e.g., data resolution, image resolution, etc.). The second processing unit 222 can determine the second traffic lane marking information by analyzing information included in the map information regarding the location, curvature, and traffic lane marking type (e.g., bus only, left turn only, etc.).

[0096] The first processing unit 221 can analyze the object recognition information received from the sensor unit, recognize traffic lane markings according to a preset period, and output the first traffic lane marking information.

[0097] The second processing unit 222 can analyze the map information of the navigation unit, identify traffic lane markings based on information about the current location of the main vehicle, and output the second traffic lane marking information.

[0098] The first and second traffic lane marking information may include point coordinates and traffic lane marking shape information.

[0099] The second processing unit 222 can analyze whether the first traffic lane marking information and the second traffic lane marking information match, determine the mismatched abnormal (e.g., mismatched) traffic lane markings based on the traffic lane marking matching results, and calculate (e.g., determine) the traffic lane marking matching information (also referred to as combined traffic lane marking information or matched traffic lane marking information) for the main vehicle.

[0100] For example, the second processing unit 222 can determine abnormal (e.g., mismatched) traffic lane markings by comparing at least one of the coordinates, slope, and angle of the first and second traffic lane marking information. An abnormal traffic lane marking may be a lane marking detected only in one of the first and second traffic lane marking information as a result of comparing the first and second traffic lane marking information. Alternatively, an abnormal traffic lane marking may be a lane line that is difficult to define (e.g., below a threshold tolerance or confidence level) as the same traffic lane marking because, as a result of comparing the first and second traffic lane marking information, its position, shape, etc., are sufficiently different to affect vehicle movement.

[0101] The second processing unit 222 can compare first traffic lane marking information and second traffic lane marking information identified within a specific range based on the current position of the main vehicle to determine whether the first traffic lane marking information and the second traffic lane marking information match. The second processing unit 222 can convert the first traffic lane marking information and the second traffic lane marking information into the same coordinate system (e.g., WGS84) and compare them.

[0102] The second processing unit 222 can determine abnormal (e.g., mismatched) traffic lane markings by comparing the distances between first and second traffic lane marking information, which will be converted to the same coordinate system. The second processing unit 222 can calculate (e.g., determine) the distance between the point coordinates of the first and second traffic lane marking information located on the same horizontal axis. The second processing unit 222 can identify traffic lane markings whose distance values ​​exceed a reference distance value as abnormal traffic lane markings. Alternatively, if a traffic lane marking whose distance value exceeds the reference distance value remains at a preset length or longer, the second processing unit 222 can identify the corresponding traffic lane marking as an abnormal traffic lane marking.

[0103] Additionally, the second processing unit 222 can determine abnormal traffic lane markings by comparing their shapes. The second processing unit 222 can compare the types of traffic lane markings included in the traffic lane marking information using coordinates, and determine mismatched traffic lane markings as abnormal traffic lane markings based on the comparison results. For example, if a traffic lane marking is classified as a solid line in the first traffic lane marking information but as a dashed line in the second traffic lane marking information, the second processing unit 222 can determine the corresponding traffic lane marking as an abnormal traffic lane marking.

[0104] Additionally, the second processing unit 222 can use the change in the coordinates between points of the traffic lane markings to calculate (e.g., determine) the angle and slope of the traffic lane markings. If the angle and slope of the traffic lane markings calculated (e.g., determined) from the first traffic lane marking information and the angle and slope of the traffic lane markings calculated (e.g., determined) from the second traffic lane marking information exceed reference values, the second processing unit 222 can identify the corresponding traffic lane markings as abnormal traffic lane markings.

[0105] For example, if map information shows three traffic lane markings on a highway entrance ramp, but due to a road construction event, the number is reduced to two, the matching result between the first traffic lane marking information and the second traffic lane marking information can indicate a mismatch.

[0106] The third processing unit 223 can establish a driving strategy to reach the destination by reflecting traffic lane marking matching information about the main vehicle (e.g., collected, determined, generated, etc. by the main vehicle).

[0107] For example, the third processing unit 223 can determine whether driving is feasible, even if there are abnormal (e.g., mismatched) traffic lane markings, or whether reaching a preset destination is feasible. If the abnormal traffic lane markings are traffic lane markings that do not affect the driving route of the main vehicle, that is, if the abnormal traffic lane markings are not traffic lane markings on either side of the main vehicle's driving route or traffic lane markings located on lane change routes, then the third processing unit 223 can determine that the abnormal traffic lane markings do not affect driving or reaching the destination.

[0108] Alternatively, if the remaining distance to the destination is less than or equal to the predetermined distance, the third processing unit 223 can determine that the abnormal traffic lane markings do not affect the arrival at the destination.

[0109] Alternatively, if the abnormal traffic lane markings are caused by a mismatch in the type of traffic lane markings, then if no lane change that crosses the corresponding traffic lane markings is planned on the current driving route, the third processing unit 223 can determine that the abnormal traffic lane markings do not affect driving or reaching the destination.

[0110] If it is determined that abnormal traffic lane markings do not affect driving or reaching the destination, the third processing unit 223 can establish a driving strategy to maintain the current driving route.

[0111] If it is determined that reaching the destination is impossible due to abnormal traffic lane markings based on the current driving strategy, the third processing unit 223 can establish an avoidance driving strategy (e.g., avoidance operation) to avoid driving on abnormal traffic lane markings.

[0112] For example, if the abnormal traffic lane markings are on either side of the driving route of the main vehicle, the third processing unit 223 can determine that the abnormal traffic lane markings affect driving or reaching the destination.

[0113] Alternatively, if the remaining distance to the destination exceeds the predetermined distance, the third processing unit 223 may determine that abnormal traffic lane markings affect the arrival at the destination.

[0114] Alternatively, if the abnormal traffic lane markings are caused by a mismatch in the type of traffic lane markings, then if a lane change that crosses the corresponding traffic lane markings is planned on the current driving route, the third processing unit 223 can determine that the abnormal traffic lane markings affect driving or reaching the destination.

[0115] If it is determined that abnormal traffic lane markings affect driving or reaching the destination, the third processing unit 223 can modify the current driving route and establish a driving strategy to avoid abnormal traffic lane markings. The third processing unit 223 can establish the driving strategy by reflecting real-time traffic conditions, road conditions, and the vehicle's current position.

[0116] For example, the third processing unit 223 can establish a driving strategy to include an avoidance path for driving or reaching a destination without changing lanes that cross abnormal traffic lane markings.

[0117] Alternatively, if the abnormal traffic lane markings are on either side of the main vehicle's driving route, the third processing unit 223 can establish a driving strategy to change the main vehicle's driving lane.

[0118] In this scenario, the third processing unit 223 can calculate (e.g., determine) multiple avoidance routes. The third processing unit 223 can calculate (e.g., determine) and output these multiple avoidance routes via an operating unit in the vehicle, a display, or the like. If there is a driver selection input, the third processing unit 223 can establish a driving strategy based on the selected avoidance route.

[0119] Alternatively, the third processing unit 223 may use the avoidance route that can reach the destination in the shortest time to establish a driving strategy when calculating (e.g., determining) multiple avoidance routes (e.g., after, during, etc.).

[0120] Alternatively, when calculating (e.g., determining) multiple avoidance routes (e.g., subsequently, simultaneously, etc.), the third processing unit 223 can establish a driving strategy by selecting the avoidance route that has the least impact on the abnormal traffic lane markings. For example, the third processing unit 223 can establish a driving strategy by selecting the avoidance route that is furthest away from the abnormal traffic lane markings.

[0121] If, based on the avoidance strategy, it is determined that reaching the destination is impossible due to abnormal traffic lane markings, the third processing unit 223 may execute an automatic driving deactivation operation (e.g., disabling automatic driving). If it is determined that the destination cannot be reached via the avoidance route of the abnormal traffic lane markings, the third processing unit 223 may release automatic driving to transfer control to the driver. The third processing unit 223 may output a driver takeover request to the driver via an operating unit, display, etc., and control the vehicle to stop in a nearby available stopping space to await the driver's decision.

[0122] Alternatively, the third processing unit 223 can output a temporary stop request to the driver via an operating unit, display, or the like in the vehicle. If a driving strategy to bypass the obstacle cannot be established, the third processing unit 223 can search for nearby available stopping space and output a temporary stop request to the driver. If there is temporary stop approval input from the driver, the third processing unit 223 can drive to the searched nearby available stopping space and execute a temporary stop.

[0123] The third processing unit 223 can use object recognition information from the sensor unit to determine whether a nearby vehicle is traveling within a predetermined distance radius based on the current driving position of the main vehicle. If a nearby vehicle exists, the third processing unit 223 can perform data communication with the nearby vehicle through the sending / receiving unit 230, and can request traffic lane marking matching information from the nearby vehicle and send and receive traffic lane marking matching information from the nearby vehicle.

[0124] When receiving lane marking matching information from nearby vehicles (e.g., subsequently, simultaneously, etc.), the third processing unit 223 can establish a driving strategy by comparing the lane marking matching information about another vehicle collected from nearby vehicles with the lane marking matching information about the main vehicle.

[0125] If the abnormal traffic lane markings are included in the traffic lane marking matching information for the primary vehicle but not in the traffic lane marking matching information for another vehicle, the third processing unit 223 can determine that an anomaly has occurred in the primary vehicle (e.g., the traffic lane marking matching information generated by the primary vehicle is inaccurate). In this case, the third processing unit 223 can determine that an anomaly has occurred in the sensor unit of the primary vehicle.

[0126] If an anomaly is detected in the main vehicle, the third processing unit 223 can perform an autopilot deactivation operation (e.g., disabling autopilot). The third processing unit 223 can release autopilot to transfer control to the driver. The third processing unit 223 can output a driver takeover request to the driver via an operating unit, display, etc. in the vehicle, and control the vehicle to stop in a nearby available parking space to await the driver's decision.

[0127] If abnormal lane markings are included in the lane marking matching information for the primary vehicle (e.g., associated with the primary vehicle) and the lane marking matching information for another vehicle (e.g., associated with another vehicle), the third processing unit 223 can send the lane marking matching information for the primary vehicle (e.g., collected, determined, generated, etc. by the primary vehicle) and the lane marking matching information for the other vehicle to the server. The third processing unit 223 can compare the coordinates of the abnormal lane markings included in the lane marking matching information for the primary vehicle with the coordinates of the abnormal lane markings included in the lane marking matching information for the other vehicle. If the coordinates are the same, it is determined that there is an error in the map information or an abnormal road condition, and the corresponding information is transmitted to the server. In this case, the third processing unit 223 can request the server to respond with map information reflecting the abnormal lane marking information or map information that modifies the abnormal lane marking information.

[0128] In this case, the third processing unit 223 can improve the reliability of the determination result by comparing the traffic lane marking matching information about the main vehicle (e.g., collected, determined, generated, etc. by the main vehicle) with the traffic lane marking matching information about a preset number or more other vehicles.

[0129] For example, the third processing unit 223 can determine whether it is still possible to drive or reach the preset destination even if there are abnormal traffic lane markings. If the abnormal traffic lane markings are traffic lane markings that do not affect the driving route of the main vehicle, that is, if the abnormal traffic lane markings are not traffic lane markings on either side of the driving route of the main vehicle or traffic lane markings located on the lane change route, then the third processing unit 223 can determine that the abnormal traffic lane markings do not affect driving or reaching the destination.

[0130] Alternatively, if the remaining distance to the destination is less than or equal to the predetermined distance, the third processing unit 223 can determine that the abnormal traffic lane markings do not affect the arrival at the destination.

[0131] Alternatively, if the abnormal traffic lane markings are caused by a mismatch in the type of traffic lane markings, then if no lane change that crosses the corresponding traffic lane markings is planned on the current driving route, the third processing unit 223 can determine that the abnormal traffic lane markings do not affect driving or reaching the destination.

[0132] If it is determined that abnormal traffic lane markings do not affect driving or reaching the destination, the third processing unit 223 can establish a driving strategy to maintain the current driving route.

[0133] If it is determined that reaching the destination is impossible due to abnormal traffic lane markings based on the current driving strategy, the third processing unit 223 can establish an avoidance driving strategy to avoid driving on abnormal traffic lane markings.

[0134] For example, if the abnormal traffic lane markings are on either side of the driving route of the main vehicle, the third processing unit 223 can determine that the abnormal traffic lane markings affect driving or reaching the destination.

[0135] Alternatively, if the remaining distance to the destination exceeds the predetermined distance, the third processing unit 223 may determine that abnormal traffic lane markings affect the arrival at the destination.

[0136] Alternatively, if the abnormal traffic lane markings are caused by a mismatch in the type of traffic lane markings, then if a lane change that crosses the corresponding traffic lane markings is planned on the current driving route, the third processing unit 223 can determine that the abnormal traffic lane markings affect driving or reaching the destination.

[0137] If it is determined that abnormal traffic lane markings affect driving or reaching the destination, the third processing unit 223 can modify the current driving route and establish a driving strategy to avoid abnormal traffic lane markings. The third processing unit 223 can establish the driving strategy by reflecting real-time traffic conditions, road conditions, and the vehicle's current position.

[0138] For example, the third processing unit 223 can establish a driving strategy to include an avoidance path for driving or reaching a destination without changing lanes that cross abnormal traffic lane markings.

[0139] Alternatively, if the abnormal traffic lane markings are on either side of the main vehicle's driving route, the third processing unit 223 can establish a driving strategy to change the main vehicle's driving lane.

[0140] In this scenario, the third processing unit 223 can calculate (e.g., determine) multiple avoidance routes. The third processing unit 223 can calculate (e.g., determine) and output multiple avoidance routes via an operating unit in the vehicle, a display, or the like. If there is a driver selection input, the third processing unit 223 can establish a driving strategy based on the selected avoidance route.

[0141] Alternatively, if multiple avoidance paths have been calculated (e.g., determined), the third processing unit 223 may use the avoidance path that reaches the destination in the shortest time to establish a driving strategy.

[0142] Alternatively, if multiple avoidance routes are calculated (e.g., determined), the third processing unit 223 can establish a driving strategy by selecting the avoidance route that has the least impact on the abnormal traffic lane markings. For example, the third processing unit 223 can establish a driving strategy by selecting the avoidance route that is furthest away from the abnormal traffic lane markings.

[0143] If, based on the avoidance strategy, it is determined that reaching the destination is impossible due to abnormal traffic lane markings, the third processing unit 223 can execute an automatic driving disengagement operation. If it is determined that the destination cannot be reached via the avoidance route of the abnormal traffic lane markings, the third processing unit 223 can release automatic driving to transfer control to the driver. The third processing unit 223 can output a driver takeover request to the driver through the vehicle's operating unit, display, etc., and control the vehicle to stop in a nearby available stopping space to await the driver's decision.

[0144] Alternatively, the third processing unit 223 can output a temporary stop request to the driver via an operating unit, display, or the like in the vehicle. If a driving strategy to bypass the obstacle cannot be established, the third processing unit 223 can search for nearby available stopping space and output a temporary stop request to the driver. If there is temporary stop approval input from the driver, the third processing unit 223 can drive to the searched nearby available stopping space and execute a temporary stop.

[0145] Figure 4 This is a conceptual diagram illustrating the operation of the vehicle control system. (Reference) Figure 4 The main vehicle may be in a situation where it must change lanes to the right along its current route, and the map information and object recognition information are different, which makes it possible to detect abnormal traffic lane markings on the right side of the main vehicle.

[0146] The vehicle control unit can receive lane marking matching information from nearby vehicles, compare the lane marking matching information with lane marking matching information about the main vehicle (e.g., collected, determined, generated, etc. by the main vehicle), and determine that an anomaly has occurred in the main vehicle if the abnormal lane marking is included in the lane marking matching information about the main vehicle but not in the lane marking matching information about another vehicle.

[0147] The vehicle control unit can send a driver takeover request to the driver through the operating unit, display, etc. in the vehicle, and control the vehicle to stop in a nearby available parking space to wait for the driver's decision.

[0148] Figure 5 This is a conceptual diagram illustrating the operation of the vehicle control system. (Reference) Figure 5 The map information and object recognition information are different, and abnormal traffic lane markings are detected on the left side of the main vehicle.

[0149] The vehicle control unit can receive traffic lane marking matching information from nearby vehicles, compare the traffic lane marking matching information with traffic lane marking matching information about the main vehicle, and determine whether the destination can be reached by the current driving route if the abnormal traffic lane marking is included in both the traffic lane marking matching information about the main vehicle (e.g., collected, determined, generated by the main vehicle, etc.) and the traffic lane marking matching information about another vehicle (e.g., collected, determined, generated by another vehicle, etc.).

[0150] The vehicle control unit can determine that abnormal traffic lane markings are unrelated to the main vehicle's driving traffic lane markings and that the destination can be reached without changing lanes, and establish a driving strategy to maintain the current driving route.

[0151] Figure 6 This is a conceptual diagram illustrating the operation of the vehicle control system. (Reference) Figure 6 The map information and object recognition information are different, and an abnormal traffic lane marking was detected on the left side of the main vehicle.

[0152] The vehicle control unit can receive traffic lane marking matching information from nearby vehicles, compare the traffic lane marking matching information with the traffic lane marking matching information for the main vehicle, and determine whether the destination can be reached via the current driving route if abnormal traffic lane markings are included in both the traffic lane marking matching information for the main vehicle and the traffic lane marking matching information for the other vehicle.

[0153] The primary vehicle may be in a situation where it must change lanes to the right along its current route, and abnormal traffic lane markings are located on the lane change route. Therefore, the vehicle control unit can determine that it is impossible to reach the destination via the current route and calculate (e.g., determine) an avoidance route to establish a driving strategy.

[0154] The vehicle control unit can output an avoidance path in the vehicle via an operating unit, display, etc., and if there is an approved input from the driver, it controls the operation of the vehicle according to the driving strategy of using the avoidance path.

[0155] Figure 7 This is a flowchart of the method for controlling the vehicle. (Reference) Figure 7 The processor can determine the first traffic lane marking information through the object recognition information detected by the sensor unit, and determine the second traffic lane marking information through the map information of the navigation unit (S701).

[0156] The processor can analyze whether the first traffic lane marking information and the second traffic lane marking information match, and determine (e.g., detect, identify, etc.) any mismatched abnormal traffic lane markings based on the traffic lane marking matching results. The processor can calculate (e.g., determine) traffic lane marking matching information for the main vehicle, including information about abnormal traffic lane markings (S702).

[0157] If no abnormal traffic lane markings are identified, the processor can maintain the driving strategy of continuing along the current route (S703).

[0158] When abnormal traffic lane markings are identified (e.g., subsequently, simultaneously, etc.), the processor uses object recognition information from the sensor unit to determine whether nearby vehicles are traveling within a predetermined distance radius based on the current driving position of the main vehicle (S704).

[0159] If no nearby vehicle is present or no lane marking matching information for another vehicle is received, the processor can determine the likelihood of reaching the destination based on the current driving strategy using the abnormal lane markings included in the lane marking matching information for the primary vehicle (S705).

[0160] If the destination can be reached according to the current driving strategy, the processor can maintain the driving strategy of following the current driving route (S706).

[0161] If the destination cannot be reached according to the current driving strategy, the processor can establish an avoidance driving strategy to avoid abnormal driving (S707 and S708).

[0162] If an avoidance driving strategy cannot be established or it is determined that the destination cannot be reached based on the avoidance driving strategy, the processor can perform an automatic driving disengagement operation (S709).

[0163] If there are nearby vehicles, the processor can perform data communication with the nearby vehicles through the sending / receiving unit, request traffic lane marking matching information from the nearby vehicles, and send and receive traffic lane marking matching information from the nearby vehicles (S710).

[0164] The processor compares the lane marking matching information for the primary vehicle with the lane marking matching information for another vehicle collected from nearby vehicles (S711).

[0165] If it is determined that the abnormal traffic lane markings are included in the traffic lane marking matching information for the primary vehicle but not in the traffic lane marking matching information for the other vehicle, the processor can determine that an anomaly has occurred in the primary vehicle (S712).

[0166] If an anomaly is detected in the main vehicle, the processor can execute an autopilot deactivation operation (S709).

[0167] Alternatively, if the abnormal lane marking is included in the lane marking matching information for the primary vehicle and the lane marking matching information for the other vehicle, the processor may send the lane marking matching information for the primary vehicle and the lane marking matching information for the other vehicle to the server (S713).

[0168] The processor can determine the likelihood of reaching the destination based on the current driving strategy by considering abnormal traffic lane markings included in the traffic lane marking matching information about the main vehicle (S705).

[0169] If the destination can be reached according to the current driving strategy, the processor can maintain the driving strategy of following the current driving route (S706).

[0170] If the destination cannot be reached according to the current driving strategy, the processor can establish an avoidance driving strategy to avoid driving on abnormal traffic lane markings (S707 and S708).

[0171] If it is determined that an avoidance driving strategy cannot be established or the destination cannot be reached based on the avoidance driving strategy, the processor can perform an automatic driving disengagement operation (S709).

[0172] A vehicle control device is provided, comprising: a first processing unit configured to determine first lane line information by means of object recognition information detected by a sensor unit and to determine second lane line information by means of map information from a navigation unit; a second processing unit configured to analyze whether the first lane line information and the second lane line information match, to determine an abnormal lane line that does not match based on the lane line matching result, and to calculate lane line matching information about a primary vehicle; and a third processing unit configured to establish a driving strategy for the vehicle by reflecting the lane line matching information about the primary vehicle.

[0173] The second processing unit can determine abnormal lane lines by comparing at least one of the coordinates, lane line slope, and angle of the first lane line information and the second lane line information.

[0174] When it is determined that reaching the destination is impossible due to the abnormal lane line according to the current driving strategy, the third processing unit can establish an avoidance driving strategy to avoid driving on the abnormal lane line.

[0175] When the driver determines that the destination cannot be reached due to an abnormal lane marking, the third processing unit can perform an automatic driving deactivation operation.

[0176] The third processing unit can establish a driving strategy by comparing lane matching information about the main vehicle with lane matching information about another vehicle collected from nearby vehicles.

[0177] When an abnormal lane line is included in the lane line matching information for the primary vehicle but not in the lane line matching information for another vehicle, the third processing unit can determine that an abnormality has occurred in the primary vehicle.

[0178] When an anomaly is detected in the main vehicle, the third processing unit can perform an automatic driving deactivation operation.

[0179] When an abnormal lane line is included in both lane line matching information for the primary vehicle and lane line matching information for the other vehicle, the third processing unit can send both lane line matching information for the primary vehicle and lane line matching information for the other vehicle to the server.

[0180] When it is determined that reaching the destination is impossible due to the abnormal lane line according to the current driving strategy, the third processing unit can establish an avoidance driving strategy to avoid driving on the abnormal lane line.

[0181] When the driver determines that the destination cannot be reached due to an abnormal lane marking, the third processing unit can perform an automatic driving deactivation operation.

[0182] A method for controlling a vehicle is provided, comprising: determining first lane line information by means of object recognition information detected by a sensor unit and determining second lane line information by means of map information of a navigation unit; analyzing whether the first lane line information and the second lane line information match, so as to determine the mismatched abnormal lane lines from the lane line matching results; calculating lane line matching information about the main vehicle including information about the abnormal lane lines; and establishing a driving strategy for reaching a destination by reflecting the lane line matching information about the main vehicle.

[0183] Determining abnormal lane lines may include comparing at least one of the coordinates, lane line slope, and angle of the first lane line information and the second lane line information to determine the abnormal lane line.

[0184] Establishing a driving strategy may include: when it is determined that reaching the destination is impossible due to an abnormal lane line according to the current driving strategy, an avoidance driving strategy is established to avoid driving on the abnormal lane line.

[0185] Establishing a driving strategy may include: when it is determined that the destination cannot be reached due to an abnormal lane marking, the automatic driving system is deactivated.

[0186] Establishing a driving strategy may include comparing lane matching information about the primary vehicle with lane matching information about another vehicle collected from nearby vehicles to establish a driving strategy.

[0187] Establishing a driving strategy may include determining that an anomaly has occurred in the primary vehicle when an abnormal lane line is included in the lane line matching information for the primary vehicle but not in the lane line matching information for another vehicle.

[0188] Establishing a driving strategy may include: when an abnormality is detected in the primary vehicle, executing an autopilot deactivation operation.

[0189] Establishing a driving strategy may include sending the lane matching information for the primary vehicle and the lane matching information for the other vehicle to the server when an abnormal lane line is included in both the lane matching information for the primary vehicle and the lane matching information for the other vehicle.

[0190] Establishing a driving strategy may include: when it is determined that reaching the destination is impossible due to an abnormal lane line according to the current driving strategy, an avoidance driving strategy is established to avoid driving on the abnormal lane line.

[0191] Establishing a driving strategy may include: when it is determined that the destination cannot be reached due to an abnormal lane marking, the automatic driving system is deactivated.

[0192] As used in this disclosure, the term "unit" can refer to a software component, a hardware component, or a combination of both, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC), and a "unit" performs certain functions. However, a "unit" is not limited to software or hardware. A "unit" can be configured to reside in an addressable memory medium or can be configured to reproduce one or more processors. Thus, for example, a "unit" includes components such as software components, object-oriented software components, class components, and task components, and includes processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided in components and "units" can be combined into a smaller number of components and "units," or can be further divided into additional components and "units." Furthermore, components and "units" can be implemented as one or more CPUs in a playback device or a secure multimedia card.

[0193] Using the vehicle control device and method according to this disclosure, stable driving can be maintained even when the traffic lane marking information identified by the autonomous vehicle does not match the traffic lane marking information in the HD map.

[0194] In addition, it can diagnose the causes of mismatches in traffic lane marking information and generate alternative routes.

[0195] In addition, it can maximize driving stability and minimize the number of times autonomous driving is released.

[0196] In this way, traffic congestion can be alleviated and the marketability of autonomous driving can be improved.

[0197] Although one or more exemplary embodiments of this disclosure have been described above, it should be understood that various changes and modifications can be made to this disclosure by those skilled in the art without departing from the spirit and scope of this disclosure as set forth in the appended claims.

Claims

1. A vehicle control device, comprising: Multiple processors, including a first processor, a second processor, and a third processor; as well as Memory, storing at least one instruction The at least one instruction is configured to, when executed by the first processor communicating with the memory, cause the vehicle control device to: Based on object recognition information received from the vehicle's sensors, the first traffic lane marking information is determined; and Based on map information received from the vehicle's navigation system, the second traffic lane marking information is determined. Wherein, the at least one instruction is configured to, when executed by the second processor communicating with the memory, further cause the vehicle control device to: Combined traffic lane marking information for the vehicle is generated by comparing the first traffic lane marking information with the second traffic lane marking information, wherein the combined traffic lane marking information indicates at least one mismatched traffic lane marking, and for at least one of the mismatched traffic lane markings, the first traffic lane marking information does not match the second traffic lane marking information. Wherein, the at least one instruction is configured to, when executed by the third processor communicating with the memory, further cause the vehicle control device to: The vehicle's driving operation is controlled based on the combined traffic lane marking information.

2. The vehicle control device according to claim 1, wherein, The at least one instruction is configured to, when executed by the second processor communicating with the memory, further cause the vehicle control device to: At least one of the following is determined by comparing the coordinates, slope, and angle of the first and second traffic lane marking information:

3. The vehicle control device according to claim 1, wherein, The at least one instruction is configured to, when executed by the third processor communicating with the memory, cause the vehicle control device to control the driving operation of the vehicle in the following manner: Based on the determination that driving in a traffic lane associated with the mismatched traffic lane marking would render the vehicle's destination unreachable, the system controls the vehicle to avoid driving in the traffic lane associated with the mismatched traffic lane marking.

4. The vehicle control device according to claim 3, wherein, The at least one instruction is configured to, when executed by the third processor communicating with the memory, cause the vehicle control device to control the driving operation of the vehicle in the following manner: Based on the determination that the vehicle's destination is unreachable by avoiding driving in the traffic lane associated with the mismatched traffic lane markings, the vehicle's autonomous driving operation is deactivated or the autonomous driving level of the vehicle's autonomous driving operation is changed.

5. The vehicle control device according to claim 1, wherein, The at least one instruction is configured to, when executed by the third processor communicating with the memory, cause the vehicle control device to control the driving operation of the vehicle in the following manner: The combined traffic lane marking information generated by the vehicle control device is compared with the second combined traffic lane marking information generated by the second vehicle.

6. The vehicle control device according to claim 5, wherein, The at least one instruction is configured to, when executed by the third processor communicating with the memory, cause the vehicle control device to control the driving operation of the vehicle in the following manner: Based on the fact that the mismatched traffic lane markings were not identified as mismatched in the second combined traffic lane marking information, it is determined that the combined traffic lane marking information generated by the vehicle is inaccurate.

7. The vehicle control device according to claim 6, wherein, The at least one instruction is configured to, when executed by the third processor communicating with the memory, cause the vehicle control device to control the driving operation of the vehicle in the following manner: Based on the determination that the combined traffic lane marking information generated by the vehicle is inaccurate, the vehicle's autonomous driving operation is deactivated or the autonomous driving level of the vehicle's autonomous driving operation is changed.

8. The vehicle control device according to claim 5, wherein, The at least one instruction is configured to, when executed by the third processor communicating with the memory, cause the vehicle control device to control the driving operation of the vehicle in the following manner: Based on the mismatched traffic lane markings being identified as mismatched in the second combined traffic lane marking information, the combined traffic lane marking information generated by the vehicle and the second combined traffic lane marking information generated by the second vehicle are sent to the server.

9. The vehicle control device according to claim 8, wherein, The at least one instruction is configured to, when executed by the third processor communicating with the memory, cause the vehicle control device to control the driving operation of the vehicle in the following manner: Based on the determination that driving in a traffic lane associated with the mismatched traffic lane marking would render the vehicle's destination unreachable, the system controls the vehicle to avoid driving in the traffic lane associated with the mismatched traffic lane marking.

10. The vehicle control device according to claim 9, wherein, The at least one instruction is configured to, when executed by the third processor communicating with the memory, cause the vehicle control device to control the driving operation of the vehicle in the following manner: Based on the determination that the vehicle's destination is unreachable by avoiding driving in the traffic lane associated with the mismatched traffic lane markings, the vehicle's autonomous driving operation is deactivated or the autonomous driving level of the vehicle's autonomous driving operation is changed.

11. A method performed by a device in a vehicle, the method comprising: Based on object recognition information received from the vehicle's sensors, first traffic lane marking information is determined; Based on map information received from the vehicle's navigation system, the second traffic lane marking information is determined; Combined traffic lane marking information for the vehicle is generated by comparing the first traffic lane marking information with the second traffic lane marking information, wherein the combined traffic lane marking information indicates at least one mismatched traffic lane marking, and for at least one said mismatched traffic lane marking, the first traffic lane marking information and the second traffic lane marking information do not match; and The vehicle's driving operation is controlled based on the combined traffic lane marking information.

12. The method of claim 11, further comprising: At least one of the following is determined by comparing the coordinates, slope, and angle of the first and second traffic lane marking information:

13. The method according to claim 11, wherein, The driving operation of the vehicle includes: Based on the determination that driving in a traffic lane associated with the mismatched traffic lane marking would render the vehicle's destination unreachable, the system controls the vehicle to avoid driving in the traffic lane associated with the mismatched traffic lane marking.

14. The method according to claim 13, wherein, The driving operation of the vehicle includes: Based on the determination that the vehicle's destination is unreachable by avoiding driving in the traffic lane associated with the mismatched traffic lane markings, the vehicle's autonomous driving operation is deactivated or the autonomous driving level of the vehicle's autonomous driving operation is changed.

15. The method according to claim 11, wherein, The driving operation of the vehicle includes: The combined traffic lane marking information generated by the device of the vehicle is compared with the second combined traffic lane marking information generated by the second vehicle.

16. The method according to claim 15, wherein, The driving operation of the vehicle includes: Based on the fact that the mismatched traffic lane markings were not identified as mismatched in the second combined traffic lane marking information, it is determined that the combined traffic lane marking information generated by the vehicle is inaccurate.

17. The method according to claim 16, wherein, The driving operation of the vehicle includes: Based on the determination that the combined traffic lane marking information generated by the vehicle is inaccurate, the vehicle's autonomous driving operation is deactivated or the autonomous driving level of the vehicle's autonomous driving operation is changed.

18. The method according to claim 15, wherein, The driving operation of the vehicle includes: Based on the mismatched traffic lane markings being identified as mismatched in the second combined traffic lane marking information, the combined traffic lane marking information generated by the vehicle and the second combined traffic lane marking information generated by the second vehicle are sent to the server.

19. The method according to claim 18, wherein, The driving operation of the vehicle includes: Based on the determination that driving in a traffic lane associated with the mismatched traffic lane marking would render the vehicle's destination unreachable, the system controls the vehicle to avoid driving in the traffic lane associated with the mismatched traffic lane marking.

20. The method according to claim 19, wherein, The driving operation of the vehicle includes: Based on the determination that the vehicle's destination is unreachable by avoiding driving in the traffic lane associated with the mismatched traffic lane markings, the vehicle's autonomous driving operation is deactivated or the autonomous driving level of the vehicle's autonomous driving operation is changed.

Images