Method for Controlling an Adaptation Module of an Autonomous Driving Foundation Model Based on Road Type and Autonomous Driving Apparatus Using the Same

Abstract

A method for controlling an adaptive module of a road type-based autonomous driving foundation model according to one technical aspect of the present invention is performed in an autonomous driving device and comprises: a step of acquiring driving environment information of a vehicle; a step of generating road type-based control information based on the driving environment information; a step of selecting or mixing-controlling at least one adaptive module coupled to a common autonomous driving foundation model based on the control information; and a step of generating an autonomous driving output by reflecting the selected or mixed-controlled at least one adaptive module in the common autonomous driving foundation model.

Description

Technology Field

[0001] The present invention relates to a technology for generating autonomous driving output corresponding to changes in road type by controlling an adaptive module coupled to a common autonomous driving foundation model according to the driving environment of an autonomous vehicle. More specifically, it relates to a technology for controlling an adaptive module of a road type-based autonomous driving foundation model that can maintain consistency of the model structure and driving stability while reflecting the characteristics of each road type. Background Technology

[0002] Recently, in the field of autonomous driving technology, large-scale AI-based driving models are being utilized to perform vehicle perception, prediction, and driving planning by integrally processing various input information, including cameras, LiDAR, radar, and map data. In particular, with the proposal of autonomous driving foundation models designed to comprehensively process diverse driving scenarios within a single learning structure, research is actively underway to secure universal performance capabilities for urban areas, highways, intersections, and complex traffic environments.

[0003] However, in real-world driving environments, road structure, lane geometry, the distribution of surrounding objects, speed limits, and vehicle interaction patterns can vary significantly depending on the road type; therefore, there are limitations to maintaining the same level of driving performance across all environments using only a single model structure. For example, while responsiveness to pedestrians and traffic signals is critical in urban areas, lane keeping stability and the ability to handle merging sections are relatively more important on highways.

[0004] To compensate for such differences, methods may be applied such as separately training multiple driving models specialized for specific road environments or selectively using separate control models in specific situations. However, in this case, inference delays may occur due to structural differences between models, or vehicle control stability may be compromised due to reduced output continuity during model switching while driving; furthermore, maintenance and management costs for multiple models may increase.

[0005] Therefore, more efficient control technology is required that can respond to various road types while maintaining a common driving model structure, and ensure the continuity and stability of autonomous driving output in response to changes in the driving environment. Prior art literature

[0006] Korean Patent Publication No. 10-270487 (Registration Date: 2024.09.04) The problem to be solved

[0007] The present invention aims to solve the aforementioned problems by mitigating the issue where recognition, prediction, and driving control performance may be degraded in specific road environments when all driving environments are consistently processed using only a single autonomous driving model, despite the fact that driving characteristics required for different road types differ. It also aims to improve the adaptability and performance of autonomous driving output generation corresponding to road types by generating road type-based control information based on driving environment information and enabling the selection or mixed control of at least one adaptive module combined with a common autonomous driving foundation model.

[0008] In addition, another objective is to mitigate the problem where, in conventional technology, when the road type changes—such as between general roads and expressways—separate driving models or policies are immediately replaced or driving modes are switched relying solely on map information, which can lead to abrupt changes in output and consequently degrade driving stability in road transition sections. Furthermore, by providing a structure that selects or mix-controls an adaptive module coupled to a common autonomous driving foundation model according to the road type, the invention aims to improve the continuity of autonomous driving output and driving stability even in road type transition sections.

[0009] In addition, another objective is to mitigate the problem of increased system burden in terms of model storage, deployment, updating, and verification when operating separate large-scale autonomous driving models for each road type to compensate for performance differences based on road environments in conventional technology, and to improve the efficiency and scalability of model operation while responding to various road types by providing a structure that selectively combines or mixes adaptation modules while maintaining a common autonomous driving foundation model.

[0010] In addition, another objective is to mitigate the problem in conventional technology where there is insufficient control adjustment system to reflect how changes in the configuration of the driving model affect actual vehicle behavior, output continuity, or model reliability when such changes occur, and to enable more consistent and stable autonomous driving control in response to changes in the driving environment based on a structure that generates autonomous driving output by reflecting at least one selectively or mixedly controlled adaptive module into a common autonomous driving foundation model.

[0011] In addition, another objective is to alleviate the problem in conventional technology where there are limitations in flexibly determining an autonomous driving model configuration suitable for the current driving situation by comprehensively reflecting various driving environment factors such as road structure, object distribution, speed limit, merging pattern, and map information, even though they interact in a complex manner, and to improve responsiveness to complex driving environment changes and flexibility of autonomous driving control by generating road type-based control information based on driving environment information and using it to select or mix-control an adaptive module.

[0012] In addition, another objective of the present invention is to improve stability, efficiency, and scalability in various driving environments by ensuring environmental adaptability while maintaining consistency of a common model structure during the autonomous driving control process for responding to changes in road type.

[0013] However, the problems intended to be solved in this disclosure are not limited to those mentioned above, and may be expanded in various ways without departing from the spirit and scope of this disclosure. means of solving the problem

[0014] One technical aspect of the present invention provides a method for controlling an adaptive module of a road type-based autonomous driving foundation model. The method is performed in an autonomous driving device and comprises: acquiring driving environment information of a vehicle; generating road type-based control information based on the driving environment information; selecting or blending control at least one adaptive module coupled to a common autonomous driving foundation model based on the control information; and generating an autonomous driving output by reflecting the selected or blended at least one adaptive module in the common autonomous driving foundation model.

[0015] In one embodiment, the step of acquiring driving environment information of the vehicle may include: collecting sensor information from at least one sensor provided in the vehicle; receiving map information; and collecting driving state information including the speed, acceleration, and steering state of the vehicle.

[0016] In one embodiment, the step of generating control information based on the road type may include: a step of analyzing road structure information, object distribution information, speed limit information, and merging pattern information included in the driving environment information; and a step of generating reliability information for each of the plurality of road types based on the analysis results.

[0017] In one embodiment, the step of selecting or mixing control of at least one adaptation module may include: a step of calculating the application ratio of each of the plurality of adaptation modules based on the reliability information; and a step of selecting at least one of the plurality of adaptation modules or mixing the plurality of adaptation modules according to the application ratio.

[0018] In one embodiment, the step of generating the autonomous driving output may include: a step of reflecting the output or parameter of the plurality of adaptation modules into the common autonomous driving foundation model according to the application ratio; and a step of generating an autonomous driving output including a driving trajectory or vehicle control command based on the reflection result.

[0019] In one embodiment, the step of selecting or mixing control of the at least one adaptive module may include: a step of setting a switching gate condition to suppress output discontinuity due to a change in the adaptive module; and a step of controlling whether to change the at least one adaptive module or the adjustment speed of the application ratio of a plurality of adaptive modules based on the switching gate condition; wherein the switching gate condition may include at least one of a hysteresis condition, a minimum holding time condition, and a switching cost condition.

[0020] In one embodiment, the step of selecting or mixing control of the at least one adaptation module may include: a step of detecting a transition section where the road type changes; and a step of gradually decreasing the application rate of the first adaptation module and gradually increasing the application rate of the second adaptation module in the transition section.

[0021] In one embodiment, the method may further include the steps of: generating stability information or uncertainty information based on the autonomous driving output after generating the autonomous driving output; readjusting the selection result of at least one adaptation module or the application ratio of a plurality of adaptation modules based on the stability information or uncertainty information; and rolling back to a previous adaptation module configuration or a pre-set conservative configuration if the stability information or uncertainty information satisfies a preset criterion; and the stability information or uncertainty information may include at least one of a steering change rate, an acceleration change rate, an output discontinuity, an object recognition reliability, and an anomaly distribution index.

[0022] Another technical aspect of the present invention proposes an autonomous driving device. The autonomous driving device comprises at least one processor and a memory for storing instructions. When the instructions are executed by the at least one processor, the at least one processor is configured to perform the following operations: acquiring driving environment information of a vehicle; generating road type-based control information based on the driving environment information; selecting or blending control at least one adaptive module coupled to a common autonomous driving foundation model based on the control information; and generating an autonomous driving output by reflecting the selected or blended adaptive module into the common autonomous driving foundation model. Effects of the invention

[0023] According to various embodiments of the present invention, road type-based control information is generated based on vehicle driving environment information, and at least one adaptive module coupled to a common autonomous driving foundation model is configured to select or mix-control, thereby allowing for more flexible reflection of differences in driving characteristics required for different road types. Accordingly, compared to the case where various driving environments are processed collectively using only a single autonomous driving model, the adaptability of autonomous driving output generation corresponding to road types and driving performance can be improved.

[0024] Furthermore, according to various embodiments of the present invention, by configuring to selectively combine adaptation modules or mix and reflect multiple adaptation modules while maintaining a common autonomous driving foundation model, the system burden associated with model storage, distribution, updating, and verification can be reduced compared to the case where separate large-scale autonomous driving models are operated for each road type. Accordingly, the efficiency and scalability of model operation can be improved while ensuring responsiveness to various road environments.

[0025] Furthermore, according to various embodiments of the present invention, by configuring the adaptive module to select or mix control in situations where the road type changes, abrupt changes in autonomous driving output can be suppressed even when the driving environment changes continuously, such as in the transition section between a general road and an expressway. Accordingly, compared to the conventional method of immediately replacing the driving model or policy, the continuity of output and the stability of vehicle behavior can be improved.

[0026] Furthermore, according to various embodiments of the present invention, at least one adaptive module that is selected or mixed-controlled is configured to be reflected in a common autonomous driving foundation model to generate autonomous driving output, thereby allowing the model configuration corresponding to changes in road type to be linked to the actual inference result. Accordingly, unlike cases where the driving environment is merely classified, changes in the driving environment are substantially reflected in the process of generating autonomous driving output, thereby improving the consistency and effectiveness of autonomous driving control.

[0027] Furthermore, according to various embodiments of the present invention, by configuring to generate road type-based control information by comprehensively reflecting multiple driving environment elements such as road structure, object distribution, speed limit, merging pattern, and map information, the responsiveness to environmental changes and the robustness of control judgment can be improved compared to cases where the autonomous driving model configuration is modified based on reliance on only partial information. Accordingly, more stable autonomous driving control can be performed even in the presence of map errors or uncertainty in individual sensor information.

[0028] In addition, according to various embodiments of the present invention, in an embodiment configured to readjust or roll back the application configuration of an adaptive module by reflecting the state after the generation of autonomous driving output, if output discontinuity, instability of vehicle behavior, or deterioration of model reliability is detected during driving, this can be reflected in subsequent control. Accordingly, adaptability to changes in the driving environment, along with stability and reliability during actual vehicle operation, can be improved.

[0029] The effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects will be obvious to those skilled in the art to which the present invention belongs from the description below. Brief explanation of the drawing

[0030] FIG. 1 is a configuration diagram illustrating the overall configuration of an adaptive module control system of a road type-based autonomous driving foundation model according to one embodiment of the present invention. FIG. 2 is a flowchart illustrating the overall procedure of an adaptive module control method for a road type-based autonomous driving foundation model according to an embodiment of the present invention. FIG. 3 is a block diagram illustrating the detailed configuration of an autonomous driving device according to one embodiment of the present invention. FIG. 4 is a configuration diagram illustrating the adaptive module control structure of a road type-based autonomous driving foundation model according to one embodiment of the present invention from a layer perspective. FIG. 5 is a flowchart illustrating the procedure of a road type-based control information generation method according to one embodiment of the present invention. FIG. 6 is a block diagram illustrating the detailed configuration of an adaptive module control unit according to one embodiment of the present invention. FIG. 7 is a block diagram illustrating the detailed configuration of a foundation model reflection part according to one embodiment of the present invention. FIG. 8 is a block diagram illustrating the detailed configuration of a readjustment unit (160) according to one embodiment of the present invention. FIG. 9 is a flowchart illustrating the procedure of an uncertainty decomposition-based adaptive module influence limitation method according to one embodiment of the present invention. FIG. 10 is a flowchart illustrating the procedure of a method for generating an autonomous driving output based on hierarchical mixing that simultaneously reflects multiple context axes according to an embodiment of the present invention. Specific details for implementing the invention

[0031] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily practice them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0032] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, each of phrases such as A or B, at least one of A and B, at least one of A or B, A, B or C, at least one of A, B and C, and at least one of A, B, or C may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as first, second, or first or second may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., first) component is referred to as coupled or connected to another (e.g., second) component, with or without the terms functionally or communicationally, it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0033] As used in the various embodiments of this document, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0034] Various embodiments of this document may be implemented as software (e.g., a program) comprising one or more instructions stored in a storage medium (e.g., memory) readable by a machine or device. For example, the processor of the machine or device may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by a machine may be provided in the form of a non-transitory storage medium. Here, "non-transitory" simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

[0035] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0036] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0038] The meanings of the key terms used in this specification are as follows, and each term is defined from a functional perspective to aid in understanding the invention. The following definitions of terms are intended to describe embodiments of the invention and should not be interpreted as unduly limiting the scope of the invention.

[0039] "Autonomous driving device" refers to an electronic processing unit configured to collect and process driving environment information of a vehicle, generate road type-based control information based on said driving environment information, and generate autonomous driving output by selecting or mixing control of at least one adaptive module coupled to a common autonomous driving foundation model. The autonomous driving device may be implemented as a single controller mounted inside the vehicle, a computing unit distributed among multiple controllers, a high-performance vehicle computing platform, or a distributed processing structure linked to the cloud. The autonomous driving device may include at least one of a driving environment information acquisition unit, a control information generation unit, an adaptive module control unit, a foundation model reflection unit, an autonomous driving output generation unit, and a readjustment unit.

[0040] "Driving environment information" refers to information indicating the external driving environment surrounding the vehicle and the driving state of the vehicle itself, and includes a set of information used as input for road type-based control information generation and adaptive module control. Driving environment information may include sensor information acquired from cameras, lidar, radar, ultrasonic sensors, inertial sensors, satellite navigation receivers, or other vehicle-mounted sensors, as well as external information such as map information, speed limit information, road connectivity information, traffic information, weather information, time zone information, and location-based information. Additionally, driving environment information may further include vehicle state information such as vehicle speed, acceleration, steering state, and vehicle attitude.

[0041] "Road type-based control information" refers to control-related information generated based on driving environment information to correspond to the road type or driving context in which the vehicle is currently driving or is scheduled to enter. The road type-based control information may include a judgment result for at least one of a general road, an expressway, a ramp section, an interchange section, an intersection section, or a transition section; reliability information for each of multiple road types; selection conditions for an adaptation module; mixing ratio information for an adaptation module; transition condition information; or transition speed control information. The road type-based control information is not limited to simple classification results and can function as a control input directly used for subsequent adaptation module control.

[0042] "Autonomous Driving Foundation Model" refers to a basic autonomous driving inference structure that can be commonly applied to multiple road types or multiple driving contexts. The common autonomous driving foundation model may include computational structures related to sensor feature interpretation, understanding of driving context, object interaction inference, path generation, and calculation of vehicle control values, and may include basic parameters or a basic backbone structure shared across multiple road types. The common autonomous driving foundation model may be configured to allow adaptive modules to be combined selectively or in a mixed manner, and the basic structure may be maintained before and after the incorporation of adaptive modules.

[0043] "Adaptation Module" refers to an additional computational structure or additional parameter structure that is selectively combined with a common Autonomous Driving Foundation Model or a combination of multiple modules to provide inference characteristics corresponding to a specific road type or specific driving context. An adaptation module may include a general road adaptation module, a highway adaptation module, a transition section adaptation module, a weather adaptation module, a time-of-day adaptation module, or a traffic density adaptation module. An adaptation module may be implemented as a low-dimensional correction parameter structure, an adapter structure, an expert-based structure, or other partial parameter structures, and can perform inference correction corresponding to a specific environment without replacing the entire common Autonomous Driving Foundation Model.

[0044] "Selective or mixed control" refers to a control action that determines at least one adaptive module coupled to a common autonomous driving foundation model, and calculates and applies the application ratio or reflection intensity of each of multiple adaptive modules as needed. Selective or mixed control may include an action of preferentially selecting a single adaptive module, an action of applying multiple adaptive modules simultaneously, an action of gradually changing the application ratio of adaptive modules, or an action of limiting the reflection speed of adaptive modules. Selective or mixed control may be dynamically updated in response to changes in road type, entry into transition sections, reduced output stability, or increased uncertainty.

[0045] "Application Ratio" refers to a value representing the relative degree of reflection each adaptation module has on the common Autonomous Driving Foundation model when multiple adaptation modules are applied simultaneously. The application ratio may be expressed in the form of a percentage, a normalized real value, a weight value, or a scaling factor. The application ratio may be calculated or recalculated based on road type reliability, transition section characteristics, transition conditions, stability evaluation results, or uncertainty judgment results.

[0046] "Transition gate condition" refers to a criterion condition for determining whether it is permissible to change the selection result of an adaptive module or to adjust the application ratio of multiple adaptive modules. The transition gate condition may include at least one of a hysteresis condition, a minimum holding time condition, a transition cost condition, an output continuity condition, a stability condition, or an uncertainty criterion. The transition gate condition may be used as a control criterion to suppress frequent changes in the adaptive module configuration and to prevent discontinuity in autonomous driving output or instability in vehicle behavior.

[0047] "Autonomous driving output" refers to driving-related output information generated based on the reflection results of a common autonomous driving foundation model and an adaptation module. The autonomous driving output may include at least one of target path information, driving trajectory, steering control value, acceleration / deceleration control value, braking control value, lane keeping control value, distance keeping information, collision avoidance path, or risk avoidance control command. The autonomous driving output can be transmitted to a vehicle control unit and used for steering, acceleration, and braking control of the actual vehicle.

[0048] "Stability information" refers to information indicating whether autonomous driving output is temporally continuous and stable from the perspective of vehicle control. Stability information may include the rate of change in steering, the rate of change in acceleration, the rate of change in braking commands, jerk, the amount of change in trajectory curvature, the degree of output discontinuity, the variability of lane centering error, or the abruptness of changes in vehicle behavior. Stability information may be used as a criterion for determining whether to readjust the configuration of the adaptive module application or to determine the need for a rollback.

[0049] "Uncertainty information" refers to information indicating how reliable the perception, prediction, and planning results are in relation to the current driving environment during the autonomous driving output or its generation process. Uncertainty information may include object recognition reliability, prediction result variance, deviation between path candidates, anomaly distribution indicators, environmental suitability degradation indicators, or model output reliability information. Uncertainty information can be used in conjunction with stability information to induce the recalculation, partial restriction, or rollback of the adaptation module configuration.

[0050] "Recalibration" refers to a control action that recalculates or modifies the selection result of the current adaptation module or the application ratio of multiple adaptation modules based on stability information or uncertainty information regarding autonomous driving output. Recalibration may include actions that increase or decrease the application ratio of a specific adaptation module, actions that change the ratio distribution among multiple adaptation modules, actions that limit the growth rate of a specific adaptation module, or actions that modify only the reflection intensity at a specific layer.

[0051] "Rollback" refers to a control action that restores the current adaptive module configuration to a previous adaptive module configuration or a preset conservative configuration when it is difficult to ensure stability or reliability solely through the readjustment of the adaptive module. Rollback may include an action that returns the entire adaptive module configuration to the previous stable state, a partial restoration action that returns only the application ratio of a specific adaptive module to a previous value, or an action that switches to a default configuration centered on a common autonomous driving foundation model.

[0052] A "transition section" refers to an intermediate section where a change occurs from one road type to another, or a section where characteristics of multiple road types appear simultaneously. Transition sections may include ramp sections entering an expressway from a general road, exit sections exiting an expressway to a general road, interchange sections, lane merging sections, or sections where multiple road connection structures overlap. In transition sections, multiple adaptation modules may be applied simultaneously, or the application ratio of adaptation modules may be gradually adjusted.

[0053] "Hierarchical blending" refers to a control method that blends adaptation modules corresponding to multiple context axes into a hierarchical relationship or a stepwise reflection structure based on the priority, role, or output characteristics of each axis, rather than simply combining them in parallel at the same level. Hierarchical blending may include a higher-level basic configuration and a lower-level correction configuration among adaptation modules corresponding to road type, weather, time zone, traffic density, or other driving context axes, and can be implemented by applying different blending weights by hierarchy or by output item.

[0055] FIG. 1 is a configuration diagram illustrating the overall configuration of an adaptive module control system of a road type-based autonomous driving foundation model according to one embodiment of the present invention.

[0056] The system according to FIG. 1 may be configured to include a vehicle (10), a sensor (11) provided in the vehicle (10), an autonomous driving device (100), an external information providing device (20), and a vehicle control device (200). The system according to FIG. 1 may be configured to collect information regarding the surrounding environment and driving state of the vehicle, generate road type-based control information based on the collected information, select or mix-control at least one adaptive module combined to a common autonomous driving foundation model accordingly, and generate an autonomous driving output based on the reflection result to control the actual driving of the vehicle.

[0057] The above road type-based control information is not limited to simple road type identification results, but can function as a control standard regarding the selection target of the adaptation module, the application ratio of multiple adaptation modules, the switching time and switching speed of the adaptation module. Accordingly, the autonomous driving device (100) can operate as a control hub connecting driving environment information and adaptation module control from the perspective of the entire system.

[0058] The vehicle (10) is a moving body to which an embodiment of the present invention is applied, and may be an automobile, commercial vehicle, shuttle, logistics vehicle, or a similar driving body capable of driving on general roads, highways, ramp sections, and other complex road environments. The vehicle (10) may perform driving control in conjunction with an autonomous driving device (100) and a vehicle control device (200), and state information of the vehicle (10), such as speed, acceleration, steering state, and vehicle behavior information, may be provided to the autonomous driving device (100).

[0059] A sensor (11) is a component mounted on a vehicle (10) for detecting the external environment around the vehicle and the state of the vehicle itself. The sensor (11) may include at least one of a camera, lidar, radar, ultrasonic sensor, satellite navigation receiver, inertial sensor, and other sensors for perceiving the vehicle driving environment. The sensor (11) can detect information regarding lane shape, road structure, location and type of surrounding objects, drivable area, presence of merging sections, road boundaries, speed limit information, and dynamic conditions around the vehicle. Additionally, information obtained through the sensor (11) can be transmitted to an autonomous driving device (100) and used as part of driving environment information.

[0060] The external information providing device (20) is a component that provides external information to the autonomous driving device (100) and can provide map information, road connection relationship information, traffic information, construction section information, weather information, time zone information, or location-based auxiliary information. The external information providing device (20) can be implemented as a remote server, a cloud system, a road infrastructure system, or an information providing platform outside the vehicle, and can be connected to the autonomous driving device (100) via wired or wireless communication. The information provided from the external information providing device (20) can be used together with information obtained from the sensor (11) to improve the accuracy and robustness of the determination of whether the environment in which the vehicle is currently driving is a general road, a highway, a transition section, or other complex section.

[0061] In one embodiment, the autonomous driving device (100) can integrate real-time detection information obtained from a sensor (11) and map or road connection information provided by an external information provider (20) into a single driving context by aligning them based on the same location and time criteria. Accordingly, control that reflects both the currently observed environment and the structure of the planned section ahead can be enabled.

[0062] The autonomous driving device (100) is a core component of the present invention and may be configured to generate an autonomous driving output by comprehensively processing driving environment information provided from a vehicle (10) and a sensor (11) and external information provided from an external information providing device (20). The autonomous driving device (100) may generate road type-based control information based on the driving environment information, and may select at least one adaptive module combined to a common autonomous driving foundation model or control a plurality of adaptive modules to mix according to the road type-based control information. Additionally, the autonomous driving device (100) may generate an autonomous driving output by reflecting the selected or mixed-controlled adaptive modules into a common autonomous driving foundation model.

[0063] Here, the common autonomous driving foundation model may include a basic inference structure that is commonly applied to multiple road types, and the adaptation module may be an additional structure that provides specialized inference characteristics corresponding to a specific road type or a specific driving context. Accordingly, the autonomous driving device (100) can flexibly respond to changes in the driving environment by adjusting the selection state or mixed configuration of the adaptation module instead of replacing the entire foundation model when the road type changes. For example, when the vehicle is driving on a general road, an adaptation module corresponding to the characteristics of the general road may be prioritized, and when driving on a highway, an adaptation module corresponding to the characteristics of the highway may be prioritized, and in transition sections such as ramp sections or interchanges, multiple adaptation modules may be mixed to mitigate abrupt changes in autonomous driving output.

[0064] Additionally, the autonomous driving device (100) can generate path information, steering control values, acceleration / deceleration control values, lane keeping control values, and control information for collision avoidance based on the generated autonomous driving output. The autonomous driving output can be transmitted to a vehicle control device (200), and the vehicle control device (200) can control the actual drive system of the vehicle (10) based on the autonomous driving output received from the autonomous driving device (100).

[0065] The vehicle control unit (200) may include a steering unit, braking unit, driving unit, or a controller associated therewith of the vehicle (10), and may convert the autonomous driving output generated by the autonomous driving unit (100) into an actual vehicle control signal or execute it directly. The vehicle control unit (200) may perform steering angle adjustment, acceleration and deceleration control, lane keeping operation, forward obstacle avoidance operation, and target path following operation. In addition, the actual control results or changes in vehicle behavior performed by the vehicle control unit (200) may be reflected back into the vehicle (10) and the autonomous driving unit (100), and thereby may be utilized for evaluating output stability during driving or readjusting the adaptive module.

[0066] In one embodiment, the autonomous driving device (100) can generate road type-based control information by considering together information obtained from the sensor (11) and information received from the external information provider (20). Accordingly, compared to cases where only map information is relied upon, or conversely, cases where only sensor information is relied upon, the accuracy of road type determination and responsiveness to environmental changes can be improved. For example, if a road is indicated as a highway entrance on the map information but the lane structure or merging pattern is not clear on the actual sensor information, the autonomous driving device (100) can determine it as a transition section through a combination of both pieces of information and adjust the mixing reflection ratio of the adaptation module.

[0067] In another embodiment, the autonomous driving device (100) may readjust the configuration of the adaptive module by reflecting the vehicle behavior or output characteristics after the generation of the autonomous driving output. For example, if the combination of adaptive modules selected on a specific road type causes a sudden increase in the rate of change of steering, discontinuity in acceleration and deceleration, or a decrease in object recognition reliability during actual driving, the autonomous driving device (100) may perform control to adjust the degree of reflection of the adaptive module or restore it to the previous configuration. This feedback structure may be described in more detail in the drawings below.

[0068] According to the system illustrated in FIG. 1, an embodiment of the present invention can provide an overall system structure in which a vehicle (10), a sensor (11), an external information providing device (20), an autonomous driving device (100), and a vehicle control device (200) are organically linked. In particular, by configuring to select or mix-control an adaptive module based on road type-based control information while maintaining a common autonomous driving foundation model, it is possible to secure both responsiveness to changes in road type and continuity of autonomous driving output. Furthermore, compared to a structure in which separate large-scale autonomous driving models are operated for each road type, the efficiency and scalability of model operation can be improved, and stability and reliability in the actual vehicle control process can be enhanced.

[0070] FIG. 2 is a flowchart illustrating the overall procedure of an adaptive module control method for a road type-based autonomous driving foundation model according to an embodiment of the present invention.

[0071] The procedure according to FIG. 2 may include acquiring driving environment information of a vehicle, generating road type-based control information based on the driving environment information, selecting or blending control at least one adaptive module combined to a common autonomous driving foundation model according to the control information, generating an autonomous driving output by reflecting the selected or blended adaptive module in the common autonomous driving foundation model, and then readjusting or rolling back the application configuration of the adaptive module based on the generated autonomous driving output. According to this procedure, one embodiment of the present invention can simultaneously achieve autonomous driving control adapting to changes in road type and secure output stability.

[0072] Referring to FIG. 2, in step S210, the autonomous driving device (100) can acquire driving environment information of the vehicle. Here, the driving environment information may include external environment information and vehicle status information collected from a sensor (11) equipped in the vehicle (10), and may further include map information, traffic information, construction zone information, speed limit information, and location-based auxiliary information provided by an external information providing device (20). The autonomous driving device (100) can receive camera images, LiDAR point clouds, radar reflection information, inertial information, vehicle speed, and steering status information to form a set of input information representing the current driving situation. For example, the autonomous driving device (100) can acquire different driving environment information, distinguishing between a situation where pedestrians and traffic lights are frequently detected in an urban area and a situation where the lane structure is simple and the vehicle speed is relatively high in a highway area.

[0073] In one embodiment of step S210, the autonomous driving device (100) may collect sensor information from at least one sensor equipped in the vehicle, receive map information from an external information provider (20), and collect driving state information including the vehicle's speed, acceleration, and steering state. By combining the sensor information, map information, and driving state information in this way to form driving environment information, the accuracy of road type determination and responsiveness to environmental changes can be improved. For example, even if the map information indicates proximity to a highway entrance ramp, if the forward lane structure in the sensor information still maintains the structure of an intersection on a general road, the autonomous driving device (100) may consider the difference between the two pieces of information together in a subsequent step. In this case, the driving environment information may be configured as a composite input structure that includes all of the sensor information, map information, and driving state information.

[0074] In step S220, the autonomous driving device (100) can generate road type-based control information based on the driving environment information. The autonomous driving device (100) can determine which type the current vehicle belongs to among a general road, a highway, a ramp section, an interchange section, or other complex road section by analyzing various elements included in the driving environment information. Here, the road type-based control information is not limited to simple road type identification results, but may include information used for selecting an adaptation module, calculating a mixing ratio, determining transition timing, and controlling output stability in subsequent steps. For example, when characteristics similar to a general road and characteristics similar to a highway are observed simultaneously, the autonomous driving device (100) can generate control information corresponding to the transition section so that multiple adaptation modules are reflected together in subsequent steps.

[0075] In one embodiment of step S220, the autonomous driving device (100) may analyze road structure information, object distribution information, speed limit information, and merging pattern information included in the driving environment information. The road structure information may include the number of lanes, lane curvature, the presence or absence of a median strip, the shape of the intersection, and the connection structure of the entrance and exit ramps. The object distribution information may include the density and spatial distribution of pedestrians, motorcycles, passenger cars, large vehicles, and stationary objects. The speed limit information may include the statutory speed limit, recommended speed, or speed change pattern of the current section. The merging pattern information may include whether lanes are merged, the shape of the entrance ramp, the direction of merging into the main road, and the flow characteristics of surrounding vehicles. Based on the analysis results, the autonomous driving device (100) may generate reliability information for each of the plurality of road types. The reliability information may be calculated independently for each of the general road, the expressway, the ramp section, and other transition sections. For example, if the straight section is long, the speed limit is high, and the pedestrian detection frequency is very low, the reliability for the highway type can be calculated highly, and if the detection frequency of traffic lights and crosswalks is high and low-speed driving objects are mixed, the reliability for the general road type can be calculated highly.

[0076] In another embodiment of step S220, when multiple road type features are observed simultaneously, such as in a transition section, the autonomous driving device (100) may maintain reliability information corresponding to multiple road types simultaneously instead of performing only a mutually exclusive single determination. According to this structure, control information suitable for continuous changes rather than abrupt changes in road types can be generated, which is advantageous for gradually switching or mixing adaptation modules in subsequent steps.

[0077] In step S230, the autonomous driving device (100) can select or combine control at least one adaptive module combined to a common autonomous driving foundation model based on the road type-based control information. The autonomous driving device (100) can select an adaptive module suitable for the current driving environment among a plurality of adaptive modules, and can perform combined control that incorporates a plurality of adaptive modules simultaneously as needed. Here, the adaptive module may be an additional structure that provides specialized reasoning characteristics corresponding to a specific road type or a specific driving context, and may be implemented as a general road adaptive module, a highway adaptive module, a transition section adaptive module, or other driving context-corresponding adaptive modules. For example, on a general road, an adaptive module with enhanced response to pedestrians, signal systems, and complex intersection flows may be selected first, and on a highway, an adaptive module advantageous for lane keeping stability, merging response, and high-speed driving stability may be selected first.

[0078] In one embodiment of step S230, the autonomous driving device (100) may calculate the application ratio of each of the plurality of adaptation modules based on the reliability information. The autonomous driving device (100) may select at least one of the plurality of adaptation modules or mix the plurality of adaptation modules according to the calculated application ratio. For example, if the reliability of the general road type is evaluated as 0.7 and the reliability of the highway type is evaluated as 0.3, the autonomous driving device (100) may create an application configuration in which the degree of reflection of the general road adaptation module is relatively large and the degree of reflection of the highway adaptation module is relatively small. In cases where transition characteristics are strong, such as in ramp sections, the general road adaptation module and the highway adaptation module may be reflected together, and a transition section adaptation module may be additionally reflected.

[0079] In another embodiment of step S230, the autonomous driving device (100) may set a switching gate condition to suppress output discontinuity resulting from changes in the adaptive module. The switching gate condition may include at least one of a hysteresis condition, a minimum holding time condition, and a switching cost condition. The hysteresis condition may suppress excessive repetition of switching of the adaptive module when the road type determination result changes frequently over a short period. The minimum holding time condition may reduce unnecessary oscillatory control by ensuring that once a selected adaptive module configuration is maintained for a certain period of time or a certain distance. The switching cost condition may be set to delay or mitigate switching when inference delay, an increase in the rate of change of output, or a decrease in stability is expected due to changes in the adaptive module. Accordingly, the autonomous driving device (100) may limit or divide in stages the speed at which the application rate of each of the multiple adaptive modules changes, as well as whether at least one adaptive module is changed.

[0080] In another embodiment of step S230, the autonomous driving device (100) may detect a transition section where the road type changes, and in said transition section, gradually decrease the application rate of the first adaptation module and gradually increase the application rate of the second adaptation module. For example, when moving from a general road to a highway entrance ramp, the application rate of the general road adaptation module may be gradually decreased while the application rate of the highway adaptation module may be gradually increased. Such gradual adjustment can reduce abrupt interruptions in the model output and allow the steering and acceleration / deceleration control of the vehicle to continue more naturally.

[0081] In step S240, the autonomous driving device (100) may reflect at least one selected or mixed-controlled adaptive module into the common autonomous driving foundation model. In this case, the selection result or mixed configuration of the adaptive module generated in step S230 may be converted into a reflection form by hierarchy or computation path in step S240, so that the adaptive module control result may be directly reflected in the actual inference structure.

[0082] Here, the reflection may be a method of combining a selected single adaptation module itself into a part of a layer or part of a computation path of a common autonomous driving foundation model, or a method of simultaneously combining each of a plurality of mixed-control adaptation modules into the common autonomous driving foundation model according to a preset application ratio. The autonomous driving device (100) may reflect the output or parameters of the adaptation module into the common autonomous driving foundation model according to the application configuration of the adaptation module. Here, the common autonomous driving foundation model may include a basic inference structure that is commonly applied to a plurality of road types, and the adaptation module may be reflected into a part of a layer, a set of parameters, or a part of a computation path of the common model to enhance characteristics corresponding to the road type or driving context. For example, the autonomous driving device (100) may form a final inference configuration by injecting the weighted reflection results calculated from a plurality of adaptation modules into a specific layer of the common model or by reflecting them in the form of adjusting some parameters.

[0083] In one embodiment of step S240, the autonomous driving device (100) may reflect the output of each of the plurality of adaptation modules or each parameter of each of the adaptation modules into the common autonomous driving foundation model according to the application ratio. For example, the autonomous driving device (100) may combine the correction parameters calculated by the general road adaptation module and the highway adaptation module, respectively, by weight according to the application ratio, and insert the result into the common autonomous driving foundation model. As another example, the autonomous driving device (100) may selectively reflect only the additional parameters of the adaptation modules while maintaining the basic parameters of the common model. According to this structure, it becomes possible to adjust the inference characteristics corresponding to changes in road type without replacing the entire model.

[0084] In step S250, the autonomous driving device (100) can generate an autonomous driving output based on the reflection results. The autonomous driving device (100) can generate an autonomous driving output including a vehicle driving trajectory or vehicle control commands using the reflection results of a common autonomous driving foundation model and an adaptation module. The autonomous driving output may include target path information, steering control values, acceleration / deceleration control values, lane keeping control values, distance keeping information, collision avoidance control commands, or equivalent driving-related information. For example, in highway sections, a driving trajectory emphasizing lane centering and forward vehicle following stability may be generated, and in general road sections, a more conservative driving trajectory considering pedestrian and intersection object avoidance may be generated. In transition sections, an autonomous driving output in a form that combines the two tendencies may be generated.

[0085] In step S260, the autonomous driving device (100) may readjust or roll back the application configuration of the adaptive module based on the autonomous driving output. The autonomous driving device (100) may separately collect the output state after the generation of the autonomous driving output to evaluate the temporal continuity of the output, the rate of change of control, and the prediction stability. The autonomous driving device (100) may collect the output state after the generation of the autonomous driving output and generate stability information or uncertainty information for the output. The stability information may include the rate of change of steering, the rate of change of acceleration, output discontinuity, or the degree of abrupt change in vehicle behavior. The uncertainty information may include object recognition reliability, the variance of the prediction result, an anomaly distribution indicator, or an indicator indicating a deterioration in environmental suitability. Based on the stability information or uncertainty information, the autonomous driving device (100) may readjust the selection result of at least one adaptive module or the application ratio of multiple adaptive modules. For example, if the rate of change of steering commands increases excessively after a specific adaptive module is applied, the autonomous driving device (100) may lower the application ratio of that adaptive module and increase the application ratio of other adaptive modules.

[0086] In one embodiment of step S260, the autonomous driving device (100) may determine whether the stability information or uncertainty information satisfies a preset standard. The standard may include an acceptable steering change rate range, an acceleration change rate range, a lower limit for object recognition reliability, an upper limit for an anomaly distribution index, or an output continuity standard. The autonomous driving device (100) may roll back to a previous adaptive module configuration or a preset conservative configuration based on the result of the determination. For example, if the object recognition reliability drops sharply after increasing the reflection rate of the highway adaptive module in the transition section, the autonomous driving device (100) may restore to the mixed configuration that was stably used in the previous step. As another example, if the output discontinuity exceeds a preset acceptable range, the autonomous driving device (100) may switch to the most conservative basic adaptive configuration to ensure the stability of vehicle control.

[0087] In another embodiment of step S260, the autonomous driving device (100) may sequentially perform readjustment and rollback. The autonomous driving device (100) may first attempt to restore output stability by fine-tuning the current adaptation module application ratio, and if stability is not restored by the fine-tuning alone, it may roll back to a previous adaptation module configuration or a preset conservative configuration. According to this sequential control structure, a more precise balance between environmental adaptability and output stability can be maintained compared to performing a full rollback immediately in all unstable situations.

[0088] The autonomous driving device (100) can input the above readjustment or rollback result back into the adaptation module reflection step for the common autonomous driving foundation model, thereby allowing the evaluation result after output generation to be directly reflected in the adaptation module configuration of the next inference cycle.

[0089] According to the procedure illustrated in FIG. 2, one embodiment of the present invention may provide a series of closed-loop control procedures that acquire driving environment information, generate road type-based control information, select or mix-control an adaptive module, generate an autonomous driving output by reflecting the adaptive module in a common autonomous driving foundation model, and then readjust or roll back the adaptive module configuration based on the generated autonomous driving output. Accordingly, environmental adaptability can be secured while maintaining a common autonomous driving foundation model in various driving environments where road types differ or change continuously, and the stability, reliability, and control efficiency of vehicle driving can be improved by mitigating output discontinuity caused by abrupt switching of the adaptive module.

[0091] FIG. 3 is a block diagram illustrating the detailed configuration of an autonomous driving device according to one embodiment of the present invention.

[0092] The autonomous driving device (100) according to FIG. 3 may be configured to include a driving environment information acquisition unit (110), a control information generation unit (120), an adaptive module control unit (130), a foundation model reflection unit (140), an autonomous driving output generation unit (150), and a readjustment unit (160). Each component illustrated in FIG. 3 may be implemented as a functionally separated logical module, executed as software on a single processor, or distributed across multiple processors or dedicated computing units. Additionally, each component may operate sequentially, and some components may operate in parallel and be performed repeatedly within the same driving cycle.

[0093] Referring to FIG. 3, the driving environment information acquisition unit (110) is a component that collects information regarding the surrounding environment and vehicle status.

[0094] In one embodiment, the driving environment information acquisition unit (110) may include a sensor information receiving unit and an external information linkage unit.

[0095] The sensor information receiving unit may be configured to receive sensor information from a sensor (11) mounted on a vehicle (10). The sensor information may include camera images, LiDAR point clouds, radar reflection data, ultrasonic detection information, inertial sensor information, vehicle speed information, steering angle information, and vehicle attitude information. For example, the sensor information receiving unit may simultaneously receive lane images acquired from a front camera and distance information of a front object acquired from LiDAR to configure the driving environment at the same time in a multi-sensor form.

[0096] The external information linkage unit may be configured to receive external information by communicating with an external information providing device (20). The external information may include map information, speed limit information, lane connection information, traffic congestion information, road construction information, weather information, and location-based auxiliary information. For example, the external information linkage unit may obtain information on changes in the road structure of a certain distance ahead from the current vehicle location in advance based on a map, and utilize information on ramp entry sections or lane reduction sections that have not yet been observed by the sensor in a subsequent stage. The driving environment information acquisition unit (110) may align sensor information and external information based on the same coordinate system or the same time axis to form driving environment information that can be processed subsequently. In this process, time delay correction between sensors, location alignment, and removal of duplicate information may be performed.

[0097] The control information generation unit (120) is a component that generates road type-based control information by analyzing driving environment information transmitted from the driving environment information acquisition unit (110).

[0098] In one embodiment, the control information generation unit (120) may include an environment feature extraction unit and a road type determination unit. The environment feature extraction unit can extract road structural features, object distribution features, speed limit change characteristics, lane connection structure, and surrounding traffic flow characteristics from driving environment information. For example, the environment feature extraction unit can calculate features such as changes in the number of lanes, the presence or absence of a median strip, intersection object density, the presence or absence of a crosswalk, entry lane angle, and vehicle density.

[0099] The road type determination unit can determine the road type of the current driving section based on extracted features or calculate the reliability for each of multiple road types. Road types may include general roads, expressways, ramp sections, interchange sections, or other transition sections. For example, the road type determination unit can calculate a high reliability for the expressway type when the speed limit is high, the frequency of pedestrian detection is low, and the proportion of straight sections is high. Conversely, it can calculate a high reliability for the general road type when traffic lights, stop lines, crosswalks, and various low-speed objects are detected. When multiple type features exist simultaneously, the road type determination unit can generate parallel reliability information for the multiple road types without determining a single type.

[0100] The adaptation module control unit (130) is a component that determines the application configuration of the adaptation module based on road type-based control information generated by the control information generation unit (120).

[0101] In one embodiment, the adaptive module control unit (130) may include an application module selection unit and a mixing ratio calculation unit. The application module selection unit may be configured to select an adaptive module suitable for the current driving situation among a plurality of adaptive modules. The plurality of adaptive modules may include a general road-compatible adaptive module, a highway-compatible adaptive module, a transition section-compatible adaptive module, and a specific driving context-compatible adaptive module. For example, if the general road type reliability is dominant, the application module selection unit may prioritize selecting a general road-compatible module.

[0102] The blending ratio calculation unit can be configured to calculate the reflection ratio of each adaptation module when multiple adaptation modules are applied simultaneously. For example, if the reliability of general road types and the reliability of highway types are both significant, the blending ratio calculation unit can calculate the application ratios of the two adaptation modules respectively to generate a blended configuration suitable for the transition section. In this process, hysteresis conditions, minimum maintenance time conditions, or transition cost conditions may be considered together. For instance, if the reliability value changes rapidly over a short period, the existing adaptation module configuration can be set to be maintained for a certain duration. Additionally, the rate of change in the application ratio can be limited to prevent the reflection of the adaptation modules from changing abruptly.

[0103] The foundation model reflection unit (140) is a component that reflects the adaptation module configuration determined by the adaptation module control unit (130) into a common autonomous driving foundation model.

[0104] In one embodiment, the foundation model reflection unit (140) may include an application module combination unit and a parameter reflection unit. The application module combination unit may be configured to combine a selected adaptation module or a plurality of mixed-control adaptation modules into some layers, some computation blocks, or some inference paths of a common autonomous driving foundation model. For example, a general road adaptation module may be strongly reflected in the object interaction interpretation layer, and a highway adaptation module may be strongly reflected in the lane following stability related layer.

[0105] The parameter reflection unit can be configured to apply the output of an adaptation module or additional parameters provided by the adaptation module to a common model. For example, correction parameters calculated by each of multiple adaptation modules can be weighted and summed according to an application ratio and then reflected in a specific set of parameters of the common model. As another example, only the additional parameters corresponding to the adaptation module can be activated while maintaining the basic parameters of the common model. Based on this structure, inference characteristics can be flexibly adjusted according to the driving environment without maintaining the entire autonomous driving model separately for each road type.

[0106] The autonomous driving output generation unit (150) is a component that generates an autonomous driving output used for actual driving control using the reflection result of the foundation model reflection unit (140).

[0107] In one embodiment, the autonomous driving output generation unit (150) may include a driving output generation unit and a vehicle control value generation unit. The driving output generation unit may generate an autonomous driving output including a target path of the vehicle, a target speed, a lane center movement path, a distance maintenance strategy between vehicles, and an avoidance path. For example, if deceleration of the vehicle ahead is detected, an output including a target deceleration trajectory may be generated, and in a lane merging section, a target lane switching path may be included.

[0108] The vehicle control value generation unit can generate steering control values, acceleration / deceleration control values, and braking control values ​​transmitted to the vehicle control unit (200) based on the generated autonomous driving output. For example, the steering control value can be calculated based on the error between the center of the target lane and the current position of the vehicle, and the acceleration / deceleration control value can be calculated based on the difference between the distance to the vehicle ahead and the target speed. The autonomous driving output generation unit (150) may select a path with high stability among multiple driving candidate paths, or rearrange output candidates according to the state of the adaptation module reflection.

[0109] The autonomous driving output generated by the autonomous driving output generation unit (150) is not only transmitted to the vehicle control unit (200) but can also be provided as an input to the readjustment unit (160). Accordingly, how the selection or mixed control result of the adaptation module affected the actual output characteristics can be subsequently evaluated within the same device.

[0110] The readjustment unit (160) is a component that re-evaluates the configuration of the adaptive module after the autonomous driving output is generated.

[0111] In one embodiment, the readjustment unit (160) may include an output stability evaluation unit and an applied configuration restoration unit. The output stability evaluation unit may evaluate the continuity of the generated autonomous driving output, the rate of change in steering, the rate of change in acceleration, the prediction consistency, and the change in object recognition reliability. For example, if the rate of change in steering increases rapidly or the vibration of the predicted trajectory increases immediately after the adaptation module change, the output stability evaluation unit may determine this as an unstable state.

[0112] In one embodiment, the adaptation module application configuration or rollback result recalculated by the re-adjustment unit (160) is transmitted back to the adaptation module control unit (130) or foundation model reflection unit (140) and can be used for the selection, mixing, or adjustment of reflection intensity of the adaptation module in the next control cycle.

[0113] The readjustment unit (160) can adjust the current adaptive module configuration or restore it to the previous configuration based on the output stability evaluation results. For example, output stability can be restored by fine-tuning the ratio of the adaptive modules currently in use, and if instability persists above a certain threshold, it can be rolled back to the adaptive module configuration used in the previous stable state. As another example, if a specific module among multiple adaptive modules is determined to be the main cause of instability, only the application ratio of that module can be reduced while maintaining the remaining modules.

[0114] According to the structure of the autonomous driving device (100) illustrated in FIG. 3, one embodiment of the present invention can structurally separate and perform functions ranging from acquiring driving environment information to selecting an adaptive module, reflecting a common model, generating autonomous driving output, and readjusting based on output. Accordingly, even in a driving environment where the road type changes continuously, only the adaptive module can be dynamically controlled while maintaining a common autonomous driving foundation model, and output discontinuity caused by the switching of the adaptive module can be suppressed through the readjustment unit (160). In addition, computational efficiency can be increased compared to the method of replacing the entire model, and scalability for various road types can be secured.

[0116] FIG. 4 is a configuration diagram illustrating the adaptive module control structure of a road type-based autonomous driving foundation model according to one embodiment of the present invention from a layer perspective.

[0117] The structure illustrated in FIG. 4 may be composed of a sensor and perception layer, an L2 meta-controller layer, an L3 foundation driving model layer, and an output and control layer. The perception result of the vehicle's surrounding environment is converted into road type-based control information, and based on the control information, the application configuration of an adaptive module combined with a common autonomous driving foundation model is determined. The result of this reflection is generated as an autonomous driving output, and a closed-loop structure may be formed in which the result is readjusted or rolled back according to output stability feedback. FIG. 4 can be understood as a detailed structure showing how the control information generation unit (120), adaptive module control unit (130), foundation model reflection unit (140), autonomous driving output generation unit (150), and readjustment unit (160) of FIG. 3 are interconnected at the layer level.

[0118] Referring to FIG. 4, the upper sensor and perception layer may be composed of an L0 sensor and data interface and an L1 perception and mapping. The L0 sensor and data interface receives data collected from cameras, lidar, radar, inertial sensors, satellite navigation receivers, and other driving sensors mounted on the vehicle, and can perform time alignment, coordinate system correction, noise removal, and data format normalization. The L1 perception and mapping can generate perception results related to lanes, road boundaries, drivable areas, surrounding objects, object types, object speeds, road connectivity relationships, and the current vehicle position using the aligned sensor data. The operation of these L0 and L1 can be understood as a form in which the driving environment information acquisition unit (110) of FIG. 3 is realized at a lower level layer. For example, the L1 perception and mapping can combine front camera and lidar information to estimate the number of lanes, the presence of an entry ramp, the presence of a median strip, and the surrounding vehicle density, and transmit the results to the downstream meta-control layer.

[0119] In one embodiment of the present invention, the L2 meta-controller layer may be a core layer that generates road type-based control information and performs selection or blending control of an adaptive module. The L2 meta-controller layer may include 1 a context integrator, 2 a model fit estimator, 3 a switching gate, 4 an uncertainty monitoring system, 5 a safety fallback manager, and 6 a feedback scheduler and a rollback. These configurations can be understood as a structure that subdivides the control information generation unit (120), the adaptive module control unit (130), and the readjustment unit (160) of FIG. 3.

[0120] A context integrator can synthesize various inputs transmitted from the sensor and perception layers to form an integrated context that represents the current driving context. This integrated context may include lane structure, changes in the number of lanes, lane curvature, speed limits, lane merging status, object distribution, potential presence of pedestrians, surrounding vehicle flow, map-based road connectivity information, and the current vehicle speed, acceleration, and steering state. The context integrator can align information from different sources based on the same time axis and spatial criteria to convert it into feature vectors or state representations suitable for subsequent processing. For example, if a highway entry section exists ahead on the map information, an increase in the number of lanes and a merging pattern are detected based on sensor perception, and the vehicle speed is gradually increasing, the context integrator can represent this as a complex driving context that is close to a transition section.

[0121] The model fit estimator receives the aforementioned integration context as input and can evaluate the fit for each of multiple adaptive module configurations that can be combined into a common autonomous driving foundation model. Indicated in the diagram in the form of "Set Score, Probability," this can be interpreted to mean that values ​​in the form of a fit score, selection probability, or confidence level can be calculated for a specific adaptive module or a specific mixed configuration. The fit can be calculated independently for each of the general road adaptive module, the highway adaptive module, the transition section adaptive module, or their mixed configurations. For example, if intersections are dense and many pedestrians and motorcycles are detected, the fit of the general road adaptive module may be calculated as high; conversely, if the lane structure is simple, the speed limit is high, and the proportion of straight-line driving is large, the fit of the highway adaptive module may be calculated as high. In the transition section, two or more adaptive modules may simultaneously have meaningful fits.

[0122] A switching gate may be a component that controls whether to change the adaptive module or the speed of adjustment of the application rate based on the above fitness estimation results. As illustrated in the drawing, the switching gate may consider hysteresis, dwell time, and switching cost. Hysteresis may be a criterion that prevents the adaptive module configuration from being switched immediately even if the fitness difference is temporarily reversed. Dwell time may be a criterion that suppresses excessive oscillatory switching by ensuring the current adaptive module configuration is maintained for a certain period. Switching cost may be a cost term reflecting inference instability, increased rate of change of output, internal state discontinuity, or additional use of computational resources that may occur due to the change in the adaptive module. For example, even if the fitness of the highway adaptive module becomes momentarily slightly higher than that of the general road adaptive module, the switching gate may maintain the current configuration if the difference does not exceed the hysteresis criterion. As another example, if the switching point is immediately after a continuous lane change or immediately before entering a sharp curve, the switching cost may be calculated significantly to limit the speed of adjustment of the application rate.

[0123] The uncertainty monitor may be a component that monitors the reliability of the output generated based on the current adaptation module configuration and the results of the foundation model reflection. The diagram shows forms such as out-of-distribution, variance, and risk, which can be understood to mean that it may include anomaly distribution indicators, predicted variance, and driving risk indicators. The uncertainty monitor can monitor for the possibility of decreased object recognition reliability, increased path prediction variance, increased control output oscillation, decreased environmental suitability, or the occurrence of out-of-distribution situations. For example, if pedestrian detection reliability drops sharply after a specific adaptation module reflection, or if the planned path oscillates unstably over time, the uncertainty monitor may determine this as a risk signal. The results of the uncertainty monitoring may be passed to the feedback scheduler and rollback as shown in the diagram, or may be directly referenced by the safety fallback manager.

[0124] The Safety Fallback Manager may be a component that performs safety-oriented output stabilization control using results passed from the Uncertainty Monitor, Feedback Scheduler, and Rollback. In the diagram, it is illustrated as a separate box at the bottom of L2 and can be understood as performing the final conservative response function within the meta-control layer. The Safety Fallback Manager can stop the use of the current adaptive module configuration and restore the adaptive module configuration that was previously operating stably, or switch to a preset conservative default configuration. For example, if a decrease in object recognition reliability and a sharp increase in steering rate of change occur simultaneously after increasing the weight of the highway adaptive module in a transition section, the Safety Fallback Manager can instruct an immediate return to the previous stable configuration. As another example, if a persistent out-of-distribution situation is detected, it can instruct a reduction in the reflection level of both the general road and highway adaptive modules and a switch to a conservative default configuration.

[0125] The feedback scheduler and rollback may be components that perform readjustment or rollback by integrating jerk or discontinuous feedback from the transition gate, uncertainty monitoring, and output layer. In the drawing, it is illustrated as a structure that receives both the outputs of the transition gate and uncertainty monitoring, and also receives "jerk or discontinuous feedback" that is fed up from the output and control layers. The feedback scheduler and rollback can schedule fine-tuning of the adaptation module application rate, slowing down of the transition speed, a stepwise return to the previous configuration, or a full rollback. For example, if the rate of change in the vehicle's steering increases rapidly after the generation of the autonomous driving output but the object recognition reliability is maintained, readjustment can be performed by partially lowering only the application rate of a specific adaptation module instead of a full rollback. Conversely, if an increase in jerk and a decrease in object recognition reliability occur simultaneously, a stronger rollback measure can be performed. This configuration can be seen as an example in which the readjustment unit (160) of FIG. 3 is implemented with a more fine layer structure.

[0126] The L3 Foundation Driving Model layer may be a layer where a common autonomous driving foundation model and multiple adaptive modules are actually combined to perform inference. The drawing illustrates "Application Module Bank LoRA, Adapter, Expert Delta," "Common Foundation Model Frozen Backbone, Shared Basic Parameters," and "Dynamic Weight Blender Partial Parameter Injection and Scaling." This configuration may correspond to a layer-level implementation of the adaptive module control unit (130) and the foundation model reflection unit (140) of FIG. 3.

[0127] The application module bank may be a set of adaptation modules corresponding to multiple road types or multiple driving contexts. The adaptation modules may be implemented as low-dimensional correction parameter structures, adapter structures, expert-based correction structures, or equivalent partial parameter modules. General road adaptation modules may be trained to enhance responsiveness to pedestrians, signal systems, intersecting objects, and complex interactions, while highway adaptation modules may be trained to be advantageous for lane following, high-speed driving stability, and merging judgment. Transition section adaptation modules may be designed to complement continuous control in areas where the characteristics of the two environments overlap.

[0128] The common foundation model may be a basic backbone or shared parameter structure that is commonly applied to multiple road types. The "frozen backbone, shared basic parameters" described in the drawing implies that there are basic model parameters that are commonly maintained instead of replacing the entire model when controlling the adaptive module. Accordingly, one embodiment of the present invention may adopt a structure that dynamically reflects only partial parameters while maintaining a common autonomous driving foundation model, instead of operating a completely separate large model for each road type. For example, while maintaining the same basic visual-behavior inference model, the general road adaptive module may be reflected more significantly when driving on a general road, and the highway adaptive module may be reflected more significantly when driving on a highway.

[0129] A dynamic weight blender may be a component that performs partial parameter injection and scaling between multiple adaptation modules and a common foundation model. The diagram is labeled "Partial parameter injection and scaling," with the explanation "Weight adjustment Alpha LoRA or adapter scaling, speed limiting" added to the left. This implies that instead of discretely turning the adaptation modules on and off, the reflection intensity of each adaptation module can be adjusted in the form of weights, and the rate of change can even be limited. For example, in the transition section from a general road to a highway, the scaling factor of the general road adaptation module can be gradually decreased from 0.8 to 0.5 and then to 0.2, while the scaling factor of the highway adaptation module can be gradually increased from 0.2 to 0.5 and then to 0.8. If a rate limiting mechanism is applied during this process, it can mitigate abrupt interruptions in autonomous driving output.

[0130] The output and control layer may consist of an L4 driving output layer and an L5 vehicle control and drive layer. The L4 driving output layer can generate a target trajectory, vehicle control commands, lane keeping control amounts, avoidance paths, or speed profiles based on the inference results of the L3 Foundation driving model layer. The outputs can be transmitted to the L5 vehicle control and drive layer to be converted into actual steering, acceleration, and braking commands. For example, the L4 driving output layer can generate a follow trajectory that considers maintaining distance from the vehicle ahead while staying in the center of the lane on highway sections, and can generate a more conservative deceleration path that considers pedestrian detection and signal conditions on general roads.

[0131] In the diagram, "jerk or discontinuous feedback" is indicated on the feedback path from the L4 driving output layer to the 6 feedback scheduler and rollback of the L2 metacontroller layer. This means that the steering change rate, acceleration / deceleration change rate, control discontinuity, and vehicle behavior shock degree observed at the output layer can be fed back to the upper layer and used for readjusting the adaptive module. For example, if steering commands fluctuate abruptly or braking commands change discontinuously immediately after changing the mixing ratio of the adaptive module, this information can be passed to the feedback scheduler and rollback to be used to re-evaluate the stability of the current application configuration.

[0132] According to the structure illustrated in FIG. 4, one embodiment of the present invention integrates, interprets, and evaluates driving environment information obtained from the sensor and perception layers in the L2 meta-controller layer, and dynamically combines a common autonomous driving foundation model and a plurality of adaptive modules in the L3 foundation driving model layer according to the evaluation results, thereby enabling actual autonomous driving output and vehicle control through the L4 and L5 layers. In this process, the selection or mixed control of the adaptive modules can be operated in a closed-loop form combined with a transition gate, uncertainty monitoring, feedback scheduling, and safety fallback. Therefore, output continuity and driving stability can be simultaneously secured while maintaining a common model structure, even in situations where environmental changes occur continuously, such as when road types differ or in transition sections.

[0133] According to the structure of FIG. 4, one embodiment of the present invention can precisely control the selection of an adaptation module, the mixing ratio, the reflection intensity, and the reflection speed based on road type-based control information, unlike a technique that simply determines the current road type and selects a separate model. In addition, by providing a hierarchical feedback structure that reflects jerk, discontinuity, uncertainty, and risk indicators after autonomous driving output, it can actively suppress instability caused by the switching of the adaptation module and return to a previous configuration or a conservative configuration if necessary. Accordingly, adaptability to various road environments, output continuity, control stability, and model operation efficiency can be improved together.

[0135] FIG. 5 is a flowchart illustrating the procedure of a road type-based control information generation method according to one embodiment of the present invention.

[0136] The procedure according to FIG. 5 may consist of a process in which an autonomous driving device receives driving environment information, generates an environment feature vector from the information, calculates a plurality of road type candidates, evaluates the reliability of each road type by reflecting map information and transition section information, and finally generates road type-based control information. Each step of FIG. 5 can be understood as an embodiment in which the functions of the driving environment information acquisition unit (110) and the control information generation unit (120) of FIG. 3 are performed in terms of the method, and can subsequently be used as an input for adaptive module control in the L2 meta-controller layer of FIG. 4.

[0137] Referring to FIG. 5, in step S510, the autonomous driving device may receive driving environment information. The driving environment information may include real-time environment information obtained from sensors mounted on the vehicle and external information received from an external information providing device. The real-time environment information may include camera images, LiDAR point clouds, radar reflection data, ultrasonic sensor information, vehicle speed information, steering angle information, inertial sensor information, and vehicle attitude information. The external information may include map information, speed limit information, lane connection information, road structure information, traffic information, construction information, and weather information.

[0138] In one embodiment, the autonomous driving device can detect lane markings from a front camera image, calculate the distance and relative speed of a vehicle ahead from LiDAR data, and obtain a road connection structure a certain distance ahead relative to the current location from map data. Additionally, data having different acquisition cycles for each sensor can be aligned based on the same point in time, and sensor data expressed in different coordinate systems can be converted into a vehicle-based coordinate system or a map-based coordinate system. During this process, noise removal, missing data correction, and redundancy information removal may be performed.

[0139] In step S520, the autonomous driving device may generate an environment feature vector based on input driving environment information. The environment feature vector may be a set of features that numerically represent the road structure, object distribution, and driving context of the current driving section. For example, the environment feature vector may include the number of lanes, lane width, lane curvature, presence of a median strip, lane merging, change in road width, forward object density, pedestrian detection frequency, presence of a stop line, presence of a traffic light, average object speed, inter-vehicle distance distribution, and current vehicle speed.

[0140] In one embodiment, the number of lanes can be determined by using image-based lane detection results and map-based lane information together. For example, if only two lanes are detected by the camera but a section entering a third lane begins according to the map information, the autonomous driving device can reflect the possibility of increasing the number of lanes in the feature vector. Additionally, object density can be calculated by weighted summing the number of vehicles, motorcycles, and pedestrians detected within a certain area. Lane curvature can be expressed as the average or maximum curvature value of the lane centerline curvature at a certain distance ahead. These feature vectors can be used as inputs for determining the road type in a subsequent step.

[0141] In step S530, the autonomous driving device may generate multiple road type candidates based on the generated environment feature vector. The road type candidates may refer to multiple road types to which the current driving section may belong, and may include general roads, expressways, ramp sections, interchange sections, intersection approach sections, or other transition sections. In this step, a single road type is not immediately determined, and multiple candidates may be maintained simultaneously.

[0142] For example, if the number of lanes increases, speed limits are high, and the frequency of detecting pedestrians ahead is very low, the autonomous driving system can set a high priority for highway candidates. Conversely, if there are many lanes but numerous traffic lights, crosswalks, and low-speed objects, general road candidates can be maintained alongside them. Additionally, if a lane merging structure is detected and an entry ramp shape exists simultaneously, ramp section candidates can be generated separately. Each candidate can be maintained along with a score or priority value, allowing for a more precise reliability evaluation in a subsequent step.

[0143] In step S540, the autonomous driving device can evaluate the reliability of each road type candidate by reflecting map information and transition section information. The above step may be a step of performing a more stable judgment on the road type candidates generated in step S530 by considering not only simple current sensor features but also spatial precedence information and section change information together.

[0144] In one embodiment, the map information may include road connection structures, ramp entry, interchange existence, lane reduction or increase sections, speed limit change sections, and branching structures at a certain distance ahead from the current vehicle location. For example, even if a general road and a highway candidate have similar scores based only on current sensor features, if a highway main line entry structure exists within 200 meters ahead on the map information, the reliability of the highway candidate can be increased.

[0145] Transition section information may be data that allows the current and future states to be reflected simultaneously in sections where the road type changes. For example, if a vehicle is still on a general road section but a ramp entry begins ahead, the reliability difference between the two candidates can be gradually adjusted while maintaining both general road candidates and ramp section candidates simultaneously. In this case, reliability evaluation can be performed by combining current feature scores, map-based future structure scores, and transition distance weights.

[0146] In addition, in one embodiment, the autonomous driving device can suppress the determination of abrupt road type changes by limiting the rate of change in reliability. For example, even if the highway candidate score increases momentarily during a continuous driving cycle, the final reliability reflection rate can be limited if it is not maintained for a certain period of time. This method can contribute to reducing unnecessary frequent switching in the subsequent adaptive module control stage.

[0147] In step S550, the autonomous driving device may generate road type-based control information based on the evaluated road type reliability. The road type-based control information may be control information used for subsequent adaptation module selection or mixing ratio control, may be a result of selecting a single road type, or may be application ratio information for each of multiple road types.

[0148] In one embodiment, when the reliability of a general road is evaluated to be 0.7 and the reliability of a highway is evaluated to be 0.3, the autonomous driving device may generate control information to predominantly reflect the general road adaptation module while applying some highway adaptation modules in combination. As another example, when the reliability of the general road and the ramp section is evaluated to be similar, the device may generate transition control information that gradually changes the reflection ratio of the two adaptation modules over time.

[0149] In addition, road type-based control information may include transition speed limit information, minimum holding time information, hysteresis criteria, and uncertainty monitoring criteria in addition to simple ratio values. For example, even if it is determined to be a current transition section, control information may be generated to lower the transition speed if steering stability is required as a priority. This information can be used directly in the transition gate and dynamic weight blender of FIG. 4.

[0150] In one embodiment, the road type-based control information generated in step S550 may be provided as an input to determine which of the plurality of adaptation modules to select, or how to set the reflection intensity of the plurality of modules, to be subsequently combined into a common autonomous driving foundation model. For example, a combination suitable for the current situation may be selected among a general road adaptation module, a highway adaptation module, and a transition section adaptation module.

[0151] According to the procedure illustrated in FIG. 5, one embodiment of the present invention can generate road type-based control information by not merely determining the road type once using only current sensor information, but by generating an environment feature vector, maintaining multiple candidates, reflecting a map-based future structure, and evaluating the reliability of the transition section. Accordingly, even in a real driving environment where road type changes occur continuously, inputs for stable adaptive module control can be generated without abrupt judgment switching.

[0152] Furthermore, by generating reliability-based control information that simultaneously considers multiple road types, discontinuities that may occur in a single model selection structure can be reduced, and the reflection of the adaptation module for the common foundation model in subsequent stages can be performed more precisely. As a result, the continuity of autonomous driving output and driving stability can be improved even in various road structures, ramp sections, interchange sections, and boundary areas between general roads and highways.

[0154] FIG. 6 is a block diagram illustrating the detailed configuration of an adaptive module control unit according to one embodiment of the present invention.

[0155] The adaptive module control unit (130) illustrated in FIG. 6 may include a module candidate determination unit (131), an application module selection unit (132), a mixing ratio calculation unit (133), a transition condition reflection unit (134), and an application configuration generation unit (135). Each component of FIG. 6 represents a more detailed representation of the function performed by the adaptive module control unit (130) of FIG. 3. It may be configured to receive road type-based control information or road type candidates and reliability information generated in FIG. 5 as input, and to generate a final application configuration that reflects the selection result of an adaptive module to be reflected in a common autonomous driving foundation model, the mixing ratio, and the transition conditions. According to such a structure, an embodiment of the present invention can precisely control the application configuration of the adaptive module even when there are different road types or complex driving contexts such as transition sections.

[0156] Referring to FIG. 6, the module candidate determination unit (131) is a component that determines an adaptation module candidate based on a road type candidate input. The module candidate determination unit (131) may receive a road type candidate, road type reliability information, transition section information, or related control information provided by the control information generation unit (120). Here, the road type candidate may include a general road, a highway, a ramp section, an interchange section, or other transition sections, and different adaptation module candidates may correspond to each road type. For example, if a general road type candidate is input, a general road adaptation module trained to respond to pedestrians, signal systems, intersection objects, and low-speed driving situations may be determined as a candidate. If a highway type candidate is input, a highway adaptation module advantageous for lane keeping stability, following a vehicle ahead, high-speed driving stability, and merging response may be determined as a candidate. If a transition section candidate is present, a general road adaptation module and a highway adaptation module may be maintained as candidates simultaneously, or an adaptation module specialized for the transition section may be determined as an additional candidate.

[0157] In one embodiment, the module candidate determination unit (131) does not operate in a manner that corresponds one adaptation module to a single road type in a one-to-one manner, but can maintain multiple adaptation module candidates in parallel when multiple road type candidates are simultaneously valid. For example, when a vehicle approaches a highway entrance ramp from a general road, the module candidate determination unit (131) can include both a general road adaptation module and a highway adaptation module in the candidate group simultaneously, and maintain an adaptation module that reflects the characteristics of the transition section as an additional candidate. This candidate maintenance structure is important in that it enables not only single module selection but also mixed control in subsequent steps.

[0158] The application module selection unit (132) is a component that determines the priority application module among the above candidate group. The application module selection unit (132) can select an adaptation module that should be applied preferentially in the current driving situation by considering the adaptation module candidate group transmitted from the module candidate determination unit (131) together with road type-based control information. Here, the priority application module may be in the form of a single module, or it may be in the form of a main module that serves as the standard for mixed control in a subsequent stage. For example, if the reliability of general roads is the highest and the reliability of highways is evaluated to be lower, the application module selection unit (132) may determine the general road adaptation module as the priority application module. Conversely, if highway driving continues for a long time, the lane structure is stable, and the frequency of pedestrian detection is almost non-existent, the highway adaptation module may be determined as the priority application module.

[0159] In one embodiment, the application module selection unit (132) may not simply mechanically select the adaptation module with the highest current reliability value, but may also consider recent driving history, map information of the forward section, whether entry into a transition section is scheduled, and the configuration of the adaptation module applied in the previous driving cycle. For example, even if the reliability of the general road and the reliability of the highway are similar at the current point in time, if entry into the main highway lane is scheduled within a few seconds according to the map information, the highway adaptation module may be selected as the priority application module, or the general road adaptation module may be maintained but set as a reference module for mixed control. This structure can mitigate the phenomenon where the adaptation module changes frequently and unnecessarily due to short-term sensor changes.

[0160] The mixing ratio calculation unit (133) is a component that performs calculation of multiple module weights. The mixing ratio calculation unit (133) can calculate the reflection ratio of each module for other adaptation module candidates maintained in parallel, along with the priority application module determined by the application module selection unit (132). Here, the mixing ratio may be a value indicating the intensity with which multiple adaptation modules are reflected in a common autonomous driving foundation model, and may be expressed in the form of a percentage, a normalized real value, or a scaling factor. For example, the application ratio of the general road adaptation module may be set to 0.8 and the application ratio of the highway adaptation module to 0.2, or multiple mixing configurations such as the general road adaptation module 0.5, the highway adaptation module 0.3, and the transition section adaptation module 0.2 may be created.

[0161] In one embodiment, the mixing ratio calculation unit (133) can calculate the weight of each adaptation module by directly reflecting road type reliability information. For example, if the general road reliability is evaluated as 0.6, the highway reliability as 0.3, and the transition section reliability as 0.1, the mixing ratio calculation unit (133) can use these values ​​as they are, or calculate a corrected mixing ratio by additionally reflecting the characteristics of each adaptation module, the results of a recent output stability evaluation, or a priority policy for a specific road type. As another example, in the transition section, a method may be applied in which the ratio between the general road adaptation module and the highway adaptation module is continuously changed over time, and the ratio of the transition section adaptation module is temporarily increased. According to this structure, the transition of the adaptation module can be performed as a continuous mixing adjustment rather than a discrete replacement.

[0162] The transition condition reflection unit (134) is a component that performs transition cost reflection and stability condition reflection. The transition condition reflection unit (134) can generate a corrected application configuration by considering the transition cost and stability condition together, rather than using the multiple module weight results calculated by the mixing ratio calculation unit (133) as is, and considering the change in the adaptation module or the change in the application ratio. Here, the transition cost may be a cost that reflects the possibility of output discontinuity, internal state change, increased inference load, or reduced control stability that may occur due to the change in the adaptation module. The stability condition may be defined based on indicators such as steering change rate, acceleration / deceleration change rate, output continuity, object recognition reliability, prediction variance, and risk.

[0163] For example, even if it is computationally possible to rapidly increase the ratio from the general road adaptation module to the highway adaptation module, if the steering change rate of the recent autonomous driving output is already large or complex lane merging occurs ahead, the transition condition reflection unit (134) may limit the rate of increase of the highway adaptation module. As another example, in a situation where there is a high possibility that the object recognition reliability will decrease immediately after changing the adaptation module, the ratio may be adjusted to apply a more conservative ratio than the result calculated by the mixing ratio calculation unit (133). In this process, hysteresis conditions, minimum maintenance time conditions, and transition cost conditions may be considered together, and as a result, the decision to change a specific adaptation module itself may be withheld, or the rate of change in the application ratio of multiple adaptation modules may be limited stepwise.

[0164] The application configuration generation unit (135) is a component that generates a module application combination output. Based on the results corrected in the transition condition reflection unit (134), the application configuration generation unit (135) can generate a final application configuration of an adaptation module to be reflected in the actual common autonomous driving foundation model. The application configuration may include which adaptation module will be activated, what ratio each adaptation module will be reflected, what level the rate of increase or decrease of a specific adaptation module will be limited to, and whether additional constraints will be imposed on a specific transition section. For example, the application configuration generation unit (135) can generate a mixed configuration of a general road adaptation module 0.65, a highway adaptation module 0.25, and a transition section adaptation module 0.10, and output control information that limits the highway adaptation module ratio to increase by a maximum of only 0.05 per next cycle.

[0165] In one embodiment, the module application combination output generated by the application configuration generation unit (135) can be transmitted to the foundation model reflection unit (140) of FIG. 3 or the L3 foundation driving model layer of FIG. 4. Accordingly, the common autonomous driving foundation model can selectively or mixedly reflect the output or parameters of the adaptation module according to the application configuration while maintaining the overall structure. For example, the application configuration generation unit (135) may additionally include layer-specific combination information that increases the reflection intensity of the general road adaptation module in a specific layer and increases the reflection intensity of the highway adaptation module in another layer. In this case, while the basic parameters of the common model are maintained, the inference characteristics corresponding to changes in road type can be finely adjusted.

[0166] The structure of the adaptive module control unit (130) illustrated in FIG. 6 can be distinguished from a structure that simply selects one adaptive module immediately based on the road type determination result. The adaptive module control unit (130) according to one embodiment of the present invention has a multi-stage structure that maintains road type candidates, selects a priority application module, calculates the mixing ratio of multiple modules, reflects switching costs and stability conditions, and then generates a final module application combination. Accordingly, the application configuration of the adaptive module can be controlled more smoothly and stably even in situations where the driving environment changes continuously, such as in the transition section between a general road and an expressway.

[0167] For example, in a situation where a vehicle is moving along a highway entrance ramp from a general road, the module candidate determination unit (131) may maintain a general road adaptation module, a highway adaptation module, and a transition section adaptation module as a group of candidates. The application module selection unit (132) may set the general road adaptation module as the priority application module, while taking into account that entry into the highway is imminent. The mixing ratio calculation unit (133) may set the ratio of the general road adaptation module to 0.7, the ratio of the highway adaptation module to 0.2, and the ratio of the transition section adaptation module to 0.1. The transition condition reflection unit (134) may limit the rate of increase of the highway adaptation module ratio by considering current steering stability. The application configuration generation unit (135) may generate a final combination reflecting these conditions and transmit it to the foundation model reflection unit (140). Through this process, adaptability to changes in road type can be secured while mitigating abrupt changes in autonomous driving output.

[0168] As another example, if a sudden construction zone is detected while driving on a highway and driving context elements similar to a general road increase slightly, the module candidate determination unit (131) may include the general road adaptation module back into the candidate group. The application module selection unit (132) may still maintain the highway adaptation module as the priority application module. The mixing ratio calculation unit (133) may generate a mixing result that slightly increases the ratio of the general road adaptation module. The transition condition reflection unit (134) may correct the ratio change so that it does not become abrupt based on the current lane keeping stability. Consequently, the application configuration generation unit (135) may generate a corrected configuration that reflects some general road adaptation characteristics while maintaining a configuration centered on the highway adaptation module.

[0169] As shown in the structure illustrated in FIG. 6, one embodiment of the present invention can convert road type-based control information into a form that can be directly reflected in a common autonomous driving foundation model by performing the selection or blending control of an adaptive module in stages. In particular, by being configured to sequentially perform priority module selection, blending ratio calculation, reflection of transition conditions, and generation of the final applied configuration while maintaining multiple module candidates, adaptability to changes in the driving environment can be enhanced compared to a single module replacement structure. Furthermore, since the speed of change of the adaptive module application configuration can be controlled, the continuity of autonomous driving output, vehicle behavior stability, and control reliability can be improved.

[0170] The adaptive module control unit (130) of FIG. 6 can function as an intermediate layer that generates control results applicable to an actual common autonomous driving foundation model by the foundation model reflection unit (140) described later in FIG. 7. Accordingly, one embodiment of the present invention can provide an adaptive control structure that stably connects a front-end structure that generates road type-based control information from driving environment information and a rear-end structure that generates autonomous driving output from a foundation model reflecting the adaptive module.

[0172] FIG. 7 is a block diagram illustrating the detailed configuration of a foundation model reflection part according to one embodiment of the present invention.

[0173] The foundation model reflection unit (140) illustrated in FIG. 7 may include an application configuration input unit (141), an insertion position determination unit (142), an adaptation parameter reflection unit (143), a common model combination unit (144), and an inference input transmission unit (145). Each component of FIG. 7 represents a more detailed representation of the function performed by the foundation model reflection unit (140) of FIG. 3, and may be configured to receive a module application combination generated by the adaptation module control unit (130) of FIG. 6 as input, actually reflect the adaptation module in the common autonomous driving foundation model, and transmit the reflection result to the autonomous driving output generation unit (150). According to such a structure, one embodiment of the present invention may provide a structure that partially injects or combines only the adaptation module while maintaining the common autonomous driving foundation model, instead of replacing the entire autonomous driving model according to the road type.

[0174] Referring to FIG. 7, the application configuration input unit (141) is a component that performs the function of receiving module application combinations. The application configuration input unit (141) can receive an adaptation module application configuration output from the application configuration generation unit (135) of FIG. 6. The application configuration may include the selection result of at least one adaptation module, the application ratio of each of a plurality of adaptation modules, a limit on the rate of increase or decrease of a specific adaptation module, mixed combination information reflecting transition conditions, and a layered reflection condition. For example, the application configuration input unit (141) can receive a weighted combination of a general road adaptation module 0.6, a highway adaptation module 0.3, and a transition section adaptation module 0.1, along with a limiting condition that the ratio of the highway adaptation module can only increase by a maximum of 0.05 per next control cycle. As another example, the application configuration input unit (141) can receive a layered application combination that specifies that a specific adaptation module is reflected only in the perception-related inference layer, and another adaptation module is reflected only in the planning-related inference layer.

[0175] In one embodiment, the application configuration input unit (141) may receive not only a simple single module selection result, but also a selection result of multiple adaptation modules and a mixed structure. Accordingly, in subsequent steps, the common autonomous driving foundation model may not receive only one adaptation module, but may adopt a structure in which the influence of multiple adaptation modules is simultaneously reflected according to changes in road type or transition section characteristics. For example, during the process of entering a highway from a general road, a general road adaptation module and a highway adaptation module may be input simultaneously, and a transition section adaptation module may be additionally included.

[0176] The insertion position determining unit (142) is a component that performs the function of selecting an application layer. Based on the module application combination received from the application configuration input unit (141), the insertion position determining unit (142) can determine which layer, which computation block, or which inference path each adaptation module will be reflected in a common autonomous driving foundation model. Here, the application layer may include a sensor feature interpretation layer, an object interaction interpretation layer, a driving context interpretation layer, a path generation layer, a control value generation layer, or an equivalent internal computation layer. For example, a general road adaptation module may be inserted preferentially into a layer where post-recognition interpretation of pedestrians, traffic lights, and cross-traffic is important, and a highway adaptation module may be inserted preferentially into a computation layer related to lane following stability, high-speed object prediction, and forward vehicle following.

[0177] In one embodiment, the insertion position determining unit (142) may be configured to reflect a single adaptation module differentially in different layers depending on the nature of the adaptation module and the road type, rather than reflecting it evenly throughout the entire common autonomous driving foundation model. For example, a transition section adaptation module may be preferentially inserted into an intermediate interpretation layer that requires sensitive interpretation of environmental transitions, and a highway adaptation module may be reflected more significantly in a planning layer related to long-term path stability. Depending on this insertion position determining structure, even with the same combination of adaptation modules, the pattern of influence within the foundation model may be implemented differently. This demonstrates that one embodiment of the present invention not only controls whether an adaptation module is selected, but also precisely controls the way the adaptation module operates within the foundation model.

[0178] The adaptive parameter reflection unit (143) is a component that performs a weight adjustment function. The adaptive parameter reflection unit (143) can actually reflect the output of the adaptive module or additional parameters provided by the adaptive module for the application layer or computation path determined by the insertion position determination unit (142). The reflection may be a method of injecting the correction parameters of a single adaptive module as they are, or a method of injecting the outputs or parameters of each of multiple adaptive modules after weighted combination according to the application ratio. For example, the adaptive parameter reflection unit (143) may weight sum the set of correction parameters generated by the general road adaptive module and the set of correction parameters generated by the highway adaptive module at a ratio of 0.7 to 0.3 and reflect this to a specific layer of the common autonomous driving foundation model.

[0179] In one embodiment, the adaptive parameter reflection unit (143) may reflect low-dimensional correction parameters, adapter parameters, expert correction parameters, or other partial parameter structures. For example, if a low-dimensional correction parameter in the form of LoRA is used, the adaptive parameter reflection unit (143) may reflect it by adding an additional low-dimensional correction value to the existing weights of the common model. If an adapter structure is used, the activation strength of an adaptive computation block inserted between specific layers may be adjusted. If an expert structure is used, it may reflect it in a way that increases the contribution of a specific expert among multiple experts. As such, the adaptive parameter reflection unit (143) may adopt various reflection methods depending on the implementation form of the adaptive module, but it is generally significant in that it adjusts only partial parameters or partial computation structures without replacing the entire common autonomous driving foundation model.

[0180] The common model combining unit (144) is a component that performs the function of maintaining common parameters. The common model combining unit (144) may be configured to combine the results adjusted by the adaptive parameter reflection unit (143) with a common autonomous driving foundation model, while maintaining the basic parameter structure or shared parameter set of the common model. Here, the common autonomous driving foundation model may be a basic inference backbone applicable to multiple road types and may include a common computational structure related to sensor feature interpretation, object relationship inference, path planning, and control command generation. The common model combining unit (144) may distinguish and process the parameter set adjusted by the adaptive module and the common parameter set that must be maintained.

[0181] For example, the common model combination unit (144) can maintain the visual feature extraction layer and basic planning layer of the common autonomous driving foundation model as they are, while correcting only some parameters of a specific intermediate layer according to the result of the adaptation parameter reflection unit (143). As another example, the common model combination unit (144) can always maintain common parameters so that basic inference is possible even when there is no adaptation module, and can additionally activate corrected partial parameters only when an adaptation module is present. According to this structure, one embodiment of the present invention can reduce storage space, model update costs, and verification burden compared to a method of operating completely independent autonomous driving models for each road type.

[0182] The inference input transmission unit (145) is a component that performs the function of transmitting to an output generation layer. The inference input transmission unit (145) can transmit the final inference structure combined in the common model combination unit (144) to the autonomous driving output generation unit (150) or to an output generation layer within the common model. Here, "transmission to an output generation layer" may mean input transmission that enables actual autonomous driving output calculation using the final foundation model state in which the adaptation module is reflected. For example, the inference input transmission unit (145) can transmit the result of a perception-prediction-planning operation in which the adaptation module is reflected to a driving trajectory generation layer, or transmit an internal representation required for generating the final control value to a vehicle control value calculation layer.

[0183] In one embodiment, the inference input transmission unit (145) may not only transmit the inference result of a single point in time, but may also transmit information on internal state changes or the state reflecting the adaptation module over a certain time interval. Accordingly, the autonomous driving output generation unit (150) may generate a driving trajectory or vehicle control command that considers time continuity rather than a discontinuous single inference. For example, if the mixing ratio of the adaptation module changes gradually in a transition interval, the inference input transmission unit (145) may continuously transmit the internal state reflecting such mixing ratio changes, thereby enabling the output generation layer to produce a smoother path and control value.

[0184] According to the structure illustrated in FIG. 7, in one embodiment of the present invention, the foundation model reflection unit (140) is not merely an intermediate interface that stores or transmits the module application combination generated in FIG. 6, but can actually perform the core function of inserting, reflecting, and combining an adaptation module within a common autonomous driving foundation model. The application configuration input unit (141) receives the module application combination, the insertion position determination unit (142) determines the layer or computation path where the adaptation module will operate, the adaptation parameter reflection unit (143) reflects the output or parameter of the adaptation module by adjusting the weights, the common model combination unit (144) combines them while maintaining the common parameter, and the inference input transmission unit (145) can transmit the result to the output generation layer.

[0185] For example, while driving on a general road, the general road adaptation module can be strongly reflected in the object interaction interpretation layer and the cross-flow prediction layer, and when entry into the highway begins, the insertion position determination unit (142) can change so that the highway adaptation module is reflected in some planning layers as well. The adaptation parameter reflection unit (143) can gradually reduce the reflection intensity of the general road adaptation module and gradually increase the reflection intensity of the highway adaptation module, while the common model combination unit (144) can update only the correction parameters while maintaining the basic parameters. The inference input transmission unit (145) can transmit the state reflecting these changes to the output generation layer, thereby preventing the autonomous driving output from changing abruptly.

[0186] As another example, in the case where the general road adaptation module and the highway adaptation module are reflected simultaneously in a meaningful ratio in the transition section, the insertion position determination unit (142) can reflect the general road adaptation module more significantly in the environment analysis layer and the highway adaptation module more significantly in the path stability related layer. The adaptation parameter reflection unit (143) can combine the correction parameters of each of the two adaptation modules by weight at different ratios for each layer. The common model combination unit (144) can enable layer-specific specialized effects while maintaining the computational consistency of the common backbone. This structure is advantageous for simultaneously securing output continuity and environment adaptability compared to a simple module replacement structure.

[0187] As such, according to the foundation model reflection section (140) of FIG. 7, one embodiment of the present invention can provide a specific structure for reflecting at least one adaptation module, which is selected or mixed-controlled, into a common autonomous driving foundation model. In particular, by performing foundation model reflection including not only the selection result of the adaptation module but also the insertion position, reflection strength, hierarchical combination form, and common parameter maintenance structure, the precision of the adjustment of inference characteristics corresponding to changes in road type can be improved.

[0188] In addition, by adopting a structure that adjusts only partial parameters while maintaining a common autonomous driving foundation model, the efficiency of model storage, updating, and verification can be improved compared to a structure that operates separate large models for each road type. Since the reflection intensity and reflection location of the adaptation module can be controlled together, the continuity of autonomous driving output, control stability, and environmental adaptability can be secured at a higher level. The structure of FIG. 7 is combined with the autonomous driving output-based readjustment and rollback structure described later in FIG. 8, demonstrating that an embodiment of the present invention can precisely control the application state of the adaptation module both before and after output generation.

[0190] FIG. 8 is a block diagram illustrating the detailed configuration of a readjustment unit (160) according to one embodiment of the present invention.

[0191] The readjustment unit (160) illustrated in FIG. 8 may include an output state collection unit (161), a stability evaluation unit (162), an uncertainty determination unit (163), a readjustment decision unit (164), and a rollback control unit (165). Each component of FIG. 8 represents a more detailed representation of the function performed by the readjustment unit (160) of FIG. 3. It may be configured to receive the autonomous driving output and related state information generated by the foundation model reflection unit (140) of FIG. 7 and the autonomous driving output generation unit (150) of FIG. 3 as input, and to determine whether to maintain the current adaptive module application configuration, readjust, or restore to the previous configuration. According to such a structure, an embodiment of the present invention can ensure output continuity and driving stability by re-evaluating the actual output results even after selecting or mixing the adaptive module based on the road type.

[0192] Referring to FIG. 8, the output state collection unit (161) is a component that performs the function of receiving driving outputs. The output state collection unit (161) can receive driving trajectories, steering control values, acceleration / deceleration control values, braking control values, lane keeping control values, distance between vehicles keeping information, and collision avoidance related output information generated from the autonomous driving output generation unit (150). The output state collection unit (161) can collect not only the final control command values ​​but also time series information regarding how the outputs change over time, interrelationships between outputs, and information on the rate of change relative to the previous cycle. For example, the output state collection unit (161) can store the amount of change in steering angle commands, the change pattern of acceleration commands, and the continuity of the planned trajectory in a series of consecutive control cycles. As another example, the output state collection unit (161) can intensively collect changes in control outputs during a certain time interval immediately after the mixing ratio of the adaptation module is changed, and provide this as input to analyze how the impact of the transition is reflected in the actual vehicle behavior.

[0193] In one embodiment, the output state collection unit (161) may also receive feedback regarding actual vehicle behavior from the vehicle control device (200) or the vehicle itself state sensor. For example, even if the steering command changes gradually, the change in lateral acceleration in the actual vehicle may be large due to road surface conditions or tire slip, so the output state collection unit (161) may collect the command output and the actual vehicle response together. Accordingly, the subsequent step can perform a more realistic stability evaluation that includes not only the simple command value itself but also the actual vehicle response.

[0194] The stability evaluation unit (162) is a component that performs the function of evaluating output continuity. The stability evaluation unit (162) can determine whether the current autonomous driving output is temporally continuous and stable from the perspective of vehicle control by using the driving output status received from the output status collection unit (161). Here, the output continuity evaluation can be performed based on the steering change rate, acceleration change rate, braking command change rate, curvature change rate of the planned trajectory, variability of lane center following error, or whether there is a sudden discontinuity in the control command. For example, if the steering control value oscillates rapidly immediately after the application rate of the adaptive module is changed, or if the acceleration / deceleration command repeatedly increases or decreases in a short period of time, the stability evaluation unit (162) can determine this as a decrease in output stability.

[0195] In one embodiment, the stability evaluation unit (162) can calculate a stability score by combining jerk, the rate of change in lateral acceleration, the rate of change in longitudinal acceleration, and the amount of change in trajectory curvature. For example, if, immediately after the vehicle lowers the ratio of the general road adaptation module and increases the ratio of the highway adaptation module in the highway entry section, the trajectory curvature suddenly decreases and lane following becomes rigid, or conversely, the steering value shakes excessively, the stability evaluation unit (162) can evaluate that the output state is outside the acceptable range of driving stability. As another example, if the deceleration command changes excessively large during a situation of following a vehicle ahead, the possibility of reduced ride comfort and subsequent control instability increases, so this can be reflected as a stability degradation indicator.

[0196] The uncertainty determination unit (163) is a component that performs the function of calculating output reliability. The uncertainty determination unit (163) can calculate uncertainty information indicating how reliably the autonomous driving output was generated for the current driving environment. The uncertainty information may include object recognition reliability, prediction result variance, deviation between path candidates, anomaly distribution indicator, environmental suitability degradation indicator, or model output consistency degradation indicator. For example, if pedestrian detection reliability drops sharply after a change in the adaptation module, or if planned trajectory candidates diverge significantly from each other under the same input conditions, the uncertainty determination unit (163) may determine that the output reliability has decreased.

[0197] In one embodiment, the uncertainty determination unit (163) can indirectly evaluate the degree of suitability between the combination of adaptive modules reflected in the common autonomous driving foundation model and the current input environment. For example, if pedestrians, traffic lights, and complex intersection objects are continuously detected in the actual input environment while the ratio of highway adaptive modules is high, the uncertainty determination unit (163) can estimate a high probability that the current output is not sufficiently suitable for the environment. As another example, if the classification results of the types of objects ahead are inconsistent or the prediction variance repeatedly increases after a specific combination of adaptive modules is applied, the uncertainty determination unit (163) can calculate a low output reliability. In this way, the uncertainty determination unit (163) can determine the state in terms of the reliability of the model output and suitability for the environment, separately from the results evaluated by the stability evaluation unit (162) based on control continuity.

[0198] The readjustment decision unit (164) is a component that performs the function of recalculating the application ratio. The readjustment decision unit (164) receives the output continuity evaluation result of the stability evaluation unit (162) and the output reliability calculation result of the uncertainty judgment unit (163) as inputs, and can decide whether to maintain the current adaptation module application configuration, adjust the application ratio of each of the multiple adaptation modules, or partially reduce the influence of only a specific adaptation module. For example, if the overall output is stable but the reliability of some object recognition decreases after the application of a specific adaptation module, the readjustment decision unit (164) can recalculate the result by slightly lowering only the application ratio of that adaptation module instead of a full rollback. Conversely, if the steering change rate and uncertainty indicators deteriorate simultaneously, a larger readjustment can be performed.

[0199] In one embodiment, the readjustment decision unit (164) may set a priority for recalculation by adaptation module. For example, if a general road adaptation module, a highway adaptation module, and a transition section adaptation module are mixed simultaneously, if the readjustment decision unit (164) determines that the main cause of the decrease in stability is the excessive weight of a specific module, it may prioritize lowering the application ratio of that module and correct the ratio of other modules. As another example, it may recalculate by temporarily increasing the weight of the transition section adaptation module to restore a smooth connection between the two environments. According to this structure, one embodiment of the present invention can readjust the adaptation configuration precisely according to the cause and situation, rather than resetting the entire configuration in the same way for all unstable situations.

[0200] The rollback control unit (165) is a component that performs the function of restoring the previous configuration. If the rollback control unit (165) determines, based on the judgment result of the readjustment decision unit (164), that it is difficult to maintain the current adaptive module application configuration or that it is difficult to restore stability or reliability through readjustment alone, it can restore the previous adaptive module configuration or a pre-set conservative configuration. Here, the previous configuration may be a combination of adaptive modules used in the previous control cycle, or a reference configuration that operated stably for a certain period of time. The conservative configuration may be a basic configuration centered on a common autonomous driving foundation model, a stability-oriented configuration centered on general road response, or a conservative mixed configuration including a transition section response module.

[0201] For example, if steering command discontinuity and object recognition reliability degradation occur simultaneously after the vehicle rapidly increases the proportion of the highway adaptation module during the process of entering the highway, the rollback control unit (165) can restore the general road-centered mixed configuration used in the previous stable state. As another example, if instability continues to accumulate even after repeatedly readjusting the adaptation module in a specific transition section, the rollback control unit (165) can immediately switch to a preset conservative configuration including the transition section adaptation module to ensure stability. The rollback control unit (165) may set the scope of the rollback to full configuration restoration, or it may set it to partial rollback that returns only the proportion of a specific adaptation module to its previous value.

[0202] In one embodiment, the rollback control unit (165) may also manage re-entry conditions after a rollback. For example, if a specific adaptive module configuration is unstable and restored to a previous state, re-application may be restricted for a certain period of time to prevent immediate switching back to the same configuration. This structure can prevent oscillation phenomena that repeatedly alternate between an unstable configuration and a stable configuration. Additionally, the rollback control unit (165) may store the rollback occurrence history, the input environment immediately prior to the rollback, the instability cause indicator, and the degree of recovery after the rollback to reflect in subsequent readjustment decisions.

[0203] According to the structure of the readjustment unit (160) illustrated in FIG. 8, one embodiment of the present invention may not merely monitor the state after the autonomous driving output is generated, but may perform multi-stage closed-loop control by collecting the output state, evaluating the stability of the output, determining the reliability and uncertainty of the output, and recalculating the application rate of the adaptation module or restoring to the previous configuration based on the results. The output state collection unit (161) collects the autonomous driving output and vehicle response state, the stability evaluation unit (162) evaluates the continuity of steering, acceleration, and trajectory, the uncertainty determination unit (163) calculates the output reliability and environmental suitability, the readjustment decision unit (164) recalculates the application rate, and the rollback control unit (165) may restore to the previous configuration if necessary.

[0204] For example, in a section transitioning from a general road to a highway, if the reflection ratio of the highway adaptation module is increased and the steering change rate is within the allowable range but the object recognition reliability is lower than expected, the readjustment decision unit (164) can recalculate by slightly reducing only the weight of the highway adaptation module and increasing the weight of the general road adaptation module without a full rollback. Conversely, if steering vibration increases and the consistency of the predicted path decreases, the rollback control unit (165) can return to the previous stable configuration. According to this structure, one embodiment of the present invention can continuously monitor the effect of the selection or mixing control of the adaptation module on the output result and correct it immediately if necessary.

[0205] As another example, in a situation where the lane structure changes rapidly and the distribution of surrounding objects becomes complex due to a construction zone while driving on a highway, if the output generated from the existing highway adaptation module center configuration exhibits discontinuous deceleration commands and high prediction variance, the output state collection unit (161) can collect the corresponding output state. The stability evaluation unit (162) can confirm an increase in the rate of change of deceleration commands, and the uncertainty judgment unit (163) can calculate a decrease in output reliability. The readjustment decision unit (164) can decide on readjustment to increase the reflection ratio of the general road adaptation module or the transition section adaptation module. If instability persists even after such readjustment, the rollback control unit (165) can restore to a more conservative stable configuration. In this way, the structure of FIG. 8 provides a cyclic control structure that reflects the actual output results again, rather than using a single adaptation module decision result in a unidirectional manner.

[0206] In this way, the readjustment unit (160) according to FIG. 8 can form a rear-end stabilization structure for adaptive module control in one embodiment of the present invention. Accordingly, even if adaptive module selection or mixed control based on road type-based control information causes unexpected discontinuities, vibrations, or a decrease in reliability in the actual autonomous driving output, this can be immediately detected and the adaptive module application ratio can be readjusted or restored to the previous configuration. Thus, while maintaining a common autonomous driving foundation model, it is possible to flexibly respond to changes in road type and improve the continuity of autonomous driving output, the stability of vehicle behavior, and the overall control reliability.

[0207] The readjustment unit (160) of FIG. 8 operates together with the adaptive module control unit (130) of FIG. 6 and the foundation model reflection unit (140) of FIG. 7 to provide a structure in which the selection, reflection, and output verification of the adaptive module are connected in a single closed loop. According to this structure, unlike a technology that uses the selection result of the adaptive module as is and in a fixed manner, one embodiment of the present invention can continuously optimize the application configuration by reflecting the state after output, thereby enabling more stable and reliable autonomous driving control in various road types, transition sections, and complex environments.

[0209] FIG. 9 is a flowchart illustrating the procedure of an uncertainty decomposition-based adaptive module influence limitation method according to one embodiment of the present invention.

[0210] The procedure according to FIG. 9 may consist of collecting autonomous driving output and evaluation information, detecting instability or uncertainty in the output, analyzing by module which of the multiple adaptation modules caused the instability or uncertainty, limiting the upper limit of reflection for the adaptation module with high influence, and regenerating the autonomous driving output with an application configuration that reflects the result of the limitation. The embodiment of FIG. 9 can be understood as an example of further enhancing the function of the readjustment unit (160) described in FIG. 8, and has technical significance in that, unlike a method of simply rolling back the entire adaptation module configuration in bulk, it breaks down the cause of instability into adaptation module units and precisely limits only the problematic module.

[0211] Referring to FIG. 9, in step S910, the autonomous driving device (100) can collect autonomous driving output and evaluation information. Here, the autonomous driving output may include a target path, steering control value, acceleration / deceleration control value, lane keeping control value, object avoidance related control value, and other driving-related outputs generated with an adaptation module reflected in a common autonomous driving foundation model. The evaluation information may be additional information for evaluating the impact of the autonomous driving output on actual vehicle behavior or the reliability of the output, and may include a steering change rate, longitudinal acceleration change rate, lateral acceleration change rate, jerk, trajectory curvature change amount, object recognition reliability, prediction variance, anomaly distribution indicator, deviation between path candidates, and forward risk calculation value.

[0212] In one embodiment, the autonomous driving device (100) may not only collect the final control command value, but also collect the current application rate of each adaptation module, the amount of change in the adaptation module ratio during the recent control cycle, the internal activation strength per adaptation module, the reflection status by hierarchy, and vehicle response information immediately after the generation of the autonomous driving output. For example, when a mixed configuration of a general road adaptation module 0.4, a highway adaptation module 0.5, and a transition section adaptation module 0.1 is applied, the steering command change rate and the object recognition reliability change may be collected together. By collecting the adaptation module application status and output evaluation information in conjunction in this way, the cause of instability can be analyzed more precisely in a subsequent step.

[0213] In step S920, the autonomous driving device (100) may detect output instability or uncertainty based on the autonomous driving output and evaluation information. Here, output instability may refer to a state in which the autonomous driving output is not temporally continuous or causes abrupt changes in terms of vehicle behavior. Output instability may appear in the form of vibrations in steering control values, abrupt increases or decreases in acceleration / deceleration commands, discontinuities in trajectories, abrupt expansion of lane centering errors, or unstable vibrations in maintaining distance between vehicles. Uncertainty may refer to a state in which the reliability of the output generated by the model or the results of perception, prediction, or planning is reduced, or the suitability to the current input environment is degraded. Uncertainty may appear in the form of reduced object recognition reliability, increased variance of prediction results, increased anomaly distribution indicators, increased deviation between multiple path candidates, or reduced suitability to the environment.

[0214] In one embodiment, the autonomous driving device (100) can determine output instability and uncertainty independently of each other, and, if necessary, can calculate them in the form of an integrated risk level that combines them. For example, if the steering change rate is within an acceptable range but only the object recognition reliability decreases, it can be classified as a situation where uncertainty predominates. Conversely, if the object recognition reliability is maintained but the trajectory curvature vibrates rapidly, it can be classified as a situation where output instability predominates. As another example, if an increase in steering vibration and a decrease in object recognition reliability occur simultaneously, it can be determined as a complex risk state. According to this detection structure, subsequent steps can not only know whether a problem has occurred, but also reflect the nature of the problem.

[0215] In step S930, the autonomous driving device (100) can decompose the contribution of the detected instability or uncertainty by adaptation module. This step is a key feature step of FIG. 9. When output instability or uncertainty occurs while multiple adaptation modules are simultaneously reflected, the autonomous driving device (100) does not treat this solely as a problem at the overall system level, but can decompose and calculate the extent to which each individual adaptation module or combination of modules contributed. Here, the decomposition of contribution can be performed by analyzing the activation strength by adaptation module, the reflection ratio by hierarchy, the recent change in the adaptation module ratio, the prediction bias by adaptation module, and the difference in output before and after module application.

[0216] For example, if steering vibration increases while a general road adaptation module, a highway adaptation module, and a transition section adaptation module are applied simultaneously, the autonomous driving device (100) can estimate which module primarily causes the increase in steering vibration by comparing the difference in output by layer reflecting each adaptation module, the amount of influence of correction parameters by module, the amount of change in path generation before and after reflecting the adaptation module, and the output sensitivity when individual modules are removed. As another example, if pedestrian recognition reliability decreases, the autonomous driving device (100) can determine whether a specific module caused the recognition bias by comparing the contribution of the general road adaptation module and the highway adaptation module to the object analysis layer.

[0217] In one embodiment, the autonomous driving device (100) may not calculate the instability contribution as a single numerical value, but may subdivide it into contributions by output item. For example, the contribution to steering stability, the contribution to acceleration / deceleration continuity, the contribution to object recognition reliability, and the contribution to path planning dispersion may be calculated for each adaptation module. Accordingly, if a specific adaptation module is useful overall but causes problems only in a specific output item, the adaptation module may not be completely excluded in a subsequent step, but only the degree of reflection corresponding to the specific output characteristic may be limited. This structure enables much more precise control than a full rollback method.

[0218] In step S940, the autonomous driving device (100) can identify the top-impact adaptive module based on the contribution decomposition results. The top-impact adaptive module may refer to an adaptive module that has a relatively high contribution to the detected output instability or uncertainty. The autonomous driving device (100) can calculate the contribution ranking or identify an adaptive module that exceeds a preset threshold as the top-impact adaptive module. For example, if the contribution to increased steering vibration is calculated as 0.2 for the general road adaptive module, 0.6 for the highway adaptive module, and 0.2 for the transition section adaptive module, the autonomous driving device (100) can identify the highway adaptive module as the top-impact adaptive module. As another example, if the general road adaptive module shows the greatest contribution to the decrease in object recognition reliability and the transition section adaptive module shows the greatest contribution to the increase in path planning variance, the autonomous driving device (100) can identify different top-impact adaptive modules for each problem type.

[0219] In one embodiment, the identification of an adaptive module with a high impact is not performed solely by a single point-in-time judgment, but may also consider the cumulative contribution over multiple control cycles or the pattern of repeated occurrences. For example, if a specific adaptive module has not shown a high contribution only once, but has been repeatedly identified as the main cause of instability in recent consecutive driving cycles, the autonomous driving device (100) may restrict the said adaptive module more strictly. Conversely, if a specific adaptive module has a high impact only temporarily and then recovers immediately, only a partial upper limit restriction may be applied rather than a full rollback. According to this structure, the identification of an adaptive module with a high impact can lead to a more stable and reliable control judgment.

[0220] In step S950, the autonomous driving device (100) may limit the reflection upper limit of the identified influence level upper adaptation module. Here, the reflection upper limit limit may mean a control that restricts the application rate or reflection intensity of a specific adaptation module from increasing beyond a certain level. The limit may be a method of uniformly lowering the overall reflection rate of the corresponding adaptation module, a method of limiting the reflection intensity only at a specific layer, or a method of lowering the reflection upper limit only at a path that affects a specific output characteristic.

[0221] For example, if the highway adaptation module is identified as the main cause of steering instability, the autonomous driving device (100) may lower the upper limit of the total application rate of the highway adaptation module from 0.6 to 0.35. As another example, if it is determined that the general road adaptation module causes a decrease in object recognition reliability, only the upper limit of the reflection of the general road adaptation module in the object interpretation layer may be limited, while the reflection in the planning layer may be maintained. As yet another example, if output vibration occurs only when the transition section adaptation module increases excessively in the short term, the autonomous driving device (100) may limit only the upper limit of the increase rate instead of lowering the absolute upper limit of the said adaptation module.

[0222] In one embodiment, the autonomous driving device (100) can be configured to continue utilizing the identified adaptive module within a limited range without completely removing it. This is an important feature of the present embodiment. That is, even if an adaptive module has caused a problem, instead of causing the module to lose all effective functions necessary for responding to a specific road type, only the level that contributes excessively to instability can be suppressed. For example, since a highway adaptive module may still be effective in terms of lane keeping stability, instead of making the entire ratio zero, it can be reflected only within a conservative upper limit. According to this structure, environmental adaptability can be maintained better than a full rollback.

[0223] In step S960, the autonomous driving device (100) can regenerate an autonomous driving output based on the above-mentioned limited application configuration. The autonomous driving device (100) can re-apply the adaptation module configuration with the limited reflection upper limit or the modified mixing ratio from step S950 to a common autonomous driving foundation model and regenerate a new driving trajectory, a new steering control value, a new acceleration / deceleration control value, or other autonomous driving outputs based thereon. For example, a new trajectory can be generated by lowering the upper limit of the ratio of the highway adaptation module, which was the main cause of steering instability, and then relatively increasing the ratio of the general road adaptation module and the transition section adaptation module. As another example, the recognition and planning results can be regenerated with a more stable combination of adaptation modules after lowering the upper limit of the reflection of the adaptation module, which had high object recognition uncertainty.

[0224] In one embodiment, the autonomous driving device (100) may re-evaluate the stability or uncertainty of the regenerated autonomous driving output. If instability remains after regeneration, the autonomous driving device (100) may perform additional upper limit restrictions, additional readjustments, or rollback to a conservative configuration. Conversely, if the output regenerated with the restricted application configuration regains continuity and reliability, the autonomous driving device (100) may maintain the restricted configuration for a certain period of time. According to this structure, the present embodiment may be implemented as an adaptive iterative stabilization structure rather than a one-time correction.

[0225] According to the procedure illustrated in FIG. 9, an additional embodiment of the present invention can be distinguished from a method of collectively canceling the entire adaptive module configuration or simply returning to a previous step when a problem occurs in the autonomous driving output. This embodiment collects autonomous driving output and evaluation information, detects output instability or uncertainty, analyzes the cause of the problem by breaking it down by adaptive module, identifies only the adaptive module with the highest influence to limit its reflection, and can regenerate the autonomous driving output with a modified application configuration. Accordingly, since only the module causing the instability can be precisely suppressed and the functions of the remaining valid adaptive modules can be maintained as much as possible, the adaptability to the autonomous driving environment and output stability can be simultaneously improved compared to a full rollback method.

[0226] For example, if increasing the proportion of the highway adaptation module during the process of entering a highway from a general road is advantageous for lane keeping but causes an excessive increase in steering vibration, this embodiment may lower only the upper limit of its reflection instead of removing the entire highway adaptation module. As another example, even if the transition section adaptation module is identified as the cause of increased path planning variance, the reflection intensity at a specific layer may be limited without completely excluding that adaptation module. This structure is particularly advantageous in that it can reduce only the influence causing instability while maintaining the valid advantages of each adaptation module.

[0227] As such, according to the additional embodiment of FIG. 9, one embodiment of the present invention can provide a precision control structure that, in addition to a road type-based adaptive module control structure, decomposes instability or uncertainty into adaptive module units and selectively limits only the upper limit of reflection of the problem module. Accordingly, compared to a simple full rollback or full reset method, high-resolution stabilization through partial influence control becomes possible, and the utilization efficiency and output stability of multiple adaptive modules can be improved while maintaining a common autonomous driving foundation model. In this respect, the embodiment of FIG. 9 can be an important additional embodiment that strengthens the inventive step of the present invention.

[0229] FIG. 10 is a flowchart illustrating the procedure of a method for generating an autonomous driving output based on hierarchical mixing that simultaneously reflects multiple context axes according to an embodiment of the present invention.

[0230] The procedure according to FIG. 10 may consist of collecting driving environment information regarding multiple contexts, separating the information by context axis, generating control information for each axis, determining candidates for the application of an adaptive module corresponding to each axis, calculating a hierarchical blending weight that reflects the influence of multiple axes, and reflecting the hierarchical blending configuration in a common autonomous driving foundation model to generate an autonomous driving output. The embodiment of FIG. 10 has technical significance in that, in addition to the basic invention having a structure for selecting or blending control of an adaptive module based on road type-based control information, it simultaneously reflects weather, time of day, traffic density, and other driving context axes, not just a single axis of road type, and hierarchically combines adaptive modules corresponding to these axes. According to such a structure, the present embodiment can enable autonomous driving control that more precisely reflects different environmental conditions even within a single road type.

[0231] Referring to FIG. 10, in step S1010, the autonomous driving device (100) may collect multiple context driving environment information. Here, the multiple context driving environment information may be an input set containing multiple types of driving environment information having different semantic axes, rather than treating the vehicle's surrounding environment as a single category of information. For example, the multiple context driving environment information may include road type information, weather information, time zone information, traffic density information, illumination information, road surface condition information, regional characteristic information, and vehicle's own driving state information. The road type information may include whether it is a general road, highway, ramp section, interchange section, and intersection section. The weather information may include clear, rain, snowfall, fog, possibility of road surface icing, or wind speed information. The time zone information may include daytime, nighttime, twilight, dawn, or time when backlight is possible. The traffic density information may include the density of vehicles ahead, occupancy rate by lane, number of surrounding objects, average speed distribution, and degree of congestion.

[0232] In one embodiment, the autonomous driving device (100) can integrally collect time and distance information obtained from sensors, map-based road structure information, weather information provided from an external server, the vehicle's current location and time information, surrounding vehicle flow information, and the vehicle's own dynamic state. For example, even within the same highway section, a situation where traffic flow is smooth during clear daytime weather and a situation where traffic density is high and road surface reflection is high during rainy nighttime weather can form different driving contexts. In this embodiment, the autonomous driving device (100) can simultaneously collect these multiple contextual elements and utilize them as inputs for multi-axis control in a subsequent step. Such a structure is advantageous in that it can reflect the complexity of the actual driving environment, which is difficult to explain with only a single road type.

[0233] In step S1020, the autonomous driving device (100) can separate the multiple context driving environment information by context axis. Here, separation by context axis means dividing various information included in a single integrated input set into a road type axis, a weather axis, a time zone axis, a traffic density axis, or other separate semantic axes, so that each axis can be used as a criterion for independent control judgment. For example, from the same input set, information regarding lane structure, speed limit, presence of ramps, and intersection structure can be classified into the road type axis, and information regarding rainfall, visibility, road surface reflection, and wiper operating status can be classified into the weather axis. Information regarding sun altitude, ambient brightness, presence of backlight, and vehicle headlight operating status can be classified into the time zone or illuminance axis. The number of surrounding vehicles, object type distribution, average driving speed, and degree of congestion can be classified into the traffic density axis.

[0234] In one embodiment, the autonomous driving device (100) may be configured so that information belonging to each axis is organized in a manner independent of one another. For example, the judgment result of the road type axis called "Highway," the judgment result of the weather axis called "Rain," and the judgment result of the time axis called "Night" may each be maintained independently. As another example, the autonomous driving device (100) may branch and extract features to be used simultaneously on multiple axes from the same sensor information. For example, lane structure information from camera images may be provided to the road type axis, and illumination and reflection information may be provided to the time axis or weather axis. Such separation of information by axis serves as the basis for individually generating control information and adaptation module candidates corresponding to each axis in a subsequent step.

[0235] In step S1030, the autonomous driving device (100) may generate control information for each context axis. Here, the control information for each axis may be information indicating how the corresponding axis should be reflected in the adaptation module control of the autonomous driving foundation model, corresponding to each context axis. For example, control information for the road type axis may indicate which module among the general road adaptation module, highway adaptation module, or transition section adaptation module should be reflected to a higher degree. Control information for the weather axis may indicate the need for a rain response adaptation module, a fog response adaptation module, or a road surface low friction response adaptation module. Control information for the time of day axis may indicate the reflection intensity of a night response adaptation module or a backlight correction response adaptation module. Control information for the traffic density axis may indicate the need for a congested traffic response adaptation module or a high-speed flow traffic response adaptation module.

[0236] In one embodiment, the autonomous driving device (100) can generate control information in the form of independent scores, reliability, or priority for each axis. For example, in the road type axis, a highway reliability of 0.8 and a transition section reliability of 0.2 can be generated, in the weather axis, a rain response need of 0.7 can be generated, and in the time of day axis, a night response need of 0.9 can be generated. In the traffic density axis, a high-density traffic response need of 0.6 can be generated. When control information for each axis is generated separately in this way, an adaptation module specialized for each axis is individually determined in a subsequent step, and they can be mixed hierarchically rather than simply combined in parallel.

[0237] In step S1040, the autonomous driving device (100) can determine candidates for application of an axis-specific adaptive module based on control information for each axis. Here, an application candidate for an axis-specific adaptive module may refer to a group of adaptive modules or a combination of adaptive modules that is judged to be suitable for the current driving situation for a specific context axis. For example, a highway adaptive module may be determined as an application candidate for the road type axis, a rain response adaptive module may be determined as an application candidate for the weather axis, and a night response adaptive module may be determined as an application candidate for the time of day axis. A high-density traffic response adaptive module may be determined as an application candidate for the traffic density axis.

[0238] In one embodiment, the autonomous driving device (100) may maintain multiple candidates for each axis. For example, in the road type axis, the highway adaptation module and the transition section adaptation module may be maintained as candidates simultaneously, and in the weather axis, the rain response adaptation module and the road surface low friction response adaptation module may be maintained as candidates simultaneously. In the time of day axis, one or both of the night response adaptation module and the backlight response adaptation module may be maintained as candidates. When candidates for each axis are maintained in this manner, the subsequent step may not simply select only one module from each axis, but may consider both internal axis selection and inter-axis mixing together.

[0239] In another embodiment, the autonomous driving device (100) may set different priorities for the application candidates of the adaptation module for each axis. For example, the road type axis and the weather axis, which directly affect autonomous driving safety, may have high priority, while the time of day axis or the traffic density axis may be set as auxiliary adjustment axes. Conversely, in nighttime rainfall situations, the influence of the time of day axis and the weather axis may be reflected as significantly as that of the road type axis. This differential assignment of priority forms the basis for the subsequent calculation of hierarchical mixed weights.

[0240] In step S1050, the autonomous driving device (100) can calculate a hierarchical blending weight. This step is a key feature step of FIG. 10. The autonomous driving device (100) can form a hierarchical blending structure that reflects the relationship and priority between multiple axes, rather than simply performing parallel weighted summation of the adaptation module application candidates for each axis at the same level. Here, hierarchical blending may mean a structure in which a combination of adaptation modules for a specific axis is reflected preferentially at a higher level than a combination of adaptation modules for another axis, or the reflection intensity of a lower axis is finely adjusted within a range determined by the upper axis.

[0241] For example, the autonomous driving device (100) may set the road type axis as the primary upper axis, the weather axis as the secondary correction axis, and the time period axis and traffic density axis as the tertiary detailed adjustment axes. In this case, a basic configuration centered on a highway adaptation module may be formed first according to the road type axis, a rain response adaptation module may be combined correctively on top of the basic configuration, and a night response adaptation module and a high-density traffic response adaptation module may be reflected in the form of additional weight adjustments. As another example, if the weather axis is determined to be in a very dangerous state, for example, heavy rain or a state of reduced visibility, the weather axis may be reconfigured to have a higher priority. According to this structure, even for the same road type, different adaptation module reflection results may be derived depending on the weather or time period.

[0242] In one embodiment, the hierarchical blending weights may be represented as a single global set of weights, or as sets of weights per layer, per module, or per output characteristic. For example, the weight of the weather axis adaptation module may be increased in the perception-related layer, the weight of the road type axis adaptation module may be increased in the path planning-related layer, and the traffic density axis adaptation module may be additionally reflected as a corrective measure in the control output-related layer. As another example, the highway adaptation module may be predominantly reflected in the lane keeping output, and the weight of the rain or night response adaptation module may be highly reflected in the object avoidance output. In this way, the hierarchical blending weights can form a structured blending rule based on axis-specific meaning and output characteristics, rather than a simple average.

[0243] In step S1060, the autonomous driving device (100) can generate an autonomous driving output with a hierarchical mixed configuration. Based on the hierarchical mixed weights calculated in step S1050, the autonomous driving device (100) can reflect adaptation modules corresponding to multiple axes into a common autonomous driving foundation model, and accordingly generate an autonomous driving output including a driving trajectory, steering control value, acceleration / deceleration control value, distance maintenance control value, and a hazard avoidance path. For example, in a situation where a highway, night, rain, and high-density traffic occur simultaneously, the night response adaptation module and the rain response adaptation module may be reflected more strongly in the perception layer on top of the basic configuration centered on the highway adaptation module, and the high-density traffic response adaptation module may be reflected correctively in the planning and control layer. Accordingly, a complex situation response output that is difficult to obtain with just "a single highway adaptation module" can be generated.

[0244] In one embodiment, the autonomous driving device (100) can perform more sophisticated output differentiation within the same road type based on a hierarchical mixed configuration. For example, even in the same general road conditions, a simple configuration centered on a general road adaptation module may be sufficient in low-density traffic conditions with clear daylight. On the other hand, in high-density traffic conditions with rain at night, a configuration hierarchically mixed with a night response adaptation module, a rain response adaptation module, and a congested traffic response adaptation module may be required in addition to the general road adaptation module. According to such an output generation structure, this embodiment can provide much higher environmental suitability than single-axis based adaptive control.

[0245] According to the procedure illustrated in FIG. 10, an additional embodiment of the present invention can provide a structure that hierarchically mixes adaptation modules by simultaneously considering multiple context axes, going beyond a basic structure that selects or mixes adaptation modules based solely on road type. Specifically, information on driving environments in multiple contexts is collected, separated by axis, control information is generated for each axis, and candidates for adaptation modules for each axis are determined. Then, hierarchical mixing weights reflecting the priority and significance of each axis are calculated, and an autonomous driving output is generated accordingly. Such a structure has technical significance in that it goes beyond merely increasing the types of input information and is designed so that each information axis acts with different hierarchical influence on the configuration of the adaptation modules of the autonomous driving foundation model.

[0246] For example, in a typical adaptive module control structure, a single adaptive module can be selected or the mixing ratio determined based solely on a single road type determination, such as "Highway." However, in this embodiment, along with the road type axis of "Highway," the time-of-day axis of "Night," the weather axis of "Rain," and the traffic density axis of "High-density Traffic" are considered simultaneously. In this case, hierarchical mixing can be performed such that basic driving stability is handled by the Highway adaptive module, visibility and response to road surface reflections are handled by the Night and Rain response adaptive modules, and maintaining distance between surrounding vehicles and risk prediction correction are handled by the High-density Traffic response adaptive module. This structure can reflect the complexity of the actual driving environment much more faithfully compared to a single-module or single-axis-based adaptive structure.

[0247] As another example, in a situation where rain begins at dusk and traffic volume increases rapidly while driving on a general urban road, the "General Road Adaptation Module" can be selected based solely on the road type axis. However, in this embodiment, in addition to the General Road Adaptation Module, an adaptation module responding to changes in illumination at dusk, an adaptation module responding to visibility and road surface conditions during rain, and an adaptation module responding to congested traffic can be hierarchically combined. In this case, the road type axis forms the upper-level basic configuration, the weather and time zone axes are strongly reflected in the perception and prediction layer, and the traffic density axis can correct the planning and control layer. As a result, even under the same general road conditions, a much more realistic and stable autonomous driving output can be generated.

[0248] As such, according to the additional embodiment of FIG. 10, one embodiment of the present invention can further expand road type-based adaptive control technology through a hierarchical blending structure that simultaneously reflects multiple context axes. Accordingly, compared to an adaptive module operation structure fixed to a single road type, the reflection structure of the adaptive module can be finely adjusted while maintaining a common autonomous driving foundation model even in real driving environments where weather, time of day, traffic density, and other environmental factors change in a complex manner. Furthermore, by performing hierarchical blending that reflects the priority and role of each axis rather than simple parallel blending, the consistency, environmental suitability, stability, and scalability of the autonomous driving output can be improved together. In this respect, the embodiment of FIG. 10 can serve as an important additional embodiment that strengthens the platform-based scalability and progressiveness of the present invention.

[0250] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0251] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave in order to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer devices and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0252] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0253] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described device, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0254] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

[0255] Although specific embodiments have been described in the detailed description of this document, it will be obvious to those skilled in the art that various modifications are possible within the scope of this document. Explanation of the symbols

[0256] 10 : Vehicle 11 : Sensor 20 : External information providing device 100: Autonomous driving device 110: Driving environment information acquisition unit 120: Control information generation unit 130: Adaptation Module Control Unit 140: Foundation Model Reflection Unit 150: Autonomous driving output generation unit 160: Re-adjustment unit 131: Module Candidate Determination Unit 132: Applicable Module Selection Unit 133: Mixing Ratio Calculation Unit 134: Conversion Condition Reflection Unit 135 : Application Configuration Generation Section 141: Application configuration input section 142: Reflection position determination section 143: Adaptive Parameter Reflection Section 144: Common Model Integration Section 145 : Inference Input Passer 161: Output status collection unit 162: Stability evaluation unit 163: Uncertainty Assessment Unit 164: Readjustment Decision Unit 165 : Rollback control unit 200: Vehicle Control Unit

Claims

Claim 1 A method performed in an autonomous driving device comprises: a step of acquiring driving environment information of a vehicle; a step of generating road type-based control information including reliability information for each of a plurality of road types, including a first road type and a second road type, based on the driving environment information; a step of selecting or blending control at least one adaptive module coupled to a common autonomous driving foundation model based on the control information; and a step of generating an autonomous driving output by reflecting the selected or blended adaptive module in the common autonomous driving foundation model; wherein the step of selecting or blending control of at least one adaptive module includes: a step of detecting a transition section where the road type changes from the first road type to the second road type; and a step of calculating the application ratio of each of the first adaptive module corresponding to the first road type and the second adaptive module corresponding to the second road type based on the reliability information. A method for controlling an adaptive module of a road type-based autonomous driving foundation model, comprising: a step of mixing the first adaptive module and the second adaptive module simultaneously by gradually decreasing the application ratio of the first adaptive module and gradually increasing the application ratio of the second adaptive module in an inference process for generating the autonomous driving output performed in the transition section; and a step of generating the autonomous driving output comprising: a step of reflecting additional parameters provided by the first adaptive module and the second adaptive module into the common autonomous driving foundation model by weighting the sum according to their respective application ratios while maintaining the basic parameters of the common autonomous driving foundation model; and a step of generating the autonomous driving output including a driving trajectory or a vehicle control command using the common autonomous driving foundation model in which the additional parameters are reflected. Claim 2 A method for controlling an adaptive module of a road type-based autonomous driving foundation model, wherein, in claim 1, the step of acquiring driving environment information of the vehicle comprises: a step of collecting sensor information from at least one sensor provided in the vehicle; a step of receiving map information; and a step of collecting driving state information including the speed, acceleration, and steering state of the vehicle. Claim 3 A method for controlling an adaptive module of a road type-based autonomous driving foundation model, wherein, in claim 1, reliability information for each of the plurality of road types is generated based on the result of analyzing road structure information, object distribution information, speed limit information, and merging pattern information included in the driving environment information. Claim 4 delete Claim 5 delete Claim 6 A method for controlling an adaptive module of a road type-based autonomous driving foundation model, wherein the step of selecting or mixing the at least one adaptive module comprises: a step of setting a switching gate condition to suppress output discontinuity due to a change in the adaptive module; and a step of controlling whether to change the at least one adaptive module or the adjustment speed of the application ratio of a plurality of adaptive modules based on the switching gate condition, wherein the switching gate condition includes at least one of a hysteresis condition, a minimum holding time condition, and a switching cost condition. Claim 7 delete Claim 8 A method for controlling an adaptive module of a road type-based autonomous driving foundation model according to claim 1, further comprising: a step of generating stability information or uncertainty information based on the autonomous driving output after generating the autonomous driving output; a step of readjusting the selection result of at least one adaptive module or the application ratio of a plurality of adaptive modules based on the stability information or uncertainty information; and a step of rolling back to a previous adaptive module configuration or a preset conservative configuration if the stability information or uncertainty information satisfies a preset criterion; wherein the stability information or uncertainty information includes at least one of a steering change rate, an acceleration change rate, an output discontinuity, an object recognition reliability, and an anomaly distribution index. Claim 9 A computer-readable storage medium storing a program for executing any one of the methods of paragraphs 1 through 3, 6 and 8 on a computer. Claim 10 The method comprises at least one processor and a memory for storing instructions, wherein when the instructions are executed by the at least one processor, the at least one processor performs the operation of acquiring driving environment information of a vehicle; the operation of generating road type-based control information including reliability information for each of a plurality of road types, including a first road type and a second road type, based on the driving environment information; and the operation of selecting or blending control of at least one adaptive module coupled to a common autonomous driving foundation model based on the control information. The operation of generating an autonomous driving output by reflecting at least one adaptive module selected or mixed-controlled above into the common autonomous driving foundation model; wherein the operation of selecting or mixed-controlling the at least one adaptive module includes detecting a transition section where the road type changes from the first road type to the second road type, calculating the application ratio of each of the first adaptive module corresponding to the first road type and the second adaptive module corresponding to the second road type based on the reliability information, and in the inference process for generating the autonomous driving output performed in the transition section, the operation of mixing the first adaptive module and the second adaptive module simultaneously by gradually decreasing the application ratio of the first adaptive module and gradually increasing the application ratio of the second adaptive module; and the operation of generating the autonomous driving output includes reflecting additional parameters provided by the first adaptive module and the second adaptive module into the common autonomous driving foundation model by weighted summing according to their respective application ratios while maintaining the basic parameters of the common autonomous driving foundation model, and driving using the common autonomous driving foundation model with the additional parameters reflected. An autonomous driving device that performs adaptive module control of a road type-based autonomous driving foundation model, including an operation to generate the autonomous driving output including a trajectory or vehicle control command.

Images