Automatic driving solution platform and application method thereof

Abstract

A method for operating an automatic driving solution platform, according to one embodiment, comprises the steps of: receiving, from a client terminal, a request for a development query necessary for development of a detailed module, which is a part of a plurality of modules included in the automatic driving solution platform; deriving, from the plurality of modules, remaining modules linked to the operation of the detailed module using a main protocol in which the type and time of information to be exchanged between the plurality of modules and conditions for exchanging the information are preset; a step of providing an infrastructure environment for developing or checking the detailed module; and a step of implementing and providing at least one of the functions of the advanced driver assistance system requested by the client of the client terminal using the remaining modules and a detailed module developed or accepted through the infrastructure environment.

Description

Technical Field

[0001] The present invention relates to an autonomous driving solution platform and its application method. Specifically, the present invention provides a client who needs to develop a specific module driven by the autonomous driving solution with a total infrastructure environment required for development or acceptance, thereby enabling any client to quickly and reliably develop the module they need. Background Art

[0002] Unless otherwise indicated herein, what is described in this section is not prior art to the claims in this application and is not admitted to be prior art by inclusion in this section.

[0003] Autonomous driving solutions are systems that enable vehicles to drive themselves without driver intervention. These solutions utilize various sensors, cameras, radar, and other technologies to detect and analyze the surrounding environment, determine, and adjust the vehicle's path. Machine learning and artificial intelligence technologies are part of these solutions. These solutions enable vehicles to safely navigate various road conditions without human intervention.

[0004] An autonomous driving solution for a vehicle consists of multiple modules. For example, this may include a sensing module, a perception module, a planning module, a control module, and an interaction module. Each module is integrated and operated within a single solution (i.e., a platform).

[0005] On the other hand, among autonomous driving solution development companies, some develop all the modules included in autonomous driving solutions, but some focus on the development of specific modules and only develop these specific modules.

[0006] After developing a specific module, a company that develops only a specific module must review not only the development process but also the results. This review process requires not only the specific module but also the remaining modules included in the autonomous driving solution. The reason for this is that, as mentioned above, each module is organically interconnected within a single platform. Specifically, each module in the autonomous driving solution must exchange data while being organically connected, and each module must operate in a predetermined sequence in order to function. Therefore, the entire platform is required not only during the development of a specific module but also during the review of the developed module.

[0007] However, it is not easy for a development company to obtain surplus modules other than the specific modules it is commissioned to develop. Even if these are obtained from outside, it is not easy to operate and adjust surplus modules obtained from outside as needed during the development or acceptance process.

[0008] Prior art literature

[0009] Patent Literature

[0010] Patent Document 1: Korean Patent No. 10-2227287 (March 8, 2021)

[0011] Patent Document 2: Korean Patent Publication No. 10-2022-0100128 (July 15, 2022)

[0012] This patent application is the result of a national project. For reference, the project number is 20024815, and the responsible government agency is the Ministry of Trade, Industry and Energy of South Korea. The project is managed by the Korea Institute for Industrial Technology and Planning. The research project is titled "Autonomous Driving Technology Development Innovation Project," and the specific research topic is "Development of Surrounding Environment Prediction Technology Based on Omnidirectional Multi-Cameras for Level 4 Autonomous Driving Vehicles." The executing agency is the applicant of this patent application, and the research period is from April 1, 2023, to December 31, 2023. Summary of the Invention

[0013] Technical issues

[0014] Problems to be solved according to embodiments include providing a client who wants to develop a specific module driven by an autonomous driving solution with an overall infrastructure environment required for development or acceptance.

[0015] However, the problem to be solved according to one embodiment is not limited to the above-mentioned problem.

[0016] Solutions to the Problem

[0017] According to one embodiment, a method for utilizing an autonomous driving solution platform is executed by including the following steps: a step of receiving a request for a development query required for the development of a detailed module that is part of a plurality of modules included in the autonomous driving solution platform from a client terminal; a step of deriving, from the plurality of modules, the remaining modules that are linked to the actions of the detailed module by using a master protocol that is preset with the type and time of information that needs to be exchanged between the plurality of modules and the condition for exchanging the information; a step of providing an infrastructure environment for developing or accepting the detailed module; and a step of implementing and providing at least one of the functions of an advanced driver assistance system (ADAS) requested by a client of the client terminal by using the remaining modules and the detailed module developed or accepted by the infrastructure environment.

[0018] Furthermore, the method may further include controlling a display unit of the client terminal to display a screen on which at least one of the plurality of modules can be selected as the detailed module.

[0019] In addition, the application method also includes: a step of comparing the first specification of the detailed module included in the development query and the second specification of the corresponding detailed module in the multiple modules based on the main protocol; a step of identifying the differences between the first specification and the second specification from the comparison result; and a step of generating an interface coupling module between the detailed module included in the development query and the remaining modules based on the identified differences. The infrastructure environment can provide an environment in which the detailed module in the development or acceptance process is coupled with the remaining modules through the interface coupling module.

[0020] In addition, the difference may include at least one of a case where the first specification has a condition not included in the second specification, a case where the first specification does not have a condition included in the second specification, and a case where the first specification has a condition different from that of the second specification.

[0021] In addition, the interface coupling module may be generated so that a result of combining a detailed module included in the development query with the interface coupling module is the same as the corresponding detailed module.

[0022] In addition, the interface coupling module can be generated by generative artificial intelligence technology.

[0023] In addition, the development query may include at least one of information about at least one of a plurality of functions supported by the advanced driver assistance system, information about an environment in which the detailed module operates, information about a testing environment for the detailed module, and information about a certification to be received by the detailed module.

[0024] In addition, the advanced driver assistance system may include at least one of forward collision warning (FCW), lane departure warning (LDW), lane keeping assist system (LKAS), automatic emergency braking (AER), adaptive cruise control (ACC), lane keeping assist (LKA), lane centering system (LCS), obstacle detection (OD) and lane detection (LD) systems.

[0025] In addition, the plurality of modules can be implemented in a manner that they can be driven independently of each other.

[0026] In addition, the multiple modules may include perception, fusion, planning, control, interaction, sensing, visualization, security, high-precision map (HD map) or localization.

[0027] In addition, each of the multiple modules can be black-boxed for security purposes, the source code of each module is kept confidential from the client, and the input data and output data of each module can be provided to the client through the main protocol.

[0028] According to another embodiment, a computer program is stored in a computer-readable storage medium, and the computer program can be programmed to execute a method comprising the following steps: a step of receiving a request for a development query required for the development of a detailed module that is part of a plurality of modules included in an autonomous driving solution platform from a client terminal; a step of deriving, from the plurality of modules, remaining modules that are linked to the actions of the detailed module by using a master protocol that is preset with the type and time of information that needs to be exchanged between the plurality of modules and the condition for exchanging the information; a step of providing an infrastructure environment for developing or accepting the detailed module; and a step of implementing and providing at least one of the functions of an Advanced Driver Assistance System (ADAS) requested by a client of the client terminal by using the remaining modules and the detailed module developed or accepted by the infrastructure environment.

[0029] According to another embodiment, an autonomous driving solution platform includes: a communication unit; a memory storing at least one instruction; and a processor. The at least one instruction is executed by the processor, wherein a client terminal requests a development query required for the development of a detailed module that is part of a plurality of modules included in the autonomous driving solution platform; utilizes a master protocol that predefines the type and timing of information to be exchanged between the plurality of modules, as well as conditions for exchanging the information; derives the remaining modules from the plurality of modules that are linked to the actions of the detailed module; provides an infrastructure environment for developing or accepting the detailed module; and utilizes the remaining modules and the detailed module developed or accepted using the infrastructure environment to implement and provide at least one Advanced Driver Assistance Systems (ADAS) function requested by a client of the client terminal.

[0030] Effects of the Invention

[0031] According to one embodiment, an autonomous driving solution platform including all remaining modules may be provided to a developer developing a specific module, so that when the specific module is accepted, the organic connection and dynamic coupling with other modules included in the autonomous driving solution may be confirmed.

[0032] In addition, by combining the developed specific modules with the remaining modules, an appropriate infrastructure environment is provided to test the developed modules in the entire autonomous driving system, thereby enabling the rapid development of autonomous driving solutions.

[0033] Furthermore, the burden of overall development and testing of autonomous driving solutions can be dispersed. This can facilitate collaboration between companies working on autonomous driving solution modules and improve overall development speed and efficiency.

[0034] The effects of the present invention are not limited to the above-described effects, but should be understood to include all effects that can be inferred from the detailed description of the present invention or the constitution of the invention described in the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a diagram illustrating a system for providing an autonomous driving solution platform according to one embodiment.

[0036] Figure 2 is an exemplary block diagram illustrating a configuration of an autonomous driving solution platform according to one embodiment.

[0037] Figure 3 An exemplary flowchart illustrating a training process of a generative language model according to one embodiment is shown.

[0038] Figure 4 The following exemplifies a process provided by an autonomous driving solution platform according to one embodiment.

[0039] Figure 5 Shows details of node configuration and autonomous driving solution platform provided according to one embodiment.

[0040] Figure 6 The following illustrates an example of an application process of an autonomous driving solution platform according to one embodiment. DETAILED DESCRIPTION

[0041] Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings. Regardless of the figure marks, the same or similar constituent elements shall be designated by the same reference marks, and their repeated descriptions shall be omitted. In the following description, the suffixes "module" and "section" used for the constituent elements are given or assigned or used interchangeably only for the convenience of writing the specification, and they themselves do not have mutually distinguishable meanings or functions. In addition, when describing the embodiments disclosed in this specification, when it is judged that the detailed description of the relevant known technology may make the subject matter of the embodiments disclosed in this specification obscure, the detailed description shall be omitted. In addition, the drawings are only used to help facilitate the understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the drawings, but should be understood to include all changes, equivalents or substitutes contained in the spirit and technical scope of the invention.

[0042] Terms including ordinal numbers (such as first, second, etc.) may be used to describe various components, but the components are not limited by these terms. These terms are only used to distinguish one component from another.

[0043] When a component is said to be “connected” or “connected” to another component, it should be understood that it can be directly connected or connected to the other component, but other components may exist between them. Conversely, when a component is said to be “directly connected” or “directly connected” to another component, it should be understood that there are no other components between them.

[0044] In this application, it should be understood that terms such as “including” or “having” are intended to specify the existence of features, numbers, steps, operations, constituent elements, parts or combinations thereof described in the specification, but do not preclude the possibility of the existence or addition of one or more other features, numbers, steps, operations, constituent elements, parts or combinations thereof.

[0045] The "unit" in this specification includes a unit implemented by hardware, a unit implemented by software, and a unit implemented by hardware and software. In addition, one unit may be implemented using two or more hardwares, or two or more units may be implemented by one hardware.

[0046] In this specification, some operations or functions described as being performed by a terminal, device, or equipment may also be performed by a server connected to the terminal, device, or equipment. Similarly, some operations or functions described as being performed by a server may also be performed by a terminal, device, or equipment connected to the server.

[0047] Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

[0048] Figure 1 is a diagram illustrating a system for providing an autonomous driving solution platform according to an embodiment.

[0049] Reference Figure 1 The system for providing an autonomous driving solution platform according to an embodiment may include an autonomous driving solution platform 100 and a client terminal 200. At this time, the autonomous driving solution platform 100 and the client terminal 200 may be connected via a network.

[0050] The term "network" refers to a wireless or wired network. A wireless network may include, for example, at least one of Long-Term Evolution (LTE), LTE-Advanced (LTE-A), Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA), Universal Mobile Telecommunications System (UMTS), Wireless Broadband (WiBro), Wireless Fidelity (WiFi), Bluetooth, Near Field Communication (NFC), and Global Navigation Satellite System (GNSS). A wired network may include, for example, at least one of Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), Recommended Standard 232 (RS-232), Local Area Network (LAN), Wide Area Network (WAN), the Internet, and a telephone network.

[0051] The autonomous driving solution platform 100 is designed to provide clients who wish to develop or validate specific modules within an autonomous driving solution with the complete infrastructure environment required for such development or validation. Clients can then use this infrastructure environment to implement at least one Advanced Driver Assistance System (ADAS) function.

[0052] The autonomous driving solution includes multiple modules, including but not limited to a driving planning module, a perception module, a sensing module, a system module, a control module, or an interaction module.

[0053] In addition, the detailed modules refer to the modules that the client expects to develop or accept among the multiple modules mentioned above. These detailed modules may include at least one module, for example, a perception module and a driving planning module as these detailed modules.

[0054] Depending on the circumstances, the specifications of the detailed module that the client wishes to develop or accept may differ from those of the modules installed on the autonomous driving solution platform 100. For example, the first specification of the detailed module included in the client's development query may differ from the second specification of the corresponding detailed module in the multiple modules. More specifically, this may include situations where the first specification contains conditions not found in the second specification, situations where the first specification does not contain conditions found in the second specification, and situations where the first specification contains conditions different from those found in the second specification. This is because the client may wish to develop a detailed module that belongs only to it.

[0055] However, if the client develops its own detailed module in this way, there may be problems with interfacing with the remaining modules of the autonomous driving solution platform 100, or even integrating them together. This is because there are differences between the client's own detailed module and the remaining modules.

[0056] In this case, according to one embodiment, the autonomous driving solution platform 100 can provide an environment that reflects the individual requirements of all these clients. For example, even if a client requests a specific module that differs from existing modules, the platform can identify the differences between the modules included in the autonomous driving solution platform 100 and these specific modules, and generate modules corresponding to these differences. For example, a coupling module can be generated to interface or integrate the specific module to be developed with the remaining modules within the autonomous driving solution platform 100, and provided to the client. This allows the client to connect or integrate the specific module they desire with the remaining modules within the autonomous driving solution platform 100, allowing them to freely develop, verify, and validate the specific module they develop. This will be described in more detail later.

[0057] On the other hand, the above is feasible because a master protocol is stored in the autonomous driving solution platform 100 according to one embodiment. The master protocol refers to a set of rules that predefine the type and time of information that needs to be exchanged between multiple modules included in the autonomous driving solution platform 100, as well as the conditions for exchanging the information. For example, the master protocol defines when or under what conditions the perception module perceives information about surrounding vehicles or the GPS information of the vehicle itself when the sensing module obtains such information. The driving planning module defines when, under what conditions, or how to establish a driving plan using the information thus perceived. The system module, control module, or interaction module defines what work each module needs to do in the background for smooth operation, what data should be received from each module, how to check the status of each module, and what subsequent measures or actions should be taken based on the inspection results.

[0058] On the other hand, as described below, each of the aforementioned modules is implemented to be independently drivable. Therefore, once a module is identified as at least one detailed module among multiple modules, it is grouped into those detailed modules and the remaining modules. Based on these master protocols, it is possible to derive information such as when and under what conditions the detailed modules should exchange with the remaining modules. Similarly, based on these master protocols, it is also possible to derive information such as when and under what conditions the remaining modules should exchange with the detailed modules. This information can be derived because, as will be described below, the autonomous driving solution platform 100 according to one embodiment utilizes a generative language model. The generative language model disclosed thus far not only allows software or source code to be understood but also generated based on user-required conditions or requests. Using this approach, since multiple modules exist and each detailed specification is defined in the master protocol, when a portion of the modules is defined as detailed modules and the remaining modules, the type, conditions, and timing of information to be exchanged between these detailed modules and the remaining modules can be easily derived or defined using these generative language models.

[0059] On the other hand, the autonomous driving solution platform 100 as described above may be implemented as including Figure 2 The configuration shown is, but not limited to, Figure 2 The autonomous driving solution platform 100 may include a communication unit 110 , a memory 120 , and a processor 130 .

[0060] The communication unit 110 can be implemented by a wired or wireless communication module. The autonomous driving solution platform 100 can communicate with an external terminal, for example, Figure 1 Communicates with the client terminal 200 and the like.

[0061] The memory 120 may be a medium for storing information. These media include at least one type of storage medium selected from the group consisting of flash memory type, hard disk type, multimedia card micro type, card-type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, and optical disk, but are not limited thereto.

[0062] Various information can be stored in these memories 120. For example, the memories 120 may store information obtained by the autonomous driving solution platform 100 from the client terminal 200 and the like via the communication unit 110. Furthermore, these memories 120 may also store a plurality of learning data used for various model or module learning described later.

[0063] In addition, various modules or models can be implemented in the memory 120. When these modules or models are executed by the processor 130 described later, the target function is performed. Each module or model will be described later.

[0064] Next, let's look at the processor 130. First, according to one embodiment, the processor 130 executes at least one instruction stored in the memory 120, thereby performing the technical features of the embodiments of the present invention described below.

[0065] In one embodiment, the processor 130 may be composed of at least one core. In addition, the processor 130 may be used to analyze and / or process data of a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), etc.

[0066] The processor 130 can cause a neural network or model designed using machine learning or deep learning to learn. To this end, the processor 130 can perform computations for learning the neural network, such as processing input data for learning, extracting features from the input data, calculating errors, and updating the weights of the neural network using backpropagation.

[0067] In addition, the processor 130 may also utilize a model implemented in an artificial neural network to perform inference for a predetermined purpose.

[0068] Next, let's look at artificial neural networks. The model in this specification may refer to any form of computer program that works based on a network function, an artificial neural network, and / or a neural network. In this specification, model, neural network, network function, and neural network (neural network) can be used as interchangeable meanings. In a neural network, at least one node is interconnected by at least one link, forming a relationship between input nodes and output nodes within the neural network. In a neural network, the characteristics of the neural network can be determined based on the number of nodes and links, the association relationship between nodes and links, and the weight value assigned to each link. A neural network can be composed of a combination of at least one node. A subset of the nodes that constitute a neural network can constitute a layer.

[0069] In neural networks, deep neural networks (DNN) refer to neural networks that include multiple hidden layers in addition to the input layer and the output layer. Figure 4 An exemplary illustration of this concept is shown in FIG, where the middle hidden layer in the deep neural network consists of one or more, preferably two or more.

[0070] These deep neural networks may include convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory (LSTM) networks, generative pre-trained transformers (GPTs), auto encoders, generative adversarial networks (GANs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), Q networks, U networks, champagne networks, generative adversarial networks (GANs), self-attention networks (transformers), etc.

[0071] Alternatively, according to an embodiment, the deep neural network may be a model learned by transfer learning. Transfer learning refers to a learning method that pre-trains a large amount of unlabeled learning data in a quasi-guided learning or self-learning manner, obtains a pre-learned model (or base part) with a first task through MLM and NSP, and fine-tunes the pre-learned model to be suitable for the second task by learning labeled learning data in a guided learning manner to achieve the target model. As one of the models learned by this transfer learning method, there are BERT (Bidirectional Encoder Representations from Transformers) and the like, but it is not limited thereto.

[0072] Neural networks, including the deep neural networks described above, can learn to minimize output errors. During neural network learning, learning data is repeatedly input into the network, and the output and target errors of the network are calculated. To reduce the error, the error is backpropagated from the output layer of the network to the input layer, thereby updating the weights of each node in the network.

[0073] On the other hand, a model according to one embodiment may be implemented by borrowing at least a portion of a self-attention network. The self-attention network may be composed of an encoder that encodes embedded data and a decoder that decodes the encoded data. The self-attention network may have a structure that receives a series of numbers and, after encoding and decoding steps, outputs a series of data of different types. In one embodiment, the series of data may be processed into a form that can be operated on by the self-attention network. Processing the series of data into a form that can be operated on by the self-attention network may include an embedding process. Expressions such as data tokens, embedded vectors, and embedded tokens may refer to embedded data in a form that can be processed by the self-attention network.

[0074] To enable the self-attention network to encode and decode a series of data, the encoder and decoder in the self-attention network can use an attention algorithm. The attention algorithm can refer to calculating the similarity of at least one key for a given query, reflecting the given similarity on the value corresponding to each key, and then weighting the value reflecting the similarity to calculate the attention value.

[0075] At this point, various attention algorithms can be categorized based on how the query, key, and value are set. For example, attention can be obtained by setting the same query, key, and value, which may mean a self-attention algorithm. Different from this, in order to process a series of input data in parallel, attention can be obtained by reducing the embedding vector dimension and calculating a separate attention head for each segmented embedding vector, which may mean a multi-head attention algorithm.

[0076] In one embodiment, the self-attention network can be composed of modules that execute multiple multi-head self-attention algorithms or multi-head encoder-decoder algorithms. In one embodiment, the self-attention network can also include additional components other than attention algorithms such as embedding, normalization, or normalized exponential (softmax). Methods for configuring a self-attention network using an attention algorithm may include the method disclosed in "Vaswani et al., Attention Is All You Need, 2017 NIPS", which is incorporated into this specification by reference.

[0077] Self-attention networks can be applied to embedding data in various domains, such as natural language, segmented image data, and audio waveforms. As a result, they can transform a series of input data into a series of output data. The ability to transform data with various domains into data that can be processed by self-attention networks is called embedding.

[0078] In addition, the self-attention network can process additional data representing the relative position relationship or phase relationship between a series of input data. Or in a series of input data, a vector representing the relative position relationship or phase relationship between the input data can be additionally reflected, so that a series of input data can be embedded. For example, the relative position relationship between a series of input data may include but is not limited to the word order in a natural language article, the relative position relationship of each segmented image, the time sequence of the segmented audio waveform, etc. The process of adding information representing the relative position relationship or phase relationship between a series of input data can be called positional encoding.

[0079] Below, we will look at various actions or functions that can be performed by the autonomous driving solution platform 100 by executing at least one instruction stored in the memory 120 through the processor 130.

[0080] First, the processor 130 can control the communication unit 110. Thus, the autonomous driving solution platform 100 can communicate with Figure 1 Communicates with the client terminal 200 shown in FIG. 1 to obtain information.

[0081] In addition, the processor 130 may read the above-mentioned data or instructions stored in the memory 120 and store new data or instructions in the memory 120. In addition, the processor 130 may modify or delete the already stored data or instructions.

[0082] In addition, the processor 130 can execute various models or modules stored in the memory 120. These models or modules can be implemented in the aforementioned artificial neural network manner or rule-based manner. For example, they can embody language models, etc. The following language model is discussed below.

[0083] A language model refers to a model generated based on human language. These language models can be obtained in a variety of ways. The language model in one embodiment can be obtained by transfer learning. In transfer learning, as described above, a large amount of unlabeled learning data (ex.copers) is pre-learned (pre-training) by quasi-guided learning or self-learning, and a process of obtaining a pre-learned (pre-trained) model (or base part) with a first task is performed. Then, a process of fine-tuning the pre-learned model is performed to suit the second task. In these fine tunings, the labeled learning data is used for learning in a guided learning manner. In one embodiment, as one of the models learned in a transfer learning manner, BERT (Bidirectional Encoder Representations from Transformers) or GPT (Generative Pre-trained Transformer) are used, but not limited to this.

[0084] Among them, such a language model according to one embodiment may be a generative language model based on GPT-3 or GPT-4. Specifically, GPT-3 is the third-generation language prediction model of the GPT-n series produced by a company called OpenAI. GPT-3 consists of 175 billion parameters, which is more than twice the size of the old version GPT-2 launched in May 2020. This is part of the natural language processing (NLP) system of the pre-trained language. Of course, the language model according to one embodiment is not limited to being implemented by the GPT-n series, and of course it can be implemented by various other generative language models.

[0085] The learning process of the generative language model is as follows Figure 3 As shown. Figure 3 According to one embodiment, the language model may be a model that has been pre-learned (S20) with a general language, or may be a model of multiple types, that is, a model that has been pre-learned based on multimodal data.

[0086] On the other hand, in the fine-tuning of the above-mentioned language model, reinforcement learning by human feedback (RLHF) can be used. RLHF refers to the use of human judgment information for learning in fine-tuning. For example, in the fine-tuning process in RLHF, a learning process using a dialogue set generated by a human and a learning process in which a human selects and ranks multiple outputs generated by the language model can be performed. For more details, refer to Figure 3, a supervised fine-tuned model (SFT) is generated through a human-generated dialogue set (S21). Then, humans select (rank) the results output by these SFT models, which is fed back to the corresponding model reward model (RM) (S22). Then, the proximal policy optimization (PPO) fine-tuning is performed (S23). Among them, the fine-tuning through PPO refers to the reinforcement learning policy algorithm, which refers to an algorithm that continuously adjusts the current policy based on the work performed by the agent and the compensation obtained. In these PPOs, the order is new prompt->PPO->Generate output->Calculate reward. After updating Calculate reward, it is provided again with a new prompt.

[0087] Next, refer to Figure 1 Client terminal 200 is a client of a development company that develops a specific module included in the autonomous driving solution. In an embodiment, client terminal 200 can request a development query required for the development of a specific module through autonomous driving solution platform 100. In an embodiment, a development query is a set of data required for the development and acceptance of a specific module.

[0088] In an embodiment, the development query may include at least one of information about a plurality of functions supported in an Advanced Driver Assistance System (ADAS), information about the environment in which the detailed module operates, information about the test environment of the detailed module, and information about a certification to be received by the detailed module, but is not limited thereto. In addition, according to an embodiment, the client can select the detailed module that it wants to develop through the client terminal 200. For example, a screen in which at least one of the plurality of modules of the autonomous driving solution platform 100 can be selected as the detailed module can be displayed on the display unit of the client terminal, but this can be achieved through the control of the above-mentioned autonomous driving solution platform 100, but is not limited thereto.

[0089] Below, let’s look at a specific example of a development or verification process performed by the autonomous driving solution platform 100.

[0090] Figure 4 2 is a diagram for explaining services provided by an autonomous driving solution platform according to an embodiment.

[0091] Reference Figure 4The specific module the client wants to develop is the Recognition module. The remaining modules required for the Perception module's operation are the Decision and Control modules. As mentioned above, these are provided to the client as the "infrastructure environment" via the autonomous driving solution platform 100.

[0092] Whether the perception module's actions require the judgment module and the control module can be derived using the aforementioned master protocol. This is because, as mentioned above, the types and timing of information that need to be exchanged between multiple modules, as well as the conditions for exchanging such information, are predefined in these master protocols. This derivation can be performed in a variety of ways, including, as mentioned above, through a generative language model.

[0093] The source code of the judgment module and control module provided as the infrastructure environment can be kept private. Therefore, developers of the autonomous driving solution platform 100 do not need to worry about the leakage of their trade secrets.

[0094] However, the protocols for the action conditions of the judgment module and the control module will be made public. Therefore, the client can adjust the infrastructure environment according to their own preferences for development or acceptance.

[0095] On the other hand, the perception module, which is a detailed module that the client wants to develop, can be used with Figure 3 The platform provider may have different detailed specifications for the perception modules owned by the platform provider. For example, the first specification of the detailed module included in the client's development query and the second specification of the corresponding detailed module in the multiple modules may be different from each other. More specifically, there may be cases where the first specification has conditions that are not included in the second specification, cases where the first specification does not have conditions that are included in the second specification, and cases where the first specification has conditions that are different from those of the second specification. This is because the client may want to develop a detailed module that belongs only to it.

[0096] However, if the client develops its own detailed module, problems may arise in interfacing or integrating it with the remaining modules of the autonomous driving solution platform 100. This is because there are differences between the client's own detailed module and the remaining modules.

[0097] Even in this case, according to one embodiment, the autonomous driving solution platform 100 can be provided to reflect all of the individual requirements of these clients. For example, even if a client requests a specific module that differs from an existing module, the platform can identify the differences between the modules included in the autonomous driving solution platform 100 and these specific modules, and generate modules corresponding to these differences. For example, a coupling module that enables an interface or integration between the specific module to be developed and the remaining modules within the autonomous driving solution platform 100 can be provided to the client. This allows the client to develop their desired specific module while connecting or integrating it with the remaining modules within the autonomous driving solution platform 100 and to fully verify and validate the specific module thus developed.

[0098] That is, these interface coupling modules can be generated in such a way that the result of combining the detailed modules included in the development query with the interface coupling module is identical to the corresponding detailed modules. Furthermore, during this generation process, the generative language model described above can be used. As described above, based on the generative language model disclosed to date, not only can software or source code be understood, but it can also be generated based on the conditions or requests desired by the user. Using this method, since there are multiple modules and the detailed specifications of each module are defined in the master protocol, when a portion is defined as a detailed module and the remaining modules, if there is a difference in the specifications between the detailed module expected by the client and the detailed module provided by the platform, the result of combining the detailed module included in the development query with the interface coupling module can be generated to be identical to the corresponding detailed module. Furthermore, the type, conditions, or timing of information that needs to be exchanged between the generated result and the remaining modules can be easily derived or defined using these generative language models and the master protocol described above.

[0099] This allows clients to easily address the difficulties they encounter in developing detailed modules. Furthermore, the autonomous driving solution platform 100 uses Docker to provide an infrastructure environment, enabling clients, such as development companies, to easily use it in other environments. In an embodiment, Docker is a platform for developing, distributing, and executing containerized applications, allowing applications to be packaged and distributed in a standardized environment. In an embodiment, a container is a software package that includes all the code, libraries, and settings required to run an application. Docker provides the tools and platform for building and managing these containers, making them easy to run in a variety of environments.

[0100] Figure 5 2 is a diagram illustrating a node configuration of an autonomous driving solution and details provided by an autonomous driving solution platform according to an embodiment.

[0101] In the autonomous driving solution according to the embodiment, multiple modules are implemented independently. In the embodiment, the autonomous driving solution can implement various modules that perform various functions such as simulation, verification, control, sensing, and system architecture management. The modules configured in the autonomous driving solution are not limited to the modules mentioned above.

[0102] Furthermore, the independently configured modules within the autonomous driving solution are black-boxed for security reasons, and all source code for each module remains confidential. However, the input and output data for each module are predefined to enable the generation of an infrastructure environment. In this embodiment, while the autonomous driving solution platform does not disclose the internal logic of each module configured within the autonomous driving solution, it does predefine the input and output of each module. This allows clients from other companies to utilize the remaining modules provided within the autonomous driving solution.

[0103] In addition, if Figure 5 As shown, each module configured in the autonomous driving solution according to an embodiment can be nodeized and implemented as a node. For example, the autonomous driving solution platform can configure each module separately in the software architecture and implement it as an independent unit node. Nodes minimize interactions or dependencies between modules, making them easier to reuse.

[0104] Reference Figure 5 According to an embodiment, the autonomous driving solution may include a simulation node 110, a verification node 120, a control node 130, a sensor node 140, and a system architecture node 150. In an embodiment, the autonomous driving solution platform may node each module in advance and store it.

[0105] In an embodiment, the simulation node 110 performs vehicle simulation. To this end, the simulation node 110 may be equipped with a vehicle and autonomous driving simulation solution (e.g., CARMAKER, etc.). In an embodiment, the vehicle and autonomous driving simulation solution provides an actual vehicle simulation environment. In addition, the simulation node 110 may be equipped with MORAI, CARLA, etc., which provide simulation validation and learning (SVL), autonomous driving and ADAS (advanced driver assistance system) testing and simulation solutions. In an embodiment, CARLA is an open source simulation platform for autonomous driving algorithm development and testing. In an embodiment, the simulation node 110 simulates the actual road environment, simulates vehicles and road infrastructure, and executes various autonomous driving solutions through the equipped solutions.

[0106] Verification node 120 verifies the effectiveness of the autonomous driving system. In this embodiment, verification node 120 is equipped with an open-source robotics software framework for developing autonomous driving systems, such as RQT (ROS Qt GUI Plugin), and provides a data logging and playback format, ROS Bag. Furthermore, it provides unit testing procedures and stores real-time data or events through real-time logging.

[0107] The control node 130 is a node that controls vehicle communication protocols such as the CAN bus, and the sensor node 140 is a node that senses the vehicle's surrounding environment. In an embodiment, the sensor node 140 senses the vehicle's surrounding environment through sensors installed on the vehicle, such as front, rear, and side cameras, Lidar, Radar, and ultra sonic. The system architecture node 150 manages the interactions between the nodes. In addition, in an embodiment, the system architecture node 150 can be equipped with the Automotive Open System Architecture (AUTOSAR) to provide a platform for automotive software development. In an embodiment, the system architecture node 150 manages the interactions between the nodes to implement Adaptive Cruise Control (ACC) and Classic Cruise Control (CCC). In an embodiment, each node exchanges data through protocols including Data Distribution Service (DDS) and Robot Operating System 2 (ROS2).

[0108] In one embodiment, the autonomous driving solution platform provides the functionality of each module of the autonomous driving solution, enabling the integrated operation of detailed modules and other modules. To this end, the autonomous driving solution platform exports the details corresponding to the development query requested by the client and selectively extracts the input and output data and source code of the details to configure the infrastructure environment. The configured infrastructure environment is then provided to the client terminal.

[0109] In an embodiment, Figure 5 As shown, the details include but are not limited to perception, fusion, planning, control, interaction, sensing, visualization, security, high-precision map, and localization.

[0110] Figure 6 FIG is a diagram showing the application process of the autonomous driving solution platform according to an embodiment. Figure 6 These flow charts shown can be Figure 1 The autonomous driving solution platform 100 shown in FIG. 1 is executed, but is not limited thereto. In addition, this flowchart is only an example, and the concept of the present invention is not limited thereto. For example, according to an embodiment, Figure 6 The steps may be performed in a different order than shown, or at least one step not shown may be added. Figure 6 The steps shown in .

[0111] Reference Figure 6 The step (S100) of receiving a development query request required for developing a detailed module as part of a plurality of modules included in the autonomous driving solution platform from a client terminal is performed. The plurality of modules may be implemented in a manner capable of being driven independently of each other.

[0112] In addition, the development query may include, but is not limited to, at least one of information about at least one of a plurality of functions supported by an Advanced Driver Assistance System (ADAS), information about an environment in which the detailed module operates, information about a test environment for the detailed module, and information about certifications to be received by the detailed module. The aforementioned functions may include, but are not limited to, at least one of Forward Collision Warning (FCW), Lane Departure Warning (LDW), Lane Keeping Assist System (LKAS), Autonomous Emergency Braking (AER), Adaptive Cruise Control (ACC), Lane Keeping Assist (LKA), Lane Centering System (LCS), Obstacle Detection (OD), and Lane Detection (LD) systems.

[0113] Furthermore, the aforementioned detailed modules may be selected on the client side via the client terminal 200. For example, a screen may be displayed on the display unit of the client terminal, allowing selection of at least one of the plurality of modules of the autonomous driving solution platform 100 as the detailed module. This may be achieved through control of the aforementioned autonomous driving solution platform 100, but is not limited thereto.

[0114] Then, a step (S200) is performed to derive the remaining modules linked to the actions of the detailed modules from the multiple modules using a master protocol preset with the type and time of information to be exchanged between the multiple modules and the conditions for exchanging the information.

[0115] The main protocol here has been explained, so additional explanation is omitted.

[0116] Then, a step (S300) of providing an infrastructure environment for developing or accepting the detailed module is performed. At this time, according to an embodiment, the infrastructure environment may further include the interface coupling module described above. That is, when the first specification of the detailed module included in the development query is compared with the second specification of the corresponding detailed module in the multiple modules based on the master protocol, if there are differences, the above-mentioned interface coupling module can be generated based on these differences. At this time, these interface coupling modules may be generated in a manner such that the result of combining the detailed module included in the development query and the interface coupling module is the same as the corresponding detailed module. In addition, such generation can also utilize generative artificial intelligence.

[0117] On the other hand, as described above, the above-mentioned differences may include at least one of the following: a case where the first specification has a condition that is not included in the second specification; a case where the first specification does not have a condition that is included in the second specification; and a case where the first specification has a condition that is different from that of the second specification.

[0118] Then, a step (S400) is performed to implement and provide at least one of the Advanced Driver Assistance Systems (ADAS) functions requested by the client of the client terminal using the remaining modules and the detailed modules developed or accepted through the infrastructure environment. That is, upon completion of development or acceptance of the detailed modules, these detailed modules, in combination with the remaining modules, implement and provide at least one of the ADAS functions. The implementation and provision of the at least one ADAS function may be provided in the form of a product solution, for which there are various known methods, and a detailed description thereof is omitted.

[0119] As described above, according to one embodiment, a developer developing a specific module may be provided with an environment identical to the autonomous driving solution platform including all remaining modules, so that upon acceptance of the specific module, the organic connection and dynamic coupling with other modules included in the autonomous driving solution may be confirmed.

[0120] In addition, by combining the developed specific modules with the remaining modules and providing an appropriate infrastructure environment to test the developed modules in the entire autonomous driving system, the development of autonomous driving solutions can be quickly achieved.

[0121] Furthermore, through the embodiments, the test results of specific modules are completed and produced and sold, thereby maximizing the profits of autonomous driving solution companies.

[0122] Furthermore, the burden of development and testing can be dispersed, facilitating collaboration between companies working on autonomous driving solution modules and improving overall development speed and efficiency.

[0123] On the other hand, the method according to the various embodiments described above can be implemented in the form of a computer program stored in a computer-readable storage medium, wherein the program is programmed to perform the various steps of the method, and can be implemented in the form of a computer-readable storage medium storing a computer program, wherein the program is programmed to perform the various steps of the method.

[0124] The above description is merely an example of the technical concept of the present invention. Any person skilled in the art in the technical field to which the present invention belongs can make various modifications and variations without departing from the essential quality of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical concept of the present invention, but to illustrate that the scope of the technical concept of the present invention is not limited according to these embodiments. The scope of protection of the present invention should be interpreted by the claims, and all technical concepts within the equivalent scope should be interpreted as included within the scope of the rights of the present invention.

Claims

1. A method for using an autonomous driving solution platform, characterized in that: include: a step of receiving, from a client terminal, a request for a development inquiry required for development of a detailed module that is part of a plurality of modules included in the autonomous driving solution platform; The step of deriving the remaining modules linked to the actions of the detailed modules from the multiple modules by using a master protocol preset with the types and timings of information to be exchanged between the multiple modules and the conditions for exchanging the information; Provide steps for developing or accepting the infrastructure environment of the detailed module; as well as A step of implementing and providing at least one of the functions of the advanced driver assistance system requested by the client of the client terminal by utilizing the remaining modules and the detailed modules developed or accepted by the infrastructure environment.

2. The method for using the autonomous driving solution platform according to claim 1, characterized in that: The method further comprises: The step of controlling the display unit of the client terminal to display a screen on which at least one of the plurality of modules can be selected as the detail module.

3. The method for using the autonomous driving solution platform according to claim 1, characterized in that: The application method also includes: a step of comparing, based on the master protocol, a first specification of a detailed module included in the development query and a second specification of a corresponding detailed module in the plurality of modules; a step of identifying differences between the first specification and the second specification from the comparison result; and generating a coupling module for an interface between the detailed module included in the development query and the remaining modules based on the identified differences, The infrastructure environment provides an environment in which detailed modules in the development or acceptance process are coupled with the remaining modules using coupling modules via the interfaces.

4. The method for using the autonomous driving solution platform according to claim 3, wherein: The difference includes at least one of a case where the first specification has a condition not included in the second specification, a case where the first specification does not have a condition included in the second specification, and a case where the first specification has a condition different from that of the second specification.

5. The method for using the autonomous driving solution platform according to claim 3, characterized in that: The interface coupling module is generated so that a result of combining the detailed module included in the development query with the interface coupling module is the same as the corresponding detailed module.

6. The method for using the autonomous driving solution platform according to claim 3, characterized in that: The interface coupling module is generated by generative artificial intelligence technology.

7. The method for using the autonomous driving solution platform according to claim 1, characterized in that: The development query includes at least one of information about at least one of a plurality of functions supported by the advanced driver assistance system, information about an environment in which the detailed module operates, information about a testing environment for the detailed module, and information about certification to be received by the detailed module.

8. The method for using the autonomous driving solution platform according to claim 7, characterized in that: The advanced driver assistance system includes at least one of forward collision warning, lane departure warning, lane keeping assist system, automatic emergency braking, adaptive cruise control, lane keeping assist, lane centering system, obstacle detection and lane detection system.

9. The method for using the autonomous driving solution platform according to claim 1, characterized in that: The plurality of modules are implemented in such a manner that they can be driven independently of each other.

10. The method for using the autonomous driving solution platform according to claim 1, characterized in that: The multiple modules include perception, fusion, planning, control, interaction, sensing, visualization, safety, high-precision mapping or mapping and positioning.

11. The method for using the autonomous driving solution platform according to claim 1, characterized in that: Each of the multiple modules is black-boxed for security purposes, and the source code of each module is kept confidential from the client. Input data and output data of each module can be provided to the client through the main protocol.

12. A computer program stored in a computer-readable storage medium, characterized in that The computer program is programmed to perform a method comprising the steps of: a step of receiving, from a client terminal, a request for a development inquiry required for development of a detailed module that is part of a plurality of modules included in the autonomous driving solution platform; The step of deriving the remaining modules linked to the actions of the detailed modules from the multiple modules by using a master protocol preset with the types and timings of information to be exchanged between the multiple modules and the conditions for exchanging the information; Provide steps for developing or accepting the infrastructure environment of the detailed module; as well as A step of implementing and providing at least one of the functions of the advanced driver assistance system requested by the client of the client terminal by utilizing the remaining modules and the detailed modules developed or accepted by the infrastructure environment.

13. An autonomous driving solution platform, characterized in that: include: Department of Communications; a memory storing at least one instruction; as well as processor, The at least one instruction is executed by the processor, The client terminal requests a development inquiry required for the development of a detailed module as part of a plurality of modules included in the autonomous driving solution platform. deriving the remaining modules of the plurality of modules that are linked to the actions of the detailed module by using a master protocol that is preset with the types and timings of the information that need to be exchanged between the plurality of modules and the conditions for exchanging the information; Provide the infrastructure environment for developing or accepting the detailed modules, At least one of the functions of the advanced driver assistance system requested by the client of the client terminal is implemented and provided by using the remaining modules and the detailed modules developed or accepted by the infrastructure environment.

Images