Method and apparatus for generating driving scenario simulation

Abstract

The present invention relates to a method and apparatus for generating a driving scenario simulation. A method according to one embodiment of the present invention is a method performed in an electronic device to generate a driving scenario simulation of a vehicle, comprising: a step of extracting a plurality of driving information related to the driving of a reference vehicle from input information; a step of generating a prompt for each driving information; a step of generating a driving scenario simulation by inputting the generated prompt as input data into a language model; and a step of retraining the language model while verifying the generated driving scenario simulation using a simulator.

Description

Technology Field

[0001] The present invention relates to a driving scenario simulation generation technology, and more specifically, to a technology for generating a specific driving scenario simulation related to input information when input information related to a vehicle's driving situation is given. Background Technology

[0003] To develop human-driven vehicles or autonomous vehicles that minimize human control, various functional, performance, or safety evaluations are required. Conventionally, for these evaluations, driving scenarios (i.e., driving scenarios) are prepared in advance, and various evaluations are performed on a standard vehicle according to the prepared scenarios.

[0004] In particular, to generate driving scenarios capable of being simulated (hereinafter referred to as "driving scenario simulations"), conventional coding work is performed by developers writing programming code based on markup languages ​​such as XML, Python, or C. According to this conventional technology, if additional driving scenario simulations are required beyond those already generated, there is a problem in that developers must perform coding work anew. In other words, conventional technology has limitations in generating numerous driving scenario simulations that may occur in real-world driving situations.

[0005] Meanwhile, generative artificial neural networks equipped with Transformer neural network structures, such as OpenAI’s ChatGPT or Google’s BARD, provide appropriate responses to input sentences. In particular, these generative artificial neural networks also provide the function of generating relevant program code for input prompt sentences. In this case, the generative artificial neural network continuously outputs tokens that are probabilistically highly associated with the input prompt sentences based on information within the network that has been trained in advance. However, although the program code output according to this method appears plausible on the surface, there is a problem in that errors occur and the code does not function when actually executed, or results different from the original intention are produced.

[0006] However, the above description merely provides background information regarding the present invention and does not constitute previously disclosed technology. The problem to be solved

[0008] In order to solve the problems of the conventional technology described above, the present invention aims to provide a technology that generates a specific driving scenario simulation related to input information when input information related to a vehicle's driving situation is given.

[0009] In addition, another objective of the present invention is to provide a technology that improves the quality of output results by attempting to execute a driving scenario simulation, which is a result generated by a language model of a generative artificial neural network, using a simulator that actually runs a driving scenario simulation, first verifying whether it can be executed because there are no grammatical errors, and secondly testing whether it matches the input scenario, and outputting results only for those results that pass the test.

[0010] However, the problems that the present invention aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by those skilled in the art to which the present invention belongs from the description below. means of solving the problem

[0012] A method according to an embodiment of the present invention for solving the above-mentioned problem is a method performed in an electronic device to generate a driving scenario simulation of a vehicle, comprising: a step of extracting a plurality of driving information related to the driving of a reference vehicle from input information; a step of generating a prompt for each driving information; a step of generating a driving scenario simulation by inputting the generated prompt as input data into a language model; and a step of retraining the language model while verifying the generated driving scenario simulation using a simulator.

[0013] The type of driving information above varies depending on whether the input information is in a first or second input state, the first input state is a state in which either a video or an accident report file is provided as input information but no additional text is provided, and the second input state may be a state in which either a video or an accident report file is provided as input information but additional text is provided.

[0014] In the case of the second input state above, first driving information is extracted from a photo, video, or accident report file and second driving information is extracted from the additional text, and the second driving information may each include first information that overlaps with at least one item of the first driving information and second information that does not overlap with the first driving information.

[0015] In the step of generating the above prompt, if there are overlapping items among each extracted driving information, fused information can be generated for the overlapping items to generate a prompt for the fused information.

[0016] In the step of generating the above prompt, for the overlapping items, the one with the larger weight can be selected according to each pre-set weight to generate the fusion information.

[0017] In the step of generating the above prompt, the fusion information can be generated by reflecting the ratio according to each pre-set weight for the overlapping items.

[0018] In the step of generating the above prompt, the prompt can be generated by listing words corresponding to each extracted driving information.

[0019] In the step of generating the above prompt, the prompt can be generated by entering the value of each extracted driving information into a selected template format.

[0020] In the step of generating the above driving scenario simulation, result text describing the driving scenario simulation in detail can be generated together.

[0021] In the above retraining step, the simulator can test for grammatical errors, execution status, and relevance with the input information regarding the generated driving scenario simulation, and if an error is found, the corresponding error code can be fed back to retrain the language model so that the error does not occur.

[0023] An apparatus according to one embodiment of the present invention includes a control unit that controls the generation of a vehicle driving scenario simulation using information stored in the memory.

[0024] An apparatus according to another embodiment of the present invention includes: a communication unit that receives input information and transmits a driving scenario simulation generated by a language model for the received input information; and a control unit that controls the generation of a driving scenario simulation of a vehicle using information stored in the memory.

[0025] The above control unit can control the extraction of multiple driving information related to the driving of a reference vehicle from input information, control the generation of a prompt for each driving information, control the generation of a driving scenario simulation by inputting the generated prompt as input data into a language model, and control the retraining of the language model while verifying the generated driving scenario simulation using a simulator. Effects of the invention

[0027] The present invention, configured as described above, has the advantage of easily generating a specific driving scenario simulation related to the input information when input information related to the driving situation of a vehicle is given.

[0028] In addition, the present invention has the advantage of improving the quality of the output result by attempting to execute a driving scenario simulation, which is a result generated by a language model of a generative artificial neural network, using a simulator that actually runs a driving scenario simulation, first checking whether it can be executed because there are no grammatical errors, and second testing whether it matches the input information, and outputting the result only for the result that passes the test.

[0029] The effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below. Brief explanation of the drawing

[0031] FIG. 1 shows a block diagram of a device (100) according to one embodiment of the present invention. FIG. 2 shows a block diagram of the control unit (150) in the driving scenario device (100) according to one embodiment of the present invention. FIG. 3 shows a flowchart of a generation method according to one embodiment of the present invention. Specific details for implementing the invention

[0032] The above-mentioned objectives, means, and resulting effects of the present invention will become clearer through the following detailed description in conjunction with the attached drawings, and accordingly, a person skilled in the art to which the present invention pertains will be able to easily implement the technical concept of the present invention. Furthermore, in describing the present invention, if it is determined that a detailed description of known technology related to the present invention may unnecessarily obscure the essence of the present invention, such detailed description will be omitted.

[0033] The terms used herein are for describing the embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form as appropriate unless specifically stated otherwise in the text. In this specification, terms such as "comprising," "providing," "making arrangements," or "having" do not exclude the presence or addition of one or more other components in addition to the components mentioned.

[0034] In this specification, terms such as "or", "at least one," etc., may indicate one of the words listed together or a combination of two or more. For example, "A or B", "at least one of A and B" may include only one of A or B, or may include both A and B.

[0035] In this specification, descriptions following "e.g." should not limit the embodiments of the invention according to various embodiments of the invention, such as variations including tolerances, measurement errors, limits of measurement accuracy, and other commonly known factors, as the information presented, such as cited characteristics, variables, or values, may not exactly match.

[0036] In this specification, where it is stated that a component is 'connected' or 'connected' to another component, it should be understood that it may be directly connected or connected to the other component, or that there may be other components in between. On the other hand, when it is mentioned that a component is 'directly connected' or 'directly connected' to another component, it should be understood that there are no other components in between.

[0037] In this specification, where a component is described as being 'on' or 'in contact' with another component, it should be understood that it may be in direct contact with or connected to the other component, but that another component may exist in between. On the other hand, where a component is described as being 'immediately above' or 'in direct contact' with another component, it should be understood that no other component exists in between. Other expressions describing the relationship between components, such as 'between' and 'directly between', may be interpreted in the same way.

[0038] In this specification, terms such as 'first,' 'second,' etc., may be used to describe various components, but such components should not be limited by these terms. Furthermore, these terms should not be interpreted as limiting the order of each component, but may be used for the purpose of distinguishing one component from another. For example, 'first component' may be named 'second component,' and similarly, 'second component' may be named 'first component.'

[0039] Unless otherwise defined, all terms used in this specification may be used in a meaning commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0041] Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the attached drawings.

[0042] FIG. 1 shows a block diagram of a device (100) according to one embodiment of the present invention.

[0043] A device (100) according to one embodiment of the present invention (hereinafter referred to as "the device") is a device capable of computing and is a device for generating specific driving scenario simulations related to given input information. In this case, the driving scenario simulation is a programming code written in a markup language such as XML, Python, or C, and is a programming code for a driving scenario that can be simulated by a computer. In the case of the prior art, the corresponding programming code was written directly by a developer, but in the present invention, the corresponding programming code can be generated by the device (100).

[0044] Of course, the device (100) can also generate detailed description text in addition to the driving scenario simulation. In this case, the detailed description text corresponds to text that explains in detail the corresponding driving scenario simulation of the driving scenario simulation, and may also be referred to as "result text."

[0045] That is, the device (100) can generate a driving scenario simulation related to text, photos, videos, or accident report files in which the representation of the given input information is ambiguous (abstract), and can also generate a detailed description related to the driving environment, situation, state, or features of the driving scenario simulation as a result text. As such result text is provided, there is an advantage that the user can understand the driving scenario simulation in advance without using the generated driving scenario simulation.

[0046] For example, the electronic device may be a general-purpose computing device such as a desktop PC, laptop PC, tablet PC, netbook computer, workstation, PDA, smartphone, smartpad, or mobile phone, or a dedicated embedded system, but is not limited thereto.

[0047] In this case, the input information may be text, photos, videos, or accident report files related to the vehicle's driving situation. In particular, photos or videos may be obtained from the vehicle's driving footage. For example, the driving footage may be footage captured by a video collection device, such as a black box, installed in the vehicle while it is in motion; it may be captured by a single vehicle or by multiple vehicles, but is not limited thereto.

[0048] In addition, an accident report file is a text file that records the detailed circumstances of an accident when a vehicle comes into contact with another vehicle, object, or person while driving.

[0049] In the present invention, a state may occur in which any one of a photograph, a video, and an accident report file is provided as input information, but no additional ambiguous (abstract) text is provided (hereinafter referred to as the "first input state"), and a state in which, in addition to any one of the photograph, a video, and an accident report file, additional text with ambiguous expression is provided (hereinafter referred to as the "second input state"). In this case, the text with ambiguous expression corresponds to text containing expressions that are more abstract than the result text.

[0050] That is, the result text corresponds to text containing a more specific and detailed explanation compared to text containing ambiguous expressions in the input information, and at the same time corresponds to text explaining the driving scenario simulation. Accordingly, the device (100) can generate a specific driving scenario simulation related to the input information according to the given first or second input state, and additionally generate result text.

[0051] Specifically, the device (100) may include an input unit (110), a communication unit (120), a display (130), a memory (140), and a control unit (150), as illustrated in FIG. 1. Of course, the device (100) may also include an image acquisition device for capturing driving images.

[0052] The input unit (110) generates input data in response to input from various users and may include various input means. For example, the input unit (110) may include a keyboard, a key pad, a dome switch, a touch panel, a touch key, a touch pad, a mouse, a menu button, etc., but is not limited thereto.

[0053] The communication unit (120) is configured to perform communication with other devices. For example, the communication unit (120) may perform wireless communication such as 5G (5th generation communication), LTE-A (long term evolution-advanced), LTE (long term evolution), Bluetooth, BLE (Bluetooth low energy), NFC (near field communication), and Wi-Fi communication, or wired communication such as cable communication, but is not limited thereto. For instance, the communication unit (120) may receive input information, etc. from other devices and transmit a generated result (a result generated according to the generation method described later) to other devices. In particular, when receiving input information from other devices and then transmitting a generated result to other devices, the device (100) may operate as a server that transmits the generated result.

[0054] The display (130) is configured to display various image data on a screen. For example, the display (130) may be composed of a non-emissive panel or an emissive panel. For example, the display (130) may include a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a micro electro mechanical systems (MEMS) display, or an electronic paper display, but is not limited thereto. For example, the display (130) may display input information, generated results, etc. on the screen. Additionally, the display (130) may be combined with an input unit (110) and implemented as a touch screen, etc.

[0055] The memory (140) stores various information necessary for the operation of the device (100). For example, the stored information may include input information, language models, simulators, generation results, or program information related to the generation method described later, but is not limited thereto. For example, the memory (140) may include, depending on its type, a hard disk type, a magnetic media type, a CD-ROM (compact disc read only memory), an optical media type, a magneto-optical media type, a multimedia card micro type, a flash memory type, a ROM type (read only memory type), or a RAM type (random access memory type), but is not limited thereto. Additionally, depending on its use / location, the memory (140) may be a cache, a buffer, a main memory, or an auxiliary memory, or a separately provided storage system, but is not limited thereto.

[0056] The control unit (150) can perform various control operations of the device (100). That is, the control unit (150) can control the execution of the generation method described later, and can control the operation of the remaining components of the device (100), namely the input unit (110), communication unit (120), display (130), memory (140), etc. For example, the control unit (150) may include a processor, which is hardware, or a process, which is software executed on said processor, but is not limited thereto.

[0057] FIG. 2 shows a block diagram of the control unit (150) in the driving scenario device (100) according to one embodiment of the present invention.

[0058] The control unit (150) controls the execution of a generation method according to one embodiment of the present invention and may include a driving information extraction unit (151), a prompt generation unit (152), a language model processing unit (153), and a simulator processing unit (154), as shown in FIG. 2. For example, the driving information extraction unit (151), the prompt generation unit (152), the language model processing unit (153), and the simulator processing unit (154) may be hardware components of the control unit (150) or processes that are software executed in the control unit (150), but are not limited thereto.

[0059] Hereinafter, the method of production according to the present invention will be described in more detail.

[0060] FIG. 3 shows a flowchart of a generation method according to one embodiment of the present invention.

[0061] A generation method according to one embodiment of the present invention (hereinafter referred to as "the generation method") can be performed under the control of a control unit (150) and is a method for generating specific driving scenario simulations related to the input information according to a first or second input state. Of course, the generation method may also generate detailed description text in addition to the driving scenario simulation. The generation method according to one embodiment of the present invention may include S310 to S340 as illustrated in FIG. 3.

[0062] S310 is a step of extracting driving information for given input information. The processing of this S310 can be controlled by the driving information extraction unit (151) of the control unit (150).

[0063] In this case, driving information is information related to the driving of a reference vehicle (i.e., the vehicle that is the subject of the input information) that can be analyzed from the input information, and is information that can affect the driving of the reference vehicle. For example, driving information may include weather, the number of surrounding vehicles, the color of surrounding vehicles, road conditions, road lanes, the presence or absence of roadside trees, etc.

[0064] Specifically, weather is information about the weather conditions under which the reference vehicle is driving. The number of surrounding vehicles is information about the number of other vehicles driving around the reference vehicle. The color of surrounding vehicles is information about the colors of other vehicles driving around the reference vehicle. If there are multiple surrounding vehicles, the color of surrounding vehicles may include color information for each vehicle. Road condition is information about the condition of the road (unpaved, paved, etc.) on which the reference vehicle is driving. Road lane is information about the lane (1st lane, 2nd lane, etc.) of the road on which the reference vehicle is driving. Presence of roadside trees is information about whether or not there are roadside trees around the road on which the reference vehicle is driving.

[0065] Each of these driving information can serve as an element included in a driving scenario simulation. That is, each driving information is extracted from input information regarding a reference vehicle and can serve as an element that influences the driving of the target vehicle (the vehicle subject to the driving scenario simulation) in a driving scenario simulation for the target vehicle. Accordingly, the present invention can generate a driving scenario simulation applicable to the target vehicle based on each driving information extracted from the reference vehicle.

[0066] Of course, the type of driving information may vary depending on whether the input information is in the first or second input state.

[0067] For example, when the input information is in a first input state, in S310, driving information (first driving information) including the number of surrounding vehicles, the color of surrounding vehicles, the number of road lanes, the presence or absence of roadside trees, and the weather can be extracted from a photo, video, or accident report file that is the input information. This is because the first driving information is information that can generally be extracted from a photo, video, or accident report file related to the driving or driving accident of a reference vehicle.

[0068] On the other hand, when the input information is in a second input state, in S310, the first driving information may be extracted from a photo, video, or accident report file, which is also input information. Of course, in this case, driving information (second driving information) may be extracted from ambiguous text, which is additional input information provided, including first information that overlaps with at least one item among the first driving information and second information that is not included in (i.e., non-overlapping with) the first driving information. For example, the second driving information may include the number of surrounding vehicles, road conditions, and weather. In this case, the number of surrounding vehicles and weather correspond to the first information, and the road conditions correspond to the second information.

[0069] As such, since the type of driving information varies depending on whether it is the first or second input state, when a user intends to use a wider variety of driving information types, the type of driving information corresponding to the first or second input state can be utilized more conveniently and easily.

[0070] Next, S320 is a step of fusing overlapping items among the driving information extracted in S310 and then generating a prompt for each driving information. Of course, a first prompt is generated for the fused driving information (i.e., fused information), and a second prompt is generated for the driving information for the remaining non-overlapping items other than the fused information (i.e., non-fused driving information). The processing of this S320 can be controlled by the prompt generation unit (152) of the control unit (150).

[0071] In this case, the fusion information corresponds to the fusion information for each identical item of driving information when multiple identical items of driving information are extracted (i.e., extracted in an overlapping manner) in S310. That is, when the input information is in a first input state, multiple first information items may be extracted in S310, and fusion information fused for each of these first information items may be generated.

[0072] Specifically, the fused information can be generated by fusing each first information according to pre-set weights. That is, a first fusion process can be performed by applying weights to each first information and selecting only those with larger weights to generate the fused information (i.e., if there may be conflicting elements or different parts to emphasize when fusing each first information, the one with the larger weight is reflected first), or a second fusion process can be performed to generate the fused information by reflecting the ratios according to the weights of each first information.

[0073] For example, when the input information is in a second input state, in S310, first driving information regarding the number of surrounding vehicles, the color of surrounding vehicles, the number of road lanes, the presence or absence of roadside trees, and the weather can be extracted from the input information of a photo, video, or accident report file. Along with this, in S310, second driving information regarding the number of surrounding vehicles, road conditions, and the weather can also be extracted from the input information of ambiguous text.

[0074] At this time, the number of surrounding vehicles and weather of the first driving information and the number of surrounding vehicles and weather of the second driving information are extracted in an overlapping manner. Of course, weights for the first and second driving information are pre-set, or weights for the elements of each driving information are pre-set. Accordingly, pre-set weights are applied to the number of surrounding vehicles and weather of the first driving information, and pre-set weights are applied to the number of surrounding vehicles and weather of the second driving information; based on the results of such application, fused information resulting from the first or second fusion processing of the driving information regarding the number of surrounding vehicles and weather can be generated.

[0075] Of course, processing of such fusion information is unnecessary for driving information of non-overlapping items. That is, since the surrounding vehicle color, number of road lanes, and presence or absence of roadside trees in the first driving information, and the road condition in the second driving information, each correspond to driving information of non-overlapping items, fusion processing is not performed on them.

[0076] Meanwhile, in S320, if only overlapping driving information exists, there is no non-fused driving information, and only fused information exists, only the first prompt is generated. Additionally, if there is no overlapping driving information, there is no fused information, and only non-fused driving information exists, only the second prompt is generated. If overlapping driving information and non-overlapping driving information exist together, and fused information and non-fused driving information exist together, both the first and second prompts are generated.

[0077] In this case, the prompt refers to fused information or non-fused driving information converted into the form of input data for a pre-trained language model. That is, fused information can be converted into a first prompt in a form that can be input into the language model, and non-fused driving information can be converted into a second prompt in a form that can be input into the language model.

[0078] Specifically, the prompt can be generated in two ways (the first and second ways).

[0079] First, there is a first method of simply listing words corresponding to fused information or non-fused driving information. For example, even if only words related to fused information or non-fused driving information, such as "weather clear, 2 vehicles, overtaking, 2 lanes," are listed and input, the language model can analyze them and derive a result.

[0080] Additionally, there is a second method in which a template format is selected in advance, and values ​​for fused or non-fused driving information are parameterized to generate a prompt according to that template format. For example, if a template format such as "driving situation where the weather is {weather} and the number of surrounding lanes is {lanes}" is selected in advance, values ​​for fused or non-fused driving information can be entered into the weather and the number of surrounding lanes, respectively, while only {weather} and {lanes} are variables, and then input into the language model.

[0081] Next, S330 is the step of inputting the prompt generated in S320 as input data into a language model to generate a driving scenario simulation accordingly. Of course, in S330, the language model can additionally generate result text along with the driving scenario simulation.

[0082] In this case, the language model is a generative artificial neural network model trained according to reinforcement learning or supervised learning techniques of machine learning, corresponding to a model trained to generate driving scenario simulations related to the input data. Of course, the language model is also trained to generate result text along with these driving scenario simulations.

[0083] For example, a language model may be a model that incorporates a transformer structure and learns weights using reinforcement learning techniques, but is not limited to this.

[0084] Additionally, in the S330, parameters regarding the level of detail can be selected and input into the language model. These parameters control how detailed the language model is in generating driving scenario simulations and resulting text. For example, if parameters for additional descriptive information, such as "approximately 100 words" or "very detailed description of surrounding road conditions," are selected, the language model can reflect this in the driving scenario simulation and output the corresponding text.

[0085] Next, S340 is the step of retraining the language model while verifying the driving scenario simulation generated in S330 using a simulator.

[0086] At this time, the simulator is a module that interprets a driving scenario simulation (e.g., markup language code or program code) generated by the language model in S340 and executes it to enable virtual driving. That is, the simulator first checks whether the generated driving scenario simulation can be executed because there are no grammatical errors, and secondarily tests whether it matches the input information (i.e., whether it is related to the input information).

[0087] As a result of these tests, errors may occur where the driving scenario simulations generated by the language model fail to run on the simulator or fail to represent the desired scenarios. To address this issue, error codes generated in the simulator can be fed back during the language model's training, and the model can be continuously retrained until the error no longer occurs. In other words, the language model can be retrained to generate driving scenario simulations in which the error has been eliminated. Accordingly, the language model outputs results only for those that pass the simulator's tests, thereby improving the quality of the output.

[0089] The present invention, configured as described above, has the advantage of easily generating a specific driving scenario simulation related to input information when input information related to a vehicle's driving situation is given. In addition, the present invention has the advantage of improving the quality of the output result by attempting to execute the driving scenario simulation, which is the result generated by the language model of a generative artificial neural network, using a simulator that actually runs the driving scenario simulation, first checking whether it can be executed because there are no grammatical errors, and secondly testing whether it matches the input information, and outputting the result only for the result that passes the test.

[0091] Although specific embodiments have been described in the detailed description of the present invention, it is understood that various modifications are possible within the scope of the invention. Therefore, the scope of the present invention is not limited to the described embodiments and should be defined by the claims set forth below and equivalents thereof. Explanation of the symbols

[0093] 100: Device 110: Input section 120: Communications Unit 130: Display 140: Memory 150: Control unit 151: Driving information extraction unit 152: Prompt generation unit 153: Language Model Processing Unit 154: Simulator Processing Unit

Claims

Claim 1 A method performed in an electronic device to generate a driving scenario simulation for a target vehicle that is the subject of the driving scenario simulation, comprising: a step of extracting a plurality of driving information that is extracted from input information for a reference vehicle and acts as a factor influencing the driving of the target vehicle in the driving scenario simulation; a step of generating a prompt for converting each driving information into a form of input data for a language model after fusing overlapping items among the extracted driving information; and a step of generating a driving scenario simulation and a result text that explains the driving scenario simulation in detail, respectively, by inputting the generated prompt into the language model as input data. The method includes the step of retraining a language model while verifying a driving scenario simulation generated using a simulator; wherein the driving information varies in type depending on whether the input information is in a first or second input state, and includes, respectively, weather information regarding the day of driving, information on the number of surrounding vehicles, information on the color of surrounding vehicles, road condition information regarding the paving status of the driving road, information on the number of road lanes regarding the lanes of the driving road, and information on the presence or absence of roadside trees regarding the presence or absence of roadside trees around the driving road; wherein the first input state is a state in which a photo, video, or accident report file of the reference vehicle is provided as input information but no additional text is provided, and the second input state is a state in which a photo, video, or accident report file of the reference vehicle is provided as input information but additional text is provided; when the input information of the first input state is input, first driving information including the information on the number of surrounding vehicles, the information on the color of surrounding vehicles, the information on the number of road lanes, the information on the presence or absence of roadside trees, and the weather information is extracted from the photo, video, or accident report file of the reference vehicle, and the input of the second input state When information is entered,The first driving information is extracted, and a second driving information including the first and second information, respectively, is further extracted from the additional text; the first information includes information that overlaps with at least one item of the first driving information, and includes the surrounding vehicle count information and the weather information; the second information includes information that does not overlap with the first driving information, and includes the road condition information; the result text includes a more detailed explanation than the additional text, and includes text explaining the driving scenario simulation; the step of generating the prompt generates a first prompt for the fused information when only the fused information, which is the driving information including the surrounding vehicle count information and the weather information that overlaps and fuses, exists; generates a second prompt for the driving information when only the driving information of the non-overlapping items, such as the surrounding vehicle color information, the road lane count information, the presence or absence of road trees, and the road condition information, exists; and generates the first and second prompts together when the fused information and the driving information of the non-overlapping items exist together; in the step of generating the prompt, the surrounding vehicle count information and the overlapping item between the first driving information and the second driving information, which are the surrounding vehicle count information and the Regarding weather information, the weights for the surrounding vehicle count information and weather information of the first driving information and the surrounding vehicle count information and weather information of the second driving information are each pre-set, and according to each weight, the one with the larger weight is selected to generate the fusion information, or the ratio according to each weight is reflected to generate the fusion information, and in the step of generating the prompt, the prompt is generated by listing words corresponding to each extracted driving information, or the prompt is generated by entering the value of each extracted driving information according to a selected template format. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 delete Claim 8 delete Claim 9 delete Claim 10 In claim 1, a method for retraining the language model in the retraining step, wherein the simulator tests for grammatical errors, execution status, and relevance with the input information regarding a generated driving scenario simulation, and if an error is found, feeds back the corresponding error code so that the error does not occur. Claim 11 Memory; The method includes a control unit that controls the generation of a driving scenario simulation of a target vehicle, which is the subject of the driving scenario simulation, using information stored in the memory; wherein the control unit controls the extraction of a plurality of driving information that acts as a factor influencing the driving of the target vehicle in the driving scenario simulation, which is extracted from input information for a reference vehicle, and controls the generation of a prompt to convert each driving information into the form of input data for a language model after fusing overlapping items among the extracted driving information, and controls the generation of a driving scenario simulation and a result text that explains the driving scenario simulation in detail by inputting the generated prompt into the language model as input data, and controls the retraining of the language model while verifying the driving scenario simulation generated using the simulator, wherein the type of driving information varies depending on whether the input information is in a first or second input state, and includes weather information regarding the day of driving, information regarding the number of surrounding vehicles, information regarding the color of surrounding vehicles, road condition information regarding the paving status of the driving road, information regarding the number of road lanes regarding the lanes of the driving road, and information regarding the presence or absence of roadside trees regarding the presence or absence of roadside trees around the driving road, respectively. The first input state is a state in which a photo, video, or accident report file of the reference vehicle is provided as input information but no additional text is provided, and the second input state is a state in which a photo, video, or accident report file of the reference vehicle is provided as input information but additional text is provided, and when the input information of the first input state is provided, first driving information including the number of surrounding vehicles information, the color information of surrounding vehicles, the number of road lanes information, the presence or absence information of roadside trees, and the weather information is extracted from the photo, video, or accident report file of the reference vehicle.When input information of the second input state is input, the first driving information is extracted, and a second driving information including the first and second information, respectively, is further extracted from the additional text; the first information includes information that overlaps with at least one item of the first driving information, and includes the surrounding vehicle count information and the weather information; the second information includes information that does not overlap with the first driving information, and includes the road condition information; the result text includes a more detailed explanation than the additional text, and includes text explaining the driving scenario simulation; and when the prompt is generated, the control unit generates a first prompt for the fused information when only the fused information, which is the driving information including the surrounding vehicle count information and the weather information that overlap and fuse, exists, and generates a second prompt for the driving information when only the driving information of the non-overlapping items, such as the surrounding vehicle color information, the road lane count information, the presence or absence of roadside trees, and the road condition information, exists, and generates the first and second prompts together when the fused information and the driving information of the non-overlapping items exist together, and between the first driving information and the second driving information Regarding the surrounding vehicle count information and weather information, which are the overlapping items mentioned above, a weight for the surrounding vehicle count information and weather information of the first driving information and a weight for the surrounding vehicle count information and weather information of the second driving information are each pre-set, and according to each weight, the one with the larger weight is selected to generate the fusion information, or the fusion information is generated by reflecting the ratio according to each weight, and the device generates the prompt by listing words corresponding to each extracted driving information, or generates the prompt by entering the value of each extracted driving information according to a selected template format. Claim 12 A communication unit that receives input information regarding a reference vehicle and transmits a driving scenario simulation generated by a language model based on the received input information; The system includes a control unit that controls the generation of a driving scenario simulation of a target vehicle, which is the subject of the driving scenario simulation, from information received by the communication unit; wherein the control unit controls the extraction of a plurality of driving information that is extracted from the input information and acts as a factor influencing the driving of the target vehicle in the driving scenario simulation, and after fusing overlapping items among the extracted driving information, controls the generation of a prompt to convert each driving information into the form of input data for a language model, and controls the generation of a driving scenario simulation and a result text that explains the driving scenario simulation in detail by inputting the generated prompt into the language model as input data, and controls the retraining of the language model while verifying the driving scenario simulation generated using a simulator, wherein the type of driving information varies depending on whether the input information is in a first or second input state, and includes weather information regarding the day of driving, information on the number of surrounding vehicles, information on the color of surrounding vehicles, road condition information regarding the paving status of the driving road, information on the number of road lanes regarding the lanes of the driving road, and information on the presence or absence of roadside trees regarding the presence or absence of roadside trees around the driving road, respectively. The first input state is a state in which a photo, video, or accident report file of the reference vehicle is provided as input information but no additional text is provided, and the second input state is a state in which a photo, video, or accident report file of the reference vehicle is provided as input information but additional text is provided, and when the input information of the first input state is input, the surrounding vehicle count information, surrounding vehicle color information, from the photo, video, or accident report file of the reference vehicleA first driving information including the above road lane count information, the above road tree presence / absence information, and the above weather information is extracted, and when input information of the above second input state is input, the first driving information is extracted, and a second driving information including the first and second information, respectively, is further extracted from the above additional text, wherein the first information includes information that overlaps with at least one item of the first driving information, and includes the surrounding vehicle count information and the above weather information, and the second information includes information that does not overlap with the first driving information, and includes the above road condition information, and the result text includes a more detailed explanation than the above additional text, and includes text explaining the driving scenario simulation, and the control unit, when generating the above prompt, generates a first prompt for the fused information when only the fused information, which is driving information including the surrounding vehicle count information and the above weather information, which are overlapping and fused items, exists, and generates a second prompt for the corresponding driving information when only the driving information of the surrounding vehicle color information, the above road lane count information, the above road tree presence / absence information, and the above road condition information, which are non-overlapping items, exists, and the driving information of the fused information and the non-overlapping items When they exist together, the first and second prompts are generated together, and regarding the surrounding vehicle count information and the weather information, which are the overlapping items between the first driving information and the second driving information, the weights for the surrounding vehicle count information and the weather information of the first driving information and the weights for the surrounding vehicle count information and the weather information of the second driving information are each pre-set, and the fusion information is generated by selecting the one with the larger weight according to each weight, or by generating the fusion information by reflecting the ratio according to each weight, or the prompt is generated by listing the words corresponding to each extracted driving information, orA device that generates the above prompt by entering the value of each extracted driving information into a selected template format.

Images