A method for managing autonomous driving training data and an autonomous driving training system implementing the same

Abstract

A method for selecting autonomous driving learning data according to an embodiment of the present invention is a method for selecting autonomous driving learning data implemented by a selection device which is a computer device mounted on a vehicle, comprising: receiving first driving information, which is driving information of a vehicle generated based on a first time point data among sensing information detected by a sensor unit of the vehicle, and a second sensing information, which is driving information of a vehicle mounted on the vehicle; evaluating whether the operation of the vehicle based on the first driving information is normal based on a predetermined deviation condition by utilizing second sensing information, which is sensing information at a second time point after the vehicle is controlled by the first driving information; and, when the predetermined deviation condition is satisfied, transmitting the selection information, which is sensing information satisfying a predetermined selection condition, to a learning device disposed outside the vehicle.

Description

Technology Field

[0001] The present invention relates to a method for managing autonomous driving learning data and an autonomous driving learning system implementing the same, and to a technology that enables continuous performance improvement of an autonomous driving model by automatically detecting error sections of the autonomous driving model and automatically collecting driving data corresponding to the error sections. Background Technology

[0003] An autonomous vehicle refers to a vehicle that drives autonomously through an artificial intelligence model that collects information about the surrounding environment via various sensors, such as cameras, lidar, and radar, and generates driving trajectories or control commands based on this information. Recently, with the introduction of end-to-end (E2E) autonomous driving models that directly output control commands from sensor inputs, there is an advantage in that the complex decision-making processes required for autonomous driving can be processed by a single model.

[0004] However, since these autonomous driving models are generally trained offline based on large-scale driving data collected in advance and applied to actual driving using fixed model parameters after training is complete, there is a problem in that existing models find it difficult to respond immediately to various exceptional situations or new domain environments not included in the training data that continuously occur in real driving conditions. Furthermore, in existing methods, there are no clear criteria for selecting which data from the vast amount generated during driving to use for training, and as most data is collected in bulk or selected manually, there are inefficiencies in terms of data collection and management.

[0005] In particular, there is a problem of reduced learning efficiency as data directly related to errors or performance degradation occurring during actual driving is processed without distinguishing between that and other data. Accordingly, there is an increasing need for technology capable of efficiently identifying, selecting, and managing valid data for learning among the data generated during driving to effectively respond to various situations occurring in real-world driving environments, and to continuously improve the performance of autonomous driving models based on this. Prior art literature

[0007] Korean Registered Patent Publication No. 10-2503443 (Registered on Feb. 21, 2023) The problem to be solved

[0008] The present invention aims to solve the aforementioned problems and provides a method for managing autonomous driving learning data that enables continuous performance improvement of an autonomous driving model, and an autonomous driving learning system that implements the same. means of solving the problem

[0010] A method for selecting autonomous driving learning data, implemented by a selection device which is a computer device mounted on a vehicle according to an embodiment of the present invention, may include: a step of receiving first driving information, which is driving information of a vehicle generated based on a first time point data among sensing information detected by a sensor unit of the vehicle and a second sensing information, which is sensing information at a second time point, and a step of evaluating whether the operation of the vehicle based on the first driving information is normal based on a predetermined deviation condition using second sensing information, which is sensing information at a second time point after the vehicle is controlled by the first driving information; and a step of transmitting the selection information, which is sensing information satisfying a predetermined selection condition, to a learning device disposed outside the vehicle when the predetermined deviation condition is satisfied.

[0011] In addition, the above-determined deviation condition may include at least one of the following conditions: a first deviation condition, which is a condition for comparing the predicted trajectory predicted by the autonomous driving model with the actual trajectory calculated based on the second sensing information; a second deviation condition, which is a condition for determining whether the actual trajectory calculated based on the second sensing information complies with safety rules; a third deviation condition, which is a condition for comparing the reference trajectory generated by the reference model with the actual trajectory calculated based on the second sensing information; and a fourth deviation condition, which is a condition for comparing the normal distribution, which is past driving information, with the actual distribution calculated based on the second sensing information.

[0012] In addition, the above-determined deviation condition may be such that the deviation index calculated by comparing the actual information calculated based on the second sensing information with the reference information serving as the basis for judgment is greater than or equal to a predetermined threshold value.

[0013] In addition, the above-mentioned predetermined threshold value may be a value adjusted by a predetermined adjustment method that considers the activity of an object capable of activity within a target area, which is a zone that includes the location where the vehicle is located.

[0014] In addition, the above-determined adjustment method may be a method of lowering the above-determined threshold value as the variability of the expected activity range of the object increases.

[0015] In addition, the above-determined adjustment method may be a method of variably setting the adjustment amount of the above-determined threshold value by considering functionality, which is an indicator representing the state of the vehicle, when a predetermined exception condition is satisfied.

[0016] In addition, the above-mentioned predetermined exception condition may be a condition satisfied when the above-mentioned predetermined threshold is adjusted to be less than a predetermined threshold.

[0017] Additionally, the above-determined selection interval may comprise at least one of the following: a first selection interval, which is a time interval from a first reference point, which is a point in time prior to the error occurrence point, to the error occurrence point; and a second selection interval, which is a time interval from an error termination point, which is a point in time after the error occurrence point where the above-determined deviation condition is not satisfied, to a second reference point, which is a point in time after the error termination point.

[0018] Meanwhile, in an autonomous driving learning data learning method according to one embodiment of the present invention, which is implemented by a learning device which is a computing device external to a vehicle, the method for learning autonomous driving learning data is,

[0019] The method may include: a step of receiving selection information, which is sensing information detected by the vehicle that satisfies a predetermined selection condition; a step of determining whether the selection information satisfies a predetermined learning condition; and a step of learning an autonomous improvement model based on the selection information if the predetermined learning condition is satisfied.

[0020] In addition, the above-mentioned predetermined learning condition may be a condition in which the diversity of error types of the above-mentioned selection information is greater than or equal to a predetermined standard.

[0021] Meanwhile, an autonomous driving learning data management method according to one embodiment of the present invention is implemented by an autonomous driving learning device—the autonomous driving learning device comprises a selection device mounted on a vehicle and a learning device disposed outside the vehicle—and comprises the steps of: receiving, by the selection device, first sensing information, which is first time point data among sensing information detected by a sensor unit of the vehicle, and first driving information, which is driving information of the vehicle generated based on an autonomous driving model mounted on the vehicle; evaluating, by the selection device, whether the operation of the vehicle based on the first driving information is normal based on a predetermined deviation condition by utilizing second sensing information, which is the sensing information at a second time point after the vehicle is controlled by the first driving information; transmitting, by the selection device, the selection information, which is the sensing information satisfying a predetermined selection condition, to the learning device; and determining by the learning device whether the selection information satisfies a predetermined learning condition. and, when the predetermined learning conditions are satisfied by the learning device, a step of learning an autonomous improvement model based on the selection information may be included. Effects of the invention

[0023] The method for managing autonomous driving learning data according to the present invention and the autonomous driving learning system implementing the same can improve the performance of an autonomous driving vehicle.

[0024] In addition, autonomous driving training data can be collected automatically.

[0025] In addition, the utilization of training data can be improved by selectively using only data valid for autonomous driving learning.

[0026] In addition, by minimizing the processing and learning of unnecessary training data, learning efficiency can be improved and operating costs can be reduced.

[0027] However, the effects of the present invention are not limited to the effects described above, and unmentioned effects will be clearly understood by those skilled in the art from this specification and the attached drawings. Brief explanation of the drawing

[0029] FIG. 1 is a relationship diagram of an autonomous driving learning system according to an embodiment of the present invention. FIG. 2 is a configuration diagram of an autonomous driving learning system according to an embodiment of the present invention. FIG. 3 is a schematic flowchart of a method for managing autonomous driving learning data according to an embodiment of the present invention. FIG. 4 is a schematic flowchart of a method for selecting autonomous driving learning data according to an embodiment of the present invention. FIG. 5 is a schematic flowchart of a method for evaluating whether the operation of a vehicle is normal in an autonomous driving learning data selection method according to an embodiment of the present invention. FIG. 6 is a flowchart of a method for calculating a deviation index of a vehicle in a method for selecting autonomous driving learning data according to an embodiment of the present invention. FIG. 7 is a flowchart of a method for determining whether a deviation index satisfies a predetermined condition in an autonomous driving learning data selection method according to an embodiment of the present invention. FIG. 8 is a drawing for explaining a method for selecting autonomous driving learning data according to an embodiment of the present invention. FIG. 9 is a configuration diagram of a sorting device according to an embodiment of the present invention. Specific details for implementing the invention

[0030] Specific embodiments of the present invention will be described in detail below with reference to the drawings. However, the concept of the present invention is not limited to the presented embodiments, and those skilled in the art who understand the concept of the present invention may easily propose other inventions that are inferior or other embodiments included within the scope of the concept of the present invention by adding, changing, or deleting other components within the same scope of the concept, and such are also to be considered to be included within the scope of the concept of the present invention.

[0031] The terms first, second, A, B, etc., as used in this specification and claims may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0032] The terms used in this specification and claims are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" should be understood as not precluding the existence or addition of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification.

[0033] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0034] Furthermore, each component, process, procedure, or method included in each embodiment of the present invention may be shared within a scope that is not technically contradictory. Additionally, components with the same function within the scope of the same concept appearing in the drawings of each embodiment are described using the same reference numerals.

[0036] FIG. 1 is a relationship diagram of an autonomous driving learning system according to one embodiment of the present invention, and FIG. 2 is a configuration diagram of an autonomous driving learning system according to one embodiment of the present invention.

[0038] Referring to FIGS. 1 and 2, an autonomous driving learning system (100) according to one embodiment of the present invention may include a control device (110) mounted on a vehicle to control the driving of the vehicle, a selection device (120) which is a computing device mounted on the vehicle, and a learning device (130) positioned outside the vehicle to learn at least a portion of the learning data for the autonomous driving of the vehicle.

[0039] The control device (110), the selection device (120), and / or the learning device (130) can be connected to a network via wired or wireless means.

[0040] The term "wireless network" as used in the present invention may refer to a core network integrated with a wired public network, a wireless mobile communication network, or the mobile internet, and may refer to a global open computer network structure that provides the TCP / IP protocol and various services existing at the upper layer, such as HTTP (Hyper Text Transfer Protocol), HTTPS (Hyper Text Transfer Protocol Secure), Telnet, FTP (File Transfer Protocol), DNS (Domain Name System), SMTP (Simple Mail Transfer Protocol), etc. It is not limited to these examples but comprehensively refers to a data communication network capable of transmitting and receiving data in various forms.

[0041] The autonomous driving learning system (100) may be a system for improving the performance of autonomous driving of a vehicle based on driving control of the vehicle and data generated during driving of the vehicle.

[0042] The autonomous driving learning system (100) may be composed of a plurality of computing devices including a processor and memory.

[0043] The autonomous driving learning system (100) can implement an autonomous driving learning data management method.

[0044] The autonomous driving learning system (100) may be a system composed of devices necessary to implement an autonomous driving learning data management method.

[0045] The computing device mentioned in the present invention may refer to a device capable of processing information processing operations.

[0046] For example, computing devices may include desktop computers, laptops, smartphones, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), portable terminals, mobile terminals and / or smart TVs, servers, etc.

[0047] Here, the vehicle may include a drive unit and a sensor unit (P10).

[0048] Specifically, a vehicle drive system may refer to a device for generating or controlling power for driving the vehicle.

[0049] For example, the drive system of a vehicle may include a housing that forms an exterior in which a person can ride, a steering system, a drive motor, an engine, a braking system, and / or a transmission system.

[0050] However, this is not limited to this, and specific examples of driving devices can be varied in many ways that are obvious to a person skilled in the art.

[0051] The sensor unit (P10) of the vehicle may refer to a device connected to or placed in the housing to detect environmental information around the vehicle and status information of the vehicle.

[0052] The sensor unit (P10) may include one or more sensors capable of measuring various physical quantities that change according to the driving of the vehicle in order to recognize the surrounding environment and vehicle status of the vehicle while driving autonomously.

[0053] For example, the sensor unit (P10) may include a camera sensor for recognizing surrounding objects, a LiDAR sensor and a radar sensor, a GPS sensor for recognizing the position of the vehicle and / or an inertial measurement unit (IMU) for recognizing the attitude of the vehicle.

[0054] However, not limited thereto, the type and configuration of the sensor included in the sensor unit (P10) may be varied in various ways within the scope obvious to a person skilled in the art.

[0056] The control device (110) is mounted on the vehicle and can control the driving of the vehicle using an autonomous driving model based on sensing information measured from the sensor unit (P10) of the vehicle.

[0057] The control device (110) may be a computing device equipped with and executing an end-to-end (E2E) autonomous driving model.

[0058] The control device (110) can execute an end-to-end (E2E) autonomous driving model that receives sensing information received from the sensor unit (P10) of the vehicle as input and directly outputs control commands for the vehicle's driving trajectory or driving without separate intermediate processing.

[0059] The control device (110) can calculate driving information that can control the driving of the vehicle using an autonomous driving model, and can control the driving device of the vehicle according to the driving information.

[0060] The control device (110) may be a computing device that has a reference model mounted on it and executes it.

[0061] Here, the reference model may include at least one of a rule-based driving policy (rule-based planner), a model trained based on driving data of human drivers collected in advance, or a model designed to simulate the driving behavior of an expert.

[0062] For example, the reference model may be a model that receives the same sensing information as input to generate a driving trajectory or control command of a vehicle, and operates according to criteria defined independently of the autonomous driving model.

[0063] For example, the reference model may be a rule-based path generation model designed to maintain lane centering, and may be a model that generates a target driving trajectory based on road lane information.

[0064] However, this is not limited to this, and specific examples of reference models can be modified in various ways to the extent that they are obvious to a person skilled in the art.

[0065] The control device (110) may be a computing device.

[0066] For example, the control device (110) may be an Electronic Control Unit (ECU) for controlling the driving of a vehicle.

[0067] However, this is not limited to this, and specific examples of the control device (110) can be varied in various ways that are obvious to a person skilled in the art.

[0068] The control device (110) can determine the driving state of the vehicle based on sensing information collected in real time while the vehicle is driving, and generate a driving control command corresponding to it.

[0070] The selection device (120) is a computing device mounted on a vehicle and can select and manage data to be used for autonomous driving learning among the data generated while the vehicle is driving.

[0071] The selection device (120) may be a computing device.

[0072] The sorting device (120) may be implemented as a separate computing device physically separated from the control device (110).

[0073] Alternatively, the sorting device (120) may be integrated with the control device (110) and implemented to perform different functions on the same hardware, but the present invention is not limited thereto, and specific examples of the sorting device (120) can be varied in various ways that are obvious to a person skilled in the art.

[0074] Specifically, the selection device (120) can analyze the deviation between the output of the autonomous driving model and the actual driving result of the vehicle based on the driving information provided from the control device (110) and the sensing information collected from the sensor unit (P10) of the vehicle.

[0075] Additionally, the selection device (120) can compare the deviation between the output of the autonomous driving model and the actual driving result of the vehicle, and select the data necessary for improving the performance of the autonomous driving model from the total data generated while the vehicle is operating or stopped.

[0077] The learning device (130) can be placed outside the vehicle.

[0078] The learning device (130) may be placed in a space separated from the control device (110) and / or the screening device (120).

[0079] The learning device (130) can be separated from the control device (110) and / or the selection device (120) to learn the autonomous improvement model separately, thereby preventing a direct impact on the control of the vehicle while it is in motion.

[0080] The learning device (130) can learn an autonomous improvement model that improves the performance of an existing learned autonomous driving model based on the selection information selected from the selection device (120).

[0081] Here, selected information may refer to data selected from the total data collected while the vehicle is autonomous driving that is deemed necessary to improve the performance of the vehicle's autonomous driving model.

[0082] The learning device (130) can provide the autonomous improvement model to the vehicle after verifying the performance of the autonomous improvement model and after all control of the vehicle using the autonomous driving model is completed.

[0083] For example, the learning device (130) may be a computing device or a server.

[0084] The server referred to in the present invention may include other components for performing the server environment. The server may include any type of device.

[0085] For example, a server is a digital device and can be a digital device equipped with a processor and memory that has computing capabilities, such as a laptop computer, notebook computer, desktop computer, web pad, or mobile phone.

[0086] For example, the server may be a web server. However, it is not limited to this, and the type of server can be varied to a level obvious to a person of ordinary technical expertise.

[0087] The computing device mentioned in the present invention may refer to a device capable of processing information processing operations.

[0088] For example, computing devices may include desktop computers, laptops, smartphones, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), portable terminals, mobile devices and / or smart TVs, servers, etc.

[0090] Below, a method for managing autonomous driving learning data, implemented by a sorting device which is a computer device mounted on a vehicle, will be described in detail.

[0092] FIG. 3 illustrates a schematic flowchart of a method for managing autonomous driving learning data according to an embodiment of the present invention, FIG. 4 illustrates a schematic flowchart of a method for selecting autonomous driving learning data according to an embodiment of the present invention, and FIG. 5 illustrates a schematic flowchart of a method for evaluating whether the operation of a vehicle is normal in the method for selecting autonomous driving learning data according to an embodiment of the present invention.

[0093] In addition, FIG. 6 shows a flowchart of a method for calculating a deviation index of a vehicle in a method for selecting autonomous driving learning data according to an embodiment of the present invention, FIG. 7 shows a flowchart of a method for determining whether a deviation index satisfies a predetermined condition in a method for selecting autonomous driving learning data according to an embodiment of the present invention, and FIG. 8 shows a diagram for explaining a method for selecting autonomous driving learning data according to an embodiment of the present invention.

[0095] Referring to FIG. 3, an autonomous driving learning data management method according to an embodiment of the present invention comprises: a step (S110) of receiving first driving information, which is driving information of a vehicle generated based on a first time point data among sensing information detected by a sensor unit of a vehicle by a selection device and an autonomous driving model mounted on the vehicle; a step (S120) of evaluating whether the operation of the vehicle based on the first driving information is normal based on a predetermined deviation condition by utilizing second sensing information, which is sensing information at a second time point after the vehicle is controlled by the first driving information; and a step (S130) of transmitting the selection information, which is sensing information satisfying a predetermined selection condition, to a learning device disposed outside the vehicle when the predetermined deviation condition is satisfied; and a step (S210) of receiving the selection information, which is sensing information detected by the vehicle satisfying the predetermined selection condition, by a learning device; a step (S220) of determining whether the selection information satisfies a predetermined learning condition, and the predetermined When learning conditions are satisfied, the method may include a method for learning autonomous driving learning data, which includes a step (S230) of learning an autonomous improvement model based on the above selection information.

[0097] Referring to FIG. 4, a method for selecting autonomous driving learning data according to one embodiment of the present invention may include the step (S110) of receiving first sensing information, which is first time point data among sensing information detected by a sensor unit (P10) of a vehicle, and first driving information, which is driving information of a vehicle generated based on an autonomous driving model mounted on the vehicle; the step (S120) of evaluating whether the operation of the vehicle based on the first driving information is normal based on a predetermined deviation condition by utilizing second sensing information, which is the sensing information of a second time point after the vehicle is controlled by the first driving information; and the step (S130) of transmitting the selected information, which is the sensing information satisfying a predetermined selection condition, to a learning device placed outside the vehicle when the predetermined deviation condition is satisfied.

[0098] The selection device (120) can receive first sensing information, which is first time point data among the sensing information detected by the sensor unit (P10) of the vehicle, and first driving information, which is driving information of the vehicle generated based on the autonomous driving model mounted on the vehicle. (S110)

[0099] Additionally, the selection device (120) can receive a reference trajectory generated by a reference model based on the first sensing information.

[0100] In step (S110), the sensing information may be information obtained by the vehicle's sensor unit (P10) measuring the vehicle's driving environment and vehicle status. The first driving information may be control command information generated by the control device (110) through an autonomous driving model to control the vehicle based on the first sensing information.

[0101] Here, the sensing information is information indicating the driving environment and vehicle status of the vehicle through the sensor unit (P10) of the vehicle, and may include image information, distance information, object recognition information, vehicle position information, speed and acceleration information, driving trajectory information, etc. obtained through at least one of a camera, LiDAR, RADAR, GPS and IMU sensor.

[0102] For example, the sensing information may include lane recognition information obtained through the vehicle's front camera, distance information between the vehicle and surrounding objects measured through LiDAR, vehicle position coordinates based on GPS, and vehicle acceleration or rotational speed information measured through IMU.

[0103] Here, the driving information is information for vehicle control generated by the autonomous driving model based on the sensing information, and may include the steering angle of the vehicle, acceleration / deceleration commands, braking commands, or target speed.

[0104] For example, the driving information may include a steering angle value for steering the vehicle left or right, a target acceleration value for accelerating or decelerating the vehicle, a braking command indicating whether the vehicle is braking, and target driving trajectory information that the vehicle must follow for a certain period of time.

[0105] Step (S110) may be a step for obtaining input information to determine the difference between the output of the autonomous driving model and actual vehicle operation or reference information.

[0106] In step (S110), the selection device (120) can obtain reference information for quantifying the difference between the driving trajectory generated by the autonomous driving model or the vehicle control command and the actual driving trajectory of the vehicle calculated based on the second sensing information.

[0107] In step (S110), the selection device (120) can additionally obtain reference information for determining the difference between the output of the autonomous driving model and the reference driving policy by using the reference trajectory generated by the reference model.

[0108] In step (S110), the sorting device (120) can obtain the first sensing information and the first driving information as mutually corresponding information.

[0109] Specifically, the first sensing information may correspond to input information for generating the first driving information, and the first driving information may correspond to output information generated in response to the first sensing information.

[0110] For example, the first sensing information is sensing information detected or acquired at a first time point by the sensor unit (P10) of the vehicle, and may mean information acquired before the time point that serves as the basis for generating the first driving information among the information indicating the driving environment and vehicle state of the vehicle.

[0111] For example, the first sensing information may refer to reference input data that is input into an autonomous driving model to generate first driving information including at least one of the vehicle's future driving trajectory, steering angle, acceleration / deceleration command, braking command, or target speed.

[0112] Accordingly, the selection device (120) can configure the first sensing information and the first driving information into a single corresponding pair and use them as data to calculate the deviation between the output of the autonomous driving model and the actual vehicle operation in a subsequent step.

[0114] Referring to FIGS. 4 and 5, the sorting device (120) can evaluate whether the operation of the vehicle based on the first driving information is normal based on predetermined deviation conditions by utilizing the second sensing information, which is the sensing information at a second time point after the vehicle is controlled by the first driving information. (S120)

[0115] Here, the predetermined deviation condition may include at least one of the following conditions: a first deviation condition, which is a condition for comparing the predicted trajectory predicted by the autonomous driving model with the actual trajectory calculated based on the second sensing information; a second deviation condition, which is a condition for determining whether the actual trajectory calculated based on the second sensing information complies with safety rules; a third deviation condition, which is a condition for comparing the reference trajectory generated by the reference model with the actual trajectory calculated based on the second sensing information; and a fourth deviation condition, which is a condition for comparing the normal distribution, which is past driving information, with the actual distribution calculated based on the second sensing information.

[0116] In addition, the predetermined deviation condition may be a condition in which the deviation index calculated by comparing the actual information calculated based on the second sensing information with the reference information serving as the basis for judgment is greater than or equal to a predetermined threshold value.

[0117] The step (S120) of evaluating whether the operation of the vehicle based on the first driving information is normal may include a step (S121) of calculating the deviation index of the vehicle, a step (S122) of determining a predetermined threshold value, and a step (S123) of determining whether the deviation index exceeds the predetermined threshold value.

[0118] Here, reference information may be information about a value that serves as a standard for determining whether the information regarding the vehicle's actual driving is normal driving.

[0119] In other words, reference information may be information regarding a value representing a driving state that serves as a standard for when a vehicle is driving normally.

[0120] Reference information can be set differently in correspondence with the types of actual information calculated based on the second sensing information above.

[0122] The sorting device (120) can calculate the deviation index of the vehicle. (S121)

[0123] In step (S121), the selection device (120) can calculate a deviation index that quantitatively represents the difference between the actual information calculated based on the second sensing information and the corresponding reference information in order to determine whether the actual driving result of the vehicle controlled according to the first driving information is normal driving.

[0124] In step (S121), the sorting device (120) can calculate a deviation index, which is an index for evaluating whether the operation of the vehicle based on the first driving information is normal, based on predetermined deviation conditions.

[0125] The step of calculating the deviation index of the vehicle (S121) may include a step of calculating a first deviation index to compare the predicted trajectory predicted by the autonomous driving model with the actual trajectory calculated based on the second sensing information (S121-1), a step of calculating a second deviation index to determine whether the actual trajectory calculated based on the second sensing information complies with safety rules (S121-2), a step of calculating a third deviation index to compare the reference trajectory generated by the reference model with the actual trajectory calculated based on the second sensing information (S121-3), and a step of calculating a fourth deviation index to compare the normal distribution, which is past driving information, with the actual distribution calculated based on the second sensing information (S121-4).

[0126] Specifically, the deviation index is calculated based on predetermined deviation conditions, and the predetermined deviation conditions may include at least one of the following conditions: a first deviation condition which compares the predicted trajectory predicted by the autonomous driving model with the actual trajectory calculated based on the second sensing information; a second deviation condition which determines whether the actual trajectory calculated based on the second sensing information complies with safety rules; a third deviation condition which compares the reference trajectory generated by the reference model with the actual trajectory calculated based on the second sensing information; and a fourth deviation condition which compares the normal distribution, which is past driving information, with the actual distribution calculated based on the second sensing information.

[0127] For example, the above deviation index may include all of the above first to fourth deviation conditions, but the present invention is not limited thereto.

[0128] Specifically, the above deviation index may include a plurality of individual deviation indices calculated according to predetermined deviation conditions.

[0129] For example, the above deviation index may include at least one of a first deviation index, a second deviation index, a third deviation index and / or a fourth deviation index.

[0131] The sorting device (120) can calculate the first deviation index. (S121-1)

[0132] In step (S121-1), the selection device (120) can calculate a first deviation index to compare the predicted trajectory predicted by the autonomous driving model with the actual trajectory calculated based on the second sensing information.

[0133] In step (S121-1), the first deviation index may be a value that quantifies the difference between the predicted trajectory, which is reference information predicted by the autonomous driving model, and the actual trajectory, which is actual information calculated based on the second sensing information.

[0134] In step (S121-1), the first deviation index may be a value for determining how accurately the predicted trajectory generated by the autonomous driving model reflects the actual driving results of the vehicle.

[0135] In step (S121-1), the predicted trajectory may be information indicating a target driving path that the vehicle must follow for a certain period of time in the future, using the first sensing information as input to the autonomous driving model.

[0136] In step (S121-1), the actual trajectory may be information representing the path actually traveled by the vehicle based on the second sensing information.

[0137] The sorting device (120) can calculate a first deviation index based on the vehicle's lateral deviation or heading error.

[0138] For example, the first deviation index may include a first-1 deviation index representing the lateral deviation between the predicted trajectory and the actual trajectory, and a first-2 deviation index representing the error in the direction of travel between the predicted trajectory and the actual trajectory.

[0139] For example, the actual trajectory used to calculate the first deviation index can be calculated based on the vehicle's position and attitude information obtained through the second sensing information.

[0140] Here, the second sensing information used may be the vehicle's position coordinates measured through a GPS sensor, the vehicle's attitude, acceleration, or rotational speed measured through an IMU sensor.

[0141] In addition, the second sensing information may be lane position or road boundary information via a camera sensor, or relative position information with respect to surrounding objects via a LiDAR or RADAR sensor.

[0142] Accordingly, the lateral deviation can be calculated based on the difference in lateral distance between the actual position of the vehicle (actual trajectory) obtained through GPS or camera sensors and the reference position on the predicted trajectory.

[0143] In addition, the direction of travel error can be calculated based on the angle difference between the actual direction of travel (actual trajectory) of the vehicle obtained through the IMU sensor and the reference direction of travel corresponding to the predicted trajectory.

[0144] However, this is not limited thereto, and specific examples of the second sensing information can be varied in many ways to the extent that they are obvious to a person skilled in the art.

[0145] For example, the above predicted trajectory may be a path (trajectory) generated based on the results of the autonomous driving model learning based on past driving data and / or driving data of a human driver.

[0146] Specifically, the autonomous driving model may be pre-trained using training data that includes the vehicle's sensing information and corresponding driving trajectory data, and the trained model may be configured to receive the first sensing information and output a target driving trajectory that the vehicle must follow.

[0147] In addition, the above-mentioned autonomous driving model can be trained based on driving data collected in a real road environment, driving data generated in a simulation environment, or data reflecting the driving patterns of experts.

[0148] For example, the above-mentioned autonomous driving model may be a model trained to simulate the driving path of a human driver.

[0149] However, not limited thereto, the method for generating the above-mentioned prediction trajectory may utilize known technology and may be modified in various ways to the extent obvious to a person skilled in the art.

[0150] For example, the 1-1 deviation index can be calculated based on the difference in lateral distance between the reference position of the vehicle on the predicted trajectory (the position coordinates of the vehicle on the predicted trajectory generated by the autonomous driving model) and the actual position of the vehicle on the actual trajectory.

[0151] In addition, the above first and second deviation indices can be calculated based on the angle difference between the reference direction of travel of the vehicle corresponding to the predicted trajectory (the direction of movement at the position coordinates of the vehicle on the predicted trajectory) and the actual direction of travel of the vehicle corresponding to the actual trajectory.

[0152] For example, the above lateral distance difference can be calculated as the shortest vertical distance value from the actual position of the vehicle driven by the first driving information to the predicted trajectory.

[0153] For example, the above first- and second deviation indices can be calculated based on the angle difference between the reference direction of travel corresponding to the predicted trajectory and the actual direction of travel corresponding to the actual trajectory of the vehicle driven by the first driving information.

[0154] For example, the sorting device (120) may determine that the first deviation condition is satisfied when at least one of the first-1 deviation index and the first-2 deviation index is greater than or equal to a predetermined threshold value.

[0155] For example, in the step (S123) described below, the sorting device (120) may determine that the first deviation condition is satisfied if at least one of the first-1 deviation index and the first-2 deviation index is greater than or equal to a predetermined threshold value.

[0156] Alternatively, the sorting device (120) may calculate the first deviation index by combining the first-1 deviation index and the first-2 deviation index, and then determine that the first deviation condition is satisfied if the first deviation index is greater than or equal to a predetermined threshold value.

[0157] For example, the first deviation index can be calculated by adding, averaging, or weighting the first-1 deviation index and the first-2 deviation index.

[0158] At this time, the predicted trajectory may also include reference position information and reference direction of travel information corresponding to the first-1 deviation index and the first-2 deviation index, respectively, and each of the reference information may be calculated and defined under the same method to correspond to the calculation method of the first-1 deviation index and the first-2 deviation index.

[0159] In other words, as the value of the first deviation index increases, it may mean that the difference between the predicted trajectory generated by the autonomous driving model and the actual driving result of the vehicle increases, which may indicate that the prediction accuracy or driving control performance of the autonomous driving model is degraded.

[0161] The sorting device (120) can calculate a second deviation index. (S121-2)

[0162] In step (S121-2), the sorting device (120) can calculate a second deviation index to determine whether the safety rule of the actual trajectory calculated based on the second sensing information is complied with.

[0163] In step (S121-2), the second deviation index may be a value that quantifies whether the actual driving state calculated based on the second sensing information complies with safety rules.

[0164] In step (S121-2), the second deviation index may be a value for determining whether the safety rule of the actual trajectory is complied with.

[0165] Specifically, the second deviation index can be calculated based on whether there is a lane departure, time to collision (TTC), responsibility-sensitive safety (RSS), or violation of traffic laws.

[0166] For example, the second deviation index may include a second-1 deviation index indicating whether there is a lane departure, a second-2 deviation index indicating the time remaining until a collision (probability of collision), or a third deviation index indicating whether the vehicle maintains a safe distance.

[0167] For example, the actual driving state used to calculate the second deviation index can be calculated based on the vehicle's position, speed, acceleration, and relative position information with respect to surrounding objects obtained through the second sensing information.

[0168] For example, the second sensing information can obtain lane information or signal information in the form of an image captured through a camera sensor, distance and relative speed information with respect to surrounding objects measured through a radar or lidar sensor, and vehicle location and driving status information through GPS and IMU sensors.

[0169] Accordingly, the above 2-1 deviation index can be calculated based on whether the actual driving state of the vehicle violates lane departure prohibition conditions or traffic law conditions.

[0170] In addition, the above 2-2 deviation index can be calculated as a value that increases when the Time To Collision (TTC), calculated based on the distance and relative speed between the vehicle and the object in front, is below a predetermined standard.

[0171] Here, the Time To Collision (TTC) may refer to the time remaining until a collision occurs, assuming the current state is maintained, based on the relative distance and relative speed between the vehicle and the object in front.

[0172] Specifically, the time to collision (TTC) can be calculated by dividing the distance between the vehicle and the forward object by the relative speed between the vehicle and the forward object.

[0173] For example, as the relative speed increases in a situation where the distance to a forward object decreases, the time remaining until the collision may decrease, and if the time remaining until the collision is below a predetermined standard (e.g., 7 seconds), it may be determined that there is a high risk of collision.

[0174] In addition, Responsibility-Sensitive Safety (RSS) is a value defined to ensure safe driving between vehicles and may serve as a criterion for defining the minimum safe distance that must be maintained between a vehicle and surrounding objects.

[0175] For example, Responsibility-Sensitive Safety (RSS) may be a value calculated based on the vehicle's current speed, possible range of acceleration and deceleration, reaction time, etc., as well as a predefined value for the minimum safe distance that must be maintained between the vehicle and the object in front.

[0176] In addition, the above 2-3 deviation indices can be calculated based on the difference between the actual distance between the vehicle and surrounding objects and a predetermined safety distance standard, and the safety distance standard can be defined based on Responsibility-Sensitive Safety (RSS).

[0177] For example, the sorting device (120) may determine that the second deviation condition is satisfied when at least one of the second-1 deviation index, the second-2 deviation index, or the second-3 deviation index is greater than or equal to a predetermined threshold value.

[0178] Alternatively, the sorting device (120) may calculate the second deviation index by combining the second-1 deviation index, the second-2 deviation index, and the second-3 deviation index, and then determine that the second deviation condition is satisfied if the second deviation index is greater than or equal to a predetermined threshold value.

[0179] For example, the second deviation index may be calculated by summing, averaging, or weighting the second-1 deviation index, the second-2 deviation index, and the second-3 deviation index.

[0180] For example, as the value of the second deviation index increases, it may mean that the actual driving condition of the vehicle is more likely to violate safety rules, which may indicate that the vehicle is approaching a collision risk state or is in an unsafe driving state, such as lane departure.

[0182] The sorting device (120) can calculate the third deviation index. (S121-3)

[0183] The selection device (120) can calculate a third deviation index to compare the reference trajectory generated by the reference model with the actual trajectory calculated based on the second sensing information.

[0184] In step (S121-3), the third deviation index may be a value indicating how much the output result of the autonomous driving model differs from general driving standards or normal driving behavior.

[0185] That is, in step (S121-3), the third deviation index may be a value that quantifies the difference between the control command generated by the reference model and the control command of the actual vehicle.

[0186] In step (S121-3), unlike the first deviation index which considers the prediction accuracy of the autonomous driving model (the degree of agreement with the target value of the autonomous driving model), the third deviation index may be a value for determining whether the actual driving result of the vehicle conforms to general driving standards.

[0187] For example, the reference model may include at least one of a rule-based driving policy (rule-based planner), a model designed to simulate driving data of human drivers collected in advance, and / or driving behavior of experts.

[0188] Here, the reference trajectory generated by the reference model is a standard driving path that the vehicle must follow using the first sensing information as input, and may be information representing an ideal driving path that the vehicle must take in normal driving situations.

[0189] For example, the above reference trajectory may be a rule-based path generated by considering traffic regulations, road structure, lane information, and the location of surrounding objects, or a path averaged or optimized based on driving data from multiple human drivers.

[0190] The third deviation index may be a value that quantifies the difference between the reference trajectory generated by the reference model and the actual trajectory calculated based on the second sensing information.

[0191] Here, the predicted trajectory is a driving path generated by the autonomous driving model to control the actual vehicle, whereas the reference trajectory may be a driving path for comparison purposes generated according to a standard defined independently of the output of the autonomous driving model.

[0192] That is, the predicted trajectory is an execution path directly used for vehicle control, whereas the reference trajectory can be used as a reference path to determine whether the actual driving result of the vehicle is normal.

[0193] For example, the predicted trajectory may be a path derived based on the learning results of an autonomous driving model, and the reference trajectory may be a standard path generated solely based on traffic laws, rule-based policies, or human driving data.

[0194] Accordingly, the first deviation index is a value for evaluating the predictive performance of the autonomous driving model itself, whereas the third deviation index may be a value for evaluating how well the output of the autonomous driving model matches the reference driving behavior.

[0195] In step (S121-3), the reference model may be a model that generates a driving trajectory or control command of a vehicle according to criteria defined independently of the autonomous driving model.

[0196] Specifically, the third deviation index can be calculated based on the difference between the reference position and reference direction of travel on the reference trajectory and the actual position and actual direction of travel on the actual trajectory.

[0197] For example, the third deviation index above can be calculated based on the sum, average, or weighted sum of the differences in lateral distance and the angle of travel between the reference trajectory and the actual trajectory, respectively.

[0198] For example, the third deviation index can be calculated by converting the difference between the steering angle, acceleration, and braking commands of the reference model and the corresponding control values ​​of the actual vehicle into a vector value, and based on the Euclidean distance (L2 distance) representing the difference in magnitude of the vector.

[0199] For example, the reference trajectory generated by the reference model may be a path created for a vehicle to travel at a constant speed along the center of the lane on a road section where a straight lane is maintained.

[0200] For example, when an actual vehicle is driving according to a predicted trajectory generated by an autonomous driving model, if the vehicle deviates from the center of the lane or performs a steering change, the actual trajectory may form a path different from the reference trajectory.

[0201] Accordingly, the third deviation index may increase through the difference in lateral distance and the difference in direction of travel between the reference trajectory and the actual trajectory, and this may be determined as the autonomous driving model performing control different from the reference driving behavior.

[0202] In addition, if the control command generated by the reference model is defined to maintain a constant steering angle and a constant acceleration, when the actual vehicle performs abrupt steering changes or unnecessary acceleration and deceleration, the difference between the control commands increases, and accordingly, the third deviation index may increase.

[0203] For example, the actual trajectory and actual control command used to calculate the third deviation index can be calculated based on the vehicle's position, attitude, speed, and relative position information with respect to surrounding objects obtained through the second sensing information.

[0204] Here, the second sensing information can acquire the vehicle's location coordinates through a GPS sensor, acquire the vehicle's direction of travel, angular velocity, and acceleration information through an IMU sensor, and acquire lane position and road structure information through a camera sensor.

[0205] In addition, the second sensing information can obtain distance and relative position information with surrounding objects through a LiDAR or RADAR sensor.

[0206] Accordingly, the third deviation index can be calculated based on the positional difference between the actual trajectory calculated through the GPS and camera sensors and the reference trajectory, and can be calculated based on the angle difference between the actual direction of travel of the vehicle obtained through the IMU sensor and the reference direction of travel corresponding to the reference trajectory.

[0207] In addition, the third deviation index can be calculated based on the difference between the actual steering angle, acceleration, and braking commands obtained from the vehicle's electronic control unit and the control commands generated by the reference model.

[0208] For example, the difference between control commands can be calculated by forming a vector of the differences between the steering angle, acceleration, and braking values, and then using the Euclidean distance (L2 distance) representing the magnitude of the vector.

[0209] However, not limited thereto, specific examples of the second sensing information for calculating the third deviation index can be varied in various ways that are obvious to a person skilled in the art.

[0210] For example, the first deviation index is a value representing the difference between the predicted trajectory generated by the autonomous driving model and the actual driving result of the vehicle, and may be an indicator for evaluating the prediction accuracy or driving control performance of the autonomous driving model.

[0211] On the other hand, the third deviation index is a value representing the difference between the reference driving trajectory or control command generated by the reference model and the actual driving result of the vehicle, and may be an indicator for evaluating how well the output of the autonomous driving model conforms to the reference normal driving behavior.

[0212] In other words, as the value of the third deviation index increases, it may mean that the difference between the actual driving result of the vehicle and the normal driving behavior defined by the reference model increases, which may indicate that the output of the autonomous driving model deviates from general driving standards.

[0214] The sorting device (120) can calculate the fourth deviation index. (S121-4)

[0215] In step (S121-4), the sorting device (120) can calculate a fourth deviation index to compare the normal distribution, which is past driving information, with the actual distribution calculated based on the second sensing information.

[0216] In step (S121-4), the fourth deviation index may be a value that quantifies how different the distribution of the actual driving state calculated based on the second sensing information is from the distribution of past normal driving data.

[0217] In step (S121-4), the fourth deviation index may be a value for determining the degree to which the actual driving condition deviates from the normal driving pattern.

[0218] Specifically, in the above-mentioned fourth deviation index, the normal distribution may be information representing the distribution of driving variables calculated based on driving data collected and judged to be a past normal driving situation.

[0219] Specifically, the actual distribution in the fourth deviation index may be information representing the distribution of driving variables calculated based on the current vehicle driving data obtained through the second sensing information.

[0220] For example, the above normal distribution may be a distribution calculated based on driving data collected from vehicles judged to be driving normally in a specific road environment, traffic situation, or time period.

[0221] Specifically, the above normal distribution may be defined in the form of an average value or a probability density function for any one of the indicators of vehicle speed, acceleration, steering angle, lateral position from the center of the lane, or distance from surrounding objects.

[0222] In addition, the actual distribution may be a distribution of driving variables calculated by accumulating the current vehicle's driving data obtained through the second sensing information over a certain time interval.

[0223] For example, the above actual distribution can be calculated by statistically processing values ​​regarding the vehicle's speed, acceleration, steering angle, lane position, or distance to surrounding objects, etc., in the same manner as the above normal distribution (in the form of an average value or a probability density function).

[0224] Accordingly, the sorting device (120) can calculate a fourth deviation index by comparing the difference between the normal distribution and the actual distribution.

[0225] For example, the fourth deviation index may be a value calculated based on the difference between the mean value of the actual distribution and the mean value of the normal distribution.

[0226] For example, the above-mentioned fourth deviation index can be calculated using a statistical distance representing the difference between the above-mentioned actual distribution and the above-mentioned normal distribution.

[0227] For example, statistical distance can be calculated according to the measurement method of Kullback-Leibler divergence, but the present invention is not limited thereto, and the method of calculating statistical distance can be varied in various ways that are obvious to a person skilled in the art.

[0228] In addition, the above-mentioned fourth deviation index can be calculated based on whether the actual distribution for a specific driving variable deviates from the confidence interval of the above-mentioned normal distribution.

[0229] In other words, as the value of the fourth deviation index increases, it may mean that the difference between the current vehicle's driving pattern and the past normal driving pattern increases, which may indicate that the current driving state is an abnormal driving state deviating from the typical driving data distribution.

[0231] Referring to FIGS. 4 to 6, the screening device (120) can determine a predetermined threshold value. (S122)

[0232] The step of determining a predetermined threshold value (S122) may include: a step of calculating the activity of an object capable of activity within a target area that includes the location where the vehicle is located (S122-1); a step of adjusting the predetermined threshold value according to a predetermined adjustment method (S122-2); a step of determining whether the adjusted predetermined threshold value satisfies a predetermined exception condition (S122-3); a step of calculating functionality, which is an indicator representing the state of the vehicle operating based on driving information (S122-4); and a step of readjusting the predetermined threshold value considering activity and functionality (S122-5).

[0233] First, the sorting device (120) can calculate the activity of an object that can be active within a target area, which is a zone that includes the location where the vehicle is located, based on the location where the vehicle is located. (S122-1)

[0234] Step (S122-1) may include a step (S122-1a) for calculating the activity of a person who is an object capable of activity within the target area and a step (S121-1b) for calculating the activity of an object who is an object capable of activity within the target area.

[0235] Here, the criterion for the location where the vehicle is situated may refer to the location of the vehicle when the above-mentioned autonomous driving learning data selection method is executed.

[0236] In addition, the target area may refer to a spatial unit within a predetermined range that includes the location where the vehicle is situated.

[0237] For example, the target area may be set at the administrative district level that includes the location where the vehicle is located, and may be a district at the level of an Eup, Myeon, or Dong.

[0238] For example, if the vehicle is located in a specific neighborhood, the above target area may be set to the entire area of ​​that neighborhood.

[0239] In addition, if the vehicle is located in a specific town or township, the above target area may be set to the administrative district of that town or township.

[0240] However, not limited thereto, the above-mentioned target area may be defined as other spatial units established based on the vehicle's location and may be modified in various ways at a level obvious to a person skilled in the art.

[0241] Here, the activity of an object may be an indicator representing the possibility that the object may move or change its position to affect the vehicle's driving path, and the degree of such variation.

[0242] For example, the activity of an object can be set variably depending on the type of object, location, surrounding environment, or time of day.

[0243] For example, the activity of an object can be calculated based on at least one of the object's movement speed, change in direction of movement, variability in the range of activity, or possibility of entering the vehicle's driving path.

[0244] In step (S122-1a), the activity level of a person among the objects can be calculated based on the characteristics of pedestrians present in the target area.

[0245] Specifically, in step (S122-1a), the activity level of a person among the subjects can be calculated based on demographic information or regional characteristic information provided from an external server for the said target area.

[0246] For example, the above external server may be a web server, and human activity may be calculated using demographic data provided by a national statistical agency (e.g., Statistics Korea).

[0247] In addition, the above external server is a web server and may include local information data provided from an internet news web server, an internet portal site, or a location-based information provider server.

[0248] For example, the above regional information data may include changes in population flow in a specific region, event information, accident occurrence information, or congestion information during a specific time period.

[0249] For example, human activity can be calculated by comprehensively considering the average age group, age distribution, and / or gender distribution of people belonging to the target area.

[0250] For example, human activity can be quantified into a predetermined range of scores.

[0251] For example, a person's activity level can be expressed as a score between 0 and 100, and the activity score can be calculated by combining individual activity scores for multiple elements.

[0252] Specifically, the human activity level can be calculated by summing a first human activity score set according to the pedestrian's age group, a second human activity score set according to gender, and a third human activity score set according to time of day or density.

[0253] For example, the above-mentioned first human activity score can be set variably depending on the age group of the person (pedestrian), and a higher score may be assigned to age groups where activity is difficult to predict.

[0254] For example, the first human activity score may be set high at 30 points for pedestrians in their teens or younger or in their 60s or older, and relatively low at 15 points for pedestrians in their 20s to 50s.

[0255] For example, this may be because pedestrians under the age of 10 are more likely to make sudden movements or rapid changes in direction, and pedestrians over the age of 60 have greater variability in walking speed and movement patterns, which may have an unpredictable impact on the driving path of vehicles.

[0256] In addition, the second human activity score can be set variably depending on the pedestrian's gender, and a relatively higher score may be assigned to the gender judged to have high variability in activity patterns.

[0257] For example, the second human activity score may be set to 30 points for male pedestrians and 15 points for female pedestrians.

[0258] For example, this may be because male pedestrians may change their speed or direction relatively frequently, which can lead to higher variability in the impact on the vehicle's driving path.

[0259] In step (S122-1a), a third human activity score considering the time period in which the vehicle is located in the target area can be set based on time-based regional activity information of the target area provided by an external server.

[0260] In step (S122-1a), the third human activity score can be variably set according to the time the vehicle is located in the target area.

[0261] The third human activity score may be a score that takes into account the density of people within the target area that changes over time, and the above time-based regional activity information may be information indicating the volume of pedestrian movement or the possibility of staying within the said target area that changes at specific times.

[0262] For example, time-based regional activity information may include at least one of time-based pedestrian flow information provided by an external server, public transportation boarding and alighting information, commercial district congestion information, school commuting time information, event schedule information, road control information, accident occurrence information, or congestion information from location-based services.

[0263] Here, the external server may be a server that provides time-based regional activity information for the target area.

[0264] The external server can be network-connected via wired or wireless connection with the control device (110), the selection device (120), and / or the learning device (130), and can provide information necessary for the autonomous driving learning system (100) to implement the autonomous driving learning data management method.

[0265] For example, an external server may include at least one of the following: a Statistics Korea or Statistical Geographic Information Service (SGIS) server, a Public Data Portal server, a National Traffic Information Center server, a City Traffic Information Center server, a Korea Expressway Corporation traffic information server, a Seoul Metropolitan Transit Corporation server, a local government server, or a map server.

[0266] As a specific example, among external servers, the Statistics Korea or Statistical Geographic Information Service (SGIS) server may provide population statistics, population movement information, population distribution information, or regional living statistics by administrative district of the target area; the Public Data Portal may provide information on floating population by time of day, resident population information, pedestrian density information, boarding and alighting passenger information by bus stop, boarding and alighting passenger information by subway station, or local facility information of the target area; the National Traffic Information Center server may provide accident information, road control information, construction information, CCTV information, or traffic congestion information of the target area; the Public Transportation Operations server may provide information on boarding and alighting passengers by time of day, usage passenger numbers, peak boarding and alighting times, or public transportation congestion levels for subway stations, railway stations, bus stops, or transfer centers within the target area; and the Map server may provide location-based information indicating congestion levels by place within the target area, road congestion levels, changes in travel routes, peak visiting times, changes in the location of user terminals, or the likelihood of staying near specific facilities.

[0267] However, this is not limited to this, and specific examples regarding external servers can be varied in many ways that are obvious to a person skilled in the art.

[0268] Accordingly, the selection device (120) can determine the amount of movement and density of pedestrians in the time period when the vehicle is located in the target area based on information provided from at least one of the external servers, and set a third human activity score according to the result of the determination.

[0269] In step (S122-1a), the third human activity score may be a score that is pre-calculated and stored in the selection device (120) for each target area and time period based on past target area time period regional activity information, or a score that is calculated in real time based on time period regional activity information received from an external server corresponding to the target area where the vehicle is currently located and the current driving time.

[0270] As a specific example, if the third human activity score is a score that has been calculated and stored in advance for each target area and time period, the selection device (120) can determine the target area where the vehicle is located based on the vehicle's current location, determine the time period to which the vehicle's driving time belongs, and then calculate the third human activity score that has been stored in advance corresponding to the target area and time period.

[0271] For example, the selection device (120) can set time intervals of one hour for each target area and pre-calculate and store a third human activity score based on past time-based regional activity information corresponding to each time interval.

[0272] Specifically, the selection device (120) can receive population distribution information by administrative district, population movement information, or living statistics information of the target area from an external server such as the National Statistical Office or Statistical Geographic Information Service (SGIS), receive information on the floating population or living population by time of day of the target area from an external server such as a public data portal server or a local government server, and receive information on the number of passengers boarding, passengers alighting, or peak boarding and alighting times by time of day for subway stations, railway stations, bus stops, or transfer centers within the target area from an external server such as a public transportation operation server.

[0273] In step (S122-1a), if the third human activity score is a score that is pre-calculated and stored for each target area and time period, the selection device (120) can pre-calculate at least one of the following for each target area and time period: a flow score corresponding to past time-based flow population information, a traffic inflow score corresponding to past public transportation boarding and alighting information, a facility time score corresponding to past school commuting time information, and a location congestion score corresponding to past location-based service congestion information.

[0274] For example, the selection device (120) can calculate a floating population score of 10 points if the number of floating populations in the past time period corresponding to 08:00 to 09:00 of the target area is greater than or equal to a predetermined first standard value (e.g., 1,500 people), calculate a traffic inflow score of 5 points if the number of people getting on and off public transportation in the past is greater than or equal to a predetermined second standard value (e.g., 600 people), calculate a facility time score of 10 points if the time period corresponds to school commuting time, and calculate a location congestion score of 5 points if the congestion grade of the past location-based service is greater than or equal to a predetermined standard.

[0275] Accordingly, the screening device (120) can calculate and store a third human activity score corresponding to 08:00 to 09:00 of the target area as 30 points by summing the above-mentioned floating population score, the above-mentioned traffic inflow score, the above-mentioned facility time score, and the above-mentioned location congestion score.

[0276] However, not limited thereto, specific numerical values ​​for the first and second reference values ​​may be varied in various ways to the extent that they are obvious to a person skilled in the art.

[0277] Alternatively, in step (S122-1a), the screening device (120) can calculate in real time at least one of a flow score corresponding to time-based flow information, a traffic inflow score corresponding to public transportation boarding and alighting information, a facility time score corresponding to school commuting time information, and a location congestion score corresponding to location-based service congestion information.

[0278] The screening device (120) can calculate a third human activity score in real time by averaging at least one of the above-mentioned floating population score, the above-mentioned traffic inflow score, the above-mentioned facility time score and / or the above-mentioned location congestion score.

[0279] As a specific example, if the third human activity score is a score calculated in real time, the selection device (120) can specify a target area based on the vehicle's current location and receive time-based regional activity information corresponding to the target area and current driving time from an external server based on the vehicle's current driving time.

[0280] For example, the selection device (120) can receive current time-based population flow information from a public data portal server or a local government server to calculate a population flow score, receive current time-based boarding and alighting information from a public transportation operation server to calculate a traffic inflow score, and receive school commuting time information from a facility operation server or a local government server to calculate a facility time score.

[0281] Additionally, the selection device (120) can calculate a location congestion score by receiving location-specific congestion information or user terminal location change information for the current time period from a map server or a location-based service server.

[0282] As a specific example, if a vehicle is located in a target area including an elementary school and a subway station and the current driving time is 08:30, the selection device (120) can calculate a floating population score of 10 points if the number of floating populations by current time period received from a public data portal server or a local government server is greater than or equal to a predetermined first standard value (e.g., 500 people), and can calculate a traffic inflow score of 5 points if the number of people getting on and off at a subway station or bus stop by current time period received from a public transportation operation server is greater than or equal to a predetermined second standard value (e.g., 700 people).

[0283] Additionally, the selection device (120) can calculate a facility time score of 10 points if the current driving time corresponds to the school commuting time based on school commuting time information received from a facility operation server or a local government server, and calculate a location congestion score of 5 points if the congestion level by place around the school received from a map server or a location-based service server or the change in location of the user terminal is greater than or equal to a predetermined standard.

[0284] Accordingly, the screening device (120) can calculate a third human activity score of 30 points in real time by summing the above-mentioned floating population score, traffic inflow score, facility time score and location congestion score.

[0285] That is, the screening device (120) can calculate a third human activity score in real time by summing at least one of the above-mentioned floating population score, traffic inflow score, facility time score and location congestion score.

[0287] Next, the selection device (120) can calculate the activity of objects that are objects capable of activity within the target area. (S122-1b)

[0288] In step (S122-1b), the activity of an object among the objects can be calculated based on the characteristics of the object existing in the target area.

[0289] Specifically, in step (S122-1b), the activity of an object among the objects can be calculated based on regional characteristic information, industry information, or operating time information provided by an external server for the said target area.

[0290] For example, an external server may provide image-based map information or street image data for a target area, and a selection device (120) may identify the arrangement of objects, industry characteristics, or road occupancy status within the section based on the image-based map information.

[0291] For example, the above image-based map information may include information indicating the location and arrangement status of buildings, stalls, mobile sales stands, or goods around the road.

[0292] Here, the object may include at least one of a building, facility, cargo, stall, mobile sales stand, construction structure, or fixed and variable objects placed around a road.

[0293] For example, the activity of an object can be calculated by comprehensively considering the types of businesses related to the object existing within the target area, the distribution of businesses, and whether they operate during business hours or by time of day.

[0294] For example, the activity of an object can be quantified into a predetermined range of scores.

[0295] For example, the activity of an object can be expressed as a score between 0 and 100, and the activity score can be calculated by combining the individual activity scores for multiple elements.

[0296] Specifically, the activity of the object can be calculated by summing the first object activity score set according to the industry, the second object activity score set according to business hours or operating hours, and the third object activity score set according to the arrangement form or occupancy status of the object.

[0297] For example, when considering the types of businesses of objects located within the above-mentioned target area, if the number of stores corresponding to business types involving frequent movement of people or goods, such as restaurants, food stalls, street vendors, traditional markets, construction sites, or logistics loading and unloading areas, is at least the first proportion (e.g., 50%) of the total number of stores (objects), the first object activity score of the object may be set high at 30 points, and if the number of stores corresponding to business types with relatively low variability in surrounding movement, such as general office facilities or residential buildings, is at least the first proportion of the total number of stores (objects), it may be set at 15 points.

[0298] This may be because, in the case of objects related to restaurants, street vendors, traditional markets, or construction sites, the variability in the impact on vehicle driving paths can be high due not only to changes in the location of the objects themselves but also to human access, loading and unloading of goods, and the installation or removal of temporary structures.

[0299] Additionally, in step (S122-1b), the second object activity score, which is set according to business hours or operating hours, can be calculated based on the number of objects actually operating or running within the target area at the vehicle's current driving time.

[0300] Specifically, the sorting device (120) receives business time information or operating time information of objects within a target area from an external server and can calculate the number of objects whose current driving time of the vehicle is included in the business time or operating time.

[0301] Here, the external server may include at least one of a local government server, a public data portal server, a business registration information provision server, a map server, a location-based service server, a market operation server, a facility operation server, or a commercial area analysis server. For example, a local government server, a public data portal server, or a market operation server may provide information on operating hours or licensing information regarding street vendors, traditional markets, shops, construction sites, road occupancy facilities, or temporary sales facilities within the target area, and a business registration information provision server, a map server, a location-based service server, or a commercial area analysis server may provide information on business hours, current operating status, or days off for restaurants, shops, logistics facilities, or service industry facilities within the target area.

[0302] The sorting device (120) can calculate a higher second object activity score by determining that the greater the number of objects operating or being in business at the current driving time of the vehicle, the greater the possibility of human access, loading or unloading of goods, use of temporary structures, or changes in the roadside occupancy status in the target area.

[0303] For example, the sorting device (120) can calculate a second object activity score of 30 points if the number of objects in operation or business at the current driving time of the vehicle is greater than or equal to a predetermined first object standard value (e.g., 200 objects), calculate a second object activity score of 20 points if the number of objects in operation or business is less than the first object standard value and greater than or equal to a predetermined second object standard value (e.g., 100 objects), and calculate a second object activity score of 10 points if the number of objects in operation or business is less than the second object standard value.

[0304] As a specific example, if the target area is a zone including a traditional market and the current driving time of the vehicle is 10:30, the sorting device (120) can receive information on the operating hours of stores, street vendors, mobile sales stands, or road occupancy facilities within the target area from a local government server or a market operation server.

[0305] In this case, the sorting device (120) can calculate a second object activity score of 30 points if the number of stores, street vendors, mobile sales stands, or road occupancy facilities that include the current driving time of 10:30 in the operating time is 200 or more.

[0306] As another example, if the number of objects in operation or business at the current driving time of the vehicle is less than 100, the sorting device (120) can calculate the second object activity score as 20 points.

[0307] However, not limited thereto, the types of objects, reference values, and scores for calculating the second object activity score may be varied by taking into account the area of ​​the target zone, road width, industry distribution, or past driving data.

[0308] In step (S122-1b), the third object activity score, which is set according to the arrangement or occupancy status of objects, can be calculated based on the number of objects occupying the roadside, pedestrian walkway, or the target area where vehicles are traveling within the target area.

[0309] In step (S122-1b), the third object activity score, which is set according to the arrangement form or occupancy status of the object, can be calculated based on whether the object within the target area is arranged in a movable form (movable object) or in a fixed form (fixed object).

[0310] In step (S122-1b), the sorting device (120) classifies the arrangement of objects within the target area into fixed objects and movable objects, and determines that the more the number of movable objects among the fixed objects and movable objects, the higher the likelihood that the occupancy range or placement location of the objects will change over time, thereby allowing for a higher third object activity score to be calculated.

[0311] Here, mobile objects may be objects whose location, arrangement direction, or scope of occupancy may change over time, such as street stalls, mobile sales stands, mobile signboards, cargo, delivery racks, temporary booths, temporary structures, loaded and unloaded goods, or construction equipment, and fixed objects may be objects whose location or scope of occupancy remains relatively constant, such as buildings, fixed signs, fixed fences, fixed facilities, fixed stalls, or road structures.

[0312] Specifically, the sorting device (120) receives information on the placement location of objects within a target area, road occupancy permit information, temporary structure installation information, construction information, or image-based map information from an external server, can determine whether the objects are mobile or fixed objects, and can calculate the number of objects adjacent to the drivable area of ​​a vehicle or occupying at least a part of the drivable area.

[0313] The sorting device (120) can calculate a higher third object activity score by determining that the greater the number (or ratio) of movable objects adjacent to or occupying at least a part of the drivable area, the greater the impact on the vehicle's avoidance path or drivable width.

[0314] For example, the sorting device (120) may calculate a third object activity score of 30 points when the ratio of the movable object among all objects is greater than or equal to a predetermined first occupancy standard value (e.g., 50%), calculate a third object activity score of 20 points when the number of the movable object among all objects is less than the first occupancy standard value and the number of the movable object is greater than or equal to a predetermined second occupancy standard value (e.g., 30%), and calculate a third object activity score of 10 points when the number of the movable object among all objects is less than the second occupancy standard value.

[0315] However, not limited thereto, the first occupancy standard value and the second occupancy standard value may be varied by a person skilled in the art, taking into consideration the area of ​​the target zone, road width, pedestrian walkway width, number of lanes, industry distribution, or past driving data.

[0317] Next, the screening device (120) can adjust a predetermined threshold value according to a predetermined adjustment method. (S122-2)

[0318] In step (S122-2), the predetermined adjustment method may be a method of lowering the predetermined threshold value as the variability of the expected activity range of the object increases.

[0319] Here, the variability of the expected range of activity can refer to the activity of people and objects.

[0320] For example, the higher a person's activity level, the larger the downward rate (adjustment amount) from a predetermined threshold may be. Conversely, the lower a person's activity level, the smaller the downward rate (adjustment amount) from a predetermined threshold may be.

[0321] For example, the higher the activity of an object, the larger the rate of downward movement from a predetermined threshold (adjustment amount). Conversely, the lower the activity of an object, the smaller the rate of downward movement from a predetermined threshold (adjustment amount).

[0322] Here, the predetermined adjustment method can adjust the predetermined threshold value only when the activity of the expected activity range of the object exists above a specific standard.

[0323] In step (S122-2), the predetermined adjustment method by the selection device (120) may be a method of adjusting a predetermined threshold value by taking into account both the activity of a person and the activity of an object.

[0324] In addition, the above-mentioned predetermined threshold values ​​may be set differently for each of the first deviation index, the second deviation index, the third deviation index, and the fourth deviation index.

[0325] Specifically, the above-mentioned predetermined threshold value can be defined as a different reference value depending on the type of deviation index, and can be individually adjusted according to the characteristics of each deviation index.

[0326] For example, a first threshold value, which is a predetermined threshold value corresponding to the first deviation index, can be set based on an acceptable error range between the predicted trajectory and the actual trajectory, and a second threshold value, which is a predetermined threshold value corresponding to the second deviation index, can be set as a criterion for determining whether there is a violation of safety rules.

[0327] In addition, the third threshold value, which is a predetermined threshold value corresponding to the third deviation index, can be set as a criterion for determining the difference from the reference model, and the fourth threshold value, which is a predetermined threshold value corresponding to the fourth deviation index, can be set as a criterion for determining the degree of deviation of the actual distribution relative to the normal distribution.

[0328] For example, the first threshold value may be set to a value corresponding to a lateral deviation of 0.5 m or a forward direction error of 5 degrees, and the second threshold value may be set based on the case where the time to collision (TTC) remaining is 7 seconds or less.

[0329] In addition, the third threshold value may be set based on the case where the distance difference between the reference trajectory and the actual trajectory is 1.0m or more, and the fourth threshold value may be set based on the case where the statistical distance value of the actual distribution relative to the normal distribution exceeds a predetermined standard (e.g., a KL divergence value of 0.3 or more).

[0330] In addition, the above-determined adjustment method can be applied independently to each of the above-determined threshold values, namely the first threshold value, the second threshold value, the third threshold value, and the fourth threshold value.

[0331] Specifically, the predetermined adjustment method can lower the first threshold value (e.g., 20), which is a predetermined threshold value, by a ratio of 20% (based on the predetermined threshold value) when the activity level of a person in the target area where the vehicle is located while driving is above the first threshold value (e.g., 60 points).

[0332] However, not limited thereto, the first standard and the first level values ​​may be varied in various ways that are obvious to a person skilled in the art.

[0333] Specifically, the predetermined adjustment method can lower the predetermined threshold value (e.g., 20) by a ratio of 20% (based on the predetermined threshold value) when the activity level of objects in the target area where the vehicle is located while driving is higher than the second criterion (e.g., 60 points).

[0334] However, not limited thereto, the second standard and the second level figures may be varied in many ways that are obvious to a person skilled in the art.

[0335] For example, the first level and the second level may be applied at the same rate to each of the first threshold, the second threshold, the third threshold, and the fourth threshold, so that the threshold can be adjusted.

[0336] Specifically, the above-determined adjustment method may apply the same downward adjustment ratio (e.g., 20%) to each of the first threshold, second threshold, third threshold, and fourth threshold.

[0337] For example, if the activity of the person or the activity of the object is within the reference range, the first threshold, second threshold, third threshold, and fourth threshold can each be adjusted to decrease at the same rate.

[0338] In addition, the predetermined adjustment method may be a method in which, when the activity of people and the activity of objects in the above target area are both above their respective standards, the predetermined threshold is further lowered by the second level after being lowered according to the first level.

[0339] Additionally, the first level may have a larger value as the activity level of the people in the target area is greater than the first criterion. Conversely, the first level may have a smaller value as the activity level of the people in the target area is greater than or closer to the first criterion.

[0340] Additionally, the second level may have a larger value as the activity of objects in the target area is greater than the second criterion. Conversely, the second level may have a smaller value as the activity of objects in the target area is greater than or closer to the second criterion.

[0341] For example, if a predetermined threshold is 20 and both the activity of a person and the activity of an object are above the standard, the predetermined threshold may be adjusted to 16 according to the first level, and then further adjusted to 12.8 according to the second level.

[0342] However, this is not limited to this, and specific examples of predetermined threshold values ​​can be varied in various ways that are obvious to a person skilled in the art.

[0343] In this way, by gradually lowering a predetermined threshold value while considering both human activity and object activity, the risk inherent in the vehicle's driving environment can be reflected more precisely.

[0344] In particular, in sections where both human and object activity are high, sudden pedestrian movements, changes in object occupancy status, or variability in the road environment may increase simultaneously; therefore, these complex risk situations can be effectively reflected by cumulatively lowering the threshold value.

[0345] In addition, as the predetermined threshold decreases, the predetermined deviation condition is satisfied even for the same deviation; thus, even minute deviations occurring in sections likely to affect the vehicle's driving path can be identified as error sections, enabling learning of small deviations in high-activity environments.

[0346] Therefore, by adjusting the threshold value according to a predetermined adjustment method, taking into account the risks that may occur in the section where the vehicle is driving, data can be selected based on stricter criteria for high-risk driving sections, and the quality of training data can be improved so that the autonomous driving model can effectively respond to the variability of the actual driving environment.

[0348] Next, the screening device (120) can determine whether a predetermined threshold value adjusted with consideration of activity satisfies a predetermined exception condition. (S122-3)

[0349] In step (S122-3), the screening device (120) can determine whether the predetermined threshold value, which was lowered according to step (S122-2) of adjusting the predetermined threshold value according to a predetermined adjustment method, has been reduced to a level below a predetermined level.

[0350] Specifically, the screening device (120) can determine whether the adjusted predetermined threshold value is less than a preset lower threshold value (e.g., 50% or less of the initial preset threshold value).

[0351] Here, the above-mentioned predetermined exception condition may be a condition satisfied when the above-mentioned predetermined threshold is adjusted to be less than a predetermined threshold.

[0352] For example, if the initial predetermined threshold value (the aforementioned predetermined threshold value before adjustment) is 20, and the threshold value is reduced to less than 10 as a result of being repeatedly lowered according to step (S122-2), it may be determined that the aforementioned predetermined exception condition is satisfied.

[0353] The screening device (120) can determine that when a predetermined exception condition is satisfied, the currently set threshold value is excessively lowered, and even minute deviations unrelated to the vehicle's driving environment may be judged as errors.

[0354] If the threshold value is excessively low in step (S122-3), even normal driving data with low actual learning value may be excessively identified as error sections, which may reduce data collection efficiency.

[0355] Accordingly, the selection device (120) may control the adjustment degree of a predetermined threshold value by additionally considering functionality, which is an indicator of the vehicle's condition, in a subsequent step when the above-determined exception condition is satisfied.

[0357] Next, the sorting device (120) can calculate a functionality (functionality index), which is an indicator representing the state of the vehicle being operated by the driving information. (S122-4)

[0358] In step (S122-4), functionality may be an indicator representing a state related to the vehicle's driving performance or control stability.

[0359] Specifically, functionality can be calculated based on one or more factors reflecting the physical condition of the vehicle, driving history, or degree of performance degradation.

[0360] For example, functionality can be calculated based on at least one of the vehicle's accumulated mileage, vehicle's usage period, vehicle's maintenance history, vehicle's accident history, or the degree of performance degradation of vehicle components.

[0361] In addition, functionality can be quantified as a score between 0 and 100.

[0362] Specifically, functionality can be calculated by combining a first functionality score set according to the vehicle's accumulated mileage, a second functionality score set according to the vehicle's usage period, and a third functionality score set according to the vehicle's accident or maintenance history.

[0363] Here, the information used to calculate the first functional score and the second functional score may be received from the control device (110), and the information used to calculate the third functional score may be received from an external server, but the present invention is not limited thereto.

[0364] For example, the sorting device (120) can set the first functional score to 30 points when the cumulative mileage of the vehicle is less than a predetermined standard (e.g., 100,000 km), and can set it to 15 points when it is greater than or equal to the predetermined standard.

[0365] Additionally, the sorting device (120) can set the second functional score to 30 points when the vehicle's usage period is less than a predetermined standard (e.g., 5 years), and to 15 points when it is greater than or equal to the predetermined standard.

[0366] Additionally, the screening device (120) can set the third functional score to 15 points if there is an accident history or major parts replacement history in the vehicle, and to 30 points if there is no accident history.

[0367] As the degree of aging or damage to the vehicle increases, the selection device (120) may cause a decrease in sensor accuracy, a delay in driving response, or an increase in control error. Since the cause of the same deviation may be due to the performance degradation of the vehicle itself rather than the autonomous driving model, the adjusted deviation may not be judged as an error as is, but the adjustment degree of a predetermined threshold value may be limited or corrected by considering functionality.

[0369] Next, the screening device (120) can readjust a predetermined threshold value in consideration of activity and functionality. (S122-5)

[0370] In step (S122-5), the selection device (120) satisfies a predetermined exception condition when the predetermined threshold value is adjusted to be below the predetermined lower limit reference value, and the predetermined threshold value may be readjusted according to a predetermined readjustment method by considering the functionality, which is an indicator representing the state of the vehicle.

[0371] In step (S122-5), the selection device (120) can finally determine the predetermined threshold value by comprehensively considering the functionality of the vehicle and the predetermined threshold value adjusted according to step (S122-2).

[0372] The predetermined readjustment method may be a method that is performed only when the above-mentioned predetermined exception condition is satisfied.

[0373] The predetermined readjustment method may be a method of readjusting the predetermined threshold value to the predetermined lower limit threshold value when the functionality is less than the third standard.

[0374] If the above-determined exception condition is satisfied but the above-determined functionality is not less than the above-determined third standard, the above-determined threshold value can be adjusted only by the above-determined adjustment method.

[0375] Specifically, if the predetermined threshold value adjusted by the predetermined adjustment method is less than the predetermined lower limit standard value and at the same time the functionality is less than the third standard (e.g., 50 points), the predetermined threshold value adjusted by the predetermined adjustment method can be readjusted to the predetermined lower limit standard value.

[0376] Here, the preset lower limit reference value may be a predetermined ratio value of the preset threshold value before being adjusted by the preset adjustment method. For example, the preset ratio may be 50%, but the present invention is not limited thereto.

[0377] When a predetermined threshold value is continuously lowered according to the activity of objects and people, and the functionality of the vehicle is determined to be low, the threshold value may be corrected to a value higher than the adjusted value adjusted considering the activity.

[0378] This may be intended to prevent the detection of excessive error sections by mistaking the deviation for a performance degradation of the autonomous driving model, as deviations may increase due to sensor errors or control delays when the vehicle's functionality is low.

[0379] Accordingly, the screening device (120) can strictly evaluate deviations by lowering the threshold value in high-activity environments, and additionally prevents excessive lowering of the threshold value when the vehicle's functionality is low, thereby enabling the determination of error sections that reflect the driving environment and vehicle condition in a balanced manner.

[0380] In step (S122-5), the selection device (120) can set the previously predetermined threshold value, which is adjusted considering the activity and functionality of the vehicle, to the final predetermined threshold value, which is readjusted or not readjusted considering the functionality of the vehicle.

[0381] The first predetermined threshold value can be defined as the initial threshold value, the initial threshold value adjusted by the predetermined adjustment method as the adjustment threshold value, and the adjustment threshold value readjusted by the predetermined adjustment method as the readjustment threshold value.

[0382] In addition, the above-mentioned predetermined threshold value finally determined through step (S122) can be defined as the final threshold value.

[0383] The predetermined readjustment method can be readjusted for each of the first threshold value, the second threshold value, the third threshold value, and the fourth threshold value.

[0385] Referring to FIGS. 4 to 7, the sorting device (120) can determine whether the deviation index exceeds the predetermined threshold value. (S123)

[0386] In step (S123), the selection device (120) can determine whether the deviation index obtained by comparing actual information and reference information exceeds a predetermined threshold value (final threshold value) set in step (S122).

[0387] In step (S123), the screening device (120) can determine whether the deviation index exceeds a predetermined threshold value according to a predetermined deviation condition.

[0388] Step (S123) may include: a step (S123-1) in which the selection device (120) determines whether a first deviation index exceeds a predetermined threshold value under a first deviation condition, which is a condition for comparing the predicted trajectory predicted by the autonomous driving model with the actual trajectory calculated based on the second sensing information; a step (S123-2) in which the second deviation index exceeds a predetermined threshold value under a second deviation condition, which is a condition for determining whether the actual trajectory calculated based on the second sensing information complies with safety rules; a step (S123-3) in which the third deviation index exceeds a predetermined threshold value under a third deviation condition, which is a condition for comparing the reference trajectory generated by the reference model with the actual trajectory calculated based on the second sensing information; and a step (S123-4) in which the fourth deviation index exceeds a predetermined threshold value under a fourth deviation condition, which is a condition for comparing the normal distribution, which is past driving information, with the actual distribution calculated based on the second sensing information.

[0389] First, in step (S123-1), the screening device (120) can determine whether at least one of the first-1 deviation index and the first-2 deviation index is greater than or equal to a first threshold value, which is a predetermined threshold value finally calculated by considering activity and functionality.

[0390] For example, if the lateral deviation, which is the 1-1 deviation index calculated based on the difference between the actual information and the reference information, is greater than or equal to the 1st threshold value (e.g., 0.5m), it can be determined that the 1st deviation condition is satisfied and the predetermined threshold value is exceeded.

[0391] In addition, even if the direction of travel error, which is the first-second deviation index calculated based on the difference between the actual information and the reference information, is greater than the first threshold value (e.g., 5 degrees), it can be determined that the first deviation condition is satisfied and the predetermined threshold value is exceeded.

[0392] Next, in step (S123-2), the screening device (120) can determine whether at least one of the 2-1 deviation index, the 2-2 deviation index, or the 2-3 deviation index is greater than or equal to a second threshold value, which is a predetermined threshold value finally calculated by considering activity and functionality.

[0393] For example, if the lane departure status, which is the 2-1 deviation index calculated based on the difference between the actual information and the reference information, is greater than or equal to the 2 threshold value, it can be determined that the 2 deviation condition is satisfied and the predetermined threshold value is exceeded.

[0394] In addition, even if the value based on the Time To Collision (TTC), which is the 2-2 deviation index calculated based on the difference between the actual information and the reference information, is greater than or equal to the 2nd threshold value, it can be determined that the 2nd deviation condition is satisfied and the predetermined threshold value is exceeded.

[0395] In addition, even if the safety distance maintenance status, which is the 2nd-3rd deviation index calculated based on the difference between the actual information and the reference information, is greater than or equal to the 2nd threshold value, it can be determined that the 2nd deviation condition is satisfied and the predetermined threshold value is exceeded.

[0396] Next, in step (S123-3), the screening device (120) can determine whether the third deviation index calculated based on the difference between the actual information and the reference information is greater than or equal to the third threshold value, which is a predetermined threshold value finally calculated by considering activity and functionality.

[0397] For example, if the value of the third deviation index that reflects the difference in lateral distance or the difference in direction of travel between the reference trajectory and the actual trajectory is greater than or equal to the third threshold value, it can be determined that the third deviation condition is satisfied and that the predetermined threshold value is exceeded.

[0398] Next, in step (S123-4), the screening device (120) can determine whether the fourth deviation index calculated based on the difference between the actual information and the reference information is greater than or equal to the fourth threshold value, which is a predetermined threshold value finally calculated by considering activity and functionality.

[0399] For example, if the value of the fourth deviation index reflecting the statistical distance between the normal distribution and the actual distribution is greater than or equal to the fourth threshold value, it can be determined that the fourth deviation condition is satisfied and the predetermined threshold value is exceeded.

[0401] Next, referring to FIGS. 3 and FIGS. 8, when a predetermined deviation condition is satisfied, the sorting device (120) can transmit the sorting information, which is the sensing information satisfying the predetermined sorting condition, to a learning device placed outside the vehicle. (S130)

[0402] In step (S130), the predetermined selection interval may comprise at least one of the following: a first selection interval which is a time interval from a first reference point, which is a time point before the error occurrence point, to the error occurrence point; and a second selection interval which is a time interval from an error termination point, which is a time point after the error occurrence point where the predetermined deviation condition is not satisfied, to a second reference point, which is a time point after the error termination point.

[0403] In step (S130), the screening device (120) can identify the point in time when at least one deviation index is determined to exceed a predetermined threshold value by step (S123) as the point in time when an error occurs.

[0404] Referring to Fig. 8, the vertical axis represents the deviation index (TH), and the horizontal axis represents time (T).

[0405] Additionally, the baseline (Z) of FIG. 8 may correspond to a predetermined threshold value, and the point in time (Y10) when the deviation index reaches or exceeds the baseline (Z) may correspond to the point in time (t2) when an error occurs.

[0406] Additionally, the point in time (Y20) when the deviation index returns to below the baseline (Z) again may correspond to the error termination point (t3).

[0407] Here, the interval (X1) between t1 and t2 shown in FIG. 8 may correspond to the first screening interval prior to the error occurrence time, and the interval (X2) between t3 and t4 may correspond to the second screening interval after the error termination time.

[0408] That is, as shown in FIG. 8, the sorting device (120) can identify as sorting information not only the central section where the deviation index exceeds the threshold value, but also the sensing information and driving information corresponding to the previous section (X1) and the subsequent section (X2).

[0409] For example, the above section (X1) may be a context section including the cause or precursor situation of the error, and the above section (X2) may be a recovery section including the process of vehicle control returning to a normal state after the error occurs.

[0410] The central section may refer to the section from the time of error occurrence to the time prior to the time of error termination.

[0411] Specifically, the above error occurrence point may be the point in time when the above deviation index becomes greater than or equal to the above predetermined threshold.

[0412] Additionally, the selection device (120) can identify the point in time when the deviation index returns to below the predetermined threshold value after the error occurrence point as the error termination point.

[0413] Accordingly, the sorting device (120) can set at least one of the first sorting section (X1) from the first reference point prior to the error occurrence point to the error occurrence point and the second sorting section (X2) from the error termination point to the second reference point as a sorting section.

[0414] For example, the first reference point may be set to a point in time prior to a predetermined first time from the error occurrence point, and the first time may be set in the range of 1 second to 100 seconds.

[0415] In addition, the second reference point may be set to a point in time after a predetermined second time from the error termination point, and the second time may be set within the range of 1 second to 100 seconds.

[0416] For example, in step (S130), the first selection section may be a section for including a driving context prior to the actual occurrence of an error, and the second selection section may be a section for including a driving context during the process of the vehicle returning to a normal state after the occurrence of an error.

[0417] That is, in step (S130), the sorting device (120) can identify the sensing information and driving information of the vehicle corresponding to a predetermined time interval before and after the point in time when the deviation index exceeds a predetermined threshold as sorting information.

[0418] Specifically, the selection information may include first sensing information, second sensing information, driving information, and corresponding vehicle status information obtained in the selection section.

[0419] Additionally, the sorting device (120) can transmit the sorting information to a learning device (130) located outside the vehicle.

[0420] Selection information may include sensing information, driving information, and corresponding control information.

[0421] In this way, by selecting not only the driving data between the time of error occurrence and the time of error termination but also the driving data before and after the time of error occurrence, it is possible to utilize for learning not only the data at the time of the error occurrence but also the contextual information that led to the error and the information on the recovery process after the error.

[0422] Accordingly, the learning device (130) can learn the cause of the error caused by the autonomous driving model and the subsequent driving response based on the above selection information, thereby more effectively improving the performance of the autonomous driving model.

[0424] Meanwhile, an autonomous driving learning data learning method according to one embodiment of the present invention, which is implemented by a learning device which is a computing device external to a vehicle, may include the step of receiving selection information, which is sensing information detected by the vehicle that satisfies a predetermined selection condition (S210); the step of determining whether the selection information satisfies a predetermined learning condition (S220); and the step of learning an autonomous improvement model based on the selection information when the predetermined learning condition is satisfied (S230).

[0425] In addition, the above-mentioned predetermined learning condition may be a condition in which the diversity of the types of the above-mentioned selection information is greater than or equal to a predetermined standard.

[0427] In step (S210), the learning device (130) can receive selection information, which is sensing information detected by the vehicle.

[0428] Specifically, the selection information may include sensing information, control information, and / or driving information extracted from a selection section predetermined by the selection device (120).

[0429] For example, the above selection information includes all driving data before and after the time of the error occurrence, and may include image information, distance information, speed information, location information obtained from the vehicle's sensors, and corresponding control command information.

[0430] In addition, the selection information may include time information corresponding to the selection section or metadata indicating the start and end times of the error section.

[0431] Additionally, the learning device (130) can be positioned outside the vehicle and receive the sorting information in an environment separated from the sorting device (120).

[0432] Specifically, by configuring the learning device (130) separately from the vehicle, the autonomous driving model can stably perform real-time control while the vehicle is driving, and the learning device (130) can perform learning using separate computing resources based on the selection information.

[0433] For example, the autonomous driving model mounted on the vehicle can be used for vehicle control in a fixed state at the time of driving, and the learning device (130) can learn the autonomous improvement model in a path independent of vehicle control after receiving the selection information.

[0434] In addition, by separating the reception of the above-mentioned selection information and the execution of learning from the vehicle's driving control in this manner, it is possible to prevent the learning process from directly affecting the vehicle's real-time driving control.

[0435] Accordingly, the learning device (130) can perform continuous model improvement using selection information corresponding to error sections while maintaining the driving stability of the vehicle.

[0437] In step (S220), the learning device (130) can determine whether the selection information satisfies predetermined learning conditions.

[0438] Specifically, the aforementioned predetermined learning conditions may serve as criteria for determining whether the selection information is data meaningful for the learning of the autonomous improvement model.

[0439] The predetermined learning conditions may include a first learning condition in which the diversity of error types included in the selection information is greater than or equal to a predetermined standard, a second learning condition in which the number of error section data of the selection information is greater than or equal to a predetermined standard, a third learning condition in which the cumulative value of the driving distance or driving time corresponding to the selection information is greater than or equal to a predetermined standard, and a fourth learning condition determined based on whether the operator approves.

[0441] For example, the first learning condition above may be a condition in which the standard deviation of the number of screening information classified by error type is less than or equal to a predetermined standard.

[0442] Here, when the standard deviation of the number of screening information classified by error type is below a predetermined standard, it can be defined that the diversity of error types included in the screening information is above a predetermined standard.

[0443] Specifically, the learning device (130) can classify the selection information into separate error types according to deviation conditions. For example, the selection information satisfying the first deviation condition, the selection information satisfying the second deviation condition, the selection information satisfying the third deviation condition, and the selection information satisfying the fourth deviation condition can each be classified as data of a different error type. If multiple deviation conditions are satisfied in a single selection information, they may be classified as duplicate error types.

[0444] In addition, the second learning condition may be satisfied when the number of error interval data included in the selection information is greater than or equal to a predetermined standard.

[0445] Specifically, the learning device (130) can identify a plurality of error sections based on the error occurrence time and error termination time included in the selection information, and calculate the number of error sections.

[0446] For example, if the number of error intervals is 100 or more, it may be determined that the second learning condition is satisfied.

[0447] In addition, the third learning condition may be a condition satisfied when the accumulated value of the driving distance or driving time corresponding to the selection information is greater than or equal to a predetermined standard.

[0448] Specifically, the learning device (130) can calculate a total by accumulating the driving distance or driving time of each error section included in the selection information.

[0449] For example, if the total driving distance included in the above selection information is 10 km or more, or the total driving time is 30 minutes or more, it may be determined that the above third learning condition is satisfied.

[0450] In addition, the above-mentioned fourth learning condition may be a condition satisfied depending on whether the operator approves it.

[0451] Specifically, the learning device (130) can provide the selection information to the operator and receive input from the operator.

[0452] For example, if an operator determines that the above-mentioned screening information is suitable for learning and performs approval input, it may be determined that the above-mentioned fourth learning condition is satisfied.

[0453] Additionally, the learning device (130) may be configured to perform learning using the selection information when at least one of the first to fourth learning conditions is satisfied.

[0455] Next, the learning device (130) can learn an autonomous improvement model based on the selection information when the above-determined learning conditions are satisfied. (S230)

[0456] In step (S230), the learning device (130) can learn an autonomous improvement model by using the sensing information included in the selection information that satisfies the predetermined learning conditions as input data, and using the corresponding reference information or driving information as correct answer data.

[0457] Specifically, the learning device (130) can take the sensing information as input to calculate the difference between the control command or prediction trajectory output by the autonomous improvement model and the correct answer data, and update the parameters of the autonomous improvement model to minimize the difference.

[0458] For example, the learning device (130) can calculate the error between the output value of the self-improvement model and the correct answer data through a loss function, and can repeatedly update the weights of the model using a backpropagation algorithm to minimize the loss value.

[0459] In addition, the learning device (130) can perform learning by assigning weights to data corresponding to error sections in which the deviation index of the selection information exceeds a predetermined threshold.

[0460] For example, the above-mentioned autonomous improvement model can be configured in an end-to-end manner that directly receives sensing information and outputs vehicle control commands.

[0461] Accordingly, the above-mentioned autonomous improvement model can be trained to have improved control performance for driving situations in which the existing autonomous driving model caused errors.

[0463] FIG. 9 is a configuration diagram of a sorting device according to one embodiment of the present invention.

[0465] Referring to FIG. 9, a sorting device (120) according to one embodiment of the present invention may include an input / output device (101, I / O), a processor (102, processor), a memory (103, memory), a storage (104, storage), an interface (105, interface), and a bus (106, bus) as a computing device.

[0466] In addition, the sorting device (120) may further include a display unit (not shown) that displays a user interface.

[0467] An input / output device (101), a processor (102), a memory (103), a storage (104), an interface (105), a display unit (not shown) and / or a communication unit (not shown) can be combined and connected to each other via a bus (106). Here, the bus (106) may correspond to a path through which data travels.

[0468] Specifically, the input / output device (101) may include at least one of a keypad, a keyboard, a touchscreen, and a display device.

[0469] Additionally, the processor (102) may include at least one of a CPU (Central Processing Unit), MPU (Micro Processor Unit), MCU (Micro Controller Unit), GPU (Graphic Processing Unit), microprocessor, digital signal processor, microcontroller, application processor (AP), and logic elements capable of performing similar functions.

[0470] The memory (103) can load data and / or programs, etc. At this time, the memory (103) is an operational memory for enhancing the operation of the processor (101) and may include high-speed DRAM and / or SRAM, etc. The memory (103) may include one or more volatile memory devices such as DDR SDRAM (Double Data Rate Static DRAM) and SDR SDRAM (Single Data Rate SDRAM) and / or one or more non-volatile memory devices such as EEPROM (Electrical Erasable Programmable ROM) and flash memory.

[0471] Storage (104) (storage device) can store and retain data and / or programs, etc. Storage (104) may include one or more non-volatile memory devices such as a solid-state drive (SSD), a hard drive, and a flash memory. In the present invention, storage (104) can store a computer program consisting of instructions for performing a document processing method.

[0472] The interface (105) can perform the function of transmitting data to a communication network or receiving data from a communication network. The interface (105) may be wired or wireless. For example, the interface (105) may include an antenna or a wired / wireless transceiver. The interface (150) may be configured to transmit and receive data with a device such as a server or other electronic device. The interface (150) may be configured to transmit and receive data with a physical server (200), a cloud server (V10), and / or a manager device (300). The interface (105) may include a Bluetooth communication unit, a BLE (Bluetooth Low Energy) communication unit, a near field communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an infrared (IrDA, infrared Data Association) communication unit, a WFD (Wi-Fi Direct) communication unit, an UWB (ultra wideband) communication unit, an Ant+ communication unit, a mobile communication network, or a wired Ethernet network.

[0473] A display unit (not shown) may include any device capable of displaying an image, such as a display device, a screen device, or a beam projector. A display (e.g., a display) may include a panel, a holographic device, a projector, or a control circuit for controlling these. The panel may be implemented, for example, in a flexible, transparent, or wearable manner. The panel may be composed of a touch panel and one or more modules. According to one embodiment, the panel may include a pressure sensor (or force sensor) capable of measuring the intensity of pressure in response to a user's touch. The pressure sensor may be implemented integrally with the touch panel or as one or more sensors separate from the touch panel. A holographic device may display a three-dimensional image in mid-air using light interference. A projector may display an image by projecting light onto a screen. The screen may be located, for example, inside or outside an electronic device.

[0474] A server according to embodiments of the present invention may be a system formed by connecting a plurality of computing devices to each other through a network. In such cases, each module or combination of modules may be implemented as a computing device. However, the present embodiment is not limited thereto.

[0475] Additionally, the server may be implemented as at least one of a workstation, a data center, an internet data center (IDC), a direct attached storage (DAS) system, a storage area network (SAN) system, a network attached storage (NAS) system, a redundant array of inexpensive disks, or a redundant array of independent disks (RAID) system, and an Electronic Document Management (EDMS) system, but the present embodiment is not limited thereto.

[0476] In some embodiments of the present invention, when a program is executed by the at least one processor, the method for selecting autonomous driving learning data is implemented by a selection device, which is a computer device mounted on a vehicle, and may perform the operation of receiving first driving information, which is driving information of a vehicle generated based on a first time point data among sensing information detected by a sensor unit of the vehicle and a second sensing information, which is sensing information at a second time point after the vehicle is controlled by the first driving information, based on a predetermined deviation condition, and evaluating whether the operation of the vehicle based on the first driving information is normal, and when the predetermined deviation condition is satisfied, the operation of transmitting the selection information, which is sensing information satisfying a predetermined selection condition, to a learning device placed outside the vehicle.

[0477] The autonomous driving learning data selection method according to the embodiment can also be implemented in the form of a computer-readable medium that stores commands and data executable by a computer.

[0478] In this case, instructions and data may be stored in the form of program code, and when executed by a processor, they may generate a specific program module to perform a specific operation. Additionally, a computer-readable medium may be any available medium accessible by a computer, and includes both volatile and non-volatile media, as well as removable and inremovable media. Furthermore, a computer-readable medium may be a computer recording medium, which may include both volatile and non-volatile, removable and inremovable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. For example, a computer recording medium may be a magnetic storage medium such as HDDs and SSDs, an optical recording medium such as CDs, DVDs, and Blu-ray discs, or memory contained in a server accessible via a network.

[0479] In addition, the autonomous driving learning data selection method according to the embodiment may be implemented as a computer program (or computer program product) including instructions executable by a computer. The computer program includes programmable machine instructions processed by a processor and may be implemented in a high-level programming language, an object-oriented programming language, assembly language, or machine language. Additionally, the computer program may be recorded on a tangible computer-readable recording medium (e.g., memory, hard disk, magnetic / optical medium, or SSD (Solid-State Drive), etc.).

[0480] Accordingly, the autonomous driving learning data selection method according to the embodiment can be implemented by executing a computer program as described above by a computing device. The computing device may include at least some of a processor, memory, storage, a high-speed interface connected to the memory and a high-speed expansion port, and a low-speed interface connected to the low-speed bus and storage. Each of these components is connected to one another using various buses and can be mounted on a common motherboard or installed in other appropriate ways.

[0481] Here, the processor can process instructions within the computing device, such as instructions stored in memory or storage devices to display graphic information for providing a Graphic User Interface (GUI) on external input and output devices, such as a display connected to a high-speed interface. In another embodiment, a plurality of processors and / or a plurality of buses may be utilized together with a plurality of memories and memory types. Additionally, the processor may be implemented as a chipset comprising chips including a plurality of independent analog and / or digital processors.

[0482] In addition, memory stores information within a computing device. For example, memory may consist of volatile memory units or a set thereof. As another example, memory may consist of non-volatile memory units or a set thereof. Furthermore, memory may be other forms of computer-readable media, such as magnetic or optical discs.

[0483] And storage can provide a large amount of storage space to computing devices. A storage device may be a computer-readable medium or a configuration containing such a medium, and may include, for example, devices or other configurations within a Storage Area Network (SAN), and may be a floppy disk device, a hard disk device, an optical disk device, or a tape device, flash memory, or other similar semiconductor memory device or device array.

[0484] Additionally, a screening device according to some embodiments of the present invention comprises: a processor; a memory for loading a computer program executed by the processor; and a storage for storing the computer program, wherein the computer program may perform the operation of receiving first sensing information, which is first time point data among sensing information detected by a sensor unit of a vehicle, and first driving information, which is driving information of a vehicle generated based on an autonomous driving model mounted on the vehicle; the operation of evaluating whether the operation of the vehicle based on the first driving information is normal based on a predetermined deviation condition by utilizing second sensing information, which is the sensing information at a second time point after the vehicle is controlled by the first driving information; and the operation of transmitting screening information, which is the sensing information satisfying a predetermined screening condition, to a learning device disposed outside the vehicle when the predetermined deviation condition is satisfied.

[0485] In addition, a computer program according to some embodiments of the present invention may be combined with a computer device which is hardware and stored in a computer-readable storage medium that performs the document processing method described above.

[0487] A control device according to one embodiment of the present invention is a computing device, similar to the sorting device described above, and may include an input / output device (101, I / O), a processor (102, processor), a memory (103, memory), a storage (104, storage), an interface (105, interface), and a bus (106, bus), and may further include a display unit (not shown) for displaying a user interface.

[0488] Further details may be omitted to the extent that they overlap with the above-mentioned content.

[0490] A learning device according to one embodiment of the present invention is a computing device, similar to the selection device described above, and may include an input / output device (101, I / O), a processor (102, processor), a memory (103, memory), a storage (104, storage), an interface (105, interface), and a bus (106, bus), and may further include a display unit (not shown) for displaying a user interface.

[0491] Further details may be omitted to the extent that they overlap with the above-mentioned content.

[0493] In addition, a learning device according to some embodiments of the present invention comprises: a processor; a memory for loading a computer program executed by the processor; and a storage for storing the computer program, wherein the computer program may execute the operation of receiving selection information, which is sensing information detected by the vehicle that satisfies a predetermined selection condition; the operation of determining whether the selection information satisfies a predetermined learning condition; and the operation of learning an autonomous improvement model based on the selection information when the predetermined learning condition is satisfied.

[0494] The autonomous driving learning data learning method and / or autonomous driving learning data management method according to the embodiment may also be implemented in the form of a computer-readable medium that stores commands and data executable by a computer.

[0495] Further details may be omitted to the extent that they overlap with the above-mentioned content.

[0497] In order to more clearly express the technical concept of the present invention, the attached drawings briefly depict or omit configurations that are unrelated to or have little relevance to the technical concept of the present invention.

[0498] Although the structure and features of the present invention have been described above based on embodiments according to the present invention, the present invention is not limited thereto, and it is obvious to those skilled in the art that various changes or modifications can be made within the spirit and scope of the present invention; therefore, it is noted that such changes or modifications fall within the scope of the appended claims. Explanation of the symbols

[0500] 110: Control device 120: Sorting device 130 : Learning device

Claims

Claim 1 A method for selecting autonomous driving learning data, implemented by a selection device which is a computer device mounted on a vehicle, comprising: a step of receiving first sensing information, which is first time point data among sensing information detected by a sensor unit of the vehicle, and first driving information, which is driving information of the vehicle generated based on an autonomous driving model mounted on the vehicle; a step of evaluating whether the operation of the vehicle based on the first driving information is normal based on a predetermined deviation condition by utilizing second sensing information, which is the sensing information of a second time point, after the vehicle is controlled by the first driving information; and a step of transmitting the selection information, which is the sensing information satisfying a predetermined selection condition, to a learning device disposed outside the vehicle when the predetermined deviation condition is satisfied.The method includes, wherein the predetermined deviation condition comprises at least one of the following conditions: a first deviation condition, which is a condition for comparing the predicted trajectory predicted by the autonomous driving model with the actual trajectory calculated based on the second sensing information; a second deviation condition, which is a condition for determining whether the actual trajectory calculated based on the second sensing information complies with safety rules; a third deviation condition, which is a condition for comparing the reference trajectory generated by the reference model with the actual trajectory calculated based on the second sensing information; and a fourth deviation condition, which is a condition for comparing the normal distribution, which is past driving information, with the actual distribution calculated based on the second sensing information. The step of evaluating whether the operation of the vehicle based on the first driving information is normal comprises determining that the first deviation condition is satisfied if the first deviation index, which is a value quantifying the difference between the predicted trajectory and the actual trajectory, is greater than or equal to a predetermined threshold value, and determining that the second deviation condition is satisfied if the second deviation index, which is a value quantifying whether the actual driving state of the vehicle complies with safety rules, is greater than or equal to a predetermined threshold value. A method for selecting autonomous driving learning data, comprising the step of determining that the third deviation condition is satisfied if the third deviation index, which is a value reflecting the difference between the reference trajectory and the actual trajectory, is greater than or equal to a predetermined threshold, and determining that the fourth deviation condition is satisfied if the fourth deviation index, which is calculated by determining the difference between the normal distribution and the actual distribution, is greater than or equal to a predetermined threshold. Claim 2 delete Claim 3 delete Claim 4 In claim 1, the predetermined threshold value is a value adjusted by a predetermined adjustment method that considers the activity of objects capable of operating within a target area, which is a zone including the location where the vehicle is located; the activity of said objects comprises the activity of people capable of operating within the target area and the activity of things capable of operating within the target area; said people’s activity is calculated by summing a first human activity score set according to the average age of pedestrians belonging to said target area based on demographic information provided from an external server regarding said target area, a second human activity score set according to the gender of said pedestrians, and a third human activity score set based on the amount of movement of pedestrians within said target area that varies according to the time change in which said vehicle is located in said target area; and said things’ activity is calculated by summing a first thing activity score set according to the type of business of things existing within said target area provided from an external server regarding said target area, a second thing activity score set according to the number of things operating at the time of the vehicle’s driving, and according to the arrangement form of things occupying said target area A method for selecting autonomous driving learning data calculated by summing the third object activity scores of the objects being set. Claim 5 In paragraph 4, the above-determined adjustment method is a method of lowering the above-determined threshold value as the variability of the expected activity range of the object increases, an autonomous driving learning data selection method. Claim 6 In paragraph 4, the above-determined adjustment method is a method of variably setting the adjustment amount of the above-determined threshold value by considering the functionality, which is an indicator representing the state of the vehicle, when a predetermined exception condition is satisfied, and the above-determined functionality is a method for selecting autonomous driving learning data, wherein the above-determined functionality is calculated by combining a first functionality score set according to the cumulative mileage of the vehicle, a second functionality score set according to the usage period of the vehicle, and a third functionality score set according to the accident or maintenance history of the vehicle. Claim 7 In claim 6, the above-mentioned predetermined exception condition is a condition satisfied when the above-mentioned predetermined threshold is adjusted to be less than a predetermined threshold, a method for selecting autonomous driving learning data. Claim 8 A method for selecting autonomous driving learning data according to claim 1, wherein the selection information includes information extracted from a predetermined selection interval, and the predetermined selection interval comprises at least one of a first selection interval which is a time interval from a first reference time point, which is a time point before a first time point before an error occurs, to the time point after an error occurs, and a second selection interval which is a time interval from an error termination time point, which is a time point after an error occurs when the predetermined deviation condition is not satisfied, to a second reference time point, which is a time point after a second time point after an error termination time. Claim 9 A method for learning autonomous driving learning data implemented by a learning device, which is a computing device external to a vehicle, comprising: a step of receiving selection information, which is sensing information detected by the vehicle that satisfies a predetermined selection condition, when a predetermined deviation condition is satisfied among all data collected from a selection device while the vehicle is driving autonomously; and a step of determining whether the selection information satisfies a predetermined learning condition. The method comprises the step of learning an autonomous improvement model based on the selection information when the above-determined learning conditions are satisfied; wherein the above-determined deviation condition is a condition in which, after the vehicle is controlled by the first driving information, which is driving information of the vehicle generated based on the autonomous driving model mounted on the vehicle, the deviation index calculated by comparing the actual information calculated based on the second sensing information, which is sensing information at a second time point, with reference information serving as a criterion for judgment is greater than or equal to a predetermined threshold value; and comprises at least one of the following conditions: a first deviation condition, which is a condition for comparing the predicted trajectory predicted by the autonomous driving model with the actual trajectory calculated based on the second sensing information; a second deviation condition, which is a condition for determining whether the actual trajectory calculated based on the second sensing information complies with safety rules; a third deviation condition, which is a condition for comparing the reference trajectory generated by the reference model with the actual trajectory calculated based on the second sensing information; and a fourth deviation condition, which is a condition for comparing the normal distribution, which is past driving information, with the actual distribution calculated based on the second sensing information; and the method of learning the autonomous improvement model. A method for learning autonomous driving learning data, wherein the step involves using the selection information as input data and the reference information corresponding to the input data as correct answer data, calculating the error between the output value of the autonomous improvement model and the correct answer data through a loss function, and repeatedly updating the weights of the autonomous improvement model using a backpropagation algorithm to minimize the loss value. Claim 10 In claim 9, the above-determined learning condition is a condition in which the diversity of error types of the above-determined selection information is greater than or equal to a predetermined standard, an autonomous driving learning data learning method. Claim 11 A method for managing autonomous driving learning data, implemented by an autonomous driving learning device—the autonomous driving learning device comprises a selection device mounted on a vehicle and a learning device disposed outside the vehicle—the method comprising: receiving, by the selection device, first sensing information, which is data at a first time point among sensing information detected by a sensor unit of a vehicle, and first driving information, which is driving information of a vehicle generated based on an autonomous driving model mounted on the vehicle; evaluating, by the selection device, whether the operation of the vehicle based on the first driving information is normal based on a predetermined deviation condition by utilizing second sensing information, which is sensing information at a second time point after the vehicle is controlled by the first driving information; transmitting, by the selection device, the selection information, which is sensing information satisfying a predetermined selection condition, to the learning device; and determining by the learning device whether the selection information satisfies a predetermined learning condition. and, when the predetermined learning conditions are satisfied by the learning device, a step of learning an autonomous improvement model based on the selection information;The method includes, wherein the predetermined deviation condition comprises at least one of the following conditions: a first deviation condition, which is a condition for comparing the predicted trajectory predicted by the autonomous driving model with the actual trajectory calculated based on the second sensing information; a second deviation condition, which is a condition for determining whether the actual trajectory calculated based on the second sensing information complies with safety rules; a third deviation condition, which is a condition for comparing the reference trajectory generated by the reference model with the actual trajectory calculated based on the second sensing information; and a fourth deviation condition, which is a condition for comparing the normal distribution, which is past driving information, with the actual distribution calculated based on the second sensing information. The step of evaluating whether the operation of the vehicle based on the first driving information is normal comprises determining that the first deviation condition is satisfied if the first deviation index, which is a value quantifying the difference between the predicted trajectory and the actual trajectory, is greater than or equal to a predetermined threshold value, and determining that the second deviation condition is satisfied if the second deviation index, which is a value quantifying whether the actual driving state of the vehicle complies with safety rules, is greater than or equal to a predetermined threshold value. A method for managing autonomous driving learning data, comprising: a step of determining that the third deviation condition is satisfied if the third deviation index, which is a value reflecting the difference between the reference trajectory and the actual trajectory, is greater than or equal to a predetermined threshold, and determining that the fourth deviation condition is satisfied if the fourth deviation index, which calculates the difference between the normal distribution and the actual distribution, is greater than or equal to a predetermined threshold; and a step of learning the autonomous improvement model, wherein the selection information is used as input data, and reference information corresponding to the input data and serving as a criterion for judgment is used as correct answer data, the error between the output value of the autonomous improvement model and the correct answer data is calculated through a loss function, and the weights of the autonomous improvement model are repeatedly updated using a backpropagation algorithm to minimize the loss value. Claim 12 A processor; a memory for loading a computer program executed by the processor; and storage for storing the computer program; wherein the computer program performs the operation of receiving first driving information, which is first time point data among the sensing information detected by the sensor unit of the vehicle, and first driving information, which is driving information of the vehicle generated based on an autonomous driving model mounted on the vehicle; the operation of evaluating whether the operation of the vehicle based on the first driving information is normal based on a predetermined deviation condition by utilizing second sensing information, which is the sensing information at a second time point after the vehicle is controlled by the first driving information; and the operation of transmitting selection information, which is the sensing information satisfying a predetermined selection condition, to a learning device disposed outside the vehicle when the predetermined deviation condition is satisfied, wherein the predetermined deviation condition is a first deviation condition, which is a condition for comparing the predicted trajectory predicted by the autonomous driving model with the actual trajectory calculated based on the second sensing information; a second deviation condition, which is a condition for determining whether the actual trajectory calculated based on the second sensing information complies with safety rules; and a reference trajectory generated by a reference model and calculated based on the second sensing information The method comprises at least one of a third deviation condition, which is a condition for comparing the actual trajectory, and a fourth deviation condition, which is a condition for comparing the actual distribution calculated based on the second sensing information with the normal distribution, which is past driving information; the operation of evaluating whether the operation of the vehicle based on the first driving information is normal is determined as if the first deviation condition is satisfied when the first deviation index, which is a value quantifying the difference between the predicted trajectory and the actual trajectory, is greater than or equal to a predetermined threshold value; the second deviation condition is determined as if the second deviation index, which is a value quantifying whether the actual driving state of the vehicle complies with safety rules, is greater than or equal to a predetermined threshold value; and the third deviation condition is determined as if the third deviation index, which is a value reflecting the difference between the reference trajectory and the actual trajectory, is greater than or equal to a predetermined threshold value.A screening device that determines that the fourth deviation condition is satisfied when the fourth deviation index, which calculates the difference between the normal distribution and the actual distribution, is greater than or equal to a predetermined threshold value.

Images