Hierarchical inference system

Abstract

To provide a hierarchical inference system using AI capable of easily performing overall optimization even when complex processing is needed.SOLUTION: The hierarchical inference system comprises: a first generation part generating one or plural pieces of second data from one or plural pieces of first data; and a second generation part generating one or plural pieces of third data from the one or plural pieces of second data. A machine-learned first prediction model is used for generation of the second data by the first generation part while a machine-learned second prediction model is used for generation of the third data by the second generation part. The prediction model or operation of one generation part of the first generation part and the second generation part is changed by input data or generation data of the other generation part.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a hierarchical reasoning system. [Background technology]

[0002] In recent years, safe driving and automated driving assistance systems have been put into practical use, and the development of automated driving is also progressing.Automated driving involves capturing images of the area in front of the vehicle with a camera, recognizing obstacles such as pedestrians and parked vehicles based on the captured images, and controlling the vehicle to avoid these obstacles.

[0003] In order to appropriately avoid obstacles during autonomous driving, future positions are predicted (see, for example, JP 2023-106554 A). Specifically, the moving object and type are distinguished from the obstacle, and the safe distance to be maintained between the moving object and the obstacle is calculated according to the type of moving object. At this time, information about the driving environment in which the vehicle is located, such as weather, time of day, and the number of lanes on each side of the road, is also taken into account.

[0004] The above-mentioned autonomous driving system uses a machine-learned neural network, known as AI, which calculates the optimal driving route based on inputs such as the above-mentioned safe distance and driving environment. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2023-106554 Summary of the Invention [Problem to be solved by the invention]

[0006] AI is being used to make decisions for autonomous driving because it is capable of making complex decisions that previously relied solely on human judgment. However, in the example above, AI is limited to determining the driving route, and traditional deterministic calculation methods are used for processes such as distinguishing between obstacles and the type of moving object, and calculating the safe distance to maintain between the moving object and the obstacle depending on its type. Using AI to make all autonomous driving decisions using obstacles as input presents challenges, such as the complexity of optimization processing and insufficient progress in machine learning.

[0007] The present disclosure has been made in light of the above circumstances, and aims to provide a hierarchical inference system using AI that can easily perform overall optimization even for complex processing. [Means for solving the problem]

[0008] A hierarchical inference system according to one embodiment of the present disclosure comprises a first generation unit that generates one or more second data from one or more first data, and a second generation unit that generates one or more third data from the one or more second data, wherein a first prediction model trained by machine learning is used to generate the second data in the first generation unit, and a second prediction model trained by machine learning is used to generate the third data in the second generation unit, and the prediction model or operation of one of the first and second generation units is changed by the input data or generated data of the other generation unit. [Effects of the Invention]

[0009] The hierarchical inference system disclosed herein uses AI to easily perform overall optimization even for complex processes. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a configuration diagram showing a hierarchical inference system according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is an explanatory diagram of the operation of the second generation unit. DETAILED DESCRIPTION OF THE INVENTION

[0011] [Description of the embodiments of the present disclosure] First, embodiments of the present disclosure will be listed and described.

[0012] (1) A hierarchical inference system according to one embodiment of the present disclosure includes a first generation unit that generates one or more second data from one or more first data, and a second generation unit that generates one or more third data from the one or more second data, wherein a machine-learned first prediction model is used to generate the second data in the first generation unit, and a machine-learned second prediction model is used to generate the third data in the second generation unit, and the prediction model or operation of one of the first and second generation units is changed by the input data or generated data of the other generation unit.

[0013] This hierarchical inference system employs a multi-stage structure in which machine-learned predictive models (so-called AI) are layered, and the second generation unit generates the next data based on the data generated by the first generation unit. In this hierarchical inference system, the second prediction model used in the second generation unit or the operation of the second generation unit is modified based on the first data input to the first generation unit or the second data resulting from the first generation unit. Alternatively, in this hierarchical inference system, the first prediction model used in the first generation unit or the operation of the first generation unit is modified based on the second data input to the second generation unit or the third data resulting from the second generation unit. These modifications efficiently link the first and second prediction models, facilitating overall optimization even for complex processes. In this specification, "machine learning" is a general term for techniques in which a computer improves itself by learning from data. This term includes deep learning, reinforcement learning, and other techniques in addition to machine learning in the narrow sense, which extracts features such as patterns, meanings, and importance from data.

[0014] (2) In the hierarchical inference system of (1) above, it is preferable that a plurality of predictive models generated by machine learning are prepared for one of the generation units, and that the change of the predictive model is a selection from the plurality of predictive models. By selecting a predictive model from the plurality of predictive models when changing the predictive model for one of the generation units in this way, it is possible to further facilitate overall optimization by switching to and using an appropriate predictive model, for example, in cases where the results are affected by environmental conditions.

[0015] (3) In the hierarchical inference system of (1) or (2) above, the change in the operation of the prediction model of one of the generation units may be a change in the function formula representing the prediction model or the coefficient values ​​used in the function formula. By changing the prediction model of one of the generation units in this way by changing the function formula or the coefficient values ​​thereof, it becomes possible to fine-tune the prediction model in accordance with, for example, environmental conditions, and to easily perform overall optimization.

[0016] (4) In any of the hierarchical inference systems (1) to (3), the first prediction model may be modified based on the second data or the results of the second prediction model. By providing a similar modification function for not only the second prediction model but also the first prediction model, overall optimization can be further advanced.

[0017] (5) In the hierarchical inference system according to any one of (1) to (4), the confidence level of the result of the first generation unit may be used to change the second prediction model. By using the confidence level of the result of the first generation unit to change the second prediction model in this way, it is possible to prioritize more likely results and increase the reliability of the third data generated by the second generation unit.

[0018] Here, "certainty" refers to the reliability of extracted or estimated data, and can be expressed as a decimal value ranging from -1 to 1. A confidence level of -1 means that the data is definitely incorrect, a confidence level of 1 means that the data is definitely correct, and a confidence level of 0 means that it is unclear whether the data is correct or incorrect. In this specification, the purpose is to infer correct data, so the confidence level ranges from 0 to 1.

[0019] [Details of the embodiments of the present disclosure] Hereinafter, a hierarchical inference system according to an embodiment of the present disclosure will be described with reference to the drawings as appropriate.

[0020] The hierarchical inference system 1 shown in Fig. 1 includes a first generation unit 10, a second generation unit 20, and a database 30. The first generation unit 10 generates one or more second data 32 from one or more first data 31. The second generation unit 20 generates one or more third data 33 from one or more second data 32. In the hierarchical inference system 1, the first data 31, the second data 32, and the third data 33 are stored in the database 30.

[0021] The hierarchical inference system 1 can be used as a processing system in which processing is performed in stages, such as generating second data 32 based on first data 31 and generating third data 33 based on the generated second data 32, for example, in collecting information necessary for a safe driving support system or automated driving. The following description will be given taking as an example a case in which information necessary for a safe driving support system or automated driving is collected, but this does not mean that the scope of use of the hierarchical inference system 1 is limited to this.

[0022] As a specific example of the use of the hierarchical inference system 1, as shown in Fig. 1, a case where an autonomously driven vehicle 100 identifies an obstacle on the road and determines the need for evasive action will be given, and the operation of the hierarchical inference system 1 will be further described in detail, but this does not mean that the scope of application of the hierarchical inference system 1 is limited to this example. Note that Fig. 1 shows, as an example of road conditions, a case where a traffic cone X1, a bicycle X2, and a pedestrian X3 are present ahead of the vehicle 100.

[0023] <Vehicle> In addition to the hierarchical inference system 1, the vehicle 100 is equipped with a sensor 101.

[0024] The information detected by the sensor 101 can include any natural phenomenon such as weather and temperature, phenomena caused by man-made objects such as traffic congestion, human vital signs such as heart rate, respiratory rate, and blood pressure, information on people approaching a vehicle, information on objects fallen on the road, time, travel speed, and location, and other information that can be detected by a sensor. In this specific example, the sensor 101 includes a camera that captures road conditions. While road images can acquire a large amount of information at once, they require complex processing. The hierarchical inference system 1 can perform such complex processing with high accuracy, making it easy to apply to, for example, autonomous driving.

[0025] The vehicle 100 stores information collected by the sensor 101 in the database 30 as first data 31 .

[0026] <First generation part> In this example, the first generation unit 10 classifies an obstacle recognized from sensor information (first data 31) detected by the sensor 101 into an object such as a traffic cone, a bicycle, or a pedestrian. The classification result is stored in the database 30 as second data 32. Note that the first data 31 is not limited to sensor information, and other data may be input, or the sensor information may be used in combination with other data.

[0027] The first generation unit 10 generates the second data 32 using a first prediction model 11 that has been machine-learned. Specifically, the first prediction model 11 may be a first prediction model 11 that has been machine-learned using known first data 31 and second data 32 corresponding to the first data 31 as training data. The machine learning may be performed using a convolutional neural network (CNN), a deep neural network (DNN), a generative adversarial network (GAN), a GPT, a diffusion model, a support vector machine (SVM), a transformer, or the like. Alternatively, the first prediction model 11 may be a prediction model that performs machine learning without using training data, such as a Kalman filter.

[0028] At this time, the first generation unit 10 generates a confidence level 32a corresponding to the generated second data 32. The confidence level 32a indicates the reliability of the classification result. The confidence level 32a is expressed as a decimal value between 0 and 1, with the closer to 1 the value, the higher the confidence level.

[0029] The significance of the confidence factor 32a is as follows: The cone X1 is classified as a cone by the first generation unit 10. If the cone X1 has the characteristics of a typical cone, the confidence factor 32a will be close to 1. However, if the cone X1 is an inconspicuous color such as gray, it is suggested that it is not a cone, and the confidence factor 32a will be low (e.g., 0.5). Alternatively, if the cone X1 is floating in the air, there is a good chance that the judgment is incorrect, and the confidence factor 32a will be even lower (e.g., 0.1). In this way, even if an obstacle is classified as a cone, its reliability will vary, and the quantity indicating this is the confidence factor 32a.

[0030] <Second generation part> In this example, the second generation unit 20 predicts the future position of the obstacle from the classification of the obstacle (second data 32) extracted by the first generation unit 10. The predicted result is stored in the database 30 as third data 33. Note that the second generation unit 20 may include data other than the second data 32 generated by the first generation unit 10, such as sensor information, in its input.

[0031] For example, the operation of the second generation unit 20 will be described by focusing on bicycle X2. Assume that the positional relationship of cone X1, bicycle X2, and pedestrian X3 based on sensor information (first data 31) detected by sensor 101 is as shown in Figure 2. Furthermore, the speed of bicycle X2 can also be obtained from the sensor information, and the future position of bicycle X2 can be predicted.

[0032] In the example of Figure 2, there is a cone X1 in front of bicycle X2. Since cone X1 does not move in principle, it is easy to predict that bicycle X2 will take action to avoid cone X1. Furthermore, since vehicle 100, which is moving faster than bicycle X2, is approaching from behind (the right side of Figure 2) in the center of the road (the upper side of Figure 2), there is a high possibility that bicycle X2 will avoid it by moving to the edge of the road (the lower side of Figure 2). For example, it is possible to determine the possibility that bicycle X2 will move, as shown in Figure 2.

[0033] The second generation unit 20 uses a second prediction model 21 that has been machine-learned to generate the third data 33. Specifically, the second prediction model 21 may be a second prediction model 21 that has been machine-learned using known second data 32 and third data 33 corresponding to the second data 32 as training data. The machine learning may be the same as that used by the first generation unit 10. The second prediction model 21 may be a prediction model that performs machine learning without using training data, similar to the first prediction model 11. The machine learning used by the second generation unit 20 may be the same type of model as the machine learning used by the first generation unit 10, or a different model.

[0034] In the hierarchical inference system 1, the operation of the second prediction model 21 or the second generation unit 20 is changed depending on the first data 31 or the result of the first generation unit 10. That is, in the hierarchical inference system 1, the prediction model or operation of the second generation unit 20, which is one of the generation units, is changed depending on the input data or generation data of the first generation unit 10, which is the other generation unit.

[0035] A plurality of second prediction models 21a that have been machine-learned are prepared, and the second prediction model 21 can be changed by selecting from the plurality of second prediction models 21a. By changing the second prediction model 21 by selecting from the plurality of second prediction models 21a in this way, it is possible to switch and use an appropriate prediction model in cases where the results are affected by environmental conditions, for example, thereby further facilitating overall optimization.

[0036] In this example, for example, if the first data 31 includes the type of road on which the vehicle 100 is traveling, the second prediction model 21 to be used can be switched between when the road is an expressway and when the road is an ordinary road. Note that the type of road on which the vehicle 100 is traveling can be recognized from, for example, sensor information, and in this example, the type of road is the first data 31.

[0037] The change in the operation of the second prediction model 21 of the second generation unit 20 may be a change in the function formula representing the second prediction model 21 or a change in the coefficient values ​​used in the function formula. By changing the prediction model of the second generation unit 20 in this way by changing the function formula or the coefficient values ​​thereof, it becomes possible to fine-tune the prediction model in accordance with, for example, environmental conditions, and to easily perform overall optimization.

[0038] For example, when a neural network is used for the second prediction model 21, the change in the second prediction model 21 can be a change in the bias value 22 corresponding to the coefficient value of the neural network.

[0039] In this example, the bias value 22 of the neural network can be adjusted depending on environmental conditions such as weather, time, and the area in which the vehicle is traveling. The weather can be recognized from sensor information, for example. The time and the area in which the vehicle is traveling can be obtained using instruments and GPS installed in the vehicle 100, and the necessary environmental information in this example is the first data 31.

[0040] The confidence factor 32a of the result of the first generator 10 can be used to change the second prediction model 21. By using the confidence factor 32a of the result of the first generator 10 to change the second prediction model 21 in this way, it is possible to give priority to a probable result and increase the reliability of the third data 33 generated by the second generator 20.

[0041] The operation of the second generation unit 20 can be changed by the update interval of the third data 33. For example, if the amount of change in the results generated by the first generation unit 10 every moment is small, the update interval of the third data 33 can be lengthened, and if the amount of change is large, the update interval can be shortened. By changing the operation of the second generation unit 20 by the update interval of the third data 33 in this way, the accuracy of the third data 33 can be maintained while the operation frequency of the second generation unit 20 can be reduced, thereby reducing the power consumption of the hierarchical inference system 1.

[0042] The change in the operation of the second generation unit 20 can be achieved by switching between execution and stop of the operation of the second generation unit 20. By changing the operation of the second generation unit 20 by switching between execution and stop of the operation of the second generation unit 20 in this way, it is possible to further reduce the power consumption of the hierarchical inference system 1.

[0043] In this case, for example, if the first generation unit 10 does not recognize an obstacle, the operation of the second generation unit 20 is unnecessary and can be stopped.

[0044] It should be noted that the hierarchical inference system 1 does not necessarily have to be equipped with all of these listed methods for changing the operation of the second prediction model 21 or the second generation unit 20. Moreover, the system may be configured to only change the second prediction model 21 or only change the operation of the second generation unit 20. Furthermore, the operation of the second prediction model 21 or the second generation unit 20 may be changed by other methods not listed here.

[0045] The operation of the second prediction model 21 or the second generation unit 20 may be changed directly by the first generation unit 10 based on the results of the first generation unit 10, or may be changed by a control unit that controls the change in the operation of the second prediction model 21 or the second generation unit 20. In either case, the change can be constructed autonomously within the hierarchical inference system 1. Here, "autonomously" means that no external control is required.

[0046] <Advantages> The hierarchical inference system 1 employs a multi-stage configuration in which machine-learned predictive models, or so-called AI, are hierarchized and the next data is generated in a second generation unit 20 based on the data generated in a first generation unit 10. In the hierarchical inference system 1, the second prediction model 21 used in the second generation unit 20 or the operation of the second generation unit 20 is changed depending on the first data 31 or the results of the first generation unit 10. This change efficiently links the first prediction model 11 and the second prediction model 21, making it easy to optimize the entire process even for complex processes.

[0047] [Other embodiments] The above-described embodiments do not limit the configuration of the present invention. Therefore, the above-described embodiments may include omissions, substitutions, or additions of components based on the description in this specification and common general technical knowledge, and all of these should be construed as falling within the scope of the present invention.

[0048] In the above embodiment, the first data, second data, and third data are stored in a database, but a database is not a required configuration, and the first data, second data, and third data may be generated sequentially and deleted after use. However, since the accumulated data can be reused, it is preferable that the first data, second data, and third data are stored in a database.

[0049] In the above embodiment, the case where certainty factors are used has been described, but the use of certainty factors is not essential, and a hierarchical inference system that does not use certainty factors is also within the scope of the present invention.

[0050] In the above embodiment, the hierarchical inference system has been described as having a two-tier configuration consisting of a first generation unit and a second generation unit. However, the hierarchical inference system of the present disclosure may have three or more tiers. In this case, for example, the first generation unit may change the prediction model of a generation unit that is not directly below or above it, such as by changing the prediction model of a third generation unit. Furthermore, multiple generation units may change the prediction model of another generation unit, such as when both the first and third generation units change the prediction model of the second generation unit. In this case, for example, the first and third generation units may refer to each other's results and change the prediction model of the second generation unit.

[0051] Furthermore, the second data generated by the first generation unit does not need to be directly input to the second generation unit, but may be indirectly input to the second generation unit. That is, the hierarchical inference system includes a case where the second data is processed by another generation unit and input to the second generation unit. The other generation unit may be an AI-type generation unit that uses a machine-learned predictive model, or may be a generation unit that deterministically generates data using signal processing or image processing.

[0052] In the above embodiment, a hierarchical inference system has been described that is configured only to change the second prediction model or the operation of the second generation unit based on the first data or the results of the first generation unit. However, the first prediction model may also be changed based on the second data or the results of the second prediction model. That is, the first prediction model or operation of the first generation unit, which is one of the generation units, may be changed based on the second data, which is input data for the second generation unit, which is the other generation unit, or the third data, which is generated data. In this case, the change of the first prediction model of the first generation unit may be performed by preparing multiple first prediction models that have been machine-learned and selecting one from among the multiple first prediction models. Furthermore, the change of the operation of the first prediction model of the first generation unit may be performed by changing the function formula representing the first prediction model or the coefficient values ​​used in the function formula.

[0053] Furthermore, a configuration may be added in which the prediction model of the other generation unit is changed based on the input data or generated data of one generation unit. That is, the hierarchical inference system may have a configuration in which the operation of the first prediction model or the first generation unit is changed based on the second data or the result of the second generation unit, in addition to a configuration in which the operation of the second prediction model or the first generation unit is changed based on the second data or the result of the second generation unit. By providing a similar change function not only for the second prediction model but also for the first prediction model, overall optimization can be further advanced. [Industrial Applicability]

[0054] As described above, the hierarchical inference system disclosed herein uses AI to easily perform overall optimization even for complex processes. [Explanation of symbols]

[0055] 1 Hierarchical Inference System 10 1st generation part 11 First forecast model 20 Second generation part 21 Second Prediction Model 21a Multiple secondary forecast models 22 Bias Value 30 databases 31 First Data 32 Second Data 32a Confidence 33 Third Data 100 vehicles 101 Sensors X1 traffic cone X2 Bicycle X3 Pedestrian

Claims

1. a first generation unit that generates one or more second data from one or more first data; a second generation unit that generates one or more third data from the one or more second data; Equipped with a first prediction model trained by machine learning is used to generate the second data in the first generation unit; a machine-learned second prediction model is used to generate the third data in the second generation unit; A hierarchical inference system in which the predictive model or operation of one of the first and second generation units is changed by the input data or generation data of the other generation unit.

2. A plurality of machine-learned prediction models are prepared for the one of the generation units, 2. The hierarchical reasoning system according to claim 1, wherein the change in the prediction model is a selection from the plurality of prediction models.

3. 2. The hierarchical inference system according to claim 1, wherein the change in the operation of the prediction model of said one of the generation units is a change in a function formula representing said prediction model or a coefficient value used in said function formula.

4. 2. A hierarchical inference system according to claim 1, wherein the predictive model of the other generator is changed by input data or generated data of the one generator.

5. 4. The hierarchical inference system according to claim 1, wherein the reliability of the data generated by said one of said generating units is used to change the prediction model of said other generating unit.

Images