Information processing device, information processing method and information processing program

Abstract

To enable a basis related to processing by an information processing device to be explained.SOLUTION: According to this disclosure, an information processing device includes: an acquisition unit for acquiring input information to be inputted to a model to be used to control a movable body that can autonomously move; and a generation unit for generating, on the basis of state information showing a state of the model after inputting the input information to the model, basis information showing a basis of controlling the movable body after inputting the input information to the model. The information processing device: outputs the basis information generated by the generation unit and showing the basis of controlling the movable body; and displays, in distinction from other areas, an area gazed by the model according to an object detected by an object detecting sensor unit.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Various systems have been provided in which machines (computers) such as automatic driving assist human actions or in which computers themselves make decisions and operate (see, for example, Patent Documents 1 and 2). In addition, in various technical fields, information processing using machine learning is being utilized, and techniques for learning models such as neural networks have been provided. For example, such learned models are used in systems such as the automatic driving described above. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-154140 A [Patent Document 2] JP 2019-109675 A Summary of the Invention [Problem to be solved by the invention]

[0004] According to the prior art (e.g., Patent Document 1), a technology has been proposed in which an emergency signal is received and a behavior that the driver should take is activated, which is preregistered by the driver. Also, according to the prior art (e.g., Patent Document 2), a technology has been proposed in which data on driving behavior is aggregated and associated with a road network, and then managed and stored in order to support automated driving. Thus, the prior art provides a technology in which a computer performs an emergency operation, but does not take into consideration how the computer makes a decision.

[0005] On the other hand, it is desired that a system in which a computer (information processing device) such as an automatic driving system makes a decision can be understood by a human being as to the basis for the decision. In particular, a model having a neural network structure has a problem that it is difficult for a human being to understand the basis for the decision due to the complexity of the structure. This problem is not limited to automatic driving, but is a common problem in fields that use models having a neural network structure. In addition, showing the basis for the processing performed by such a computer is not limited to the case of a neural network structure, but is a common problem in general to processing performed by a computer. Therefore, it is desired to be able to explain the basis for the processing by an information processing device.

[0006] Therefore, the present disclosure proposes an information processing device, an information processing method, and an information processing program that can make it possible to explain the basis for processing by the information processing device. [Means for solving the problem]

[0007] In order to solve the above problems, an information processing device of one embodiment of the present disclosure includes an acquisition unit that acquires input information to be input into a model used to control a moving body that can move autonomously, and a generation unit that generates basis information indicating a basis for the output of the model after the input information is input into the model, based on state information indicating the state of the model after the input information is input into the model, and outputs the basis information indicating the basis for the control of the moving body generated by the generation unit, and displays the area that the model is gazing at separately from other areas depending on an object detected by a sensor unit that detects the object. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of information processing according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a diagram illustrating an example of a flow of information processing according to the embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of the configuration of a mobile device according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a model information storage unit according to an embodiment of the present disclosure. [Diagram 5] FIG. 2 is a diagram illustrating an example of a log information storage unit according to an embodiment of the present disclosure. [Figure 6] FIG. 1 illustrates an example of a model according to an embodiment of the present disclosure. [Figure 7] 4 is a flowchart illustrating a procedure of information processing according to the embodiment. [Figure 8] 10 is a flowchart showing a procedure for a control process of a moving object. [Figure 9] FIG. 11 is a diagram illustrating another example of information processing according to the embodiment. [Figure 10] FIG. 1 is a diagram illustrating an example of a conceptual diagram of a configuration of an in-vehicle system. [Figure 11] FIG. 13 is a diagram illustrating a configuration example of an information processing system according to a modified example of the present disclosure. [Figure 12] FIG. 13 is a diagram illustrating a configuration example of an information processing device according to a modified example of the present disclosure. [Figure 13] 1 is a block diagram showing an example of a schematic functional configuration of a mobile object control system to which the present technology can be applied; [Figure 14] FIG. 2 is a hardware configuration diagram showing an example of a computer that realizes the functions of a mobile device or an information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the information processing device, information processing method, and information processing program according to the present application are not limited to these embodiments. In addition, in the following embodiments, the same components are denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0010] The present disclosure will be described in the following order. 1. Embodiment 1-1. Overview of information processing according to an embodiment of the present disclosure 1-1-1. Issues and benefits of autonomous driving 1-1-2. Other visualization examples 1-1-2-1. Visualization analysis method using composite algorithms 1-1-3.Other application examples 1-1-4.AI ethics 1-2. Configuration of the Mobile Device According to the Embodiment 1-2-1. Model example 1-3. Information Processing Procedure According to the Embodiment 1-4.Other information processing examples 1-5. Conceptual diagram of in-vehicle system configuration 2. Other embodiments 2-1.Other configuration examples 2-2. Configuration of moving objects 2-3.Other 3. Effects of the present disclosure 4. Hardware Configuration

[0011] [1. Embodiment] [1-1. Overview of information processing according to an embodiment of the present disclosure] 1 is a diagram illustrating an example of information processing according to an embodiment of the present disclosure. The information processing according to the embodiment of the present disclosure is realized by a mobile device 100 illustrated in FIG.

[0012] The mobile body device 100 is an information processing device that executes information processing according to the embodiment. The mobile body device 100 is a mobile body that travels by automatic driving. For example, the mobile body device 100 is a mobile body that travels automatically by appropriately using various conventional technologies related to automatic driving. The mobile body device 100 may be a vehicle that is automated at any level of levels 0 to 5 defined by the Society of Automotive Engineers (SAE). In the example of FIG. 1, the mobile body device 100 is a vehicle that is automated at level 3. That is, in the example of FIG. 1, the mobile body device 100 is a vehicle that autonomously controls the travel of the mobile body device 100 itself and can be operated by a user riding in the mobile body device 100 as necessary. Note that the mobile body device 100 may be a highly automated vehicle of level 4 or higher that does not require a driver to ride, or may be level 2 or lower.

[0013] In the example of FIG. 1, a mobile device 100, which is a vehicle that runs by automatic driving, so-called an automobile, is shown as an example of a mobile body, but the mobile body may be any object that runs autonomously. For example, the mobile body is not limited to a four-wheeled one, but may be a mobile body having a type of wheel other than four, such as two or three wheels, a mobile body without wheels, a drone, a bot, or other types of mobile bodies. In the example of FIG. 1, a mobile body that a person rides on is shown as an example, but the mobile body may be a type that moves autonomously without a person on board, which will be described in detail later.

[0014] Hereinafter, the details of the process shown in FIG. 1 will be described with reference to FIG. 2. FIG. 2 is a diagram showing an example of the flow of information processing according to the embodiment. In the example shown in FIG. 1, a user (hereinafter also referred to as "user U") is riding on the mobile device 100, and the mobile device 100 is traveling on a road RD11. Specifically, in the example shown in FIG. 1, the direction from the mobile device 100 toward objects OB11 and OB12 described below is the front of the mobile device 100, and the mobile device 100 is traveling forward. Note that the side of the road RD11 is an area such as a wall surface into which the mobile device 100 cannot enter. First, the mobile device 100 performs detection using the sensor unit 14 (see FIG. 3) (step S11). The mobile device 100 detects image information (also simply referred to as "image") using the image sensor 141 (see FIG. 3). In the example shown in FIG. 1, the mobile device 100 detects (takes an image) IM1 using the image sensor 141.

[0015] Then, the mobile device 100 performs a recognition process (step S12). The mobile device 100 performs the recognition process based on the image IM1 captured by the image sensor 141. The mobile device 100 performs a process to recognize objects and the like included in the image IM1. The mobile device 100 performs the recognition process using a model M1 for image recognition as shown in FIG. 2. For example, the mobile device 100 inputs the image IM1 to the model M1 as input information IND1, thereby causing the model M1 to output a recognition result. The model M1 outputs information corresponding to the input of the model M2 as the recognition result. For example, the model M1 outputs information indicating the type (class) of an object included in the image and information indicating its position (area) in response to the input of an image. Note that the above is an example, and the model M1 may output any information as long as it corresponds to the input of the model M2, and may output information indicating the status of the captured image, such as "the traffic light is red."

[0016] Here, the model M1 is a multi-layered neural network, and has a structure such as a deep neural network with four or more layers, so-called deep neural network (deep learning), as shown in FIG. 6. The model M1 also includes a CNN (Convolutional Neural Network), and is a model that outputs information indicating objects included in an image and their positions in response to an input of the image. The model M1 is a model having a neural network structure. For example, the model M1 is a model having a CNN structure with a so-called convolutional layer.

[0017] In the example of FIG. 1, the mobile device 100 inputs an image IM1 to a model M1, thereby causing the model M1 to output information indicating that an object OB11 of the type "person" is present in a position (area) on the left side of the image IM1. In addition, the mobile device 100 inputs an image IM1 to the model M1, thereby causing the model M1 to output information indicating that an object OB12 of the type "vehicle" is present in a position (area) on the right side of the image IM1. Then, the mobile device 100 performs the processes of steps S13 to S16. Note that, in the example of FIG. 1, for convenience of explanation, steps S13 to S16 are attached to each process, but this does not mean that step S15 is performed after step S14. For example, the processes of steps S13 to S14 and the processes of steps S15 to S16 are performed in parallel.

[0018] First, the processing of steps S13 to S14 will be described. The mobile device 100 performs a generation process (step S13). The mobile device 100 generates basis information indicating a basis for the output of the model after input information is input to the model, based on state information indicating the state of the model M1 after input information is input to the model M1. The mobile device 100 generates basis information indicating a basis for the output of the model M1 after image IM1 is input to the model M1, based on state information indicating the state of the model M1 after image IM1 is input to the model M1.

[0019] In the example of FIG. 1, the mobile device 100 generates basis information indicating the basis for the output of the model M1 after the image IM1 is input by Gradient-weighted Class Activation Mapping (Grad-CAM). For example, the mobile device 100 generates basis information indicating the basis for the output of the model M1 after the image IM1 is input by processing related to Grad-CAM as disclosed in the following document. The mobile device 100 generates basis information indicating the basis for the output of the model M1 after the image IM1 is input by using the Grad-CAM technology, which is a visualization method applicable to networks including CNN in general. For example, the mobile device 100 can visualize the parts that affect each class by calculating the weight of each channel from the final layer of the CNN and multiplying the weights. In this way, the mobile device 100 can visualize which part of the image was focused on when making a decision in a neural network including a CNN.

[0020] ·Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization<https: / / arxiv.org / abs / 1610.02391>

[0021] Although a description of the Grad-CAM technology will be omitted, the mobile device 100 generates basis information using the Grad-CAM technique (see the above-mentioned document). For example, the mobile device 100 specifies a target type (class) and generates information (image) corresponding to the specified class. For example, the mobile device 100 uses the Grad-CAM technology to generate information (image) for the specified class through various processes such as backpropagation.

[0022] In the example of FIG. 1, the mobile device 100 specifies the class of the type "person" and generates an image related to the basis information in the basis information generating unit RSD1 (see FIG. 2) corresponding to the type "person". The basis information generating unit RSD1 generates an image showing, for example, a range (area) that is being watched for recognizing (classifying) the type "person" in the form of a so-called heat map (color map). Since FIG. 1 is expressed in black and white, the darker the hatching, the more the area is being watched for recognizing (classifying) the type "person". Thus, in the example of FIG. 1, the basis information generating unit RSD1 generates, as the basis information RINF1, an image showing that the position of the object OB11, which is a person, in the image IM1 is being most watched and that the object OB11, which is a person, is being recognized. In the example of FIG. 1, it is also assumed that the mobile device 100 properly recognizes the object OB12, which is a vehicle.

[0023] Then, the mobile device 100 performs a display process (step S14). The mobile device 100 displays the basis information RINF1 generated in the basis information generation unit RSD1 on the display unit 11 (see FIG. 3). As shown in FIG. 2, the mobile device 100 realizes visualization by a display DP related to the basis information generated in the basis information generation unit RSD1.

[0024] As described above, the mobile device 100 can explain the basis for the output of a model having a neural network structure. In this way, the mobile device 100 can explain the basis for the processing by the mobile device 100, which is an information processing device. This allows the user U riding in the mobile device 100 to grasp the basis information for the models M1 to M3 in real time. For example, when there is a discrepancy between the actual state and the basis information of the basis information generating unit RSD1, the user U riding in the mobile device 100 can stop the automatic driving and operate the mobile device 100 by himself. In the example of FIG. 1, the object OB11, which is a person, is appropriately recognized as shown in the basis information generating unit RSD1, so the user U may maintain the automatic driving state of the mobile device 100.

[0025] Furthermore, the mobile device 100 associates the basis information generated in the basis information generation unit RSD1 with the image IM1, which is the input information IND1 on which the basis information is based, and stores the information as history in the storage unit 12 (see FIG. 3). For example, the mobile device 100 associates the input information IND1 with basis information such as the basis information RINF1 and stores the association in the log information storage unit 122 (see FIG. 3). This makes it possible to verify what input the mobile device 100 used to make a decision before performing a subsequent action.

[0026] From here, the processing of steps S15 to S16 will be explained. Note that the processing of step S15 is performed immediately after the processing of step S12 is completed.

[0027] The mobile device 100 performs a prediction process based on the recognition result of the recognition process (step S15). The mobile device 100 performs a prediction process based on the output of the model M1. The mobile device 100 performs a process of predicting the behavior (movement mode) of an object included in the image IM1, such as movement. The mobile device 100 performs a prediction process using a prediction model M2 as shown in FIG. 2. For example, the mobile device 100 inputs information (recognition result information) output by the model M1 to the model M2, thereby causing the model M2 to output a prediction result. For example, the model M2 outputs information indicating the moving direction and speed of the object included in the recognition result information in response to the input of the recognition result information. For example, the model M2 is a multi-layered neural network, and has a structure such as a deep neural network with four or more layers, such as that shown in FIG. 6, a so-called deep neural network (deep learning). Note that the model M2 may perform a prediction process using various information, such as sensor information, as an input, in addition to the information output by the model M1.

[0028] The mobile body device 100 predicts the motion of an object OB11, which is a person. The mobile body device 100 predicts the moving direction and speed of the object OB11. In the example of FIG. 1, the mobile body device 100 predicts that the object OB11 is moving toward the mobile body device 100. The mobile body device 100 also predicts the motion of an object OB12, which is a vehicle. The mobile body device 100 predicts the moving direction and speed of the object OB12. In the example of FIG. 1, the mobile body device 100 predicts that the object OB12 is moving toward the mobile body device 100.

[0029] The mobile device 100 performs a process of determining an action plan based on the prediction result of the prediction process (step S16). The mobile device 100 performs a process of generating an action plan based on the output of the model M2. The mobile device 100 determines an action plan based on the predicted motion patterns of the objects OB11 and OB12. The mobile device 100 performs a process of determining an action plan using a model M3 for action plans as shown in FIG. 2. For example, the mobile device 100 inputs information (prediction result information) output by the model M2 to the model M3, thereby causing the model M3 to output an action plan. For example, the model M3 outputs information indicating the action plan of the mobile device 100 in response to the input of the prediction result information. For example, the model M3 is a multi-layered neural network, and has a structure such as a deep neural network with four or more layers, a so-called deep neural network (deep learning), as shown in FIG. 6. The model M3 may perform a process of generating an action plan using various information, such as sensor information, as an input, in addition to the information output by the model M2. In this way, the mobile device 100 is an information processing device that performs actions using models M1 to M3, which are models learned by machine learning (machine learning models). Note that the actions referred to here include various actions such as information processing performed by the information processing device and actions by robots, mobile objects, etc.

[0030] In the example of FIG. 1, the mobile device 100 determines an action plan to avoid the object OB11 because the objects OB11 and OB12 are located in the traveling direction of the mobile device 100 and are approaching the mobile device 100, and it is not possible to avoid both the objects OB11 and OB12. Specifically, the mobile device 100 plans a route PP11 that proceeds to the right side of the traveling direction to avoid colliding with the object OB11 located on the left side of the traveling direction. The mobile device 100 generates action plan information indicating the route PP11. Then, the mobile device 100 controls automatic driving based on the action plan information indicating the route PP11. For example, the mobile device 100 associates the input information IND1 with the route PP11 and the actual traveling information of the mobile device 100 based on the route PP11, and stores the input information IND1 in the log information storage unit 122 (see FIG. 3). This makes it possible to verify what input the mobile device 100 made what plan or what traveling the mobile device 100 made.

[0031] As described above, in the example of Fig. 1, the mobile device 100 detects (recognizes) an object OB11, which is a pedestrian, from an image IM1 detected by the image sensor 141, and displays the grounds information generated in the grounds information generator RSD1, which is a heat map in which the object OB11 is highlighted, on the display unit 11. In this way, in the example of Fig. 1, a pedestrian is detected by a sensor in autonomous driving, and the pedestrian is highlighted in a heat map by the visualization display device (for example, the display unit 11, etc.).

[0032] For example, in the example of FIG. 1, the mobile device 100 approaches the object OB12, which is a vehicle, by taking a course toward the object OB12, which is a vehicle, in order to avoid the object OB11, which is a pedestrian. At this time, the mobile device 100 displays grounds information in real time, which is a heat map in which the object OB11, which is a pedestrian, is highlighted. This allows the passenger of the mobile device 100 to recognize that the mobile device 100 is taking a course toward the vehicle in order to avoid the person. In this way, the passenger of the mobile device 100 knows that the mobile device 100, which is traveling in an automatic driving mode, is trying to avoid the pedestrian. Therefore, the passenger of the mobile device 100 can avoid an accident with an oncoming vehicle by switching to emergency manual driving, stopping the automatic driving, and operating the steering wheel, brakes, etc., by himself / herself.

[0033] In the example of FIG. 1, assume that the mobile device 100 collides with an oncoming object OB12 because the autonomous driving determines that the mobile device 100 will move to the right in order to avoid a pedestrian, which is an object OB11. Even in such a case, the mobile device 100 can provide information indicating the reason (basis) that the collision with the object OB12 is to avoid the pedestrian, which is an object OB11, because the basis information generated by the basis information generating unit RSD1, which is a heat map in which the pedestrian, the object OB11, is highlighted, is stored in the storage unit 12. Furthermore, various information recorded by the sensor unit 14, such as the speed, acceleration, surrounding outside temperature, road surface conditions (wetness due to rain, freezing, etc.), and further, sensor information such as the physical condition of the driver in the vehicle (whether or not the driver is conscious, blood pressure, body temperature, heart rate, posture, etc.) of the mobile device 100 can also be provided. For example, the mobile device 100 may output sensor information such as the speed, acceleration, surrounding outside temperature, road surface conditions, and physical condition of the driver in the vehicle detected by the sensor unit 14 as information indicating the basis of the action of the mobile device 100. That is, the mobile device 100 can comprehensively consider these pieces of information and prove from the log data that the collision with the oncoming vehicle was caused to avoid a pedestrian.

[0034] Here, FIG. 2 will be described in detail. The mobile device 100 includes a basis information generating unit RSD1 having a plurality of basis generation algorithms for generating basis information for an action. For example, the basis information generating unit RSD1 corresponds to the generating unit 136 in FIG. 3, the storage unit 12 in which the plurality of basis generation algorithms are stored, and the like. In this case, specifically, the storage unit 12 stores a plurality of computer programs (also simply referred to as "programs") in which each of the plurality of basis generation algorithms is implemented. In this manner, a program in which an algorithm is implemented may be described as an algorithm. The plurality of basis generation algorithms here include algorithms using techniques such as Grad-CAM, LIME (Local Interpretable Model-agnostic Explanations) to be described later, and TCAV (Testing with Concept Activation Vectors). For example, the plurality of basis generation algorithms include three algorithms: an algorithm based on the Grad-CAM technique, an algorithm based on the LIME technique, and an algorithm based on the TCAV technique. Note that the basis generation algorithm is not limited to Grad-CAM, LIME, and TCAV, and may be an algorithm using various techniques.

[0035] The mobile device 100 generates basis information for an action using one of a plurality of basis generation algorithms. The mobile device 100 selects one of a plurality of basis generation algorithms, and generates basis information for an action using the selected basis generation algorithm. In the case of processing that requires real-time performance, the mobile device 100 selects an algorithm based on the Grad-CAM method from a plurality of basis generation algorithms, and generates basis information for an action using the selected algorithm based on the Grad-CAM method. In addition, in the case of obtaining locally approximated basis, the mobile device 100 selects an algorithm based on the LIME method, and generates basis information for an action using the selected algorithm based on the LIME method. In addition, in the case of adding a directionality that validates a concept, the mobile device 100 selects an algorithm based on the TCAV method, and generates basis information for an action using the selected algorithm based on the TCAV method.

[0036] The mobile device 100 outputs information indicating the basis of an action based on basis information generated based on one or more basis generation algorithms and / or sensor information. The mobile device 100 outputs first basis information generated by an algorithm based on the Grad-CAM technique, second basis information generated by an algorithm based on the LIME technique, and third basis information generated by an algorithm based on the TCAV technique as information indicating the basis of the action. When one basis generation algorithm is selected, the mobile device 100 outputs basis information for the action generated using the selected basis generation algorithm. When real-time processing is required, the mobile device 100 selects an algorithm based on the Grad-CAM technique and outputs basis information for the action generated using the selected algorithm based on the Grad-CAM technique.

[0037] For example, the mobile device 100 has a stop function. The mobile device 100 detects an abnormality and executes an emergency switch from automatic driving to manual driving. The concept of stopping the AI ​​to perform automatic driving is very important, and for example, the AI ​​of the mobile device 100 itself detects an abnormality and requests a human (passenger, etc.) to switch from automatic driving to manual driving. The mobile device 100 can then visualize in detail what the problem is, drawing the human's attention to the areas that require attention and preventing accidents from occurring. Such a mobile device 100 corresponds to a system in which the AI ​​itself presses an emergency stop button and explains what the problem is by visualizing it.

[0038] For example, traffic conditions and rules differ from country to country. Therefore, the mobile device 100 uses a model (neural network) trained using data from the area where the mobile device 100 is used (operated) as a training data set. For example, the conditions in Japan and the United States are different. Therefore, when the mobile device 100 is driven in Japan, the mobile device 100 uses a network trained by collecting data in Japan. This allows the mobile device 100 to realize automatic driving that appropriately responds to the left-hand traffic and narrow road conditions in Japan. For example, if a data set from a different region is used, a network will be created based on undesirable data such as training data for right-hand traffic and data with different traffic rules for right and left turns. On the other hand, depending on the region where the mobile device 100 is used, a model trained using a data set optimal for that region can be used. An appropriate model corresponding to the usage situation of the mobile device 100 can be used. It can be explained that the training data used to train the model used in the mobile device 100 is data corresponding to an appropriate environment.

[0039] As described above, the mobile device 100 can realize safer driving by adding sound, smell, and the like to the basis for judgment, not limited to images. For example, if a person hears a horn from the right side, the person's interest (attention) is directed to the sound. Therefore, the mobile device 100 works in conjunction with a sound sensor that detects sound, and when it detects a sound such as an abnormal sound, for example, the image capture (detection by the image sensor) is also focused in the same direction as the source of the sound, and an image is taken in that direction, thereby improving the accuracy of the specific direction. In addition, judgment is also necessary based on smell. For example, if you notice a strange smell coming from the car, you can discover a malfunction of the car early. For example, in the mobile device 100, it is useful to feed back the smell sensor to the control system of the automatic driving and inform the person of the abnormality. In this way, in the mobile device 100, the anomaly detection system using deep learning can be judged based on sound and smell. That is, the mobile device 100 is made to make a judgment using not only images (sight) but also sensor information corresponding to various senses such as sound (hearing) and smell (olfaction), that is, multimodal information. This allows the mobile device 100 to approach a car driven by a human and drive with peace of mind.

[0040] For example, Grad-Cam, which operates in real time, is one way of visualizing the decision basis of deep learning. Grad-Cam displays the decision basis of CNN in a heat map, but the visualization method is not limited to this. For example, there are various other ways to interpret deep learning, and each method has a different perspective, so the decision basis is different. For example, LIME specifies a certain category and performs forward calculations on a large number of test images. Grad-Cam specifies a certain category and performs calculations in the completely opposite direction to backward calculations. In terms of interpretation in each case, a system in which multiple basis generation algorithms work simultaneously or selectively is effective in order to understand the decision basis more deeply. This results in an explanation algorithm that selects the optimal explanation algorithm (basis generation algorithm) depending on the situation. Here, technology to prevent accidents from occurring is being researched day and night. In terms of identifying the cause when an accident occurs, it is still in the development stage. When an accident occurs, a system that analyzes logs using multiple basis generation algorithms is useful. Therefore, the mobile device 100 can appropriately explain the basis by using a plurality of basis generation algorithms. For example, even if a calculation takes time, sufficient time can be secured when investigating the cause of an accident after the accident occurs.

[0041] [1-1-1. Issues and benefits of autonomous driving] From here, we will explain the issues in autonomous driving as shown in Figure 1. Autonomous driving using artificial intelligence has been realized. However, the basis for the artificial intelligence's judgment is not known, and autonomous driving is currently performed based on the judgment of a black box. In recent years, accidents have occurred during autonomous driving, and AI (Artificial Intelligence) ethics have become an issue. For example, there is a problem that the passenger (driver) cannot know the reason why a vehicle traveling in autonomous driving suddenly evades to the left. On the other hand, if the reason (basis) is known, it will be clear whether it is a misjudgment by the artificial intelligence, and it is expected that accidents can be avoided and prevented in advance. Therefore, in autonomous driving, it is desirable to visualize and provide the reason (basis) for the artificial intelligence's judgment to the passenger. In the above-mentioned SAE, for example, autonomous driving levels 1 and 2 are defined as driving assistance, and level 3 is defined as manual driving in an emergency. If the basis for the artificial intelligence's judgment can be visualized in real time at these levels, useful information can be provided to the driver.

[0042] Until now, artificial intelligence has been described as a black box, despite its advanced capabilities. For example, deep learning has a structure that mimics human neurons, and models are formed by optimizing a large number of parameters. Due to its complexity, it has been said to be impossible to explain. In recent years, research into explainable artificial intelligence has become more active, and technology that visualizes the basis for decisions has been attracting attention. Although various algorithms have been proposed, research remains at the academic level, and deployment to practical systems has been delayed.

[0043] Therefore, the mobile device 100 visualizes the basis of the judgment made by the neural network including deep learning. As a result, the mobile device 100 can provide useful information to the driver of the mobile device 100 by visualizing the basis of the judgment made by the neural network in real time. In the example of FIG. 1, the mobile device 100 can display basis information generated in the basis information generating unit RSD1, which indicates local parts that became the basis of the judgment made by the model M1, which is deep learning, in the form of a heat map when an image captured by the image sensor 141 is input. The mobile device 100 can visualize the judgment made by the convolution neural network (CNN) included in the model M1 that solves the image classification problem, and the driver (human) of the mobile device 100 can know the basis of the judgment. In addition, the visualization described above includes a method capable of real-time operation (including a method that is not real-time), and therefore can be applied to an automatic driving system such as the mobile device 100, and therefore the driver of the mobile device 100 can know the reason (basis) for the movement of the mobile device 100 in real time. In this way, by incorporating a deep learning decision-making basis visualization algorithm into the vehicle driving system, the driver can understand in real time the reasons for the actions of the autonomous vehicle.

[0044] With the mobile device 100 as described above, even if an accident occurs during autonomous driving, it is possible to explain what kind of judgment the autonomous driving was based on by using a visualization technique of deep learning. Furthermore, with the mobile device 100 as described above, even if an accident occurs, it is possible to show the basis for the operation of the autonomous vehicle based on, for example, log information stored in the log information storage unit 122, etc., and, if necessary, sensor information from the sensor unit 14. In this way, with the mobile device 100 as described above, in the event of an accident occurring, it is possible to provide an explanation based on the visualized display of the judgment basis stored in the log and the sensor information.

[0045] In this way, the mobile device 100 visualizes the deep learning decision basis in autonomous driving. This allows the mobile device 100 to assist the driver, avoid accidents, and investigate the cause of accidents. Furthermore, the mobile device 100 uses a heat map to show the location determined by deep learning in real time in the analysis of driving image data acquired by a sensor. This allows the mobile device 100 to visualize the deep learning decision basis in real time, allowing a human to switch to manual driving and avoid accidents. Depending on the result of the basis information obtained from the basis judgment generation unit RSD1, the mobile device 100 may be stopped as a system while ensuring safety, etc.

[0046] [1-1-2. Other visualization examples] In the example of FIG. 1, an image such as a heat map is generated as the basis information. However, the basis information is not limited to an image, and may be information in various formats such as text information or audio information.

[0047] Furthermore, the basis information generating unit RSD1 of the mobile device 100 may generate the basis information by using various techniques as appropriate, not limited to Grad-CAM, as a method for generating the basis information. For example, the mobile device 100 may generate the basis information by using the LIME technique. For example, the mobile device 100 may generate the basis information by a process related to LIME as disclosed in the following document:

[0048] ·"Why Should I Trust You?": Explaining the Predictions of Any Classifier<https: / / arxiv.org / abs / 1602.04938>

[0049] Although a description of the LIME technology will be omitted, the mobile device 100 generates the basis information by the LIME method (see the above-mentioned document). For example, the mobile device 100 generates another model (basis model) that locally approximates the model to show the reason (basis) why the model made such a decision. The mobile device 100 generates a locally approximating basis model for a combination of input information and an output result corresponding to the input information. The mobile device 100 then generates the basis information using the basis model. Furthermore, the mobile device 100 may use a calculation method (generation method) for the basis information, such as "Testing with Concept Activation Vectors" (a test that takes into account the directionality of activating a concept) called TCAV, as disclosed in the following document: ·Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)<https: / / arxiv.org / pdf / 1711.11279.pdf>

[0050] For example, the mobile device 100 generates a plurality of pieces of input information by duplicating or modifying input information (target input information) that is the basis of an image or the like. The mobile device 100 then inputs each of the plurality of pieces of input information into a model (explanation target model) for which grounds information is to be generated, and causes the explanation target model to output a plurality of pieces of output information corresponding to each of the input information. The mobile device 100 then learns the grounds model using combinations (pairs) of each of the plurality of pieces of input information and each of the corresponding plurality of pieces of output information as learning data. In this way, the mobile device 100 generates a grounds model that locally approximates the target input information with another interpretable model (such as a linear model).

[0051] In this way, when the mobile device 100 obtains the output of a model for a certain input, it generates a basis model to show the basis (local explanation) of the output.

[0052] For example, the mobile device 100 generates an interpretable model such as a linear model as a grounds model. The mobile device 100 generates grounds information based on information such as each parameter of the grounds model such as a linear model. For example, the mobile device 100 generates grounds information indicating that, among the features of the grounds model such as a linear model, a feature with a large weight has a large influence.

[0053] As described above, the mobile device 100 generates the basis information based on the basis model trained using the input information and output results of the model. In this manner, the mobile device 100 may generate the basis information based on state information including the output results of the model after the input information is input to the model.

[0054] In the example of FIG. 1, the mobile device 100 may generate basis information for models M2 and M3, not limited to model M1. For example, the mobile device 100 may generate basis information for prediction of model M2 based on a basis model trained using input information and output results of model M2. For example, the mobile device 100 may generate basis information for an action plan of model M3 based on a basis model trained using input information and output results of model M3. The mobile device 100 may generate basis information for model M2 using LIME technology. The mobile device 100 may generate basis information indicating a basis for prediction of model M2, which is a model that receives the output of model M1 as input, using LIME technology. The mobile device 100 may generate basis information for model M3 using LIME technology. The mobile device 100 may generate basis information indicating a basis for an action plan of model M3, which is a model that receives the output of model M2 as input, using LIME technology.

[0055] [1-1-2-1. Visualization analysis method using composite algorithms] The mobile device 100 may realize visualization by combining multiple algorithms. In this way, analysis by combining multiple visualization algorithms is more advantageous. For example, the mobile device 100 may combine a first algorithm that visualizes the decision basis with a second algorithm different from the first algorithm to analyze the decision basis from a composite perspective.

[0056] For example, the mobile device 100 may combine multiple algorithms, such as Grad-Cam, which is a first algorithm, and LIME, which is a second algorithm, to analyze the basis for judgment from a composite perspective according to the situation and characteristics. The mobile device 100 may combine Grad-Cam, which visualizes the basis for judgment based on the feature amount of deep learning, with LIME, which generates a large amount of sample data and visualizes the basis for judgment from a mask image as a local classification problem. Grad-Cam can visualize the point of interest from the feature amount of a convolution layer. In addition, LIME is a visualization technique that is obtained by inference using a large amount of sample data. In this way, the mobile device 100 can improve the analysis accuracy by combining different visualization techniques. For example, the mobile device 100 may provide the user with the first basis information generated by Grad-Cam and the second basis information generated by LIME. The mobile device 100 may display the first basis information generated by Grad-Cam and the second basis information generated by LIME.

[0057] [1-1-3. Other application examples] Note that the autonomous driving shown in Fig. 1 is an example, and the present invention may be applied to various technologies other than autonomous driving. For example, the mobile device 100 is not limited to an automobile, and may be another type of mobile device such as an electric bicycle, a motorcycle, a drone, etc. When the mobile device 100 is a drone, the image sensor 141 widely views (captures) the entire celestial sphere, but when a target object is recognized, the image sensor 141 may crop to that part and perform zoom photography.

[0058] The information processing device may also be applied to various technologies, such as entertainment robots, robots, cooking robots, medical robots, and humanoids. That is, the information processing device that generates the basis information may be a device used in technical fields such as entertainment robots, robots, cooking robots, medical robots, and humanoids. In the case of an entertainment robot, the information processing device that generates the basis information may generate basis information indicating where the entertainment robot looked (recognized) and performed an action using an image sensor. In the case of a medical robot, the information processing device that generates the basis information may generate basis information indicating what the medical robot recognized using an image sensor and performed an action related to surgery. In this case, the information processing device that generates the basis information may generate basis information indicating the basis of the action of the medical robot when a medical accident occurs. This makes it possible to determine whether a medical accident caused by a medical robot is due to a mistake by the medical robot.

[0059] Further, the information processing device that generates the basis information is not limited to a form having a moving mechanism such as the mobile device 100 or a robot, and may be an information processing device 100A that performs only information processing as shown in FIG. 12. The information processing device that generates the basis information may be applied to the field of finance. For example, the information processing device that generates the basis information may be applied to prediction of an index related to finance (financial index). For example, the information processing device that generates the basis information may be applied to stock price prediction, which is an example of a financial index. In this case, the information processing device that generates the basis information may generate basis information indicating the basis of the predicted stock price. For example, the information processing device that generates the basis information may generate basis information indicating the basis of the predicted stock price by using LIME technology for a model that receives meteorological information such as weather and social information such as politics as input and outputs stock price prediction information.

[0060] [1-1-4.AI ethics] The above-mentioned autonomous driving process will be briefly explained from the perspective of AI ethics. Below, we will explain the case where an autonomous vehicle detects a similar object as an example.

[0061] For example, an autonomous vehicle detects four objects (object group A) of the same type (category X) in the direction of travel, and detects two objects (object group B) of category X to the right of the direction of travel. Category X may be any category, including living objects such as dogs and people, or inanimate objects such as utility poles, cars, and houses.

[0062] In addition, it is assumed that the autonomous vehicle is unable to make an emergency stop in time even if the automatic brakes are activated. In this case, the autonomous vehicle will be unable to avoid contact with either object group A or object group B. Therefore, the autonomous vehicle will select (decide) an action that involves contact with either object group A or object group B.

[0063] In such a case, by generating grounds information indicating the grounds for judgment as described above, a person can recognize what grounds information the autonomous vehicle used to select the action. In the above example, the visualization technology can explain whether the autonomous vehicle correctly recognized both object group A and object group B. That is, in the above example, it is possible to explain whether the autonomous vehicle correctly recognized both object group A and object group B before selecting (determining) the action, or whether the autonomous vehicle correctly recognized object group A or object group B before selecting (determining) the action, or whether the autonomous vehicle correctly recognized object group A or object group B.

[0064] More generally, the basis for actions taken by AI (artificial intelligence) in self-driving vehicles, etc. can be shown to people in a format that people can understand, making it possible to properly determine whether the AI ​​correctly recognized the actions and then took those actions, or whether it acted without being able to properly recognize the actions.

[0065] As described above, the mobile device 100 can explain whether it has properly recognized the outside world by using visualization technology. Specifically, the mobile device 100 can explain whether it has correctly recognized all of the objects by using visualization technology. This allows the mobile device 100 to explain whether the selected (decided) behavior is caused by judgment (decision-making), or whether the recognition is incomplete and the cause is not the judgment but the previous recognition (sensing). For example, when the behavior performed by the mobile device 100 is problematic from an ethical point of view, the mobile device 100 can explain whether the behavior is caused by judgment (decision-making), or whether the recognition is incomplete and the cause is not the judgment but the previous recognition (sensing). Note that in the above example, objects of the same type (same category) are used as an example to simplify the explanation, but the above points can be similarly applied to the case of objects of different types.

[0066] [1-2. Configuration of the Mobile Device According to the Embodiment] Next, a configuration of the mobile body device 100, which is an example of an information processing device that executes information processing according to the embodiment, will be described. Fig. 3 is a diagram showing an example of the configuration of the mobile body device 100 according to the embodiment.

[0067] 3, the mobile device 100 has a display unit 11, a storage unit 12, a control unit 13, a sensor unit 14, and a drive unit 15. The mobile device 100 has a configuration that realizes a function of accepting driving operations by a user U or other user while riding in the mobile device, but since this is a configuration that an ordinary vehicle (automobile) has, a description thereof will be omitted as appropriate. For example, although not shown in the figure, the mobile device 100 has a handle unit (operation unit), a brake unit, an accelerator unit, and the like that accept various driving operations by the user.

[0068] The mobile device 100 may have a communication unit when transmitting and receiving information to and from an external device. The communication unit is realized by, for example, a network interface card (NIC) or a communication circuit. The communication unit is connected to a network N (such as the Internet) by wire or wirelessly, and transmits and receives information to and from other devices via the network N.

[0069] The display unit 11 displays various information. The display unit 11 is a display device (display unit) such as a display, and displays various information. For example, the display unit 11 may be disposed inside the mobile device 100. The display unit 11 may be disposed at a position visible to a user in the mobile device 100, for example, at the front side inside the mobile device 100. The display unit 11 may be the windshield of the mobile device 100. In this case, the display unit 11 may display various information using technology related to AR (Augmented Reality) or MR (Mixed Reality). The display unit 11, which is the windshield, displays the grounds information in a transparent manner. For example, the display unit 11 displays the grounds information having transparency by superimposing it on a range in which an image is captured. For example, the display unit 11 displays the grounds information RINF1 by matching a corresponding area in the grounds information RINF1, which is a heat map, to the position of the object OB11. The display unit 11 displays the information recognized by the recognition unit 132. The display unit 11 displays the information predicted by the prediction unit 133. The display unit 11 displays the information generated by the generation unit 136.

[0070] The display unit 11 displays the basis information. The display unit 11 displays the basis information as a diagram. The display unit 11 displays the basis information as image information. The display unit 11 displays the basis information as a heat map. The display unit 11 displays the basis information as characters. The display unit 11 displays the basis information as numerical values.

[0071] 1, the display unit 11 displays basis information RINF1. The display unit 11 realizes visualization of the basis information by displaying the basis information RINF1. The display unit 11 displays information indicating the basis of an action based on basis information generated based on one or more basis generation algorithms and / or sensor information.

[0072] Furthermore, the mobile device 100 may have a functional configuration for outputting information, not limited to the display unit 11. The mobile device 100 may have a function for outputting information as sound. For example, the mobile device 100 may have a sound output unit such as a speaker that outputs sound.

[0073] The storage unit 12 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 12 has a model information storage unit 121 and a log information storage unit 122. The storage unit 12 stores various information, including but not limited to the model information storage unit 121 and the log information storage unit 122. For example, the storage unit 12 stores various information related to roads and maps on which the mobile device 100, which is an automobile, travels. The storage unit 12 has a map information storage unit that stores various information related to maps.

[0074] The map information storage unit stores various types of map-related information. The map information storage unit stores various types of map-related information necessary for autonomous driving.

[0075] The model information storage unit 121 according to the embodiment stores information related to a model. For example, the model information storage unit 121 stores information (model data) indicating the structure of a model (network). FIG. 4 is a diagram illustrating an example of the model information storage unit according to the embodiment of the present disclosure. FIG. 4 illustrates an example of the model information storage unit 121 according to the embodiment. In the example illustrated in FIG. 4, the model information storage unit 121 includes items such as "model ID," "purpose," and "model data."

[0076] "Model ID" indicates identification information for identifying a model. "Use" indicates the use of the corresponding model. "Model Data" indicates the data of the model. Figure 4 shows an example in which conceptual information such as "MDT1" is stored in "Model Data", but in reality, various information that constitutes the model, such as information and functions related to the network included in the model, is included.

[0077] In the example shown in Fig. 4, the model (model M1) identified by the model ID "M1" indicates that its application is "image recognition." Also, the model data of the model M1 indicates that it is model data MDT1. For example, the model data MDT1 of the model M1 includes various information such as the network structure of the model M1, such as a deep neural network, and parameters such as weights.

[0078] Also, the model (model M2) identified by the model ID "M2" indicates that its use is "prediction." Also, the model data of the model M2 indicates that it is model data MDT2. For example, the model data MDT2 of the model M2 includes various information such as the network structure of the model M2, such as a deep neural network, and parameters such as weights.

[0079] Also, the model (model M3) identified by the model ID "M3" indicates that its use is "action planning." Also, the model data of the model M3 indicates that it is model data MDT3. For example, the model data MDT3 of the model M3 includes various information such as the network structure of the model M3, such as a deep neural network, and parameters such as weights.

[0080] The model information storage unit 121 is not limited to the above, and may store various types of information depending on the purpose.

[0081] The log information storage unit 122 according to the embodiment stores information related to logs (history). For example, the log information storage unit 122 stores information indicating history related to recognition, prediction, and action plans in autonomous driving. The log information storage unit 122 stores information that associates input information to a model in autonomous driving with grounds information related to the output of the model. FIG. 5 is a diagram illustrating an example of a log information storage unit according to the embodiment of the present disclosure. In the example shown in FIG. 5, the log information storage unit 122 includes items such as "log ID," "input information," and "grounds information."

[0082] "Log ID" indicates identification information for identifying a log (history). "Input information" indicates the corresponding input information. FIG. 5 shows an example in which conceptual information such as "IND1" is stored in "input information", but in reality, various data such as input images, or file path names indicating the storage location, etc. are stored.

[0083] "Basis information" indicates the corresponding basis information. In Fig. 5, an example is shown in which conceptual information such as "RINF1" is stored in "Basis information", but in reality, various data such as generated images or file path names indicating the storage locations are stored as basis information.

[0084] In the example shown in Fig. 5, a log identified by a log ID "LG1" (log LG1) indicates that the input information is "IND1" and the basis information is "RINF1." Thus, the example in Fig. 5 shows a case in which the input information IND1 and the basis information RINF1 are associated with each other. In other words, the log LG1 indicates that the basis information indicating the basis for the output corresponding to the input information IND1 is the basis information RINF1.

[0085] The log information storage unit 122 may store various information according to the purpose, not limited to the above. The log information storage unit 122 stores various information, not limited to the basis information, in association with the input information. The log information storage unit 122 stores the input information in association with the recognition result, the prediction result, the action plan, the travel information of the mobile body, etc., corresponding to the input information. For example, the log information storage unit 122 stores the input information IND1 in association with the route PP11 and the travel information of the actual mobile body device 100 based on the route PP11.

[0086] Returning to Fig. 3, the description will be continued. The control unit 13 is realized, for example, by a central processing unit (CPU), a micro processing unit (MPU), or the like executing a program (for example, an information processing program according to the present disclosure) stored inside the mobile device 100 using a random access memory (RAM) or the like as a working area. The control unit 13 is a controller, and may be realized, for example, by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0087] 3, the control unit 13 has an acquisition unit 131, a recognition unit 132, a prediction unit 133, a behavior planning unit 134, an execution unit 135, and a generation unit 136, and realizes or executes the functions and actions of information processing described below. Note that the internal configuration of the control unit 13 is not limited to the configuration shown in FIG. 3, and may be other configurations as long as they perform the information processing described below.

[0088] The acquisition unit 131 acquires various information. The acquisition unit 131 acquires various information from an external information processing device. The acquisition unit 131 acquires various information from the storage unit 12. The acquisition unit 131 acquires various information from the model information storage unit 121 and the log information storage unit 122.

[0089] The acquisition unit 131 acquires sensor information detected by the sensor unit 14. The acquisition unit 131 stores the acquired information in the storage unit 12. The acquisition unit 131 acquires image information detected by the image sensor 141. The acquisition unit 131 acquires sensor information detected by the distance measurement sensor.

[0090] The acquisition unit 131 acquires a model having a neural network structure and input information to be input to the model. The acquisition unit 131 acquires a model used to control an autonomously acting device. The acquisition unit 131 acquires a model used to control a moving body that can move autonomously. The acquisition unit 131 acquires a model used to control a moving body that is a vehicle that operates by automatic driving.

[0091] The acquisition unit 131 acquires a model that performs output in response to input of sensor information, and input information that is sensor information detected by a sensor. The acquisition unit 131 acquires a model that outputs a recognition result of image information in response to input of image information, and input information that is image information. The acquisition unit 131 acquires a model including a CNN. The acquisition unit 131 acquires a model that performs output in response to input of output information output by another model, and input information that is output information output by the other model.

[0092] In the example of FIG. 1, the acquisition unit 131 acquires models M1 to M3, etc. from the model information storage unit 121. The acquisition unit 131 acquires a model M1 used in recognition processing. The acquisition unit 131 acquires a model M2 used in prediction processing. The acquisition unit 131 acquires a model M3 used in action plan processing. The acquisition unit 131 acquires an image IM1 detected by the image sensor 141. The acquisition unit 131 acquires the image IM1 as input information IND1 to the model M1.

[0093] The recognition unit 132 performs recognition processing. The recognition unit 132 performs various types of recognition. The recognition unit 132 recognizes an object. The recognition unit 132 recognizes an object using various types of information. The recognition unit 132 generates various types of information related to the object recognition results. The recognition unit 132 recognizes an object based on the information acquired by the acquisition unit 131. The recognition unit 132 recognizes an object using various types of sensor information detected by the sensor unit 14. The recognition unit 132 recognizes an object using image information (sensor information) captured by the image sensor 141. The recognition unit 132 recognizes an object included in the image information. The recognition unit 132 recognizes various types of information based on information stored in the model information storage unit 121 and the log information storage unit 122.

[0094] 1, the recognition unit 132 performs recognition processing based on an image IM1 captured by the image sensor 141. The recognition unit 132 performs processing to recognize objects and the like included in the image IM1. The recognition unit 132 performs recognition processing using a model M1. For example, the recognition unit 132 inputs the image IM1 to the model M1 as input information IND1, thereby causing the model M1 to output a recognition result.

[0095] The prediction unit 133 performs a prediction process. The prediction unit 133 predicts various pieces of information. The prediction unit 133 predicts various pieces of information based on information acquired from an external information processing device. The prediction unit 133 predicts various pieces of information based on information stored in the storage unit 12. The prediction unit 133 predicts various pieces of information based on information stored in the model information storage unit 121 and the log information storage unit 122.

[0096] In the example of Fig. 1, the prediction unit 133 performs prediction processing based on the output of the model M1. The prediction unit 133 performs processing to predict behavior (movement pattern) such as movement of an object included in the image IM1. The prediction unit 133 performs prediction processing using the model M2. For example, the prediction unit 133 inputs information (recognition result information) output by the model M1 to the model M2, thereby causing the model M2 to output a prediction result.

[0097] The prediction unit 133 predicts the motion manner of the object OB11, which is a person. The prediction unit 133 predicts the moving direction and speed of the object OB11. The prediction unit 133 predicts that the object OB11 is moving towards the prediction unit 133. The prediction unit 133 also predicts the motion manner of the object OB12, which is a vehicle. The prediction unit 133 predicts the moving direction and speed of the object OB12. The prediction unit 133 predicts that the object OB12 is moving towards the mobile device 100.

[0098] The behavior planning unit 134 performs various plans. The behavior planning unit 134 determines a behavior plan. The behavior planning unit 134 generates various information related to the behavior plan. The behavior planning unit 134 performs various plans based on the information acquired by the acquisition unit 131. The behavior planning unit 134 performs various plans using information predicted by the prediction unit 133. The behavior planning unit 134 performs the behavior plan using various techniques related to the behavior plan.

[0099] The behavior planning unit 134 determines a behavior plan based on the information predicted by the prediction unit 133. The behavior planning unit 134 determines a behavior plan for moving so as to avoid obstacles included in the obstacle map based on the information predicted by the prediction unit 133.

[0100] In the example of FIG. 1, the behavior planning unit 134 performs processing to generate a behavior plan based on the output of the model M2. The behavior planning unit 134 determines a behavior plan based on the predicted motion patterns of the objects OB11 and OB12. The behavior planning unit 134 performs processing to determine a behavior plan using the model M3. The behavior planning unit 134 inputs information (prediction result information) output by the model M2 to the model M3, thereby causing the model M3 to output a behavior plan.

[0101] Since objects OB11 and OB12 are located in the traveling direction of the mobile device 100 and approach the mobile device 100, and it is impossible to avoid both objects OB11 and OB12, the behavior planning unit 134 determines a behavior plan to avoid object OB11. The behavior planning unit 134 plans a route PP11 that travels to the right side of the traveling direction to avoid collision with object OB11 located on the left side of the traveling direction. The behavior planning unit 134 generates behavior plan information indicating the route PP11.

[0102] The execution unit 135 executes various information. The execution unit 135 executes various processes based on information from an external information processing device. The execution unit 135 executes various processes based on information stored in the storage unit 12. The execution unit 135 executes various information based on information stored in the map information storage unit. The execution unit 135 determines various information based on information acquired by the acquisition unit 131.

[0103] The execution unit 135 executes various processes based on the information predicted by the prediction unit 133. The execution unit 135 executes various processes based on the action plan planned by the action planning unit 134. The execution unit 135 executes processes related to actions based on the information of the action plan generated by the action planning unit 134. The execution unit 135 controls the driving unit 15 to execute an action corresponding to the action plan based on the information of the action plan generated by the action planning unit 134. The execution unit 135 executes a movement process of the mobile device 100 in accordance with the action plan by controlling the driving unit 15 based on the information of the action plan.

[0104] 1, the execution unit 135 controls the autonomous driving based on the action plan information indicating the route PP11. The execution unit 135 controls the movement of the mobile device 100 based on the action plan information indicating the route PP11 so as to avoid an object OB11, which is a person.

[0105] The generation unit 136 performs various generation. The generation unit 136 generates various information based on information stored in the storage unit 12. The generation unit 136 generates various information based on information stored in the model information storage unit 121 and the log information storage unit 122. The generation unit 136 generates various information based on sensor information detected by the sensor unit 14. The generation unit 136 generates various information based on image information detected by the image sensor 142.

[0106] The generation unit 136 generates various information based on the information acquired by the acquisition unit 131. The generation unit 136 generates various information based on the recognition result by the recognition unit 132. The generation unit 136 generates various information based on the prediction result by the prediction unit 133. The generation unit 136 generates various information based on the action plan by the action plan unit 134.

[0107] The generating unit 136 generates basis information indicating a basis for an output of the model after the input information is input to the model, based on state information indicating a state of the model after the input information is input to the model. The generating unit 136 generates basis information indicating a basis for a process using the output of the model.

[0108] The generation unit 136 generates basis information indicating a basis for controlling the device after input information is input to the model. The generation unit 136 generates basis information indicating a basis for controlling the moving object after input information is input to the model. The generation unit 136 generates basis information indicating a basis for the moving direction of the moving object.

[0109] The generating unit 136 generates basis information of the model to which the input information is input in response to detection by the sensor. The generating unit 136 generates image information indicating the basis for the output of the model as the basis information. The generating unit 136 generates a heat map indicating the basis for the output of the model as the basis information.

[0110] The generating unit 136 generates the basis information based on state information including the state of the convolutional layer of the model. The generating unit 136 generates the basis information by processing related to CAM (Class Activation Mapping). The generating unit 136 generates the basis information by Grad-CAM.

[0111] The generation unit 136 generates basis information for a model to which input information has been input in accordance with output by another model. The generation unit 136 generates basis information based on state information including an output result of a model after input of input information to the model. The generation unit 136 generates basis information based on a basis model trained using the input information and the output result. The generation unit 136 generates basis information using a basis model that locally approximates a combination of input information and output result as a target. The generation unit 136 generates basis information by processing related to LIME.

[0112] The generating unit 136 stores log information in which the input information and the basis information are associated with each other in the storage unit. The generating unit 136 generates various information to be displayed on the display unit 11. The generating unit 136 generates various information such as character information and image information such as graphs to be displayed on the display unit 11.

[0113] The generating unit 136 may generate information (images) related to a screen such as the grounds information RINF1, which is the heat map shown in Fig. 1, by appropriately using various conventional techniques related to images. The generating unit 136 generates images such as the graph GR11 shown in Fig. 1, by appropriately using various conventional techniques related to GUIs. For example, the generating unit 136 may generate images such as the heat map HM11 by using CSS, JavaScript (registered trademark), HTML, or any language capable of describing information processing such as the above-mentioned information display and operation reception.

[0114] 1, the generation unit 136 generates basis information indicating a basis for an output of the model after input information is input to the model, based on state information indicating a state of the model M1 after input information is input to the model M1. The generation unit 136 generates basis information indicating a basis for an output of the model M1 after image IM1 is input to the model M1, based on state information indicating a state of the model M1 after image IM1 is input to the model M1.

[0115] The generation unit 136 generates, by Grad-CAM, basis information indicating a basis for the output of the model M1 after the input of the image IM1. The generation unit 136 generates, by the above-mentioned processing related to Grad-CAM, basis information indicating a basis for the output of the model M1 after the input of the image IM1.

[0116] The generating unit 136 specifies a target type (class) and generates information (image) corresponding to the specified class. For example, the generating unit 136 uses Grad-CAM technology to generate information (image) for the specified class through various processes such as back error propagation. The generating unit 136 specifies the class of type "person" and generates an image that is basis information RINF1 corresponding to the type "person".

[0117] The generation unit 136 generates evidence information RINF1, which is an image showing a range (area) in which attention is paid to recognition (classification) of the type "person" in the form of a heat map (color map). The generation unit 136 generates evidence information RINF1 indicating that the position of the object OB11, which is a person, in the image IM1 is most gazed upon and that the object OB11, which is a person, is recognized. The generation unit 136 functions as a evidence information generation unit that generates evidence information for an action based on a plurality of evidence generation algorithms. The generation unit 136 generates evidence information for an action based on a plurality of evidence generation algorithms, such as an algorithm based on the Grad-CAM method, an algorithm based on the LIME method, and an algorithm based on the TCAV method. The generation unit 136 generates evidence information for an action using one of the plurality of evidence generation algorithms. The generation unit 136 selects one of the plurality of evidence generation algorithms and generates evidence information for an action using the selected evidence generation algorithm. In the case of processing that requires real-time performance, the generation unit 136 selects an algorithm based on the Grad-CAM method from among multiple basis generation algorithms, and generates basis information for the action using the selected algorithm based on the Grad-CAM method.

[0118] The sensor unit 14 detects predetermined information. The sensor unit 14 has an image sensor 141. The image sensor 141 functions as an imaging unit that captures an image. The image sensor 141 detects image information.

[0119] The sensor unit 14 may have various sensors, not limited to the image sensor 141. For example, the sensor unit 14 may have various sensors, such as a position sensor, a distance sensor, a sound sensor, an acceleration sensor, a gyro sensor, a temperature sensor, a humidity sensor, an illuminance sensor, a pressure sensor, a proximity sensor, and a sensor for acquiring biological information such as odor, sweat, heart rate, pulse rate, and brain waves.

[0120] For example, the distance measurement sensor detects the distance between the object to be measured and the distance measurement sensor. The distance measurement sensor detects distance information between the object to be measured and the distance measurement sensor. The distance measurement sensor may be an optical sensor. The sensor unit 14 has a LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) as a distance measurement sensor. The distance measurement sensor is not limited to LiDAR, and may be various sensors such as a ToF (Time of Flight) sensor or a stereo camera. The distance measurement sensor may be a distance measurement sensor using a millimeter wave radar. The distance measurement sensor is not limited to LiDAR, and may be various sensors such as a ToF sensor or a stereo camera.

[0121] Also, for example, the position sensor detects the position of the mobile device 100. The position sensor may be various sensors such as a GPS (Global Positioning System) sensor. Also, the sensors that detect the above-mentioned various information in the sensor unit 14 may be a common sensor, or may be realized by different sensors.

[0122] The driving unit 15 has a function of driving the physical configuration of the moving body device 100. The driving unit 15 has a function for moving the position of the moving body device 100. The driving unit 15 has a function for moving the position of the moving body device 100, which is an automobile. The driving unit 15 is, for example, a motor. The driving unit 15 drives the tires of the moving body device 100, which is an automobile. The driving unit 15 may have any configuration as long as the moving body device 100 can realize a desired operation. The driving unit 15 may have any configuration as long as the driving unit 15 can realize the movement of the position of the moving body device 100. For example, the driving unit 15 drives a movement mechanism of the moving body device 100 in response to a driving operation by a user or an instruction from the execution unit 135, thereby moving the moving body device 100 and changing the position of the moving body device 100.

[0123] [1-2-1. Model example] As described above, the mobile device 100 may use various types of models (functions). For example, the mobile device 100 may use any type of model (function), such as a regression model such as a Support Vector Machine (SVM) or a neural network. The mobile device 100 may use various regression models, such as a nonlinear regression model or a linear regression model.

[0124] In this regard, an example of a network structure of a model will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of a model according to an embodiment of the present disclosure. The network NW1 shown in Fig. 6 shows a neural network including multiple (multiple) intermediate layers between an input layer INL and an output layer OUTL. The network NW1 has a structure such as a deep neural network of four or more layers, a so-called deep neural network (deep learning).

[0125] The network NW1 shown in Fig. 6 corresponds to, for example, the network of the model M1, and is a conceptual diagram showing a neural network (model) used for image recognition. For example, when an image is input from the input layer INL side, the network NW1 outputs the recognition result from the output layer OUTL. For example, the mobile device 100 inputs information to the input layer INL in the network NW1, thereby outputting the recognition result corresponding to the input from the output layer OUTL.

[0126] 6 shows the network NW1 as an example of a model (network), but the network NW1 may be in various forms depending on the purpose, etc. In addition, not only the model M1 but also the models M2, M3, etc. have a similar structure such as a deep neural network (deep learning).

[0127] [1-3. Information Processing Procedure According to the Embodiment] Next, the procedure of information processing according to the embodiment will be described with reference to Fig. 7 and Fig. 8. First, the flow of learning processing according to the embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the procedure of information processing according to the embodiment.

[0128] 7, the mobile device 100 acquires a model having a neural network structure (step S101). For example, the mobile device 100 acquires a model M1 from the model information storage unit 121 (see FIG. 3).

[0129] The mobile device 100 acquires input information to be input to the model (step S102). For example, the mobile device 100 acquires an image IM1 as input information IND1 to be input to the model M1.

[0130] Then, the mobile device 100 generates basis information indicating a basis for the output of the model after the input information is input to the model, based on state information indicating the state of the model after the input information is input to the model (step S103). For example, the mobile device 100 generates basis information RINF1 indicating a basis for the output of the model M1 after the input information IND1 is input to the model M1, based on state information indicating the state of the model M1 after the input information IND1 is input to the model M1. Then, the mobile device 100 displays the generated basis information RINF1.

[0131] Next, the procedure of the control process of the moving body will be described with reference to Fig. 8. A detailed flow of the movement control process of the moving body device 100 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the procedure of the control process of the moving body.

[0132] 8, the mobile device 100 acquires an image from a sensor (step S201). For example, the mobile device 100 acquires an image from the image sensor 141.

[0133] Then, the mobile device 100 stores the acquired image as log data (step S202). For example, the mobile device 100 stores the image in the log information storage unit 122 as log data.

[0134] Then, when the mobile device 100 detects a pedestrian (step S203: Yes), it displays a heat map to the pedestrian (step S204). For example, when the mobile device 100 detects a pedestrian, it generates and displays basis information that is a heat map in a manner that draws attention to the position of the pedestrian.

[0135] Then, the mobile device 100 stores the generated basis information as log data (step S202). For example, the mobile device 100 stores the generated basis information in the log information storage unit 122 in association with the image that is the basis for generation.

[0136] Then, the mobile device 100 performs a steering operation to avoid the pedestrian (step S205). For example, the mobile device 100 accepts a steering operation by the user, and performs movement control in accordance with the accepted steering operation.

[0137] Then, the mobile device 100 stores the received information on the steering operation as log data (step S202). For example, the mobile device 100 stores the received information on the steering operation in the log information storage unit 122 in association with the corresponding image and grounds information.

[0138] On the other hand, if the mobile device 100 has not detected a pedestrian (step S203: No), the mobile device 100 ends the process without performing the processes of steps S204 and S205.

[0139] [1-4. Other information processing examples] In the example of FIG. 1, the mobile body device 100 appropriately recognizes an object such as a pedestrian. However, the mobile body device 100 is not limited to being able to appropriately recognize an object, and may not be able to appropriately recognize an object due to various conditions. Even in such a case, it is useful for the mobile body device 100 to generate grounds information. This point will be described with reference to FIG. 9. FIG. 9 is a diagram showing another example of information processing according to an embodiment. Note that the description of the same points as those in FIG. 1 will be omitted as appropriate.

[0140] In the example shown in FIG. 9, a user (user U) is riding on the mobile device 100, and the mobile device 100 is traveling on a road RD21. Specifically, in the example shown in FIG. 9, the direction from the mobile device 100 toward objects OB21 and OB22 described below is the front of the mobile device 100, and the mobile device 100 is traveling forward. Note that the side of the road RD21 is an area such as a wall surface that the mobile device 100 cannot enter. First, the mobile device 100 performs detection using the sensor unit 14 (see FIG. 3) (step S21). In the example shown in FIG. 9, the mobile device 100 detects (captures) an image IM21 using the image sensor 141.

[0141] Then, the mobile body device 100 performs a recognition process (step S22). The mobile body device 100 performs the recognition process based on the image IM21 captured by the image sensor 141. The mobile body device 100 performs a process to recognize objects and the like included in the image IM21. The mobile body device 100 performs the recognition process using the model M1 in the same manner as in FIG. 1.

[0142] In the example of FIG. 9, the mobile device 100 inputs the image IM21 to the model M1, thereby causing the model M1 to output information indicating that an object OB21 of the type "person" is present in a position (area) on the left side of the image IM21. Also, the mobile device 100 inputs the image IM21 to the model M1, thereby causing the model M1 to output information indicating that an object OB22 of the type "vehicle" is present in a position (area) on the right side of the image IM21. Then, the mobile device 100 performs the processes of steps S23 to S26. Note that, in the example of FIG. 9, for convenience of explanation, steps S23 to S26 are attached to each process, but this does not mean that step S25 is performed after step S24. For example, the processes of steps S23 to S24 and the processes of steps S25 to S26 are performed in parallel.

[0143] First, the processing of steps S23 to S24 will be described. The mobile device 100 performs a generation process (step S23). The mobile device 100 generates basis information indicating a basis for the output of the model after input information is input to the model, based on state information indicating the state of the model M1 after input information is input to the model M1. The mobile device 100 generates basis information indicating a basis for the output of the model M1 after image IM21 is input to the model M1, based on state information indicating the state of the model M1 after image IM21 is input to the model M1.

[0144] In the example of FIG. 9, the mobile device 100 generates, by Grad-CAM, basis information indicating basis regarding the output of the model M1 after the image IM21 is input. In the example of FIG. 9, the mobile device 100 specifies the class of the type "person" and generates an image which is basis information RINF21 corresponding to the type "person". The basis information RINF21 is an image which indicates the range (area) which is gazed upon for recognizing (classifying) the type "person" in the form of a so-called heat map (color map), similar to the basis information RINF1 of FIG. 1. In the example of FIG. 9, the basis information RINF21 indicates that the position of the object OB21 which is a person in the image IM21 is not gazed upon, and the object OB21 which is a person is not properly recognized. Also, in the example of FIG. 9, it is assumed that the mobile device 100 properly recognizes the object OB12 which is a vehicle.

[0145] Then, the mobile body device 100 performs a display process (step S24). The mobile body device 100 displays the generated basis information RINF21 on the display unit 11 (see FIG. 3). In the example of FIG. 9, since the object OB21, which is a person, is not properly recognized as shown in the basis information RINF21, the user U may switch the automatic driving state of the mobile body device 100 to manual driving and switch to driving by the user's own operation.

[0146] Furthermore, the mobile device 100 associates the grounds information RINF21 with the image IM21, which is the input information (input information IND21) on which the grounds information RINF21 is based, and stores the association as history in the storage unit 12 (see FIG. 3). For example, the mobile device 100 associates the input information IND21 with the grounds information RINF21 and stores them in the log information storage unit 122 (see FIG. 3). This makes it possible to verify what input the mobile device 100 used to make a decision before carrying out a subsequent action.

[0147] From here, the processes of steps S25 to S26 will be described. Note that the process of step S25 is performed immediately after the process of step S22 is completed.

[0148] The mobile body device 100 performs a prediction process based on the recognition result of the recognition process (step S25). The mobile body device 100 performs a prediction process based on the output of the model M1. The mobile body device 100 performs a process of predicting the behavior (movement mode) such as movement of an object included in the image IM21. The mobile body device 100 performs a prediction process using the model M2, as in FIG. 1. The mobile body device 100 predicts the movement mode of the object OB22, which is a recognized vehicle. The mobile body device 100 predicts the movement direction and speed of the object OB22. In the example of FIG. 9, the mobile body device 100 predicts that the object OB22 is moving toward the mobile body device 100.

[0149] The mobile device 100 performs a process of determining an action plan based on the prediction result of the prediction process (step S26). The mobile device 100 performs a process of generating an action plan based on the output of the model M2. The mobile device 100 determines an action plan based on the predicted motion mode of the object OB22. The mobile device 100 performs a process of determining an action plan using the model M3, as in FIG. 1. In the example of FIG. 9, the mobile device 100 determines an action plan to avoid the object OB22, since the object OB22 is located in the traveling direction of the mobile device 100 and is approaching the mobile device 100. Specifically, the mobile device 100 plans a route PP21 that proceeds to the left side of the traveling direction to avoid collision with the object OB22 located on the right side of the traveling direction. The mobile device 100 generates action plan information indicating the route PP21. Then, the mobile device 100 controls automatic driving based on the action plan information indicating the route PP21. For example, the mobile device 100 associates the input information IND21 with the route PP21 and actual travel information of the mobile device 100 based on the route PP21 and stores them in the log information storage unit 122 (see FIG. 3). This makes it possible to verify what input the mobile device 100 received caused what plan the mobile device 100 made or what travel it performed.

[0150] 9, the mobile device 100 cannot detect (recognize) an object OB21, which is a pedestrian, from the image IM21 detected by the image sensor 141, and therefore displays the grounds information RINF21, which is a heat map in which the object OB21 is not highlighted, on the display unit 11. In this way, in the example of Fig. 9, a pedestrian cannot be detected by the sensor in autonomous driving, and therefore the pedestrian is not highlighted in a heat map by the visualization display device (for example, the display unit 11, etc.).

[0151] For example, in the example of FIG. 9, the mobile device 100 cannot properly recognize the object OB21, which is a pedestrian, and therefore takes a course toward the pedestrian object OB21 and approaches the object OB21 in order to avoid the object OB22, which is a vehicle. At this time, the mobile device 100 displays in real time the grounds information RINF21, which is a heat map in which the pedestrian object OB21 is not highlighted. This allows the passenger of the mobile device 100 to recognize that the mobile device 100 is taking a course toward the person because it cannot recognize the person. In this way, the passenger of the mobile device 100 can see that the mobile device 100, which is traveling in an automatic driving mode, is trying to take a course toward the person in order to avoid an oncoming vehicle because it cannot properly recognize the person. Therefore, the passenger of the mobile device 100 can avoid an accident in which the mobile device 100 comes into contact with a person by switching to emergency manual driving, stopping automatic driving, and operating the handlebars, brakes, etc., by himself / herself.

[0152] In the example of Fig. 9, assume that the mobile device 100 comes into contact with a pedestrian object OB21 because the automatic driving determines that the mobile device 100 should move left to avoid an oncoming vehicle that is an object OB22. Even in such a case, the mobile device 100 can provide information indicating the reason (basis) that the contact with the pedestrian object OB21 was due to the failure to properly recognize the object OB21, because the basis information RINF21, which is a heat map in which the pedestrian object OB21 is not highlighted, is stored in the storage unit 12. In other words, the mobile device 100 can prove from the log data that the contact accident with a person occurred because the automatic driving system (sensor system) was unable to recognize a pedestrian, as an explanation for the contact accident.

[0153] [1-5. Conceptual diagram of in-vehicle system configuration] Here, the functions, hardware configuration, and processing in the in-vehicle system are conceptually shown using Fig. 10. Fig. 10 is a diagram showing an example of a conceptual diagram of the configuration of the in-vehicle system. The in-vehicle system FCB1 shown in Fig. 10 is a system mounted on a vehicle (mobile body) for automatic driving. For example, the mobile body device 100 is an automobile mounted with the in-vehicle system FCB1 as shown in Fig. 10.

[0154] The in-vehicle system FCB1 shown in Fig. 10 includes a sensor unit, artificial intelligence, an automatic driving control unit, etc. In addition, the in-vehicle system FCB1 performs processes such as visualization display, log storage, and manual operation in an emergency.

[0155] The sensor unit of the in-vehicle system FCB1 detects, for example, information outside the vehicle. The sensor unit of the in-vehicle system FCB1 corresponds to the sensor unit 14 of the mobile device 100. The sensor unit of the in-vehicle system FCB1 captures an image.

[0156] The artificial intelligence of the in-vehicle system FCB1 includes a recognition system and a judgment system. The recognition system of the in-vehicle system FCB1 performs external environment recognition and prediction processing. The recognition system of the in-vehicle system FCB1 corresponds to the recognition unit 132 and the prediction unit 133 of the mobile device 100. The recognition system of the in-vehicle system FCB1 recognizes the external environment based on information (sensor information) detected by the sensor unit of the in-vehicle system FCB1. In addition, the recognition system of the in-vehicle system FCB1 performs prediction based on the result of the external environment recognition.

[0157] The determination system of the in-vehicle system FCB1 processes the behavior plan. The determination system of the in-vehicle system FCB1 corresponds to the behavior plan unit 134 of the mobile device 100. The determination system of the in-vehicle system FCB1 performs the behavior plan based on the prediction result of the recognition system of the in-vehicle system FCB1.

[0158] The automatic driving control unit of the in-vehicle system FCB1 controls automatic driving. The automatic driving control unit of the in-vehicle system FCB1 corresponds to the execution unit 135 of the mobile device 100 and each configuration that controls driving. The automatic driving control unit of the in-vehicle system FCB1 controls driving based on the action plan generated by the determination system of the in-vehicle system FCB1.

[0159] The visualized display of the in-vehicle system FCB1 is a process of displaying various types of information. The visualized display of the in-vehicle system FCB1 is realized by functions such as the display unit 11 and the generation unit 136 of the mobile device 100. The visualized display of the in-vehicle system FCB1 displays information from the sensor unit and artificial intelligence. The visualized display of the in-vehicle system FCB1 displays basis information that indicates the basis for the artificial intelligence's judgment. For example, the visualized display of the in-vehicle system FCB1 generates and displays basis information based on artificial intelligence information.

[0160] The log storage of the in-vehicle system FCB1 is a process of storing various types of information as logs. The log storage of the in-vehicle system FCB1 is realized by functions such as the memory unit 12 of the mobile device 100. The log storage of the in-vehicle system FCB1 stores information on the visualized display and judgment system of the in-vehicle system FCB1 as logs. The log storage of the in-vehicle system FCB1 stores sensor information and information on external recognition, prediction, and action plans based on the sensor information in association with each other as logs.

[0161] The emergency manual driving of the in-vehicle system FCB1 is a process of controlling the in-vehicle system FCB1 in response to manual driving by a user (passenger) riding in the vehicle in which the in-vehicle system FCB1 is mounted. The emergency manual driving of the in-vehicle system FCB1 is realized by a configuration that accepts various driving operations by the user of the bundle unit, accelerator unit, brake unit, etc. of the moving body device 100. For example, when the in-vehicle system FCB1 is controlled by emergency manual driving, the control is stopped by the automatic driving control unit, and the traveling of the vehicle in which the in-vehicle system FCB1 is mounted is controlled in response to manual driving by the user.

[0162] The in-vehicle system FCB1 described above visualizes the basis on which deep learning, known as a black box, makes decisions in AI-based autonomous driving. As shown in Figure 10, the artificial intelligence part of the in-vehicle system FCB1 consists of a recognition system and a judgment system. The recognition system recognizes and predicts the outside world, and provides information that serves as the basis for the judgment system. The recognition system can visualize and display the image that serves as the basis for the judgment by highlighting it in a heat map. If the driver of the vehicle sees the visualization and senses a dangerous situation, he or she can avoid an accident by switching from autonomous driving to emergency manual driving. In addition, in the unlikely event that an accident occurs, an explanation can be given based on the visualization of the basis for the judgment stored in the log and the sensor image.

[0163] The in-vehicle system FCB1 described above makes it possible to visualize the decisions made by the artificial intelligence in autonomous driving. The driver can know in real time the basis for the decision made by the artificial intelligence, which was previously called a black box, and can support safe driving. For example, if the autonomous driving tries to turn left suddenly, the visualization technology for the deep learning decision basis can tell whether it is to avoid a person or an obstacle. For example, a sudden turn to the left may indicate that there is a person or other object ahead. If the autonomous driving had not turned the steering wheel to the left and had instead moved toward the obstacle, it would have only caused property damage and could have avoided a personal injury accident. In addition, in the unlikely event of an accident occurring, the basis for the autonomous driving decision is recorded, so it is possible to prove negligence in the autonomous driving.

[0164] [2. Other embodiments] The process according to each of the above-described embodiments may be implemented in various different forms (modified examples) other than the above-described embodiments. For example, in the above-described example, the information processing device that performs information processing is the mobile device 100, but the information processing device may be a server device. For example, the information processing device may be a server device that generates basis information using information received from another device. That is, the information processing device may have only a configuration necessary for performing processing to generate basis information. An information processing system including an information processing device that generates basis information may be configured. In this case, the information processing system may be configured to include an information processing device that generates basis information and a display device that displays the basis information generated by the information processing device. That is, the information processing system may be configured to include a device that generates basis information and a device that displays the basis information.

[0165] [2-1. Other configuration examples] In addition, in the above-mentioned example, the information processing device and the mobile device (mobile body) are integrated, but the information processing device and the mobile device (mobile body) may be separate. This point will be described with reference to Figs. 11 and 12. Fig. 11 is a diagram showing a configuration example of an information processing system according to a modified example of the present disclosure. Fig. 12 is a diagram showing a configuration example of an information processing device according to a modified example of the present disclosure.

[0166] As shown in FIG. 11, the information processing system 1 includes a mobile body device 10 and an information processing device 100A. The mobile body device 10 and the information processing device 100A are connected to each other via a network N so as to be able to communicate with each other by wire or wirelessly. The information processing system 1 shown in FIG. 11 may include a plurality of mobile body devices 10 and a plurality of information processing devices 100A. In this case, the information processing device 100A may communicate with the mobile body device 10 via the network N and may instruct the mobile body device 10 to control the mobile body device 10 based on information collected by the mobile body device 10 and various sensors. The information processing device 100A may be located anywhere. For example, the information processing device 100A may be located outside the mobile body device 10 or may be mounted on the mobile body device 10.

[0167] The mobile body device 10 is an automobile that travels autonomously. The mobile body device 10 transmits sensor information detected by a sensor such as an image sensor to the information processing device 100A. The mobile body device 10 transmits an image captured by the image sensor to the information processing device 100A. As a result, the information processing device 100A acquires the image captured by the image sensor. Note that the mobile body device 10 may be any device that can transmit and receive information to and from the information processing device 100A, and may be, for example, various mobile bodies such as an autonomous mobile robot or a drone.

[0168] The information processing device 100A is an information processing device that performs various information processes using information received from the mobile device 10. The information processing device 100A provides information for controlling the mobile device 10, such as action plan information, to the mobile device 10. The mobile device 10 that receives the action plan information from the information processing device 100A controls and moves based on the action plan information. The information processing device 100A provides the generated basis information to the mobile device 10. The mobile device 10 that receives the basis information from the information processing device 100A displays the basis information.

[0169] 12, the information processing device 100A has a storage unit 12, a control unit 13A, and a communication unit 16. The communication unit 16 is connected to a network N (such as the Internet) by wire or wirelessly, and transmits and receives information to and from the mobile body device 10 via the network N. The storage unit 12 stores information for controlling the movement of the mobile body device 10, various information received from the mobile body device 10, and various information to be transmitted to the mobile body device 10. In this way, the information processing device 100A has a configuration for transmitting and receiving information to and from an external device such as the mobile body device 10.

[0170] 12, the information processing device 100A does not have a sensor unit, a drive unit, etc., and may not have a configuration for realizing a function as a mobile device. Note that the information processing device 100A may have an input unit (e.g., a keyboard, a mouse, etc.) that accepts various operations from an administrator who manages the information processing device 100A, and a display unit (e.g., a liquid crystal display, etc.) that displays various information.

[0171] The control unit 13A includes an acquisition unit 131 , a recognition unit 132 , a prediction unit 133 , an action planning unit 134 , an execution unit 135 , a generation unit 136 , and a transmission unit 137 .

[0172] The transmission unit 137 transmits various information. The transmission unit 137 provides various information. The transmission unit 137 provides various information to an external information processing device. The transmission unit 137 transmits various information to an external information processing device. The transmission unit 137 transmits information stored in the storage unit 12. The transmission unit 137 transmits information generated by the generation unit 136.

[0173] The transmission unit 137 transmits information to the mobile device 10. The transmission unit 137 transmits information on the action plan generated by the action planning unit 134 to the mobile device 10. The transmission unit 137 controls the operation of the mobile device 10 by transmitting information on the action plan generated by the action planning unit 134 to the mobile device 10. The transmission unit 137 controls automatic driving of the mobile device 10 by transmitting information on the action plan to the mobile device 10.

[0174] [2-2. Configuration of moving objects] The above-described mobile device 100 and information processing system 1 may have a configuration as shown in Fig. 13. For example, the mobile device 100 may have the following configuration in addition to the configuration shown in Fig. 3. Note that each unit shown below may be included in the configuration shown in Fig. 3, for example.

[0175] That is, the above-described mobile object device 100 and information processing system 1 can also be configured as a mobile object control system as shown below. Fig. 13 is a block diagram showing an example of a schematic functional configuration of a mobile object control system to which the present technology can be applied.

[0176] The autonomous driving control unit 212 and the operation control unit 235 of the vehicle control system 200, which is an example of a mobile body control system, correspond to the execution unit 135 of the mobile body device 100. The detection unit 231, the self-position estimation unit 232, and the situation analysis unit 233 of the autonomous driving control unit 212 correspond to the recognition unit 132 and the prediction unit 133 of the mobile body device 100. The planner 234 of the autonomous driving control unit 212 corresponds to the action planner 134 of the mobile body device 100. The autonomous driving control unit 212 may have blocks corresponding to each processing unit of the control unit 13, in addition to the blocks shown in FIG. 13 .

[0177] In the following description, when the vehicle in which the vehicle control system 200 is installed is to be distinguished from other vehicles, it will be referred to as the host vehicle or the host vehicle.

[0178] The vehicle control system 200 includes an input unit 201, a data acquisition unit 202, a communication unit 203, an in-vehicle device 204, an output control unit 205, an output unit 206, a drive system control unit 207, a drive system 208, a body system control unit 209, a body system 210, a storage unit 211, and an automatic driving control unit 212. The input unit 201, the data acquisition unit 202, the communication unit 203, the output control unit 205, the drive system control unit 207, the body system control unit 209, the storage unit 211, and the automatic driving control unit 212 are connected to each other via a communication network 221. The communication network 221 is, for example, an in-vehicle communication network or a bus that complies with any standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), or FlexRay (registered trademark). Note that each unit of the vehicle control system 200 may be directly connected without going through the communication network 221.

[0179] In the following description, when each unit of the vehicle control system 200 communicates with each other via the communication network 221, the description of the communication network 221 will be omitted. For example, when the input unit 201 and the autonomous driving control unit 212 communicate with each other via the communication network 221, it will be simply described that the input unit 201 and the autonomous driving control unit 212 communicate with each other.

[0180] The input unit 201 includes a device used by the passenger to input various data, instructions, etc. For example, the input unit 201 includes operation devices such as a touch panel, a button, a microphone, a switch, and a lever, as well as an operation device that allows input by a method other than manual operation such as voice or gesture. In addition, for example, the input unit 201 may be a remote control device that uses infrared rays or other radio waves, or an externally connected device such as a mobile device or a wearable device that supports operation of the vehicle control system 200. The input unit 201 generates an input signal based on data, instructions, etc. input by the passenger, and supplies the input signal to each unit of the vehicle control system 200.

[0181] The data acquisition unit 202 includes various sensors and the like that acquire data used for processing by the vehicle control system 200 , and supplies the acquired data to each unit of the vehicle control system 200 .

[0182] For example, the data acquisition unit 202 includes various sensors for detecting the state of the vehicle, etc. Specifically, for example, the data acquisition unit 202 includes a gyro sensor, an acceleration sensor, an inertial measurement unit (IMU), and sensors for detecting the amount of operation of an accelerator pedal, the amount of operation of a brake pedal, the steering angle of a steering wheel, the engine rotation speed, the motor rotation speed, or the rotation speed of wheels, etc.

[0183] Moreover, for example, the data acquisition unit 202 includes various sensors for detecting information outside the vehicle. Specifically, for example, the data acquisition unit 202 includes imaging devices such as a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. Furthermore, for example, the data acquisition unit 202 includes an environmental sensor for detecting weather or climate, and a surrounding information detection sensor for detecting objects around the vehicle. The environmental sensor includes, for example, a raindrop sensor, a fog sensor, a sunlight sensor, a snow sensor, and the like. The surrounding information detection sensor includes, for example, an ultrasonic sensor, a radar, a LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), a sonar, and the like.

[0184] Furthermore, for example, the data acquisition unit 202 includes various sensors for detecting the current position of the vehicle. Specifically, for example, the data acquisition unit 202 includes a GNSS receiver that receives GNSS signals from Global Navigation Satellite System (GNSS) satellites.

[0185] Furthermore, for example, the data acquisition unit 202 includes various sensors for detecting information inside the vehicle. Specifically, for example, the data acquisition unit 202 includes an imaging device for capturing an image of the driver, a biosensor for detecting bioinformation of the driver, and a microphone for collecting sound inside the vehicle. The biosensor is provided, for example, on the seat or steering wheel, and detects bioinformation of a passenger sitting in the seat or a driver holding the steering wheel.

[0186] The communication unit 203 communicates with the in-vehicle device 204 as well as various devices, servers, base stations, and the like outside the vehicle, transmits data supplied from each part of the vehicle control system 200, and supplies received data to each part of the vehicle control system 200. Note that the communication protocol supported by the communication unit 203 is not particularly limited, and the communication unit 203 can also support multiple types of communication protocols.

[0187] For example, the communication unit 203 performs wireless communication with the in-vehicle device 204 by wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), WUSB (Wireless USB), etc. Also, for example, the communication unit 203 performs wired communication with the in-vehicle device 204 by USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface) (registered trademark), MHL (Mobile High-definition Link), etc. via a connection terminal (and a cable, if necessary) not shown.

[0188] Furthermore, for example, the communication unit 203 communicates with devices (for example, application servers or control servers) present on an external network (for example, the Internet, a cloud network, or a network specific to a business operator) via a base station or an access point. Also, for example, the communication unit 203 communicates with terminals (for example, pedestrian or store terminals, or MTC (Machine Type Communication) terminals) present in the vicinity of the vehicle using a P2P (Peer To Peer) technology. Furthermore, for example, the communication unit 203 performs V2X communication such as vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication. Also, for example, the communication unit 203 includes a beacon receiving unit, and receives radio waves or electromagnetic waves transmitted from a wireless station or the like installed on a road, and acquires information such as a current position, congestion, traffic regulations, or required time.

[0189] The in-vehicle devices 204 include, for example, mobile devices or wearable devices possessed by passengers, information devices carried into or attached to the vehicle, and navigation devices that search for a route to an arbitrary destination.

[0190] The output control unit 205 controls the output of various information to the passengers of the vehicle or to the outside of the vehicle. For example, the output control unit 205 generates an output signal including at least one of visual information (e.g., image data) and audio information (e.g., audio data) and supplies the output signal to the output unit 206, thereby controlling the output of the visual information and audio information from the output unit 206. Specifically, for example, the output control unit 205 synthesizes image data captured by different imaging devices of the data acquisition unit 202 to generate an overhead image or a panoramic image, and supplies an output signal including the generated image to the output unit 206. Also, for example, the output control unit 205 generates audio data including a warning sound or a warning message against danger such as a collision, contact, or entry into a dangerous area, and supplies an output signal including the generated audio data to the output unit 206.

[0191] The output unit 206 includes a device capable of outputting visual or audio information to the passengers of the vehicle or to the outside of the vehicle. For example, the output unit 206 includes a display device, an instrument panel, an audio speaker, headphones, a wearable device such as a glasses-type display worn by the passenger, a projector, a lamp, etc. The display device included in the output unit 206 may be a device having a normal display, or may be a device that displays visual information within the field of view of the driver, such as a head-up display, a see-through display, or a device having an AR (Augmented Reality) display function.

[0192] The drive system control unit 207 generates various control signals and supplies them to the drive system 208, thereby controlling the drive system 208. In addition, the drive system control unit 207 supplies control signals to each unit other than the drive system 208 as necessary, and notifies the control state of the drive system 208, etc.

[0193] The drivetrain system 208 includes various devices related to the drivetrain of the vehicle, such as a drive force generating device for generating drive force from an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle, a braking device for generating braking force, an antilock brake system (ABS), an electronic stability control (ESC), and an electric power steering device.

[0194] The body system control unit 209 generates various control signals and supplies them to the body system 210, thereby controlling the body system 210. Furthermore, the body system control unit 209 supplies control signals to each unit other than the body system 210 as necessary, and notifies the control state of the body system 210, etc.

[0195] The body system 210 includes various body system devices mounted on the vehicle body, such as a keyless entry system, a smart key system, a power window device, a power seat, a steering wheel, an air conditioning device, and various lamps (e.g., head lamps, back lamps, brake lamps, blinkers, fog lamps, etc.).

[0196] The storage unit 211 includes, for example, a read only memory (ROM), a random access memory (RAM), a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, and a magneto-optical storage device. The storage unit 211 stores various programs and data used by each unit of the vehicle control system 200. For example, the storage unit 211 stores map data such as a three-dimensional high-precision map such as a dynamic map, a global map that is less accurate than the high-precision map and covers a wide area, and a local map that includes information about the surroundings of the vehicle.

[0197] The automatic driving control unit 212 performs control related to automatic driving such as autonomous driving or driving assistance. Specifically, for example, the automatic driving control unit 212 performs cooperative control for the purpose of realizing the functions of an ADAS (Advanced Driver Assistance System) including collision avoidance or impact mitigation of the host vehicle, following driving based on the distance between vehicles, maintaining vehicle speed, collision warning of the host vehicle, or lane departure warning of the host vehicle. In addition, for example, the automatic driving control unit 212 performs cooperative control for the purpose of automatic driving that runs autonomously without relying on the operation of the driver. The automatic driving control unit 212 includes a detection unit 231, a self-position estimation unit 232, a situation analysis unit 233, a planning unit 234, and an operation control unit 235.

[0198] The detection unit 231 detects various types of information necessary for controlling the autonomous driving. The detection unit 231 includes an outside-vehicle information detection unit 241, an inside-vehicle information detection unit 242, and a vehicle state detection unit 243.

[0199] The outside-vehicle information detection unit 241 performs a detection process of information outside the vehicle based on data or signals from each unit of the vehicle control system 200. For example, the outside-vehicle information detection unit 241 performs a detection process, a recognition process, and a tracking process of objects around the vehicle, as well as a detection process of the distance to the objects. Objects to be detected include, for example, vehicles, people, obstacles, structures, roads, traffic lights, traffic signs, road markings, and the like. In addition, for example, the outside-vehicle information detection unit 241 performs a detection process of the environment around the vehicle. The surrounding environment to be detected includes, for example, weather, temperature, humidity, brightness, and road surface conditions. The outside-vehicle information detection unit 241 supplies data indicating the results of the detection process to the self-position estimation unit 232, the map analysis unit 251, the traffic rule recognition unit 252, and the situation recognition unit 253 of the situation analysis unit 233, the emergency situation avoidance unit 271 of the operation control unit 235, and the like.

[0200] The in-vehicle information detection unit 242 performs detection processing of information inside the vehicle based on data or signals from each unit of the vehicle control system 200. For example, the in-vehicle information detection unit 242 performs authentication processing and recognition processing of the driver, detection processing of the driver's state, detection processing of passengers, detection processing of the environment inside the vehicle, etc. The driver's state to be detected includes, for example, physical condition, alertness level, concentration level, fatigue level, line of sight direction, etc. The environment inside the vehicle to be detected includes, for example, temperature, humidity, brightness, odor, etc. The in-vehicle information detection unit 242 supplies data indicating the result of the detection processing to the situation recognition unit 253 of the situation analysis unit 233, the emergency situation avoidance unit 271 of the operation control unit 235, etc.

[0201] The vehicle state detection unit 243 performs detection processing of the state of the vehicle based on data or signals from each unit of the vehicle control system 200. The vehicle state to be detected includes, for example, the speed, acceleration, steering angle, the presence or absence and content of an abnormality, the driving operation state, the position and inclination of the power seat, the door lock state, and the states of other in-vehicle devices. The vehicle state detection unit 243 supplies data indicating the result of the detection processing to the situation recognition unit 253 of the situation analysis unit 233, the emergency situation avoidance unit 271 of the operation control unit 235, etc.

[0202] The self-position estimation unit 232 performs estimation processing of the position and attitude of the vehicle based on data or signals from each unit of the vehicle control system 200, such as the outside-vehicle information detection unit 241 and the situation recognition unit 253 of the situation analysis unit 233. The self-position estimation unit 232 also generates a local map (hereinafter referred to as a self-position estimation map) used for estimating the self-position as necessary. The self-position estimation map is a high-precision map using a technology such as SLAM (Simultaneous Localization and Mapping). The self-position estimation unit 232 supplies data indicating the result of the estimation processing to the map analysis unit 251, the traffic rule recognition unit 252, the situation recognition unit 253, and the like of the situation analysis unit 233. The self-position estimation unit 232 also stores the self-position estimation map in the storage unit 211.

[0203] The situation analyzing unit 233 performs an analysis process of the situation of the vehicle itself and the surroundings. The situation analyzing unit 233 includes a map analyzing unit 251, a traffic rule recognizing unit 252, a situation recognizing unit 253, and a situation predicting unit 254.

[0204] The map analysis unit 251 performs analysis processing of various maps stored in the storage unit 211 while using data or signals from each unit of the vehicle control system 200, such as the self-position estimation unit 232 and the outside vehicle information detection unit 241, as necessary, to construct a map including information necessary for autonomous driving processing. The map analysis unit 251 supplies the constructed map to the traffic rule recognition unit 252, the situation recognition unit 253, the situation prediction unit 254, and the route planning unit 261, the action planning unit 262, the operation planning unit 263, and the like of the planner 234.

[0205] The traffic rule recognition unit 252 performs recognition processing of traffic rules around the vehicle based on data or signals from each unit of the vehicle control system 200, such as the self-position estimation unit 232, the vehicle outside information detection unit 241, and the map analysis unit 251. This recognition processing recognizes, for example, the positions and states of traffic signals around the vehicle, the details of traffic regulations around the vehicle, and lanes in which travel is possible. The traffic rule recognition unit 252 supplies data indicating the results of the recognition processing to the situation prediction unit 254 and the like.

[0206] The situation recognition unit 253 performs recognition processing of the situation regarding the vehicle based on data or signals from each unit of the vehicle control system 200, such as the self-position estimation unit 232, the vehicle outside information detection unit 241, the vehicle inside information detection unit 242, the vehicle state detection unit 243, and the map analysis unit 251. For example, the situation recognition unit 253 performs recognition processing of the situation of the vehicle, the situation around the vehicle, the situation of the driver of the vehicle, and the like. In addition, the situation recognition unit 253 generates a local map (hereinafter referred to as a situation recognition map) used for recognizing the situation around the vehicle, as necessary. The situation recognition map is, for example, an occupancy grid map.

[0207] The vehicle's status to be recognized includes, for example, the vehicle's position, posture, and movement (e.g., speed, acceleration, moving direction, etc.), as well as the presence or absence of an abnormality and its contents, etc. The vehicle's surroundings to be recognized include, for example, the type and position of surrounding stationary objects, the type, position and movement (e.g., speed, acceleration, moving direction, etc.) of surrounding moving objects, the configuration of surrounding roads and the condition of the road surface, as well as the surrounding weather, temperature, humidity, brightness, etc. The driver's status to be recognized includes, for example, physical condition, alertness, concentration, fatigue, eye movement, driving operation, etc.

[0208] The situation recognition unit 253 supplies data indicating the result of the recognition process (including a situation recognition map, if necessary) to the self-position estimation unit 232, the situation prediction unit 254, etc. In addition, the situation recognition unit 253 stores the situation recognition map in the storage unit 211.

[0209] The situation prediction unit 254 performs prediction processing of the situation regarding the vehicle based on data or signals from each unit of the vehicle control system 200, such as the map analysis unit 251, the traffic rule recognition unit 252, and the situation recognition unit 253. For example, the situation prediction unit 254 performs prediction processing of the situation of the vehicle, the situation around the vehicle, the situation of the driver, and the like.

[0210] The vehicle's status to be predicted includes, for example, the vehicle's behavior, the occurrence of an abnormality, and the driving distance, etc. The vehicle's surroundings to be predicted include, for example, the behavior of animals around the vehicle, changes in traffic signals, and changes in the environment such as weather, etc. The driver's status to be predicted includes, for example, the driver's behavior and physical condition, etc.

[0211] The situation prediction unit 254 supplies data indicating the results of the prediction process to the route planning unit 261, the action planning unit 262, the operation planning unit 263, etc. of the planner 234, together with the data from the traffic rule recognition unit 252 and the situation recognition unit 253.

[0212] The route planning unit 261 plans a route to the destination based on data or signals from each unit of the vehicle control system 200, such as the map analysis unit 251 and the situation prediction unit 254. For example, the route planning unit 261 sets a route from the current position to a specified destination based on a global map. In addition, for example, the route planning unit 261 appropriately changes the route based on conditions such as congestion, accidents, traffic restrictions, and construction, and the driver's physical condition, etc. The route planning unit 261 supplies data indicating the planned route to the action planning unit 262 and the like.

[0213] The behavior planning unit 262 plans the behavior of the vehicle for safely traveling the route planned by the route planning unit 261 within the planned time based on data or signals from each unit of the vehicle control system 200, such as the map analysis unit 251 and the situation prediction unit 254. For example, the behavior planning unit 262 plans starting, stopping, traveling direction (for example, moving forward, backward, turning left, turning right, changing direction, etc.), traveling lane, traveling speed, overtaking, etc. The behavior planning unit 262 supplies data indicating the planned behavior of the vehicle to the operation planning unit 263, etc.

[0214] The action planning unit 263 plans the action of the vehicle to realize the action planned by the action planning unit 262, based on data or signals from each unit of the vehicle control system 200, such as the map analysis unit 251 and the situation prediction unit 254. For example, the action planning unit 263 plans acceleration, deceleration, a driving trajectory, etc. The action planning unit 263 supplies data indicating the planned action of the vehicle to the acceleration / deceleration control unit 272 and the direction control unit 273 of the action control unit 235, etc.

[0215] The operation control unit 235 controls the operation of the host vehicle. The operation control unit 235 includes an emergency situation avoidance unit 271, an acceleration / deceleration control unit 272, and a direction control unit 273.

[0216] The emergency avoidance unit 271 performs detection processing of an emergency such as a collision, contact, entry into a dangerous area, driver abnormality, or vehicle abnormality based on the detection results of the vehicle outside information detection unit 241, the vehicle inside information detection unit 242, and the vehicle state detection unit 243. When the emergency avoidance unit 271 detects the occurrence of an emergency, it plans an operation of the vehicle to avoid an emergency such as a sudden stop or a sharp turn. The emergency avoidance unit 271 supplies data indicating the planned operation of the vehicle to the acceleration / deceleration control unit 272, the direction control unit 273, etc.

[0217] The acceleration / deceleration control unit 272 performs acceleration / deceleration control to realize the operation of the host vehicle planned by the operation planning unit 263 or the emergency situation avoidance unit 271. For example, the acceleration / deceleration control unit 272 calculates a control target value of a driving force generation device or a braking device to realize the planned acceleration, deceleration, or sudden stop, and supplies a control command indicating the calculated control target value to the drive system control unit 207.

[0218] The direction control unit 273 performs direction control to realize the operation of the host vehicle planned by the operation planning unit 263 or the emergency situation avoidance unit 271. For example, the direction control unit 273 calculates a control target value of a steering mechanism to realize a traveling trajectory or a sharp turn planned by the operation planning unit 263 or the emergency situation avoidance unit 271, and supplies a control command indicating the calculated control target value to the drive system control unit 207.

[0219] [2-3.Other] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

[0220] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0221] Furthermore, the above-described embodiments and modifications can be appropriately combined as long as the processing contents are not contradictory.

[0222] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0223] The present invention is not limited to mobile devices, but can also be applied to TVs that provide functions such as program recommendations using machine learning models, cameras that provide functions such as autofocus and automatic shutter, and other home appliances and smartphones. For example, an information processing device that realizes information processing according to the present disclosure is not limited to mobile devices, but can be applied to various devices such as the above-mentioned TVs, cameras, other home appliances, and devices such as smartphones.

[0224] [3. Effects of this disclosure] As described above, the information processing device according to the present disclosure (in the embodiments, the mobile device 100 and the information processing device 100A) includes an acquisition unit (in the embodiments, the acquisition unit 131) and a generation unit (in the embodiments, the generation unit 136). The acquisition unit acquires a model having a neural network structure and input information to be input to the model. The generation unit generates basis information indicating a basis for the output of the model after the input information is input to the model, based on state information indicating the state of the model after the input information is input to the model.

[0225] As a result, the information processing device according to the present disclosure can show the basis for the output of a model having a neural network structure when input information is input to the model, and can make the basis for the output of the model having a neural network structure explainable. In other words, the information processing device can make the basis for processing by the information processing device explainable.

[0226] The generation unit also generates basis information indicating the basis of processing using the output of the model. This enables the information processing device to indicate the basis for the output of the model using the output of the model, and makes it possible to explain the basis for processing by the information processing device.

[0227] The acquisition unit acquires a model used to control the autonomously acting device. The generation unit generates basis information indicating a basis for control of the device after input of input information to the model. This enables the information processing device to indicate a basis for output of the model in controlling the autonomously acting device, and makes it possible to explain the basis for processing by the information processing device.

[0228] The acquisition unit acquires a model used for controlling the autonomously movable mobile body. The generation unit generates basis information indicating a basis for control of the mobile body after input of input information to the model. This enables the information processing device to indicate a basis for output of the model in controlling the autonomously movable mobile body, and makes it possible to explain a basis for processing by the information processing device.

[0229] The generation unit also acquires a model used for controlling a moving body that is a vehicle that operates by autonomous driving. This enables the information processing device to show the basis for the output of the model in controlling the vehicle that operates by autonomous driving, and makes it possible to explain the basis for the processing by the information processing device.

[0230] Furthermore, the generating unit generates basis information indicating the basis of the moving direction of the moving object. This enables the information processing device to indicate the basis of the moving direction of the moving object, and makes it possible to explain the basis for processing by the information processing device.

[0231] The acquisition unit acquires a model that outputs in response to input of sensor information, and input information that is sensor information detected by the sensor. The generation unit generates basis information of the model to which the input information is input in response to detection by the sensor. This enables the information processing device to show the basis for the output of the model when sensor information is input to a model having a neural network structure, and makes it possible to explain the basis for processing by the information processing device.

[0232] The acquisition unit acquires the model that outputs a recognition result of the image information in response to the input of the image information, and the input information that is the image information. This enables the information processing device to show the basis for the output of the model when the image information is input to the model having the neural network structure, and makes it possible to explain the basis for the processing by the information processing device.

[0233] In addition, the generation unit generates image information indicating the basis for the output of the model as the basis information. This allows the information processing device to generate the image information indicating the basis for the output of the model, thereby making it possible to explain the basis for the processing by the information processing device.

[0234] The generating unit generates a heat map showing the basis for the output of the model as the basis information. This allows the information processing device to make the basis for the processing by the information processing device explainable by generating a heat map showing the basis for the output of the model.

[0235] In addition, the acquisition unit acquires a model including CNN. This enables the information processing device to show the basis for the output of the model including CNN, and makes it possible to explain the basis for the processing by the information processing device.

[0236] In addition, the generation unit generates the basis information based on state information including the state of the convolutional layer of the model. This enables the information processing device to show the basis for the output of the model based on the state of the convolutional layer of the model, and makes it possible to explain the basis for the processing by the information processing device.

[0237] The generating unit generates the basis information by Grad-CAM, which enables the information processing device to show the basis for the output of the model by the Grad-CAM technique, and makes it possible to explain the basis for the processing by the information processing device.

[0238] The acquisition unit acquires a model that outputs in response to an input of output information output by another model, and input information that is the output information output by the other model. The generation unit generates basis information for the model to which the input information is input in response to the output by the other model. This enables the information processing device to show the basis for the output of the model that receives the output of the other model as input, and makes it possible to explain the basis for the processing by the information processing device.

[0239] Furthermore, the basis information is generated based on state information including the output result of the model after the input information is input to the model. This enables the information processing device to show the basis for the output of the model using the output of the model based on the output result of the model, thereby making it possible to explain the basis for the processing by the information processing device.

[0240] The generation unit also generates the evidence information through processing related to LIME, which enables the information processing device to show the evidence related to the output of the model using the LIME technology, thereby making it possible to explain the evidence related to the processing by the information processing device.

[0241] The information processing device also has a display unit (display unit 11 in the embodiment). The display unit displays the basis information. This allows the information processing device to provide appropriate information on the basis for the output of the model.

[0242] The generation unit also stores log information in which the input information and the basis information are associated with each other in the storage unit. This allows the information processing device to associate and manage the input and the basis for the output, thereby enabling the information processing device to appropriately provide information indicating the basis for the output at a certain point in time for a certain input.

[0243] Moreover, the information processing device is an information processing device that performs an action using a machine learning model, and includes a sensor unit (sensor unit 14 in the embodiment) and a basis information generation unit (basis information generation unit RSD1 in the embodiment) equipped with a plurality of basis generation algorithms that generate basis information for an action, and outputs information indicating the basis for an action based on the basis information generated based on one or more basis generation algorithms and / or the sensor information. This allows the information processing device to indicate the basis for an action of the information processing device by outputting information indicating the basis for an action based on the basis information generated based on the basis generation algorithm and / or the sensor information. Therefore, the information processing device can make the basis for processing by the information processing device explainable.

[0244] [4. Hardware configuration] Information equipment such as the mobile body device 100 and the information processing device 100A according to each embodiment described above is realized by a computer 1000 having a configuration as shown in Fig. 14, for example. Fig. 14 is a hardware configuration diagram showing an example of the computer 1000 that realizes the functions of an information processing device such as the mobile body device 100 and the information processing device 100A. The mobile body device 100 according to the embodiment will be described below as an example. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. Each unit of the computer 1000 is connected by a bus 1050.

[0245] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processes corresponding to the various programs.

[0246] The ROM 1300 stores boot programs such as a basic input output system (BIOS) executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000.

[0247] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records an information processing program according to the present disclosure, which is an example of program data 1450.

[0248] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0249] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading a program or the like recorded on a predetermined recording medium. The medium may be, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0250] For example, when the computer 1000 functions as the mobile device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes an information processing program loaded onto the RAM 1200 to realize functions of the control unit 13, etc. Also, the information processing program according to the present disclosure and data in the storage unit 12 are stored in the HDD 1400. Note that the CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, these programs may be obtained from other devices via an external network 1550.

[0251] The present technology can also be configured as follows. (1) An acquisition unit that acquires a model having a neural network structure and input information to be input to the model; a generation unit that generates, based on state information indicating a state of the model after the input information is input to the model, grounds information indicating a grounds for an output of the model after the input information is input to the model; An information processing device comprising: (2) The generation unit is Generate the basis information indicating the basis of processing using the output of the model. The information processing device according to (1). (3) The acquisition unit is obtaining said model for use in controlling an autonomously acting device; The generation unit is Generate the basis information indicating the basis for control of the device after the input information is input to the model. The information processing device according to (1) or (2). (4) The acquisition unit is Obtaining the model used for controlling a moving object capable of moving autonomously; The generation unit is Generate the basis information indicating the basis for control of the moving object after the input information is input to the model. The information processing device according to any one of (1) to (3). (5) The acquisition unit is The model used to control the moving body, which is a vehicle that operates by autonomous driving, is obtained. The information processing device according to (4). (6) The generation unit is Generate the grounds information indicating the grounds for the direction of movement of the moving object. The information processing device according to (4) or (5). (7) The acquisition unit is Acquire the model that outputs in response to input of sensor information, and the input information that is the sensor information detected by a sensor; The generation unit is The basis information of the model to which the input information is input in response to detection by the sensor is generated. The information processing device according to any one of (1) to (6). (8) The acquisition unit is The model outputs a recognition result of image information in response to input of the image information, and the input information is the image information. The information processing device according to any one of (1) to (7). (9) The generation unit is Image information showing a basis for the output of the model is generated as the basis information. The information processing device according to any one of (1) to (8). (10) The generation unit is A heat map showing evidence regarding the output of the model is generated as the evidence information. The information processing device according to (9). (11) The acquisition unit is Obtain the above model, including CNN (Convolutional Neural Network) The information processing device according to any one of (1) to (10). (12) The generation unit is Generate the basis information based on the state information including a state of a convolutional layer of the model. The information processing device according to (11). (13) The generation unit is The basis information is generated by processing related to CAM (Class Activation Mapping). The information processing device according to (11) or (12). (14) The generation unit is The basis information is generated using Grad-CAM (Gradient-weighted Class Activation Mapping). The information processing device according to (13). (15) The acquisition unit is Obtaining the model that performs output in response to input of output information output by another model, and the input information that is the output information output by the other model; The generation unit is Generate the basis information of the model to which the input information has been input in response to the output by the other model. The information processing device according to any one of (1) to (14). (16) The generation unit is generating the basis information based on the state information including an output result of the model after the input information is input to the model; The information processing device according to any one of (1) to (15). (17) The generation unit is The basis information is generated based on a basis model trained using the input information and the output result. The information processing device according to (16). (18) The generation unit is The basis information is generated using the basis model that locally approximates a combination of the input information and the output result. The information processing device according to (17). (19) The generation unit is Generate the evidence information through processing related to LIME (Local Interpretable Model-agnostic Explanations) The information processing device according to (17) or (18). (20) A display unit for displaying the grounds information; Equipped The information processing device according to any one of (1) to (19). (twenty one) The display unit is Display the evidence information as a diagram The information processing device according to (20). (twenty two) The display unit is Displaying the grounds information which is image information The information processing device according to (21). (twenty three) The display unit is Displaying the evidence information as a heat map The information processing device according to (22). (twenty four) The display unit is Display the basis information as text The information processing device according to any one of (20) to (23). (twenty five) The display unit is Display the basis information as a numerical value The information processing device according to any one of (20) to (24). (26) The generation unit is Log information in which the input information and the basis information are associated with each other is stored in a storage unit. The information processing device according to any one of (1) to (25). (27) Obtaining a model having a neural network structure and input information to be input to the model; generating, based on state information indicating a state of the model after the input information is input to the model, grounds information indicating a grounds for an output of the model after the input information is input to the model; An information processing method for performing processing. (28) Obtaining a model having a neural network structure and input information to be input to the model; generating, based on state information indicating a state of the model after the input information is input to the model, grounds information indicating a grounds for an output of the model after the input information is input to the model; An information processing program that executes processing. (29) In an information processing device that performs an action using a machine learning model, A sensor unit; a basis information generating unit including a plurality of basis information generating algorithms for generating basis information for the action; Equipped with 13. An information processing device comprising: an information processing apparatus that outputs information indicating a basis for the action based on basis information generated based on one or more basis generation algorithms and / or sensor information. [Explanation of symbols]

[0252] 100 Mobile device 100A Information processing equipment 11 Display section 12 Storage section 121 Model information storage unit 122 Log information storage unit 13, 13A Control section 131 Acquisition Department 132 Recognition part 133 Prediction Department 134 Action Planning Department 135 Executive Department 136 Generation part 137 Transmitter 14 Sensor section 141 Image Sensor 15 Drive unit 16 Communications Department

Claims

1. An acquisition unit that acquires input information to be input to a model used for controlling a moving object that can move autonomously; a generation unit that generates, based on state information indicating a state of the model after the input information is input to the model, grounds information indicating a grounds for control of the moving object after the input information is input to the model; Equipped with Outputting grounds information indicating grounds for control of the moving body generated by the generation unit, and displaying an area that the model is gazing at separately from other areas according to an object detected by a sensor unit that detects the object. Information processing device.

2. The gazed area is the area in which the model recognizes the object as a person. The information processing device according to claim 1 .

3. The generation unit is Generate the basis information indicating the basis of processing using the output of the model. The information processing device according to claim 1 .

4. The acquisition unit is obtaining said model for use in controlling an autonomously acting device; The generation unit is Generate the basis information indicating the basis for control of the device after the input information is input to the model. The information processing device according to claim 1 .

5. The acquisition unit is The model used to control the moving body, which is a vehicle that operates by autonomous driving, is obtained. The information processing device according to claim 1 .

6. The generation unit is Generate the grounds information indicating the grounds for the direction of movement of the moving object. The information processing device according to claim 1 .

7. The acquisition unit is Acquire the model that outputs in response to input of sensor information, and the input information that is the sensor information detected by a sensor; The generation unit is The basis information of the model to which the input information is input in response to detection by the sensor is generated. The information processing device according to claim 1 .

8. The acquisition unit is The model outputs a recognition result of image information in response to input of the image information, and the input information is the image information. The information processing device according to claim 1 .

9. The generation unit is Image information showing a basis for the output of the model is generated as the basis information. The information processing device according to claim 1 .

10. The generation unit is A heat map showing evidence regarding the output of the model is generated as the evidence information. The information processing device according to claim 9.

11. The acquisition unit is Obtain the model including CNN (Convolutional Neural Network) The information processing device according to claim 1 .

12. The generation unit is Generate the basis information based on the state information including a state of a convolutional layer of the model. The information processing device according to claim 11.

13. The generation unit is The basis information is generated by Grad-CAM (Gradient-weighted Class Activation Mapping). The information processing device according to claim 12.

14. The acquisition unit is Obtaining the model that performs output in response to input of output information output by another model, and the input information that is the output information output by the other model; The generation unit is Generate the basis information of the model to which the input information has been input in response to the output by the other model. The information processing device according to claim 1 .

15. The generation unit is generating the basis information based on the state information including an output result of the model after the input information is input to the model; The information processing device according to claim 1 .

16. The generation unit is The basis information is generated by processing related to LIME (Local Interpretable Model-agnostic Explanations). The information processing device according to claim 15.

17. A display unit for displaying the grounds information; Equipped The information processing device according to claim 1 .

18. Acquire input information to be input to a model used for controlling a mobile object capable of autonomous movement; generating grounds information indicating a grounds for control of the moving object after the input information is input to the model, based on state information indicating a state of the model after the input information is input to the model; outputting grounds information indicating the grounds for the control of the moving body that has been generated, and displaying an area that the model is gazing at in distinction from other areas in accordance with an object detected by a sensor unit that detects the object; An information processing method for performing a process.

19. Acquire input information to be input to a model used for controlling a mobile object capable of autonomous movement; generating grounds information indicating a grounds for control of the moving object after the input information is input to the model, based on state information indicating a state of the model after the input information is input to the model; outputting grounds information indicating the grounds for the control of the moving body that has been generated, and displaying an area that the model is gazing at in distinction from other areas in accordance with an object detected by a sensor unit that detects the object; An information processing program that executes processing.

20. An information processing device that controls a moving object that can move autonomously using a machine learning model, A sensor unit that detects an object; a basis information generating unit including a plurality of basis generation algorithms for generating basis information for controlling the moving object; Equipped with An information processing device characterized by outputting basis information indicating the basis for control of the moving body generated based on one or more basis generation algorithms, and displaying the area that the machine learning model is focusing on separately from other areas depending on the object detected by the sensor unit.

Images