Information processing system, information processing device, and information processing method

Abstract

[Problem] To provide an information processing system capable of supporting driving in accordance with different automatic driving levels. [Solution] An information processing system according to an embodiment of the present disclosure comprises: at least one moving device for which an automatic driving level can be set; and an external network device which can communicate with the moving device. The external network device has: a communication device which communicates with the moving device; an arithmetic model determination device which determines an arithmetic model corresponding to the automatic driving level and provides the arithmetic model to the moving device via the communication device; and a registration determination device which determines, on the basis of information pertaining to possession of the arithmetic model, whether to permit registration of the automatic driving level, and notifies the moving device via the communication device of permission to register the automatic driving level. The moving device has: an arithmetic model request unit which requests the arithmetic model from the arithmetic model determination device, and, upon being provided with the requested arithmetic model, causes information to be transmitted to the registration determination device; and a movement control unit which, upon receiving the notification from the registration determination device, starts movement control based on the automatic driving level that has been permitted to be registered.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing device, and an information processing method. [Background technology]

[0002] In recent years, vehicles have been equipped with various sensors, and driving assistance functions are being introduced. The IEEE has also developed a vehicle-to-vehicle communication system called Dedicated Short Range Communication (DSRC) based on 802.11p, and 3GPP has also formulated the C-V2X standard in Rel-14, which is based on LTE (Long Term Evolution) Device-to-Device (D2D) communication. Advanced safety driving systems called ADAS (Advanced Driver-Assistance Systems) are beginning to be introduced, utilizing this vehicle-to-vehicle communication and sensor fusion that utilizes various sensors installed in vehicles. Furthermore, fully autonomous driving without human intervention is expected to be realized using artificial intelligence (AI), machine learning (ML), and deep learning (DL). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-174355 Summary of the Invention [Problem to be solved by the invention]

[0004] There are several levels of autonomous driving defined according to the degree of human intervention, and until all vehicles are capable of fully autonomous driving, which requires neither a steering wheel nor an accelerator, as defined by the Society of Automotive Engineers (SAE) International as Level 5, vehicles capable of various levels of autonomous driving will coexist. In this situation, providing MaaS (Mobility as a Service) will require driving assistance according to the different levels of autonomous driving.

[0005] The present disclosure provides an information processing system, an information processing device, and an information processing method that are capable of providing driving assistance according to different autonomous driving levels. [Means for solving the problem]

[0006] An information processing system according to an embodiment of the present disclosure includes one or more mobile devices capable of setting an autonomous driving level, and an external network device capable of communicating with the mobile devices, The external network device a communication device for communicating with the mobile device; a calculation model determination device that determines a calculation model corresponding to the autonomous driving level and provides the calculation model to the mobile device via the communication device; a registration determination device that determines whether or not the autonomous driving level can be registered based on information about ownership of the computation model, and notifies the mobile device of registration permission via the communication device; The moving device is a calculation model request unit that requests the calculation model from the calculation model determination device and, when the requested calculation model is provided, causes the information to be transmitted to the registration determination device; The mobile control unit has a function of starting mobile control based on the autonomous driving level for which registration is permitted when the notification is received from the registration determination device.

[0007] The communication device may be a base station device that wirelessly communicates with the mobile device.

[0008] The computation model determination device may be a server having a communication unit that communicates with the mobile device via the base station device.

[0009] The registration determination device may be an operation management device having a communication unit that communicates with the mobile device via the base station device.

[0010] In addition, the mobile device may further have a calculation model determination unit that checks the expiration date or valid area of ​​the first calculation model it owns, and the calculation model request unit may request a second calculation model with a valid expiration date or valid area depending on the confirmation result of the calculation model determination unit.

[0011] The mobile device may further include a computational model storage unit that stores the computational model, and the first computational model may be updated to the second computational model in the computational model storage unit.

[0012] In addition, the registration determination device may further have a compatibility test execution instruction unit that determines whether a compatibility test is necessary based on the autonomous driving level and instructs the mobile device to perform a compatibility test based on the determination result.

[0013] In addition, the registration determination device may further have a registration permission determination unit that instructs a change in the setting of the autonomous driving level based on the location of the mobile device, and the mobile device may further have an autonomous driving level setting unit that changes the setting of the autonomous driving level based on the instruction of the registration permission determination unit.

[0014] In addition, the computation model request unit may request a computation model corresponding to the changed autonomous driving level from the computation model determination device, and update the first computation model it owns to the second computation model obtained from the computation model determination device.

[0015] Furthermore, the compatibility test execution instruction unit may instruct the mobile device to periodically execute the compatibility test when the autonomous driving level is equal to or higher than a predetermined level.

[0016] An information processing device according to an embodiment of the present disclosure includes: a computation model request unit that requests and acquires a computation model corresponding to an autonomous driving level set in the mobile device; and a mobility control unit that, when registration of the autonomous driving level is permitted based on information regarding ownership of the computational model, starts controlling the mobility of the mobile device based on the autonomous driving level permitted for registration.

[0017] The mobile device may further include a registration request processing unit that requests an external network device capable of communicating with the mobile device to register the autonomous driving level, and the calculation model request unit may request the calculation model from the external network device and acquire the calculation model from the external network device.

[0018] The mobile device may further include a driving assistance processing unit that controls the mobile control unit based on the results of calculation of detection data from a sensor provided in the mobile device using the calculation model, and the calculation model may be determined by the external network device based on at least one of the detection content of the sensor, the type of the mobile device, and the performance of the driving assistance processing unit.

[0019] The device may further include a computation model determination unit that checks the expiration date or valid area of ​​the first computation model owned, and the computation model request unit may request a second computation model with a valid expiration date or valid area according to the confirmation result of the computation model determination unit.

[0020] The system may further include a computational model storage unit for storing the computational model, and the first computational model may be updated to the second computational model in the computational model storage unit.

[0021] An information processing method according to an embodiment of the present disclosure includes: One or more mobile devices capable of setting an autonomous driving level request a calculation model corresponding to the autonomous driving level from an external network device; The external network device determines the computation model and provides it to the mobile device; the mobile device transmits information regarding ownership of the computational model to the external network device; The external network device determines whether or not the autonomous driving level can be registered based on the information, and notifies the mobile device of permission for registration; When the mobile device receives the notification, it starts mobility control based on the autonomous driving level for which registration is permitted. [Brief explanation of the drawings]

[0022] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a driving assistance system according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of a mobile device. [Figure 3] FIG. 2 is a block diagram showing an example of the configuration of an operation management device. [Figure 4] FIG. 2 is a block diagram illustrating an example of the configuration of a server. [Figure 5] 10 is a flowchart illustrating an example of a process for setting an autonomous driving level. [Figure 6] FIG. 10 is a sequence diagram illustrating an example of a registration process for an autonomous driving level. [Figure 7] FIG. 10 is a sequence diagram illustrating an example of a process related to validity confirmation of a computation model. [Figure 8] FIG. 10 is a sequence diagram of another example of processing related to validity confirmation of a computation model. [Figure 9] 10 is a flowchart showing a series of processes of a mobile device related to setting an autonomous driving level. [Figure 10] 10 is a flowchart showing another example of the process of registering an autonomous driving level. [Figure 11] 10 is a flowchart illustrating an example of operation management processing of the operation management device. [Figure 12] 10 is a flowchart showing an example of a part of a traffic management process. [Figure 13] 10 is a flowchart showing another example of a part of the operation management process. DETAILED DESCRIPTION OF THE INVENTION

[0023] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The following description will focus on the main components of the present disclosure, but the present disclosure may include components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

[0024] FIG. 1 is a diagram illustrating an example of the configuration of a driving assistance system according to an embodiment. The driving assistance system 100 according to this embodiment includes an operation management device 110, a server 120, a base station device 130, and a mobile device 140. While two mobile devices 140 are illustrated in FIG. 1, the number of mobile devices 140 may be one or three or more. That is, the driving assistance system 100 includes one or more mobile devices 140. The driving assistance system 100 is an example of an information processing system. The operation management device 110, the server 120, and the base station device 130 constitute external network devices. The operation management device 110, the server 120, and the base station device 130 are examples of a registration determination device, a computation model determination device, and a communication device, respectively. Although FIG. 1 illustrates one base station device 130, two or more base station devices 130 may be deployed to provide a nationwide communication area. Here, the base station device 130 may include various types of devices such as base stations with different maximum transmission powers and operating frequency bands, such as macrocells, picocells, microcells, small cells, and femtocells, RRHs (Remote Radio Heads), and even TRPs (Transmission / Reception Points).

[0025] The operation management device 110 collects data from the mobile device 140 via the base station device 130 and manages the movement, i.e., the traveling, of the mobile device 140. The management related to traveling broadly includes the control and management of the driving assistance processing unit 142 provided in the mobile device 140.

[0026] The server 120 executes various calculation processes instructed by the operation management device 110. For example, the server 120 performs machine learning or deep learning using data acquired by the operation management device 110 from the mobile device 140, and generates a neural network model as a result.

[0027] The server 120 may be provided in (integrated with) the operation management device 110. Furthermore, the data acquired from the mobile device 140 may include, for example, data acquired by a sensor 141 provided in the mobile device 140, or processed data, as well as control information and output information of a power system, a braking device, and a steering device provided in the mobile device 140.

[0028] The server 120 may generate and manage a neural network model for each autonomous driving level of each mobile device 140. The autonomous driving levels established by SAE (Society of Automotive Engineers) International are defined as follows: Level 0: The driver controls everything Level 1: The system supports either steering or acceleration / deceleration. Level 2: The system supports both steering and acceleration / deceleration. Level 3: The system operates everything in specific locations, and the driver takes control in emergencies. Level 4: The system controls everything in a specific location Level 5: The system controls everything, regardless of location. Specific locations defined as Level 4 are expected to include, for example, highways for private cars and trucks in logistics services, or areas with relatively low traffic volume and good visibility, such as depopulated areas, university campuses, airport facilities, and other areas with relatively simple driving environments, for mobility services. In the following explanation, the autonomous driving level refers to, for example, one of these five levels. Furthermore, the autonomous driving levels may be classified more finely than these five levels.

[0029] 2 is a block diagram showing an example configuration of the mobile device 140. The mobile device 140 has a sensor 141, a driving assistance processing unit 142, a movement control unit 143, a communication unit 144, a registration request processing unit 145, a computation model request unit 146, an autonomous driving level setting unit 147, a computation model determination unit 148, a computation model storage unit 149, and a compatibility test execution unit 150. Each unit except for the sensor 141 constitutes an information processing device.

[0030] The sensor 141 includes, for example, a position information sensor 141a, a camera module (including an image sensor) 141b, a LiDAR (Light Detection and Ranging or Laser Imaging Detection and Ranging) 141c, and a radar 141d. The sensor 141 may include at least one of these sensors. The sensor 141 may also include an inertial measurement unit called an IMU (Inertial Measurement Unit), which is a unit that integrates an acceleration sensor, a rotational angular acceleration sensor / gyro sensor, a magnetic field sensor, a barometric pressure sensor, a temperature sensor, and the like.

[0031] The position information sensor 141a may broadly include positioning technologies using not only a Global Navigation Satellite System (GNSS) such as a Global Positioning System (GPS), but also an odometer, and signals transmitted and received via a communication unit 144 compatible with a Long Term Evolution (LTE) or 4G or 5G cellular system. Positioning technologies in cooperation with a 4G or 5G cellular system may broadly include the position information sensor 141a acquiring information to assist positioning from a Location Management Function (LMF) via an LTE Positioning Protocol (LPP), or providing data detected by the position information sensor 141a to the LMF via the LPP, which then calculates the position. Furthermore, the position information sensor 141a may include positioning technologies that utilize high-precision three-dimensional geospatial information called a dynamic map that is stored in advance by the driving assistance processing unit 142 or dynamically acquired and updated via the communication unit 144. The camera module 141b is equipped with a plurality of image sensors and acquires image information of the outside of the vehicle and image information of the inside of the vehicle, including the driver's movements and facial expressions.

[0032] The driving assistance processing unit 142 controls the movement control unit 143 based on the calculation results obtained by inputting the detection data of the sensor 141 into a neural network model generated by AI, for example, machine learning or deep learning, and performs processing to realize ADAS / AD (Autonomous Driving).

[0033] The neural network model may be pre-installed in the driving assistance processing unit 142, or may be acquired from the driving management device 110 or the server 120 via the base station device 130 and updated and stored as appropriate. The neural network model may consist of one neural network model or may be composed of multiple neural network models. The multiple neural network models may be so-called edge AI, such as a neural network model prepared for each sensor 141, or a neural network model prepared for each control of the mobility control unit 143.

[0034] The movement control unit 143 supplies, for example, control information such as acceleration and deceleration to the power system, control information such as deceleration and stopping to the braking device, and control information such as xx [cm] left, yy [cm] right to the steering device.

[0035] The communication unit 144 performs wireless communication with the server 120 and the operation management device 110 via the base station device 130. The communication unit 144 also inputs and outputs data between the driving assistance processing unit 142, the registration request processing unit 145, the calculation model request unit 146, and the compatibility test execution unit 150. Here, the base station device 130 may include a roadside unit or an RSU (Road Side Unit), which is a transportation infrastructure.

[0036] Furthermore, the communication unit 144 performs V2X communication such as vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, vehicle-to-network communication, and vehicle-to-pedestrian communication. V2X communication can be realized by providing the communication unit 144 with a V2X module.

[0037] The registration request processing unit 145 performs processing with the operation management device 110 regarding a request to register the autonomous driving level of the mobile device 140. The calculation model request unit 146 performs processing with the server 120 regarding a request to provide a calculation model corresponding to the autonomous driving level for which registration is requested. The autonomous driving level setting unit 147 performs processing related to setting the autonomous driving level of the mobile device 140. The calculation model determination unit 148 performs processing related to determining the validity of the calculation model. The calculation model storage unit 149 stores the calculation model. The compatibility test execution unit 150 performs processing related to the execution of a compatibility test that is set in advance according to the autonomous driving level or that is instructed by the operation management device 110. The processing content of each unit will be explained in detail later.

[0038] 3 is a block diagram showing an example configuration of the operation management device 110. The operation management device 110 includes a communication unit 111, a computation model confirmation unit 112, a computation model acquisition instruction unit 113, a registration request reception unit 114, a registration permission determination unit 115, an autonomous driving level management unit 116, a compatibility test execution instruction unit 117, a compatibility test result acquisition unit 118, and a compatibility test determination unit 119.

[0039] The communication unit 111 outputs data received from the server 120 or data received from the mobile device 140 via the base station device 130 to each unit of the operation management device 110 according to the content of the data. In addition, the communication unit 111 transmits data input from each unit of the operation management device 110 to the mobile device 140 or the server 120.

[0040] The computation model confirmation unit 112 performs confirmation processing of a computation model corresponding to the autonomous driving level for which registration is requested from the mobile device 140. The computation model acquisition instruction unit 113 performs processing related to an instruction to acquire the computation model confirmed by the computation model confirmation unit 112, to the mobile device 140. The registration request receiving unit 114 performs processing related to reception of a registration request for the autonomous driving level from the mobile device 140. The registration permission determination unit 115 performs determination processing related to whether or not the autonomous driving level of the mobile device 140 can be registered. The autonomous driving level management unit 116 manages the autonomous driving level of each mobile device 140. The compatibility test execution instruction unit 117 performs processing related to an instruction to execute a compatibility test according to the autonomous driving level of the mobile device 140. The compatibility test result acquisition unit 118 performs processing related to acquisition of test results from the mobile device 140 that has executed the compatibility test. The compatibility test determination unit 119 performs determination processing related to the pass / fail of the compatibility test, based on the test results acquired by the compatibility test result acquisition unit 118. The processing contents of each unit will be explained in detail later.

[0041] 4 is a block diagram showing an example of the configuration of the server 120. The server 120 includes a communication unit 121, a computation model request receiving unit 122, a computation model managing unit 123, a data collecting unit 124, and a computation model generating unit 125.

[0042] The communication unit 121 outputs data received from the operation management device 110 or data received from the mobile device 140 via the base station device 130 to the computation model request receiving unit 122 or the data collecting unit 124 depending on the content of the data. In addition, the communication unit 121 transmits data input from the computation model request receiving unit 122 to the mobile device 140 or the operation management device 110.

[0043] The computation model request receiving unit 122 performs a receiving process related to a request to provide a computation model from the mobile device 140. The computation model managing unit 123 performs a process related to the management of the computation model to be provided to each mobile device 140. The data collecting unit 124 performs a process related to the collection of data detected by the mobile device 140. The computation model generating unit 125 performs a process related to the generation of a computation model using the data collected by the data collecting unit 124. The processing content of each unit will be explained in detail later.

[0044] 5 is a flowchart showing an example of a process for setting an autonomous driving level. The process for setting an autonomous driving level by the moving device 140 will be described below.

[0045] First, the autonomous driving level setting unit 147 of the mobile device 140 sets an autonomous driving level prior to driving and notifies the registration request processing unit 145 (step S201). In step S201, the autonomous driving level setting unit 147 may, for example, set an autonomous driving level related to the most recent operation, or may dynamically change or update the autonomous driving level.

[0046] Next, the registration request processing unit 145 determines whether the autonomous driving level is level 1 or higher (step S202). If the autonomous driving level is level 1 or higher, the registration request processing unit 145 starts registration processing in the operation management device 110 (step S203). On the other hand, if the autonomous driving level is lower than level 1 in step S202, the registration request processing unit 145 does not perform registration processing and ends the autonomous driving level setting processing.

[0047] 5 shows an example in which the registration process is initiated when the autonomous driving level is level 1 or higher, but the present invention is not limited to this example. For example, the operation management device 110 may set the mobile device 140 to initiate the registration process when the autonomous driving level is level 3 or higher. Furthermore, the operation management device 110 may vary the autonomous driving level at which the registration process is initiated depending on the area or time period.

[0048] 6 is a sequence diagram showing an example of a process for registering an autonomous driving level. The process for registering the autonomous driving level of the mobile device 140 in the operation management device 110 will be described below.

[0049] When the mobile device 140 is powered on (the engine is started) (step S301), the registration request processor 145 starts the registration process (step S302).

[0050] Next, the communication unit 144 transmits a registration request including the autonomous driving level to the driving management device 110 via the base station device 130 (step S303). In step S303, the registration request may include, in addition to the autonomous driving level, information on, for example, the location information of the mobile device 140, the vehicle model, the performance (hardware configuration) of the driving assistance processing unit 142, or the hardware type, model number, year, OS (Operating System) version, and firmware version of the installed ECU (Electronic Control Unit). Furthermore, the registration request may include information on capabilities such as applicability to a specific driving mode, for example, platooning.

[0051] In the operation management device 110, when the registration request receiving unit 114 receives a registration request via the communication unit 111, the autonomous driving level management unit 116 notifies the computation model confirmation unit 112 of the autonomous driving level included in the registration request. The computation model confirmation unit 112 confirms the computation model corresponding to the autonomous driving level and notifies the computation model acquisition instruction unit 113. The computation model acquisition instruction unit 113 instructs the requesting mobile device 140, via the base station device 130, to acquire a computation model corresponding to the autonomous driving level that is the target of the registration request (step S304). In step S304, for example, the computation model acquisition instruction unit 113 may instruct the mobile device 140 to use a computation model that it already holds if the autonomous driving level included in the registration request is level 2 or lower, and may instruct the mobile device 140 to acquire a computation model if the autonomous driving level is level 3 or higher. In addition, the calculation model instructed by the calculation model acquisition instruction unit 113 may be selected according to the configuration of the sensor 141, the type of vehicle of the mobile device 140, and the performance of the driving assistance processing unit 142, in addition to responding to the requested level of autonomous driving.

[0052] In the mobile device 140 that has received the instruction to acquire the computation model, the computation model request unit 146 transmits a request for the computation model via the communication unit 111 (step S305). The request for the computation model is transmitted to the server 120 via the base station device .

[0053] In the server 120, the computation model request receiving unit 122 receives a request for a computation model through the communication unit 121. Subsequently, the computation model management unit 123 provides the computation model indicated in the request (step S306). The provided computation model is transmitted from the communication unit 121 to the requesting mobile device 140 via the base station device 130. The computation model is, for example, a neural network model obtained by machine learning or deep learning, and a different neural network model is provided for each autonomous driving level. Furthermore, an expiration date and a valid geographical or spatial area may be set for the computation model.

[0054] This neural network model is composed of layers called input layers, hidden layers (or intermediate layers), and output layers, each containing multiple nodes, and each node is connected via an edge. Each layer has a function called an activation function, and each edge is weighted. A neural network model based on deep learning is composed of multiple hidden layers.

[0055] The neural network model is, for example, a model in a form called a CNN (Convolution Neural Network), an RNN (Recurrent Neural Network), or an LSTM (Long Short-Term Memory).

[0056] In a CNN, the hidden layer is composed of layers called a convolution layer and a pooling layer. In the convolution layer, filtering is performed using a convolution operation to extract data called a feature map. In the pooling layer, the information of the feature map output from the convolution layer is compressed and downsampled. CNN is used, for example, for image recognition applications, and information about each pixel, also called a pixel of an image, is input to the input layer, and information about the recognized image can be obtained as the output layer. For example, information about each pixel of an image detected by the camera module 141b is input to the CNN.

[0057] RNNs have a network structure in which values ​​of hidden layers are recursively input to hidden layers, and are used to process, for example, short-term time-series data.

[0058] In LSTM, by introducing parameters that hold the state of the intermediate layer called memory cells into the intermediate layer output of RNN, it is possible to retain the influence of outputs from the distant past. In other words, LSTM processes time series data over a longer period than RNN.

[0059] The input layer of the RNN or LSTM receives, for example, time series data of information related to the position detected by the position information sensor 141a, time series data of information related to ranging detected by the LiDAR 141c and the radar 141d, time series data of angular velocity and acceleration detected by the IMU, time series data of control information from the movement control unit 143, information related to the positions and speeds of surrounding vehicles acquired via V2X communication by the communication unit 144, or time series data of information detected by sensors equipped in the surrounding vehicles, and time series data of information related to a recognized image obtained as the output of the CNN constituting the computation model. Also, the input layer of the RNN or LSTM receives, for example, time series data of high-precision three-dimensional geospatial information included in a dynamic map.

[0060] The computation model is composed of one or more CNNs, RNNs, and / or LSTMs, and is processed in a dependent or parallel manner. The output of the computation model is used, for example, as control information for the movement control unit 143. The computation results of the computation model are output, for example, as detection processing and recognition processing of objects around the movement device 140. The object detection processing is, for example, processing to detect the presence or absence, size, shape, position, movement, etc. of an object. The object recognition processing is, for example, processing to recognize attributes such as the type of object, or to identify a specific object. However, the detection processing and the recognition processing are not necessarily clearly separated, and may overlap.

[0061] Furthermore, the computational model detects objects around the vehicle 1 by performing clustering to classify a point cloud based on sensor data from the LiDAR 141c, radar 141d, or the like into clusters of points as output results. This detects the presence, size, shape, and position of objects around the vehicle 1. In this case, the computational model may be a computational model provided as an edge AI as described above. The movement of objects around the vehicle 1 is detected by tracking the movement of the clusters of points classified by clustering. This detects the speed and traveling direction (movement vector) of objects around the mobile device 140.

[0062] The computational model also detects or recognizes vehicles, people, bicycles, obstacles, structures, roads, traffic lights, traffic signs, road markings, etc. from image data supplied from the camera module (including an image sensor) 141b. The computational model may also recognize the type of object around the mobile device 140 by performing recognition processing such as semantic segmentation.

[0063] In addition, sensor fusion processing is performed to combine multiple different types of sensor data (for example, data supplied from two or more of the camera module 141b, the LiDAR 141c, and the radar 141d) to obtain new information. Methods for combining different types of sensor data include early fusion, which integrates data from each sensor without preprocessing, late fusion, which integrates data after preprocessing, and other integration methods such as integrating only feature amounts, integrating only recognition results, and integrating at different processing levels for each sensor.

[0064] Furthermore, the computational model can perform recognition processing of traffic rules around the mobile device 140 and output the results based on the stored map information, the estimation result of the self-position by the position information sensor 141a, and the recognition result of objects around the mobile device 140. This processing makes it possible to recognize the positions and states of traffic signals, the contents of traffic signs and road markings, the contents of traffic regulations, and lanes that can be traveled.

[0065] Furthermore, for example, the computational model can perform a process of recognizing the environment around the vehicle 1. The surrounding environment to be recognized by the recognition unit 73 may include weather, temperature, humidity, brightness, and road surface conditions.

[0066] Also, for example, the computational model creates a behavior plan for the mobile device 140. The behavior plan is created by performing path planning and path following processing using the output from the computational model as described above.

[0067] Global path planning is a process for planning a rough route from the start to the goal. This route planning is called trajectory planning, and also includes local path planning, which takes into account the motion characteristics of the vehicle 1 on the route planned by the route planning, and enables safe and smooth progress in the vicinity of the mobile device 140. Path planning may be distinguished as long-term path planning, and trajectory generation may be distinguished as short-term path planning or local path planning. A safety-priority path represents a concept similar to trajectory generation, short-term path planning, or local path planning.

[0068] Path following is a process of planning an operation for traveling safely and accurately along a route planned by a route planner within a planned time. The computational model can, for example, calculate the target speed and target angular velocity of the mobile device based on the results of this path following process.

[0069] Furthermore, in order to realize the action plan created as the output of the computational model, the movement of the mobile device 140 is controlled by the movement control unit 143.

[0070] For example, the movement control unit 143 performs steering, acceleration, deceleration, etc., and controls the movement device 140 to move along the trajectory calculated by the trajectory plan. For example, the movement control unit 143 performs cooperative control with the aim of realizing ADAS functions such as collision avoidance or impact mitigation, following driving, maintaining vehicle speed, collision warning for the own vehicle, and lane departure warning for the own vehicle. For example, the movement control unit 143 performs cooperative control with the aim of automatic driving, which drives autonomously without relying on driver operation.

[0071] Furthermore, the system may recognize and detect information inside the vehicle, and perform authentication of the driver and recognition of the driver's state based on sensor data from an in-vehicle sensor, such as a camera module (including an image sensor) 141b installed inside the vehicle, input data input to an HM (Human Machine Interface), etc. In this case, the driver's state to be recognized may be, for example, physical condition, level of alertness, level of concentration, level of fatigue, line of sight, level of intoxication, driving operation, posture, etc.

[0072] As a result of the vehicle interior recognition and detection process, authentication process for passengers other than the driver and recognition process for the passengers' conditions may be performed. Furthermore, for example, recognition process for the vehicle interior conditions may be performed based on sensor data from an in-vehicle sensor. Possible vehicle interior conditions to be recognized include, for example, temperature, humidity, brightness, and odor.

[0073] In the mobile device 140, the computation model request unit 146 receives the computation model through the communication unit 144 and stores it in the computation model storage unit 149. Next, the computation model request unit 146 causes the communication unit 144 to transmit information related to the possession of a valid computation model (step S307). This information is transmitted to the operation management device 110 via the base station device 130. In addition, the information related to the possession of a valid computation model includes, for example, information related to the computation model, such as an ID (Identification) that identifies the computation model, an expiration date, and a valid area.

[0074] In the operation management device 110, the above information is input to the registration permission determination unit 115 via the communication unit 111 and the computation model confirmation unit 112. The registration permission determination unit 115 determines whether or not to permit registration based on the above information (step S308). In step S308, the registration permission determination unit 115 permits registration when, for example, an ID (Identification) that identifies the computation model matches a pre-registered genuine ID, and denies registration if not.

[0075] If the information regarding possession of a valid computation model satisfies the registration requirements, the registration permission determination unit 115 notifies the mobile device 140 of registration permission via the base station device 130 (step S309).

[0076] In the mobile device 140, the registration request processing unit 145 receives the notification of registration permission via the communication unit 144 and confirms the registration permission (step S310). This puts the mobile control unit 143 into an operable state (step S311). Thereafter, the mobile control unit 143 starts operation at an autonomous driving level corresponding to the calculation model (step S312).

[0077] If the mobile device 140 already owns a valid calculation model, in step S303, a registration request including information regarding the ownership of the valid calculation model in addition to the autonomous driving level may be transmitted to the operation management device 110. Upon receiving the information regarding the ownership of the valid calculation model in addition to the autonomous driving level, the operation management device 110 skips the processes from step S304 to step S307 and executes the processes from step S308 onwards.

[0078] After starting operation at the autonomous driving level, the mobile device 140 appropriately checks the validity of the computation model it owns. The following describes the process of checking the validity of the computation model.

[0079] 7 is a sequence diagram showing an example of processing related to confirmation of the validity of a computation model. After starting operation at an autonomous driving level, the computation model determination unit 148 of the mobile device 140 confirms the expiration date of the computation model it owns (step S313). If the confirmation result shows that the remaining period is within a certain period, the computation model request unit 146 requests a computation model with a valid remaining period from the server 120 via the communication unit 144 and the base station device 130 in order to update the computation model (step S314).

[0080] In the server 120, the computation model request receiving unit 122 receives the request for the computation model via the communication unit 121. Subsequently, the computation model managing unit 123 provides the computation model that matches the request to the mobile device 140 via the base station device 130 (step S315).

[0081] In the mobile device 140, the computation model request unit 146 receives the computation model through the communication unit 144 and stores it in the computation model storage unit 149. As a result, the computation model stored in the computation model storage unit 149 is updated to a valid computation model whose remaining period is equal to or longer than a certain period (step S316).

[0082] Next, the computation model request unit 146 transmits information relating to the ownership of the updated computation model to the operation management device 110 from the communication unit 144 via the base station device 130 (step S317).

[0083] In the operation management device 110, the registration request receiving unit 114 receives the information through the communication unit 111, and the autonomous driving level management unit 116 updates the received information by associating it with the mobile device 140 that sent the information (step S318).

[0084] Furthermore, in the mobile device 140, the calculation model request unit 146 transmits information regarding the ownership of the updated calculation model to the operation management device 110, and then the mobile control unit 143 starts mobile control based on the updated calculation model (step S319).

[0085] FIG. 8 is a sequence diagram of another example of processing related to validity confirmation of a computation model.

[0086] In this process, after starting operation at the autonomous driving level, the computation model determination unit 148 of the mobile device 140 checks whether the computation model it owns is within the valid area (step S413). If the check shows that the computation model is outside the valid area, the computation model request unit 146 requests a computation model whose area is valid from the server 120 via the communication unit 144 and the base station device 130 (step S414).

[0087] In the server 120, the computation model request receiving unit 122 receives the request for the computation model through the communication unit 121. Subsequently, the computation model managing unit 123 provides the computation model that matches the request to the mobile device 140 via the base station device 130 (step S415).

[0088] In the mobile device 140, the computation model request unit 146 receives the computation model through the communication unit 144 and stores it in the computation model storage unit 149. As a result, the computation model stored in the computation model storage unit 149 is updated to a computation model for which the area is valid (step S416).

[0089] Next, the computation model request unit 146 transmits information relating to the ownership of the updated computation model to the operation management device 110 from the communication unit 144 via the base station device 130 (step S417).

[0090] In the operation management device 110, the registration request receiving unit 114 receives the information through the communication unit 111, and the autonomous driving level management unit 116 updates the received information by associating it with the mobile device 140 that sent the information (step S418).

[0091] Furthermore, in the mobile device 140, the calculation model request unit 146 transmits information regarding the ownership of the updated calculation model to the operation management device 110, and then the mobile control unit 143 starts mobile control based on the updated calculation model (step S419).

[0092] 9 is a flowchart showing a series of processes related to setting the autonomous driving level by the moving device 140. The operational contents from step S201 to step S203 are the same as those in FIG. 5, and therefore a description thereof will be omitted.

[0093] When the registration request processing unit 145 starts the process of registering the automatic driving level to the operation management device 110 (step S203), the communication unit 144 transmits a registration request for the automatic driving level to the operation management device 110 (step S204).

[0094] Thereafter, the computation model requesting unit 146 acquires a computation model corresponding to the autonomous driving level from the server 120 based on an instruction from the operation management device 110 (step S205). Subsequently, the computation model requesting unit 146 transmits information regarding possession of a valid computation model to the operation management device 110 (step S206).

[0095] Thereafter, when the registration request processing unit 145 receives a notification of registration permission from the operation management device 110 (step S207), the mobility control unit 143 starts mobility control (step S208). Note that even if the autonomous driving level is not 1 or higher in step S202, that is, the autonomous driving level is level 0, the mobility control unit 143 starts mobility control.

[0096] 9 shows an example in which the registration process is initiated when the autonomous driving level is level 1 or higher, but the present invention is not limited to this example. For example, the operation management device 110 may set the mobile device 140 to initiate the registration process when the autonomous driving level is level 3 or higher. That is, in step S202, if the autonomous driving level is level 0, level 1, or level 2, the mobile control unit 143 starts mobile control.

[0097] Fig. 10 is a flowchart showing another example of the process of registering the autonomous driving level. The operation contents from step S301 to step S303 shown in Fig. 10 are the same as those in Fig. 6, so the explanation will be omitted.

[0098] When the communication unit 144 of the mobile device 140 transmits a registration request including the autonomous driving level to the operation management device 110 via the base station device 130 (step S303), the registration request receiving unit 114 of the operation management device 110 receives the registration request via the communication unit 111. Next, the autonomous driving level management unit 116 notifies the compatibility test execution instruction unit 117 of the autonomous driving level indicated in the registration request.

[0099] The compatibility test execution instruction unit 117 determines whether a compatibility test is necessary depending on the autonomous driving level (step S503). For example, the compatibility test execution instruction unit 117 determines that a compatibility test needs to be performed for a mobile device 140 that requests registration of an autonomous driving level of 3 or higher. The compatibility test is performed to verify whether the driving assistance processing unit 142, sensor 141, and mobile control unit 143 that assist autonomous driving have the performance required to achieve the autonomous driving level requested to be registered, or to detect any malfunctions or defects in the driving assistance processing unit 142, sensor 141, and mobile control unit 143.

[0100] When the compatibility test execution instruction unit 117 determines that a compatibility test needs to be executed, it instructs the mobile device 140 via the base station device 130 to execute a compatibility test corresponding to the autonomous driving level for which registration has been requested (step S504).

[0101] In the mobile device 140, the instruction to execute the compatibility test is received by the driving assistance processing unit 142 via the communication unit 144. As the compatibility test, the driving assistance processing unit 142 inputs the data acquired by the sensor 141 into an AI, for example, a neural network model generated by machine learning or deep learning, and calculates the output (step S505). This neural network model may be pre-implemented in the driving assistance processing unit 142, or may be acquired by the communication unit 144 from the driving management device 110 via the base station device 130.

[0102] When the compatibility test is completed, the driving support processing unit 142 transmits information about the executed compatibility test from the communication unit 144 to the driving management device 110 via the base station device 130 (step S506). This information includes, for example, data acquired by the sensor 141 and an output calculated using a neural network model. The data acquired by the sensor 141 is transmitted in association with the identification IDs of the position information sensor 141a, the camera module (including an image sensor) 141b, the LiDAR 141c, and the radar 141d.

[0103] In the operation management device 110, the compatibility test result acquisition unit 118 receives information about the compatibility test performed by the mobile device 140. Next, the compatibility test determination unit 119 verifies the compatibility test (step S507). In the compatibility test verification, for example, the detection accuracy of each sensor corresponding to the autonomous driving level requested for registration is evaluated from data acquired by the sensor 141. In other words, the detection accuracy required for the sensor 141 may differ for each autonomous driving level. In addition, in the compatibility test verification, the operation management device 110 inputs the data acquired by the sensor 141 into a neural network model generated by machine learning or deep learning, and compares the output obtained with the output acquired from the mobile device 140 to evaluate the accuracy of the driving assistance processing unit 142 corresponding to the autonomous driving level requested for registration.

[0104] The compatibility test determination unit 119 may also verify the compatibility test based on whether a specific object on the dynamic map is detected with sufficient accuracy. The camera module 141b of the sensor 141 detects image information. The position information sensor 141a detects information related to the position. The LiDAR 141c and radar 141d detect information related to the distance measurement of an arbitrary object on the dynamic map. The driving assistance processing unit 142 calculates information related to a correspondence relationship with an arbitrary object on the dynamic map and vector information based on the image information, information related to the position, and information related to the distance measurement of the arbitrary object on the dynamic map. The driving assistance processing unit 142 reports information related to the position and information related to the vector of the specific object on the dynamic map instructed by the driving management device 110 to the compatibility test determination unit 119. The compatibility test determination unit 119 verifies whether the reported information related to the vector of the specific object is sufficiently accurate.

[0105] The compatibility test determination unit 119 notifies the registration permission determination unit 115 of the verification result of the compatibility test. The registration permission determination unit 115 determines whether or not to permit registration based on the verification result of the compatibility test (step S508). For example, if the compatibility test determination unit 119 determines that the sensor 141 and the driving assistance processing unit 142 do not have a detection accuracy corresponding to the autonomous driving level for which registration is requested, the registration permission determination unit 115 rejects the registration request from the mobile device 140. On the other hand, if the compatibility test determination unit 119 determines that the sensor 141 and the driving assistance processing unit 142 have a detection accuracy corresponding to the autonomous driving level for which registration is requested, the registration permission determination unit 115 permits the registration request from the mobile device 140.

[0106] Furthermore, if it is determined that the sensor 141 and the driving assistance processing unit 142 do not have the detection accuracy required for the autonomous driving level for which registration is requested, the registration permission determination unit 115 may determine whether the detection accuracy required for the next lower autonomous driving level is available. For example, if the autonomous driving level for which registration was initially requested is level 3, the next lower autonomous driving level is level 2. In this case, if it is determined that the sensor 141 and the driving assistance processing unit 142 have the detection accuracy required for the next lower autonomous driving level, the registration permission determination unit 115 may instruct the mobile device 140 via the base station device 130 to reset the autonomous driving level to the next lower autonomous driving level.

[0107] 11 is a flowchart showing an example of operation management processing by the operation management device 110. The operation management processing by the operation management device 110 will be described below.

[0108] The registration permission determination unit 115 of the operation management device 110 instructs the mobile device 140, which has left or is about to leave a specific area, to change the autonomous driving level setting via the base station device 130 (step S601). Changing the autonomous driving level setting means, for example, changing the setting to an autonomous driving level different from the currently registered autonomous driving level. For example, if the mobile device 140, which is compatible with level 3, is about to leave a specific area where level 3 is permitted, the registration permission determination unit 115 instructs the mobile device 140 to change the setting to one of the autonomous driving levels from level 0 to level 2. Furthermore, if the mobile device 140, which is compatible with level 4, is about to leave a specific area where level 4 is permitted, the registration permission determination unit 115 instructs the mobile device 140 to change the setting to one of the autonomous driving levels from level 0 to level 3. In other words, the registration permission determination unit 115 instructs the mobile device 140 to change the autonomous driving level setting based on the location of the mobile device 140.

[0109] On the other hand, for example, when mobile device 140 compatible with level 3 attempts to enter a specific area where level 3 is permitted from an area other than the specific area where level 3 is permitted, registration permission determination unit 115 instructs the setting to be changed from an autonomous driving level of level 2 or lower to level 3. Also, when mobile device 140 compatible with level 4 attempts to enter a specific area where level 4 is permitted from an area other than the specific area where level 4 is permitted, registration permission determination unit 115 instructs the setting to be changed from an autonomous driving level of level 3 or lower to level 4.

[0110] For example, even within a specific area where level 3 is permitted, a level 3-compatible mobile device 140 does not need to drive at level 3; it simply has the performance to drive at level 3. Therefore, depending on the surrounding environmental conditions and the accuracy of the driving assistance processing unit 142, the registration permission determination unit 115 can instruct the level 3-compatible mobile device 140 to drive at a lower autonomous driving level. In other words, the registration permission determination unit 115 instructs the level 3-compatible mobile device 140 to change the setting to one of the autonomous driving levels from level 0 to level 2. This instruction to dynamically change the setting of the autonomous driving level is important for optimizing the entire autonomous operation system.

[0111] When the autonomous driving level setting unit 147 of the mobile device 140 receives the instruction to change the setting of the autonomous driving level via the communication unit 144 and the registration request processing unit 145, it changes the setting to the instructed autonomous driving level (step S602). Next, the registration request processing unit 145 requests the operation management device 110 to change the registration of the autonomous driving level from the communication unit 144 via the base station device 130 (step S603).

[0112] In the operation management device 110, when the registration request receiving unit 114 receives the request, the autonomous driving level management unit 116 notifies the computation model confirmation unit 112 of the autonomous driving level included in the registration request. The computation model confirmation unit 112 confirms the computation model corresponding to the autonomous driving level and notifies the computation model acquisition instructing unit 113. The computation model acquisition instructing unit 113 instructs the mobile device 140 that sent the registration request, via the base station device 130, to acquire the computation model corresponding to the autonomous driving level included in the registration request (step S604).

[0113] In the mobile device 140, the computation model request unit 146 transmits a request for a computation model through the communication unit 111 (step S605). The request for a computation model is transmitted to the server 120 via the base station device .

[0114] In the server 120, the computation model request receiving unit 122 receives the request for a computation model through the communication unit 121. Subsequently, the computation model management unit 123 provides the computation model indicated in the request (step S606). The provided computation model is transmitted from the communication unit 121 to the requesting mobile device 140 via the base station device 130.

[0115] In the mobile device 140, the computation model request unit 146 receives the computation model through the communication unit 144 and stores it in the computation model storage unit 149. This updates the computation model (step S607). Next, the computation model request unit 146 causes the communication unit 144 to transmit information related to the ownership of the updated computation model (step S608).

[0116] In the operation management device 110, the above information is input to the registration permission determination unit 115 via the communication unit 111 and the computation model confirmation unit 112. The registration permission determination unit 115 determines whether or not to permit the registration change based on the above information (step S609). In step S609, the registration permission determination unit 115 permits the registration change if, for example, the ID identifying the changed computation model matches a pre-registered genuine ID, and denies the registration change if not.

[0117] If the information relating to ownership of the changed computation model satisfies the registration change requirements, the registration permission determination unit 115 notifies the mobile device 140 of registration change permission via the base station device 130 (step S610).

[0118] In the mobile device 140, the registration request processing unit 145 receives the notification of permission to change the registration via the communication unit 144 and confirms permission to change the registration (step S611).

[0119] In the above-described operation management process, the operation management device 110 issues an instruction to change the autonomous driving level setting when the mobile device 140 leaves or enters a specific area, but the timing of the instruction is not limited to this. For example, the sensitivity of the sensor may change due to changes in weather or during specific time periods. Therefore, the operation management device 110 may issue an instruction to change the autonomous driving level setting when triggered by changes in weather or time.

[0120] Furthermore, the autonomous driving level setting unit 147 of the mobile device 140 may request a change in the autonomous driving level setting itself without receiving an instruction to change the autonomous driving level setting from the operation management device 110. For example, in addition to the above-mentioned weather changes and time of day, when fuel or the remaining battery power is low, the autonomous driving level may be lowered (for example, from level 3 to level 2) due to the communication unit 144 being outside the communication area of ​​the base station device 130, or the sensitivity of a specific sensor may be lowered or a specific function may be disabled.

[0121] Furthermore, when an instruction to change the setting to a lower autonomous driving level is received from the operation management device 110 in step S602, the autonomous driving level setting unit 147 may request the operation management device 110 to change the registered autonomous driving level, including information related to the ownership of the current computation model, in S603. For example, if the operation management device 110 determines that the current computation model is valid even at a lower autonomous driving level, it is possible to omit the processes from S604 to S608 and execute the processes from S609 onwards.

[0122] 12 is a flowchart showing an example of a part of the operation management process, which explains the process flow regarding whether or not a compatibility test needs to be performed.

[0123] When the registration request receiving unit 114 receives a registration request including an updated autonomous driving level from the mobile device 140 (step S701), the autonomous driving level management unit 116 determines whether the autonomous driving level included in the registration request is level 5 (step S702).

[0124] If the autonomous driving level is level 5, the compatibility test execution instruction unit 117 instructs the execution of a compatibility test at the time of registration (step S703). Next, the autonomous driving level management unit 116 determines whether the autonomous driving level set for the registered mobile device 140 is level 1 or higher (step S704). Note that even if it is determined in step S702 that the autonomous driving level is not level 5, the processing of step S704 is subsequently executed.

[0125] If the autonomous driving level is not level 1 or higher, that is, if the autonomous driving level is level 0, the processing ends. On the other hand, if the autonomous driving level is 1 or higher, the compatibility test execution instruction unit 117 sets the mobile device 140 to execute a periodic compatibility test (step S705). For example, the execution of a periodic compatibility test is set to detect failures or malfunctions in the driving assistance processing unit 142, sensor 141, and movement control unit 143, which assist autonomous driving. Therefore, the period is set based on statistical information on the history of failures or malfunctions in the driving assistance processing unit 142, sensor 141, and movement control unit 143.

[0126] Furthermore, when the driving assistance processing unit 142, the sensor 141, and the movement control unit 143 require periodic calibration, periodic execution of the compatibility test may be set to determine the timing of the calibration. Furthermore, the compatibility test period may be set based on the frequency of maintenance required for the moving device 140, the driving assistance processing unit 142, the sensor 141, or the movement control unit 143 as stipulated by law. Furthermore, the compatibility test period may be variable depending on the area, time of day, and weather. For example, in areas where accidents occur frequently based on accident history, the compatibility test may be set to be performed at shorter intervals. Furthermore, qualitatively, the compatibility test may be set to be performed at nighttime rather than during the day, or in rainy or snowy weather rather than in sunny weather.

[0127] Next, the registration permission determination unit 115 determines whether the mobile device 140 has entered a specific area (step S706). If the mobile device 140 has not entered a specific area, the processing of step S706 is periodically executed. On the other hand, if the mobile device 140 has entered a specific area, the autonomous driving level management unit 116 determines whether the autonomous driving level set by the registered mobile device 140 is level 3 or higher (step S707).

[0128] If the autonomous driving level set by the registered mobile device 140 is not level 3 or higher, the process ends. On the other hand, if the autonomous driving level set by the registered mobile device 140 is level 3 or higher, the compatibility test execution instruction unit 117 instructs the mobile device 140 to execute a compatibility test (step S708). Thereafter, the process of step S706 is executed periodically, and it is determined whether the mobile device 140 has entered a specific area.

[0129] 12, in S701, the compatibility test execution instructing unit 117 instructs the execution of a compatibility test at the time of registration when the autonomous driving level is level 5. However, this is not limiting. For example, if the area where the registration request is received is a specific location where the autonomous driving level is level 4 and the system can perform all operations, the compatibility test execution instructing unit 117 may instruct the execution of a compatibility test when the autonomous driving level is level 4 or higher. Furthermore, if the area where the registration request is received is a specific location where the autonomous driving level is level 3 and the system can perform all operations, the compatibility test execution instructing unit 117 may instruct the execution of a compatibility test when the autonomous driving level is level 3 or higher.

[0130] 13 is a flowchart showing another example of a part of the operation control process, in which the process flow relating to determining the compatibility test results is explained.

[0131] When the compatibility test execution instruction unit 117 instructs the mobile device 140 to execute a compatibility test (step S801), the compatibility test result acquisition unit 118 then acquires the results of the compatibility test from the mobile device 140 (step S802).

[0132] Next, the compatibility test determination unit 119 determines whether the mobile device 140 passes the compatibility test (step S803). If the mobile device 140 passes the compatibility test, the registration permission determination unit 115 permits the mobile device 140 to operate at the registered autonomous driving level (step S804).

[0133] On the other hand, if the mobile device 140 fails the compatibility test, the registration permission determination unit 115 determines whether the mobile device 140 is capable of driving at level 0 (step S805). For example, a mobile device 140 designed for level 5 autonomous driving may not be designed to involve human intervention in driving.

[0134] If the mobile device 140 is capable of operating at level 0, the registration permission determination unit 115 instructs the mobile device 140 to operate at level 0 (step S806).

[0135] On the other hand, if the mobile device 140 is not capable of driving at level 0, the registration permission determination unit 115 instructs the mobile device 140 to stop driving (step S807). For example, unlike level 3 autonomous driving, level 4 autonomous driving does not require driver operation in an emergency, and is therefore expected to not have the performance required for human driving. In other words, a mobile device 140 that is compatible with level 4 autonomous driving may not be capable of driving at level 0. Therefore, the operation of the mobile device 140 is stopped. Here, if the vehicle traveling ahead of the mobile device 140 is a vehicle that supports platooning via V2X communication and the mobile device 140 has platooning capability, the registration permission determination unit 115 may instruct the mobile device 140 to drive by platooning depending on the vehicle distance from the vehicle traveling ahead.

[0136] In addition, in accordance with the instruction to stop operation, operation support may be provided to stop the vehicle in a location that does not interfere with the operation of other vehicles. For example, this operation support is remote driving operation by the operation management device 110. Furthermore, the operation management process shown in Fig. 13 may be performed in an autonomous and decentralized manner by the driving support processing unit 142 installed in the mobile device 140, instead of by the operation management device 110.

[0137] Furthermore, if the mobile device 140 does not pass the compatibility test in step S803, the compatibility test execution instruction unit 117 may instruct the execution of a compatibility test at the next lower level (for example, from level 5 to level 4) before performing the processing of step S805. In this case, the compatibility test determination unit 119 determines the compatibility test result, thereby making it possible to support autonomous driving at a level higher than level 0 as much as possible.

[0138] According to the embodiment described above, the autonomous driving level of each mobile device 140 is registered in advance in the driving management device 110 before the device starts driving. Furthermore, the calculation model differs for each autonomous driving level, and each mobile device 140 cannot start driving unless it has an appropriate calculation model. In this way, the autonomous driving level and calculation model of each mobile device 140 are collectively managed by the driving management device 110, making it possible to provide driving assistance according to different autonomous driving levels.

[0139] In this specification, the mobile device 140 is described as a vehicle traveling on a road, but the scope of the technology disclosed in this specification is not limited to this. The mobile device 140 can be an aircraft that flies or travels on the ground or in the air, or an unmanned aerial vehicle (UAV) such as a drone. Here, the aircraft can include small aircraft also known as flying cars. When the mobile device 140 is such an aircraft, a spatial area that also includes information on the height direction is applied as the area. Furthermore, the definitions and classifications of the autonomous driving levels are merely examples, and more detailed classifications are possible, for example.

[0140] Furthermore, the processes executed in the above-described operation management device 110, server 120, and mobile device 140 can be realized by software (programs) executed by a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), etc. When realized by software, programs that realize at least some of the functions of the operation management device 110, server 120, and mobile device 140 may be stored in a recording medium such as a semiconductor memory, a hard disk, a flexible disk, or a CD-ROM, and may be read and executed by a computer. Furthermore, programs that realize at least some of the functions of the operation management device 110, server 120, and mobile device 140 may be dynamically downloaded from the base station device 130 or an RSU and stored in the above-mentioned recording medium.

[0141] Note that instead of executing all of the processes of the operation management device 110, the server 120, and the mobile device 140 by software, some of the processes may be executed by hardware such as a dedicated circuit. Furthermore, the operation management device 110 or the server 120 may be distributed and implemented on multiple devices, for example, cloud servers. Here, the implementation form may include dynamic implementation using virtualization. Furthermore, the cloud server may include the concept of an edge server.

[0142] In the above explanation, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0143] The present technology can be configured as follows: (1) A system including one or more mobile devices capable of setting an autonomous driving level, and an external network device capable of communicating with the mobile devices; The external network device a communication device for communicating with the mobile device; a calculation model determination device that determines a calculation model corresponding to the autonomous driving level and provides the calculation model to the mobile device via the communication device; a registration determination device that determines whether or not the autonomous driving level can be registered based on information about ownership of the computation model, and notifies the mobile device of registration permission via the communication device; The moving device is a calculation model request unit that requests the calculation model from the calculation model determination device and, when the requested calculation model is provided, causes the information to be transmitted to the registration determination device; An information processing system having a mobility control unit that, upon receiving the notification from the registration determination device, starts mobility control based on the autonomous driving level for which registration is permitted. (2) The information processing system according to (1), wherein the communication device is a base station device that wirelessly communicates with the mobile device. (3) The information processing system according to (2), wherein the computation model determination device is a server having a communication unit that communicates with the mobile device via the base station device. (4) The information processing system according to (2) or (3), wherein the registration determination device is an operation management device having a communication unit that communicates with the mobile device via the base station device. (5) An information processing system described in any one of (1) to (4), wherein the mobile device further has a calculation model determination unit that checks the expiration date or valid area of ​​the first calculation model it owns, and the calculation model request unit requests a second calculation model with a valid expiration date or area depending on the confirmation result of the calculation model determination unit. (6) The information processing system according to (5), wherein the mobile device further has a calculation model storage unit that stores the calculation model, and the first calculation model is updated to the second calculation model within the calculation model storage unit. (7) An information processing system described in any one of (1) to (4), wherein the registration determination device further has a compatibility test execution instruction unit that determines whether a compatibility test is necessary based on the autonomous driving level and instructs the mobile device to perform a compatibility test based on the determination result. (8) The registration determination device further includes a registration permission determination unit that instructs a change in the setting of the autonomous driving level based on the location of the mobile device, The information processing system according to any one of (1) to (4), wherein the mobile device further has an autonomous driving level setting unit that changes the setting of the autonomous driving level based on an instruction from the registration permission determination unit. (9) The information processing system described in (8), wherein the calculation model request unit requests a calculation model corresponding to the changed autonomous driving level from the calculation model determination device, and updates the first calculation model it possesses to the second calculation model obtained from the calculation model determination device. (10) The information processing system described in (7), wherein the compatibility test execution instruction unit instructs the mobile device to periodically execute the compatibility test when the autonomous driving level is a predetermined level or higher. (11) a calculation model request unit that requests and acquires a calculation model corresponding to an autonomous driving level set in the mobile device; An information processing device comprising: a mobility control unit that, when registration of the autonomous driving level is permitted based on information regarding ownership of the computational model, starts mobility control of the mobile device based on the autonomous driving level permitted for registration. (12) A registration request processing unit is further provided that requests an external network device capable of communicating with the mobile device to register the autonomous driving level, The information processing device according to (11), wherein the computation model request unit requests the computation model from the external network device and acquires the computation model from the external network device. (13) A driving assistance processing unit controls the movement control unit based on a result of calculating detection data of a sensor provided in the movement device using the calculation model, The information processing device according to (12), wherein the calculation model is determined by the external network device based on at least one of the detection content of the sensor, the type of the mobile device, and the performance of the driving assistance processing unit. (14) Further comprising a calculation model determination unit for checking the expiration date or valid area of ​​the first calculation model owned; The information processing device according to any one of (11) to (13), wherein the computation model request unit requests a second computation model with a valid time limit or area according to a confirmation result of the computation model determination unit. (15) The information processing device according to (14), further comprising a computation model storage unit for storing the computation model, wherein the first computation model is updated to the second computation model in the computation model storage unit. (16) One or more mobile devices capable of setting an autonomous driving level request a calculation model corresponding to the autonomous driving level from an external network device; The external network device determines the computation model and provides it to the mobile device; the mobile device transmits information regarding ownership of the computational model to the external network device; The external network device determines whether or not the autonomous driving level can be registered based on the information, and notifies the mobile device of permission for registration; An information processing method in which, upon receiving the notification, the mobile device starts mobility control based on the autonomous driving level for which registration is permitted.

[0144] The above-described embodiment shows an example for realizing the present disclosure, and the present disclosure can be implemented in various other forms. For example, various modifications, substitutions, omissions, or combinations thereof are possible without departing from the spirit of the present disclosure. Such modifications, substitutions, omissions, etc. are also included within the scope of the present disclosure, as well as the scope of the inventions described in the claims and their equivalents. [Explanation of symbols]

[0145] 100: Driving assistance system 110: Operation management device 117: Conformance test execution instruction unit 115: Registration permission determination unit 120: Server 130:Base station equipment 140: Mobile device 141: Sensor 142: Driving assistance processing unit 143: Movement control unit 145: Registration request processing unit 146: Calculation model request unit 147: Autonomous driving level setting unit 148: Operation model determination unit 149: Calculation model storage unit

Claims

1. The system includes one or more mobile devices capable of setting an autonomous driving level, and an external network device capable of communicating with the mobile devices, The external network device a communication device for communicating with the mobile device; a computation model determination device that provides the mobile device with a computation model, which is a neural network model determined according to the autonomous driving level set in the mobile device prior to driving, from among the neural network models that differ for each autonomous driving level, via the communication device; a registration determination device that determines whether or not the autonomous driving level can be registered based on information about ownership of the computation model, and notifies the mobile device of registration permission via the communication device; The moving device is a calculation model requesting unit that requests the calculation model from the calculation model determination device and, when the requested calculation model is provided, transmits the information to the registration determination device; An information processing system having a mobility control unit that, upon receiving the notification from the registration determination device, starts mobility control based on the autonomous driving level for which registration is permitted.

2. The information processing system according to claim 1 , wherein the communication device is a base station device that wirelessly communicates with the mobile device.

3. 3. The information processing system according to claim 2, wherein the computation model determination device is a server having a communication unit that communicates with the mobile device via the base station device.

4. The information processing system according to claim 2 , wherein the registration determination device is an operation management device having a communication unit that communicates with the mobile device via the base station device.

5. The information processing system of claim 1, wherein the mobile device further has a calculation model determination unit that checks whether the deadline or area of ​​a first calculation model owned by the mobile device itself is valid after the start of operation, and the calculation model request unit requests a second calculation model in which the deadline or area is valid from the calculation model determination device if the deadline or area of ​​the first calculation model is not valid based on the confirmation result of the calculation model determination unit.

6. The information processing system according to claim 5 , wherein the mobile device further comprises a computational model storage unit that stores the computational model, and the first computational model is updated to the second computational model in the computational model storage unit.

7. The information processing system of claim 1, wherein the registration determination device further has a compatibility test execution instruction unit that determines whether a compatibility test is necessary based on the autonomous driving level and instructs the mobile device to perform a compatibility test based on the determination result.

8. the registration determination device further includes a registration permission determination unit that instructs a change in the setting of the autonomous driving level based on a location of the mobile device; The information processing system according to claim 1 , wherein the mobile device further comprises an autonomous driving level setting unit that changes a setting of the autonomous driving level based on an instruction from the registration permission determination unit.

9. 9. The information processing system according to claim 8, wherein the calculation model request unit requests the calculation model determination device for a calculation model corresponding to the autonomous driving level whose setting has been changed by the autonomous driving level setting unit, and obtains a first calculation model owned by the mobile device from the calculation model determination device before making the request to the calculation model determination device, and updates the first calculation model to a second calculation model corresponding to the autonomous driving level after the setting has been changed by the autonomous driving level setting unit.

10. The information processing system according to claim 7 , wherein the compatibility test execution instruction unit instructs the mobile device to periodically execute the compatibility test when the autonomous driving level is equal to or higher than a predetermined level.

11. A computational model request unit that requests an external network device that can communicate with the mobile device to obtain a computational model, which is a neural network model corresponding to the autonomous driving level set in the mobile device prior to driving, from the external network device, from among neural network models that differ for each autonomous driving level; An information processing device comprising: a mobility control unit that, when registration of the autonomous driving level is permitted based on information regarding ownership of the computational model, starts mobility control of the mobile device based on the autonomous driving level permitted for registration.

12. a registration request processing unit that requests the external network device to register the autonomous driving level; The information processing apparatus according to claim 11 , wherein the computation model requesting unit requests the external network device for the computation model and acquires the computation model from the external network device.

13. a driving assistance processing unit that controls the movement control unit based on a result of calculating detection data from a sensor provided in the movement device using the calculation model; The information processing device according to claim 12 , wherein the calculation model is determined by the external network device based on at least one of the detection content of the sensor, the type of the mobile device, and the performance of the driving assistance processing unit.

14. The method further comprises a calculation model determination unit for determining whether or not the term or area of ​​a first calculation model owned by the mobile device after the start of operation is valid, among the calculation models; The information processing device according to claim 11, wherein the calculation model request unit requests a second calculation model in which the deadline or the area is valid from the external network device when the confirmation result of the calculation model determination unit indicates that the deadline or the area of ​​the first calculation model is not valid.

15. The information processing apparatus according to claim 14 , further comprising a computation model storage unit that stores the computation model, wherein the first computation model is updated to the second computation model in the computation model storage unit.

16. One or more mobile devices capable of setting an autonomous driving level request an external network device for a computation model that is a neural network model corresponding to the autonomous driving level set in the mobile device prior to driving, from among neural network models that differ for each autonomous driving level; the external network device determines the computation model in response to a request from the mobile device and provides the computation model to the mobile device; the mobile device transmits information regarding ownership of the computational model to the external network device; The external network device determines whether or not the autonomous driving level can be registered based on the information, and notifies the mobile device of permission for registration; An information processing method in which, upon receiving the notification, the mobile device starts mobility control based on the autonomous driving level for which registration is permitted.

Images