Information processing method, information processing device, and server device

Abstract

This disclosure relates to an information processing method, an information processing apparatus, and a server apparatus. A technique is provided that enables efficient joint learning between a server apparatus and multiple information processing devices respectively mounted on a mobile body. One aspect of the information processing method includes the following actions: each information processing device uses local data to perform machine learning on a global model, thereby generating a local model; each information processing device calculates an evaluation value based on an evaluation metric; each information processing device exchanges the calculated evaluation values ​​via near-field communication; each information processing device determines whether its evaluation is high based on a comparison of its own evaluation value with the evaluation values ​​of other information processing devices; and at least one information processing device determined to have a high evaluation sends local model information representing the generated local model to the server apparatus.

Description

Technical Field

[0001] This disclosure relates to a technique for joint learning of machine learning models implemented between a server device and multiple information processing devices respectively mounted on a mobile body such as a vehicle. Background Technology

[0002] Traditional machine learning methods involve aggregating various training data and using the aggregated data to train a machine learning model. However, this method suffers from drawbacks such as the cost of aggregating training data (e.g., time and communication costs) and increased computational load on the computer performing the machine learning.

[0003] Therefore, joint learning is proposed as a solution to these problems. Joint learning is a method of implementing machine learning decentralizedly without aggregating learning data. In the case of joint learning between a server device and multiple client devices, each client device collects local data independently. Each client device generates a local model by performing machine learning on the global model using the collected local data. Next, each client device sends its generated local model to the server device. The server device generates a new global model by integrating the local models received from each client device.

[0004] According to this joint learning method, a fully trained machine learning model (a new global model) reflecting the local data obtained from each client device can be generated without uploading local data to a server device. In other words, the cost of aggregating local data to a server device can be saved. Furthermore, the computational load of machine learning can be distributed among the client devices. Therefore, compared to the previous methods, a fully trained model based on a large amount of data can be generated while aiming to reduce costs and avoid concentrating computational load.

[0005] Patent Document 1 provides an example of a joint learning process between a server device and a client device. Non-Patent Document 1 proposes a method for joint learning between a central server and individual vehicle clients, which, based on two-dimensional contract theory, facilitates interaction between the central server and individual vehicle clients to select vehicle clients to participate in the joint learning process.

[0006] Existing technical documents

[0007] Patent documents

[0008] Patent Document 1: U.S. Patent Application Publication No. 2020 / 0242514

[0009] Non-patent literature

[0010] Non-Patent Document 1: Dongdong Ye, Rong Yu, Miao Pan, Zhu Han, "Federated Learning in Vehicular Edge Computing: A Selective Model Aggregation Approach", [online], [retrieved November 30, Reiwa 3], Internet <URL:https: / / ieeexplore.ieee.org / document / <8964354> Summary of the Invention

[0011] Problems to be Solved by the Invention

[0012] An object of the present disclosure is to provide a technique capable of efficiently performing federated learning between a server device and a plurality of information processing devices respectively mounted on mobile bodies. <​​​​​​The information processing apparatus of the second technical solution of this disclosure is mounted on a mobile body, wherein the information processing apparatus includes a control unit configured to perform the following actions: collecting local data; performing machine learning on a global model using the collected local data to generate a local model; calculating an evaluation value based on an evaluation index; sending the calculated evaluation value to other information processing apparatuses mounted on one or more other mobile bodies located nearby via near-field communication; receiving, via near-field communication, the evaluation value calculated by the other information processing apparatuses based on the evaluation index from the other information processing apparatuses mounted on the one or more other mobile bodies; determining whether the evaluation is high by comparing the calculated evaluation value with the evaluation value received from the other information processing apparatuses; and, if the evaluation is determined to be high, sending local model information representing the generated local model to a server device.

[0016] The server device of the third technical solution of this disclosure is configured to communicate with each of a plurality of information processing devices, the plurality of information processing devices being configured to collect local data respectively and use the collected local data to perform machine learning on a global model, thereby generating local models respectively, the plurality of information processing devices being mounted on different mobile bodies, wherein the server device includes a control unit configured to perform the following actions: receiving local model information representing a local model from at least one of the plurality of information processing devices; and generating a new global model by integrating the local model represented by the local model information received from the at least one information processing device. The plurality of information processing devices are further configured to: calculate an evaluation value based on an evaluation index; transmit the calculated evaluation value via near-field communication to other information processing devices mounted on one or more other mobile devices located near their own mobile device; receive, via the near-field communication, the evaluation value calculated by the other information processing devices based on the evaluation index from the other information processing devices mounted on the one or more other mobile devices; and determine whether the evaluation is high by comparing the calculated evaluation value with the evaluation value received from the other information processing devices. The at least one information processing device is the one determined to have a high evaluation as a result of comparing the evaluation values.

[0017] The effects of the invention

[0018] According to this disclosure, joint learning can be efficiently implemented between a server device and multiple information processing devices respectively mounted on a mobile body. Attached Figure Description

[0019] Figure 1 This is an illustrative example of a scenario in which the present disclosure is applied.

[0020] Figure 2 An example of the hardware structure of a server device in an embodiment is shown schematically.

[0021] Figure 3 This schematically illustrates an example of the hardware structure of the vehicle-mounted device in an embodiment.

[0022] Figure 4 This schematically illustrates an example of the software structure of a server device in an implementation method.

[0023] Figure 5 This schematically illustrates an example of the software structure of an onboard device in an embodiment.

[0024] Figure 6 This is an example illustrating the process of machine learning in an in-vehicle device.

[0025] Figure 7 This is a sequence diagram illustrating an example of the joint learning process performed between the server device and each vehicle-mounted device in the implementation embodiment.

[0026] Figure 8 This is a flowchart illustrating an example of the processing steps related to the generation of a local model and the selection of autonomous distribution by the vehicle-mounted device of the embodiment.

[0027] Explanation of reference numerals in the attached figures

[0028] 1…Server device,

[0029] 11…Control unit, 12…Storage unit, 13…Communication module,

[0030] 14…Input device, 15…Output device, 16…Driver,

[0031] 81…program, 91…storage medium,

[0032] 111…Model Distribution Department, 112…Indicator Distribution Department,

[0033] 113…Model Receiving Unit, 114…Integration Processing Unit,

[0034] 2…Onboard device (information processing device), V…Vehicle (moving body),

[0035] 21…Control unit, 22…Storage unit, 23…Communication module,

[0036] 24…Input device, 25…Output device, 26…Driver,

[0037] 27…Near Field Communication Module,

[0038] 82… program, 92… storage medium,

[0039] 211… Model Acquisition Department, 212… Indicator Acquisition Department,

[0040] 213…Data Collection Department, 214…Machine Learning Department,

[0041] 215… Evaluation value calculation department, 216… Evaluation value sending department,

[0042] 217…Evaluation value receiving department, 218…Evaluation department,

[0043] 219…Model Sending Department,

[0044] 3… Local data,

[0045] 40…Global model, 400…Global model information

[0046] 45…(New) Global Model

[0047] 50… local model, 500… local model information,

[0048] 6… rating value

[0049] 70… evaluation indicators, 700… evaluation indicator information Detailed Implementation

[0050] All participating client devices can participate in joint learning, but if all client devices participate, the efficiency of updating the machine learning model (generating a new global model) deteriorates. Furthermore, it's not limited to generating a well-trained model with high accuracy. Therefore, sometimes it's necessary to select (adjust the number of) client devices participating in joint learning.

[0051] As a simple method of selection, one could enumerate client devices randomly selected from multiple client devices to participate in joint learning. However, in this method, the server device selects client devices to participate in joint learning without knowing the attributes of the client devices (e.g., the quality of available local data, computing power, communication capabilities, etc.). Therefore, it is not limited to obtaining a local model suitable for model updates from the selected client devices. Consequently, model updates become inefficient and may lead to increased communication overhead.

[0052] One approach to address this issue is to have the server device investigate the attributes (such as status, current condition, etc.) of each client device to determine which client device can generate a local model suitable for model updates. However, this method involves each client device reporting its own attributes to the server device, potentially raising privacy concerns. Furthermore, the selection process is centralized, generating data communication between the server device and the client devices, which could degrade the efficiency of joint learning.

[0053] In the method proposed in Non-Patent Literature 1, the asymmetry of information between the server device and each client device (vehicle client) can be ensured by employing two-dimensional contract theory. That is, even if each client device does not report its attributes to the server device as is, the server device can still select client devices to participate in joint learning based on the attributes of the client devices. Therefore, the aforementioned privacy issue can be resolved. However, in this method, the selection process is also centralized, and complex data communication occurs between the server device and each client device to address the problems of two-dimensional contract theory. Therefore, it may lead to a deterioration in the efficiency of joint learning. Thus, the efficiency of joint learning remains an issue in previous methods.

[0054] In contrast, the information processing method of the first technical solution of this disclosure includes: each of a plurality of information processing devices mounted on different mobile bodies collects local data; each of the plurality of information processing devices uses the collected local data to perform machine learning on a global model, thereby generating a local model; each of the plurality of information processing devices calculates an evaluation value based on an evaluation index; each of the plurality of information processing devices sends the calculated evaluation value to other information processing devices mounted on one or more other mobile bodies located near its own mobile body via near-field communication; each of the plurality of information processing devices receives the evaluation value calculated by other information processing devices from other information processing devices mounted on one or more other mobile bodies via near-field communication; each of the plurality of information processing devices determines whether the evaluation is high by comparing the calculated evaluation value with the evaluation value received from other information processing devices; and at least one information processing device that determines the evaluation is high sends local model information representing the generated local model to a server device.

[0055] It is highly likely that multiple information processing devices mounted on mobile bodies moving within geographically similar areas will collect similar local data. Therefore, the capabilities of the local models generated by each of these information processing devices are also similar, and it is highly likely that it will be sufficient to reflect the local model obtained from at least one information processing device in the update of the global model. In the information processing method of the first technical solution of this disclosure, the selection process is implemented by exchanging evaluation values ​​between the information processing devices via near-field communication. According to this near-field communication-based selection process, information processing devices participating in joint learning can be selected from multiple information processing devices mounted on mobile bodies existing within geographically close areas, and the local models generated by the selected information processing devices can be reflected in the update of the global model. Furthermore, it is possible to autonomously and decentralizedly select information processing devices to participate in joint learning without relying on a server device (i.e., to determine whether each information processing device should participate in joint learning). Therefore, according to the information processing method of the first technical solution of this disclosure, it is possible to achieve high efficiency in joint learning implemented between a server device and multiple information processing devices mounted on a mobile body.

[0056] Hereinafter, an embodiment of one aspect of the present disclosure (hereinafter also referred to as "this embodiment") will be described based on the accompanying drawings. However, the embodiment described below is merely an example of the present disclosure in all respects. Various modifications or variations can be made without departing from the scope of the present disclosure. When implementing the present disclosure, specific structures corresponding to the embodiments may also be appropriately adopted. It should be noted that the data appearing in this embodiment is described using natural language, and more specifically, is specified using pseudo-language, commands, parameters, machine language, etc., that can be recognized by a computer.

[0057] [1 Application Example]

[0058] Figure 1 This illustration shows an example of a scenario in which the present disclosure is applied. The system of this embodiment consists of a server device 1 and multiple vehicle-mounted devices 2.

[0059] The server device 1 in this embodiment is one or more computers configured to acquire local models 50 from vehicle-mounted devices 2 and integrate the acquired local models 50. The vehicle-mounted device 2 in this embodiment is one or more computers configured to collect local data 3, use the collected local data 3 to perform machine learning on a global model 40, thereby generating a local model 50. Multiple vehicle-mounted devices 2 are respectively mounted on different vehicles V. Figure 1In this example, vehicle-mounted device 2A is mounted on vehicle VA, and vehicle-mounted device 2B is mounted on vehicle VB (in the following description, reference numerals A, B, etc., are used to distinguish them; reference numerals are omitted unless otherwise specified). Vehicle V is, for example, an automobile. In this embodiment, vehicle V is an example of a mobile body, and vehicle-mounted device 2 is an example of an information processing device mounted on a mobile body.

[0060] In this embodiment, each vehicle-mounted device 2 collects local data 3. Each vehicle-mounted device 2 uses the collected local data 3 to perform machine learning on the global model 40, thereby generating a local model 50. Performing machine learning using local data 3 corresponds to localization for this device. The global model 40 is the machine learning model before localization using the local data 3. The local model 50 is the localized, trained machine learning model generated by updating the global model 40 using the local data 3.

[0061] Each on-board device 2 calculates an evaluation value 6 based on evaluation index 70. Each on-board device 2 sends the calculated evaluation value 6 to other on-board devices 2 of one or more other vehicles V located near its own vehicle V via near-field communication. Each on-board device 2 receives the evaluation value 6 calculated by those other on-board devices 2 of one or more other vehicles V located near its own vehicle V via near-field communication. Each on-board device 2 determines whether the evaluation is high by comparing its calculated evaluation value 6 with the evaluation values ​​6 received from the other on-board devices 2 of the one or more other vehicles V. At least one on-board device 2 that is determined to have a high evaluation sends local model information 500 representing the generated local model 50 to the server device 1.

[0062] exist Figure 1 In this example, assume that vehicles VA and VB exist within the range of vehicle-mounted devices 2A and 2B, which can send and receive data to each other via near-field communication. In this example, vehicle-mounted device 2A collects local data 3A and uses the collected local data 3A to generate a local model 50A. Vehicle-mounted device 2B collects local data 3B and uses the collected local data 3B to generate a local model 50B.

[0063] Vehicle-mounted device 2A calculates an evaluation value 6A based on evaluation index 70 and transmits the calculated evaluation value 6A to other vehicle-mounted devices mounted in other vehicles via near-field communication. Vehicle-mounted device 2B obtains its own evaluation value 6A by receiving this evaluation value. Similarly, vehicle-mounted device 2B calculates an evaluation value 6B based on evaluation index 70 and transmits the calculated evaluation value 6B to other vehicle-mounted devices via near-field communication. Vehicle-mounted device 2A obtains its own evaluation value 6B by receiving this evaluation value.

[0064] Each on-board unit (2A, 2B) compares its own evaluation value (6A, 6B) with the evaluation values ​​of other units (6B, 6A) to determine whether the evaluation is high. Figure 1 In the example, suppose that vehicle-mounted device 2A determines its own evaluation is high while vehicle-mounted device 2B determines its own evaluation is low. In this scenario, vehicle-mounted device 2A sends its local model information 500A, representing its generated local model 50A, to server device 1. It should be noted that... Figure 1 The scenario shown is an example, and this disclosure is not limited to this example. The number of vehicle-mounted devices 2 that can communicate with server device 1, the number of other vehicle-mounted devices 2 that exist within the range of data transmission and reception based on near-field communication relative to the target vehicle-mounted device 2 (i.e., exchanging evaluation value 6 with the target vehicle-mounted device 2 through near-field communication), and the status of vehicle-mounted devices 2 that are judged to have high evaluation can be appropriately determined according to the specific implementation scenario.

[0065] Server device 1 receives local model information 500 representing a local model 50 from at least one of a plurality of vehicle-mounted devices 2. Server device 1 integrates the local models 50 represented by the local model information 500 received from at least one vehicle-mounted device 2. As a result, server device 1 generates a new global model 45. The new global model 45 is a trained machine learning model generated by integrating the local models 50 obtained from at least one vehicle-mounted device 2.

[0066] As described above, in this embodiment, the selection process for choosing the on-board devices 2 (providing the local model 50) to participate in joint learning is implemented by exchanging evaluation values ​​6 between the on-board devices 2 based on near-field communication. According to this near-field communication-based selection process, on-board devices 2 participating in joint learning can be selected from multiple on-board devices 2 mounted on vehicles V located in geographically close proximity, and the local model 50 generated by the selected on-board devices 2 is reflected in the update of the global model 40. Furthermore, on-board devices 2 participating in joint learning can be selected autonomously and decentralizedly without relying on the server device 1. Therefore, according to this embodiment, the efficiency of joint learning implemented between the server device 1 and the on-board devices 2 can be improved. Moreover, since the data exchanged between the on-board devices 2 is only the evaluation values ​​6, the privacy issue can be resolved.

[0067] It should be noted that, in this embodiment, at least one of the global model 40 and the evaluation metric 70 can be provided from the server device 1 to each vehicle device 2. The number of vehicle devices 2 that may participate in joint learning is not particularly limited and can be appropriately determined according to the implementation method. The server device 1 can provide the new global model 45 generated by integrating the local model 50 to each vehicle device 2. Then, the new global model 45 can be processed as the global model 40, and the generation and integration of the local model 50 (as well as the generation of the new global model) can be repeated. The system of this embodiment can also generate a global model that can perform inference tasks with good accuracy by repeatedly executing the generation and integration process of the local model 50.

[0068] The global model (40, 45) and the local model 50 are configured to perform computational processing to solve inference tasks related to the moving body (vehicle V in this embodiment) through a machine learning model. The machine learning model has one or more computational parameters for performing computational processing to solve the inference tasks. The machine learning model can also be configured to solve multiple inference tasks. The machine learning model can be, for example, composed of neural networks, regression models, support vector machines, etc.

[0069] The reasoning task can be any task related to the moving body and is not particularly limited; it can be appropriately selected depending on the implementation method. For example, each moving body can be equipped with sensors, and the reasoning task can be to derive a certain reasoning result based on the sensing data obtained from the sensors. Sensors can be, for example, image sensors (cameras, etc.), proximity sensors, LiDAR (light detection and ranging) sensors, Radar (radio detection and ranging), etc. As a specific example, the sensor can be an image sensor such as a camera, and the reasoning task can be to detect objects reflected in the image obtained by the image sensor (e.g., pedestrians, traffic lights, median strips, obstacles, etc.). In this case, detecting objects can include segmenting and identifying objects. Furthermore, the sensor can be an in-vehicle sensor such as an image sensor, proximity sensor, LiDAR sensor, Radar, etc., and the reasoning task can be to determine the control content of the moving body based on the sensing data (e.g., braking the vehicle V, locking the doors, determining the content of autonomous driving, etc.). When the reasoning task derives a reasoning result from sensing data, the collection of local data 3 can use sensors.

[0070] Evaluation metric 70 can be appropriately defined by selecting vehicle-mounted devices 2 that generate local models 50 suitable for updating global model 40 (i.e., whose performance is improved by integrating the newly generated global model 45). The quality of local data 3 affects the accuracy of the newly generated global model 45. Furthermore, the higher the computing power of vehicle-mounted devices 2, the faster they can generate local models 50, thus increasing the probability of successful upload of local models 50 within the timeout period. Additionally, it is possible to shorten the time spent aggregating local models 50 on server device 1. Similarly, the higher the communication capability of vehicle-mounted devices 2, the shorter the time spent aggregating local models 50 on server device 1. Therefore, in this embodiment, evaluation metric 70 can be configured to include at least one of a first metric for evaluating the quality of local data 3, a second metric for evaluating the computing power of each vehicle-mounted device 2, and a third metric for evaluating the communication capability of each vehicle-mounted device 2. Furthermore, evaluation metric 70 can be expressed as a function. In this case, each vehicle-mounted device 2 can calculate an evaluation value by performing arithmetic operations on the function representing evaluation metric 70.

[0071] The first metric can be appropriately defined as follows: the more beneficial the local data 3 is to the performance improvement of the global model 40, the higher the evaluation; the less beneficial it is to the performance improvement, the lower the evaluation. The first metric can be represented by a function (f1). The global model 40 can be used to evaluate whether the local data 3 is beneficial. As an example, the objective function used in the machine learning of the global model 40 can also be used to evaluate the beneficialness of the local data 3. The objective function can be replaced by a cost function, loss function, or error function. In the case where the optimization problem in machine learning is constituted by minimizing the value of the objective function, the lower the inference accuracy of the global model 40 for the local data 3, the higher the value calculated by the objective function. Lower inference accuracy can correspond to the local data 3 containing many samples that are not adequately trained for the global model 40. By reflecting the local model 50 trained on such samples into the joint learning, there is a high probability of obtaining a new global model 45 with good performance. Therefore, when the optimization problem in machine learning is constituted by minimizing the value of the objective function, the first metric (f1) can be defined as follows: the larger the error calculated by providing local data 3 to the global model 40 and using the objective function, the higher the evaluation; the smaller the error, the lower the evaluation. Since the evaluation value directly corresponds to the overall evaluation, the objective function can be simply used directly as the first metric (function f1). It should be noted that, in one example, the objective function can be the same across all vehicle-mounted devices 2. In another example, the objective function can also differ from that of other vehicle-mounted devices 2 in at least some of them. Furthermore, when the number of samples of local data 3 collected by each vehicle-mounted device 2 differs, operations such as standardization and sample size adjustment can be performed to facilitate appropriate comparisons.

[0072] The second metric can be appropriately defined as follows: higher computing power results in a higher evaluation, and lower computing power results in a lower evaluation. This second metric can be expressed as a function (f²). Computing power can be calculated based on the performance of computing resources such as processor resources and memory resources. Processor resources can include, for example, CPUs (Central Processing Units), microprocessors, FPGAs (field-programmable gate arrays), and GPUs (Graphics Processing Units). Memory resources can include, for example, memory (hard disk drives, solid-state drives, etc.), RAM (Random Access Memory), and cache memory (e.g., Last Level Cache). For example, computing power can be evaluated based on the computing speed of processor resources, the capacity of memory resources, and the read speed of memory resources.

[0073] The third metric can be appropriately defined as follows: higher communication capability results in a higher evaluation, and lower communication capability results in a lower evaluation. The third metric can be expressed as a function (f3). Communication capability can be calculated based on the performance of the communication modules that can be used in the transmission of the local model 50. Communication capability can be evaluated based on, for example, communication bandwidth and communication speed.

[0074] [Number 1]

[0075]

[0076] When evaluation index 70 includes index 1, index 2, and index 3, evaluation index 70 can be constructed from the function of equation 1 above. k represents the on-board device 2 (vehicle V) of the object, S... k This represents the evaluation value 6 calculated by the vehicle-mounted device 2 of the object. n This refers to the weight (priority) of each indicator. n The value can be set appropriately. In one example, the weight c of each indicator... n This can be configured by the operator. It should be noted that the behavior of functions (f1, f2, f3) is an example, and the order of the indicators is not strictly limited. The behavior of each function can be appropriately determined according to the implementation method.

[0077] The relationship between the evaluation value 6 calculated by evaluation index 70 and the evaluation is not particularly limited and can be appropriately determined according to the implementation method. In one example, a high evaluation value 6 can correspond to a high evaluation. In another example, a low evaluation value 6 can correspond to a high evaluation. It should be noted that a high (low) evaluation corresponds to a high (low) priority of participating in joint learning. Participating in joint learning corresponds to the case where the local model 50 generated in this device is provided to the server device 1 in order to generate a new global model 45.

[0078] In another example, such as Figure 1 As shown, the server device 1 and each vehicle-mounted device 2 can be configured to communicate with each other via a network (connection is possible). The type of network is not particularly limited, and can be appropriately selected from, for example, the Internet, wireless communication network, mobile communication network, telephone network, private network, etc.

[0079] [2 Structural Examples]

[0080] [Hardware Structure Example]

[0081] <Server Device>

[0082] Figure 2 This schematically illustrates an example of the hardware structure of the server device 1 according to this embodiment. For example... Figure 2 As shown, the server device 1 in this embodiment is a computer electrically connected to a control unit 11, a storage unit 12, a communication module 13, an input device 14, an output device 15, and a driver 16.

[0083] The control unit 11 includes a CPU, RAM, ROM (Read Only Memory), cache memory, etc., which are hardware processors, and is configured to perform information processing based on programs and various data. The control unit 11 (CPU) is an example of a processor resource. The storage unit 12 is configured, for example, a hard disk drive, a solid-state drive, etc. The storage unit 12 is an example of a memory resource. In this embodiment, the storage unit 12 stores various information such as program 81, global model information 400, and evaluation index information 700.

[0084] Program 81 is used to enable server device 1 to perform information processing related to joint learning (described later). Figure 7 The program 81 contains a series of instructions for processing this information. Global model information 400 is configured to represent global model 40. Evaluation index information 700 is configured to represent evaluation index 70.

[0085] The communication module 13 may be a wired LAN (Local Area Network) module, a wireless LAN module, or the like, configured to perform wired or wireless communication via a network. The server device 1 can communicate data with other computers (e.g., each vehicle-mounted device 2) via the communication module 13.

[0086] Input device 14 is, for example, a mouse, keyboard, or other device for input. Output device 15 is, for example, a display, speaker, or other device for output. The operator can operate server device 1 by using input device 14 and output device 15. Input device 14 and output device 15 may also be integrated into a single unit, such as a touchpad or display.

[0087] The driver 16 is a device for reading various information, such as programs, stored in the storage medium 91. At least one of the aforementioned program 81, global model information 400, and evaluation index information 700 may also be stored in the storage medium 91. The storage medium 91 is a medium that stores various information, such as programs, through electrical, magnetic, optical, mechanical, or chemical means in a manner that allows computers and other devices or machines to read the stored information.

[0088] Here, in Figure 2 In this example, storage medium 91 is illustrated with disc-type storage media such as CDs and DVDs. However, the type of storage medium 91 is not limited to disc types. Other storage media besides disc types include, for example, semiconductor memories such as flash memory. The type of drive 16 can be appropriately selected depending on the type of storage medium 91.

[0089] It should be noted that the specific hardware structure of server device 1 can be appropriately omitted, substituted, or added depending on the implementation method. For example, the control unit 11 may include multiple hardware processors. The hardware processors may be composed of microprocessors, FPGAs, GPUs, etc. At least one of the input device 14, output device 15, and driver 16 may also be omitted. Server device 1 may be composed of multiple computers. In this case, the hardware structure of each computer may be the same or different. Server device 1 may be a computer designed specifically for providing services, or it may be a general-purpose computer, etc.

[0090] <Vehicle-mounted device>

[0091] Figure 3 This schematically illustrates an example of the hardware structure of the vehicle-mounted device 2 according to this embodiment. For example... Figure 3As shown, the vehicle-mounted device 2 in this embodiment is a computer electrically connected to a control unit 21, a storage unit 22, a communication module 23, an input device 24, an output device 25, a driver 26, a near-field communication module 27, and an external interface 28.

[0092] The control unit 21 includes a CPU, RAM, ROM, etc., which are hardware processors, and is configured to perform various information processing based on programs and data. The control unit 21 (CPU) is an example of a processor resource. The storage unit 22 is an example of a memory resource, such as a hard disk drive, a solid-state drive, etc. In this embodiment, the storage unit 22 stores various information such as program 82, global model information 400, evaluation index information 700, local data 3, and local model information 500.

[0093] Program 82 is used to enable the onboard device 2 to perform information processing related to joint learning (described later). Figure 7 and Figure 8 The program 82 contains a series of instructions for processing this information. The local model information 500 is configured to represent information related to the generated local model 50.

[0094] The communication module 23, such as a wireless LAN module, is configured to perform wireless communication via a network. The vehicle-mounted device 2 can communicate data with other computers (e.g., server device 1) via the communication module 23. The input device 24 and output device 25 can be configured similarly to the input device 14 and output device 15. An operator (e.g., a passenger of vehicle V) can operate the vehicle-mounted device 2 using the input device 24 and output device 25.

[0095] Drive 26 and storage medium 92 can be configured in the same way as drive 16 and storage medium 91. At least one of the above-mentioned program 82, global model information 400, evaluation index information 700, local data 3, and local model information 500 can also be stored in storage medium 92.

[0096] The near-field communication module 27 is configured to perform data transmission and reception based on near-field communication. In one example, the near-field communication module 27 may be a beacon module. The type of beacon module may be appropriately selected according to the implementation method. The near-field communication module 27 may employ a known type of beacon module. In this embodiment, the near-field communication module 27 is used for near-field communication with other vehicle-mounted devices 2 besides this device. However, the use of the near-field communication module 27 is not limited to this. The near-field communication module 27 may also be used for other purposes. In addition, the type of near-field communication module 27 may not be limited to a beacon module. In another example, the near-field communication module 27 may be configured to perform near-field communication such as Wi-Fi (registered trademark), Bluetooth (registered trademark), and D2D.

[0097] External interface 28, such as a USB (Universal Serial Bus) port or a dedicated port, is an interface used to connect to external devices. The type and number of external interfaces 28 can be appropriately determined according to the type and number of external devices connected. In this embodiment, the vehicle-mounted device 2 can be connected to the sensor S via external interface 28. The sensor S can be, for example, an image sensor (camera, etc.), a proximity sensor, a LiDAR (light detection and ranging) sensor, a Radar, etc. The inference task performed by the global model (40, 45) and the local model 50 can be to derive inference results from the sensing data obtained from the sensor S. In this case, the collection of local data 3 can be performed using the sensor S.

[0098] It should be noted that the specific hardware structure of the vehicle-mounted device 2 can be appropriately modified by omitting, replacing, or adding components depending on the implementation method. For example, the control unit 21 may include multiple hardware processors. The hardware processors may be composed of microprocessors, FPGAs, GPUs, etc. At least one of the input device 24, output device 25, driver 26, and external interface 28 may also be omitted. The vehicle-mounted device 2 may be composed of multiple computers. In this case, the hardware structures of each computer may be identical or different. In addition to being a computer designed specifically for providing services, the vehicle-mounted device 2 may also be a general-purpose computer, including smartphones, mobile phones, tablet PCs (personal computers), etc.

[0099] [Software Structure Example]

[0100] <Server Device>

[0101] Figure 4 This schematically illustrates an example of the software structure of the server device 1 according to this embodiment. The control unit 11 of the server device 1 expands the program 81 stored in the storage unit 12 into RAM. Then, the control unit 11 executes the instructions contained in the program 81 expanded in RAM via the CPU. Thus, as... Figure 4 As shown, the server device 1 in this embodiment operates as a computer having a model distribution unit 111, an index distribution unit 112, a model receiving unit 113, and an integration processing unit 114 as software modules. That is, in this embodiment, each software module of the server device 1 is implemented by the control unit 11 (CPU).

[0102] The model distribution unit 111 is configured to send global model information 400, representing the global model 40, to multiple vehicle-mounted devices 2. The indicator distribution unit 112 is configured to send evaluation indicator information 700, representing the evaluation indicator 70, to multiple vehicle-mounted devices 2. In this embodiment, the global model 40 and the evaluation indicator 70 are distributed from the server device 1 to each vehicle-mounted device 2 through the operation of the model distribution unit 111 and the indicator distribution unit 112.

[0103] The model receiving unit 113 is configured to receive local model information 500 representing the local model 50 from at least one of the plurality of vehicle-mounted devices 2. The at least one vehicle-mounted device 2 is determined to be a high-rated device as a result of a selection process based on evaluation values ​​6. The selection process includes exchanging evaluation values ​​6 between the device and other vehicle-mounted devices 2 via near-field communication, and comparing the evaluation values ​​6 of the device with those of the other vehicle-mounted devices 2.

[0104] The integration processing unit 114 is configured to generate a new global model 45 by integrating the local model 50 represented by the local model information 500 received from at least one vehicle-mounted device 2. The integration processing unit 114 may also be configured to generate global model information 450 representing the new global model 45 and store the generated global model information 450 in a predetermined storage area (e.g., the RAM of the control unit 11, the storage unit 12, the storage medium 91, the external storage device, etc.).

[0105] <Vehicle-mounted device>

[0106] Figure 5 This schematically illustrates an example of the software structure of the vehicle-mounted device 2 according to this embodiment. The control unit 21 of the vehicle-mounted device 2 expands the program 82 stored in the storage unit 22 into RAM. Then, the control unit 21 executes the instructions contained in the program 82 expanded in RAM via the CPU. Thus, as... Figure 5 As shown, in this embodiment, the vehicle-mounted device 2 operates as a computer comprising a model acquisition unit 211, an index acquisition unit 212, a data collection unit 213, a machine learning unit 214, an evaluation value calculation unit 215, an evaluation value transmission unit 216, an evaluation value receiving unit 217, an evaluation unit 218, and a model transmission unit 219 as software modules. That is, in this embodiment, similar to the server device 1, each software module of the vehicle-mounted device 2 is also implemented by the control unit 21 (CPU).

[0107] The model acquisition unit 211 is configured to acquire global model information 400 representing the global model 40. In this embodiment, the model acquisition unit 211 is configured to receive the global model information 400 distributed from the server device 1. The indicator acquisition unit 212 is configured to acquire evaluation indicator information 700 representing the evaluation indicator 70. In this embodiment, the indicator acquisition unit 212 is configured to receive the evaluation indicator information 700 distributed from the server device 1.

[0108] The data collection unit 213 is configured to collect local data 3. The machine learning unit 214 is configured to perform machine learning on the global model 40 using the collected local data 3, thereby generating a local model 50. The machine learning unit 214 may also be configured to generate local model information 500 representing the local model 50, and store the generated local model information 500 in a predetermined storage area (e.g., RAM of the control unit 21, storage unit 22, storage medium 92, external storage device, etc.).

[0109] The evaluation value calculation unit 215 is configured to calculate the evaluation value 6 (6s) based on the evaluation index 70 represented by the acquired evaluation index information 700. The evaluation value transmission unit 216 is configured to transmit the calculated evaluation value 6 (6s) to other vehicle-mounted devices 2 mounted on one or more other vehicles V located near its own vehicle V via near-field communication. The evaluation value receiving unit 217 is configured to receive the evaluation value 6 (6t) calculated by the other vehicle-mounted devices 2 mounted on one or more other vehicles V via near-field communication. It should be noted that in the following description, the evaluation value of this device (the target vehicle-mounted device 2) is sometimes marked with the symbol "6s", and the evaluation value of other vehicle-mounted devices 2 observed from the target vehicle-mounted device 2 is marked with the symbol "6t".

[0110] The evaluation unit 218 is configured to determine whether the evaluation is high by comparing the calculated evaluation value 6s with the evaluation value 6t received from other vehicle-mounted devices 2. The model sending unit 219 is configured to send local model information 500, representing the generated local model 50, to the server device 1 if the evaluation of this device is determined to be high.

[0111] (An example of a machine learning method)

[0112] Figure 6This illustration shows an example of the machine learning process performed by the machine learning unit 214 on the global model 40. The global model (40, 45) and the local model 50 are composed of machine learning models. Each machine learning model has one or more computational parameters for performing computational processing to solve reasoning tasks, and the values ​​of these parameters are adjusted through machine learning. As long as computation can be performed to solve reasoning tasks related to the moving body, the type, structure, and configuration of the machine learning model are not particularly limited and can be appropriately determined according to the implementation method.

[0113] As a typical example Figure 6 This illustration shows an example of a scenario where a neural network is used in the machine learning model constituting the global model 40, and supervised learning is employed in the machine learning method. In this case, the weights of the connections between nodes (neurons), the thresholds of each node, etc., are examples of computational parameters. The type and structure of the neural network (e.g., the type of layers, the number of layers, the number of nodes in each layer, the connection relationships between nodes, etc.) can be appropriately determined according to the implementation method. The global model 40 can, for example, be composed of a fully connected neural network, a convolutional neural network, a recurrent neural network, or a combination thereof.

[0114] exist Figure 6 In one example, local data 3 consists of multiple datasets 30. Each dataset 30 is a combination of training data 31 and correct labels 33, used as samples for training in machine learning. Training data 31 are samples of explanatory variables, input into the machine learning model. In cases where the inference task derives inference results from sensor data, training data 31 can consist of sensor data obtained through sensor S. Correct labels 33 are configured to represent the truth value of the target variable for the input explanatory variable samples, i.e., the solution (truth value) of the inference task for the corresponding training data 31. In machine learning, training data 31 is used as input data, and correct labels 33 are used as training signals.

[0115] Training data 31 can be collected appropriately. In one example, the control unit 21 of each vehicle-mounted device 2 can collect training data 31 as a data collection unit 213 while executing an inference task based on the global model 40. In this case, the control unit 21 can prioritize collecting input data from which the global model 40 derives incorrect inference results as training data 31. In the case of repeatedly performing joint learning processing, where a local model 50 is temporarily generated in past processing, the control unit 21 can collect training data 31 while executing an inference task based on the local model 50. In contrast, the correct label 33 can be appropriately obtained in a manner that indicates the correctness of the inference based on the obtained training data 31. In one example, the correct label 33 can be obtained through operator input. In another example, the correct label 33 can also be obtained through arbitrary inference processing. The correct label 33 can also be obtained based on information obtained from other sensors. Each dataset 30 can be generated automatically by computer actions or manually by actions that at least partially include operator actions.

[0116] As part of the machine learning process, the machine learning unit 214 performs initial settings for the global model 40 with reference to the global model information 400. This process can be omitted if the information for the global model 40 has already been expanded. Next, the machine learning unit 214 trains the global model 40 for each dataset 30 in such a way that the output values ​​(inference results) obtained from the global model 40 by providing training data 31 match the true values ​​represented by the correct labels 33. The global model 40 is trained by adjusting (optimizing) the values ​​of the operational parameters that constitute it. The method for adjusting the operational parameters (the method for solving the optimization problem) can be appropriately determined based on the type, structure, and construction of the machine learning model used in the global model 40.

[0117] As an example of the adjustment method, when the global model 40 is composed of a neural network, the machine learning unit 214 inputs the training data 31 of each dataset 30 into the global model 40 and performs forward propagation processing of the global model 40. As a result of this processing, the machine learning unit 214 obtains the output value corresponding to the inference result for the training data 31 from the global model 40. The machine learning unit 214 calculates the error between the obtained output value and the true value represented by the corresponding correct label 33, and further calculates the gradient of the calculated error. The machine learning unit 214 calculates the error of the values ​​of the operational parameters of the global model 40 by backpropagating the calculated error gradient using the error backpropagation method. Based on the calculated error, the machine learning unit 214 updates the values ​​of the operational parameters.

[0118] Through this series of update processes, the machine learning unit 214 adjusts the values ​​of the operational parameters of the global model 40, thereby reducing the sum of errors between the output value (inference result) obtained by providing training data 31 and the true value represented by the correct label 33. This adjustment of the operational parameters can be repeated, for example, until predetermined conditions such as performing the set number of repetitions or the calculated sum of errors falling below a threshold are met. Furthermore, machine learning conditions such as the objective function (cost function, loss function, error function) and learning rate used to calculate the error can be appropriately set according to the implementation method. Through this machine learning process, a local model 50 can be generated. The machine learning unit 214 generates local model information 500 representing the local model 50.

[0119] It should be noted that machine learning methods are not limited to Figure 6 The example illustrates supervised learning. In another example, the machine learning method can employ unsupervised learning (including adversarial learning) or reinforcement learning. The structure of local data 3 can be appropriately determined based on the machine learning method used, the type of machine learning model, etc. Furthermore, the machine learning model constituting the global model 40 is not limited to neural networks. In another example, the global model 40 can be composed of regression models, support vector machines, etc.

[0120] <Other>

[0121] In this embodiment, an example has been described in which each software module of the server device 1 and the vehicle-mounted device 2 is implemented using a general-purpose CPU. However, some or all of the above-mentioned software modules may also be implemented using one or more dedicated processors. The above-mentioned modules may also be implemented as hardware modules. Regarding the software structure of the server device 1 and the vehicle-mounted device 2, modules may be omitted, replaced, or added as appropriate, depending on the implementation method.

[0122] [3 Action Examples]

[0123] Figure 7 This is a sequence diagram illustrating an example of the joint learning process performed between the server device 1 and each vehicle-mounted device 2 in this embodiment. The following processing steps are an example of an information processing method. However, the following processing steps are merely an example, and each step can be modified as much as possible. Furthermore, regarding the following processing steps, steps can be appropriately omitted, substituted, or added according to the embodiment.

[0124] <Steps S11 and S21>

[0125] In step S11, the control unit 11 of the server device 1 acts as the model distribution unit 111 and uses the communication module 13 to send the global model information 400 representing the global model 40 to each vehicle device 2.

[0126] The transmission range and transmission method can be appropriately selected according to the implementation method. For example, the control unit 11 can also transmit the global model information 400 (global model 40) to each vehicle-mounted device 2 via broadcast. This saves bandwidth for wireless communication. Furthermore, the geographical range for integrating the local model 50 can be arbitrarily defined, and the global model 40 to be distributed can be selected for each defined range. In this case, the global model 40 distributed to one part and the global model 40 distributed to another part can also be different from each other. When the global model 40 is distributed via broadcast, the geographical range can be defined according to the broadcast range.

[0127] The global model 40 can be either a machine learning model before training or a machine learning model after training. If the global model 40 is a machine learning model after training, the global model 40 can be generated by machine learning from the server device 1 or by integration processing of prior joint learning (for example, after the integration processing of step S33 described above, the new global model 45 is used as the global model 40 and the processing is performed again from step S11).

[0128] When server device 1 generates global model 40 by implementing machine learning, control unit 11 can acquire global data. Global data can be constructed in the same way as local data 3. Global data can be collected, for example, through data generation in an experimental environment, or provided from vehicle-mounted device 2. Then, control unit 11 can also generate global model 40 by implementing machine learning using the acquired global data. The machine learning method for generating global model 40 can be the same as the machine learning method for generating local model 50. In one example, with... Figure 6 Similarly, global data can consist of multiple datasets, each containing a combination of training data and correct labels, and global model 40 can be generated through supervised learning.

[0129] The structure of the global model information 400 is not particularly limited as long as it can reproduce the global model 40, and can be appropriately determined according to the implementation method. In one example, the global model information 400 may include information representing the values ​​of the operational parameters contained in the global model 40. Depending on the situation, the global model information 400 may also include information representing the construction of the global model 40. The construction can be determined, for example, by the number of layers from the input layer to the output layer, the types of each layer, the number of neurons contained in each layer, and the connection relationships between neurons in adjacent layers.

[0130] According to the processing in step S11, in step S21, the control unit 21 of each vehicle-mounted device 2 acts as a model acquisition unit 211 and uses the communication module 23 to receive global model information 400 from the server device 1. As a result, each vehicle-mounted device 2 acquires the global model 40.

[0131] <Steps S13 and S23>

[0132] In step S13, the control unit 11 of the server device 1 acts as the index distribution unit 112 and uses the communication module 13 to send the evaluation index information 700 representing the evaluation index 70 to each vehicle device 2.

[0133] Similar to step S11, the transmission range and transmission method can be appropriately selected according to the implementation method. For example, the control unit 11 can also transmit the evaluation index information 700 to each vehicle-mounted device 2 via broadcast. This saves wireless communication bandwidth. Furthermore, the evaluation index 70 to be distributed can be selected for each geographical area. In this case, the evaluation index 70 distributed to one part can be different from the evaluation index 70 distributed to other parts. When broadcasting is used as the method for distributing the evaluation index information 700, the geographical area can be defined according to the broadcast range.

[0134] The structure of the evaluation index information 700 is not particularly limited, as long as it can determine the evaluation index 70 and calculate the evaluation value 6. It can be appropriately determined according to the implementation method. In one example, the evaluation index information 700 can be configured to represent the relational expression in Equation 1 above. In another example, where the functions of each index are pre-shared among the vehicle-mounted devices 2, the evaluation index information 700 can be configured to represent the weight c of each index. n (Information about the functions of each indicator can be omitted). In this case, the weights c of each indicator... n The settings can be appropriately configured for each geographical area, and the control unit 11 can broadcast the weights of each indicator. n As an evaluation indicator, information 700 is sent to each vehicle-mounted device 2.

[0135] According to the processing in step S13, in step S23, the control unit 21 of each vehicle-mounted device 2 operates as an index acquisition unit 212, and uses the communication module 23 to acquire evaluation index information 700 from the server device 1. Thus, each vehicle-mounted device 2 acquires the evaluation index 70.

[0136] <Step S25>

[0137] In step S25, the control unit 21 of each vehicle-mounted device 2 performs the generation of the local model 50 and the selection of autonomously distributed vehicles V.

[0138] Figure 8This is a flowchart illustrating an example of the processing steps related to the local model 50 generation process and the autonomous decentralized selection process performed by each on-board device 2 in this embodiment. The processing of step S25 in this embodiment may include the processing of steps S251 to S258 below. However, Figure 8 The processing steps shown are merely one example; each process can be modified as much as possible. Additionally, regarding... Figure 8 The processing steps shown can be appropriately omitted, substituted, or added according to the implementation method.

[0139] (Steps S251 and S252)

[0140] In step S251, the control unit 21 of each vehicle-mounted device 2 operates as a data collection unit 213, collecting local data 3. In step S252, the control unit 21 of each vehicle-mounted device 2 operates as a machine learning unit 214, using the collected local data 3 to perform machine learning on the received global model 40, thereby generating a local model 50. In one example, such as Figure 6 As shown, local data 3 can consist of multiple datasets 30 that respectively contain combinations of training data 31 and correct labels 33, and local model 50 can be generated through supervised learning.

[0141] After machine learning is completed, the control unit 21 of each vehicle-mounted device 2 generates local model information 500 representing the local model 50. The structure of the local model information 500 is not particularly limited as long as it can reproduce the local model 50, and can be appropriately determined according to the implementation method. In one example, similar to the global model information 400, the local model information 500 may include information representing the values ​​of the computational parameters included in the local model 50, adjusted through machine learning. Depending on the situation, the local model information 500 may also include information representing the construction of the local model 50. In another example, the local model information 500 may be configured to represent the update from the global model 40 to the local model 50 (i.e., the difference in the values ​​of the computational parameters between the global model 40 and the local model 50).

[0142] It should be noted that after generating the local model 50, each vehicle-mounted device 2 can either directly retain the global model information 400, or delete the global model information 400 (for example, it can be overwritten with the local model information 500). Furthermore, the generated local model 50 can be used to solve inference tasks at any time in each vehicle-mounted device 2. After machine learning is completed, the control unit 21 of each vehicle-mounted device 2 initiates the next step, S253.

[0143] (Step S253)

[0144] In step S253, the control unit 21 of each vehicle-mounted device 2 operates as an evaluation value calculation unit 215, calculating the evaluation value 6s of the device based on the evaluation index 70 represented by the received evaluation index information 700. In this embodiment, the evaluation index 70 may include at least one of the first index, the second index, and the third index described above. Furthermore, the evaluation index 70 can be represented by a function. Accordingly, the control unit 21 of each vehicle-mounted device 2 can calculate the evaluation value 6s of the device by executing the arithmetic processing of the function representing the evaluation index 70 (for example, Equation 1 above). If the evaluation value 6s of the device is calculated, the control unit 21 causes the processing to proceed to the next step S254.

[0145] (Steps S254 to S256)

[0146] In step S254, the control unit 21 of each vehicle-mounted device 2 operates as an evaluation value transmitting unit 216, transmitting the evaluation value 6s calculated by the device to other vehicle-mounted devices 2 of one or more other vehicles V located near its own vehicle V via near-field communication. In step S255, the control unit 21 of each vehicle-mounted device 2 operates as an evaluation value receiving unit 217, receiving the evaluation value 6t calculated by the other vehicle-mounted devices 2 of one or more other vehicles V based on the evaluation index 70.

[0147] The timing of executing steps S254 and S255 can be adjusted among the vehicle-mounted devices 2 in any way. In one example, the trial period for sending the evaluation value 6s of step S254 and receiving the evaluation value 6t of step S255 can be predetermined. In this case, information about the time period to be executed can be appropriately shared among the vehicle-mounted devices 2.

[0148] Near-field communication module 27 is used in the near-field communication in steps S254 and S255. In this embodiment, each vehicle-mounted device 2 may be equipped with a beacon module as near-field communication module 27. Correspondingly, the control unit 21 may also use a beacon module for near-field communication. Thus, near-field communication can be performed inexpensively. It should be noted that the transmission range of the evaluation value 6s can be appropriately determined according to the type of near-field communication module 27, etc.

[0149] Furthermore, the transmission method can be appropriately selected according to the implementation method. In one example, the control unit 21 of each vehicle-mounted device 2 can also transmit the evaluation value 6s via broadcast during near-field communication in step S254. In step S255, the control unit 21 of each vehicle-mounted device 2 can listen to broadcasts from other vehicle-mounted devices 2 to attempt to obtain the evaluation value 6t from other vehicle-mounted devices 2. As a result, the bandwidth of near-field communication can be saved.

[0150] In step S256, the control unit 21 of each vehicle-mounted device 2 determines whether an evaluation value 6t has been obtained from other vehicle-mounted devices 2 within a certain period. The inability to obtain an evaluation value 6t from other vehicle-mounted devices 2 within a certain period corresponds to the situation where no other vehicle-mounted devices 2 (other vehicles V) are present within the range where near-field communication is possible during that period. The information processing method of this embodiment is configured to enable such vehicle-mounted devices 2 to participate in joint learning. That is, when no evaluation value 6t is obtained from other vehicle-mounted devices 2 within a certain period (timeout), the control unit 21 proceeds to step S27. On the other hand, when an evaluation value 6t is obtained from other vehicle-mounted devices 2 mounted on one or more other vehicles V within a certain period, the control unit 21 proceeds to step S257.

[0151] (Steps S257 and S258)

[0152] In step S257, the control unit 21 operates as the evaluation unit 218, and determines whether the evaluation is high by comparing the evaluation value 6s calculated by this device with the evaluation value 6t received from other vehicle-mounted devices 2.

[0153] The range for determining a high rating can be appropriately determined based on the implementation method. In one example, the control unit 21 may determine a high rating if the comparison result of the rating values ​​(6s, 6t) is the highest, and otherwise determine a low rating. In another example, the control unit 21 may determine a high rating if the comparison result of the rating values ​​(6s, 6t) is within any number of the highest rating, and otherwise determine a low rating.

[0154] In step S258, the control unit 21 determines the branch destination for processing based on the comparison result of step S257. If the evaluation is determined to be high, the control unit 21 causes the processing to proceed to step S27. On the other hand, if the evaluation is determined to be low, the control unit 21 of the vehicle-mounted device 2 skips the processing in step S27 and ends the processing steps of this operation example.

[0155] <Step S27>

[0156] In step S27, the control unit 21 operates as a model sending unit 219, using the communication module 23 to send local model information 500, representing the local model 50 generated by this device, to the server device 1. In this embodiment, the processing in step S27 is performed by the vehicle device 2 that has not received an evaluation value 6t from other vehicle devices 2 within a certain period of time in the processing of step S256, and at least one vehicle device 2 whose evaluation is determined to be high based on the comparison result of step S257. When the transmission of local model information 500 is completed, the control unit 21 of the vehicle device 2 ends the processing steps of this operation example.

[0157] <Step S31>

[0158] return Figure 7 In step S31, the control unit 11 of the server device 1 operates as a model receiving unit 113, receiving local model information 500 representing the local model 50 from at least one vehicle-mounted device 2 that has performed the processing in step S27. When the local model information 500 is received, the control unit 11 causes the processing to proceed to the next step S33.

[0159] <Step S33>

[0160] In step S33, the control unit 11 operates as an integration processing unit 114, generating a new global model 45 by integrating the local models 50 represented by the local model information 500 received from at least one vehicle-mounted device 2. Integrating multiple local models 50 can be achieved by averaging the values ​​of the corresponding operational parameters contained in each local model 50, or by summing the values ​​of the corresponding operational parameters. The values ​​of the operational parameters contained in each local model 50 can also be integrated after weighting. The weights of each local model 50 can be specified in any way.

[0161] The integration of the local model 50 can be performed using all the calculation parameters or only a portion of them. When the integration of the local model 50 is performed in a partial manner, the information of the range that is not to be integrated can be omitted from the local model information 500 sent from the vehicle-mounted device 2 to the server device 1.

[0162] The control unit 11 can generate global model information 450 representing the new global model 45. Global model information 450 can be configured similarly to global model information 400. After generating the new global model 45, in one example, the server device 1 can directly retain the global model information 400 before the update and the local model information 500 received from the vehicle-mounted device 2. In another example, the server device 1 can also delete at least one of the global model information 400 and the local model information 500.

[0163] When the new global model 45 is generated, the control unit 11 of the server device 1 terminates the processing steps of this action example. It should be noted that the server device 1 and each vehicle-mounted device 2 can treat the new global model 45 as the global model 40, repeatedly executing the joint learning process from step S11 onwards. In the repeatedly executed joint learning process, steps S13 and S23 can be omitted if the evaluation index 70 is not changed. Steps S13 and S23 can be executed appropriately when the evaluation index 70 is changed.

[0164] [feature]

[0165] In this embodiment, the selection process in steps S253 to S257 is implemented by exchanging evaluation values ​​6 between the vehicle-mounted devices 2 based on near-field communication. According to this near-field communication-based selection process, vehicle-mounted devices 2 participating in joint learning can be selected from multiple vehicle-mounted devices 2 mounted on vehicles V located in geographically close proximity. Then, the local model 50 generated by the selected vehicle-mounted devices 2 can be reflected in the update of the global model 40 in step S33. Furthermore, vehicle-mounted devices 2 participating in joint learning can be selected autonomously and decentralizedly without relying on the server device 1. Therefore, according to this embodiment, the efficiency of joint learning implemented between the server device 1 and the vehicle-mounted devices 2 can be improved. Moreover, since the data exchanged between the vehicle-mounted devices 2 in steps S254 and S255 is only the evaluation value 6, the privacy issue can be resolved.

[0166] Furthermore, in this embodiment, the global model 40 and evaluation metrics 70 are provided from the server device 1 to each vehicle device 2. This allows for flexible modification of the global model 40 and evaluation metrics 70. For example, the global model 40 and evaluation metrics 70 can be appropriately modified for each arbitrary geographical area. By adjusting the global model 40 and evaluation metrics 70 in a manner suitable for performing inference tasks generated in each area, a new global model 45 that can achieve better performance in each area can be generated.

[0167] [4 Variations]

[0168] The embodiments of this disclosure have been described in detail above, but the description up to the foregoing is merely illustrative in all respects. Various modifications or variations can be made without departing from the scope of this disclosure. For example, the following modifications can be made. The following variations can be appropriately combined.

[0169] In the above embodiments, vehicle V is an example of a mobile body, and vehicle-mounted device 2 is an example of an information processing device mounted on the mobile body. However, the mobile body of this disclosure is not limited to vehicles. The mobile body may be, for example, a drone, a robotic device, or other device configured to be mobile. The information processing device may be appropriately replaced, for example, depending on the type of mobile body on which it is mounted.

[0170] In the above embodiment, the global model 40 and evaluation index 70 are provided from the server device 1 to each vehicle device 2. However, the method of providing the global model 40 and evaluation index 70 is not limited to this example. In another example, the global model information 400 (global model 40) can be provided to each vehicle device 2 from an external storage device, an external server, a storage medium 92, etc. When the global model 40 is provided to each vehicle device 2 by a method other than distribution from the server device 1, the model distribution unit 111 can be omitted from the software structure of the server device 1, and the processing steps S11 and S21 can be omitted from the processing steps of the information processing method of the above embodiment. In addition, in another example, the evaluation index information 700 can be provided to each vehicle device 2 from an external storage device, an external server, a storage medium 92, etc. The evaluation index information 700 can also be pre-assembled in each vehicle device 2. When the evaluation index information 700 is provided to each vehicle device 2 by a method other than the method of distribution from server device 1, the index distribution unit 112 can be omitted from the software structure of server device 1, and the processing steps S13 and S23 can be omitted from the processing steps of the information processing method of the above embodiment.

[0171] In the above embodiments, Figure 7 and Figure 8 The processing order of the steps involved in the information processing can be appropriately changed without causing contradictions. For example, steps S11 and S21 can be executed at any time before step S252. Steps S13 and S23 can be executed at any time before step S253. The processing order of steps S11 and S13 can be arbitrarily changed. For example, steps S11 and S13 can be executed simultaneously. Alternatively, step S13 can be executed before step S11. Step S251 can be executed at any time before step S252. Step S252 can be executed at any time before step S27. Step S253 can be executed at any time before step S254. The processing order of steps S254 and S255 can be arbitrarily changed. For example, steps S254 and S255 can be executed in parallel.

[0172] Furthermore, the information processing method of the above embodiment is configured such that, through the processing in step S256, vehicle devices 2 that have not obtained evaluation values ​​6t from other vehicle devices 2 within a certain period of time participate in joint learning. However, the processing of vehicle devices 2 that cannot obtain evaluation values ​​6t from other vehicle devices 2 is not limited to this example. The information processing method of the above embodiment can also be configured such that vehicle devices 2 that have not obtained evaluation values ​​6t from other vehicle devices 2 do not participate in joint learning. That is, if the determination process in step S256 is YES, the control unit 21 can end the processing steps of the above-described operation example. In another example, the processing in step S256 can be omitted.

[0173] [5 Supplements]

[0174] The processes and components described in this disclosure can be freely combined and implemented as long as they do not create technical contradictions.

[0175] Furthermore, the processing described by one device can be performed by multiple devices. Alternatively, the processing described by different devices can be performed by one device. In a computer system, the hardware structure used to implement each function can be flexibly changed.

[0176] This disclosure can also be implemented by supplying a computer program with the functions described in the above embodiments to a computer, which is then read and executed by one or more processors. Such a computer program can be provided to the computer either by a non-volatile computer-readable storage medium connectable to the computer's system bus or via a network. Non-volatile computer-readable storage media include, for example, any type of disk such as a hard disk (floppy disk, hard disk drive (HDD)), an optical disk (CD-ROM, DVD, Blu-ray disc, etc.), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards, flash memory, optical cards, semiconductor drivers (solid-state drives, etc.), and any type of medium suitable for storing electronic instructions.

Claims

1. An information processing method in which, the information processing method includes: each information processing device of a plurality of information processing devices mounted on different mobile bodies collects local data; each information processing device of the plurality of information processing devices uses the collected local data to implement machine learning of a global model, thereby generating a local model; each information processing device of the plurality of information processing devices calculates an evaluation value in accordance with an evaluation index; each information processing device of the plurality of information processing devices transmits the calculated evaluation value to other information processing devices mounted on one or more other mobile bodies existing in the vicinity of one's own mobile body through near field communication; each information processing device of the plurality of information processing devices receives the evaluation value calculated by the other information processing devices from the other information processing devices mounted on the one or more other mobile bodies through the near field communication; each information processing device of the plurality of information processing devices determines whether the evaluation is high by comparing the calculated evaluation value with the evaluation value received from the other information processing devices; and at least one information processing device whose evaluation is determined to be high transmits local model information indicating the generated local model to a server device.

2. The information processing method according to claim 1, wherein, each information processing device of the plurality of information processing devices is provided with a beacon module, the near field communication is performed using the beacon module.

3. The information processing method according to claim 1 or 2, wherein, the evaluation index includes a first index for evaluating the quality of the local data.

4. The information processing method according to claim 1 or 2, wherein, the evaluation index includes a second index for evaluating the calculation capability of each information processing device of the plurality of information processing devices.

5. The information processing method according to claim 3, wherein, the evaluation index further includes a second index for evaluating the calculation capability of each information processing device of the plurality of information processing devices.

6. The information processing method according to claim 1 or 2, wherein, the evaluation index includes a third index for evaluating the communication capability of each information processing device of the plurality of information processing devices.

7. The information processing method according to claim 3, wherein, the evaluation index further includes a third index for evaluating the communication capability of each information processing device of the plurality of information processing devices.

8. The information processing method according to claim 4, wherein, the evaluation index further includes a third index for evaluating the communication capability of each information processing device of the plurality of information processing devices.

9. The information processing method according to claim 5, wherein, the evaluation index further includes a third index for evaluating the communication capability of each information processing device of the plurality of information processing devices.

10. The information processing method according to claim 1 or 2, wherein, the evaluation index is expressed by a function, each information processing device of the plurality of information processing devices calculates the evaluation value by performing an arithmetic process of the function expressing the evaluation index.

11. The information processing method according to claim 1 or 2, wherein the information processing method further includes: the server device transmits the global model to each of the plurality of information processing devices; and each of the plurality of information processing devices receives the global model from the server device.

12. The information processing method according to claim 1 or 2, wherein the information processing method further includes: the server device transmits evaluation index information indicating the evaluation index to each of the plurality of information processing devices; and each of the plurality of information processing devices receives the evaluation index information from the server device, each of the plurality of information processing devices calculates the evaluation value based on the evaluation index indicated by the received evaluation index information.

13. An information processing device mounted on a mobile body, wherein the information processing device includes a control section configured to perform: collecting local data; implementing machine learning of a global model using the collected local data, thereby generating a local model; calculating an evaluation value based on an evaluation index; transmitting the calculated evaluation value to other information processing devices mounted on one or more other mobile bodies existing in the vicinity, through near field communication; receiving evaluation values calculated by the other information processing devices based on the evaluation index, from the other information processing devices mounted on the one or more other mobile bodies, through the near field communication; determining whether or not the evaluation is high, by comparing the calculated evaluation value with the received evaluation values; and in a case where it is determined that the evaluation is high, transmitting local model information indicating the generated local model to a server device.

14. The information processing device according to claim 13, wherein it further includes a beacon module, the control section is configured to perform the near field communication using the beacon module.

15. The information processing device according to claim 13 or 14, wherein the evaluation index includes a first index for evaluating a quality of the local data.

16. The information processing device according to claim 13 or 14, wherein the evaluation index includes a second index for evaluating a calculation capability.

17. The information processing device according to claim 15, wherein the evaluation index further includes the second index for evaluating the calculation capability.

18. The information processing device according to claim 13 or 14, wherein the evaluation index includes a third index for evaluating a communication capability.

19. The information processing device according to claim 15, wherein the evaluation index further includes the third index for evaluating the communication capability.

20. The information processing device according to claim 16, wherein the evaluation index further includes the third index for evaluating the communication capability.

21. The information processing device according to claim 17, wherein the evaluation index further includes the third index for evaluating the communication capability.

22. The information processing apparatus according to claim 13 or 14, wherein the evaluation index is represented by a function, the control section is configured to calculate the evaluation value by executing an operation process of the function representing the evaluation index.

23. The information processing apparatus according to claim 13 or 14, wherein the control section is configured to transmit the evaluation value by broadcast in the near field communication.

24. A server apparatus configured to be able to communicate with each information processing apparatus of a plurality of information processing apparatuses configured to respectively collect local data and to implement machine learning of a global model using the collected local data, thereby to respectively generate a local model, the plurality of information processing apparatuses being respectively mounted on different moving bodies, wherein The server apparatus is provided with a control section configured to execute the following actions: receive local model information representing a local model from at least one information processing apparatus of a plurality of information processing apparatuses; and generate a new global model by integrating local models represented by the local model information received from the at least one information processing apparatus, the plurality of information processing apparatuses are further configured to: calculate an evaluation value according to an evaluation index, transmit the calculated evaluation value to other information processing apparatuses respectively mounted on one or more other mobile bodies existing in the vicinity of one's own mobile body by near field communication, receive evaluation values calculated by the other information processing apparatuses according to the evaluation index from the other information processing apparatuses respectively mounted on the one or more other mobile bodies by the near field communication, and determine whether the evaluation is high by comparing the calculated evaluation value with the evaluation values received from the other information processing apparatuses, the at least one information processing apparatus is an information processing apparatus determined to have a high evaluation as a result of comparison using the evaluation value.

25. The server apparatus according to claim 24, wherein the control section is configured to further execute the following action: transmit the global model to each information processing apparatus of the plurality of information processing apparatuses, each information processing apparatus of the plurality of information processing apparatuses is configured to generate the local model by implementing machine learning of the received global model using the collected local data.

26. The server apparatus according to claim 25, wherein the control section is configured to transmit the global model to each information processing apparatus of the plurality of information processing apparatuses by broadcast.

27. The server apparatus according to any one of claims 24 to 26, wherein the control section is configured to further execute the following action: transmit evaluation index information representing the evaluation index to each information processing apparatus of the plurality of information processing apparatuses, each information processing apparatus of the plurality of information processing apparatuses is configured to calculate the evaluation value according to the evaluation index represented by the received evaluation index information.

28. The server apparatus according to claim 27, wherein the control section is configured to transmit the evaluation index information to each information processing apparatus of the plurality of information processing apparatuses by broadcast.

Images