Information processing device, information processing method, and program

Abstract

Provided is an information processing device comprising: an acquiring unit that acquires an image of the ground around a moving body and information indicating the behavior of the moving body when the moving body moves; an estimating unit that estimates the type of the ground on the basis of the image; and a predicting unit that predicts the behavior of the moving body when the moving body moves on the ground of the type estimated by the estimating unit on the basis of a regression model for the type of the ground, and updates the regression model for the type of the ground on the basis of the predicted behavior of the moving body and the behavior of the moving body acquired by the acquiring unit. This allows the behavior of the moving body to be estimated more appropriately.

Description

Information Processing Apparatus, Information Processing Method, and Program

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

[0002] Patent Document 1 discloses a technique for estimating the behavior of a moving object in a first environment by inputting environment analysis data based on the state of the first environment into a model for estimating the behavior of the moving object in the first environment.

[0003] International Publication No. 2022 / 091305

[0004] However, in the technique described in Patent Document 1, for example, there may be room for more appropriately estimating the behavior of the moving object.

[0005] An object of the present disclosure is to provide a technique capable of more appropriately estimating the behavior of a moving object in view of the above-described problems.

[0006] In a first aspect according to the present disclosure, an acquisition unit that acquires an image of the ground around a moving object and information indicating the behavior of the moving object when the moving object moves, an estimation unit that estimates the type of the ground based on the image, and the moving object moves on the ground of the type estimated by the estimation unit. A prediction unit that predicts the behavior of the moving object based on a regression model for the type of the ground and updates the regression model for the type of the ground based on the predicted behavior of the moving object and the behavior of the moving object acquired by the acquisition unit is provided.

[0007] Further, in a second aspect according to the present disclosure, an image of the ground around a moving object and information indicating the behavior of the moving object when the moving object moves are acquired, the type of the ground is estimated based on the image, and the moving object moves on the ground of the estimated type. An information processing method is provided that predicts the behavior of the moving object based on a regression model for the type of the ground and updates the regression model for the type of the ground based on the predicted behavior of the moving object and the acquired behavior of the moving object.

[0008] Furthermore, a third aspect of the present disclosure provides a program that causes a computer to perform the following processes: acquire an image of the ground around a moving object and information indicating the behavior of the moving object when it moves; estimate the type of ground based on the image; predict the behavior of the moving object when it moves on the estimated type of ground based on a regression model for the type of ground; and update the regression model for the type of ground based on the predicted behavior of the moving object and the acquired behavior of the moving object.

[0009] From one perspective, it is possible to more accurately estimate the behavior of a moving object.

[0010] This figure shows an example of the configuration of the information processing device related to this disclosure. This figure shows an example of the configuration of the mobile device related to this disclosure. This figure shows an example of the hardware configuration of the information processing device related to this disclosure. This flowchart shows an example of the processing performed by the information processing device related to this disclosure. This figure shows an example of the information stored in the regression model DB (database) related to this disclosure.

[0011] The principles of this disclosure will be described with reference to several exemplary embodiments. These embodiments are described for illustrative purposes only and should be understood as helping those skilled in the art to understand and implement this disclosure without implying any limitation on the scope of this disclosure. The disclosures described herein may be implemented in various ways other than those described below.

[0012] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meanings as those generally understood by those skilled in the art to which this disclosure belongs.

[0013] Embodiments of the present disclosure will be described below with reference to the drawings. Each drawing is merely illustrative for illustrating one or more embodiments. Each drawing may be associated not only with one specific embodiment but also with one or more other embodiments. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments not explicitly shown or described. Not all features or steps shown in any one drawing are necessarily required to illustrate an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any of the drawings may be changed as appropriate.

[0014] (Embodiment 1) <Configuration> Referring to Figure 1, the configuration of the information processing device 10 according to the embodiment will be described. Figure 1 is a diagram showing an example of the configuration of the information processing device 10 according to the present disclosure. The information processing device 10 has an acquisition unit 11, an estimation unit 12, and a prediction unit 13. Each of these units may be realized through the cooperation of one or more programs installed in the information processing device 10 and hardware such as the processor and memory of the information processing device 10.

[0015] The acquisition unit 11 acquires an image of the ground around the moving object and information indicating the behavior of the moving object when it moves. The estimation unit 12 estimates the type of ground based on the image acquired by the acquisition unit 11.

[0016] The prediction unit 13 predicts the behavior of a moving object when it moves on a type of ground estimated by the estimation unit 12, based on a regression model for the ground type. The prediction unit 13 also updates the regression model for the estimated ground type based on the predicted behavior of the moving object and the behavior of the moving object acquired by the acquisition unit 11. This allows for a more accurate estimation of the moving object's behavior, for example.

[0017] (Embodiment 2) <System Configuration> Next, with reference to Figure 2, the configuration of the mobile body 1 according to the embodiment will be described. Figure 2 is a diagram showing an example of the configuration of the mobile body 1 according to the present disclosure. In the example of Figure 2, the mobile body 1 has an information processing device 10, a shooting device 20, a sensor 30, and a control device 40. In the example of Figure 2, the information processing device 10, the shooting device 20, the sensor 30, and the control device 40 are connected so that they can communicate via a network N. Note that the number of information processing devices 10, shooting devices 20, sensors 30, and control devices 40 is not limited to the example in Figure 2.

[0018] Examples of network N include, for example, in-vehicle networks, LANs (Local Area Networks), wireless LANs, buses, the Internet, and mobile communication systems. Examples of mobile communication systems include, for example, fifth-generation mobile communication systems (5G), sixth-generation mobile communication systems (6G, Beyond 5G), fourth-generation mobile communication systems (4G), and third-generation mobile communication systems (3G).

[0019] Mobile unit 1 may be, for example, a vehicle or robot that travels on land by wheels, such as a road vehicle, construction vehicle, military vehicle, or industrial vehicle. Alternatively, mobile unit 1 may be a robot that moves on land by means of mechanical legs, for example. Mobile unit 1 may travel on rough roads or off-road terrain, for example.

[0020] The information processing device 10 is, for example, an ECU (Electronic Control Unit), a computer, a server, a cloud server, or the like. The information processing device 10 controls the movement of the mobile body 1 using the control device 40, for example, based on automatic or manual operation.

[0021] The imaging device 20 is an imaging device that captures an image of the ground in the direction of movement of the moving object 1. The imaging device 20 may be, for example, a camera (digital camera) that takes still images. Alternatively, the imaging device 20 may be, for example, a LiDAR (Light Detection and Ranging) that irradiates an object with laser light, measures the time it takes for the light to reflect back, and measures the distance and direction to the object. Alternatively, the imaging device 20 may be, for example, an infrared camera that takes still images using infrared light.

[0022] Sensor 30 is a sensor for measuring the state of the moving object 1. Sensor 30 may be, for example, a GNSS (Global Navigation Satellite System) receiver, an inertial measurement unit (IMU), or a motor encoder that detects and encodes the rotation state of a motor.

[0023] The control device 40 is a device that controls the movement of the mobile body 1 based on automatic driving or manual operation by the driver and the predicted behavior of the mobile body 1 by the information processing device 10. The control device 40 may, for example, control the engine, brakes, and steering wheel of the mobile body 1.

[0024] <Hardware Configuration> Figure 3 shows an example of the hardware configuration of the information processing device 10 according to this disclosure. In the example in Figure 3, the information processing device 10 (computer 100) includes a processor 101, a memory 102, and a communication interface 103. These parts may be connected by a bus or the like. The memory 102 stores at least a portion of the program 104. The communication interface 103 includes an interface necessary for communication with other network elements.

[0025] When program 104 is executed in cooperation with the processor 101 and memory 102, etc., the computer 100 performs at least some of the processing of embodiments of the present disclosure. Memory 102 may be of any type. Memory 102 may, in non-limiting examples, be a non-temporary computer-readable storage medium. Memory 102 may also be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. Although only one memory 102 is shown for computer 100, computer 100 may have several physically different memory modules. Processor 101 may be of any type. Processor 101 may include one or more general-purpose computers, dedicated computers, microprocessors, digital signal processors (DSPs), and, in non-limiting examples, processors based on multicore processor architectures. Computer 100 may have multiple processors, such as application-specific integrated circuit chips that are time-dependent to a clock that synchronizes the main processor.

[0026] Embodiments of the present disclosure may be implemented in hardware or in dedicated circuitry, software, logic, or any combination thereof. Some embodiments may be implemented in hardware, while others may be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device.

[0027] This disclosure also provides at least one computer program product tangibly stored on a non-temporary computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions contained in a program module, and is executed on a device on a target real or virtual processor to perform the processes or methods of this disclosure. The program module includes routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. The functionality of the program module may be combined or divided among the program module as desired in various embodiments. The machine-executable instructions of the program module can be executed on a local or distributed device. On a distributed device, the program module can reside on both local and remote storage media.

[0028] Program code for performing the methods of this disclosure may be written in any combination of one or more programming languages. These program codes are provided to a processor or controller of a general-purpose computer, a dedicated computer, or other programmable data processing device. When the program code is executed by the processor or controller, the functions / operations in the flowchart and / or block diagrams it implements are performed. The program code runs entirely on the machine, partially on the machine, as a standalone software package, partially on the machine, partially on a remote machine, or entirely on a remote machine or server.

[0029] Programs can be stored and supplied to a computer using various types of non-temporary computer-readable media. Non-temporary computer-readable media include various types of tangible recording media. Examples of non-temporary computer-readable media include magnetic recording media, magneto-optical recording media, optical disc media, and semiconductor memory. Magnetic recording media include, for example, flexible disks, magnetic tapes, and hard disk drives. Magneto-optical recording media include, for example, magneto-optical disks. Optical disc media include, for example, Blu-ray discs, CD (Compact Disc)-ROM (Read Only Memory), CD-R (Recordable), and CD-RW (ReWritable). Semiconductor memory includes, for example, solid-state drives, mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (random access memory). Programs may also be supplied to a computer using various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. Temporary computer-readable media can supply programs to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0030] <Processing> Next, an example of the processing of the information processing device 10 according to the embodiment will be described with reference to Figures 4 and 5. Figure 4 is a flowchart showing an example of the processing of the information processing device 10 according to the present disclosure. Figure 5 is a diagram showing an example of information stored in the regression model DB (database) 501 according to the present disclosure. Note that the processing in Figure 4 may be executed at intervals such as periodic intervals. Note that the following processing order can be changed as appropriate, as long as it is not contradictory. Also, the processing cycle of step S101 and the processing cycles from step S102 to step S105 do not have to be the same.

[0031] In step S101, the acquisition unit 11 acquires an image of the ground around the mobile body 1. Here, the mobile body 1 may move on ground where roads are not well paved, for example. In this case, the mobile body 1 may move in sites such as civil engineering and construction, forestry, mining, transportation on unpaved roads, agriculture, defense, disaster response, and lunar exploration. Alternatively, the mobile body 1 may move on roads that are paved but slippery due to rain, snow, etc.

[0032] Next, the estimation unit 12 estimates the type of ground surrounding the mobile body 1 based on the image acquired by the acquisition unit 11 (step S102). Here, the estimation unit 12 may use supervised learning, such as a CNN (Convolutional Neural Network), to estimate (infer) the type of ground at each location around the mobile body 1 from the image of the ground surrounding the mobile body 1.

[0033] Next, the prediction unit 13 predicts the behavior of the mobile body 1 when it moves over the type of ground estimated by the estimation unit 12, based on a regression model for the type of ground (step S103). Here, the prediction unit 13 may predict the behavior of the mobile body 1 using a regression model for the type of ground recorded in the regression model DB 501 shown in Figure 5. In the example in Figure 5, the regression model DB 501 records regression models associated with the type of ground.

[0034] The types of ground may include, for example, mud, sand, hard soil, soft soil, grass, wood, water, asphalt, rock, and bedrock. The regression model may be, for example, an equation estimated by statistical methods. In this case, the regression model may use, for example, the behavior instructed to the mobile body 1 (instruction behavior) as an explanatory variable (independent variable). The instruction behavior of the mobile body 1 may include, for example, the speed of movement, direction of movement, and trajectory of movement of the mobile body 1. Furthermore, the regression model may also use, for example, the position of the mobile body 1 and the shape of the ground (for example, the angle of inclination and the degree of unevenness) as explanatory variables. The regression model may also use the predicted behavior of the mobile body 1 on a certain type of ground as the dependent variable (dependent variable). The predicted behavior of the mobile body 1 may include, for example, the speed of movement, direction of movement, trajectory, change in posture, slip, vibration, and power consumption (fuel consumption) of the mobile body 1.

[0035] The prediction unit 13 may, for example, repeat the process in step S103 to calculate an instructed behavior that matches the automatically or manually instructed behavior (desired behavior). Then, the prediction unit 13 may, for example, control the amounts of the accelerator, brake, and steering wheel of the moving body 1 using the control device 40 according to the calculated instructed behavior.

[0036] Next, the acquisition unit 11 acquires information indicating the actual behavior of the moving body 1 when it moves (step S104). Here, the acquisition unit 11 may acquire, for example, information indicating the moving speed of the moving body 1 and the slip of the moving body 1, as measured by the sensor 30.

[0037] Next, the prediction unit 13 determines whether or not to update the regression model for the estimated ground type (step S105).

[0038] (Example of updating at specific intervals, etc.) The prediction unit 13 may, for example, determine to update the regression model when the acquisition unit 11 has acquired a specific number or more of data (images and information showing the actual behavior of the moving object 1) for a specific type of ground estimated by the estimation unit 12. The prediction unit 13 may then update the regression model corresponding to the specific type based on the data. This allows the regression model to be updated at specific intervals (for example, every minute) even if there are uneven ground areas with poor soil conditions at the same site, thus enabling a more appropriate estimation of the predicted behavior.

[0039] (Example of updating when prediction accuracy is below a threshold) The prediction unit 13 may, for example, determine to update the regression model if the prediction accuracy of the actual behavior of the moving object 1 relative to the regression model is below a threshold. In this case, the prediction unit 13 may, for example, predict a specific number of predicted behaviors using a regression model that predicts the probability (confidence level) of each of multiple predicted behaviors for the estimated type of ground.

[0040] Furthermore, the prediction unit 13 may determine to update the regression model if, for example, the prediction accuracy of the actual behavior of the moving object 1 measured by the sensor 30 and the predicted number of predicted behaviors is below a threshold. The prediction accuracy in this disclosure may be, for example, the root mean squared error (RMSE) or likelihood between the predicted behavior and the actual behavior. The prediction unit 13 may also determine to update the regression model if, for example, the actual behavior of the moving object 1 measured by the sensor 30 is not contained within the space that contains the predicted number of predicted behaviors, or if the degree of deviation from that space is at a threshold market.

[0041] (Example of determining the method for judging whether to update based on the type of ground) The prediction unit 13 may determine, for example, a method for judging whether to update the regression model based on the type of ground estimated by the estimation unit 12. Thereby, for example, when the ground is covered with snow and the state under the snow is unknown, or when the ground is sandy and the difference in ground strength is relatively large, etc., the update frequency of the regression model can be increased. Therefore, the predicted behavior can be estimated more appropriately.

[0042] In this case, for example, when the ground estimated by the estimation unit 12 is of the first type, the prediction unit 13 may update the regression model for the type of ground when a specific number or more of data is acquired by the acquisition unit 11. And, for example, when the ground estimated by the estimation unit 12 is of the second type different from the first type, the prediction unit 13 may update the regression model for the type of ground when the prediction accuracy of the behavior of the moving body 1 with respect to the regression model for the type of ground is below a threshold value.

[0043] (Example of determining the threshold value for update based on the type of ground) The prediction unit 13 may determine, for example, a threshold value for updating the regression model based on the type of ground estimated by the estimation unit 12. Thereby, for example, in the case of a snow-covered ground, a sandy ground, etc., the update frequency of the regression model can be increased. Therefore, the predicted behavior can be estimated more appropriately.

[0044] In this case, for example, the prediction unit 13 determines a specific threshold value based on the type of ground estimated by the estimation unit 12, and may update the regression model for the type of ground when the prediction accuracy of the behavior of the moving body with respect to the regression model for the type of ground is below the specific threshold value.

[0045] When the regression model is not updated (NO in step S105), the process ends. On the other hand, when the regression model is updated (YES in step S105), the prediction unit 13 updates the regression model for the estimated type of ground so that the difference between the predicted behavior of the moving body 1 and the behavior of the moving body 1 acquired by the acquisition unit 11 becomes small (step S106), and the process ends.

[0046] This allows for the calculation of appropriate predicted behavior even in environments where the estimation unit 12 might misclassify the environment as a different type. For example, even if the estimation unit 12 estimates the surface as "soil" from an image, but the actual surface is "sand," which is more slippery than "soil," the vehicle's movement can still be appropriately controlled.

[0047] Furthermore, even for road surfaces and other conditions not included in the pre-trained data, the system can appropriately calculate predicted behavior. For example, even if the estimation unit 12 estimates the surface as "sand" from the image, the system can still appropriately control the vehicle's movement even if the sandy surface is more slippery than the "sand" in the pre-trained data.

[0048] The prediction unit 13 may also save the regression models for each type of ground recorded in the regression model DB 501 as default values. Furthermore, the prediction unit 13 may reset the regression models for each type of ground recorded in the regression model DB 501 to their default values ​​when the mobile unit 1 moves to a different site, or when a reset operation is performed by the administrator of the mobile unit 1.

[0049] (Example of reusing regression models for other types) The prediction unit 13 may update the regression model for ground type estimated by the estimation unit 12 to the regression model with the highest prediction accuracy among the regression models for each type. This allows the system to appropriately calculate the predicted behavior even if, for example, sandy ground after rain has dried is misestimated (classified) as hard soil ground by the estimation unit 12, using the updated regression model for hard soil.

[0050] In this case, the prediction unit 13 may predict the behavior of the moving body 1 based on each regression model recorded in the regression model DB 501, for example. The prediction unit 13 may then record in the regression model DB 501, for example, the regression model that has the highest prediction accuracy between the predicted behavior and the actual behavior of the moving body 1 measured by the sensor 30, as the regression model for the type of ground estimated by the estimation unit 12.

[0051] (Example of mixing multiple regression models) The prediction unit 13 may update the regression model for the type of ground estimated by the estimation unit 12 to a regression model based on multiple regression models from among the regression models for each type. This allows for more appropriate calculation of predicted behavior, for example, even if the ground estimated as sand by the estimation unit 12 at a particular site is actually a mixture of sand, soil, grass, etc.

[0052] In this case, the prediction unit 13 may predict the behavior of the moving object 1 based on each regression model recorded in the regression model DB 501, for example. The prediction unit 13 may also record a model obtained by weighting and adding the prediction accuracy of each regression model in the regression model DB 501 as a regression model for the type of ground estimated by the estimation unit 12.

[0053] Furthermore, the prediction unit 13 may, for example, record a model obtained by weighting and adding the probabilities (confidence level, likelihood) of each of the multiple ground types estimated by the estimation unit 12 in the regression model DB 501 as a regression model for the ground types estimated by the estimation unit 12. In this case, for example, if the estimation unit 12 estimates the likelihood of sand to be 0.5, the likelihood of soil to be 0.3, and the likelihood of grass to be 0.2, the regression model pc for sandy ground is calculated by the following equation (1). １ ~p ３ Each of these can also be the initial value of a regression model for sand, soil, and grass ground. ｃ = 0.5p １ +0.3p ２ +0.2p ３ ... (1)

[0054] (Example of transfer learning of regression model) The prediction unit 13 may update the regression model for the type of ground estimated by the estimation unit 12 with a regression model that has been transfer-learned based on the actual behavior of the moving object 1 acquired by the acquisition unit 11 measured by the sensor 30. This allows for more appropriate calculation of predicted behavior, for example, even if the ground estimated to be sand by the estimation unit 12 at a particular site is more slippery than sand that has been learned in advance.

[0055] (Example of determining an update method based on the type of ground) The prediction unit 13 may determine a method for updating the regression model for the type of ground based on the type of ground estimated by the estimation unit 12. This allows the model to be updated using an appropriate method depending on the type of ground.

[0056] In this case, the prediction unit 13 may, for example, update the regression model for the first type by reusing the regression models for the other types described above if the ground estimated by the estimation unit 12 is of the first type. The prediction unit 13 may also, for example, update the regression model for the second type by mixing the multiple regression models described above if the ground estimated by the estimation unit 12 is of the second type. The prediction unit 13 may also, for example, update the regression model for the third type by transferring the regression models described above if the ground estimated by the estimation unit 12 is of the third type.

[0057] <Other> The driving behavior of vehicles when traveling on uneven or undulating roads depends heavily on the shape and condition of the road surface. Therefore, when operating vehicles on ground where there are types (classes) that have not been previously trained, the predictive performance of the vehicle's driving behavior will decrease. For example, if the system has been trained with data including the soil class, but wet soil or mud has not been trained, the predictive performance of the driving behavior will decrease. Also, for example, if the system has been trained with data including the sand class, but the slip on the sand the system is about to operate on is greater than the slip on the sand the system has been trained on, the predictive performance of the driving behavior will decrease.

[0058] To improve prediction performance, one might consider pre-training the model on a wide variety of ground (road) types. However, since off-road environments have diverse road surface types and conditions, it is extremely difficult to pre-define categories that cover all road surface types and conditions and collect training data accordingly.

[0059] Furthermore, to improve prediction performance, it is conceivable to retrain the estimation unit 12 by collecting new training data for each new environment. However, for example, collecting data and annotating images (manually assigning correct labels) for a new environment and then retraining the deep learning model is costly.

[0060] On the other hand, according to this disclosure, the prediction unit 13 updates the regression model associated with the type by using the type label estimated by the estimation unit 12 as is, thereby improving the prediction accuracy in the current environment. Therefore, if the operator (developer) of the information processing device 10 defines only typical types, the regression model for each type will be adjusted to suit the local environment based on local data. As a result, the effort required for image annotation work can be reduced. In addition, the time required for training and inference of the estimation unit 12, which is a classifier, can be shortened. For example, instead of defining detailed ground types such as loosely deposited sand, compacted sand, wet sand, large particle size gravel, small particle size gravel, etc., only broad types such as smooth road surface, road surface with small irregularities, road surface with large irregularities, muddy road surface, and impassable road surface can be defined.

[0061] <Modifications> The information processing device 10 may be a device contained in a single housing, but the information processing device 10 of this disclosure is not limited to this. Each part of the information processing device 10 may be realized by cloud computing, which is composed of one or more computers, for example. The information processing device 10 and the imaging device 20 may also be housed in the same housing and configured as an integrated information processing device. Furthermore, the imaging device 20 may perform processing on at least some of the functions of each functional part of the information processing device 10. Such information processing devices 10 are also included as examples of the "information processing device" of this disclosure.

[0062] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0063] Some or all of the above embodiments may also be described as follows, but are not limited to the following. Some or all of the elements (e.g., configuration and function) described in each appendix that is dependent on Appendix 1 may be dependent on other independent appendices of other categories in a similar manner. Some or all of the elements described in any appendix may be applied to various hardware, software, recording means, systems, and methods for recording software. (Appendix 1) An information processing device having: an acquisition unit that acquires an image of the ground around a moving object and information indicating the behavior of the moving object when the moving object moves; an estimation unit that estimates the type of ground based on the image; and a prediction unit that predicts the behavior of the moving object when the moving object moves on the type of ground estimated by the estimation unit, based on a regression model for the type of ground, and updates the regression model for the type of ground based on the predicted behavior of the moving object and the behavior of the moving object acquired by the acquisition unit. (Note 2) The information processing device according to Note 1, wherein the prediction unit updates the regression model for the type of ground when a specific number or more of data are acquired by the acquisition unit for the specific type of ground estimated by the estimation unit, and when the prediction accuracy of the behavior of the moving object with respect to the regression model for the type of ground is below a threshold, in at least one of the above cases. (Note 3) The information processing device according to Note 2, wherein the prediction unit updates the regression model for the type of ground when a specific number or more of data are acquired by the acquisition unit if the ground estimated by the estimation unit is a first type, and updates the regression model for the type of ground when the prediction accuracy of the behavior of the moving object with respect to the regression model for the type of ground is below a threshold, if the ground estimated by the estimation unit is a second type. (Note 4) The information processing device according to Note 2, wherein the prediction unit determines a specific threshold based on the type of ground estimated by the estimation unit, and updates the regression model for the type of ground when the prediction accuracy of the behavior of the moving object with respect to the regression model for the type of ground is below the specific threshold.(Note 5) The information processing device according to Note 1 or 2, wherein the prediction unit updates the regression model for the type of ground estimated by the estimation unit to the regression model with the highest prediction accuracy among a plurality of regression models for each type. (Note 6) The information processing device according to Note 1 or 2, wherein the prediction unit updates the regression model for the type of ground estimated by the estimation unit to a regression model based on a plurality of regression models among the regression models for each type. (Note 7) The information processing device according to Note 1 or 2, wherein the prediction unit updates the regression model for the type of ground estimated by the estimation unit to a regression model that has been transfer-learned based on the behavior of the moving object acquired by the acquisition unit. (Note 8) The information processing device according to Note 1 or 2, wherein the prediction unit determines a method for updating the regression model for the type of ground based on the type of ground estimated by the estimation unit. (Note 9) An information processing method comprising: acquiring an image of the ground around a moving object and information indicating the behavior of the moving object when the moving object moves; estimating the type of ground based on the image; predicting the behavior of the moving object when it moves on the estimated type of ground based on a regression model for the type of ground; and updating the regression model for the type of ground based on the predicted behavior of the moving object and the acquired behavior of the moving object. (Note 10) A program that causes a computer to perform the following processing: acquiring an image of the ground around a moving object and information indicating the behavior of the moving object when the moving object moves; estimating the type of ground based on the image; predicting the behavior of the moving object when it moves on the estimated type of ground based on a regression model for the type of ground; and updating the regression model for the type of ground based on the predicted behavior of the moving object and the acquired behavior of the moving object.

[0064] This application claims priority based on Japanese Patent Application No. 2024-163821, filed on 20 September 2024, and incorporates all of its disclosures herein.

[0065] 1 Mobile device 10 Information processing device 11 Acquisition unit 12 Estimation unit 13 Prediction unit 20 Imaging device 30 Sensor 40 Control device

Claims

1. An information processing device comprising: an acquisition unit that acquires an image of the ground around a moving object and information indicating the behavior of the moving object when it moves; an estimation unit that estimates the type of ground based on the image; and a prediction unit that predicts the behavior of the moving object when it moves on the type of ground estimated by the estimation unit, based on a regression model for the type of ground, and updates the regression model for the type of ground based on the predicted behavior of the moving object and the behavior of the moving object acquired by the acquisition unit.

2. The information processing apparatus according to claim 1, wherein the prediction unit updates the regression model for the type of ground estimated by the estimation unit when a certain number or more of data are acquired by the acquisition unit for that specific type of ground, and when the prediction accuracy of the behavior of the moving object with respect to the regression model for the type of ground is below a threshold, in at least one of the above cases.

3. The information processing apparatus according to claim 2, wherein the prediction unit updates the regression model for the type of ground when the acquisition unit has acquired a specific number of data points or more if the ground estimated by the estimation unit is of a first type, and updates the regression model for the type of ground when the prediction accuracy of the behavior of the moving object with respect to the regression model for the type of ground is below a threshold if the ground estimated by the estimation unit is of a second type.

4. The information processing apparatus according to claim 2, wherein the prediction unit determines a specific threshold based on the type of ground estimated by the estimation unit, and updates the regression model for the type of ground when the prediction accuracy of the behavior of the moving object with respect to the regression model for the type of ground is less than or equal to the specific threshold.

5. The information processing apparatus according to claim 1 or 2, wherein the prediction unit updates the regression model for the type of ground estimated by the estimation unit to the regression model with the highest prediction accuracy among a plurality of regression models for each type.

6. The information processing apparatus according to claim 1 or 2, wherein the prediction unit updates the regression model for the type of ground estimated by the estimation unit to a regression model based on a plurality of regression models among the regression models for each type.

7. The information processing apparatus according to claim 1 or 2, wherein the prediction unit updates the regression model for the type of ground estimated by the estimation unit to a regression model that has been transfer-learned based on the behavior of the moving object acquired by the acquisition unit.

8. The information processing apparatus according to claim 1 or 2, wherein the prediction unit determines a method for updating the regression model for the type of ground based on the type of ground estimated by the estimation unit.

9. An information processing method comprising: acquiring an image of the ground around a moving object and information indicating the behavior of the moving object when it moves; estimating the type of ground based on the image; predicting the behavior of the moving object when it moves on the estimated type of ground based on a regression model for the type of ground; and updating the regression model for the type of ground based on the predicted behavior of the moving object and the acquired behavior of the moving object.

10. The information processing method according to claim 9, wherein the regression model for the type of ground is updated in at least one of the following cases: when a certain number or more of data have been obtained for the estimated specific type of ground, and when the prediction accuracy of the behavior of the moving object with respect to the regression model for the type of ground is below a threshold.

11. The information processing method according to claim 10, wherein if the estimated ground is of type 1, the regression model for the ground type is updated when a certain number or more of data have been acquired, and if the estimated ground is of type 2, the regression model for the ground type is updated when the prediction accuracy of the behavior of the moving object relative to the regression model for the ground type is below a threshold.

12. The information processing method according to claim 10, comprising determining a specific threshold based on the estimated type of ground, and updating the regression model for the type of ground if the prediction accuracy of the behavior of the moving object with respect to the regression model for the type of ground is less than or equal to the specific threshold.

13. The information processing method according to claim 9 or 10, wherein the regression model for the estimated type of ground is updated to the regression model with the highest prediction accuracy among a plurality of regression models for each type.

14. The information processing method according to claim 9 or 10, which updates the estimated regression model for the type of ground to a regression model based on a plurality of regression models among the regression models for each type.

15. The information processing method according to claim 9 or 10, wherein the regression model for the estimated type of ground is updated with a regression model that has been transfer-learned based on the acquired behavior of the moving body.

16. The information processing method according to claim 9 or 10, which determines a method for updating a regression model for the type of ground based on the estimated type of ground.

17. A program that causes a computer to perform the following processes: acquire an image of the ground around a moving object and information indicating the behavior of the moving object when it moves; estimate the type of ground based on the image; predict the behavior of the moving object when it moves on the estimated type of ground based on a regression model for the type of ground; and update the regression model for the type of ground based on the predicted behavior of the moving object and the acquired behavior of the moving object.

18. The program according to claim 17, which causes a computer to perform a process to update the regression model for the type of ground when a certain number or more of data have been obtained for the estimated specific type of ground, and when the prediction accuracy of the behavior of a moving object with respect to the regression model for the type of ground is below a threshold, in at least one of the above cases.

19. The program according to claim 18, which causes a computer to perform the following processing: if the estimated ground is of type 1, it updates the regression model for the ground type when a certain number or more of data have been acquired; and if the estimated ground is of type 2, it updates the regression model for the ground type when the prediction accuracy of the behavior of the moving object relative to the regression model for the ground type is below a threshold.

20. The program according to claim 18, which causes a computer to perform a process of determining a specific threshold based on the estimated type of ground, and updating the regression model for the type of ground if the prediction accuracy of the behavior of the moving object with respect to the regression model for the type of ground is less than or equal to the specific threshold.

Images