Information processing method and information processing device

Abstract

An information processing device according to the present embodiment acquires a captured image, which is an image of a wheel of a target vehicle captured by a camera disposed above a suspension of the target vehicle, and estimates a stroke amount of the suspension from the acquired captured image.

Description

Information processing method and information processing device

[0001] The present invention relates to an information processing method and an information processing device.

[0002] Various technologies have been known for controlling the suspensions of vehicles such as automobiles and detecting the stroke amounts of the suspensions. For example, Patent Document 1 listed below discloses a suspension control device that prevents the front bumper from rubbing against the front bumper when descending a step without reducing ride comfort, even when a sufficient approach angle cannot be set or when a large step is encountered. That is, the suspension control device discloses a suspension control device that, when a downward step is detected by a camera or the like, reduces the damping force when the variable dampers of the left and right front wheels extend and increases the damping force when the variable dampers compress, compared to when a downward step is not detected.

[0003] JP 2010-184512 A

[0004] However, the above-described conventional technology has a problem in that it is not possible to detect the suspension stroke amount for vehicles that are not equipped with a stroke sensor.

[0005] In one aspect, the present invention has been made in consideration of the above circumstances, and an object of the present invention is to provide an information processing method and an information processing device that can estimate the suspension stroke amount for a vehicle that is not equipped with a stroke sensor.

[0006] In order to solve the above-mentioned problems, in an information processing method according to one aspect of the present invention, a computer executes the steps of acquiring an image of the wheels of a target vehicle using a camera positioned above the suspension of the target vehicle, and estimating the stroke amount of the suspension from the acquired image.

[0007] According to the present invention, it is possible to provide an information processing method and an information processing device that can estimate the suspension stroke amount for a vehicle that is not equipped with a stroke sensor.

[0008] 1 is a block diagram showing a schematic configuration of a vehicle equipped with an information processing device according to an embodiment. FIG. 10 is a schematic diagram showing an example of a hardware configuration of an information processing device according to an embodiment. FIG. 11 is a schematic diagram showing an example of a software configuration of an information processing device according to an embodiment. FIG. 12 is a flowchart illustrating details of an example of the processing procedure of step S140 of FIG. 10.

[0009] An embodiment according to one aspect of the present invention (hereinafter also referred to as "the present embodiment") will be described below with reference to the drawings. However, the present embodiment described below is merely an example of the present invention in all respects. Needless to say, various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiment may be appropriately adopted. Note that, although data appearing in the present embodiment are described in natural language, more specifically, they are specified using pseudo-language, commands, parameters, machine language, etc. that can be recognized by a computer.

[0010] §1 Application Example FIG. 1 is a block diagram showing the schematic configuration of a target vehicle TV equipped with an information processing device (information processing device 1) according to this embodiment. The target vehicle TV is an example of the "target vehicle" of the present invention. The target vehicle TV does not have a stroke sensor that detects the stroke amount SA of the suspension. However, the target vehicle TV has the following configuration. That is, the target vehicle TV illustrated in FIG. 1 includes an information processing device 1, a camera 2, and a vehicle control system 3.

[0011] The camera 2 is disposed above the suspension of the target vehicle TV and captures images below it, particularly of the wheels of the target vehicle TV. The camera 2 may be, for example, a camera equipped with an imaging element such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), an ultrasonic camera, or an infrared camera. A plurality of cameras 2 may be provided on the target vehicle TV. The cameras 2 may be disposed above the suspensions of the left and right front wheels (for example, below the left and right door mirrors), or above the suspensions of the left and right rear wheels. The camera 2 and the information processing device 1 may be connected, for example, via a controller area network (CAN) or other in-vehicle LAN, and may communicate with each other (transmit and receive information). The camera 2 periodically captures images of the wheels of the target vehicle TV and periodically outputs the captured images (captured images CI) to the information processing device 1.

[0012] The vehicle control system 3 is a system that controls the driving of the target vehicle TV, for example, by controlling the acceleration, steering, and deceleration (braking) of the target vehicle TV. The vehicle control system 3 may be realized by a so-called vehicle control ECU (Electronic Control Unit). In this embodiment, the vehicle control system 3 can control the driving of the target vehicle TV by communicating with the information processing device 1 (mutually transmitting and receiving information). For example, the information processing device 1 outputs a signal (control signal CS) to the vehicle control system 3 that specifies or changes the torque value DT of the drive shaft of the target vehicle TV. The vehicle control system 3 can control the torque value DT of the drive shaft of the target vehicle TV in accordance with the control signal CS output from the information processing device 1. Furthermore, for example, the information processing device 1 obtains steering angle information IF indicating the steering angle RA of the target vehicle TV (the steering angles RA of each of the multiple wheels) from the vehicle control system 3. The vehicle control system 3 and the information processing device 1 may be connected, for example, via an in-vehicle LAN, and transmit and receive related information to each other.

[0013] The information processing device 1 estimates the suspension stroke amount SA of the target vehicle TV from the captured image CI (i.e., an image of the wheels of the target vehicle TV captured from above the suspension) captured by the camera 2. Therefore, the information processing device 1 can detect (estimate) the suspension stroke amount SA for a target vehicle TV that is not equipped with a stroke sensor. The information processing device 1, whose overview has been explained above, will be explained in detail below using Figures 2 to 11.

[0014] §2 Configuration Example [Hardware Configuration] Fig. 2 schematically illustrates an example of the hardware configuration of the information processing device 1 according to this embodiment. As shown in Fig. 2, the information processing device 1 according to this embodiment is a computer to which a control unit 11, a storage unit 12, a communication interface 13, an external interface 14, an input device 15, an output device 16, and a drive 17 are electrically connected. Note that in Fig. 2, the communication interface and the external interface are referred to as a "communication I / F" and an "external I / F."

[0015] The control unit 11 includes a hardware processor such as a central processing unit (CPU), a random access memory (RAM), and a read-only memory (ROM), and is configured to execute information processing based on programs and various data. The CPU is an example of a processor resource. A graphics processing unit (GPU) may be used as the processor resource instead of or in addition to the CPU. The storage unit 12 is an example of a memory resource, and is configured, for example, with a hard disk drive or a solid-state drive. In this embodiment, the storage unit 12 stores various information such as an information processing program PG and an estimation model EM.

[0016] The information processing program PG is a program for causing the information processing device 1 to execute information processing (see FIG. 10 and the like) described below for estimating a stroke amount SA of the suspension of the target vehicle TV from the captured image CI. Details will be described later, but in this embodiment, the information processing device 1 executes the above-described information processing using, for example, an estimation model EM. However, it is not essential for the information processing device 1 to estimate the stroke amount SA using the estimation model EM, and the information processing device 1 may estimate the stroke amount SA on a rule-based basis. The information processing program PG includes a series of instructions for the information processing.

[0017] The estimation model EM is a trained machine learning model (in other words, a learned statistical model). For example, when a captured image CI is given, the estimation model EM is configured to perform an estimation task of estimating any of the following three items (in other words, to output an output value corresponding to the result of performing the estimation task). That is, the estimation model EM performs an estimation task of estimating any of (1) the "stroke amount SA of the suspension of the target vehicle TV," (2) the "contour shape OS of the wheels of the target vehicle TV (imaged in the captured image CI)," and (3) the "center position CP of the wheels (or wheels) of the target vehicle TV (imaged in the captured image CI)." The machine learning of the estimation model EM will be described in detail below.

[0018] The communication interface 13 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, or the like, and is an interface for performing wired or wireless communication via a network. As described above, the communication interface 13 may be an interface for performing communication via a CAN or other in-vehicle LAN. The information processing device 1 may use this communication interface 13 to perform data communication via a network with other information processing devices (e.g., the camera 2, the vehicle control system 3, etc.). The external interface 14 is, for example, a USB (Universal Serial Bus) port, a dedicated port, or the like, and is an interface for connecting to an external device. The type and number of external interfaces 14 may be selected appropriately depending on the type and number of external devices to be connected.

[0019] For example, the information processing device 1 is connected to another information processing device via at least one of the communication interface 13 and the external interface 14, and performs the following communications with the other information processing device. That is, the information processing device 1 is connected to a camera 2 and acquires "a captured image CI of the wheels of the target vehicle TV captured by the camera 2 arranged above the suspension of the target vehicle TV." The information processing device 1 is also connected to a vehicle control system 3 and outputs a control signal CS to the vehicle control system 3, which specifies or changes the torque value DT of the drive shaft of the target vehicle TV. The information processing device 1 is also connected to the vehicle control system 3 and acquires steering angle information IF indicating the steering angle RA of the target vehicle TV from the vehicle control system 3.

[0020] The input device 15 is a device for inputting information, such as a mouse or a keyboard. The output device 16 is a device for outputting information, such as a display or a speaker. An operator such as a user can operate the information processing device 1 by using the input device 15 and the output device 16.

[0021] The drive 17 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading various information, such as programs, stored in a storage medium 91. The storage medium 91 is a medium that stores information, such as programs, electrically, magnetically, optically, mechanically, or chemically, so that a computer or other device, machine, or the like can read the stored information. At least one of the information processing program PG and the estimation model EM may be stored in the storage medium 91. The information processing device 1 may acquire at least one of the information processing program PG and the estimation model EM from the storage medium 91. Note that FIG. 2 illustrates a disk-type storage medium, such as a CD or a DVD, as an example of the storage medium 91. However, the type of the storage medium 91 is not limited to a disk-type storage medium and may be other than a disk-type storage medium. Examples of storage media other than a disk-type storage medium include semiconductor memories, such as flash memories. The type of the drive 17 may be selected arbitrarily depending on the type of the storage medium 91.

[0022] Note that, with regard to the specific hardware configuration of the information processing device 1, components may be omitted, replaced, or added as appropriate depending on the embodiment. For example, the processor resource may include multiple hardware processors. The hardware processor may be configured with a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), or the like. The storage unit 12 may be configured with RAM and ROM included in the control unit 11. At least one of the communication interface 13, the external interface 14, the input device 15, the output device 16, and the drive 17 may be omitted. The information processing device 1 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be identical. Furthermore, the information processing device 1 may be an information processing device designed specifically for the service provided, as well as a general-purpose server device, a PC (Personal Computer), or the like.

[0023] [Software Configuration] FIG. 3 schematically illustrates an example of the software configuration of the information processing device 1 according to this embodiment. The control unit 11 of the information processing device 1 loads the information processing program PG stored in the storage unit 12 onto the RAM. The control unit 11 then uses the CPU to interpret and execute instructions included in the information processing program PG loaded onto the RAM to control each component. As a result, as shown in FIG. 3 , the information processing device 1 according to this embodiment operates as a computer including an image acquisition unit 110, an imaging range determination unit 120, a steering angle information acquisition unit 130, an estimation unit 140, a vibration frequency determination unit 150, and a torque control unit 160 as software modules. That is, in this embodiment, each software module of the information processing device 1 is realized by the control unit 11 (CPU).

[0024] The image acquisition unit 110 acquires captured images CI from the camera 2. That is, the image acquisition unit 110 acquires "captured images CI in which the camera 2 arranged above the suspension of the target vehicle TV captures images of the wheels of the target vehicle TV." For example, the image acquisition unit 110 periodically (periodically) acquires four captured images CI in which the cameras 2 arranged above each of the four suspensions corresponding to the left and right front wheels and the left and right rear wheels (four wheels) of the target vehicle TV capture the images of the four wheels. The image acquisition unit 110 notifies the captured images CI to the imaging range determination unit 120 and the estimation unit 140.

[0025] The imaging range determination unit 120 identifies an "imaged area AC, which is the area below the wheels of the target vehicle TV," captured in the captured image CI acquired by the image acquisition unit 110, and compares the identified imaged area AC with a predetermined reference area TA. For example, the imaging range determination unit 120 identifies an imaged area AC for each of the four wheels of the target vehicle TV captured in each of the four captured images CI, and compares the identified imaged area AC with a predetermined reference area TA. The imaging range determination unit 120 notifies the torque control unit 160 of the result of the comparison, particularly whether "the imaged area AC is equal to or smaller than the reference area TA." The reference area TA may be stored in advance in the storage unit 12, for example. For example, the reference area TA is set assuming a situation in which the target vehicle TV travels on a snowy or sandy road, causing the vehicle (tires) of the target vehicle TV to be buried in the ground, making it difficult for the target vehicle TV to travel. As an example, in the above-described situation, the reference area TA is set based on the area below the "wheels of the target vehicle TV" that is captured in the "image CI captured by the camera 2 positioned above the suspension of the target vehicle TV, capturing the wheels of the target vehicle TV."

[0026] The steering angle information acquisition unit 130 acquires steering angle information IF indicating the steering angle RA of the target vehicle TV from the vehicle control system 3 that controls the acceleration, steering, deceleration (braking), etc. of the target vehicle TV. For example, the steering angle information acquisition unit 130 periodically acquires steering angle information IF indicating the steering angles RA of each of the four wheels of the target vehicle TV from the vehicle control system 3. The steering angle information acquisition unit 130 notifies the estimation unit 140 of the acquired steering angle information IF.

[0027] The estimation unit 140 estimates the stroke amount SA of the suspension of the target vehicle TV from the captured images CI acquired by the image acquisition unit 110. For example, the estimation unit 140 periodically estimates the stroke amount SA of each of the four suspensions corresponding to each of the four wheels of the target vehicle TV from each of the four captured images CI. The estimation unit 140 may estimate the stroke amount SA from the captured images CI on a rule-based basis or on a model-based basis using a trained machine learning model (in other words, a learned statistical model). The estimation unit 140 illustrated in FIG. 3 includes a contour estimation unit 142, a center position estimation unit 144, and a stroke amount estimation unit 146.

[0028] The contour estimation unit 142 estimates, from the captured images CI, a contour shape OS of the wheels (or wheels) of the target vehicle TV captured in the captured images CI. For example, the contour estimation unit 142 periodically estimates the contour shape OS for each of the four wheels of the target vehicle TV captured in each of the four captured images CI.

[0029] For example, the contour estimation unit 142 may use steering angle information IF in addition to the captured image CI to estimate the contour shape OS, that is, the contour shape OS may be estimated from the steering angle RA of the target vehicle TV (in particular, the steering angles RA of each of the four wheels) and the captured image CI.

[0030] For example, the contour estimation unit 142 first identifies the corresponding contour shape COS from the steering angle RA of the target vehicle TV indicated by the steering angle information IF. The corresponding contour shape COS is the "contour shape OS of the wheels (or wheels) of the target vehicle TV, which is estimated to be captured by the camera 2," corresponding to the steering angle RA of the target vehicle TV. For example, the contour estimation unit 142 may identify the corresponding contour shape COS from the steering angle RA of the target vehicle TV by using the steering angle-shape correspondence information IR. The steering angle-shape correspondence information IR is information that previously associates the steering angle RA of the target vehicle TV with the "contour shape OS of the wheels (or wheels) of the target vehicle TV, which is estimated to be captured by the camera 2." The turning angle of the wheels of the target vehicle TV varies depending on the steering angle RA of the target vehicle TV. Naturally, if the turning angle of the wheels of the target vehicle TV varies, the shape (contour shape OS) of the "wheels of the target vehicle TV" captured by the camera 2 arranged above the suspension of the target vehicle TV will also vary. In other words, the contour shape OS of the "wheels of the target vehicle TV" captured by the camera 2 varies depending on the steering angle RA of the target vehicle TV. Therefore, for example, the above-described steering angle-shape correspondence information IR can be prepared by checking the "contour shape OS of the wheels (or wheels) of the target vehicle TV captured by the camera 2" in advance for each steering angle RA. The contour estimation unit 142 may identify the corresponding contour shape COS from the steering angle-shape correspondence information IR and the steering angle RA of the target vehicle TV indicated by the steering angle information IF. The steering angle-shape correspondence information IR may be stored in advance in the storage unit 12, for example.

[0031] Furthermore, for example, the contour estimation unit 142 extracts (cuts out) an image (partial image IC) of the part where the wheels (or wheels) of the target vehicle TV are captured from the captured image CI, and performs preprocessing such as noise removal and distortion correction on the partial image IC.

[0032] Next, the contour estimation unit 142 estimates the shape (contour shape OS) of the wheels (or wheels) of the target vehicle TV captured in the partial image IC after the above-described preprocessing has been performed, and detects matching points between the estimated contour shape OS and the corresponding contour shape COS. The contour estimation unit 142 then repeats the estimation of the contour shape OS from the partial image IC (captured image CI) until the number of matching points reaches or exceeds a predetermined reference matching number TN. In other words, the contour estimation unit 142 estimates the contour shape OS of the wheels (or wheels) of the target vehicle TV from the captured image CI so that the number of matching points with the corresponding contour shape COS corresponding to the steering angle RA of the target vehicle TV indicated by the steering angle information IF reaches or exceeds the reference matching number TN. The contour estimation unit 142 can estimate the contour shape OS from the captured image CI with high accuracy by using the corresponding contour shape COS corresponding to the steering angle RA. The reference matching number TN may be stored in advance in the storage unit 12, for example.

[0033] As described above, the contour estimation unit 142 (estimation unit 140) estimates the contour shape OS of the "wheels (or wheels) of the target vehicle TV" from the captured image CI. For example, the contour estimation unit 142 estimates the contour shape OS from the captured image CI using the steering angle RA of the target vehicle TV. As described above, the contour estimation unit 142 can estimate the contour shape OS with high accuracy from the captured image CI by using the steering angle RA of the target vehicle TV. Then, as will be described in detail later, the estimation unit 140 estimates the stroke amount SA of the suspension of the target vehicle TV from the estimated contour shape OS. Therefore, the estimation unit 140 can estimate the stroke amount SA with high accuracy by using the contour shape OS estimated with high accuracy from the captured image CI using the steering angle RA of the target vehicle TV.

[0034] The center position estimation unit 144 estimates the center position CP of the wheels (or wheels) of the target vehicle TV from the "contour shape OS of the wheels (or wheels) of the target vehicle TV" estimated by the contour estimation unit 142. For example, the center position estimation unit 144 periodically estimates the contour shapes OS of each of the four wheels from the "contour shapes OS of each of the four wheels of the target vehicle TV" estimated by the contour estimation unit 142.

[0035] As described above, the center position estimation unit 144 estimates the center position CP of the wheel (or wheels) of the target vehicle TV. Then, as will be described in detail later, the estimation unit 140 estimates the stroke amount SA of the suspension of the target vehicle TV using the estimated center position CP. Here, if the wheels or tires of the target vehicle TV are replaced, the dynamic radius of the wheel (or wheels) of the target vehicle TV changes, and it is thought that, for example, the "contour shape OS of the wheel of the target vehicle TV" captured by the camera 2 changes. Even in such a case, the estimation unit 140 can use the center position CP of the wheel (or wheels) of the target vehicle TV to estimate the stroke amount SA with high accuracy, for example, as the distance between the center position CP and a fixed point of the target vehicle TV.

[0036] The stroke amount estimation unit 146 estimates the stroke amount SA of the suspension of the target vehicle TV from the "center position CP of the wheels (or wheels) of the target vehicle TV" estimated by the center position estimation unit 144. For example, the stroke amount estimation unit 146 estimates the stroke amount SA from the distance between the center position CP and a fixed point of the target vehicle TV (for example, the wheelhouse of the target vehicle TV). The stroke amount estimation unit 146 may periodically estimate the stroke amount SA of each of the four suspensions corresponding to each of the four wheels from the "center positions CP of each of the four wheels of the target vehicle TV" estimated by the center position estimation unit 144.

[0037] As described above, the estimation unit 140 estimates the stroke amount SA of the suspension of the target vehicle TV from the captured images CI. For example, the estimation unit 140 periodically estimates the stroke amount SA from the captured images CI periodically (periodically) acquired from the camera 2. The estimation unit 140 calculates the difference between the stroke amount SA for a plurality of consecutive periods. For example, the estimation unit 140 calculates the difference between the stroke amount SA estimated in a certain period and the stroke amount SA estimated in the period immediately following the certain period. If the difference exceeds a predetermined threshold, the estimation unit 140 may perform the following process. That is, for example, the estimation unit 140 may correct the "stroke amount SA estimated in the next period" using the "stroke amount SA estimated in the certain period." Then, the estimation unit 140 may use the stroke amount SA after such correction as the "stroke amount SA estimated in the next period." By performing such processing, the estimation unit 140 can prevent the occurrence of impossible changes (changes over time) in the suspension stroke amount SA, and even if an error occurs in the periodic estimation of the stroke amount SA, it can adopt a reasonable value as the ``stroke amount SA estimated in each period.''

[0038] (Example of Estimation Using a Trained Machine Learning Model) As described above, the estimation unit 140 may estimate the stroke amount SA from the captured image CI on a rule-based basis or on a model-based basis using a trained machine learning model. Below, a detailed description is given of an example in which the estimation unit 140 estimates the stroke amount SA from the captured image CI on a model-based basis.

[0039] Note that, up to this point, an example has been described in which the estimation unit 140 includes the contour estimation unit 142, the center position estimation unit 144, and the stroke amount estimation unit 146, which are each different functional units. However, the contour estimation unit 142, the center position estimation unit 144, and the stroke amount estimation unit 146 may be configured, for example, as an integrated unit, and as one example, may be configured as an integrated neural network (neural network module). For example, the entire estimation unit 140 may be configured as a neural network module. In other words, the estimation unit 140 may be configured as a neural network module that integrally performs the processes performed by the contour estimation unit 142, the center position estimation unit 144, and the stroke amount estimation unit 146.

[0040] Furthermore, at least one of the contour estimation unit 142, the center position estimation unit 144, and the stroke amount estimation unit 146 may be configured as a neural network module. Below, we will first describe an example in which the entire estimation unit 140 is configured as a neural network module, and then we will describe an example in which each of the contour estimation unit 142 and the center position estimation unit 144 is configured as a neural network module.

[0041] (Example of using a model that estimates the suspension stroke amount from a captured image) For example, the estimation unit 140 may estimate the stroke amount SA from the captured image CI using "an estimation model EM that is a trained machine learning model that has been trained to output the suspension stroke amount SA from the captured image CI." Hereinafter, the "estimation model EM that is a trained machine learning model that has been trained to output the suspension stroke amount SA from the captured image CI" will also be particularly referred to as a first estimation model EM(0).

[0042] As described above, for example, the estimation unit 140 is configured as a neural network module that integrally performs the processes performed by the contour estimation unit 142, the center position estimation unit 144, and the stroke amount estimation unit 146. The neural network module may then operate as the above-described estimation model EM (first estimation model EM(0)), thereby estimating the suspension stroke amount SA from the captured image CI. For example, the estimation unit 140 refers to the storage unit 12 to acquire the above-described estimation model EM (first estimation model EM(0)), particularly various parameters for realizing the estimation model EM by a neural network. The estimation unit 140 then inputs the captured image CI into the acquired estimation model EM (first estimation model EM(0)), thereby estimating the stroke amount SA. For example, the estimation unit 140 realizes a trained estimation model EM (first estimation model EM(0)) by applying the various acquired parameters described above to a neural network, and estimates the stroke amount SA by inputting the captured image CI into the realized estimation model EM.

[0043] As illustrated in FIG. 4 , each training data set DS(0) used for machine learning of the first estimation model EM(0) is composed of a combination of a training image LI captured by a camera positioned above the suspension of the training vehicle LV and the suspension stroke amount (training stroke amount LSA) of the training vehicle LV. The training stroke amount LSA indicates the correct answer to the estimation task for the training image LI. The training vehicle LV and the target vehicle TV may be the same, or the training vehicle LV may be a vehicle other than the target vehicle TV that has similar conditions to the target vehicle TV (e.g., at least one of the same model, year of manufacture, manufacturing plant, etc.). In the illustrated example, the training data set DS(0(1)) is composed of a combination of a training image LI(0(1)) and a training stroke amount LSA(1): +5. The learning data set DS(0(2)) is composed of a combination of the learning image LI(0(2)) and the "learning stroke amount LSA(2): 0". The learning data set DS(0(3)) is composed of a combination of the learning image LI(0(3)) and the "learning stroke amount LSA(3): -2".

[0044] Performing machine learning of the first estimation model EM(0) includes the following training step. That is, it includes a training step of training the first estimation model EM(0) so that, when the learning images LI are provided to the first estimation model EM(0) for each of the above-described learning data sets DS(0), the result of performing the estimation task of the first estimation model EM(0) conforms to (matches) the learning stroke amount LSA. The first estimation model EM(0) may be constructed using, for example, a random forest. However, the learning method (machine learning method) of the first estimation model EM(0) is not limited to this. A known algorithm, such as an algorithm used in machine learning, can be used as the algorithm for constructing the first estimation model EM(0). Examples of machine learning algorithms include, in addition to random forest, a support vector machine with a linear kernel (SVM linear), a support vector machine with an rbf kernel (SVM rbf), a neural net, a generalized linear model, a regularized linear discriminant analysis, and a regularized logistic regression.

[0045] As illustrated in Figure 5, the estimation unit 140 estimates the suspension stroke amount SA by inputting the captured image CI into an estimation model EM (first estimation model EM(0)), which is a trained machine learning model constructed by the above-mentioned machine learning.

[0046] As described above, the "contour shape of the vehicle's wheels captured by a camera positioned above the vehicle's suspension" varies depending on the steering angle of the vehicle. Therefore, each training data set DS(0) used in the machine learning of the first estimation model EM(0) may be composed of, for example, a combination of a training image LI, a steering angle of the training vehicle LV (training steering angle LRA), and a training stroke amount LSA. The first estimation model EM(0) may be trained so that, for each of the training data sets DS(0), when the training image LI and the training steering angle LRA are provided to the first estimation model EM(0), the result of the estimation task performed by the first estimation model EM(0) matches the training stroke amount LSA. In the machine learning of the first estimation model EM(0), for example, each training data set DS(0) is first classified by the training steering angle LRA. The first estimation model EM(0) is then trained so that the results of performing the estimation task for each learning data set DS(0) classified by learning steering angle LRA match the learning stroke amount LSA. Through this training, it is possible to learn the correspondence between the learning captured image LI and the learning stroke amount LSA for each learning steering angle LRA, thereby improving the estimation accuracy of the first estimation model EM(0).

[0047] As described above, each learning data set DS(0) may include additional learning information other than "the learning images LI and the learning stroke amounts LSA indicating the correct answers of the estimation task for the learning images LI." The additional learning information may be not only the above-mentioned learning steering angle LRA, but also the vehicle speed of the learning vehicle LV, the yaw rate of the learning vehicle LV, etc. For example, each learning data set DS(0) may include at least one of the learning steering angle LRA, the vehicle speed of the learning vehicle LV, and the yaw rate of the learning vehicle LV as the additional learning information.

[0048] If each learning data set DS(0) used for machine learning of the first estimation model EM(0) includes additional learning information in addition to the learning captured image LI and the learning stroke amount LSA, the estimation unit 140 inputs the following values ​​into the first estimation model EM(0). That is, the estimation unit 140 inputs additional information related to the target vehicle TV corresponding to the above-mentioned additional learning information into the first estimation model EM(0) in addition to the captured image CI. For example, if the learning data set DS(0) includes a learning steering angle LRA as additional learning information, the estimation unit 140 inputs the captured image CI and the "steering angle RA of the target vehicle TV indicated by the steering angle information IF" into the first estimation model EM(0). With this input, the estimation unit 140 can estimate the suspension stroke amount SA of the target vehicle TV with higher accuracy using the above-mentioned first estimation model EM(0).

[0049] (Example of using a model that estimates the contour shape of a vehicle wheel from a captured image) For example, the estimation unit 140 (particularly, the contour estimation unit 142) may estimate the contour shape OS from the captured image CI using "an estimation model EM that is a trained machine learning model that has been trained to output the contour shape OS from the captured image CI." Hereinafter, "the estimation model EM that is a trained machine learning model that has been trained to output the contour shape OS from the captured image CI" will also be particularly referred to as a second estimation model EM(1).

[0050] As described above, at least a portion of the estimation unit 140 may be configured as a neural network module. Specifically, at least one of the contour estimation unit 142, the center position estimation unit 144, and the stroke amount estimation unit 146 may be configured as a neural network module. For example, the contour estimation unit 142 may be configured as a neural network module. The neural network module may then operate as the above-mentioned estimation model EM (second estimation model EM(1)), thereby estimating the contour shape OS from the captured image CI. For example, the contour estimation unit 142 refers to the storage unit 12 to acquire the above-mentioned estimation model EM (second estimation model EM(1)), particularly various parameters for realizing the estimation model EM using a neural network. The contour estimation unit 142 then inputs the captured image CI into the acquired estimation model EM (second estimation model EM(1)), thereby estimating the contour shape OS. For example, the contour estimation unit 142 realizes a trained estimation model EM (second estimation model EM(1)) by applying the various acquired parameters described above to a neural network, and estimates the contour shape OS by inputting the captured image CI into the realized estimation model EM.

[0051] As illustrated in FIG. 6 , each training data set DS(1) used for machine learning of the second estimation model EM(1) is composed of a combination of a training image LI and a contour shape (training contour shape LOS) of a wheel (or wheels) of a training vehicle LV. The training contour shape LOS indicates the correct answer to the estimation task for the training image LI. In the illustrated example, the training data set DS(1(1)) is composed of a combination of the training image LI(1(1)) and the training contour shape LOS(1). The training data set DS(1(2)) is composed of a combination of the training image LI(1(2)) and the training contour shape LOS(2). The training data set DS(1(3)) is composed of a combination of the training image LI(1(3)) and the training contour shape LOS(3).

[0052] Performing machine learning of the second estimation model EM(1) includes the following training step: training the second estimation model EM(1) so that, when the learning captured image LI is provided to the second estimation model EM(1) for each of the above-mentioned learning data sets DS(1), the result of the second estimation model EM(1) performing an estimation task matches the learning contour shape LOS. In other respects, the second estimation model EM(1) is similar to the first estimation model EM(0).

[0053] As illustrated in Figure 7, the contour estimation unit 142 estimates the contour shape OS of the target vehicle TV by inputting the captured image CI into an estimation model EM (second estimation model EM(1)), which is a trained machine learning model constructed by the above-mentioned machine learning.

[0054] Each training data set DS(1) used in the machine learning of the second estimation model EM(1) may include additional training information, similar to each training data set DS(0) used in the machine learning of the first estimation model EM(0). That is, each training data set DS(1) may include additional training information other than "training images LI and training contour shapes LOS indicating correct answers to the estimation task for the training images LI." For example, each training data set DS(1) may include, as the additional training information, at least one of a training steering angle LRA, a vehicle speed of the training vehicle LV, and a yaw rate of the training vehicle LV.

[0055] If each learning data set DS(1) used for machine learning of the second estimation model EM(1) includes additional learning information in addition to the learning captured image LI and the learning contour shape LOS, the contour estimation unit 142 inputs the following values ​​into the second estimation model EM(1). That is, the contour estimation unit 142 inputs additional information related to the target vehicle TV corresponding to the above-mentioned additional learning information into the second estimation model EM(1) in addition to the captured image CI. For example, if the learning steering angle LRA is included in the learning data set DS(1) as additional learning information, the contour estimation unit 142 inputs the captured image CI and the "steering angle RA of the target vehicle TV indicated by the steering angle information IF" into the second estimation model EM(1). With this input, the contour estimation unit 142 can use the above-mentioned second estimation model EM(1) to more accurately estimate the contour shape OS of the wheel (or wheels) of the target vehicle TV.

[0056] Furthermore, as described above, the contour estimation unit 142 may estimate the contour shape OS with higher accuracy by using the corresponding contour shape COS corresponding to the steering angle RA of the target vehicle TV indicated by the steering angle information IF. For example, the contour estimation unit 142 may estimate the contour shape OS of the target vehicle TV from the captured image CI using the second estimation model EM(1) so that the number of matching points with the corresponding contour shape COS corresponding to the steering angle RA of the target vehicle TV indicated by the steering angle information IF is equal to or greater than a reference matching number TN.

[0057] (Example of using a model that estimates the center position of a wheel from a captured image) For example, the estimation unit 140 (particularly, the center position estimation unit 144) may estimate the center position CP from the captured image CI using "an estimation model EM that is a trained machine learning model that has been trained to output the center position CP from the captured image CI." Hereinafter, "the estimation model EM that is a trained machine learning model that has been trained to output the center position CP from the captured image CI" will also be particularly referred to as a third estimation model EM(2).

[0058] For example, the center position estimation unit 144 may be configured as a neural network module. The neural network module may operate as the above-described estimation model EM (third estimation model EM(2)), thereby estimating the center position CP from the captured image CI. For example, the center position estimation unit 144 refers to the storage unit 12 to acquire the above-described estimation model EM (third estimation model EM(2)), particularly various parameters for realizing the estimation model EM using a neural network. The center position estimation unit 144 then inputs the captured image CI into the acquired estimation model EM (third estimation model EM(2)), thereby estimating the center position CP. For example, the center position estimation unit 144 applies the above-described acquired various parameters to a neural network to realize a trained estimation model EM (third estimation model EM(2)), and then inputs the captured image CI into the realized estimation model EM, thereby estimating the center position CP.

[0059] As illustrated in FIG. 8 , each training data set DS(2) used for machine learning of the third estimation model EM(2) is composed of a combination of a training image LI and the center position (training center position LCP) of a wheel (or wheels) of the training vehicle LV. The training center position LCP indicates the correct answer to the estimation task for the training image LI. In the illustrated example, the training data set DS(2(1)) is composed of a combination of a training image LI(2(1)) and a training center position LCP(1). The training data set DS(2(2)) is composed of a combination of a training image LI(2(2)) and a training center position LCP(2). The training data set DS(2(3)) is composed of a combination of a training image LI(2(3)) and a training center position LCP(3).

[0060] Performing machine learning of the third estimation model EM(2) includes the following training step: training the third estimation model EM(2) so that, when the learning captured images LI are provided to the third estimation model EM(2) for each of the above-mentioned learning datasets DS(2), the result of performing the estimation task of the third estimation model EM(2) matches the learning center position LCP. In other respects, the third estimation model EM(2) is similar to the first estimation model EM(0) and the second estimation model EM(1).

[0061] As illustrated in Figure 9, the center position estimation unit 144 estimates the center position CP of the target vehicle TV by inputting the captured image CI into an estimation model EM (third estimation model EM(2)), which is a trained machine learning model constructed by the above-mentioned machine learning.

[0062] The training data set DS(2), like the training data set DS(0) and the training data set DS(1), may include additional training information. That is, each training data set DS(2) may include additional training information other than "training images LI and training center positions LCP indicating correct answers to the estimation task for the training images LI." For example, each training data set DS(2) may include, as the additional training information, at least one of the training steering angle LRA, the vehicle speed of the training vehicle LV, and the yaw rate of the training vehicle LV.

[0063] If each learning data set DS(2) used for machine learning of the third estimation model EM(2) includes additional learning information in addition to the learning captured image LI and the learning center position LCP, the center position estimation unit 144 inputs the following values ​​into the third estimation model EM(2). That is, in addition to the captured image CI, the center position estimation unit 144 inputs additional information related to the target vehicle TV corresponding to the above-mentioned additional learning information into the third estimation model EM(2). For example, if the learning data set DS(2) includes a learning steering angle LRA as additional learning information, the center position estimation unit 144 inputs the captured image CI and the "steering angle RA of the target vehicle TV indicated by the steering angle information IF" into the third estimation model EM(2). With this input, the center position estimation unit 144 can estimate the center position CP of the wheels (or wheels) of the target vehicle TV with higher accuracy using the above-mentioned third estimation model EM(2).

[0064] As described above, the estimation unit 140 may estimate the stroke amount SA of the suspension of the target vehicle TV from the captured image CI using an estimation model EM, which is a trained machine learning model. That is, the estimation unit 140 may estimate the stroke amount SA by inputting the captured image CI to the estimation model EM. For example, the estimation unit 140 may estimate the stroke amount SA by inputting the captured image CI to the estimation model EM (first estimation model EM(0)). For example, the estimation unit 140 may estimate a contour shape OS by inputting the captured image CI to the estimation model EM (second estimation model EM(1)), and then estimate the stroke amount SA using the estimated contour shape OS. For example, the estimation unit 140 may estimate a center position CP from the contour shape OS estimated using the second estimation model EM(1), and then estimate the stroke amount SA from the estimated center position CP. For example, the estimation unit 140 may estimate the center position CP by inputting the captured image CI into an estimation model EM (third estimation model EM(2)), and may estimate the stroke amount SA using the estimated center position CP. As an example, the estimation unit 140 may estimate the stroke amount SA as the distance between the center position CP estimated using the third estimation model EM(2) and a fixed point of the target vehicle TV.

[0065] The first estimation model EM(0) is constructed in advance by performing machine learning on a plurality of training data sets DS(0). Each training data set DS(0) is constructed by combining a training image LI of the wheels of the training vehicle LV captured by a camera positioned above the suspension of the training vehicle LV with the suspension stroke amount (training stroke amount LSA) of the training vehicle LV. The second estimation model EM(1) is constructed in advance by performing machine learning on a plurality of training data sets DS(1). Each training data set DS(1) is constructed by combining a training image LI with the contour shape (training contour shape LOS) of the wheels (or wheels) of the training vehicle LV. The third estimation model EM(2) is constructed in advance by performing machine learning on a plurality of training data sets DS(2). Each learning data set DS(2) is configured by a combination of a learning image LI and the center position (learning center position LCP) of a wheel (or wheels) of the learning vehicle LV. With this configuration, the estimation unit 140 can estimate the stroke amount SA with high accuracy from the captured image CI, for example, by using the estimation model EM trained using each learning data set DS related to a learning vehicle LV that has similar conditions to the target vehicle TV.

[0066] 3 , the vibration frequency determination unit 150 identifies a vibration frequency VF, which is the frequency of the change over time in the stroke amount SA estimated by the estimation unit 140 (e.g., the stroke amount estimation unit 146). For example, the vibration frequency determination unit 150 determines the period of expansion and contraction of the suspension of the target vehicle TV from the change over time (amount of change) in the stroke amount SA periodically estimated by the stroke amount estimation unit 146, and further determines the vibration frequency VF. The vibration frequency determination unit 150 may periodically identify the vibration frequency VF for the "stroke amounts SA of the four suspensions corresponding to the four wheels of the target vehicle TV" estimated by the estimation unit 140 (stroke amount estimation unit 146).

[0067] The vibration frequency determination unit 150 then compares the determined vibration frequency VF with a predetermined reference frequency TF and notifies the torque control unit 160 of the comparison result, particularly whether the vibration frequency VF is greater than the reference frequency TF. The reference frequency TF may be stored in advance in, for example, the storage unit 12. Generally, the vibration frequency (resonance frequency) of the vehicle's "vehicle body and powertrain" is a value (eigenvalue) specific to the "vehicle body and powertrain." Therefore, first, for the target vehicle TV, a "frequency range (of vibration) in which the "vehicle body and powertrain" and the "joint connected to the drive shaft, which generates vibration in response to the suspension stroke" do not resonate (the likelihood of resonance occurring)" is determined in advance. For example, for the target vehicle TV, a "frequency range in which the "vehicle body and powertrain" and the tripod joint do not resonate" is determined in advance through experiments or the like. The reference frequency TF may then be set in advance from the determined "frequency range in which resonance does not occur."

[0068] The torque control unit 160 changes the torque value DT of the drive shaft of the target vehicle TV according to the stroke amount SA estimated by the estimation unit 140 (e.g., the stroke amount estimation unit 146). For example, the torque control unit 160 changes the torque value DT (driving torque amount) for each of the four wheels of the target vehicle TV according to the stroke amount SA of each of the four suspensions corresponding to the four wheels. When the estimation unit 140 estimates the "stroke amount SA for a certain period" from the "images CI captured in a certain period," the torque control unit 160 may change the torque value DT for the period following the certain period in accordance with the estimated "stroke amount SA for the certain period." In this embodiment, the torque control unit 160 outputs a control signal CS to the vehicle control system 3 to specify or change the torque value DT of the drive shaft of the target vehicle TV (e.g., the above-mentioned "torque value DT for the next period"). For example, the torque control unit 160 outputs a control signal CS to the vehicle control system 3 to suppress (reduce) the torque value DT in accordance with the stroke amount SA. If the stroke amount SA is greater than a predetermined reference stroke amount, the torque control unit 160 may suppress the torque value DT in accordance with the stroke amount SA. Furthermore, the torque control unit 160 may suppress the torque value DT in a cycle following a certain cycle in accordance with the stroke amount SA in that cycle and the torque value DT in that cycle. The above-mentioned reference stroke amount may be stored in advance in the storage unit 12, for example.

[0069] The torque control unit 160 achieves the following effects by changing the torque value DT of the drive shaft of the target vehicle TV using the stroke amount SA estimated by the stroke amount estimation unit 146. That is, the torque control unit 160 can be applied to suppressing lateral vibration when the target vehicle TV starts, suppressing pitch behavior of the target vehicle TV (e.g., behavior of the target vehicle TV that occurs during acceleration and deceleration), and adjusting the wheel load balance of the target vehicle TV. For example, if a driving / regenerative torque is generated while the suspension of the target vehicle TV is extended or contracted, vibrations may occur in the target vehicle TV, with the joint of the drive shaft as a vibration source. The torque control unit 160 can suppress such vibrations by suppressing the torque value DT of the drive shaft in accordance with the stroke amount SA of the suspension of the target vehicle TV. In particular, the information processing device 1 can adjust the torque value DT of the drive shaft of the target vehicle TV (e.g., the torque value DT for each of the four wheels) with higher precision by estimating the stroke amount SA of each suspension corresponding to each wheel of the target vehicle TV.

[0070] When the imaging range determination unit 120 determines that the imaged area AC is equal to or smaller than the reference area TA, the torque control unit 160 does not execute the process of changing the torque value DT of the drive shaft of the target vehicle TV in accordance with the stroke amount SA. For example, when the imaged area AC is equal to or smaller than the reference area TA, the torque control unit 160 does not output a control signal CS to the vehicle control system 3. When the imaged area AC is equal to or smaller than the reference area TA, the torque control unit 160 may output a control signal CS to the vehicle control system 3 indicating that the torque value DT will not be changed. Furthermore, when the imaged area AC is equal to or smaller than the reference area TA, the torque control unit 160 may output a control signal CS to the vehicle control system 3 indicating that the amount of change in the torque value DT in accordance with the stroke amount SA is zero.

[0071] In a scene where the vehicle (tires) of the target vehicle TV are buried in snow or sand while traveling on a snowy or sandy road, it is not easy to estimate the outline shape OS of the wheels (tires) of the target vehicle TV from the captured image CI. Furthermore, such a scene is extremely rare, i.e., it is not a normal situation in which vibration of the target vehicle TV is suppressed (e.g., vibration is suppressed for ride comfort). Therefore, if the area below the "wheels of the target vehicle TV" captured in the captured image CI (imaged area AC) is equal to or smaller than a predetermined reference area TA, the torque control unit 160 does not perform (masks) the process of changing the torque value DT of the drive shaft shaft according to the suspension stroke amount SA. This allows the torque control unit 160 to reduce the computational load (e.g., the computational load of a motor controller) of control related to the drive of the torque value DT and the like.

[0072] When the vibration frequency determination unit 150 determines that "the vibration frequency VF is greater than the reference frequency TF," the torque control unit 160 does not execute (masks) the process of "changing the torque value DT of the drive shaft of the target vehicle TV in accordance with the stroke amount SA." For example, when the vibration frequency VF is greater than the reference frequency TF, the torque control unit 160 does not output a control signal CS to the vehicle control system 3. When the vibration frequency VF is greater than the reference frequency TF, the torque control unit 160 may output a control signal CS to the vehicle control system 3 indicating that the torque value DT will not be changed. Furthermore, when the vibration frequency VF is greater than the reference frequency TF, the torque control unit 160 may output a control signal CS to the vehicle control system 3 indicating that the "amount of change in the torque value DT in accordance with the stroke amount SA" is "0 (zero)."

[0073] As described above, the vibration frequency of the vehicle body and powertrain of the target vehicle TV is a characteristic value and is generally low. Therefore, when the vibration frequency VF (the frequency of expansion and contraction of the suspension of the target vehicle TV) is high (i.e., when the period of change in the stroke amount SA over time is fast), resonance with the vibration frequency of the vehicle body and powertrain is unlikely. In other words, when the vibration frequency VF is high, resonance between the vehicle body and powertrain and a joint (e.g., a tripod joint) connected to the drive shaft, which generates vibration in accordance with the suspension stroke, is unlikely to occur. Therefore, when the vibration frequency VF is high, specifically when the vibration frequency VF is greater than the reference frequency TF, the torque control unit 160 does not need to execute the process of "changing the torque value DT of the drive shaft of the target vehicle TV in accordance with the stroke amount SA." This allows the torque control unit 160 to reduce the computational load (e.g., the computational load of a motor controller, etc.) of control related to the drive of the torque value DT and the like.

[0074] §3 Operation Example (Overall Overview) Figure 10 is a flowchart showing an example of the processing procedure of the information processing device 1 according to this embodiment. The processing procedure described below is an example of the processing procedure of an information processing method PM that causes a processor (e.g., the CPU of the information processing device 1) to execute the process of "estimating the stroke amount SA of the suspension of the target vehicle TV from the captured image CI." However, the processing procedure described below is merely an example, and each step may be modified as much as possible. Furthermore, steps in the processing procedure described below may be omitted, replaced, or added as appropriate depending on the embodiment.

[0075] (Step S110) In step S110, the control unit 11 operates as the image acquisition unit 110 and acquires captured images CI of the wheels of the target vehicle TV from the camera 2 arranged above the suspension of the target vehicle TV. For example, the control unit 11 periodically (periodically) acquires four captured images CI of each of the four wheels of the target vehicle TV by the camera 2 arranged above the suspension of each of the four wheels.

[0076] (Step S120) In step S120, the control unit 11 operates as the imaging range determination unit 120 and identifies the area (imaged area AC) below the wheels of the target vehicle TV that is captured in the captured image CI acquired in step S110. For example, the control unit 11 periodically identifies the imaged area AC for each of the four captured images CI capturing each of the four wheels of the target vehicle TV.

[0077] (Step S130) In step S130, the control unit 11 operates as the imaging range determination unit 120 and determines whether the imaged area AC determined in step S120 is greater than the reference area TA. For example, the control unit 11 periodically performs this determination. If the imaged area AC is equal to or smaller than the reference area TA (No in step S130), the control unit 11 terminates the process; that is, the control unit 11 does not execute the process of step S170, which is to "change the torque value DT of the drive shaft of the target vehicle TV according to the stroke amount SA." For example, the control unit 11 terminates the process if the imaged area AC is equal to or smaller than the reference area TA for at least one of the four captured images CI. However, the control unit 11 may terminate the process only for the wheels captured in the captured images CI whose imaged areas AC are equal to or smaller than the reference area TA. The control unit 11 may proceed to step S140 for a wheel captured in an image CI of the four captured images CI whose captured area AC is larger than the reference area TA. If the captured area AC is larger than the reference area TA (Yes in step S130), the control unit 11 proceeds to step S140. For example, if the captured areas AC of all of the four captured images CI are larger than the reference area TA, the control unit 11 proceeds to step S140.

[0078] (Step S140) In step S140, the control unit 11 operates as the estimation unit 140 and estimates the stroke amount SA of the suspension of the target vehicle TV from the captured images CI acquired in step S110. For example, the control unit 11 periodically estimates the stroke amount SA of each of the four suspensions corresponding to each of the four wheels of the target vehicle TV from each of the four captured images CI.

[0079] (Step S150) In step S150, the control unit 11 operates as the vibration frequency determination unit 150 and determines the frequency (vibration frequency VF) of the change over time in the stroke amount SA estimated in step S140. For example, the control unit 11 periodically determines the vibration frequency VF for the stroke amount SA estimated in step S140, and in particular, periodically determines the vibration frequency VF for each of the stroke amounts SA of the four suspensions described above.

[0080] (Step S160) In step S160, the control unit 11 operates as the vibration frequency determination unit 150 and determines whether the vibration frequency VF determined in step S150 is equal to or less than the reference frequency TF. For example, the control unit 11 periodically performs this determination. If the vibration frequency VF is greater than the reference frequency TF (NO in step S160), the control unit 11 terminates the process; that is, the control unit 11 does not execute the process of "changing the torque value DT of the drive shaft of the target vehicle TV according to the stroke amount SA" in step S170. For example, the control unit 11 terminates the process if all of the vibration frequencies VF of the four suspensions described above are greater than the reference frequency. However, the control unit 11 may terminate the process only for suspensions (wheels corresponding to the suspensions) whose vibration frequencies VF are greater than the reference frequency. The control unit 11 may proceed to step S170 for suspensions (wheels corresponding to the suspensions) whose vibration frequencies VF are equal to or less than the reference frequency. If the vibration frequency VF is equal to or less than the reference frequency TF (Yes in step S160), the control unit 11 proceeds to step S170. For example, if all of the vibration frequencies VF of the four suspensions are equal to or less than the reference frequency, the control unit 11 proceeds to step S170.

[0081] (Step S170) In step S170, the control unit 11 operates as the torque control unit 160 and changes the torque value DT of the drive shaft of the target vehicle TV in accordance with the stroke amount SA estimated in step S140. For example, the control unit 11 suppresses (reduces) the torque value DT in accordance with the stroke amount SA. The control unit 11 may change the torque value DT (driving torque amount) of each of the four wheels corresponding to each of the four suspensions in accordance with the stroke amount SA of each of the four suspensions estimated in step S140. Furthermore, the control unit 11 may change the torque value DT for the period following a certain period in accordance with the "stroke amount SA for a certain period" estimated from the "images CI captured in a certain period" in step S140.

[0082] As described above, when all of the vibration frequencies VF of the four suspensions are equal to or lower than the reference frequency, the control unit 11 may change the torque values ​​DT of the four wheels corresponding to the four suspensions in accordance with the stroke amounts SA of the four suspensions. Furthermore, the control unit 11 may change the torque values ​​DT of at least the wheels corresponding to the suspensions whose vibration frequencies VF are equal to or lower than the reference frequency in accordance with the stroke amounts SA estimated in step S140.

[0083] 10 shows an example in which the control unit 11 ends the process when the answer is No in step S130, that is, when the imaged area AC is equal to or smaller than the reference area TA. However, when the answer is No in step S130, instead of "ending the process," the control unit 11 may output, for example, a control signal CS indicating that the torque value DT is not to be changed to the vehicle control system 3 in step S170. Also, when the answer is No in step S130, the control unit 11 may output, for example, a control signal CS indicating that the "amount of change in the torque value DT according to the stroke amount SA" is "0 (zero)" to the vehicle control system 3 in step S170.

[0084] 10 shows an example in which the control unit 11 ends the process when the result of step S160 is No, that is, when the vibration frequency VF is greater than the reference frequency TF. However, when the result of step S160 is No, instead of "ending the process," the control unit 11 may output, for example, a control signal CS indicating that the torque value DT is not to be changed to the vehicle control system 3 in step S170. Furthermore, when the result of step S160 is No, the control unit 11 may output, for example, a control signal CS indicating that the "amount of change in the torque value DT according to the stroke amount SA" is "0 (zero)" to the vehicle control system 3 in step S170.

[0085] (Details of Step of Estimating Suspension Stroke Amount from Captured Image) Fig. 11 is a flowchart illustrating details of the processing procedure for an example of the process of "estimating suspension stroke amount SA of the target vehicle TV from captured image CI" in step S140 illustrated in Fig. 10. However, it is not essential to execute the processing related to each step illustrated in Fig. 11 as the processing in step S140. For example, in step S140, the control unit 11 may estimate the suspension stroke amount SA of the target vehicle TV by inputting the captured image CI to the estimation model EM (first estimation model EM(0)).

[0086] (Step S1410) In step S1410, the control unit 11 operates as the steering angle information acquisition unit 130 and acquires steering angle information IF indicating the steering angle RA of the target vehicle TV from the vehicle control system 3. For example, the control unit 11 periodically acquires steering angle information IF indicating the steering angle RA of each of the four wheels of the target vehicle TV.

[0087] (Step S1420) In step S1420, the control unit 11 operates as the contour estimation unit 142 and identifies the corresponding contour shape COS from the steering angle information IF acquired in step S1410. For example, the control unit 11 periodically identifies the corresponding contour shape COS corresponding to the steering angle RA for each of the four wheels of the target vehicle TV indicated by the steering angle information IF.

[0088] (Step S1430) In step S1430, the control unit 11 operates as the contour estimation unit 142 and estimates the "shape (contour shape OS) of the wheels (or wheels) of the target vehicle TV" from the captured images CI acquired in step S110 of FIG. 10. For example, the control unit 11 periodically estimates the contour shape OS for each of the four wheels of the target vehicle TV from each of four captured images CI. For example, the control unit 11 may estimate the contour shape OS from the captured images CI using an estimation model EM (second estimation model EM(1)), which is a trained machine learning model trained to output the contour shape OS from the captured images CI.

[0089] (Step S1440) In step S1440, the control unit 11 operates as the contour estimation unit 142 and detects matching portions between the contour shape OS estimated in step S1430 and the corresponding contour shape COS identified in step S1420. For example, the control unit 11 periodically detects matching portions between the contour shapes OS of each of the four wheels estimated in step S1430 and the corresponding contour shapes COS of each of the four wheels identified in step S1420.

[0090] (Step S1450) In step S1450, the control unit 11 operates as the contour estimation unit 142 and determines whether the "matching locations between the contour shape OS and the corresponding contour shape COS" detected in step S1440 are equal to or greater than the reference match number TN. If the matching locations are equal to or greater than the reference match number TN (Yes in step S1450), the control unit 11 proceeds to step S1470. If the matching locations are less than the reference match number TN (No in step S1450), the control unit 11 corrects the contour shape OS estimated from the captured image CI, that is, estimates the corrected contour shape OS from the captured image CI (step S1460). Then, the control unit 11 detects matching locations between the estimated "corrected contour shape OS" and the corresponding contour shape COS (step S1440) and determines whether the matching locations are equal to or greater than the reference match number TN (step S1450). That is, the control unit 11 repeatedly estimates the contour shape OS from the captured image CI until the number of matching portions between the contour shape OS estimated from the captured image CI and the corresponding contour shape COS reaches or exceeds the reference matching number TN. In other words, the control unit 11 estimates the contour shape OS from the captured image CI so that the number of matching portions with the corresponding contour shape COS reaches or exceeds the reference matching number TN. For example, the control unit 11 periodically estimates the contour shape OS from the captured image CI for each of the four wheels of the target vehicle TV so that the number of matching portions with the corresponding contour shape COS reaches or exceeds the reference matching number TN.

[0091] (Step S1470) In step S1470, the control unit 11 operates as the center position estimation unit 144 and estimates the center positions CP of the wheels (or wheels) of the target vehicle TV from the contour shapes OS estimated in step S1430 or step S1460. For example, the control unit 11 periodically estimates the center positions CP of each of the four wheels from the contour shapes OS of each of the four wheels estimated in step S1430. For example, the control unit 11 may estimate the center positions CP from the captured images CI using an estimation model EM (third estimation model EM(2)), which is a trained machine learning model trained to output the center positions CP from the captured images CI. When estimating the center positions CP from the captured images CI using the estimation model EM (third estimation model EM(2)), the control unit 11 may omit the processing of steps S1410 to S1460.

[0092] (Step S1480) In step S1480, the control unit 11 operates as the stroke amount estimating unit 146 and estimates the stroke amount SA of the suspension of the target vehicle TV from the center position CP estimated in step S1470. For example, the control unit 11 periodically estimates the stroke amount SA of each of the four suspensions corresponding to each of the four wheels from the center positions CP of each of the four wheels estimated in step S1470.

[0093] [Features] As described above, the information processing device 1 according to this embodiment includes the image acquisition unit 110 and the estimation unit 140. The image acquisition unit 110 acquires a captured image CI of the wheels of the target vehicle TV captured by the camera 2 arranged above the suspension of the target vehicle TV. The estimation unit 140 estimates the stroke amount SA of the suspension of the target vehicle TV from the captured image CI acquired by the image acquisition unit 110.

[0094] Furthermore, the information processing method PM according to this embodiment is an information processing method in which a computer (for example, the information processing device 1) executes steps S110 and S140. In step S110, the information processing device 1 acquires a captured image CI of the wheels of the target vehicle TV captured by a camera 2 arranged above the suspension of the target vehicle TV. In step S140, the information processing device 1 estimates a stroke amount SA of the suspension of the target vehicle TV from the captured image CI acquired in step S110.

[0095] According to this configuration, the information processing device 1 (information processing method PM) can detect (estimate) the suspension stroke amount for a vehicle (in this embodiment, the target vehicle TV) that does not have a stroke sensor.

[0096] §4 Modifications Although the embodiments of the present invention have been described above in detail, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. For example, the following modifications are possible. Note that, in the following, the same reference numerals are used for components similar to those in the above embodiment, and descriptions of similar points to those in the above embodiment are omitted where appropriate. The following modifications can be combined as appropriate.

[0097] In the above embodiment, an example has been described in which the information processing device 1, the camera 2, and the vehicle control system 3 are each configured as separate computers. However, the configuration of the information processing device according to this embodiment is not limited to this example and may be determined appropriately depending on the embodiment. For example, the information processing device 1 and the camera 2 may be integrated into a single computer. Furthermore, the information processing device 1 and the vehicle control system 3 may be integrated into a single computer. Furthermore, at least one of the information processing device 1, the camera 2, and the vehicle control system 3 may be configured as a single computer. Furthermore, at least one of the information processing device 1, the camera 2, and the vehicle control system 3 may be configured as a multiple computer.

[0098] In the above embodiment, an example has been described in which the target vehicle TV has four wheels in total, including left and right front wheels and left and right rear wheels, but the number of wheels (number) provided by the target vehicle TV does not necessarily have to be "4." The number of wheels provided by the target vehicle TV may be "1," or may be a number other than "4."

[0099] 1...information processing device, 2...camera, 110...image acquisition unit, 140...estimation unit, AC...imaged area, CI...captured image, CP...center position, DS...learning data set, DT...torque value, EM...estimation model, IF...steering angle information, LI...learning captured image, LV...learning vehicle, PM...information processing method, RA...steering angle, SA...stroke amount, TA...reference area, TF...reference frequency, TV...target vehicle, VF...vibration frequency

Claims

1. An information processing method in which a computer executes the steps of: acquiring an image of a wheel of a target vehicle using a camera positioned above the suspension of the target vehicle; and estimating the stroke amount of the suspension from the acquired image.

2. The information processing method of claim 1, wherein the computer further executes a step of acquiring steering angle information indicating the steering angle of the target vehicle, and in the step of estimating the stroke amount, the computer estimates the stroke amount from the captured image and the steering angle.

3. An information processing method as described in claim 1 or 2, wherein in the step of estimating the stroke amount, the computer identifies the center position of the wheel of the target vehicle from the captured image, and estimates the stroke amount from the identified center position.

4. The information processing method of claim 1 or 2, wherein in the step of estimating the stroke amount, the computer estimates the stroke amount by inputting the captured image into an estimation model that has been trained by machine learning, and the trained estimation model has been constructed in advance by performing machine learning on a plurality of training data sets each consisting of a combination of a training captured image of the wheels of the training vehicle taken by a camera positioned above the suspension of the training vehicle, and the stroke amount of the suspension of the training vehicle, or the contour shape of the wheels of the training vehicle, or the center position of the wheels of the training vehicle.

5. The information processing method according to claim 1 or 2, wherein the computer further executes the step of changing the torque value of the drive shaft of the target vehicle in accordance with the estimated stroke amount.

6. The information processing method of claim 5, wherein the computer further executes a step of identifying a vibration frequency, which is the frequency of the change over time of the estimated stroke amount, and if the identified vibration frequency is greater than a predetermined reference frequency, does not execute a step of changing the torque value according to the stroke amount.

7. The information processing method of claim 5, wherein the computer further executes a step of identifying an imaged area, which is the area below the wheels of the target vehicle that is captured in the captured image, and if the identified imaged area is equal to or smaller than a predetermined reference area, does not execute a step of changing the torque value according to the stroke amount.

8. An information processing device comprising: an image acquisition unit that acquires an image of a wheel of a target vehicle using a camera arranged above the suspension of the target vehicle; and an estimation unit that estimates the stroke amount of the suspension from the image acquired by the image acquisition unit.

Images