Control device, control method, and control program

Abstract

Provided is a control device comprising: an extraction unit that extracts a feature portion of a tire of a traveling mobile body from a first image obtained by imaging of the tire and that extracts, from a second image obtained by imaging of the tire a prescribed time after a time at which the first image was captured, a feature portion identical to the feature portion of the tire extracted from the first image; a rotation speed calculation unit that uses a movement amount of the respective feature portions extracted from the first image and the second image by the extraction unit and uses a time difference in imaging timing between the first image and the second image, so as to calculate a rotation speed of the tire; and a determination unit that uses the rotation speed of the tire calculated by the rotation speed calculation unit and uses a travel speed of the mobile body, so as to determine the presence or absence of a slip of the tire.

Description

Control Device, Control Method, and Control Program

[0001] The present disclosure relates to a control device, a control method, and a control program.

[0002] Japanese Patent Application Laid-Open No. 2016-5328 discloses a control device for a vehicle, which includes a drive unit that drives a tire via a drive shaft, a reference rotation speed calculation unit that calculates a torsional amount of the drive shaft based on a required torque and outputs a reference rotation speed for slip determination obtained by adding the torsional amount to a rotation speed of the drive unit corresponding to a vehicle speed, a slip determination unit that compares the rotation speed of the drive unit with the reference rotation speed to determine whether the tire is slipping, and a rotation speed control unit that controls the rotation speed of the drive unit with respect to a target rotation speed when it is determined by the slip determination unit that the tire is slipping and controls the rotation speed of the tire with respect to the target rotation speed when the slip of the tire tends to converge.

[0003] In the technology disclosed in Japanese Patent Application Laid-Open No. 2016-5328, the rotation speed of the tire is calculated from the torsional amount of the drive shaft, but it is required to be able to determine the slip of the tire with a simpler configuration.

[0004] In view of the above points, the present disclosure aims to provide a control device, a control method, and a control program that determine the presence or absence of tire slip using a captured image of a tire during travel.

[0005] A control device according to a first aspect of the present disclosure includes an extraction unit that extracts a characteristic portion of the tire from a first image capturing the tire of a moving body during travel and extracts a characteristic portion identical to the characteristic portion of the tire extracted from the first image from a second image capturing the tire after a predetermined time from the time when the first image was captured, and a rotation speed calculation unit that calculates a rotation speed of the tire using a movement amount of the characteristic portions respectively extracted by the extraction unit from the first image and the second image and a time difference of imaging timings between the first image and the second image, and a determination unit that determines the presence or absence of tire slip using the rotation speed of the tire calculated by the rotation speed calculation unit and a travel speed of the moving body.

[0006] A control device according to a second aspect of the present disclosure is a control device according to a first aspect, further comprising a fuel control unit that controls the amount of fuel injected by the moving body when the determination unit determines that the tire is slipping.

[0007] A control device according to a third aspect of this disclosure is a control device according to the first or second aspect, further comprising a travel speed calculation unit that calculates the travel speed of the moving body using the position information of the moving body acquired by a position sensor that acquires the current position.

[0008] A control method according to a fourth aspect of the present disclosure involves a processor extracting a feature portion of a tire from a first image of a moving body while it is in motion, extracting the same feature portion of the tire as that extracted from the first image from a second image of the tire taken a predetermined time after the first image was taken, calculating the rotational speed of the tire using the amount of movement of the feature portions extracted from the first and second images, respectively, and the time difference in the imaging timing between the first and second images, and executing a process to determine whether or not the tire is slipping using the calculated rotational speed of the tire and the speed of the moving body.

[0009] A control program according to a fifth aspect of the present disclosure causes a computer to extract characteristic portions of the tires from a first image of the tires of a moving body while it is in motion, extract the same characteristic portions of the tires as those extracted from the first image from a second image of the tires taken a predetermined time after the first image was taken, calculate the rotational speed of the tires using the amount of movement of the characteristic portions extracted from the first and second images, respectively, and the time difference in the timing of the images taken between the first and second images, and determine whether or not the tires are slipping using the calculated rotational speed of the tires and the speed of the moving body.

[0010] According to this disclosure, a control device, a control method, and a control program can be provided that determine whether or not a tire is slipping using an image of the tire while it is in motion.

[0011] This figure shows a schematic configuration of a mobile body equipped with a control device according to an embodiment of the disclosed technology. This is a block diagram showing the hardware configuration of the control device. This is a block diagram showing an example of the functional configuration of the control device. This is a flowchart showing the flow of control processing by the control device.

[0012] Hereinafter, an example of an embodiment of this disclosure will be described with reference to the drawings. In each drawing, identical or equivalent components and parts are given the same reference numerals. Also, the dimensional ratios in the drawings are exaggerated for illustrative purposes and may differ from the actual ratios.

[0013] Figure 1 is a diagram showing a schematic configuration of a mobile body equipped with a control device according to this embodiment. The mobile body 1 shown in Figure 1 is, for example, a vehicle that travels through a mine. In this embodiment, the mobile body 1 has a total of six tires 2, one on each side at the front left and right, and two on each side at the rear left and right. The mobile body 1 is equipped with a camera 3 for imaging at least one of the tires 2. The camera 3 images the tire 2 as a moving image. The camera 3 captures moving images at, for example, 30 frames, 60 frames, or other frame rates per second.

[0014] Furthermore, the mobile unit 1 is equipped with a control device 10. The control device 10 has the function of determining whether or not the tires 2 are slipping using images of the tires 2 captured by the camera 3 while the mobile unit 1 is moving. In order to detect slipping, it is necessary to know the speed of the mobile unit 1 and the rotation speed of the tires 2. The control device 10 calculates the rotation speed of the tires 2 using images of the tires 2 captured by the camera 3 while the mobile unit 1 is moving for at least two frames.

[0015] The mobile body 1 equipped with the control device 10 according to this embodiment can determine whether or not the tire 2 is slipping, despite its simple configuration, by using the rotational speed of the tire 2 and the travel speed of the mobile body 1 calculated from the image of the tire 2 taken by the camera 3 while the tire is in motion.

[0016] Figure 2 is a block diagram showing the hardware configuration of the control device 10.

[0017] As shown in Figure 2, the control device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, storage 14, an input unit 15, a display unit 16, a communication interface (I / F) 17, and a positioning sensor 18. Each component is connected to the others via a bus 19 so as to be able to communicate with each other.

[0018] The CPU 11 is a central processing unit that executes various programs and controls various parts. Specifically, the CPU 11 reads a program from the ROM 12 or storage 14 and executes the program using the RAM 13 as a working area. The CPU 11 controls each of the above components and performs various calculations according to the program recorded in the ROM 12 or storage 14. In this embodiment, the ROM 12 or storage 14 stores a control program that determines whether or not the tire 2 is slipping.

[0019] ROM 12 stores various programs and data. RAM 13 temporarily stores programs or data as a working area. Storage 14 consists of a storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory, and stores various programs including the operating system, and various data.

[0020] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used for various types of input.

[0021] The display unit 16 is, for example, a liquid crystal display and displays various information. The display unit 16 may also function as an input unit 15 by employing a touch panel system.

[0022] The communication interface 17 is an interface for communicating with other devices, and standards such as Ethernet®, FDDI, and Wi-Fi® can be used.

[0023] The positioning sensor 18 is a sensor that determines the current location. Examples of sensors that determine the current location include GPS (Global Positioning System) sensors and GNSS (Global Navigation Satellite System) sensors.

[0024] When executing the control program described above, the control device 10 uses the hardware resources described above to implement various functions. The functional configuration implemented by the control device 10 will now be described.

[0025] Figure 3 is a block diagram showing an example of the functional configuration of the control device 10.

[0026] As shown in Figure 3, the control device 10 has the following functional configuration: an extraction unit 101, a rotational speed calculation unit 102, a driving speed calculation unit 103, a determination unit 104, and a fuel control unit 105. Each functional configuration is realized by the CPU 11 reading and executing a control program stored in the ROM 12 or storage 14.

[0027] The extraction unit 101 extracts characteristic features of the tire 2 from an image (first image) obtained when the camera 3 images the tire 2 while the moving body 1 is in motion. These characteristic features may include, for example, a logo engraved on the tire 2 or the wheel portion of the tire 2, or a mark attached to the tire 2 or the wheel portion of the tire 2. Furthermore, the extraction unit 101 extracts the same characteristic features of the tire 2 as those extracted from the first image from an image (second image) obtained when the camera 3 images the tire 2 a predetermined time after the first image was captured (for example, one frame later).

[0028] The extraction unit 101 may also use a pre-trained model generated by machine learning to extract the characteristic parts of the tire 2.

[0029] The rotation speed calculation unit 102 calculates the rotation speed of the tire 2 using the amount of movement of the feature portion extracted by the extraction unit 101 from the first and second images. The rotation speed calculation unit 102 can calculate the rotation speed of the tire 2 by using the amount of circumferential movement of the feature portion and the time difference between the imaging time of the first image and the imaging time of the second image. The size of the tire 2 and the installation location of the camera 3 are known. Therefore, the rotation speed calculation unit 102 can determine the actual amount of circumferential movement of the feature portion in the tire 2 by determining the amount of circumferential movement of the feature portion in the image of the tire 2 captured by the camera 3. If the time difference between the imaging time of the first image and the imaging time of the second image is the time of one frame, and the camera 3 was capturing 30 frames of moving images per second, then the time difference between the imaging time of the first image and the imaging time of the second image is 1 / 30 of a second.

[0030] The speed calculation unit 103 calculates the speed of the moving object 1 at the time the rotation speed calculation unit 102 calculates the rotation speed of the tires 2, using the position information of the moving object 1 measured by the positioning sensor 18. Here, the speed calculation unit 103 can calculate the speed of the moving object 1 by using, for example, the difference between the position information at the time of imaging of the first image and the position information at the time of imaging of the second image, and the time difference between the time of imaging of the first image and the time difference between the time of imaging of the second image. Alternatively, as an example of measurement, the speed calculation unit 103 can calculate the distance traveled by comparing the positions of feature points in the second image that are different from the tires and body (for example, objects on the road surface, unevenness, etc.) with the positions of feature points extracted from the first image, and then calculate the speed of the moving object 1 by using this distance traveled and the time difference between the time of imaging of the first image and the time difference between the time of imaging of the first image and the time difference between the time of imaging of the second image. The speed of the moving object 1 can be calculated using known methods, not limited to these examples.

[0031] Furthermore, if the travel speed of the mobile body 1 at the time the rotation speed calculation unit 102 calculates the rotation speed of the tire 2 can be directly obtained from the speed sensor installed on the mobile body 1, the travel speed calculation unit 103 does not need to perform the travel speed calculation.

[0032] The determination unit 104 uses the rotational speed of the tire 2 calculated by the rotational speed calculation unit 102 and the moving speed of the moving body 1 at that rotational speed to determine whether or not the tire 2 is slipping. The determination of whether or not the tire 2 is slipping by the determination unit 104 is performed at any time. Specifically, the determination unit 104 refers to a table that records the relationship between the rotational speed of the tire 2 and the moving speed of the moving body 1, and determines that the tire 2 is slipping if the rotational speed of the tire 2 calculated by the rotational speed calculation unit 102 is faster than the rotational speed of the tire 2 in a normal state at the moving speed of the moving body 1. On the other hand, the determination unit 104 refers to a table that records the relationship between the rotational speed of the tire 2 and the moving speed of the moving body 1, and determines that the tire 2 is not slipping if the rotational speed of the tire 2 calculated by the rotational speed calculation unit 102 is not faster than the rotational speed of the tire 2 in a normal state at the moving speed of the moving body 1.

[0033] The determination unit 104 may also determine whether or not the tires 2 are slipping by taking into account information on the inclination angle of the road surface on which the mobile body 1 is traveling and information on the gears used while the mobile body 1 is traveling. Information on the inclination angle of the road surface on which the mobile body 1 is traveling can be obtained, for example, by an acceleration sensor, a gyro sensor, etc.

[0034] Furthermore, the mobile body 1 may be equipped with multiple cameras 3 capable of imaging different tires 2. In this case, the extraction unit 101 may extract characteristic parts of the tire 2 from the first and second images captured by each camera 3, and the rotation speed calculation unit 102 may calculate the rotation speed of each tire 2 using the amount of movement of the characteristic parts extracted by the extraction unit 101 from the first and second images captured by each camera 3, and then determine the rotation speed of the tire 2 by taking the arithmetic mean of the calculated rotation speeds. The rotation speed calculation unit 102 can determine the rotation speed of the tire 2 with greater accuracy by determining the rotation speed of the tire 2 from the first and second images captured by multiple cameras 3 compared to the case where the rotation speed of the tire 2 is determined from the first and second images captured by a single camera 3.

[0035] When the determination unit 104 determines that the tire 2 is slipping, the fuel control unit 105 increases or decreases the amount of fuel injected into the moving body 1 according to the slip situation. For example, if the tire 2 is slipping on the road surface, the fuel control unit 105 may increase the amount of fuel injected, and if it is spinning freely, the fuel control unit 105 may decrease the amount of fuel injected. However, it is desirable to select an appropriate control method according to the situation so that the tire 2 grips and the moving speed is commensurate with the rotation speed of the tire 2, and so that the moving body 1 behaves as intended after gripping.

[0036] With this configuration, the control device 10 can determine whether or not the tire 2 is slipping from the image captured by the camera 3 that images the tire 2.

[0037] Next, the operation of the control device 10 will be explained.

[0038] Figure 4 is a flowchart showing the flow of control processing by the control device 10. The CPU 11 reads the control program from the ROM 12 or storage 14, loads it into the RAM 13, and executes it to perform the control processing. The control processing shown in Figure 4 is executed at any time while the mobile body 1 is moving.

[0039] In step S101, the CPU 11 extracts characteristic features of the tire 2 from an image (first image) obtained when the camera 3 captures the tire 2 while the mobile body 1 is moving. These characteristic features of the tire 2 may include, for example, a logo engraved on the tire 2 or the wheel portion of the tire 2, or a mark attached to the tire 2 or the wheel portion of the tire 2.

[0040] Following step S101, in step S102, the CPU 11 extracts the same feature portion of the tire 2 that was extracted from the first image from an image (second image) obtained by the camera 3 when it photographs the tire 2 a predetermined time after the first image was taken (for example, one frame later).

[0041] Following step S102, in step S103, the CPU 11 calculates the rotational speed of the tire 2 using the amount of movement of the feature portion extracted from the first and second images.

[0042] Following step S103, in step S104, the CPU 11 acquires the traveling speed of the moving body 1 at the time when the rotational speed of the tire 2 is calculated. For example, the CPU 11 calculates the traveling speed of the moving body 1 at the time when the rotational speed of the tire 2 is calculated in step S103, using the position information of the moving body 1 measured by the positioning sensor 18. Here, the CPU 11 can calculate the traveling speed of the moving body 1 by using, for example, the difference between the position information at the imaging time of the first image and the position information at the imaging time of the second image, and the time difference between the imaging time of the first image and the imaging time of the second image.

[0043] Following step S104, in step S105, the CPU 11 determines the presence or absence of slip of the tire 2 by using the rotational speed of the tire 2 calculated in step S103 and the moving speed of the moving body 1 at the time of the rotational speed acquired in step S104. Specifically, the CPU 11 refers to a table in which the relationship between the rotational speed of the tire 2 and the moving speed of the moving body 1 is recorded, and if the calculated rotational speed of the tire 2 is faster than the rotational speed of the tire 2 in a normal state at the moving speed of the moving body 1, it is determined that the tire 2 is slipping. On the other hand, the CPU 11 refers to a table in which the relationship between the rotational speed of the tire 2 and the moving speed of the moving body 1 is recorded, and if the calculated rotational speed of the tire 2 is not faster than the rotational speed of the tire 2 in a normal state at the moving speed of the moving body 1, it is determined that the tire 2 is not slipping.

[0044] Following step S105, in step S106, the CPU 11 determines whether the tire 2 is slipping.

[0045] As a result of the determination in step S106, if the tire 2 is slipping (step S106; Yes), the CPU 11 controls the fuel injection amount of the moving body 1 in step S107. Since the state where the tire 2 is slipping is a state where a moving speed corresponding to the rotational speed of the tire 2 is not obtained, the CPU 11 controls the fuel injection amount in the state where the tire 2 is slipping and controls the fuel injection amount of the moving body 1.

[0046] On the other hand, if, as a result of the determination in step S106, the tire 2 is not slipping (step S107; Yes), the CPU 11 skips the process in step S107.

[0047] By executing such processing, the control device 10 can determine the presence or absence of slip of the tire 2 from the image captured by the camera 3 that captures the tire 2.

[0048] In the above embodiment, one camera 3 is provided for one tire 2, but a plurality of cameras 3 may be provided for one tire 2. The control device 10 may determine the presence or absence of slip of the tire 2 by extracting the characteristic portion of the tire 2 from each of the images captured by the plurality of cameras 3. By extracting the characteristic portion of the tire 2 from each of the images captured by the plurality of cameras 3 and determining the presence or absence of slip of the tire 2, it becomes possible to improve the determination accuracy of slip.

[0049] In the above embodiment, when the control device 10 determines the slip of the tire 2, it performs fuel control. However, alternatively, when the control device 10 determines the slip of the tire 2, it may perform control on other control systems such as the engine output, brake system, and traction control system of the moving body 1.

[0050] When the control device 10 determines the slip of the tire 2, it may accumulate information on the situation in which the slip has occurred. By accumulating information when slip occurs, it becomes possible to analyze the pattern and tendency of slip occurrence using big data analysis or machine learning, and utilize it for future slip prediction or improvement of the control of the moving body 1.

[0051] As described above, the embodiments of the present disclosure have been described in detail with reference to the accompanying drawings. However, the technical scope of the present disclosure is not limited to such examples. It is obvious that those having ordinary knowledge in the technical field of the present disclosure can come up with various modification examples or correction examples within the scope of the technical idea described in the claims. It is naturally understood that these modification examples or correction examples also belong to the technical scope of the present disclosure.

[0052] Furthermore, the effects described in the above embodiments are descriptive or illustrative, and are not limited to those described in the above embodiments. In other words, the technology relating to this disclosure may produce other effects that would be obvious to a person of ordinary skill in the art of this disclosure from the descriptions in the above embodiments, in addition to or in lieu of the effects described in the above embodiments.

[0053] In addition, the control processing that the CPU reads and executes in each of the above embodiments may be executed by various processors other than the CPU. Examples of such processors include PLDs (Programmable Logic Devices) whose circuit configuration can be changed after manufacturing, such as FPGAs (Field-Programmable Gate Arrays), and dedicated electrical circuits that have a circuit configuration specifically designed to execute a particular process, such as ASICs (Application Specific Integrated Circuits). Furthermore, the control processing may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (for example, multiple FPGAs, and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor elements.

[0054] Furthermore, while the above embodiments describe a configuration in which the control processing program is pre-stored (installed) in ROM or storage, the invention is not limited thereto. The program may be provided in a form recorded on a non-transitor recording medium such as a CD-ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versatile Disk Read Only Memory), or USB (Universal Serial Bus) memory. The program may also be provided in a form that can be downloaded from an external device via a network. Moreover, this disclosure may also be applied to program products.

[0055] Furthermore, the disclosure of Japanese Patent Application No. 2024-195489, filed on November 7, 2024, is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

Claims

1. A control device comprising: an extraction unit that extracts characteristic portions of a tire from a first image of a moving body's tire while it is in motion, and extracts the same characteristic portions of the tire as those extracted from the first image from a second image of the tire taken a predetermined time after the first image was taken; a rotational speed calculation unit that calculates the rotational speed of the tire using the amount of movement of the characteristic portions extracted by the extraction unit from the first image and the second image, respectively, and the time difference in the imaging timing between the first image and the second image; and a determination unit that determines whether or not the tire is slipping using the rotational speed of the tire calculated by the rotational speed calculation unit and the travel speed of the moving body.

2. The control device according to claim 1, further comprising a fuel control unit that controls the amount of fuel injected by the moving body when the determination unit determines that the tire is slipping.

3. The control device according to claim 1, further comprising a speed calculation unit that calculates the speed of the moving object using the position information of the moving object acquired by a position sensor that acquires the current position.

4. A control method comprising: a processor extracting characteristic portions of the tire from a first image of the tire of a moving object while it is in motion; extracting the same characteristic portions of the tire as those extracted from the first image from a second image of the tire taken a predetermined time after the first image was taken; calculating the rotational speed of the tire using the amount of movement of the characteristic portions extracted from the first and second images, respectively, and the time difference in the imaging timing between the first and second images; and executing a process to determine whether or not the tire is slipping using the calculated rotational speed of the tire and the speed of the moving object.

5. A control program that causes a computer to perform the following steps: extract characteristic parts of the tire from a first image of the tire of a moving object while it is in motion; extract the same characteristic parts of the tire as those extracted from the first image from a second image of the tire taken a predetermined time after the first image was taken; calculate the rotational speed of the tire using the amount of movement of the characteristic parts extracted from the first and second images, respectively, and the time difference in the imaging timing between the first and second images; and determine whether or not the tire is slipping using the calculated rotational speed of the tire and the speed of the moving object.

Images