Control device, control method, and control program

Abstract

The system uses images of the tires while they are in motion to determine whether or not the tires are slipping. [Solution] A control device is provided, comprising: an extraction unit that extracts characteristic parts of a tire from a first image of a moving body's tire while it is in motion, and extracts the same characteristic parts from a second image of the tire taken a predetermined time after the first image was taken, as well as the characteristic parts of the tire extracted from the first image; a rotational speed calculation unit that calculates the rotational speed of the tire using the amount of movement of the characteristic parts extracted by the extraction unit from the first image and the second image, respectively, and the time difference in the imaging timing between the first image and the second image; and a determination unit that determines whether or not the tire is slipping using the rotational speed of the tire calculated by the rotational speed calculation unit and the travel speed of the moving body.

Description

Technical Field

[0003] ,

[0001] The present disclosure relates to a control device, a control method, and a control program.

Background Art

[0002] Patent Document 1 discloses a control device for a vehicle, comprising: a drive unit that drives a tire via a drive shaft; a reference rotation speed calculation unit that calculates a torsional amount of the drive shaft based on a required torque and outputs a reference rotation speed for slip determination obtained by adding the torsional amount to a rotation speed of the drive unit corresponding to a vehicle speed; a slip determination unit that compares the rotation speed of the drive unit with the reference rotation speed to determine whether the tire is slipping; and a rotation speed control unit that controls the rotation speed of the drive unit with respect to a target rotation speed when it is determined by the slip determination unit that the tire is slipping, and controls the rotation speed of the tire with respect to the target rotation speed when the slip of the tire tends to converge.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the technique disclosed in Patent Document 1, the rotation speed of the tire is calculated from the torsional amount of the drive shaft, but it is required to be able to determine the slip of the tire with a simpler configuration.

[0005] The present disclosure has been made in view of the above points, and an object thereof is to provide a control device, a control method, and a control program for determining the presence or absence of tire slip using a captured image of a tire during traveling.

Means for Solving the Problems

[0006] A control device according to a first aspect of the present disclosure includes: an extraction unit that extracts characteristic portions of a tire from a first image of a moving body's tire while it is in motion, and extracts the same characteristic portions of the tire as those extracted from the first image from a second image of the tire taken a predetermined time after the first image was taken; a rotational speed calculation unit that calculates the rotational speed of the tire using the amount of movement of the characteristic portions extracted by the extraction unit from the first image and the second image, respectively, and the time difference in the imaging timing between the first image and the second image; and a determination unit that determines whether or not the tire is slipping using the rotational speed of the tire calculated by the rotational speed calculation unit and the travel speed of the moving body.

[0007] A control device according to a second aspect of the present disclosure is a control device according to a first aspect, further comprising a fuel control unit that controls the amount of fuel injected by the moving body when the determination unit determines that the tire is slipping.

[0008] A control device according to a third aspect of this disclosure is a control device according to a first aspect, further comprising a travel speed calculation unit that calculates the travel speed of the moving body using position information of the moving body acquired by a position sensor that acquires the current position.

[0009] A control method according to a fourth aspect of the present disclosure involves a processor extracting a feature portion of a tire from a first image of a moving body while it is in motion, extracting the same feature portion of the tire as that extracted from the first image from a second image of the tire taken a predetermined time after the first image was taken, calculating the rotational speed of the tire using the amount of movement of the feature portions extracted from the first and second images, respectively, and the time difference in the imaging timing between the first and second images, and executing a process to determine whether or not the tire is slipping using the calculated rotational speed of the tire and the speed of the moving body.

[0010] A control program according to a fifth aspect of this disclosure causes a computer to extract characteristic portions of the tire from a first image of the tire of a moving body while it is in motion, extract the same characteristic portions of the tire as those extracted from the first image from a second image of the tire taken a predetermined time after the first image was taken, calculate the rotational speed of the tire using the amount of movement of the characteristic portions extracted from the first and second images, respectively, and the time difference in the timing of the images taken between the first and second images, and determine whether or not the tire is slipping using the calculated rotational speed of the tire and the speed of the moving body. [Effects of the Invention]

[0011] According to this disclosure, a control device, a control method, and a control program can be provided that determine whether or not a tire is slipping using an image of the tire while it is in motion. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows a schematic configuration of a mobile body equipped with a control device according to an embodiment of the disclosed technology. [Figure 2] Block diagram showing the hardware configuration of the control unit. [Figure 3] This is a block diagram showing an example of the functional configuration of a control device. [Figure 4] This is a flowchart showing the flow of control processing by the control device. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of this disclosure will be described with reference to the drawings. In each drawing, identical or equivalent components and parts are given the same reference numerals. Also, the dimensional ratios in the drawings are exaggerated for illustrative purposes and may differ from the actual ratios.

[0014] Figure 1 is a diagram showing a schematic configuration of a mobile body equipped with a control device according to this embodiment. The mobile body 1 shown in Figure 1 is, for example, a vehicle that travels through a mine. In this embodiment, the mobile body 1 has a total of six tires 2, one on each side at the front left and right, and two on each side at the rear left and right. The mobile body 1 is equipped with a camera 3 for imaging at least one of the tires 2. The camera 3 images the tire 2 as a moving image. The camera 3 captures moving images at, for example, 30 frames, 60 frames, or other frame rates per second.

[0015] Furthermore, the mobile unit 1 is equipped with a control device 10. The control device 10 has the function of determining whether or not the tires 2 are slipping using images of the tires 2 captured by the camera 3 while the mobile unit 1 is moving. In order to detect slipping, the speed of the mobile unit 1 and the rotation speed of the tires 2 must be known. The control device 10 calculates the rotation speed of the tires 2 using at least two frames of images of the tires 2 captured by the camera 3 while the mobile unit 1 is moving.

[0016] The mobile body 1 equipped with the control device 10 according to this embodiment can determine whether or not the tire 2 is slipping, despite its simple configuration, by using the rotational speed of the tire 2 and the travel speed of the mobile body 1 calculated from the image of the tire 2 taken by the camera 3 while the tire is in motion.

[0017] Figure 2 is a block diagram showing the hardware configuration of the control device 10.

[0018] As shown in Figure 2, the control device 10 includes a CPU (Central Processing Unit) 11, ROM (Read Only Memory) 12, RAM (Random Access Memory) 13, storage 14, input unit 15, display unit 16, communication interface (I / F) 17, and positioning sensor 18. Each component is connected to the others via a bus 19 so that they can communicate with each other.

[0019] The CPU 11 is a central processing unit that executes various programs and controls each part. That is, the CPU 11 reads a program from the ROM 12 or the storage 14 and executes the program using the RAM 13 as a working area. The CPU 11 performs control of each of the above components and various arithmetic processes according to the program recorded in the ROM 12 or the storage 14. In the present embodiment, a control program for determining the presence or absence of slip of the tire 2 is stored in the ROM 12 or the storage 14.

[0020] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores a program or data as a working area. The storage 14 is composed of a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory, and stores various programs including an operating system and various data.

[0021] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.

[0022] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may adopt a touch panel method and function as the input unit 15.

[0023] The communication interface 17 is an interface for communicating with other devices. For example, standards such as Ethernet (registered trademark), FDDI, Wi-Fi (registered trademark), etc. are used.

[0024] The positioning sensor 18 is a sensor for positioning the current position. Examples of the sensor for positioning the current position include a GPS (Global Positioning System) sensor, a GNSS (Global Navigation Satellite System) sensor, etc.

[0025] When executing the control program described above, the control device 10 uses the hardware resources described above to implement various functions. The functional configuration implemented by the control device 10 will now be described.

[0026] Figure 3 is a block diagram showing an example of the functional configuration of the control device 10.

[0027] As shown in Figure 3, the control device 10 has the following functional configuration: extraction unit 101, rotational speed calculation unit 102, travel speed calculation unit 103, determination unit 104, and fuel control unit 105. Each functional configuration is realized by the CPU 11 reading and executing a control program stored in the ROM 12 or storage 14.

[0028] The extraction unit 101 extracts characteristic features of the tire 2 from an image (first image) obtained when the camera 3 images the tire 2 while the moving body 1 is in motion. These characteristic features may include, for example, a logo engraved on the tire 2 or the wheel portion of the tire 2, or a mark attached to the tire 2 or the wheel portion of the tire 2. Furthermore, the extraction unit 101 extracts the same characteristic features of the tire 2 as those extracted from the first image from an image (second image) obtained when the camera 3 images the tire 2 a predetermined time after the first image was taken (for example, one frame later).

[0029] The extraction unit 101 may also use a pre-trained model generated by machine learning to extract the characteristic parts of the tire 2.

[0030] The rotation speed calculation unit 102 calculates the rotation speed of tire 2 using the amount of movement of the feature portion extracted by the extraction unit 101 from the first and second images. The rotation speed calculation unit 102 can calculate the rotation speed of tire 2 by using the amount of circumferential movement of the feature portion and the time difference between the imaging time of the first image and the imaging time of the second image. The size of tire 2 and the installation location of camera 3 are known. Therefore, the rotation speed calculation unit 102 can determine the actual amount of circumferential movement of the feature portion in tire 2 by determining the amount of circumferential movement of the feature portion in the image of tire 2 captured by camera 3. If the time difference between the imaging time of the first image and the imaging time of the second image is the time of one frame, and camera 3 was capturing 30 frames of moving images per second, then the time difference between the imaging time of the first image and the imaging time of the second image is 1 / 30 of a second.

[0031] The speed calculation unit 103 uses the position information of the moving body 1, which has been measured by the positioning sensor 18, to calculate the speed of the moving body 1 at the time the rotation speed calculation unit 102 calculates the rotation speed of the tires 2. Here, the speed calculation unit 103 can calculate the speed of the moving body 1 by using, for example, the difference between the position information at the time of imaging of the first image and the position information at the time of imaging of the second image, and the time difference between the time of imaging of the first image and the time difference between the time of imaging of the second image. Alternatively, as an example of measurement, the speed calculation unit 103 can calculate the distance traveled by comparing the positions of feature points in the second image that are different from the tires and body (for example, objects on the road surface, unevenness, etc.) with the positions of feature points extracted from the first image, and then calculate the speed of the moving body 1 by using this distance traveled and the time difference between the time of imaging of the first image and the time difference between the time of imaging of the first image and the time difference between the time of imaging of the second image. The speed of the moving body 1 can be calculated using known methods, not limited to these examples.

[0032] Furthermore, if the travel speed of the mobile body 1 at the time the rotation speed calculation unit 102 calculates the rotation speed of the tire 2 can be directly obtained from the speed sensor installed on the mobile body 1, the travel speed calculation unit 103 does not need to perform the travel speed calculation.

[0033] The determination unit 104 uses the rotational speed of tire 2 calculated by the rotational speed calculation unit 102 and the moving speed of the moving body 1 at that rotational speed to determine whether or not tire 2 is slipping. The determination of whether or not tire 2 is slipping by the determination unit 104 is performed at any time. Specifically, the determination unit 104 refers to a table that records the relationship between the rotational speed of tire 2 and the moving speed of the moving body 1, and determines that tire 2 is slipping if the rotational speed of tire 2 calculated by the rotational speed calculation unit 102 is faster than the rotational speed of tire 2 in a normal state at the moving speed of the moving body 1. On the other hand, the determination unit 104 refers to a table that records the relationship between the rotational speed of tire 2 and the moving speed of the moving body 1, and determines that tire 2 is not slipping if the rotational speed of tire 2 calculated by the rotational speed calculation unit 102 is not faster than the rotational speed of tire 2 in a normal state at the moving speed of the moving body 1.

[0034] The determination unit 104 may also determine whether or not the tires 2 are slipping by taking into account information on the inclination angle of the road surface on which the mobile body 1 is traveling and information on the gears used while the mobile body 1 is traveling. Information on the inclination angle of the road surface on which the mobile body 1 is traveling can be obtained, for example, by an acceleration sensor, a gyroscope, or the like.

[0035] Furthermore, the mobile body 1 may be equipped with multiple cameras 3 capable of imaging different tires 2. In this case, the extraction unit 101 may extract characteristic parts of the tire 2 from the first and second images captured by each camera 3, and the rotation speed calculation unit 102 may calculate the rotation speed of each tire 2 using the amount of movement of the characteristic parts extracted by the extraction unit 101 from the first and second images captured by each camera 3, and then determine the rotation speed of the tire 2 by taking the arithmetic mean of the calculated rotation speeds. The rotation speed calculation unit 102 can determine the rotation speed of the tire 2 with greater accuracy by determining the rotation speed of the tire 2 from the first and second images captured by multiple cameras 3 compared to the case where the rotation speed of the tire 2 is determined from the first and second images captured by a single camera 3.

[0036] When the detection unit 104 determines that the tire 2 is slipping, the fuel control unit 105 increases or decreases the amount of fuel injected into the mobile body 1 according to the slip situation. For example, if the tire 2 is sliding on the road surface, the fuel control unit 105 may increase the amount of fuel injected, and if it is spinning freely, the fuel control unit 105 may decrease the amount of fuel injected. However, it is desirable to select an appropriate control method according to the situation so that the tire 2 grips and the mobile body 1 moves at a speed commensurate with the rotation speed of the tire 2, and so that the mobile body 1 behaves as intended after gripping.

[0037] With this configuration, the control device 10 can determine whether or not the tire 2 is slipping from the image captured by the camera 3 that images the tire 2.

[0038] Next, the operation of the control device 10 will be explained.

[0039] Figure 4 is a flowchart showing the flow of control processing by the control device 10. The CPU 11 reads the control program from the ROM 12 or storage 14, loads it into the RAM 13, and executes it to perform the control processing. The control processing shown in Figure 4 is executed at any time while the mobile body 1 is moving.

[0040] In step S101, the CPU 11 extracts characteristic features of the tire 2 from the image (first image) obtained when the camera 3 captures the tire 2 while the mobile body 1 is moving. These characteristic features of the tire 2 may include, for example, a logo engraved on the tire 2 or the wheel portion of the tire 2, or a mark attached to the tire 2 or the wheel portion of the tire 2.

[0041] Following step S101, in step S102, the CPU 11 extracts the same feature portion of tire 2 that was extracted from the first image from an image (second image) obtained when camera 3 photographs tire 2 a predetermined time after the first image was taken (for example, 1 frame later).

[0042] Following step S102, in step S103, the CPU 11 calculates the rotation speed of the tire 2 using the displacement of the feature portions extracted from the first and second images.

[0043] Following step S103, in step S104, the CPU 11 obtains the travel speed of the mobile body 1 at the time the rotation speed of the tire 2 was calculated. For example, the CPU 11 uses the position information of the mobile body 1, which was determined by the positioning sensor 18, to calculate the travel speed of the mobile body 1 at the time the rotation speed of the tire 2 was calculated in step S103. Here, the CPU 11 can calculate the travel speed of the mobile body 1 by using, for example, the difference between the position information at the time of capturing the first image and the position information at the time of capturing the second image, and the time difference between the time of capturing the first image and the time of capturing the second image.

[0044] Following step S104, in step S105, the CPU 11 uses the rotational speed of tire 2 calculated in step S103 and the moving speed of the moving body 1 at that rotational speed, obtained in step S104, to determine whether or not tire 2 is slipping. Specifically, the CPU 11 refers to a table that records the relationship between the rotational speed of tire 2 and the moving speed of the moving body 1, and determines that tire 2 is slipping if the calculated rotational speed of tire 2 is faster than the rotational speed of tire 2 in a normal state at the moving speed of the moving body 1. On the other hand, the CPU 11 refers to a table that records the relationship between the rotational speed of tire 2 and the moving speed of the moving body 1, and determines that tire 2 is not slipping if the calculated rotational speed of tire 2 is not faster than the rotational speed of tire 2 in a normal state at the moving speed of the moving body 1.

[0045] Following step S105, in step S106, the CPU 11 determines whether tire 2 is slipping.

[0046] If, as a result of the judgment in step S106, tire 2 is slipping (step S106; Yes), then in step S107, the CPU 11 controls the fuel injection amount of the mobile body 1. When tire 2 is slipping, the mobile body is not moving at a speed commensurate with the rotational speed of tire 2. Therefore, the CPU 11 controls the fuel injection amount of tire 2 in the state where tire 2 is slipping, and controls the fuel injection amount of the mobile body 1.

[0047] On the other hand, if the determination in step S106 is that tire 2 is not slipping (step S107; Yes), the CPU 11 skips the processing in step S107.

[0048] By performing this process, the control device 10 can determine whether or not the tire 2 is slipping from the image captured by the camera 3 that images the tire 2.

[0049] In the above embodiment, one camera 3 was provided for each tire 2, but multiple cameras 3 may be provided for each tire 2. The control device 10 may determine whether or not the tire 2 is slipping by extracting characteristic parts of the tire 2 from each of the images captured by the multiple cameras 3. By extracting characteristic parts of the tire 2 from each of the images captured by the multiple cameras 3 and determining whether or not the tire 2 is slipping, the control device 10 can improve the accuracy of slip detection.

[0050] In the above embodiment, the control device 10 performed fuel control when it detected tire 2 slippage. However, when it detects tire 2 slippage, it may also perform control on other control systems such as the engine output, brake system, and traction control system of the mobile body 1.

[0051] When the control device 10 detects a slip in the tire 2, it may store information about the circumstances under which the slip occurred. By accumulating information about when a slip occurs, it becomes possible to analyze the patterns and trends of slip occurrences using big data analysis or machine learning, and utilize this information for future slip prediction or improvement of the control of the moving body 1.

[0052] While embodiments of the present disclosure have been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to these examples. It is clear that a person with ordinary skill in the art of the present disclosure may conceive of various modifications or alterations within the scope of the technical idea set forth in the claims, and these modifications or alterations are also understood to fall within the technical scope of the present disclosure.

[0053] Furthermore, the effects described in the above embodiments are descriptive or illustrative, and are not limited to those described in the above embodiments. In other words, the technology relating to this disclosure may produce other effects that would be obvious to a person of ordinary skill in the art of this disclosure from the descriptions in the above embodiments, in addition to or in lieu of the effects described in the above embodiments.

[0054] In addition, the control processing that the CPU reads and executes in each of the above embodiments may be executed by various processors other than the CPU. Examples of such processors include PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) whose circuit configuration can be changed after manufacturing, and dedicated electrical circuits that are processors with circuit configurations specifically designed to execute specific processing, such as ASICs (Application Specific Integrated Circuits). Furthermore, the control processing may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (for example, multiple FPGAs, and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor elements.

[0055] Furthermore, while the above embodiments describe a configuration in which the control processing program is pre-stored (installed) in ROM or storage, the invention is not limited thereto. The program may be provided in a form recorded on a non-transitory recording medium such as a CD-ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versatile Disk Read Only Memory), or USB (Universal Serial Bus) memory. The program may also be provided in a form that can be downloaded from an external device via a network. This disclosure may also be applied to program products. [Explanation of Symbols]

[0056] 1 Mobile Unit 2 tires 3 cameras 10 Control device

Claims

1. An extraction unit extracts characteristic parts of the tire from a first image of the tire of a moving object while it is in motion, and extracts the same characteristic parts as those extracted from the first image from a second image of the tire taken a predetermined time after the first image was taken. A rotation speed calculation unit calculates the rotation speed of the tire using the amount of movement of the feature portions extracted by the extraction unit from the first image and the second image, respectively, and the time difference in the imaging timing between the first image and the second image. A determination unit determines whether or not the tire is slipping using the rotational speed of the tire and the travel speed of the moving body calculated by the rotational speed calculation unit, A control device equipped with the following features.

2. The control device according to claim 1, further comprising a fuel control unit that controls the amount of fuel injected by the moving body when the determination unit determines that the tire is slipping.

3. The control device according to claim 1, further comprising a speed calculation unit that calculates the speed of the moving object using the position information of the moving object acquired by a position sensor that acquires the current position.

4. The processor, A characteristic portion of the tire is extracted from a first image of the tire of a moving object in motion, and a characteristic portion identical to that extracted from the first image is extracted from a second image of the tire taken a predetermined time after the first image was taken. The rotational speed of the tire is calculated using the amount of movement of the feature portion extracted from the first image and the second image, respectively, and the time difference in the imaging timing between the first image and the second image. The presence or absence of tire slippage is determined using the calculated rotational speed of the tire and the travel speed of the moving body. A control method for executing a process.

5. On the computer, A characteristic portion of the tire is extracted from a first image of the tire of a moving object in motion, and a characteristic portion identical to that extracted from the first image is extracted from a second image of the tire taken a predetermined time after the first image was taken. The rotational speed of the tire is calculated using the amount of movement of the feature portion extracted from the first image and the second image, respectively, and the time difference in the imaging timing between the first image and the second image. The presence or absence of tire slippage is determined using the calculated rotational speed of the tire and the travel speed of the moving body. A control program that executes a process.

Images