Image data generating device, lean vehicle, and image data generating method

Abstract

Achieved is an image data generation device which is used in a support system that assists the operation of the rider of a leaning vehicle, and is capable of suppressing a delay until the start of a support operation for a rider in a support system more than in the related art. The image data generation device is used in a support system that assists the operation of the rider of a leaning vehicle, and comprises: an input unit into which imaging data on an imaging device which detects the surrounding environment of the leaning vehicle is input; a movement conversion unit which obtains first image data obtained by moving and converting the imaging data at least in a vertical direction; and a rotation conversion unit which obtains second image data obtained by rotating and converting the first image data.

Description

[Technical Field]

[0001] The present invention relates to an image data generating device used in an assistance system that assists a rider of a lean vehicle in driving, a lean vehicle equipped with the image data generating device, and a method for generating image data used in an assistance system that assists a rider of a lean vehicle in driving. [Background technology]

[0002] A conventional motorcycle, which is a type of lean vehicle, has been proposed that employs an assistance system that assists the rider in driving. The assistance system detects the surrounding environment of the motorcycle based on image data captured by an imaging device mounted on the motorcycle, and performs operations to assist the rider in driving the motorcycle (e.g., a warning function, an emergency braking function, a cruising function, etc.) based on the detection results. A lean vehicle is a vehicle whose body leans in the direction of the turn when turning.

[0003] As described above, when a lean vehicle turns, the body tilts in the turning direction. Therefore, when the lean vehicle is in a leaned state, the image data of the imaging device mounted on the lean vehicle is also tilted. Therefore, a conventional assistance system mounted on a motorcycle, which is one type of lean vehicle, rotates and converts the image data of the imaging device when the image data is tilted, and uses the rotated and converted image data to consider the necessity (necessity) of driving assistance (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2015 / 174208 Summary of the Invention [Problem to be solved by the invention]

[0005] A lean-mounted vehicle has a short length. Therefore, the lean-mounted vehicle pitches more, and the optical axis of the imaging device mounted on the lean-mounted vehicle also moves significantly in the pitching direction. In other words, when a lean-mounted vehicle pitches, the center position of the imaging data from the imaging device before the pitching occurs shifts vertically. Therefore, an imaging device mounted on a lean-mounted vehicle needs to have a large imaging range (a large number of pixels) so that it can capture the area necessary for determining whether or not driving assistance is required, even when the vehicle pitches. Furthermore, using an imaging device with a large imaging range requires that areas of the imaging data that are not necessary for determining whether or not driving assistance is required be rotated when tilted imaging data is converted, resulting in a large amount of calculation. Furthermore, the calculations required for rotating and converting the imaging data involve sine and cosine calculations, which are time-consuming. Therefore, conventional assistance systems mounted on motorcycles have a problem of a long delay before providing assistance to the rider.

[0006] The present invention has been made in light of the above-mentioned problems, and aims to provide an image data generation device used in an assistance system that assists the driving of a rider of a lean vehicle, which can reduce the delay before the assistance system starts providing assistance to the rider more than ever before. Another object of the present invention is to provide a lean vehicle equipped with an image data generation device configured as above. Another object of the present invention is to provide an image data generation method used in an assistance system that assists the driving of a rider of a lean vehicle, which can reduce the delay before the assistance system starts providing assistance to the rider more than ever before. [Means for solving the problem]

[0007] The image data generating device of the present invention is an image data generating device used in an assistance system that assists the driving of a rider of a lean vehicle, and is equipped with an input unit to which image data from an imaging device that detects the surrounding environment of the lean vehicle is input, a movement conversion unit that obtains first image data by moving and converting the image data at least in the vertical direction, and a rotation conversion unit that obtains second image data by rotating and converting the first image data.

[0008] A lean vehicle according to the present invention includes the image data generating device according to the present invention.

[0009] In addition, the image data generation method of the present invention is a method for generating image data used in an assistance system that assists the driving of a rider of a lean vehicle, and includes an input step in which image data from an imaging device that detects the surrounding environment of the lean vehicle is input, a movement transformation step in which first image data is obtained by moving and transforming the image data at least in the vertical direction, and a rotation transformation step in which second image data is obtained by rotating and transforming the first image data. [Effects of the Invention]

[0010] In the present invention, first image data is obtained by performing a translation transformation on the image data captured by the imaging device at least in the vertical direction. Then, the present invention obtains second image data by performing a rotation transformation on the first image data. By obtaining the second image data used to consider the necessity of driving assistance in this manner, the amount of calculation (time-consuming calculation) required to rotate and transform the image data can be reduced compared to conventional methods. Therefore, the present invention can reduce the delay before the assistance system starts providing assistance to the rider compared to conventional methods. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a side view of a lean vehicle equipped with an image data generating device according to an embodiment of the present invention; [Figure 2]1 is a schematic view of the area around the front wheels of a lean vehicle according to an embodiment of the present invention, observed from the front of the lean vehicle in a state in which the lean vehicle is tilted to the side. [Figure 3] 1 is a block diagram showing an example of a support system including an image data generating device according to an embodiment of the present invention; [Figure 4] 1 is a diagram showing an example of imaging data of an imaging device of a support system including an image data generating device according to an embodiment of the present invention; [Figure 5] 1 is a diagram showing an example of imaging data of an imaging device of a support system including an image data generating device according to an embodiment of the present invention; [Figure 6] 10A and 10B are diagrams for explaining rotational transformation of imaging data by a conventional support system. [Figure 7] 4 is a diagram showing an example of first image data obtained by a movement conversion unit of the image data generating device according to the embodiment of the present invention. FIG. [Figure 8] 5A and 5B are diagrams showing an example of second image data obtained by a rotation conversion unit of the image data generating device according to the embodiment of the present invention. [Figure 9] 3 is a diagram showing an example of a table stored in a storage unit of the image data generating device according to the embodiment of the present invention; FIG. [Figure 10] FIG. 4 is a diagram showing a control flow of an example of the operation of the image data generating device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] An image data generating device, a lean vehicle, and an image data generating method according to the present invention will be described below with reference to the drawings.

[0013] The configuration and operation described below are an example of the present invention, and the present invention is not limited to such configuration and operation.

[0014] For example, in the following, a motorcycle is used as an example of a lean vehicle. However, a lean vehicle generally refers to a vehicle whose body leans in the turning direction when turning. Therefore, a lean vehicle is not limited to a motorcycle. For example, lean vehicles include motorcycles (motorcycles and motor tricycles whose body leans in the turning direction when turning) whose body leans in the turning direction when turning, and bicycles. Furthermore, motorcycles whose body leans in the turning direction when turning may be propelled by an engine or an electric motor, and include, for example, motorcycles, scooters, and electric scooters. Furthermore, a bicycle generally refers to a vehicle that can be propelled on a road by the rider's pedaling force applied to the pedals. Bicycles include standard bicycles, electrically assisted bicycles, and electric bicycles.

[0015] In the following, descriptions of identical or similar parts are appropriately simplified or omitted. In addition, in each drawing, reference numerals are omitted for identical or similar parts or components, or the same reference numerals are used. In addition, illustrations of detailed structures are appropriately simplified or omitted.

[0016] Embodiment An image data generating device according to an embodiment, a lean vehicle equipped with the image data generating device, and a method for generating image data according to the embodiment will be described below.

[0017] <Configuration of a lean vehicle equipped with an image data generating device> The configuration of an image data generating device according to an embodiment and a lean vehicle equipped with the image data generating device will be described.

[0018] Fig. 1 is a side view of a lean vehicle equipped with an image data generating device according to an embodiment of the present invention. Fig. 2 is a schematic view of the area around the front wheels of the lean vehicle according to the embodiment of the present invention, observed from the front of the lean vehicle when the lean vehicle is tilted to the side. In Fig. 1, the right side of the paper surface is the front side of the lean vehicle 1. The lean vehicle 1 is, for example, a motorcycle. The lean vehicle 1 includes a front wheel 3 and a handlebar 2 connected to the front wheel 3 and serving as an operating section for the front wheel 3. In this embodiment, the lean vehicle 1 also includes a suspension 4 between the vehicle body and the front wheel 3.

[0019] The lean vehicle 1 is equipped with an assistance system 10 that assists the rider of the lean vehicle 1 in driving. The assistance system 10 includes an imaging device 11, an image data generation device 20, and a control device 12. The imaging device 11 detects the surrounding environment of the lean vehicle 1 (takes an image of the surroundings). In this embodiment, the imaging device 11 detects the surrounding environment at least in front of the lean vehicle 1 (takes an image of the surroundings). The image data generation device 20 uses the imaging data of the imaging device 11 to generate image data used in the assistance system 10. A detailed configuration of the image data generation device 20 will be described later.

[0020] The control device 12 considers the necessity of an operation to assist the rider of the lean vehicle 1 in driving (for example, a warning function, an emergency braking function, a cruising function, etc.) based on the image data generated by the image data generating device 20, and executes the operation to assist if necessary. For example, part or all of the control device 12 is configured with a microcomputer, a microprocessor unit, etc. Furthermore, for example, part or all of the control device 12 may be configured with updatable firmware, etc., or may be a program module executed by commands from a CPU, etc. The control device 12 may be, for example, a single device or may be divided into multiple devices. The running condition detection device 40 shown in FIG. 1 detects the running condition of the lean vehicle 1, and will be described in detail later.

[0021] As shown in Fig. 2, the body of the lean vehicle 1 tilts in the turning direction when turning. Fig. 2 shows the lean vehicle 1 tilted in the turning direction at a lean angle α.

[0022] FIG. 3 is a block diagram showing an example of a support system including an image data generating device according to an embodiment of the present invention. The image data generating device 20 generates image data to be used in the assistance system 10 from the image data captured by the imaging device 11 based on the detection result of the running condition detecting device 40 that detects information on the running condition of the lean vehicle 1.

[0023] The running condition detection device 40 includes at least one detection device. The detection device constituting the running condition detection device 40 may be a detection device provided exclusively for the running condition detection device 40, or a detection device used for another purpose may be used as the detection device of the running condition detection device 40.

[0024] The running state detection device 40 includes, for example, a detection device that detects posture information in the pitching direction of the lean vehicle 1. The pitching direction is the direction indicated by the arc-shaped arrow P in FIG.

[0025] For example, the running condition detection device 40 includes a detection device that detects the pitching angle of the lean vehicle 1 as a detection device that detects attitude information in the pitching direction of the lean vehicle 1. This is because when the attitude of the lean vehicle 1 in the pitching direction changes, the pitching angle of the lean vehicle 1 also changes. The detection device that detects the pitching angle of the lean vehicle 1 may also detect a physical quantity that can be substantially converted into the pitching angle of the lean vehicle 1. In this embodiment, the running condition detection device 40 includes an inertial measurement device as a detection device that detects the pitching angle of the lean vehicle 1, and detects the pitching angle of the lean vehicle 1 using the inertial measurement device. The inertial measurement device includes a three-axis gyro sensor and a three-directional acceleration sensor, and outputs the detection results of the three-axial acceleration and three-axial angular velocity of the lean vehicle 1.

[0026] Furthermore, for example, the detection device that the traveling condition detection device 40 has for detecting posture information of the lean vehicle 1 in the pitching direction may be a detection device that detects the amount of expansion and contraction of the suspension 4 on the front wheel 3 side of the lean vehicle 1. This is because when the posture of the lean vehicle 1 in the pitching direction changes, the amount of expansion and contraction of the suspension 4 also changes. Specifically, the greater the amount of sinking of the front of the body of the lean vehicle 1 relative to the rear, the greater the amount of contraction of the suspension 4. Also, the greater the amount of lift of the front of the body of the lean vehicle 1 relative to the rear, the greater the amount of expansion of the suspension 4. Note that the detection device that detects the amount of expansion and contraction of the suspension 4 may be one that detects a physical quantity that can be substantially converted into the amount of expansion and contraction of the suspension 4.

[0027] Further, for example, the detection device for detecting posture information of the lean vehicle 1 in the pitching direction, which is provided in the traveling condition detection device 40, may be a detection device for detecting the acceleration / deceleration in the longitudinal direction of the lean vehicle 1. This is because, when the posture of the lean vehicle 1 in the pitching direction changes, the acceleration / deceleration in the longitudinal direction of the lean vehicle 1 also changes. Specifically, the greater the deceleration when the lean vehicle 1 moves forward, the greater the amount of front sinking of the body of the lean vehicle 1 relative to the rear. Furthermore, the greater the acceleration when the lean vehicle 1 moves forward, the greater the amount of front lift of the body of the lean vehicle 1 relative to the rear. Note that the detection device for detecting the acceleration / deceleration in the longitudinal direction of the lean vehicle 1 may be one that detects a physical quantity that can be substantially converted into the acceleration / deceleration in the longitudinal direction of the lean vehicle 1. In this embodiment, the above-mentioned inertial measurement unit is used as the detection device for detecting the acceleration / deceleration in the longitudinal direction of the lean vehicle 1. Note that the acceleration / deceleration in the longitudinal direction of the lean vehicle 1 may also be detected from the amount of change per unit time in the speed of the lean vehicle 1.

[0028] The running condition detection device 40 also includes, for example, a detection device that detects the steering angle of the lean vehicle 1. The detection device that detects the steering angle of the lean vehicle 1 may detect a physical quantity that can be substantially converted into the steering angle of the lean vehicle 1.

[0029] Furthermore, as the detection device included in the traveling condition detection device 40, for example, the imaging device 11 may be used.

[0030] The running state detection device 40 also includes, for example, a detection device that detects posture information in the tilt direction of the lean vehicle 1. Here, the tilt direction is the direction shown in FIG. 2, which is the direction in which the lean vehicle 1 tilts sideways.

[0031] For example, the traveling condition detection device 40 includes a detection device that detects at least one of the lean angle, yaw rate, and lateral acceleration of the lean vehicle 1 as a detection device that detects posture information in the lean direction of the lean vehicle 1. Note that the detection device that detects at least one of the lean angle, yaw rate, and lateral acceleration of the lean vehicle 1 may detect a physical quantity that can be substantially converted into this information. As the lean vehicle 1 leans sideways, the lean angle of the lean vehicle 1 also increases. Furthermore, the lean vehicle 1 leans in most cases when turning, and when the lean vehicle 1 turns, the yaw rate and lateral acceleration of the lean vehicle 1 generally increase and decrease in response to an increase or decrease in the lean angle of the lean vehicle 1. Therefore, the lean angle, yaw rate, and lateral acceleration of the lean vehicle 1 can be used as posture information in the lean direction of the lean vehicle 1.

[0032] When detecting multiple items of the lean angle, yaw rate, and lateral acceleration of the lean vehicle 1, the running condition detection device 40 may be provided with separate detection devices for detecting each item of information, or may detect at least two items of information using a single detection device. In this embodiment, the running condition detection device 40 uses the inertial measurement unit described above as a detection device for detecting posture information of the lean vehicle 1 in the tilt direction.

[0033] The image data generating device 20 may be partially or entirely configured with a microcomputer, microprocessor unit, or the like. Furthermore, for example, part or all of the image data generating device 20 may be configured with updatable components such as firmware, or may be a program module executed by commands from a CPU or the like. The image data generating device 20 may be, for example, a single unit, or may be divided into multiple units. Furthermore, at least a portion of the image data generating device 20 may be integrally formed with at least a portion of the control device 12. In this embodiment, the image data generating device 20 includes, as functional units, an input unit 21, a movement conversion unit 22, and a rotation conversion unit 23. Furthermore, the image data generating device 20 according to this embodiment also includes, as a functional unit, a storage unit 24.

[0034] The input unit 21 receives image data from the imaging device 11 that detects the surrounding environment of the lean vehicle 1. For example, the image data from the imaging device 11 is as follows.

[0035] 4 and 5 are diagrams showing examples of imaging data of an imaging device of a support system including an image data generating device according to an embodiment of the present invention.

[0036] The imaging data 30 of the imaging device 11 shown in Fig. 4 is imaging data when the lean vehicle 1 is traveling straight (not tilted) and no pitching is occurring in the lean vehicle 1. In this embodiment, the imaging device 11 is mounted on the lean vehicle 1 so that the vanishing point is approximately at the center of the imaging data 30. The area around the center position of the imaging data 30 is an area 33 required for the assistance system 10 to determine whether driving assistance is required. The position and size of the area 33 required for determining whether driving assistance is required are merely examples.

[0037] The image data 30 shown in Fig. 4 also shows a tree growing vertically. The base of this tree is located at the center of the image data 30 shown in Fig. 4. This tree is also located in the image data 30 shown in Fig. 5 and in image data described later. That is, in the image data 30 shown in Fig. 5 and in the image data described later, the base of the tree is located at the center of the image data 30 shown in Fig. 4 (image data in a state where no pitching occurs in the leaning vehicle 1).

[0038] Here, when the lean vehicle 1 turns, the vehicle body tilts in the turning direction. Therefore, when the lean vehicle 1 is in a tilted state, the imaging data 30 of the imaging device 11 mounted on the lean vehicle 1 also tilts. Furthermore, the lean vehicle 1 has a short length. Therefore, the lean vehicle 1 pitches more. When the posture information of the lean vehicle 1 in the pitching direction changes, the position that was centered before the pitching occurred in the imaging data 30 of the imaging device 11 moves in the vertical direction. Therefore, when pitching occurs in the lean vehicle 1 while turning, the imaging data 30 becomes as shown in FIG. 5.

[0039] In a conventional assistance system proposed for installation on a lean vehicle, the tilted image data 30 shown in Figure 5 is rotated and converted as follows and used to consider the need for assistance operations for the rider of the lean vehicle 1.

[0040] Fig. 6 is a diagram for explaining the rotational transformation of imaging data by a conventional support system. The hatched portion shown in image data 101 in Fig. 6 is a portion where image information is lost when the tilted imaging data 30 shown in Fig. 5 is rotationally transformed. This is because, in the image data after rotational transformation, only coordinate positions where image information exists both before and after the rotational transformation remain as images.

[0041] When generating image data 101 shown in FIG. 6 from imaging data 30 shown in FIG. 5, the conventional assistance system rotates imaging data 30 according to the tilt of imaging data 30, with the center of imaging data 30 shown in FIG. 5 as the center of rotation. As described above, when the conventional assistance system rotates tilted imaging data 30 shown in FIG. 5, it must also rotate regions of imaging data 30 that are not necessary for determining whether driving assistance is required, resulting in a large amount of calculation. Furthermore, the calculations required for rotational conversion of tilted imaging data 30 shown in FIG. 5 involve sine and cosine calculations, which takes time. As a result, the conventional assistance system experiences a large delay before starting rider assistance operations.

[0042] On the other hand, the image data generating device 20 according to this embodiment generates image data used to consider the necessity of providing assistance to the rider, using the movement conversion unit 22 and the rotation conversion unit 23. Therefore, as will be described later, in the assistance system 10, the delay before the assistance to the rider is started can be reduced more than in the past.

[0043] The movement conversion unit 22 obtains first image data by performing movement conversion on the imaging data 30 of the imaging device 11 at least in the vertical direction. Specifically, when pitching occurs in the lean vehicle 1 while the lean vehicle 1 is tilted, the movement conversion unit 22 obtains first image data by performing movement conversion on the imaging data 30 of the imaging device 11 at least in the vertical direction. More specifically, when pitching occurs in the body of the lean vehicle 1, in which the front sinks relative to the rear, the movement conversion unit 22 obtains first image data by performing movement conversion on the imaging data 30 of the imaging device 11 at least in the downward direction. Furthermore, when pitching occurs in the body of the lean vehicle 1, in which the front rises relative to the rear, the movement conversion unit 22 obtains first image data by performing movement conversion on the imaging data 30 of the imaging device 11 at least in the upward direction. That is, the movement conversion unit 22 obtains first image data by performing movement conversion on the imaging data 30 shown in FIG. 5 at least in the vertical direction.

[0044] Fig. 7 is a diagram showing an example of first image data calculated by a motion conversion unit of an image data generating device according to an embodiment of the present invention. The first image data 31 shown in Fig. 7 was calculated using the imaging data 30 shown in Fig. 5. The hatched portion shown in the first image data 31 in Fig. 7 is a portion where image information is missing because no image information exists in the imaging data 30 shown in Fig. 5. As can be seen from a comparison between the first image data 31 shown in Figure 7 and the imaging data 30 shown in Figure 5, the center of the first image data 31 shown in Figure 7 is closer to the position where the center of the imaging data 30 was when no pitching occurred in the leaning vehicle 1 (the position where the center was in the imaging data 30 in Figure 4, i.e., the position of the tree base) than the center of the imaging data 30 shown in Figure 5.

[0045] For example, the amount of movement between the imaging data 30 and the first image data 31 may be a fixed value. In this case, for example, when the pitching of the body of the lean vehicle 1, in which the front sinks relative to the rear, becomes greater than a specified pitching amount, the movement conversion unit 22 converts the first image data 31 into the imaging data 30 by moving it downward by a specified value. Also, for example, when the pitching of the body of the lean vehicle 1, in which the front rises relative to the rear, becomes greater than a specified pitching amount, the movement conversion unit 22 converts the first image data 31 into the imaging data 30 by moving it upward by a specified value.

[0046] Furthermore, the movement conversion unit 22 may determine the amount of movement between the imaging data 30 and the first image data 31 based on posture information in the pitching direction of the lean vehicle 1. That is, the movement conversion unit 22 may determine a larger value for the amount of movement between the imaging data 30 and the first image data 31 as the pitching amount of the lean vehicle 1 increases. Specifically, the movement conversion unit 22 moves the imaging data 30 downward more when calculating the first image data 31 as the pitching amount of the front of the lean vehicle 1 decreases relative to the rear of the vehicle body increases. Furthermore, for example, the movement conversion unit 22 moves the imaging data 30 upward more when calculating the first image data 31 as the pitching amount of the front of the lean vehicle 1 increases relative to the rear of the vehicle body increases.

[0047] The method of changing the amount of movement between the image data 30 and the first image data 31 is arbitrary. As the pitching amount of the lean-in vehicle 1 increases, the amount of movement between the image data 30 and the first image data 31 may be increased linearly, or the amount of movement between the image data 30 and the first image data 31 may be increased in a stepwise manner. By determining the amount of movement between the image data 30 and the first image data 31 based on posture information in the pitching direction of the lean-in vehicle 1, the center of the first image data 31 becomes closer to the position that was the center of the image data when the lean-in vehicle 1 was not pitching (the position that was the center of the image data 30 in FIG. 4, i.e., the position of the tree roots) compared to the center of the image data 30 when the lean-in vehicle 1 was pitching.

[0048] For example, the movement conversion unit 22 uses the pitching angle of the lean vehicle 1 as posture information in the pitching direction of the lean vehicle 1. That is, the greater the pitching angle in the direction in which the front of the body of the lean vehicle 1 sinks relative to the rear, the greater the movement conversion unit 22 moves the imaging data 30 downward when obtaining the first image data 31. Also, the greater the pitching angle in the direction in which the front of the body of the lean vehicle 1 rises relative to the rear, the greater the movement conversion unit 22 moves the imaging data 30 upward when obtaining the first image data 31.

[0049] For example, the movement conversion unit 22 uses the amount of expansion and contraction of the suspension 4 on the front wheel 3 side of the lean vehicle 1 as posture information in the pitching direction of the lean vehicle 1. That is, the greater the amount of contraction of the suspension 4, the greater the movement conversion unit 22 moves the imaging data 30 downward when calculating the first image data 31. Also, the greater the amount of expansion of the suspension 4, the greater the movement conversion unit 22 moves the imaging data 30 upward when calculating the first image data 31.

[0050] For example, the movement conversion unit 22 uses the acceleration and deceleration in the longitudinal direction of the lean vehicle 1 as posture information in the pitching direction of the lean vehicle 1. That is, the greater the deceleration when the lean vehicle 1 is moving forward, the greater the movement conversion unit 22 moves the imaging data 30 downward when calculating the first image data 31. Also, the greater the acceleration when the lean vehicle 1 is moving forward, the greater the movement conversion unit 22 moves the imaging data 30 upward when calculating the first image data 31.

[0051] The movement conversion unit 22 may use, as the posture information in the pitching direction of the lean vehicle 1, two or more of the pitching angle of the lean vehicle 1, the expansion / contraction amount of the suspension 4, and the acceleration / deceleration in the longitudinal direction, to determine the amount of movement between the imaging data 30 and the first image data 31. As a result, the center of the first image data 31 becomes closer to the position where the center of the imaging data was when the lean vehicle 1 was not pitching (the position where the center was in the imaging data 30 in FIG. 4, i.e., the position of the tree roots) compared to the center of the imaging data 30 when the lean vehicle 1 was pitching.

[0052] Here, it is assumed that the imaging device 11 is provided directly on the steering wheel 2 or on a structure that moves together with the steering wheel 2. That is, it is assumed that the imaging device 11 moves together with the steering wheel 2 of the lean-in vehicle 1. In such a configuration, when the steering wheel 2 is turned, the position in the imaging data 30 that was the center of the imaging data when no pitching occurred in the lean-in vehicle 1 moves in the left-right direction opposite to the direction in which the steering wheel 2 is turned.

[0053] Therefore, when the movement conversion unit 22 uses at least one of the pitching angle of the lean vehicle 1, the expansion / contraction amount of the suspension 4, and the acceleration / deceleration in the longitudinal direction as the posture information in the pitching direction of the lean vehicle 1, the movement conversion unit 22 may further determine the amount of movement (more specifically, the amount of movement in the lateral direction) between the imaging data 30 and the first image data 31 based on the steering angle of the lean vehicle 1. As a result, the center of the first image data 31 becomes closer to the position where the center of the imaging data was when the lean vehicle 1 was not pitching (the position where the center was in the imaging data 30 in FIG. 4, i.e., the position of the tree roots) compared to the center of the imaging data 30 when the lean vehicle 1 was pitching.

[0054] Further, for example, the movement conversion unit 22 may determine the amount of movement between the imaging data 30 and the first image data 31 based on the imaging data 30 of the imaging device 11. By comparing the imaging data 30 when the lean vehicle 1 is not tilting and is not pitching with the imaging data 30 when the lean vehicle 1 is pitching while leaning, the amount of movement in the up / down and left / right directions between the imaging data 30 and the first image data 31 can be directly determined. Note that in the first embodiment, when the lean vehicle 1 is not tilting and is not pitching, the vanishing point of the imaging data 30 is the center of the imaging data 30. On the other hand, in the imaging data 30 when the lean vehicle 1 is pitching while leaning, the vanishing point is shifted from the center. In the present embodiment, the amount of movement between the imaging data 30 and the first image data 31 is determined based on the amount of shift of the vanishing point.

[0055] In this embodiment, the motion transformation unit 22 obtains the first image data 31 by so-called inverse transformation. That is, the motion transformation unit 22 according to this embodiment is configured to obtain the first image data 31 by image transformation in which coordinate information (pixel number, pixel coordinate value, etc.) of the first image data 31 is input and coordinate information of the imaging data 30 is output. By obtaining the first image data 31 by inverse transformation, it is possible to suppress image omissions that occur in the first image data 31 when the first image data 31 is generated by so-called forward transformation. Of course, the motion transformation unit 22 may obtain the first image data 31 by forward transformation. That is, the motion transformation unit 22 may be configured to obtain the first image data 31 by image transformation in which coordinate information of the imaging data 30 is input and coordinate information of the first image data 31 is output.

[0056] The rotation conversion unit 23 obtains second image data by rotationally converting the first image data 31. Specifically, the rotation conversion unit 23 obtains second image data by rotationally converting the first image data 31 in a direction that reduces the tilt of the captured object. This second image data is used by the control device 12 to consider the need for driving assistance. That is, the rotation conversion unit 23 obtains second image data by rotationally converting the first image data 31 shown in FIG. 7.

[0057] Fig. 8 is a diagram showing an example of second image data obtained by a rotation conversion unit of an image data generating device according to an embodiment of the present invention. The second image data 32 shown in Fig. 8 was obtained using the first image data 31 shown in Fig. 7. The hatched portion shown in the second image data 32 in Fig. 8 is a portion where image information was lost when the first image data 31 shown in Fig. 7 was rotationally converted.

[0058] As can be seen from the image data 101 shown in Fig. 6 and the second image data 32 shown in Fig. 8, the portion of the second image data 32 shown in Fig. 8 where image information is missing is larger than the portion of the image information missing in the image data 101 shown in Fig. 6 by the portion of the image information where image information was missing when the first image data 31 was generated. In other words, the second image data 32 shown in Fig. 8 can reduce the amount of calculation (time-consuming sine and cosine calculations) required to rotate and transform the image data compared to the image data 101 shown in Fig. 6.

[0059] Furthermore, as can be seen from the image data 101 shown in FIG. 6 and the second image data 32 shown in FIG. 8, image information around corners is lost in the image data after rotational transformation. For this reason, if the area 33 necessary for determining whether driving assistance is required is located around the center in the image data before rotational transformation, image information loss in the area 33 is less likely to occur in the image data after rotational transformation. Here, in the first image data 31, the area 33 necessary for determining whether driving assistance is required in the assistance system 10 is located around the center. For this reason, an imaging device with a smaller imaging range (smaller number of pixels) than conventional imaging devices can be used as the imaging device 11 according to this embodiment. From this perspective, the second image data 32 can reduce the amount of calculation required for rotational transformation of image data compared to the image data 101.

[0060] Furthermore, as described above, in the first image data 31, the area 33 necessary for determining whether or not driving assistance is required in the assistance system 10 is located around the center. Therefore, the second image data 32 can be generated simply by rotationally transforming the area around the center of the first image data 31. In this respect, the second image data 32 can reduce the amount of calculation required to rotate and transform image data compared to the image data 101.

[0061] In this way, the second image data 32 can reduce the amount of calculation required to rotate and convert image data compared to the image data 101. Therefore, the image data generation device 20 according to this embodiment can reduce the delay before the assistance system 10 starts providing assistance to the rider, more than ever before.

[0062] In conventional assistance systems, sine and cosine calculations are performed in a calculation library. Calculating sine and cosine in this calculation library takes time. Therefore, the image data generation device 20 according to the present embodiment has the following configuration to further reduce the calculation time required to perform rotational transformation on the first image data 31, thereby further minimizing the delay before the assistance system 10 starts providing assistance to the rider.

[0063] FIG. 9 is a diagram showing an example of a table stored in the storage unit of the image data generating device according to the embodiment of the present invention. As described above, the image data generating device 20 according to this embodiment includes a storage unit 24. The storage unit 24 stores a table 50, which stores sine values of multiple angles and cosine values of multiple angles, as shown in FIG. 9 . The rotational transformation unit 23 uses the table 50 when calculating the second image data 32. By referencing the sine and cosine values of angles required for rotational transformation of the first image data 31 from the table 50, the calculation time required for rotational transformation of the first image data 31 can be reduced compared to when sine and cosine values are calculated using a calculation library. Therefore, by using the table 50 when calculating the second image data 32, the delay before the assistance system 10 starts providing assistance to the rider can be further reduced.

[0064] Here, for example, the amount of rotation between the first image data 31 and the second image data 32 may be a fixed value. In this case, for example, when the lean angle of the lean vehicle 1 becomes equal to or greater than a specified lean angle, the rotation conversion unit 23 converts the second image data 32 into the first image data 31 by rotating it in a direction that reduces the tilt by the specified angle.

[0065] Furthermore, the rotation conversion unit 23 may determine the amount of rotation between the first image data 31 and the second image data 32 based on posture information in the tilt direction of the lean vehicle 1. That is, the rotation conversion unit 23 may determine the amount of rotation between the first image data 31 and the second image data 32 to be a larger value as the tilt angle of the lean vehicle 1 increases.

[0066] The method of changing the amount of rotation between the first image data 31 and the second image data 32 is arbitrary. As the lean angle of the lean vehicle 1 increases, the amount of rotation between the first image data 31 and the second image data 32 may be increased linearly, or the amount of rotation between the first image data 31 and the second image data 32 may be increased in a stepwise manner. By determining the amount of rotation between the first image data 31 and the second image data 32 based on posture information in the lean direction of the lean vehicle 1, the inclination of the second image data 32 becomes smaller, improving the detection accuracy of the surrounding environment of the lean vehicle 1, and thereby improving the assistance accuracy of the assistance system 10.

[0067] For example, the rotation conversion unit 23 uses the lean angle of the lean vehicle 1 as the attitude information of the lean vehicle 1 in the tilt direction. Also, for example, the rotation conversion unit 23 uses the yaw rate of the lean vehicle 1 as the attitude information of the lean vehicle 1 in the tilt direction. Also, for example, the rotation conversion unit 23 uses the lateral acceleration of the lean vehicle 1 as the attitude information of the lean vehicle 1 in the tilt direction.

[0068] The rotation conversion unit 23 may use two or more of the lean angle, yaw rate, and lateral acceleration of the lean vehicle 1 as posture information in the lean direction of the lean vehicle 1 to determine the amount of rotation between the first image data 31 and the second image data 32. This makes it possible to more accurately grasp the lean angle of the lean vehicle 1 and reduce the inclination of the second image data 32, thereby improving the accuracy of detecting the environment around the lean vehicle 1 and the accuracy of assistance in the assistance system 10.

[0069] Furthermore, in this embodiment, the rotational transformation unit 23 obtains the second image data 32 by inverse transformation. That is, the rotational transformation unit 23 according to this embodiment is configured to obtain the second image data 32 by image transformation in which coordinate information of the second image data 32 is input and coordinate information of the first image data 31 is output. By obtaining the second image data 32 by inverse transformation, it is possible to prevent image omissions that occur in the second image data 32 when the second image data 32 is generated by forward transformation. Of course, the rotational transformation unit 23 may also obtain the second image data 32 by forward transformation. That is, the rotational transformation unit 23 may also be configured to obtain the second image data 32 by image transformation in which coordinate information of the first image data 31 is input and coordinate information of the second image data 32 is output.

[0070] <Operation of the image data generating device> The operation of the image data generating device 20 according to the embodiment will be described.

[0071] FIG. 10 is a diagram showing a control flow of an example of the operation of the image data generating device according to the embodiment of the present invention. When the conditions for starting the control are met, in step S1, the image data generating device 20 starts the control shown in Fig. 10. The conditions for starting the control are, for example, when the lean vehicle 1 tilts and pitches.

[0072] Step S2 after step S1 is an input step. In step S2, the imaging data 30 of the imaging device 11 is input to the input unit 21 of the image data generating device 20.

[0073] Step S3 after step S2 is a translation transformation step. In step S3, the translation transformation unit 22 of the image data generation device 20 obtains first image data 31 by translation-transforming the imaging data 30 at least in the vertical direction using one of the methods described above.

[0074] Step S4 after step S3 is a rotation transformation step. In step S4, the rotation transformation unit 23 of the image data generation device 20 obtains second image data 32 by rotationally transforming the first image data 31 using one of the methods described above. The second image data 32 is used by the control device 12 to consider the need for an operation to assist the rider in driving, etc.

[0075] Step S5 after step S4 is an end determination step. In step S5, the image data generating device 20 determines whether or not the control end condition has been met. The control end condition is, for example, when tilting or pitching no longer occurs in the lean vehicle 1. If the control end condition has been met, the image data generating device 20 proceeds to step S6 and ends the control shown in FIG. 10. On the other hand, if the control end condition has not been met, the image data generating device 20 repeats steps S2 to S5.

[0076] <Effects of the image data generation device> The image data generating device 20 according to this embodiment is an image data generating device used in an assistance system 10 that assists the driving of a rider of a lean vehicle 1. The image data generating device 20 includes an input unit 21 to which image data 30 from an imaging device 11 that detects the surrounding environment of the lean vehicle 1 is input, a movement conversion unit 22 that obtains first image data 31 by performing movement conversion on the image data 30 at least in the vertical direction, and a rotation conversion unit 23 that obtains second image data 32 by performing rotation conversion on the first image data 31.

[0077] The image data generating device 20 configured in this manner obtains first image data 31 by shifting and converting the image data 30 from the imaging device 11 at least in the vertical direction. The image data generating device 20 then rotates and converts the first image data 31 to obtain second image data 32 used to consider the need for driving assistance. By obtaining the second image data 32 in this manner, the amount of calculation required for rotating and converting image data can be reduced compared to conventional methods. Therefore, the image data generating device 20 configured in this manner can reduce the delay before the assistance system 10 starts providing assistance to the rider compared to conventional methods.

[0078] The image data generating device 20 according to this embodiment has been described above, but the image data generating device according to the present invention is not limited to the description of this embodiment. For example, the image data generating device according to the present invention may be implemented in only a part of this embodiment. Furthermore, for example, the image data generating device according to the present invention may be used in an assistance system equipped with an imaging device that detects the surrounding environment at least behind a leaning vehicle. [Explanation of symbols]

[0079] 1 lean vehicle, 2 steering wheel, 3 front wheel, 4 suspension, 10 assistance system, 11 imaging device, 12 control device, 20 image data generation device, 21 input unit, 22 movement conversion unit, 23 rotation conversion unit, 24 memory unit, 30 imaging data, 31 first image data, 32 second image data, 33 area, 40 running condition detection device, 50 table, 101 image data (conventional).

Claims

1. An image data generating device (20) used in an assistance system (10) that assists a rider in driving a lean vehicle (1), comprising: an input unit (21) to which image data (30) of an image pickup device (11) that detects the surrounding environment of the lean vehicle (1) is input; a translation transformation unit (22) for obtaining first image data (31) by translation-transforming the imaging data (30) at least in the vertical direction; a rotation transformation unit (23) for obtaining second image data (32) by rotationally transforming the first image data (31); An image data generating device (20) comprising:

2. The movement conversion unit (22) is configured to determine the amount of movement between the imaging data (30) and the first image data (31) based on posture information in the pitching direction of the lean vehicle (1). The image data generating device (20) of claim 1.

3. The pitching angle of the lean vehicle (1) is used as the attitude information in the pitching direction. The image data generating device (20) of claim 2.

4. The amount of expansion and contraction of the suspension (4) on the front wheel (3) side of the lean vehicle (1) is used as the attitude information in the pitching direction. An image data generating device (20) according to claim 2 or claim 3.

5. The acceleration / deceleration in the longitudinal direction of the lean vehicle (1) is used as the posture information in the pitching direction. The image data generating device (20) according to any one of claims 2 to 4.

6. When mounted on the lean vehicle, the imaging device (11) moves together with a steering wheel (2) of the lean vehicle, The movement conversion unit (22) is configured to determine the amount of movement between the imaging data (30) and the first image data (31) based on the posture information in the pitching direction and the steering angle of the leaning vehicle (1). The image data generating device (20) according to any one of claims 2 to 5.

7. The movement conversion unit (22) is configured to determine the amount of movement between the imaging data (30) and the first image data (31) based on the imaging data (30). The image data generating device (20) of claim 1.

8. The rotation conversion unit (23) is configured to determine the amount of rotation between the first image data (31) and the second image data (32) based on posture information of the lean vehicle (1) in the tilt direction. The image data generating device (20) according to any one of claims 1 to 7.

9. The movement conversion unit (22) is configured to receive coordinate information of the first image data (31) and to obtain the first image data (31) by image conversion in which coordinate information of the imaging data (30) is output. The image data generating device (20) according to any one of claims 1 to 8.

10. The movement transformation unit (22) is configured to receive coordinate information of the imaging data (30) and to obtain the first image data (31) by image transformation in which coordinate information of the first image data (31) is output. The image data generating device (20) according to any one of claims 1 to 8.

11. The rotation transformation unit (23) is configured to receive coordinate information of the second image data (32) and to obtain the second image data (32) by image transformation in which coordinate information of the first image data (31) is output. The image data generating device (20) according to any one of claims 1 to 10.

12. The rotation transformation unit (23) is configured to receive coordinate information of the first image data (31) and to obtain the second image data (32) by image transformation in which coordinate information of the second image data (32) is output. The image data generating device (20) according to any one of claims 1 to 10.

13. a storage unit (24) storing a table (50) in which values of sine of a plurality of angles and values of cosine of a plurality of angles are stored; The rotation conversion unit (23) is configured to use the table (50) when determining the second image data (32). The image data generating device (20) according to any one of claims 1 to 9.

14. A lean vehicle (1) comprising the image data generating device (20) according to any one of claims 1 to 13.

15. A method for generating image data used in an assistance system (10) that assists a rider in driving a lean vehicle (1), comprising: an input step (S2) of inputting image data (30) from an image pickup device (11) that detects the surrounding environment of the lean vehicle (1); a translation transformation step (S3) of obtaining first image data (31) by translation-transforming the imaging data (30) at least in the vertical direction; a rotational transformation step (S4) of obtaining second image data (32) by rotationally transforming the first image data (31); An image data generation method comprising:

Images