Hitch angle estimation device, hitch angle estimation method, and program

Abstract

To provide a hitch angle estimation device used for learning a model capable of estimating a hitch angle by using images of a training trailer that does not include sensors or markers.SOLUTION: A hitch angle estimation device 14 includes: an acquisition unit 3A that acquires images of a trailer 2, which are picked up by a camera 11 mounted on a vehicle 1 towing the trailer 2; and an estimation unit 3B that estimates the hitch angle of the trailer 2 based on the images of the trailer 2 acquired by the acquisition unit 3A by using a model that is obtained by learning an image of the learning trailer taken by the learning camera mounted on the learning vehicle towing the learning trailer and the training data, which is a data set with labels indicating the hitch angle of the training trailer. As the hitch angle of the training trailer, the difference between the heading of the training vehicle and the heading of the training trailer is used.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a hitch angle estimation device, a hitch angle estimation method, and a program. [Background technology]

[0002] Patent Document 1 describes that a deep neural network (DNN) can calculate the angular difference between the front and rear axles of a towing vehicle and the front and rear axles of a trailer. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2020-535077 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, Patent Document 1 does not describe a method for obtaining the hitch angle of a training trailer, which must be input to the DNN as training data together with the image during training. If a sensor, marker, etc. for obtaining the hitch angle of the training trailer were included in the image used for training the DNN, the DNN would not be able to train properly, and as a result, the DNN may not be able to properly estimate the trailer hitch angle during inference based on an image that does not include the sensor, marker, etc.

[0005] In view of the above, an object of the present disclosure is to provide a hitch angle estimation device, a hitch angle estimation method, and a program that can use images of an appropriate training trailer that do not include sensors, markers, etc., to train a model used to estimate the hitch angle. is. [Means for solving the problem]

[0006] (1) One aspect of the present disclosure is a hitch angle estimation device that includes an acquisition unit that acquires images of the trailer taken by a camera mounted on a vehicle towing the trailer, and an inference unit that estimates the hitch angle of the trailer based on the images of the trailer acquired by the acquisition unit by using a model obtained by learning using training data, which is a dataset of images of the training trailer taken by a training camera mounted on a training vehicle towing the training trailer and labels indicating the hitch angle of the training trailer, and in which the hitch angle of the training trailer is determined by using the difference between the orientation of the training vehicle calculated based on a GPS signal received by a GPS receiver mounted on the training vehicle and the orientation of the training trailer calculated based on a GPS signal received by a GPS receiver mounted on the training trailer.

[0007] (2) One aspect of the present disclosure is a hitch angle estimation method comprising: an acquisition step in which a hitch angle estimation device acquires an image of the trailer taken by a camera mounted on a vehicle towing the trailer; and an inference step in which the hitch angle estimation device estimates the hitch angle of the trailer based on the image of the trailer acquired in the acquisition step by using a model obtained by learning using training data, which is a dataset of images of the training trailer taken by a training camera mounted on a training vehicle towing the training trailer and labels indicating the hitch angle of the training trailer; wherein the hitch angle of the training trailer is determined by using the difference between the orientation of the training vehicle calculated based on a GPS signal received by a GPS receiver mounted on the training vehicle and the orientation of the training trailer calculated based on a GPS signal received by a GPS receiver mounted on the training trailer.

[0008] (3) One aspect of the present disclosure is a program for causing a processor to execute an acquisition step of acquiring an image of the trailer taken by a camera mounted on a vehicle towing the trailer, and an inference step of estimating the hitch angle of the trailer based on the image of the trailer acquired in the acquisition step by using a model obtained by learning using training data, which is a dataset of images of the training trailer taken by a training camera mounted on a training vehicle towing the training trailer and labels indicating the hitch angle of the training trailer, wherein the program uses, as the hitch angle of the training trailer, the difference between the orientation of the training vehicle calculated based on a GPS signal received by a GPS receiver mounted on the training vehicle and the orientation of the training trailer calculated based on a GPS signal received by a GPS receiver mounted on the training trailer. [Effects of the Invention]

[0009] According to the present disclosure, images of suitable training trailers that do not include sensors, markers, etc. may be used to train a model used to estimate the hitch angle. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating an example of a vehicle to which a hitch angle estimation device according to a first embodiment is applied. [Figure 2] FIG. 2 is a diagram showing an example of the relationship between the vehicle and the trailer shown in FIG. [Figure 3] FIG. 2 is a diagram showing an example of a training vehicle and a training trailer used to obtain a model used by an inference unit. [Figure 4] 4 is a flowchart illustrating an example of processing executed in the hitch angle estimation device of the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of a hitch angle estimation device, a hitch angle estimation method, and a program according to the present disclosure will be described with reference to the drawings.

[0012] First Embodiment Fig. 1 is a diagram showing an example of a vehicle 1 to which a hitch angle estimation device 14 of the first embodiment is applied. Fig. 2 is a diagram showing an example of the relationship between the vehicle 1 and trailer 2 shown in Fig. 1. In detail, Fig. 2(A) is a diagram showing the vehicle 1 and trailer 2 as viewed from above, and Fig. 2(B) is a diagram showing an example of an image IM of the trailer 2 captured by a camera 11 mounted on the vehicle 1. In the example shown in Figures 1 and 2, a vehicle 1 tows a trailer 2. The vehicle 1 is equipped with a camera 11, an HMI (Human Machine Interface) 12, a vehicle control device 13, a steering actuator 13A, a braking actuator 13B, a driving actuator 13C, and a hitch angle estimation device 14. The camera 11 is disposed, for example, at the rear end 1R of the vehicle 1. The camera 11 captures an image of the rear of the vehicle 1 (the right side in Figure 2(A)) and transmits an image (for example, a fisheye lens image) IM (see Figure 2(B)) of the trailer 2 to the hitch angle estimation device 14. As shown in FIG. 2, the trailer 2 is connected to the vehicle 1 so as to be rotatable around a hitch ball (not shown).

[0013] The HMI 12 has functions such as accepting various operations by the driver of the vehicle 1, and transmits signals indicating the operations by the driver of the vehicle 1 to the vehicle control device 13. The vehicle control device 13 controls the steering actuator 13A, the braking actuator 13B, and the drive actuator 13C based on the signals transmitted from the HMI 12.

[0014] The hitch angle estimation device 14 is configured by a microcomputer equipped with a communication interface (I / F) 141, a memory 142, and a processor 143. The communication interface 141 has an interface circuit for connecting the hitch angle estimation device 14 to the camera 11, the HMI 12, and the vehicle control device 13. The memory 142 stores programs and various data used in the processing executed by the processor 143. The processor 143 has a function as an acquisition unit 3A and a function as an inference unit 3B. The acquisition unit 3A acquires an image IM of the trailer 2 captured by the camera 11. The inference unit 3B estimates the hitch angle θ (see FIG. 2(A)) of the trailer 2 based on the image IM of the trailer 2 acquired by the acquisition unit 3A, using a model described below.

[0015] Fig. 3 is a diagram showing an example of a training vehicle L1 and a training trailer L2 used to obtain a model used by the inference unit 3B. In detail, Fig. 3(A) is a diagram showing the training vehicle L1 and the training trailer L2 from above, Fig. 3(B) is a diagram showing an example of components of the training vehicle L1, and Fig. 3(C) is a diagram showing an example of components of the training trailer L2. In the example shown in Fig. 3, the training trailer L2 is equipped with a GPS (Global Positioning System) receiver L21 and a communication device L22. The GPS receiver L21 receives a GPS signal and calculates the orientation D2 [deg] (see Fig. 3(A)) of the training trailer L2 based on the GPS signal. The communication device L22 transmits the orientation D2 of the training trailer L2 calculated by the GPS receiver L21 to the training vehicle L1. The training vehicle L1 is equipped with a training camera L11, a GPS receiver L12, a communication device L13, and a learning device L14. Like the camera 11 of the vehicle 1, the training camera L11 is disposed at the rear end of the training vehicle L1 and captures the rear of the training vehicle L1 (the right side in FIG. 3A). The GPS receiver L12 receives a GPS signal and calculates the orientation D1 [deg] of the training vehicle L1 based on the GPS signal. The communication device L13 receives the orientation D2 of the training trailer L2 transmitted from the communication device L22 of the training trailer L2. The learning device L14 calculates the difference (D1-D2) between the orientation D1 of the training vehicle L1 and the orientation D2 of the training trailer L2 as the hitch angle Φ of the training trailer L2 (see FIG. 2A). Furthermore, the learning device L14 generates a model used by the inference unit 3B of the hitch angle estimation device 14 (trains the model). In detail, the learning device L14 generates a model to be used by the inference unit 3B of the hitch angle estimation device 14 by performing learning using learning data, which is a data set of images of the training trailer L2 taken by the training camera L11 and labels indicating the hitch angle Φ of the training trailer L2.

[0016] FIG. 4 is a flowchart illustrating an example of processing executed in the hitch angle estimation device 14 of the first embodiment. In the example shown in FIG. 4, the acquisition unit 3A acquires an image IM of the trailer 2 taken by the camera 11 in step S10. In step S11, the inference unit 3B estimates the hitch angle θ of the trailer 2 based on the image IM of the trailer 2 acquired in step S10 by using a model obtained by performing learning using learning data, which is a data set of images of the training trailer L2 taken by the training camera L11 and labels indicating the hitch angle Φ of the training trailer L2. The hitch angle Φ of the training trailer L2 is determined by using the difference (D1-D2) between the orientation D1 of the training vehicle L1 calculated based on the GPS signal received by the GPS receiver L12 mounted on the training vehicle L1 and the orientation D2 of the training trailer L2 calculated based on the GPS signal received by the GPS receiver L21 mounted on the training trailer L2. In the example shown in FIGS. 1 to 4, the model used to estimate the hitch angle θ of the trailer 2 can be learned using an image of the training trailer L2 that does not include sensors, markers, etc.

[0017] Second Embodiment As described above, in the first embodiment (the example shown in FIGS. 1 to 4), the learning device L14 that performs learning using learning data to generate a model is mounted on the learning vehicle L1. On the other hand, in the second embodiment, a learning device (learning computer) that generates a model by performing learning using learning data does not necessarily have to be mounted on the learning vehicle L1.

[0018] In the first embodiment (the example shown in FIGS. 1 to 4), the learning camera L11, GPS receiver L12, and learning device L14 of the learning vehicle L1 are interconnected. On the other hand, in the second embodiment, the learning camera L11 of the learning vehicle L1 is not connected to the learning computer, and the GPS receiver L12 of the learning vehicle L1 is not connected to the learning computer. That is, in the second embodiment, an image of the learning trailer L2 captured by the learning camera L11 is input to the learning computer by, for example, an annotator, and the orientation D1 of the learning vehicle L1 calculated by the GPS receiver L12 is input to the learning computer by, for example, an annotator.

[0019] In the first embodiment (the example shown in FIGS. 1 to 4), the learning vehicle L1 is equipped with a communication device L13, and the learning trailer L2 is equipped with a communication device L22. On the other hand, in the second embodiment, the learning vehicle L1 does not have to be equipped with the communication device L13, and the learning trailer L2 does not have to be equipped with the communication device L22. That is, in the second embodiment, the orientation D2 of the learning trailer L2 calculated by the GPS receiver L21 is input to the learning computer by, for example, an annotator.

[0020] As described above, embodiments of the hitch angle estimation device, hitch angle estimation method, and program of the present disclosure have been described with reference to the drawings. However, the hitch angle estimation device, hitch angle estimation method, and program of the present disclosure are not limited to the above-described embodiments and may be modified as appropriate without departing from the spirit and scope of the present disclosure. The configurations of the above-described embodiments may be combined as appropriate. In the above-described embodiments, the processing performed by the hitch angle estimation device 14 has been described as software processing performed by executing a program. However, the processing performed by the hitch angle estimation device 14 may also be hardware processing. Alternatively, the processing performed by the hitch angle estimation device 14 may be a combination of both software and hardware. Furthermore, the program stored in the memory 142 of the hitch angle estimation device 14 (the program that realizes the functions of the processor 143 of the hitch angle estimation device 14) may be provided, distributed, etc., by being recorded on a computer-readable storage medium such as a semiconductor memory, a magnetic recording medium, an optical recording medium, etc. [Explanation of symbols]

[0021] 1...vehicle, 1R...rear end, 11...camera, 12...HMI, 13...vehicle control device, 13A...steering actuator, 13B...braking actuator, 13C...driving actuator, 14...hitch angle estimation device, 141...communication interface, 142...memory, 143...processor, 3A...acquisition unit, 3B...inference unit, 2...trailer, L1...learning vehicle, L11...learning camera, L12...GPS receiver, L13...communication device, L14...learning device, L2...learning trailer, L21...GPS receiver, L22...communication device

Claims

1. an acquisition unit that acquires an image of the trailer taken by a camera mounted on a vehicle towing the trailer; an inference unit that estimates the hitch angle of the trailer based on the trailer image acquired by the acquisition unit by using a model obtained by performing learning using learning data, which is a data set of images of the training trailer taken by a training camera mounted on a training vehicle that tows the training trailer and labels indicating the hitch angle of the training trailer; The hitch angle of the learning trailer is determined by a difference between the orientation of the learning vehicle calculated based on a GPS signal received by a GPS receiver mounted on the learning vehicle and the orientation of the learning trailer calculated based on a GPS signal received by a GPS receiver mounted on the learning trailer. Hitch angle estimator.

2. an acquisition step in which the hitch angle estimation device acquires an image of the trailer taken by a camera mounted on a vehicle towing the trailer; the hitch angle estimation device comprises an inference step of estimating the hitch angle of the trailer based on the trailer image acquired in the acquisition step by using a model obtained by performing learning using learning data, which is a data set of images of the training trailer taken by a training camera mounted on a training vehicle that tows the training trailer and labels indicating the hitch angle of the training trailer; The hitch angle of the learning trailer is determined by a difference between the orientation of the learning vehicle calculated based on a GPS signal received by a GPS receiver mounted on the learning vehicle and the orientation of the learning trailer calculated based on a GPS signal received by a GPS receiver mounted on the learning trailer. Hitch angle estimation method.

3. The processor an acquisition step of acquiring an image of the trailer taken by a camera mounted on a vehicle towing the trailer; a program for executing an inference step of estimating the hitch angle of the trailer based on the trailer image acquired in the acquisition step by using a model obtained by performing learning using learning data, which is a data set of images of the training trailer taken by a training camera mounted on a training vehicle that tows the training trailer and labels indicating the hitch angle of the training trailer, The hitch angle of the learning trailer is determined by a difference between the orientation of the learning vehicle calculated based on a GPS signal received by a GPS receiver mounted on the learning vehicle and the orientation of the learning trailer calculated based on a GPS signal received by a GPS receiver mounted on the learning trailer. program.

Images