Vanishing Point Exploration Device

Abstract

To provide a vanishing point search device capable of searching for a vanishing point again at an appropriate timing.SOLUTION: A vanishing point search device 1 comprises a loading state determination unit 11 and a vanishing point search unit 12. The loading state determination unit 11 determines whether or not a loading state of a vehicle changes based on a detection result of an in-vehicle sensor 3 that detects the loading state of the vehicle. The vanishing point search unit 12 searches for a vanishing point in a captured image based on the captured image of a camera of the vehicle. The vanishing point search unit 12 searches again for a vanishing point when the loading state changes.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a vanishing point search device.

Background Art

[0002] Conventionally, as a technical document related to a vanishing point search device, Japanese Patent Application Laid-Open No. 2012-045706 is known. Japanese Patent Application Laid-Open No. 2012-045706 discloses a technique for searching for a vanishing point on a captured image based on a captured image of a camera and using the vanishing point for calculating a physical quantity such as TOC [Time Of Contact].

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology as described above, a vanishing point once searched is continuously used during a predetermined period. However, if, for example, the orientation of the camera changes during the predetermined period, the position of the vanishing point in the captured image may deviate from the accurate position.

[0005] An object of the present disclosure is to provide a vanishing point search device capable of re-searching for a vanishing point at an appropriate timing.

Means for Solving the Problems

[0006] The vanishing point search device of the present disclosure includes a loading state determination unit that determines whether or not the loading state of the vehicle has changed based on a detection result of an in-vehicle sensor that detects the loading state of the vehicle, and a vanishing point search unit that searches for a vanishing point on a captured image based on a captured image of a camera of the vehicle. The vehicle-mounted sensor detects the number of towed vehicles connected to the vehicle, and when the number of towed vehicles changes, the loading state determination unit determines that the loading state has changed. When the loading state changes, the vanishing point search unit re-searches for the vanishing point.

Advantages of the Invention

[0007] According to the present disclosure, it is possible to provide a vanishing point search device capable of re-searching for a vanishing point at an appropriate timing.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0010] FIG. 1 is a block diagram of a vanishing point search device according to an embodiment. As shown in FIG. 1, the vanishing point search device 1 includes a camera 2, an in-vehicle sensor 3, a display 4, and an ECU [Electronic Control Unit] 10. The vanishing point search device 1 is a device for searching for a vanishing point [FOE: Focus of Expansion].

[0011] The vanishing point is a point at an infinite distance. A plurality of straight lines parallel to a predetermined direction intersect at the vanishing point when viewed from the predetermined direction. When the viewpoint (for example, a human eye) moves forward along a direction parallel to the line of sight, a plurality of objects appearing in the visual field move along a radial trajectory starting from the vanishing point.

[0012] The camera 2 is provided inside the vehicle compartment of the vehicle 20 (see FIG. 2). The camera 2 is fixed to the vehicle 20. The camera 2 images the front of the vehicle 20 through, for example, the windshield of the vehicle 20. The camera 2 is, for example, a monocular camera. The camera 2 may be, for example, a stereo camera. The camera 2 transmits the captured image to the ECU 10.

[0013] The in-vehicle sensor 3 detects the loading state of the vehicle 20. The loading state of the vehicle 20 is an index corresponding to the weight of the load or passengers carried by the vehicle 20. In the present embodiment, the loading state of the vehicle 20 is the number of towed vehicles 30 (see FIG. 2(b)) connected to the vehicle 20. That is, the in-vehicle sensor 3 detects the number of towed vehicles 30 connected to the vehicle 20. The in-vehicle sensor 3 is, for example, a TDM [Trailer detection module] or the like. The in-vehicle sensor 3 transmits the loading state of the vehicle 20 to the ECU 10.

[0014] The display 4 is provided, for example, on the instrument panel of the vehicle 20. The display 4 has a display screen for displaying an image. The display 4 displays an image on the display screen based on a control signal from the ECU 10. The display 4 displays, for example, the captured image of the camera 2. Note that the display 4 may be an HUD [Head Up Display] or the like that performs projection display on the windshield or a projection screen of the vehicle 20.

[0015] The ECU 10 is an electronic control unit having a CPU [Central Processing Unit] and a storage unit. The storage unit is, for example, a ROM [Read Only Memory] or a RAM [Random Access Memory] or the like. In the ECU 10, for example, various functions are realized by the program stored in the storage unit being executed by the CPU.

[0016] FIG. 2 is a diagram showing the attitude change of the vehicle 20 when the loaded state changes. FIG. 2(a) shows the case where the loaded state of the vehicle 20 is in the first state. FIG. 2(b) shows the case where the loaded state of the vehicle 20 is in the second state. The number of towed vehicles 30 in the second state is larger than the number of towed vehicles 30 in the first state. In the present embodiment, the number of towed vehicles 30 in the first state is "0", and the number of towed vehicles 30 in the second state is "1".

[0017] As shown in FIG. 2(a), in the case of the first state, the optical axis 21 of the camera 2 is substantially parallel to the traveling direction (for example, the horizontal direction) D of the vehicle 20. That is, in the case of the first state, the camera 2 faces the front of the vehicle 20 in the horizontal direction. As shown in FIG. 2(b), in the case of the second state, since the towed vehicle 30 is connected to the rear of the vehicle 20, compared with the first state, the front end of the vehicle 20 is displaced upward in the vertical direction and the rear end of the vehicle 20 is displaced downward in the vertical direction. That is, the attitude of the vehicle 20 in the first state and the attitude of the vehicle 20 in the second state are different from each other.

[0018] Since the camera 2 is fixed to the vehicle 20, when the attitude of the vehicle 20 changes, the orientation of the camera 2 also changes. In the case of the second state, the optical axis 21 of the camera 2 intersects the traveling direction D of the vehicle 20. That is, in the case of the second state, the camera 2 faces upward in the vertical direction with respect to the horizontal direction. Thus, even when the traveling direction D of the vehicle 20 does not change, the orientation of the camera 2 may change. When the orientation of the camera 2 changes, the position of the vanishing point in the captured image of the camera 2 also changes. Therefore, for example, if the vanishing point in the first state continues to be used even in the second state, it may be difficult to accurately calculate the physical quantity in the second state.

[0019] The ECU 10 includes, as functional components, a loading state determination unit 11, a vanishing point search unit 12, a vanishing point storage unit 13, and a physical quantity calculation unit 14.

[0020] The loading state determination unit 11 determines the loading state of the vehicle 20 based on the detection results of the in-vehicle sensor 3. The loading state determination unit 11 determines whether a towed vehicle 30 is connected to the vehicle 20. The loading state determination unit 11 determines the number of towed vehicles 30 connected to the vehicle 20.

[0021] The loading state determination unit 11 determines whether the loading state of the vehicle 20 has changed based on the detection results of the in-vehicle sensor 3. The loading state determination unit 11 determines whether the number of towed vehicles 30 connected to the vehicle 20 has increased. The loading state determination unit 11 determines whether the number of towed vehicles 30 connected to the vehicle 20 has decreased. When the number of towed vehicles 30 connected to the vehicle 20 has changed, the loading state determination unit 11 determines that the loading state of the vehicle 20 has changed.

[0022] The vanishing point search unit 12 searches for the vanishing point on the captured image based on the captured image of the camera 2 of the vehicle 20. When the loading state of the vehicle 20 changes, the vanishing point search unit 12 re-searches for the vanishing point. Each time the loading state of the vehicle 20 changes, the vanishing point search unit 12 searches for the vanishing point. The vanishing point search unit 12 searches for the vanishing point by various known methods.

[0023] FIG. 3(a) shows the captured image of the camera 2 in the first state. As shown in FIG. 3(a), in the captured image 22, for example, a pair of lane boundary lines 40 parallel to each other exist. The vanishing point search unit 12 extracts the pair of lane boundary lines 40 and searches for the intersection of the extension lines of the pair of lane boundary lines 40 as the vanishing point 51 in the first state.

[0024] The horizontal coordinate X1 of the vanishing point 51 in the captured image 22 is, for example, 100 pixels. The vertical coordinate Y1 of the vanishing point 51 in the captured image 22 is, for example, 100 pixels. That is, the vanishing point 51 is 100 pixels away from the left end of the captured image 22 and 100 pixels away from the lower end of the captured image 22.

[0025] FIG. 3(b) shows the captured image of the camera 2 in the second state. The vanishing point search unit 12 extracts a pair of lane boundary lines 40 and searches for the intersection of the extension lines of the pair of lane boundary lines 40 as the vanishing point 52 in the second state. As described above, in the case of the first state, the camera 2 faces forward of the vehicle 20 in the horizontal direction, while in the case of the second state, the camera 2 faces upward in the vertical direction with respect to the horizontal direction. Therefore, in the captured image 22, the vanishing point 52 in the second state is located below the vanishing point 51 in the first state.

[0026] The horizontal coordinate X2 of the vanishing point 52 in the captured image 22 is the same as the horizontal coordinate X1 of the vanishing point 51 in the captured image 22. The vertical coordinate Y2 of the vanishing point 52 in the captured image 22 is smaller than the vertical coordinate Y1 of the vanishing point 51 in the captured image 22. The vertical coordinate Y2 of the vanishing point 52 in the captured image 22 is, for example, 80 pixels. That is, the vanishing point 52 is 100 pixels away from the left end of the captured image 22 and 80 pixels away from the lower end of the captured image 22.

[0027] The vanishing point storage unit 13 stores the vanishing points 51 and 52 searched by the vanishing point search unit 12. The vanishing point storage unit 13 stores the vanishing point 51 when the loading state of the vehicle 20 is in the first state as the first vanishing point, and stores the vanishing point 52 when the loading state of the vehicle 20 is in the second state as the second vanishing point. The vanishing point storage unit 13 stores the vanishing point for each loading state of the vehicle 20.

[0028] The vanishing point search unit 12 searches for the vanishing point within a predetermined search area in the captured image 22 starting from the initial position. FIG. 4 is a diagram showing the search area for searching the vanishing point. As shown in FIG. 4, in the case of the first state, the vanishing point search unit 12 searches for the vanishing point 51 within the search area 61 starting from the initial position. The initial position is the position of the vanishing point immediately before the change in the loading state. When calculating the first vanishing point after the vehicle 20 starts, the vanishing point search unit 12 may search for the vanishing point starting from the initially stored initial position.

[0029] When the loading state of the vehicle 20 changes, the vanishing point exploration unit 12 re-explores a new vanishing point after the change in the loading state, with the position of the vanishing point immediately before the change in the loading state as the initial position. When the vehicle 20 changes from the first state to the second state, the vanishing point exploration unit 12 re-explores the vanishing point 52 within the exploration region 62 with the position of the vanishing point 51 (the first vanishing point) as the initial position. The exploration region 62 is larger than the exploration region 61. That is, when the loading state of the vehicle 20 changes, the vanishing point exploration unit 12 expands the exploration region and then re-explores a new vanishing point.

[0030] When the vanishing point 51 is stored by the vanishing point storage unit 13, if the loading state of the vehicle 20 changes from a state other than the first state to the first state, the vanishing point exploration unit 12 re-explores the vanishing point with the position of the vanishing point 51 as the initial position. For example, when the loading state of the vehicle 20 changes from the first state to the second state and then returns to the first state, the vanishing point exploration unit 12 uses the position of the vanishing point 51 (the first vanishing point) instead of the vanishing point 52 (the second vanishing point) as the initial position to re-explore the vanishing point when the loading state of the vehicle 20 returns to the first state.

[0031] FIG. 5 is a diagram showing the change in the vanishing point when the loading state of the vehicle 20 changes from the first state to the second state and then returns to the first state. As shown in FIG. 5, the position of the vanishing point 53 when the loading state of the vehicle 20 changes from the first state to the second state and then returns to the first state may be different from the position of the vanishing point 51 in the first state before changing to the second state. This is considered to be caused by, for example, a decrease in the fuel of the vehicle 20 or a change in the loading weight of the towed vehicle 30 while the loading state of the vehicle 20 changes from the first state to the second state and then returns to the first state.

[0032] The horizontal coordinate X3 of the vanishing point 53 in the captured image 22 is the same as the horizontal coordinate X1 of the vanishing point 51 in the captured image 22. The vertical coordinate Y3 of the vanishing point 53 in the captured image 22 is larger than the vertical coordinate Y1 of the vanishing point 51 in the captured image 22. The vertical coordinate Y3 of the vanishing point 53 in the captured image 22 is, for example, 120 pixels. That is, the vanishing point 53 is 100 pixels away from the left end of the captured image 22 and 120 pixels away from the lower end of the captured image 22.

[0033] Although the position of the vanishing point 53 is different from the position of the vanishing point 51, since the vanishing point 53 and the vanishing point 51 are vanishing points in the same loading state (the first state), the position of the vanishing point 53 tends to be closer to the vanishing point 51 than the vanishing point 52 in the second state. Therefore, when searching for the vanishing point 53 with the position of the vanishing point 51 as the initial position, the learning efficiency of the vanishing point is improved compared to when searching for the vanishing point 53 with the position of the vanishing point 52 as the initial position.

[0034] The physical quantity calculation unit 14 calculates each physical quantity based on the latest vanishing point searched by the vanishing point search unit 12. The physical quantity calculation unit 14 calculates the distance between the obstacle and the vehicle 20, for example, based on the distance between the obstacle and the vanishing point in the captured image 22.

[0035] Next, the processing of the ECU 10 will be described. FIG. 6 is a flowchart showing the processing by the ECU 10. As shown in FIG. 6, in step S1, the ECU 10 searches for the vanishing point 51 in the first state.

[0036] In step S2, the ECU 10 determines whether the loading state of the vehicle 20 has changed from the first state to the second state. That is, in step S2, the ECU 10 determines whether the number of towed vehicles 30 connected to the vehicle 20 has increased to the number in the second state. If the loading state of the vehicle 20 has changed from the first state to the second state (step S2: YES), the ECU 10 proceeds to step S3. If the loading state of the vehicle 20 has not changed from the first state to the second state (step S2: NO), the ECU 10 ends the current processing.

[0037] In step S3, the ECU 10 stores the vanishing point 51 in the first state as the first vanishing point. In step S4, the ECU 10 expands the search area. In step S5, the ECU 10 searches for the vanishing point 52 in the second state. In step S5, with the position of the vanishing point 51 in the first state as the initial position, the ECU 10 searches for the vanishing point 52 in the second state within the search area 62 expanded in step S4.

[0038] In step S6, the ECU 10 determines whether the loading state of the vehicle 20 has changed from the second state to the first state. That is, in step S6, the ECU 10 determines whether the number of towed vehicles 30 connected to the vehicle 20 has decreased to the number in the first state. If the loading state of the vehicle 20 has changed from the second state to the first state (step S6: YES), the ECU 10 proceeds to step S7. If the loading state of the vehicle 20 has not changed from the second state to the first state (step S6: NO), the ECU 10 ends the current process.

[0039] In step S7, the ECU 10 sets, not the vanishing point 52 (second vanishing point) in the second state, but the position of the vanishing point 51 (first vanishing point) in the first state stored in step S3 as the initial position. In step S8, starting from the initial position set in step S7, the ECU 10 searches for a new vanishing point 53 in the first state.

[0040] As described above, in the vanishing point search device 1, when the loading state of the vehicle 20 changes, the vanishing point search unit 12 re-searches for the vanishing point. Thus, even if the orientation of the camera 2 changes due to a change in the loading state of the vehicle 20, a new vanishing point can be re-searched at an appropriate timing. That is, since the vanishing point search unit 12 searches for the vanishing point triggered by a change in the loading state of the vehicle 20, it can re-search for a new vanishing point without waiting until the normal search start conditions are met. Therefore, it is suppressed that a vanishing point deviated from the accurate position is used for the calculation of physical quantities. Thus, according to the vanishing point search device 1, high-precision calculation of physical quantities becomes possible.

[0041] The vehicle-mounted sensor 3 detects the number of towed vehicles 30 connected to the vehicle 20. When the number of towed vehicles 30 changes, the loading state determination unit 11 determines that the loading state of the vehicle 20 has changed. When the number of towed vehicles 30 changes, the attitude change of the vehicle 20 tends to be relatively large. When the attitude change of the vehicle 20 is large, the change in the orientation of the camera 2 of the vehicle 20 also tends to be relatively large. In such a case, since the vanishing point may deviate relatively greatly from the accurate position, the above-described effects become particularly prominent.

[0042] The vanishing point storage unit 13 stores the vanishing point 51 when the loading state of the vehicle 20 is in the first state as the first vanishing point. When the first vanishing point is stored in the vanishing point storage unit 13, the vanishing point search unit 12 re-searches for the vanishing point 52 with the position of the vanishing point 51 as the initial position when the loading state of the vehicle 20 changes from a state other than the first state to the first state. As a result, when the loading state of the vehicle 20 returns from a state other than the first state to the first state, in order to re-search for the new vanishing point 53 in the first state with the position of the vanishing point 51 (the first vanishing point) as the initial position, for example, compared with the case of re-searching for the new vanishing point 53 in the first state with the position of the vanishing point 52 (the second vanishing point) as the initial position, the new vanishing point 53 in the first state can be searched for more efficiently.

[0043] As described above, the embodiments of the present disclosure have been described. However, the present disclosure is not limited to the above-described embodiments. The present disclosure can be implemented in various forms with various changes and improvements based on the knowledge of those skilled in the art, including the above-described embodiments.

[0044] Although an example in which the camera 2 of the vehicle 20 images the front of the vehicle 20 has been shown, the camera 2 may image the rear of the vehicle 20 through the rear glass of the vehicle 20.

[0045] An example where the number of towed vehicles 30 in the first state is "0" and the number of towed vehicles 30 in the second state is "1" has been shown. However, the number of towed vehicles 30 in the first state may be "1" or more, and the number of towed vehicles 30 in the second state may be "2" or more. It is only necessary that the number of towed vehicles 30 in the first state and the number of towed vehicles 30 in the second state are different. The number of towed vehicles 30 in the second state may be less than the number of towed vehicles 30 in the first state.

[0046] Although an example of the number of towed vehicles 30 has been shown as the loading state of the vehicle 20, the loading state of the vehicle 20 may be an index corresponding to the weight of the passengers. In this case, the in-vehicle sensor 3 may detect, for example, the number of seat belts worn in the vehicle 20. The loading state determination unit 11 may determine that the loading state of the vehicle 20 has changed when the number of seat belts worn changes. The in-vehicle sensor 3 may be a load sensor provided on the seat or a load sensor provided on the loading platform.

[0047] The ECU 10 may not include the physical quantity calculation unit 14. That is, the vanishing points 51, 52, 53 searched by the vanishing point search unit 12 may not be used for the calculation of physical quantities. The vehicle 20 may be an autonomous vehicle.

Explanation of Signs

[0048] 1... Vanishing point search device, 2... Camera, 3... In-vehicle sensor, 11... Loading state determination unit, 12... Vanishing point search unit, 13... Vanishing point storage unit, 20... Vehicle, 22... Captured image, 30... Towed vehicle, 51, 52, 53 Vanishing points

Claims

1. A loading state determination unit that determines whether or not the loading state of the vehicle has changed based on a detection result of an in-vehicle sensor that detects the loading state of the vehicle; A vanishing point search unit that searches for a vanishing point on the captured image based on a captured image of a camera of the vehicle, the vanishing point search device comprising: The in-vehicle sensor detects the number of towed vehicles connected to the vehicle; The loading state determination unit determines that the loading state has changed when the number of towed vehicles changes; The vanishing point search unit re-searches for the vanishing point when the loading state changes, the vanishing point search device.

2. The vanishing point search device according to claim 1, further comprising a vanishing point storage unit that stores the vanishing point when the loading state is in a first state as a first vanishing point; The vanishing point search unit re-searches for the vanishing point with the position of the first vanishing point as an initial position when the loading state changes from a state other than the first state to the first state when the first vanishing point is stored by the vanishing point storage unit.

Images