Information processor, information processing method, program, and recording medium

Abstract

To provide an information processor capable of predicting an operation of the following vehicle from behavior of the following vehicle viewed through a rear window even if the rear window is small, using a camera provided in a vehicle cabin; and to provide an information processing method, a program, and a recording medium therefor.SOLUTION: There are provided a captured image acquisition part for acquiring a captured image on which a rear window of a vehicle cabin is included, that is captured by an imaging part provided in the vehicle cabin of one's own vehicle; a specification part for specifying a following vehicle area as an area where the following vehicle captured through the rear window appears on the captured image; and a detection part for detecting specific behavior of the following vehicle according to movement of an outer-periphery of the following vehicle area, with respect to one reference line extending in a vertical direction, that is set at a predetermined position inside an area where the rear window is present on the captured image.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, a program, and a recording medium. [Background technology]

[0002] A control device is disclosed that warns the driver of the vehicle when another vehicle approaches the vehicle. For example, Patent Document 1 discloses a vehicle control device that recognizes another vehicle traveling behind the vehicle from an image captured by a camera installed in a rear portion of the exterior of the vehicle, such as a back door, and issues a warning to the driver of the vehicle according to the movement of the recognized other vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-151314 A Summary of the Invention [Problem to be solved by the invention]

[0004] For example, in Patent Document 1, when a camera is installed in the front of the vehicle, such as around the rear-view mirror, and while capturing images of the interior of the vehicle, a vehicle traveling behind the vehicle (hereinafter also referred to as a rear vehicle) is captured, the rear vehicle is only visible in the rear window of the camera image, making it difficult to predict the behavior of the rear vehicle. In particular, in the case of a vehicle with a small rear window, the range in which the rear vehicle can be viewed is narrow, making it difficult to predict the behavior of the rear vehicle, such as overtaking.

[0005] The present invention has been made in consideration of the above-mentioned points, and aims to provide an information processing device, an information processing method, a program, and a recording medium that are capable of predicting the operation of a rear vehicle from the behavior of the rear vehicle seen through the rear window using a camera installed inside the vehicle cabin, even if the rear window is small. [Means for solving the problem]

[0006] The information processing device described in claim 1 is characterized by having an image acquisition unit that acquires an image including the rear window of the vehicle cabin captured by an imaging unit provided in the vehicle cabin of the vehicle itself, an identification unit that identifies a rear vehicle area, which is an area in the captured image where a rear vehicle is present as seen through the rear window, and a detection unit that detects specific behavior of the rear vehicle according to movement of the outer edge of the rear vehicle area relative to a reference line that is set at a predetermined position in the area in the captured image where the rear window is present and extends vertically.

[0007] The information processing method described in claim 12 is characterized by having an image acquisition step of acquiring an image including the rear window of the vehicle cabin captured by an imaging unit provided in the vehicle cabin of the vehicle itself; an identification step of identifying a rear vehicle area, which is an area in the captured image in which a rear vehicle is present as seen through the rear window; and a detection step of detecting a specific behavior of the rear vehicle in accordance with movement of the outer edge of the rear vehicle area relative to a reference line that is set at a predetermined position in the area in which the rear window is present in the captured image and extends vertically.

[0008] The program described in claim 13 is a program to be executed by a computer, and is characterized in that it includes an image acquisition step of acquiring an image including the rear window of the vehicle cabin captured by an imaging unit provided in the vehicle cabin of the host vehicle; an identification step of identifying a rear vehicle area, which is an area in the captured image in which a rear vehicle is present as seen through the rear window; and a detection step of detecting a specific behavior of the rear vehicle in accordance with movement of the outer edge of the rear vehicle area relative to a reference line that is set at a predetermined position in the area in which the rear window is present in the captured image and extends vertically.

[0009] The recording medium described in claim 14 is a recording medium having a program stored therein to be executed by a computer, and is characterized in having an image acquisition step of acquiring an image including the rear window of the vehicle cabin captured by an imaging unit provided in the vehicle cabin of the vehicle itself, an identification step of identifying a rear vehicle area, which is an area in the captured image in which a rear vehicle is present as seen through the rear window, and a detection step of detecting a specific behavior of the rear vehicle in accordance with movement of the outer edge of the rear vehicle area relative to a reference line that is set at a predetermined position in the area in which the rear window is present in the captured image and extends vertically. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing a configuration of a front seat portion of a vehicle according to a first embodiment and a view in front of the vehicle; [Diagram 2] FIG. 2 is a diagram showing an example of a captured image captured by an in-vehicle camera in the first embodiment. [Diagram 3] 1 is a block diagram illustrating an example of a configuration of an information processing device according to a first embodiment. [Figure 4] 4 is a diagram showing an image when another vehicle is recognized by the information processing device in the first embodiment. FIG. [Diagram 5] 4 is an enlarged view of an image when another vehicle is recognized by the information processing device in the first embodiment. FIG. [Figure 6] 4 is a diagram illustrating an example of detecting the behavior of another vehicle by the information processing device according to the first embodiment; FIG. [Figure 7] 4 is a table showing an example of the behavior of another vehicle determined by the information processing device according to the first embodiment. [Figure 8] 4 is a flowchart showing a control routine of the information processing device according to the first embodiment. [Figure 9] FIG. 11 is a top view of a host vehicle and a rear vehicle traveling on a road in a second embodiment. [Figure 10] FIG. 11 is a block diagram illustrating an example of a configuration of an information processing device according to a second embodiment. [Figure 11]10 is a flowchart showing a control routine of an information processing device according to a second embodiment. [Figure 12] 13 is a flowchart showing a control routine of an information processing device according to a modified example of the second embodiment. [Figure 13] FIG. 11 is a block diagram illustrating an example of a configuration of an information processing device according to a third embodiment. [Figure 14] 11 is a flowchart showing a control routine of an information processing device according to a third embodiment. [Figure 15] FIG. 13 is a block diagram showing an example of a configuration of an information processing device according to a modified example of the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the drawings, the same components are designated by the same reference numerals, and the description of the same components will be omitted. EXAMPLES

[0012] Fig. 1 is a diagram showing a configuration of a front seat portion of a vehicle M as a host vehicle according to the first embodiment and a view ahead of the vehicle M. In this embodiment, the vehicle M is described as traveling in the center lane of a three-lane road with good visibility such as an expressway as shown in Fig. 1. In addition, the description is based on the premise that the vehicle drives on the left side of the road.

[0013] The in-vehicle device 10 as an information processing device is connected to a touch panel display 12, a speaker 13, and an in-vehicle camera 14 installed in the cabin of the vehicle M, and includes a control unit that controls these. The in-vehicle device 10 is disposed in the center of a dashboard DB in the front seat of the vehicle M.

[0014] The touch panel display 12 is a display device that combines a display that displays a screen based on the control of the in-vehicle device 10 and a touch panel that accepts input operations from an occupant of the vehicle M. The touch panel display 12 is provided in the center portion on the outer side of the dashboard DB.

[0015] The speaker 13 is an audio output device that outputs audio under the control of the in-vehicle device 10. The speaker 13 is provided on each of two A-pillars AP that are provided on either side of the front window FW.

[0016] The in-vehicle camera 14 as an imaging unit is an imaging device that captures images of the interior of the vehicle M and the expressions of the occupants of the vehicle M. The in-vehicle camera 14 is provided on the ceiling of the vehicle M adjacent to the rearview mirror RM.

[0017] Here, an image captured by the in-vehicle camera 14 and processing performed by the in-vehicle device 10 will be described with reference to Fig. 2. Fig. 2 is a diagram showing a captured image CI as an example of an image captured by the in-vehicle camera 14. The in-vehicle camera 14 captures an image of the interior of the vehicle M including the rear window RW as shown in Fig. 2. The captured image CI shown in Fig. 2 was captured on a straight portion of a road.

[0018] The in-vehicle device 10 recognizes a vehicle FV (hereinafter referred to as the rear vehicle FV) traveling behind the vehicle M from an image CI captured by the in-vehicle camera 14 through the rear window RW, and obtains a superimposed image in which a rectangular bounding box surrounding the recognized rear vehicle FV is superimposed on the captured image (details will be described later).

[0019] In addition, the in-vehicle device 10 detects overtaking behavior, which is the behavior of the rear vehicle FV attempting to overtake the host vehicle M, in accordance with the movement of the outer edge of the bounding box relative to a reference line extending vertically and placed on the above-mentioned superimposed image.

[0020] For example, when the in-vehicle device 10 detects an overtaking behavior of the rear vehicle FV, the in-vehicle device 10 notifies the driver of the vehicle M via the speaker 13 that the rear vehicle FV is about to overtake the vehicle M. This enables the driver of the vehicle M to drive while paying attention to the behavior of the rear vehicle FV while predicting the overtaking by the rear vehicle FV.

[0021] Next, the configuration of the in-vehicle device 10 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of a hardware configuration of the in-vehicle device 10 according to the first embodiment. The in-vehicle device 10 is a device in which a control unit 17, an input unit 19, an output unit 21, and a large-capacity storage device 22 cooperate with each other via a system bus 16.

[0022] The control unit 17 is configured with a CPU (Central Processing Unit) 17A, a ROM (Read Only Memory) 17B, a RAM (Random Access Memory) 17C, etc., and functions as a computer. The CPU 17A reads out and executes various programs stored in the ROM 17B and the large-capacity storage device 22 to realize various functions.

[0023] The above-mentioned various programs may be acquired from another server device or the like via a network by communication of a communication unit (not shown) connected to the in-vehicle device 10, or may be recorded on a recording medium and read via various drive devices. That is, the various programs (including a program for executing the processes in the in-vehicle device 10 described later) can be transmitted via a network, and can be transferred by recording on a computer-readable recording medium.

[0024] The input unit 19 is an interface unit that communicatively connects the control unit 17 of the in-vehicle device 10 with the touch panel display 12 and the in-vehicle camera 14. The control unit 17, for example, sequentially acquires captured images CI (see FIG. 2) captured by the in-vehicle camera 14 via the input unit 19. In other words, the control unit 17 functions as an image acquisition unit that acquires the captured images CI.

[0025] The output unit 21 is an interface unit that communicatively connects the control unit 17 of the in-vehicle device 10 with the touch panel display 12 and the speaker 13. The control unit 17 transmits a video or image signal to the touch panel display 12 via the output unit 21 to display the video or image. The control unit 17 also transmits an audio signal to the speaker 13 via the output unit 21 to output sound.

[0026] The large-capacity storage device 22 is a storage device that stores and manages data necessary for processing by the control unit 17. The large-capacity storage device 22 is configured, for example, with a hard disk device, an SSD (solid state drive), a flash memory, etc., and stores various programs such as an operating system and software for the in-vehicle device 10.

[0027] The recognition model 22A, which is included in the software for the in-vehicle device 10 and is used by the control unit 17, is an AI model that can recognize a rear vehicle FV in a captured image CI captured by the in-vehicle camera 14. The recognition model 22A has previously learned about the features of an image to be recognized as a rear vehicle FV of the vehicle M from the captured image CI.

[0028] That is, the recognition model 22A is trained in advance to be able to recognize the rear vehicle FV based on the characteristics of the vehicle, such as the shape and size, when the rear vehicle is viewed through the rear window from a camera installed, for example, at the front of the vehicle interior of the vehicle. Therefore, the recognition model 22A does not recognize a vehicle that is traveling far away from the vehicle M, i.e., a vehicle that appears small in size, as the rear vehicle FV, even if the vehicle is traveling behind the vehicle M, and can recognize only a vehicle traveling closer to the vehicle M than a predetermined standard among the vehicles traveling behind the vehicle M as the rear vehicle FV.

[0029] The control unit 17 inputs the captured image CI to the recognition model 22A, and obtains an image in which the area in which the rear vehicle FV, which is captured in the captured image CI with a size equal to or larger than a predetermined size, exists is surrounded by a rectangular bounding box as a recognition result of the rear vehicle FV by the recognition model 22A. In other words, the control unit 17 functions as an identification unit that identifies the rear vehicle area, which is the area in which the rear vehicle FV exists in the captured image CI.

[0030] Fig. 4 is a diagram showing an example of an image received from the recognition model 22A by the control unit 17. As shown in Fig. 4, the control unit 17 acquires from the recognition model 22A a superimposed image SI in which a rectangular bounding box BB surrounding the rear vehicle FV is superimposed on the captured image CI.

[0031] As shown in FIG. 4, the control unit 17 places a reference line BL extending in the vertical direction at a position on the superimposed image SI that is set in advance by the user of the in-car device 10 via the touch panel display 12 or the like.

[0032] Fig. 5 is an enlarged view of part A in Fig. 4. The position of the reference line BL is set to pass directly behind the vehicle M, that is, the midpoint of the vehicle M in the vehicle width direction. The user of the in-vehicle device 10 can adjust the position of the reference line BL in the left-right direction by, for example, performing an operation to specify the position on the touch panel display 12. In the case of Fig. 5, the reference line BL is positioned to pass through the midpoint MP between the road boundary line L1 and the road boundary line L2, which are the boundary lines of the lane in which the vehicle M is traveling.

[0033] When the position of the reference line BL is set in this manner, when the rear vehicle FV is traveling directly behind the vehicle M, the reference line BL will be sandwiched between the right side RS and the left side LS of the bounding box BB that extend vertically, as shown in Figure 5.

[0034] [Detection of overtaking behavior of rear vehicles] 6 and 7, the detection of the overtaking behavior of the rear vehicle FV performed by the control unit 17 of the in-vehicle device 10 will be described below. Note that in the following description, it is assumed that the vehicle M and the rear vehicle FV are traveling in the same lane.

[0035] Fig. 6 is an enlarged view of part A in Fig. 4, similar to Fig. 5. Fig. 6 shows an example of the movement of a bounding box BB surrounding a following vehicle FV relative to a reference line BL.

[0036] When one of the two vertical sides (right side RS and left side LS) of the bounding box BB surrounding the rear vehicle FV in the superimposed image SI crosses the reference line BL, the control unit 17 determines whether the rear vehicle FV has performed an overtaking behavior based on the direction of the motion vector of a point on the crossed side.

[0037] For example, as shown in FIG. 6, when the following vehicle FV moves from the position of bounding box BB1 (dashed line in the figure) to the position of bounding box BB2 (solid line in the figure), the right side RS of the bounding box BB1 crosses the reference line BL.

[0038] When the right side RS of the bounding box BB1 crosses the reference line BL, the control unit 17 focuses on the midpoint MR of the right side RS, and determines that the rear vehicle FV has performed an overtaking behavior if the movement vector Ve of the midpoint MR before and after crossing the reference line BL is pointing in the overtaking direction, which is a direction that indicates overtaking behavior.

[0039] Fig. 7 is a behavior determination table TB1 (hereinafter, simply referred to as table TB1) which is a table showing an example of a determination of an overtaking behavior for each movement of a bounding box BB with respect to a reference line BL. Specifically, using the table TB1 shown in Fig. 7, the control unit 17 determines that the behavior performed by the following vehicle FV is an overtaking behavior when, for example, the vertical side of the bounding box BB that straddles the reference line BL is the right side and the motion vector of the midpoint of the right side faces leftward.

[0040] Therefore, in FIG. 6, the control unit 17 determines that the following vehicle FV has performed an overtaking behavior because the motion vector Ve of the midpoint MR of the right side RS faces leftward before and after crossing the reference line BL.

[0041] In addition, using table TB1, the control unit 17 determines that the behavior performed by the rear vehicle FV is an overtaking behavior, for example, when the side of the bounding box BB that straddles the reference line BL is the left side and the motion vector of the midpoint of the left side is pointing in the right direction.

[0042] The control unit 17 determines that the behavior of the rear vehicle FV is not an overtaking behavior when the motion vector of the midpoint of the right side RS faces to the right or the motion vector of the midpoint of the left side LS faces to the left other than the above. Such a behavior of the rear vehicle FV is detected, for example, when the rear vehicle FV changes lanes from a passing lane to the driving lane in which the vehicle M is present.

[0043] In this way, the control unit 17 of the in-vehicle device 10 of this embodiment can detect the overtaking behavior of the rear vehicle FV based on the direction of the motion vector of the midpoint of one of the two vertical sides of the bounding box BB surrounding the rear vehicle FV when the side crosses the reference line BL, which is a trigger. In other words, the control unit 17 functions as a detection unit that detects the overtaking behavior of the rear vehicle FV.

[0044] When the control unit 17 detects an overtaking behavior by the rear vehicle FV, it notifies the driver of the vehicle M that the rear vehicle FV is about to overtake the vehicle M, for example, by playing a voice message such as "A vehicle is approaching. Please be careful" through the speaker 13 of the vehicle M. In other words, the control unit 17 functions as a notification unit that notifies the occupants of the vehicle M that the rear vehicle FV is about to overtake the vehicle M.

[0045] The specific operation of the in-vehicle device 10 in this embodiment will be described below with reference to Fig. 8. In the following description, it is assumed that the vehicle M and the rear vehicle FV are traveling in the same lane, and that the right side RS and left side LS of the bounding box BB in which the rear vehicle FV is recognized are positioned on either side of the reference line BL in the superimposed image SI.

[0046] 8 is a flowchart showing a behavior detection routine RT1 executed by the control unit 17 of the in-vehicle device 10. The control unit 17 starts the behavior detection routine RT1, for example, when the in-vehicle device 10 is powered on, and repeatedly executes the behavior detection routine RT1 while the power is on.

[0047] First, the control unit 17 determines whether or not one of the two vertical sides of the bounding box BB on the superimposed image SI crosses the reference line BL (step S101). When the control unit 17 determines that the vertical side does not cross the reference line BL (step S101: NO), the control unit 17 ends the behavior detection routine RT1.

[0048] When the control unit 17 determines that one of the vertical directions of the bounding box BB crosses the reference line BL (step S101: YES), it determines whether the direction of the motion vector before and after the midpoint of the edge that crosses the reference line BL is facing the overtaking direction, which is a direction indicating overtaking behavior (step S102).

[0049] If the direction of the motion vector of the midpoint of the side that crosses the reference line BL is pointing in the overtaking direction (step S102: YES), the control unit 17 determines that the rear vehicle FV has performed an overtaking behavior and notifies the occupants of the vehicle that the rear vehicle FV is attempting to overtake the vehicle (step S103).

[0050] Specifically, using table TB1 in Figure 7, the control unit 17 determines that the rear vehicle FV has performed an overtaking behavior when the midpoint of the right side RS of the bounding box BB is facing left or when the midpoint of the left side LS is facing right.

[0051] After step S103 is completed, or if the direction of the motion vector of the midpoint of the side straddling the reference line BL is not in the overtaking direction (step S102: NO), the control unit 17 ends the behavior detection routine RT1.

[0052] According to the in-vehicle device 10 of this embodiment, the above-mentioned behavior detection routine RT1 can detect the overtaking behavior of the rear vehicle FV attempting to overtake the host vehicle, that is, the vehicle M. As a result, when the driver of the vehicle M hears the above-mentioned attention-calling voice, the driver can predict that the rear vehicle FV will soon overtake the host vehicle, and can keep in mind safe driving, such as by not forcing the vehicle to overtake even if there is a slow vehicle ahead.

[0053] Furthermore, for example, when the in-vehicle camera 14 is installed in a vehicle with a small rear window, or when the in-vehicle camera 14 is installed at the front of the vehicle interior at a distance from the rear window, the rear area that can be observed through the rear window in the image CI captured by the in-vehicle camera 14 may be narrow; however, according to the in-vehicle device 10 of this embodiment, even in such cases, the overtaking behavior of the rear vehicle FV can be suitably detected.

[0054] In other words, most vehicles have at least the area directly behind the vehicle visible so that the driver can check for safety behind the vehicle through the rear window. Therefore, if, as in this embodiment, the reference line BL for determining the overtaking behavior of the rear vehicle FV is set directly behind vehicle M, the overtaking behavior of the rear vehicle FV can be detected even if the rear window in the captured image CI is small.

[0055] Therefore, according to the in-vehicle device 10 of this embodiment, by using the in-vehicle camera 14 installed at the front of the vehicle cabin, it is possible to predict the movement of the rear vehicle FV from the behavior of the rear vehicle FV recognized through the rear window RW, even if the rear window RW is small.

[0056] In this embodiment, the in-vehicle camera 14 is provided on the ceiling of the vehicle M adjacent to the rearview mirror RM, but this is not limited thereto, and the in-vehicle camera 14 may be provided, for example, on the dashboard DB or directly on the front window FW.

[0057] In this embodiment, the reference line BL is placed at a position previously set by the user of the in-vehicle device 10, but this is not limited to the above, and the placement position may be determined automatically, for example, when the control unit 17 acquires the captured image CI from the in-vehicle camera 14.

[0058] For example, when the vehicle M is located on a straight road, the control unit 17 may perform image recognition of white lines indicating lanes in the captured image CI, identify a horizontal midpoint in the image between the white lines, and place the reference line BL so as to pass through the midpoint. Note that the midpoint is not the only thing to be identified, and the vanishing point of the lane in the captured image CI may be identified.

[0059] By automatically determining the position of the reference line BL in this manner, it is possible to eliminate the trouble of setting the position of the reference line BL by the user of the in-vehicle device 10. Note that the position setting of the reference line BL may be switchable so that the position is automatically determined when there are passengers only in the driver's seat and the passenger seat, and the position can be manually set by the user of the in-vehicle device 10 when there are passengers in the rear seats as well.

[0060] In this embodiment, the reference line BL is placed on the superimposed image SI, but this is not limited to the above, and the reference line BL may be placed at a position on the captured image CI that is set in advance at the time the captured image CI is acquired.

[0061] In addition, in this embodiment, the control unit 17 detects the overtaking behavior of the rear vehicle FV based on the direction of the movement vector of a point on one of the vertical sides of the bounding box BB at the time of crossing the reference line BL as a trigger, but the sides that are subject to the determination of crossing the reference line BL are not limited to the vertical sides of the bounding box BB.

[0062] For example, the control unit 17 may set a right determination line RS1, which is to be used to determine whether the line crosses the reference line BL, instead of the right side RS of the bounding box BB, by offsetting the line RS parallel to the right side RS within a predetermined range based on the position of the right side RS. Also, the control unit 17 may set a left determination line LS1, which is to be used to determine whether the line crosses the reference line BL, instead of the left side LS of the bounding box BB, by offsetting the line LS parallel to the left side LS within a predetermined range based on the position of the left side LS.

[0063] In this case, the more the setting position of the right judgment line RS1 is offset from the right side RS toward the center of the bounding box BB, the higher the detection sensitivity of the overtaking behavior of the rear vehicle FV, and the more the setting position of the right judgment line RS1 is offset from the right side RS toward the outside of the bounding box BB, the lower the detection sensitivity of the overtaking behavior of the rear vehicle FV.

[0064] Similarly, the more the setting position of the left judgment line LS1 is offset from the left side LS toward the center of the bounding box BB, the higher the detection sensitivity of the overtaking behavior of the rear vehicle FV, and the more the setting position of the left judgment line LS1 is offset from the left side LS toward the outside of the bounding box BB, the lower the detection sensitivity of the overtaking behavior of the rear vehicle FV.

[0065] When a right judgment line RS1 and a left judgment line LS1 are provided as described above, a user of the in-vehicle device 10 may be able to adjust the offset position of the right judgment line RS1 from the right side RS and the offset position of the left judgment line LS1 from the left side LS, for example, by specifying a setting position on the touch panel display 12.

[0066] In addition, in this embodiment, the control unit 17 detects the overtaking behavior of the rear vehicle FV based on the direction of the motion vector of the midpoint of one of the vertical edges of the bounding box BB that crosses the reference line BL, but the point at which the direction of the motion vector is observed is not limited to the midpoint of the edge that crosses the reference line BL.

[0067] For example, the control unit 17 may detect the overtaking behavior of the following vehicle FV based on the directions of motion vectors at the upper and lower ends of the right side RS and the left side LS that cross the reference line BL, i.e., at the corners of the bounding box BB. Alternatively, the control unit 17 may detect the overtaking behavior of the following vehicle FV based on the directions of motion vectors of points on the right determination line RS1 and the left determination line LS1 that cross the reference line BL.

[0068] In this embodiment, the control unit 17 notifies the driver of vehicle M that the rear vehicle FV is about to overtake vehicle M by playing a warning sound through the speaker 13, but the method of notifying the driver of vehicle M is not limited to this.

[0069] For example, the control unit 17 may display a message indicating a warning on the touch panel display 12 in addition to or instead of outputting a voice through the speaker 13. When playing a voice to the driver through the speaker 13, a continuous sound such as a buzzer may simply be output instead of the above-mentioned voice announcement.

[0070] The control unit 17 may not only notify the driver of the vehicle M, but also, for example, an occupant of a leading vehicle traveling in front of the vehicle M. For example, the control unit 17 may output a buzzer sound or the like to the leading vehicle when detecting an overtaking behavior of the rear vehicle FV. EXAMPLES

[0071] Next, the processing of the in-vehicle device 10 according to the second embodiment will be described with reference to Fig. 9 and Fig. 10. In the second embodiment, the configuration of the in-vehicle device 10 and the processing of the control unit 17 are different from those in the first embodiment, but other points, such as the method of detecting the overtaking behavior of the rear vehicle FV by the control unit 17, are the same as those in the first embodiment. Note that, in this embodiment as well, the vehicle M and the rear vehicle FV will be described as traveling in the center lane of a three-lane road.

[0072] Fig. 9 is a top view of vehicle M as a host vehicle traveling on a road in Example 2 and a following vehicle FV traveling behind vehicle M. Fig. 9 shows the moment when vehicle M enters a curved road CR turning left from a straight road SR, and shows a situation where the following vehicle FV is traveling on the straight road SR behind vehicle M. In Fig. 9, the reference line BL shown in Fig. 5 is shown extended backward along the traveling direction of vehicle M.

[0073] As shown in Figure 9, when the posture of vehicle M changes as vehicle M enters a curved road CR turning left, the rear vehicle FV traveling on the straight road SR moves horizontally in the angle of view of the in-vehicle camera 14 of vehicle M, even if the rear vehicle FV itself does not perform an overtaking behavior. That is, in the case shown in Figure 9, the rear vehicle FV moves to the right of the reference line BL as viewed from the in-vehicle camera 14.

[0074] Therefore, in a case such as that shown in Figure 9, the left side LS of the bounding box BB crosses the reference line BL, and the movement vector of the midpoint of the left side LS faces to the right, which is the overtaking direction, so the control unit 17 may mistakenly detect that the rear vehicle FV has performed an overtaking behavior even though in fact the rear vehicle FV has not performed an overtaking behavior.

[0075] Fig. 10 is a block diagram showing an example of a hardware configuration of the in-vehicle device 10 according to Example 2. In this example, the vehicle M is provided with an angular velocity sensor 24 connected to an input unit 19 as shown in Fig. 10, and the control unit 17 is configured to be able to sequentially acquire the lateral angular velocity of the vehicle M from the angular velocity sensor 24 via the input unit 19.

[0076] The control unit 17 suspends detection of the overtaking behavior of the rear vehicle FV when the angular velocity of the vehicle M acquired from the angular velocity sensor 24 exceeds a predetermined value. Then, the control unit 17 resumes detection of the overtaking behavior of the rear vehicle FV when the angular velocity of the vehicle M falls below the predetermined value while the detection of the overtaking behavior is suspended.

[0077] That is, in this embodiment, the control unit 17 determines that the vehicle M has entered the curved road CR when the angular velocity of the vehicle M exceeds a predetermined value, and suspends detection of the overtaking behavior of the rear vehicle FV until the angular velocity of the vehicle M falls below the predetermined value. This makes it possible to prevent erroneous detection of the overtaking behavior of the rear vehicle FV even when the vehicle M has entered the curved road CR, for example, as shown in Fig. 9.

[0078] A specific operation of the in-vehicle device 10 in this embodiment will be described below with reference to Fig. 11. Fig. 11 is a flowchart showing a behavior detection interruption routine RT2 executed by the control unit 17 of the in-vehicle device 10. The control unit 17 starts the behavior detection interruption routine RT2, for example, when the in-vehicle device 10 is powered on, and repeatedly executes the routine while the power is on.

[0079] First, the control unit 17 determines whether or not the current angular velocity of the vehicle M exceeds a predetermined value (step S201). When the control unit 17 determines that the angular velocity does not exceed the predetermined value (step S201: NO), for example, when the vehicle M is traveling on a straight road SR, the control unit 17 ends the behavior detection interruption routine RT2.

[0080] When the control unit 17 determines that the current angular velocity of the vehicle M exceeds the predetermined value (step S201: YES), it interrupts the detection of the overtaking behavior of the following vehicle FV (step S202). That is, the control unit 17 temporarily stops the above-mentioned behavior detection routine RT1.

[0081] After step S202, the control unit 17 determines whether the current angular velocity of the vehicle M has become equal to or less than a predetermined value (step S203). If the angular velocity of the vehicle M is not equal to or less than the predetermined value, that is, if the angular velocity remains higher than the predetermined value (step S203: NO), the control unit 17 repeats step S203.

[0082] When the control unit 17 determines that the current angular velocity of the vehicle M is equal to or less than the predetermined value (step S203: YES), it resumes the interrupted detection of the overtaking behavior of the following vehicle FV (step S204). That is, the control unit 17 resumes the above-mentioned behavior detection routine RT1. The control unit 17 ends the behavior detection interruption routine RT2 after step S204.

[0083] According to the in-vehicle device 10 of this embodiment, the above-mentioned behavior detection interruption routine RT2 can prevent erroneous detection of the overtaking behavior of the rear vehicle FV, even when, for example, when the vehicle M as the host vehicle enters a curved road CR and the rear vehicle FV moves horizontally without performing an overtaking behavior in the field of view of the in-vehicle camera 14.

[0084] Therefore, according to the in-vehicle device 10 of this embodiment, it is possible to prevent erroneous detection of the behavior of the rear vehicle FV, while predicting the movement of the rear vehicle FV from the behavior of the rear vehicle FV recognized through the rear window RW using the in-vehicle camera 14 installed at the front of the vehicle cabin.

[0085] [Modification of Example 2] Next, a modified example of the second embodiment will be described with reference to Fig. 12. Fig. 12 is a diagram showing a behavior detection interruption routine RT2'. In the behavior detection interruption routine RT2', only step S203 is different from the behavior detection interruption routine RT2 of the second embodiment, and the rest is the same as the second embodiment.

[0086] In this modification, the control unit 17 determines whether a predetermined time has elapsed when the detection of the overtaking behavior of the following vehicle FV is interrupted in step S202 (step S203), and if the predetermined time has not elapsed (step S203: NO), repeats step S203. If the predetermined time has elapsed (step S203: YES), the control unit 17 resumes the detection of the overtaking behavior of the following vehicle FV (step S204).

[0087] In the second embodiment, detection of the overtaking behavior of the rear vehicle FV is suspended until the angular velocity of the vehicle M falls below a predetermined value. However, for example, if the rear vehicle FV performs an overtaking behavior while the vehicle M is traveling on a curved road CR, the control unit 17 may not be able to detect the overtaking behavior.

[0088] Therefore, in this modification, detection of the overtaking behavior of the rear vehicle FV is suspended only for a predetermined time after the angular velocity of the vehicle M exceeds a predetermined value, such as for a few seconds when the vehicle M enters the curved road CR. This allows the control unit 17 to detect the overtaking behavior of the rear vehicle FV even while the vehicle M and the rear vehicle FV are traveling on the curved road CR.

[0089] For example, when vehicle M is traveling around a left curve in the middle lane of a three-lane road as shown in Figure 9, the driver of vehicle M can relatively easily notice a rear vehicle FV overtaking from the inside of the vehicle, but cannot easily notice a rear vehicle FV overtaking from the outside of the vehicle.

[0090] The in-vehicle device 10 of this modified example detects overtaking behavior by the rear vehicle FV from the outside of the vehicle, which is particularly difficult for the driver to notice when traveling around a curve, and alerts the driver that the rear vehicle FV is attempting to overtake, allowing the driver of vehicle M to drive safely even when traveling around a curve.

[0091] In addition, according to the in-vehicle device 10 of this modified example, it is possible to detect overtaking behavior from the outside of the vehicle when turning right as well as when turning left, so that the driver of vehicle M can be aware of the detection of overtaking behavior of the rear vehicle FV while traveling on a curved road on either the left or right side. EXAMPLES

[0092] Next, the process of the in-vehicle device 10 according to the third embodiment will be described with reference to Fig. 13. Fig. 13 is a block diagram showing an example of a hardware configuration of the in-vehicle device 10 according to the third embodiment. In the third embodiment, the configuration of the in-vehicle device 10 and the method of interrupting the overtaking behavior are different from those in the second embodiment, and other points, such as a method of detecting the overtaking behavior of the following vehicle FV, are the same as those in the second embodiment.

[0093] In this embodiment, the vehicle M is provided with a speed sensor 25 connected to the input unit 19 in addition to the angular velocity sensor 24 as shown in FIG. 13, and the control unit 17 is configured to be able to sequentially acquire the speed of the vehicle M from the speed sensor 25 via the input unit 19.

[0094] In this embodiment, the control unit 17 calculates the curve radius of the road on which the vehicle M is currently traveling from the angular velocity of the vehicle M acquired from the angular velocity sensor 24 and the speed of the vehicle M acquired from the speed sensor 25, and suspends detection of the overtaking behavior of the rear vehicle FV if the curve radius is equal to or smaller than a predetermined value. Then, the control unit 17 resumes detection of the overtaking behavior if the curve radius exceeds the predetermined value while the detection of the overtaking behavior is suspended.

[0095] For example, when the curve radius of the road on which vehicle M is currently traveling is 450 m or less, the control unit 17 assumes that the rear vehicle FV, which has been traveling directly behind vehicle M as shown in Example 2, is moving outside the field of view of the in-vehicle camera 14, and interrupts detection of the overtaking behavior of the rear vehicle FV.

[0096] A specific operation of the in-vehicle device 10 in this embodiment will be described below with reference to Fig. 14. Fig. 14 is a flowchart showing a behavior detection interruption routine RT3 executed by the control unit 17 of the in-vehicle device 10. The control unit 17 starts the behavior detection interruption routine RT3, for example, when the in-vehicle device 10 is powered on, and repeatedly executes the behavior detection interruption routine RT3 while the power is on.

[0097] First, the control unit 17 acquires the angular velocity and the speed of the vehicle M, and calculates the curve radius of the road on which the vehicle M is currently traveling from the acquired angular velocity and speed (step S301).

[0098] After step S301, the control unit 17 judges whether or not the current curve radius calculated in step S301 is equal to or less than a predetermined value (step S302). For example, the control unit 17 judges whether or not the curve radius is equal to or less than 450 m. When the control unit 17 judges that the curve radius is not equal to or less than the predetermined value (step S302: NO), the control unit 17 ends the behavior detection interruption routine RT3.

[0099] When the control unit 17 determines that the current curve radius is equal to or smaller than the predetermined value (step S302: YES), it suspends the detection of the overtaking behavior of the following vehicle FV (step S303). That is, the control unit 17 temporarily stops the above-mentioned behavior detection routine RT1 while the vehicle M is traveling on a curved road with a small curvature radius.

[0100] After step S303, the control unit 17 determines whether or not the current curve radius of the vehicle M has exceeded the predetermined value (step S304). If the curve radius of the vehicle M has not exceeded the predetermined value, that is, if the curve radius remains equal to or smaller than the predetermined value (step S304: NO), the control unit 17 repeats step S304 again.

[0101] When the control unit 17 determines that the current curve radius of the vehicle M exceeds the predetermined value (step S304: YES), it resumes the interrupted detection of the overtaking behavior of the following vehicle FV (step S305). That is, the control unit 17 resumes the above-mentioned behavior detection routine RT1. The control unit 17 ends the behavior detection interruption routine RT3 after step S305.

[0102] In the in-vehicle device 10 of this embodiment as well, the above-mentioned behavior detection interruption routine RT3 can prevent erroneous detection of the overtaking behavior of the rear vehicle FV, even when, for example, the vehicle M as the host vehicle enters a curved road CR and the rear vehicle FV moves horizontally without performing an overtaking behavior in the field of view of the in-vehicle camera 14.

[0103] In addition to determining whether to suspend detection of the overtaking behavior based on the magnitude of the curve radius as shown in this embodiment, the control unit 17 may also determine whether to suspend detection of the overtaking behavior based on the magnitude of the angular velocity as shown in Example 2.

[0104] [Modification of Example 3] Next, a modified example of the third embodiment will be described with reference to Fig. 15. Fig. 15 is a block diagram showing an example of a hardware configuration of the in-vehicle device 10 according to this modified example. In this modified example, the vehicle M has a GNSS receiver 26 instead of the angular velocity sensor 24 and the velocity sensor 25 shown in Fig. 13.

[0105] The GNSS receiver 26 is a receiver that receives signals (GNSS signals) from Global Navigation Satellite System (GNSS) satellites. In this modification, the control unit 17 acquires the current position of the vehicle M from the GNSS signals received by the GNSS receiver 26.

[0106] In this modification, the mass storage device 22 has a map information database 22B (hereinafter referred to as map information DB 22B) which is a database in which map information is stored. In this modification, the control unit 17 functions as a map information acquisition unit that acquires map information from the map information DB 22B.

[0107] In this modification, the control unit 17 calculates the curve radius of the road on which the vehicle M is currently traveling, based on the GNSS signal received by the GNSS receiver 26 and the map information stored in the map information DB 22B.

[0108] In this modification, the control unit 17 suspends detection of the overtaking behavior of the following vehicle FV when the calculated curve radius is equal to or smaller than a predetermined value, as in the third embodiment. Then, the control unit 17 resumes detection of the overtaking behavior when the curve radius exceeds the predetermined value while the detection of the overtaking behavior is suspended.

[0109] In the in-vehicle device 10 of this modified example as well, the behavior detection interruption routine RT3 shown in the third embodiment can prevent erroneous detection of the overtaking behavior of the rear vehicle FV, even when, for example, the vehicle M as the host vehicle enters a curved road CR and the rear vehicle FV moves horizontally without performing an overtaking behavior in the field of view of the in-vehicle camera 14.

[0110] In Example 3 and the modified example of Example 3, the control unit 17 suspends detection of the overtaking behavior of the rear vehicle FV when the calculated curve radius is equal to or smaller than a predetermined value. However, the detection of the overtaking behavior may not be suspended and the position of the reference line BL may be shifted according to the calculated curve radius.

[0111] For example, the control unit 17 may shift the position of the reference line BL horizontally from the midpoint MP as the calculated curve radius becomes smaller. This makes it easier to detect the overtaking behavior of the rear vehicle FV while traveling around a curve, compared to when the reference line BL is positioned so as to always pass through the midpoint MP of the lane.

[0112] The control routines shown in the above-mentioned first to third embodiments are merely examples, and can be appropriately selected and modified according to the application or conditions of use.

[0113] Furthermore, in the above-mentioned embodiments 1 to 3, the case where the road has three lanes on each side has been described, but the on-vehicle device 10 can also detect the overtaking behavior of the rear vehicle FV on other roads, such as general roads with two lanes on each side and expressways with four lanes on each side. [Explanation of symbols]

[0114] 10 Onboard equipment 12 Touch panel display 13 Speakers 14 In-car camera 16 System Bus 17 Control section 19 Input section 21 Output section 22 Mass storage 24 Angular rate sensor 25 Speed ​​Sensor 26 GNSS receivers

Claims

1. an image acquisition unit that acquires an image including a rear window of the vehicle interior captured by an imaging unit provided in the vehicle interior; an identification unit that identifies a rear vehicle area in the captured image, the rear vehicle area being an area in which a rear vehicle captured through the rear window exists; a detection unit that detects a specific behavior of the rear vehicle in accordance with a movement of an outer edge of the rear vehicle area with respect to a reference line that is set at a predetermined position within an area in which the rear window is present in the captured image and that extends in a vertical direction; 13. An information processing device comprising:

2. The information processing apparatus according to claim 1 , wherein the detection unit detects, as the specific behavior, an overtaking behavior in which the rear vehicle attempts to overtake the host vehicle.

3. The rear vehicle area has two outer edge lines that face each other and extend in a vertical direction so as to sandwich the rear vehicle therebetween, The information processing device according to claim 2, characterized in that the detection unit detects the overtaking behavior based on a direction of a motion vector of a point on one of the two outer edge lines before and after the one of the two outer edge lines crosses the reference line.

4. 3. The information processing apparatus according to claim 1, wherein the imaging unit is provided at a front portion of the vehicle interior.

5. 3 . The information processing device according to claim 1 , wherein the reference line passes through a horizontal midpoint of a lane in which the host vehicle is traveling in the captured image. 4 .

6. 3. The information processing apparatus according to claim 1, wherein the position of the reference line is set based on an operation by a user of the information processing apparatus.

7. 3 . The information processing device according to claim 2 , wherein the detection unit suspends detection of the overtaking behavior when the angular velocity of the host vehicle exceeds a predetermined value until the angular velocity falls below the predetermined value.

8. 3. The information processing device according to claim 2, wherein the detection unit suspends detection of the overtaking behavior for a predetermined time when the angular velocity of the host vehicle exceeds a predetermined value.

9. 9. The information processing device according to claim 2, wherein the detection unit calculates a curve radius of the road on which the vehicle is traveling from the speed and angular velocity of the vehicle, and interrupts detection of the overtaking behavior when the magnitude of the curve radius is equal to or smaller than a predetermined value.

10. A map information acquisition unit for acquiring map information, 9. The information processing device according to claim 2, wherein the detection unit calculates a curve radius of the road on which the vehicle is traveling from the current position of the vehicle and the map information, and interrupts detection of the overtaking behavior when the magnitude of the curve radius is equal to or smaller than a predetermined value.

11. 3. The information processing device according to claim 2, further comprising a notification unit that notifies an occupant of the host vehicle that the rear vehicle is attempting to overtake the host vehicle when the detection unit detects the overtaking behavior.

12. An information processing method executed by an information processing device, an image acquisition step of acquiring an image including a rear window of the vehicle interior captured by an imaging unit provided in the vehicle interior; a specifying step of specifying a rear vehicle area in the captured image, the rear vehicle area being an area in which a rear vehicle captured through the rear window exists; a detection step of detecting a specific behavior of the rear vehicle in accordance with a movement of an outer edge of the rear vehicle area with respect to a reference line that is set at a predetermined position in an area in which the rear window is present in the captured image and that extends in a vertical direction; 13. An information processing method comprising:

13. A program for causing a computer to execute the program, an image acquisition step of acquiring an image including a rear window of the vehicle interior captured by an imaging unit provided in the vehicle interior; a specifying step of specifying a rear vehicle area in the captured image, the rear vehicle area being an area in which a rear vehicle captured through the rear window exists; a detection step of detecting a specific behavior of the rear vehicle in accordance with a movement of an outer edge of the rear vehicle area with respect to a reference line that is set at a predetermined position in an area in which the rear window is present in the captured image and that extends in a vertical direction; A program for causing the computer to execute the above.

14. In the information processing device, an image acquisition step of acquiring an image including a rear window of the vehicle interior captured by an imaging unit provided in the vehicle interior; a specifying step of specifying a rear vehicle area in the captured image, the rear vehicle area being an area in which a rear vehicle captured through the rear window exists; a detection step of detecting a specific behavior of the rear vehicle in accordance with a movement of an outer edge of the rear vehicle area with respect to a reference line that is set at a predetermined position in an area in which the rear window is present in the captured image and that extends in a vertical direction; A recording medium for recording a program for executing the above.

Images