Wrong-way driving determination device, wrong-way driving determination method, and wrong-way driving determination program

The wrong-way driving determination device uses image analysis to identify and interpret signboards to prevent vehicles from reversing direction, enhancing safety by detecting and preventing wrong-way maneuvers.

JP7594133B2Active Publication Date: 2024-12-03J-QUAD DYNAMICS INC
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2023559614
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-11-12
Filing Date
2022-11-07
Publication Date
2024-12-03
Estimated Expiration
2042-11-07

AI Technical Summary

Technical Problem

Existing systems fail to quickly detect when a vehicle is likely to run in the wrong direction, leading to potential accidents during merging maneuvers.

Method used

A wrong-way driving determination device that analyzes image data from a forward-facing camera to identify predetermined objects, such as signboards, and determines the vehicle's direction by evaluating their arrangement and displacement within the image frame, triggering interventions like forced stopping if conditions indicate a potential wrong-way maneuver.

Benefits of technology

Enables rapid detection of potential wrong-way driving, reducing the risk of accidents by preventing vehicles from entering merging lanes in the wrong direction through precise determination and timely intervention.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007594133000001
    Figure 0007594133000001
  • Figure 0007594133000002
    Figure 0007594133000002
  • Figure 0007594133000003
    Figure 0007594133000003
Patent Text Reader

Abstract

This wrong-way travel determination device is configured to execute wrong-way travel determination processing (S20, S22) for determining, on the basis of a prescribed object (52) represented in image data corresponding to the output of a camera (10) directed to the forward direction of a vehicle, whether the vehicle is traveling the wrong-way. The prescribed object is an object disposed periodically along a merging road that merges with the road on which the vehicle is traveling. The wrong-way travel determination processing includes at least one of two processes, namely: a process of determining that the vehicle is traveling the wrong-way if the prescribed object is captured in each of two regions, namely a left side region and a right side region in a horizontal direction, represented in the image data; and a process of determining that the vehicle is traveling the wrong-way if the prescribed object is displaced from one end to the other end of the left-right ends of a region represented in the image data.
Need to check novelty before this filing date? Find Prior Art

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Japanese Application No. 2021-184762 filed on November 12, 2021, the contents of which are incorporated herein by reference. [Technical field]

[0002] The present disclosure relates to a wrong-way driving determination device, a wrong-way driving determination method, and a wrong-way driving determination program. [Background technology]

[0003] For example, Patent Document 1 listed below describes a device that recognizes marks on a road based on an image captured by a camera and determines whether or not the vehicle is traveling in the wrong direction. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-66948 A Summary of the Invention

[0005] The above-mentioned device determines that the vehicle is traveling in the wrong direction only when the vehicle is actually traveling in the wrong direction, and therefore cannot detect the vehicle in the wrong direction at an early stage when the vehicle is likely to start traveling in the wrong direction. The following describes the means and effects of one embodiment of the present disclosure.

[0006] The reverse driving detection device is configured to execute a reverse driving detection process for determining whether or not the vehicle is driving in the wrong direction based on a predetermined object indicated by image data corresponding to the output of a camera pointed forward of the vehicle, the predetermined object being an object periodically arranged along a merging road that merges with the road on which the vehicle is driving, and the reverse driving detection process includes at least one of two processes: a process for determining that the vehicle is driving in the wrong direction when the predetermined object is imaged in each of two areas, a left area and a right area, in the horizontal direction indicated by the image data, and a process for determining that the vehicle is driving in the wrong direction when the predetermined object is displaced from one of the left and right ends of the area indicated by the image data to the other.

[0007] When a vehicle is attempting to normally enter a merging road, the predetermined object tends to be present in only one of the two regions, the left region and the right region, in the horizontal direction, shown in the image data. On the other hand, when a vehicle is attempting to run in the wrong direction, the predetermined object tends to be present in each of the two regions, the left region and the right region, in the horizontal direction, shown in the image data. Therefore, according to the process of determining that the vehicle is running in the wrong direction when the predetermined object is imaged in each of the two regions, it is possible to determine that the vehicle is attempting to run in the wrong direction before it actually starts running in the wrong direction.

[0008] In addition, the direction in which a specific object is displaced in the area indicated by the image data tends to be opposite between when the vehicle is attempting to enter the merging road normally and when the vehicle is attempting to reverse direction. Therefore, according to the process of determining that the vehicle is attempting to reverse direction when the specific object is displaced from one side to the other, it is possible to determine that the vehicle is attempting to reverse direction before it actually starts reverse direction.

[0009] Therefore, according to the above configuration, it is possible to quickly determine whether the vehicle is about to travel in the wrong direction. The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing a partial configuration of a vehicle according to a first embodiment; [Diagram 2] FIG. 2 is a diagram illustrating a field of view of a camera according to the embodiment; [Diagram 3] FIG. 3 is a flowchart showing a procedure of a process executed by the control device according to the embodiment; [Figure 4] FIG. 4 is a diagram illustrating a signboard used for determining a wrong-way driving risk scene according to the embodiment; [Diagram 5] FIG. 5 is a diagram illustrating an image when traveling in the wrong direction according to the embodiment; [Figure 6] 6(a) and 6(b) are diagrams illustrating image displacements at the time of merging according to the embodiment; [Figure 7] FIG. 7 is a diagram illustrating an image for explaining the effect of the embodiment. [Figure 8] FIG. 8 is a flowchart showing a procedure of a process executed by the control device according to the second embodiment. [Figure 9] FIG. 9 is a diagram illustrating a method of determining whether or not a vehicle is running in the wrong direction according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] <First embodiment> Hereinafter, the first embodiment will be described with reference to the drawings. A camera 10 shown in FIG. 1 outputs image data Dp, which is data obtained by capturing an image of the outside of a vehicle VC.

[0012] As shown in Fig. 2, the field of view FV of the camera 10 is the area in front of the vehicle VC. In this embodiment, the field of view FV is a symmetrical area with respect to the front-rear direction of the vehicle. Here, the horizontal angle of the field of view FV is 180° or less.

[0013] Returning to FIG. 1, the image ECU 20 executes a recognition process of an object around the vehicle based on the image data Dp. More specifically, the image ECU 20 includes a PU 22 and a storage device 24. The PU 22 is a software processing device including at least one of a CPU, a GPU, and a TPU. The storage device 24 stores an object recognition program 24a and object recognition mapping data 24b. The object recognition mapping data 24b specifies an object recognition mapping that receives image data Dp as input and outputs a variable value related to the recognition result of an object. The object recognition mapping includes both a discrimination model and a regression model. The discrimination model uses a variable indicating whether or not an object is a predetermined object as an output variable. The regression model uses a coordinate value at a predetermined position on the contour of a predetermined object as an output variable. When the PU 22 executes the object recognition program 24a, the image data Dp is input to the object recognition mapping, thereby executing the object recognition process.

[0014] The ADASECU 30 executes a process for controlling the running of the vehicle VC. When executing the process for controlling the running, the ADASECU 30 receives the recognition result by the image ECU 20 via the local network 28. The ADASECU 30 also operates a drive system 40 and a braking system 42.

[0015] The drive system 40 includes at least one of the two thrust generating devices of the vehicle, an internal combustion engine and a rotating electric machine. The drive system 40 may also include a drive control device that controls the internal combustion engine and the rotating electric machine. In this case, "the ADASECU 30 operates the drive system 40" means that the ADASECU 30 outputs a command signal to the drive control device.

[0016] The braking system 42 includes at least one of a device that uses frictional force to decelerate the rotation of the wheels, and a device that converts the power of the wheels into electrical energy to decelerate the rotation of the wheels. The device that decelerates the rotation of the wheels by converting it into electrical energy may be shared with the rotating electric machine of the drive system. The braking system may include a braking control device that controls the device that decelerates the rotation of the wheels. In this case, "the ADASECU 30 operates the braking system 42" means that the ADASECU 30 outputs a command signal to the braking control device.

[0017] The ADASECU 30 includes a PU 32 and a storage device 34. The PU 32 is a software processing device including at least one of a CPU, a GPU, and a TPU. A driving assistance program 34a is stored in the storage device 34. The driving assistance program 34a includes, in particular, command data for causing the PU 32 to execute assistance processing for suppressing reverse running of the vehicle VC.

[0018] Fig. 3 shows the procedure of the support process for suppressing reverse running of the vehicle VC. The process shown in Fig. 3 is realized by the PU 32 repeatedly executing the driving support program 34a stored in the storage device 34, for example, at a predetermined interval. In the following, the step number of each process is represented by a number preceded by "S".

[0019] In the series of processes shown in FIG. 3, the PU 32 first obtains the object recognition result by the image ECU 20 (S10). Next, the PU 32 judges whether or not the flag F is "1" (S12). The flag F is "1" when the scene is a wrong-way driving risk scene, and is "0" when the scene is not a wrong-way driving risk scene. The wrong-way driving risk scene is a driving scene in which a wrong-way driving risk occurs. When the PU 32 judges that the flag F is "0" (S12: NO), the PU 32 judges whether or not an object indicating a junction point has been detected based on the recognition result obtained by the process of S10 (S14).

[0020] An object indicating a merging point is, for example, a signboard or the like shown in Fig. 4. Fig. 4 shows an example of a signboard indicating merging onto a main lane. In this embodiment, when the lane in which the vehicle VC is traveling merges into another lane, it is determined that there is a risk of wrong-way driving. Here, the travel direction of the lane in which the vehicle VC is traveling and the travel direction of the other lane form an acute angle.

[0021] The recognition result can be obtained by using a discriminative model in the object recognition map. That is, the recognition result can be obtained by including a variable indicating whether or not the object is a signboard in the output variable of the discriminative model. The discriminative model is a trained model trained using the image data Dp of the signboard and the image data Dp not including the signboard as training data.

[0022] Returning to FIG. 3, when it is determined that an object at the meeting point has been detected (S14: YES), the PU 32 assigns "1" to the flag F (S16). On the other hand, when the PU 32 determines that the flag F is "1" (S12: YES), the PU 32 determines whether or not an arrow signboard has been detected based on the recognition result acquired in the process of S10 (S18).

[0023] FIG. 5 illustrates an example of an arrow sign 52. The arrow sign 52 indicates the travel direction of the lane into which the lane in which the vehicle VC is traveling merges. A plurality of arrow signs 52 are placed at predetermined intervals along the lane into which the vehicle VC is traveling. Placing "a plurality of arrow signs at predetermined intervals along the lane into which the vehicle VC is traveling" means the following. That is, when there is no lane adjacent to the lane into which the vehicle VC is traveling in the opposite direction, the arrow signs are placed along the travel direction in an area adjacent to the lane into which the vehicle VC is traveling. Also, when there is a lane adjacent to the lane into which the vehicle VC is traveling in the opposite direction, the arrow signs are placed along the travel direction in an area adjacent to the opposite lane.

[0024] The recognition result used in the process of S18 can be obtained by using a discriminative model of the object recognition map. That is, the result can be obtained by including a variable indicating whether or not the object is an arrow signboard 52 in the output variable of the discriminative model. The discriminative model is a trained model trained using as training data the image data Dp in which the arrow signboard 52 is captured and the image data Dp that does not include the arrow signboard 52.

[0025] Returning to FIG. 3, when the PU 32 determines that the arrow signboard 52 has been detected (S18: YES), it determines whether or not the logical product of the following condition (i) and condition (ii) is true (S20).

[0026] Condition (i): The arrow signboard 52 is present on both the left and right sides ahead of and below the vehicle VC in the image represented by the image data Dp. FIG. 5 illustrates an example where condition (i) is satisfied. FIG. 5 shows an example where arrow signs 52 exist on both sides of a left-right determination line 54. FIG. 5 shows an example where the left-right determination line 54 is a line that divides an area into left and right halves at the horizontal center of an image 50 represented by image data Dp. FIG. 5 also shows an example where arrow sign 52 exists below a top-bottom determination line 56. FIG. 5 shows an example where the top-bottom determination line 56 is a line that divides an area into right and left halves at the vertical center of an image 50 represented by image data Dp.

[0027] Condition (ii): The displacement direction of the arrow sign 52 is to the left. In other words, the displacement direction of the arrow sign 52 is the direction proceeding from the driver's seat to the passenger's seat. FIG. 6 illustrates an example of the displacement of the arrow sign 52 in the image 50 when the vehicle VC is traveling in the wrong direction. FIG. 6(a) shows the time when the lane 60 in which the vehicle VC is traveling reaches a point where it merges with the destination lane 62. In this case, the arrow sign 52 is located on the right side of the image 50. In other words, the arrow sign 52 is located on the driver's seat side. FIG. 6(b) shows the time when the vehicle VC is about to travel in the wrong direction on the lane 62. In this case, the arrow sign 52 is located on the left side of the image 50. In other words, the arrow sign 52 is located on the passenger's seat side.

[0028] In this way, when the vehicle VC is about to travel in the wrong direction, the arrow sign 52 is displaced in the image 50 in the forward direction from the driver's seat to the passenger's seat. The recognition result used by the PU 32 in determining whether the recognition result acquired by the process of S10 satisfies the conditions (i) and (ii) can be acquired using a regression model of the object recognition mapping. That is, this can be realized by including, for example, the coordinates of the four vertices of the arrow signboard 52 in the output variables of the object recognition mapping. The regression model is a trained model trained using the image data Dp of the image of the arrow signboard 52 and the vertex coordinates of the arrow signboard 52 in the image data Dp as training data.

[0029] 3, when the logical AND is determined to be true (S20: YES), the PU 32 performs a reverse running determination (S22). Then, the PU 32 forcibly operates the drive system 40 and the braking system 42 to forcibly stop the vehicle VC (S24). Here, forcibly stopping the vehicle VC means executing control to stop the vehicle VC regardless of the accelerator operation by the user, etc.

[0030] When the process of S24 is completed or when a negative determination is made in the process of S20, the PU 32 assigns "0" to the flag F (S26). When the PU 32 completes the processes of S16 and S26, or when a negative determination is made in the processes of S14 and S18, the PU 32 temporarily ends the series of processes shown in FIG.

[0031] Here, the operation and effects of this embodiment will be described. When the lane in which the vehicle VC is traveling merges into another lane, the PU 32 determines that the vehicle is traveling in the wrong direction based on the arrangement and displacement direction of the arrow sign 52 indicating the traveling direction of the lane at the merging destination. This makes it possible to determine that the vehicle is traveling in the wrong direction before the vehicle actually travels in the lane at the merging destination.

[0032] According to the present embodiment described above, the following actions and effects can be obtained. (1) The PU 32 determines that the vehicle is traveling in the wrong direction when the logical product of the condition (i) and the condition (ii) is true. Therefore, it is possible to reduce erroneous determinations that the vehicle is traveling in the wrong direction, compared to a case in which the determination is made based only on the condition (i).

[0033] 7 shows a case where the lane into which the vehicle VC is to merge is curved. In this case, when the vehicle VC starts to travel in the lane into which the vehicle VC is to merge, the arrow sign 52 may be detected on both the left and right of the left / right determination line 54. Therefore, depending on the radius of curvature of the curve, the arrow sign 52 may be detected below the up / down determination line 56 and on both the left and right of the left / right determination line 54. However, in this case, the displacement direction of the arrow sign 52 does not satisfy condition (ii).

[0034] (2) Condition (i) is set to a condition that the logical product of the condition that the arrow sign 52 is located on both the left and right sides of the left-right determination line 54 and the condition that the arrow sign 52 is located below the up-down determination line 56 is true. This makes it possible to prevent the occurrence of a situation in which condition (i) is satisfied in the situation illustrated in FIG. 7, compared to a case in which condition (i) is set to a condition that the arrow sign 52 is located on both the left and right sides of the left-right determination line 54.

[0035] (3) The PU 32 executes the process of S20 on the condition that the wrong-way driving risk scene determination has been made. This makes it possible to suppress a positive determination in the process of S20 when, for example, an object that may be mistakenly recognized as an arrow signboard 52 is present at the rear of the vehicle traveling ahead.

[0036] (4) When the PU32 determines that the vehicle VC is traveling in the wrong direction, it forcibly stops the vehicle VC. This makes it possible to prevent the vehicle VC from interfering with the traveling of the vehicle in the lane where the vehicle is to merge. <Second embodiment> The second embodiment will be described below with reference to the drawings, focusing on the differences from the first embodiment.

[0037] Fig. 8 shows the procedure of the support process for suppressing reverse running of the vehicle VC according to this embodiment. The process shown in Fig. 8 is realized by the PU 32 repeatedly executing the driving support program 34a stored in the storage device 34, for example, at a predetermined interval. For convenience, the same step numbers are given to the processes in Fig. 8 that correspond to those shown in Fig. 3.

[0038] In the series of processes shown in FIG. 8, when an affirmative determination is made in the process of S18, the PU 32 determines whether or not the logical product of the condition (i) and the following condition (iii) is true (S20a).

[0039] Condition (iii): The difference in size between the arrow signs 52 is equal to or smaller than a predetermined value. This is a condition for not making a wrong-way driving determination when condition (i) is satisfied in the situation illustrated in FIG. 7.

[0040] Whether or not the difference in size is equal to or smaller than a predetermined value can be realized, for example, by calculating the area of ​​the arrow signboard 52 in the image 50 from the coordinates of the four vertices of the arrow signboard 52. If the PU 32 determines that the logical product is true (S20a: YES), the process proceeds to S22. If the PU 32 determines that the logical product is false (S20a: NO), the process proceeds to S26.

[0041] <Third embodiment> The third embodiment will be described below with reference to the drawings, focusing on the differences from the first embodiment.

[0042] In this embodiment, the condition (i) is changed to the condition shown in FIG. As shown in Fig. 9, in this embodiment, a pair of left-right determination lines 54l, 54r are provided. The left-right determination lines 54l, 54r are disposed symmetrically with respect to the left-right determination line 54 illustrated in Fig. 5. As a result, the left-right determination line 54l is located to the left of the horizontal center in the image 50. The left-right determination line 54r is located to the right of the horizontal center in the image 50.

[0043] In this embodiment, instead of condition (i), a condition that the arrow signboard 52 exists below the up-down determination line 56 in the image 50, to the left of the left-right determination line 54l and to the right of the left-right determination line 54r is adopted.

[0044] <Other embodiments> This embodiment can be modified as follows: This embodiment and the following modifications can be combined with each other to the extent that no technical contradiction occurs.

[0045] "About merging judgment processing" It is not essential to provide the processes of S12, S14, S16, and S26. Even if they are not provided, the risk of erroneous determination that may occur when only condition (i) is used can be reduced by adding, for example, the above condition (ii) or condition (iii) to the conditions for determining whether or not the vehicle is running in the wrong direction.

[0046] "About the wrong-way driving detection process" The condition for determining whether or not the vehicle is traveling in the wrong direction is not limited to either the condition that the logical product of the above conditions (i) and (ii) is true or the condition that the logical product of the above conditions (i) and (iii) is true. For example, the condition may be a condition that the logical product of the above conditions (i), (ii), and (iii) is true. For example, the condition may be a condition that the logical sum of the condition that the logical product of the above conditions (i) and (ii) is true and the condition that the logical product of the above conditions (i) and (iii) is true is true. For example, the condition may be a condition that the logical sum of the above conditions (i) and (ii) is true.

[0047] In the above-described embodiment and modified examples thereof, condition (i) may be replaced with a condition obtained by deleting the condition that the arrow signboard 52 is located below the up-down determination line 56 from the above-described condition (i).

[0048] "For a given object" In the above embodiment, a signboard with an arrow drawn on it is used as an example of a predetermined object used for wrong-way driving detection, but the present invention is not limited to this. For example, rubber poles placed at predetermined intervals may also be used.

[0049] "About handling reverse driving" In the above embodiment, the process of stopping the vehicle VC has been exemplified as the process of dealing with reverse running, but the present invention is not limited to this. For example, the process may be a process of warning the user that the user is about to run in the reverse direction. This can be realized, for example, by issuing a predetermined alarm sound. However, in this case, visual information may be added.

[0050] "About driving control devices" In the above embodiment, the ADASECU 30 as the driving control device receives the object recognition result obtained by performing image recognition processing or the like on the image data Dp by the image ECU 20, but this is not limited to the above. For example, the ADASECU 30 may execute the processing executed by the image ECU 20 in the above embodiment.

[0051] The driving control device is not limited to a device that includes the PU 32 and the storage device 34 and executes software processing. For example, a dedicated hardware circuit such as an ASIC may be provided that performs hardware processing of at least a part of what was processed by software in the above embodiment. That is, the execution device may have any of the following configurations (a) to (c). (a) A processing device that executes all of the above processing according to a program, and a program storage device that stores the program. (b) A processing device and program storage device that execute part of the above processing according to a program, and a dedicated hardware circuit that executes the remaining processing. (c) A dedicated hardware circuit that executes all of the above processing. Here, there may be a plurality of software execution devices including a processing device and a program storage device, and a dedicated hardware circuit.

[0052] "About Computers" The computer is not limited to a single PU 32 as illustrated in Fig. 1. Also, it is not limited to a computer installed in a vehicle as illustrated in Fig. 1. For example, the processes in Fig. 3 other than the process of S24 may be executed by a PU installed in a mobile terminal of a driver. In this case, the computer installed in the vehicle and a computer not installed in the vehicle communicate with each other and execute the driving assistance program 34a in cooperation.

[0053] Although the present disclosure has been described based on the embodiment, it is understood that the present disclosure is not limited to the embodiment or structure. The present disclosure also includes various modifications and modifications within the equivalent range. In addition, various combinations and forms, and other combinations and forms including only one element, more than one element, or less than one element, are also within the scope and concept of the present disclosure.

[0054] The statement "at least one of A and B" in this specification should be understood to mean "only A, or only B, or both A and B."

Claims

1. The system is configured to execute a reverse running determination process (S20, S22) for determining whether or not the vehicle is running in the wrong direction based on a predetermined object (52) indicated by image data corresponding to an output from a camera (10) facing forward of the vehicle, the predetermined object is an object that is periodically arranged along a merging road that merges with a road on which the vehicle is traveling, The reverse running determination process includes: a process of determining that the vehicle is traveling in the wrong direction when the predetermined object is captured in each of two areas, a left area and a right area in the horizontal direction, which are indicated by the image data, and when the predetermined object is located below an up-down determination line (56) that divides the area indicated by the image data in the up-down direction; a process of determining that the vehicle is traveling in the wrong direction when the predetermined object is captured in each of two areas, a left area and a right area in the horizontal direction indicated by the image data, and the difference in size between the captured predetermined objects is equal to or smaller than a predetermined value; and a process of determining that the vehicle is traveling in the wrong direction when the specified object displaces from one of the left and right ends of the area indicated by the image data to the other.

2. A merging determination process (S14) is executed to determine whether or not the vehicle will merge with the merging road, 2. The wrong-way driving determination device according to claim 1, wherein the determination of wrong-way driving by the wrong-way driving determination process is made on condition that a merging has been determined by the merging determination process.

3. 2. The wrong-way driving determination device according to claim 1, wherein the wrong-way driving determination process (S20, S22) includes a process of determining that the vehicle is driving in the wrong direction when the predetermined object is imaged in each of the two areas and the predetermined object is displaced from one of the predetermined areas to the other.

4. 2. The wrong-way running determination device according to claim 1, wherein when it is determined that the vehicle is running in the wrong direction by the wrong-way running determination process, a stop process (S24) is executed to stop the vehicle from running in the wrong direction.

5. A method for determining whether a vehicle is running in the wrong direction, comprising the steps of: executing each process in the wrong direction determination device (30) according to claim 1.

6. A wrong-way driving determination program for causing a computer to execute each process in the wrong-way driving determination device according to claim 1.

Citation Information

Patent Citations

  • Wrong-way driving detector

    JP2015121952A

  • Information processing device

    JP2017073073A

  • Reverse run determination system

    JP2017134441A

  • Reverse driving prevention device, reverse driving prevention method, and reverse driving prevention system

    JP2018084873A

  • Drive recorder

    JP2018195117A