Determination device and determination method

The determination device accurately determines the orientation of a curved mirror in an image using an analysis unit and determination unit, addressing the challenge of visibility issues by employing methods based on mirror components and surrounding context.

JP7595470B2Active Publication Date: 2024-12-06DENSO TEN LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2021009004
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-01-22
Publication Date
2024-12-06
Estimated Expiration
2041-01-22

AI Technical Summary

Technical Problem

Conventional techniques struggle to accurately determine the orientation of a curved mirror in an image, especially when the pillar portion is not visible.

Method used

A determination device equipped with an analysis unit and a determination unit that analyzes the image of the curved mirror, utilizing various methods to determine its orientation based on the relationship between the mirror surface, support, eaves, marks, and surrounding objects.

Benefits of technology

Enables accurate determination of the curved mirror's orientation in an image, even when the pillar portion is not visible, by employing multiple methods that leverage the mirror's components and surrounding context.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007595470000001
    Figure 0007595470000001
  • Figure 0007595470000002
    Figure 0007595470000002
  • Figure 0007595470000003
    Figure 0007595470000003
Patent Text Reader

Abstract

To provide a determination apparatus which accurately determines a direction of a curve mirror in an image, and a determination method.SOLUTION: A determination apparatus includes an analysis unit and a determination unit. The analysis unit analyzes an image of a curve mirror captured from a vehicle. The determination unit determines a direction of the curve mirror on the basis of information obtained by analysis of the analysis unit. For example, the determination unit determines a direction of the curve mirror on the basis of a relationship between a center position of a mirror surface of the curve mirror and a position of a pillar of the curve mirror. For example, the determination unit determines a direction of the curve mirror on the basis of a pattern of an image of a region of the mirror surface of the curve mirror.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a determination device and a determination method. [Background technology]

[0002] Conventionally, a technique for determining the orientation of a convex mirror is known (for example, see Patent Document 1). For example, the process for determining the orientation of a convex mirror is sometimes used as part of the process for determining whether or not an object approaching a vehicle is reflected in the convex mirror. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-8701 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology has a problem that the orientation of the convex mirror in the image may not be accurately determined. For example, the technology described in Patent Document 1 analyzes the image of the pillar part of the convex mirror to determine the orientation. Therefore, if the pillar part is not reflected in the image, it is difficult to determine the convex mirror.

[0005] The present invention has been made in consideration of the above, and aims to provide a determination device and a determination method that can accurately determine the orientation of a curved mirror in an image. [Means for solving the problem]

[0006] In order to solve the above problems and achieve the object, the determination device according to the present invention has an analysis unit and a determination unit. The analysis unit analyzes an image of a convex mirror captured from a vehicle. The determination unit determines the orientation of the convex mirror based on information obtained by the analysis by the analysis unit. Effect of the Invention

[0007] According to the present invention, the orientation of a curved mirror in an image can be accurately determined. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a determination device according to an embodiment. [Diagram 2] FIG. 2 is a diagram explaining the different orientations of convex mirrors. [Diagram 3] FIG. 3 is a diagram explaining the different orientations of convex mirrors. [Figure 4] FIG. 4 is a diagram illustrating the first determination method. [Diagram 5] FIG. 5 is a flowchart showing the procedure of the first determination method. [Figure 6] FIG. 6 is a diagram illustrating the second determination method. [Figure 7] FIG. 7 is a flowchart showing the procedure of the second determination method. [Figure 8] FIG. 8 is a diagram illustrating the third determination method. [Figure 9] FIG. 9 is a flowchart showing the procedure of the third determination method. [Figure 10] FIG. 10 is a diagram illustrating the fourth determination method. [Figure 11] FIG. 11 is a flowchart showing the procedure of the fourth determination method. [Figure 12] FIG. 12 is a diagram illustrating the fifth determination method. [Figure 13] FIG. 13 is a flowchart showing the procedure of the fifth determination method. [Figure 14] FIG. 14 is a diagram illustrating the sixth determination method. [Figure 15] FIG. 15 is a diagram illustrating the sixth determination method. [Figure 16] FIG. 16 is a flowchart showing the procedure of the sixth determination method. [Figure 17]FIG. 17 is a diagram illustrating the seventh determination method. [Figure 18] FIG. 18 is a flowchart of the entire determination process regarding an approaching object. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of a determination device and a determination method disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.

[0010] First, a determination device according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of a determination device according to an embodiment.

[0011] The determination device 10 receives as input an image captured by a camera mounted on a moving vehicle.

[0012] Then, the determination device 10 determines the orientation of the convex mirror reflected in the input image. For example, the determination device 10 determines whether the convex mirror reflected in the image faces right or left.

[0013] The determination device 10 may be provided in a vehicle, or may be a server connected to an in-vehicle device provided in the vehicle via a network such as the Internet.

[0014] For example, the determination of the direction of a convex mirror may be used to determine the approach of an object reflected in the convex mirror.

[0015] For example, it is conceivable to determine the approach of an object based on the direction of the optical flow of an object reflected in a curved mirror.

[0016] Here, the optical flow is a vector representation of changes in the positions of feature points in a series of images (eg, video images).

[0017] However, there are cases where approach determination cannot be performed accurately based only on the direction of the optical flow, as shown in Figures 2 and 3. Figures 2 and 3 are diagrams for explaining differences in the directions of convex mirrors.

[0018] In Figures 2 and 3, the convex mirrors are placed in the same position near the intersection. However, the convex mirror M10 in Figure 2 faces to the right, whereas the convex mirror M20 in Figure 3 faces to the left.

[0019] In Fig. 2, in an image IMG10 captured from a vehicle V11, an object, an optical flow F11 generated by a vehicle V12 faces leftward. M11a is the area of ​​the curved mirror M10 reflected in the image. V12a is the vehicle V12 reflected in the curved mirror M10.

[0020] On the other hand, in Fig. 3, in an image IMG20 taken from a vehicle V21, which is the subject vehicle, an optical flow F21 generated by a vehicle V22, which is an approaching object, is directed to the right. M21a is the area of ​​the curved mirror M20 reflected in the image. V22a is the vehicle V22 reflected in the curved mirror M20.

[0021] In this way, since the direction of the optical flow of an approaching object changes depending on the direction of the convex mirror, in order to determine an approaching object, it is necessary to determine the direction of the convex mirror.

[0022] As shown in FIG. 1, the determination device 10 includes an input / output unit 11, a storage unit 12, and a control unit 13.

[0023] The input / output unit 11 is an interface for inputting and outputting data, and can receive input of an image from a camera or other device.

[0024] Furthermore, the input / output unit 11 outputs the determination result, for example, the direction of the curved mirror reflected in the image, in the form of a text message, voice, binary data, or the like.

[0025] The control unit 13 and the memory unit 12 of the determination device 10 are realized by, for example, a computer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, an input / output port, etc., or various circuits.

[0026] The storage unit 12 corresponds to a RAM or a flash memory. The RAM or the flash memory can store information about various programs executed by the determination device 10, and the like.

[0027] The determination device 10 may acquire the above-mentioned programs and various information via another computer or a portable recording medium connected via a wired or wireless network.

[0028] The CPU of the computer functions as the analysis unit 131 and the determination unit 132 of the control unit 13 by, for example, reading and executing a program stored in the ROM.

[0029] The analysis unit 131 analyzes an image of a convex mirror captured from a vehicle. For example, the analysis unit 131 recognizes the convex mirror and other objects using existing object recognition technology, etc. Furthermore, the analysis unit 131 calculates parameters such as position coordinates and pixel values ​​of the recognized convex mirror and other objects.

[0030] Specifically, the analysis unit 131 recognizes the mirror surface, eaves, pillars, signs, etc. that constitute the convex mirror. Furthermore, the analysis unit 131 recognizes structures, vehicles, road signs, etc. in the image.

[0031] The determination unit 132 determines the orientation of the convex mirror based on the information obtained by the analysis by the analysis unit 131.

[0032] In this way, the determination device 10 can accurately determine the orientation of the convex mirror in the image by utilizing the analysis results of the image.

[0033] The determination unit 132 can make the determination by using one or more of the determination methods described below.

[0034] Moreover, the following Figures 4, 6, 8, 10, 12, 14, 15 and 17 show images captured by a vehicle camera.

[0035] Also, please note that the terms right and left in the explanation are based on the viewpoint from the vehicle, not on the viewpoint from the mirror surface of the convex mirror.

[0036] Thus, facing right means facing the right side of each image, and facing left means facing the left side of each image.

[0037] [First Judgment Method] In the first determination method, the determination unit 132 determines the orientation of the convex mirror based on the relationship between the center position of the mirror surface of the convex mirror and the positions of the support pillars of the convex mirror.

[0038] Fig. 4 is a diagram for explaining the first determination method, and Fig. 5 is a flowchart showing the procedure of the first determination method.

[0039] As shown in FIG. 4 and FIG. 5, first, the analysis unit 131 detects the position coordinates (horizontal coordinate: x1) of the center of the mirror surface portion of the curved mirror M101 (step S101).

[0040] However, in the following explanation, the coordinate values ​​on the horizontal axis increase toward the right side of the figure.

[0041] Next, the analysis unit 131 detects the position coordinates (horizontal coordinate: x2) of the center of the support part of the curved mirror M101 (step S102).

[0042] If x1>x2 is satisfied (step S103: Yes), the determining unit 132 determines that the curved mirror M101 faces right (step S104).

[0043] On the other hand, if x1>x2 is not true (step S103: No), the determining unit 132 determines that the curved mirror M101 faces left (step S105).

[0044] According to the first determination method, the direction can be determined from only the components of the convex mirror.

[0045] [Second Judgment Method] In the second determination method, the determination unit 132 determines the orientation of the convex mirror based on the relationship between the center position of the mirror surface of the convex mirror and the position of a sign provided on the support pillar of the convex mirror.

[0046] Fig. 6 is a diagram for explaining the second determination method, and Fig. 7 is a flowchart showing the procedure of the second determination method.

[0047] As shown in FIG. 6 and FIG. 7, first, the analysis unit 131 detects the position coordinates (horizontal coordinate: x3) of the center of the mirror surface portion of the curved mirror M102 (step S111).

[0048] Next, the analysis unit 131 detects the position coordinates (horizontal coordinate: x4) of the center of an auxiliary sign with the word "Caution" written on it that is attached to the support part of the convex mirror M102 (step S112).

[0049] If x3>x4 is satisfied (step S113: Yes), the determining unit 132 determines that the curved mirror M102 faces right (step S114).

[0050] On the other hand, if x3>x4 is not true (step S113: No), the determining unit 132 determines that the curved mirror M102 faces left (step S115).

[0051] According to the second determination method, the direction can be determined from only the components of the convex mirror.

[0052] [Third Judgment Method] In the third determination method, the determination unit 132 determines the orientation of the convex mirror M103 based on the relationship between the center position of the mirror surface of the convex mirror M103 and the position of the eaves area of ​​the convex mirror M103.

[0053] Fig. 8 is a diagram for explaining the third determination method, and Fig. 9 is a flowchart showing the procedure of the third determination method.

[0054] As shown in FIG. 8 and FIG. 9, first, the analysis unit 131 detects the position coordinates of the center of the mirror surface of the curved mirror M103 (step S121).

[0055] Next, analysis unit 131 calculates the proportion of orange in the upper right and upper left parts of the center of the mirror surface part (step S122).

[0056] When the proportion of orange in the upper right portion is greater than that in the upper left portion (step S123: Yes), the determination unit 132 determines that the convex mirror M103 faces right (step S124).

[0057] On the other hand, if the proportion of orange in the upper right portion is not large compared to the upper left portion (step S123: No), the determination unit 132 determines that the convex mirror M103 faces left (step S125).

[0058] The third method of judgment makes use of the fact that the awning of a convex mirror is orange in color and that the awning on the same side as the direction of the convex mirror appears larger than the awning on the opposite side.

[0059] In the third method, the judgment may be made based on not only the proportion of orange but also the difference in brightness.

[0060] For example, the determination unit 132 determines that the direction in which the luminance of the eaves portion (upper right portion, upper left portion) is smaller is the direction of the convex mirror.

[0061] This is because the inside of the eaves is visible on the same side as the direction of the convex mirror, and the outside of the eaves is visible on the opposite side. In this case, the brightness of the eaves part on the same side as the direction of the convex mirror is smaller than the eaves part on the opposite side.

[0062] According to the third determination method, the orientation can be determined from only the mirror surface and eaves portions that make up the convex mirror.

[0063] [Fourth Judgment Method] In the fourth determination method, the determination unit 132 determines the orientation of the convex mirror based on the inclination of the center line of the mirror surface of the convex mirror.

[0064] Fig. 10 is a diagram for explaining the fourth determination method, and Fig. 11 is a flowchart showing the procedure of the fourth determination method.

[0065] As shown in FIGS. 10 and 11, first, the analysis unit 131 calculates the inclination of the center line of the elliptical shape of the mirror surface portion (step S131).

[0066] If the center line is inclined to the right (step S132: Yes), the determination unit 132 determines that the curved mirror M104 faces right (step S133).

[0067] On the other hand, if the center line is not tilted to the right (step S132: No), the determination unit 132 determines that the curved mirror M104 faces left (step S134).

[0068] The fourth judgment method takes advantage of the fact that convex mirrors are elliptical in shape and installed at a forward inclination.

[0069] According to the fourth determination method, the orientation can be determined only from the mirror surface portion that makes up the convex mirror.

[0070] [Fifth Judgment Method] In the fifth determination method, the determination unit 132 determines the orientation of the convex mirror based on the pattern of the image of the mirror surface area of ​​the convex mirror.

[0071] Fig. 12 is a diagram for explaining the fifth determination method, and Fig. 13 is a flowchart showing the procedure of the fifth determination method.

[0072] As shown in FIG. 12 and FIG. 13, first, the analysis unit 131 extracts an image reflected in the curved mirror M105 (step S141).

[0073] Furthermore, the analysis unit 131 performs pattern matching of the extracted image (step S142). For example, the analysis unit 131 performs pattern matching with an area other than the curved mirror M105 in the image.

[0074] Then, the analysis unit 131 detects the position coordinates in the matching image (step S143). For example, when the image reflected on the curve mirror M105 matches the area near the road sign 202, the analysis unit 131 detects the position coordinates near the road sign 202.

[0075] If the detected position coordinates are to the right of the convex mirror M105 (step S144: Yes), the determination unit 132 determines that the convex mirror M105 faces right (step S145).

[0076] If the detected position coordinates are not to the right of the convex mirror M105 (step S144: No), the determination unit 132 determines that the convex mirror M105 faces left (step S146).

[0077] Furthermore, the determination unit 132 may determine the orientation of the curved mirror 105 based on the similarity between the arrangement of parameters (RGB values, brightness, etc.) of the area reflected on the curved mirror M105 and the arrangement of parameters of areas other than the curved mirror 105 in the image.

[0078] For example, in the example of FIG. 12, if the curved mirror M105 faces to the right, it is considered that the parameters of the structure and vehicle on the right side of the drawing are distributed in the area reflected in the curved mirror M105.

[0079] However, in reality, the convex mirror M105 faces left, so the area reflected in the convex mirror M105 has parameters distributed therein similar to those of the road signs and structures on the left side of the drawing.

[0080] According to the fifth determination method, the properties of the mirror surface of the convex mirror can be utilized to perform a highly accurate determination.

[0081] [6th Judgment Method] In the sixth determination method, the determination unit 132 determines the orientation of the convex mirror based on the direction indicated by a predetermined object reflected in the image. The object is, for example, a road sign, a vehicle, a structure, or the like.

[0082] Figures 14 and 15 are diagrams for explaining the sixth determination method, and Figure 16 is a flowchart showing the procedure of the sixth determination method.

[0083] As shown in FIGS. 14, 15, and 16, first, the analysis unit 131 detects a specific object in an image (step S151).

[0084] For example, the analysis unit 131 detects the road sign 204 in Fig. 14 as the object. The analysis unit 131 may also detect objects in the areas 205 and 206 in Fig. 15.

[0085] Furthermore, the analysis unit 131 may detect a road sign 201 and a vehicle 203 in FIG. 12 as objects.

[0086] The determination unit 132 determines, based on the detected object, whether or not a condition indicating that the convex mirror faces right is satisfied (step S152).

[0087] If the condition indicating that the object faces right is satisfied (step S152: Yes), the determination unit 132 determines that the convex mirror faces right (step S153).

[0088] On the other hand, if the condition indicating that the object faces right is not satisfied (step S152: No), the determination unit 132 determines that the convex mirror faces left (step S154).

[0089] For example, since the arrow on the road sign 204 (No entry except in the specified direction) in Fig. 14 does not point to the left, it is possible that a vehicle is approaching from the left. In that case, if a convex mirror is installed, it is estimated that it will face left.

[0090] Therefore, the determining unit 132 determines that the road sign 204 does not satisfy the condition indicating that the road sign faces right.

[0091] 15, there is a structure in area 205, but there is no structure in area 206. In such a case, if a curved mirror is installed to reduce the blind spot caused by the structure in area 205, it is estimated that the mirror faces left.

[0092] Therefore, the determining unit 132 determines that the area 205 and the area 206 do not satisfy the condition indicating that they face right.

[0093] Also, for example, since the road sign 201 (one-way street) in FIG. 12 points to the right, it is possible that a vehicle is approaching from the left, and in this case, if a curved mirror is installed, it is estimated that it will face left.

[0094] Therefore, the determining unit 132 determines that the road sign 201 does not satisfy the condition indicating that the road sign faces right.

[0095] Furthermore, for example, since the vehicle 203 in FIG. 12 is traveling in a rightward direction, if a curved mirror were installed, it would be estimated that the vehicle would be facing left.

[0096] Therefore, the determination unit 132 determines that the condition indicating that the vehicle 203 is facing right is not satisfied.

[0097] According to the sixth determination method, even if the state of the convex mirror itself is unclear, the orientation of the convex mirror can be determined from the surrounding conditions. Also, according to the sixth determination method, even if the convex mirror is outside the image capture range, the orientation can be determined.

[0098] [Seventh Judgment Method] In the seventh determination method, when two convex mirrors that satisfy a predetermined positional relationship are reflected in the image, the determination unit 132 determines one of the two convex mirrors to face left and the other to face right.

[0099] Fig. 17 is a diagram for explaining the seventh determination method. As shown in Fig. 17, the analysis unit 131 detects the distance between two curve mirrors M107a and M107b.

[0100] If the detected distance is within a certain value, the determination unit 132 determines that the convex mirror M107a on the left side faces left, and determines that the convex mirror M107b on the right side faces right.

[0101] The seventh method makes use of the fact that two convex mirrors are sometimes installed in pairs facing in opposite directions so that the vehicle can see both to the left and right.

[0102] According to the seventh determination method, the orientations of a plurality of convex mirrors can be determined simultaneously.

[0103] [Processing flow of the entire system] The determination device 10 may be a part of a danger prediction system used for automatic driving of a vehicle, driver assistance, etc. In that case, for example, the determination result of the direction of a curve mirror by the determination device 10 is used to determine an approaching object.

[0104] Fig. 18 is a flowchart of the entire determination process regarding an approaching object. As shown in Fig. 18, first, the danger prediction system detects the shape of a convex mirror from an image (step S11).

[0105] Next, the risk prediction system extracts edges of the detected shape of the convex mirror (step S12), and then extracts optical flow within the convex mirror (step S13).

[0106] Next, the risk prediction system determines the orientation of the convex mirror (step S14). Step S14 is executed by the determination device 10 of the present embodiment. For example, in step S14, any one of the determination methods described above is executed.

[0107] Furthermore, the danger prediction system determines an approaching object reflected in the curved mirror (step S15), and determines the direction of the approaching object (step S16).

[0108] As described above, the determination device 10 according to the embodiment includes the analysis unit 131 and the determination unit 132. The analysis unit 131 analyzes an image of a convex mirror captured from a vehicle. The determination unit 132 determines the orientation of the convex mirror based on information obtained by the analysis by the analysis unit 131.

[0109] In this way, the determination device 10 determines the orientation of the convex mirror in the image by using the analysis result of the image. Therefore, according to this embodiment, it is possible to accurately determine the orientation of the convex mirror in the image.

[0110] Further advantages and modifications may readily occur to those skilled in the art. Thus, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof. [Explanation of symbols]

[0111] 10 Judgment device 11 Input / output section 12 Storage section 13 Control section 131 Analysis Department 132 Judgment Department

Claims

1. An analysis unit that analyzes an image of a curve mirror taken from a vehicle; A determination unit that determines the orientation of the convex mirror based on the relationship between the center position of the mirror surface of the convex mirror obtained by the analysis by the analysis unit and the position of the support of the convex mirror; A determination device comprising:

2. The determination device according to claim 1, characterized in that the determination unit determines the orientation of the convex mirror based on the relationship between the center position of the mirror surface of the convex mirror and the position of a sign attached to a support pillar of the convex mirror.

3. The determination device according to claim 1 or 2, characterized in that the determination unit determines the orientation of the convex mirror based on the relationship between the center position of the mirror surface of the convex mirror and the position of the eaves area of ​​the convex mirror.

4. 3. The determination device according to claim 1, wherein the determination unit determines the orientation of the convex mirror based on an inclination of a center line of the mirror surface of the convex mirror.

5. The determination device according to claim 1 or 2, characterized in that the determination unit determines the orientation of the convex mirror based on a pattern of an image of an area of ​​the mirror surface of the convex mirror.

6. The determination device according to claim 1 or 2, characterized in that the determination unit determines the orientation of the curved mirror based on a direction indicated by a predetermined object reflected in the image.

7. The determination device described in claim 1 or 2, characterized in that when two curved mirrors that satisfy a predetermined positional relationship are reflected in the image, the determination unit determines one of the two curved mirrors to be facing left and the other to be facing right.

8. A determination method executed by a determination device, comprising: An analysis step of analyzing an image of a curve mirror taken from a vehicle; A determination step of determining the orientation of the convex mirror based on the relationship between the center position of the mirror surface of the convex mirror obtained by the analysis step and the position of the support of the convex mirror; A method for determining whether or not a particular

Citation Information

Patent Citations

  • Obstacle collision determination device and obstacle collision alarm system

    JP2007279805A

  • Object detection device

    JP2017097581A

  • Image recognition device, road reflector and image recognition method

    JP2019008701A