System and method for determining the position of a sign using a front camera

Abstract

The present invention relates to a system for determining the location of a sign using a front camera and a method for determining the location thereof. More specifically, the invention relates to a technology that, when the lane information of the lane in which the vehicle is traveling is relatively clear, can use the lane information to clearly determine whether a no-entry sign corresponds to the lane in which the vehicle is traveling or is a sign located outside the lane (e.g., the opposite lane) that can be ignored.

Description

Technology Field

[0001] The present invention relates to a system for determining the location of a sign using a front camera and a method for determining the location thereof. More specifically, the invention relates to a system for determining the location of a sign using a front camera and a method for determining the location thereof, which can determine the accurate location of a sign recognized in front of a vehicle and determine whether to provide information about the recognized sign to the driver of a moving vehicle. Background Technology

[0003] Generally, a front camera sensor mounted on the vehicle is used to recognize road traffic signs located on the road, and safe driving is encouraged by visually providing the driver with traffic-related information (warning signs, regulatory signs, directional signs, auxiliary signs, and road surface signs, etc.) based on the recognized road traffic signs.

[0004] Recently, in European regions (such as Germany), regulations are being established to prevent driving against the flow of traffic by placing "No Entry" signs on both sides of highway entrances to warn drivers. Specifically, warning technology is being commercialized that displays a notification on the vehicle's instrument cluster when identical "No Entry" signs are detected on both sides of the lane—specifically, in the forward left and forward right areas relative to the vehicle—to inform the driver that they are entering a restricted road. To explain further, since the purpose of these "No Entry" signs is to alert drivers of vehicles currently traveling in the wrong direction that they are doing so, the front of the sign is positioned facing the direction of travel in reverse, rather than the normal direction of travel. Consequently, vehicles traveling in the normal direction fail to recognize the front of the "No Entry" sign, while only vehicles traveling in the abnormal direction (the direction of travel in reverse) recognize it.

[0005] If it is a straight road, the problem described below will not occur, but as shown in Fig. 1, if the road on which the vehicle is traveling is a curved road, there is a problem in that a vehicle traveling in the opposite direction may misrecognize the front of the no-entry sign depending on the curvature.

[0006] In other words, even if the vehicle is not driving in the wrong direction but is simply traveling in the opposite lane, the front of a "no entry" sign placed in the opposite lane may be recognized, causing a warning that the vehicle is driving in the wrong direction even though it is traveling in the normal direction of travel.

[0008] In this regard, Korean published patent No. 10-2017-0060449 ("Entry Permissible Notification System and Method Using Sign Recognition") discloses a technology capable of recognizing one-way signs or no-entry signs from a front image acquired through a front camera mounted on a vehicle, informing the user that entry is permitted if the direction of the arrow on the one-way sign matches the direction of the vehicle's turn signal, warning that entry is prohibited if they do not match, and warning that entry is prohibited if the direction of the arrow on the no-entry sign matches the direction of the vehicle's turn signal. However, this technology also does not determine whether the sign is located on the road the vehicle is traveling on, and thus still contains the possibility of misrecognition, which could lead to misidentifying and providing guidance for a sign located on the opposite road. Prior art literature

[0010] Korean Published Patent Application No. 10-2017-0060449 (Publication Date: June 1, 2017) The problem to be solved

[0011] The present invention has been devised to solve the problems described above, and provides a system for determining the location of a sign using a front camera and a method for determining the location thereof, which determines the precise location of a sign recognized in front of a vehicle and determines whether to provide information about the recognized sign to the driver of a moving vehicle; thereby preventing a false warning that the vehicle is driving in the wrong direction even though it is simply driving in the opposite lane and not in a situation of driving in the wrong direction, by recognizing the front of a "no entry" sign placed in the opposite lane. means of solving the problem

[0013] A sign position determination system using a front camera according to the present invention for solving the problems described above preferably includes: a sign recognition unit that recognizes a sign located in front by analyzing image data acquired through a front camera sensor mounted on a vehicle; a lane analysis unit that analyzes information related to the lane in which the vehicle is currently driving using the image data; a lane prediction unit that analyzes information regarding the lateral separation distance of the lane in which the vehicle is currently driving, which is located on the same line as the recognized sign, based on the distance information between the recognized sign and the vehicle and the analyzed lane-related information; and a position determination unit that determines whether the recognized sign is located in the lane in which the vehicle is currently driving or is located outside the lane in which the vehicle is currently driving, using the analyzed lateral separation distance information of the lane.

[0014] Furthermore, the sign recognition unit recognizes two or more signs displaying the same regulatory information located ahead, and it is preferable that the sign be a no-entry sign.

[0015] Furthermore, it is desirable that the lane-related information includes information regarding the left lane and the right lane of the lane currently being driven relative to the vehicle.

[0016] Furthermore, the lateral separation distance information of the above lane is analyzed based on a preset front center point of the vehicle, and it is preferable to include a first lateral separation distance information, which is the lateral separation distance information between the left lane and the front center point, and a second lateral separation distance information, which is the lateral separation distance information between the right lane and the front center point.

[0017] Furthermore, it is preferable that the position determination unit includes an additional analysis unit that analyzes the image data and, when the front center point is located on the same line as the recognized sign, analyzes the separation distance information between each sign and the front center point; a first determination unit that determines that the recognized sign is located outside the lane in which the vehicle is currently driving, by using the two or more analyzed separation distance information and the analyzed lane's lateral separation distance information, if both of the two or more analyzed separation distance information are smaller than the first lateral separation distance information and less than a predetermined value; and a second determination unit that determines that the recognized sign is located outside the lane in which the vehicle is currently driving, if both of the two or more analyzed separation distance information and the analyzed lane's lateral separation distance information are larger than the second lateral separation distance information and more than a predetermined value.

[0018] Furthermore, it is preferable that the above-mentioned location determination unit further includes a warning output unit that outputs entry prohibition warning information through a display means mounted on the vehicle when, according to the determination result of the first determination unit or the second determination unit, the recognized sign is located in the lane in which the vehicle is currently driving.

[0020] A method for determining the position of a sign using a front camera, wherein each step is performed by a computational processing means to solve the problem described above, preferably comprises: a sign recognition step (S100) for recognizing a sign located in front by analyzing image data acquired through a front camera sensor mounted on a vehicle; a lane analysis step (S200) for analyzing information related to the lane in which the vehicle is currently driving using the acquired image data; a lane prediction step (S300) for analyzing lateral separation distance information of the lane in which the vehicle is currently driving that is on the same line as the recognized sign, based on the distance information between the sign recognized through the sign recognition step (S100) and the vehicle, using the lane-related information analyzed by the lane analysis step (S200); and a position determination step (S400) for determining whether the recognized sign is located in the lane in which the vehicle is currently driving or is located outside the lane in which the vehicle is currently driving, using the lateral separation distance information of the lane analyzed through the lane prediction step (S300).

[0021] Furthermore, the sign recognition step (S100) recognizes two or more signs displaying the same regulatory information located ahead, and it is preferable that the sign be a no-entry sign.

[0022] Furthermore, the lane analysis step (S200) preferably includes information related to the left lane and the right lane of the lane currently being driven relative to the vehicle as the lane-related information.

[0023] Furthermore, the lane prediction step (S300) analyzes the lateral separation distance information of the lane based on a preset front center point of the vehicle, and it is preferable that the lateral separation distance information of the lane includes a first lateral separation distance information, which is the lateral separation distance information between the left lane and the front center point, and a second lateral separation distance information, which is the lateral separation distance information between the right lane and the front center point.

[0024] Furthermore, the position determination step (S400) analyzes the acquired image data and, if the front center point is located on the same line as the recognized sign, performs an additional analysis step (S410) for analyzing the separation distance information between each sign and the front center point; a first determination step (S420) for determining that the recognized sign is located outside the lane in which the vehicle is currently driving, if, using the two or more separation distance information analyzed by the additional analysis step (S410) and the lane lateral separation distance information analyzed by the lane prediction step (S300), both of the two or more separation distance information are smaller than the first lateral separation distance information by a predetermined value or less; and if, using the two or more separation distance information analyzed by the additional analysis step (S410) and the lane lateral separation distance information analyzed by the lane prediction step (S300), both of the two or more separation distance information are larger than the second lateral separation distance information by a predetermined value or more, the recognized sign is the It is desirable to include a second judgment step (S430) that determines that the vehicle is located outside the lane it is currently driving in.

[0025] Furthermore, it is preferable that the above location determination step (S400) further includes a warning output step (S440) which outputs entry prohibition warning information through a display means mounted on the vehicle when, according to the result of the determination of the above first determination step (S420) or the above second determination step (S430), the recognized sign is located in the lane where the vehicle is currently driving.

[0026] Furthermore, the method for determining the location of a sign using the front camera described above further includes a sign analysis step (S10) in which, after performing the sign recognition step (S100), if it is difficult to analyze information related to the lane in which the vehicle is currently driving through the lane analysis step (S200), the acquired image data is analyzed to determine whether there are signs in both the right lane and the left lane centered on the lane, and a warning output step (S20) in which, according to the analysis result of the sign analysis step (S10), if signs are located in both the right lane and the left lane, no-entry warning information is output through a display means mounted on the vehicle, wherein the sign analysis step (S10) preferably analyzes the acquired image data to determine whether each recognized sign is located in the right area and the left area based on the front center point when the front center point is located on the same line as the recognized sign. Effects of the invention

[0028] According to the system and method for determining the location of a sign using a front camera according to the present invention as described above, when the lane information of the lane in which the vehicle is traveling is relatively clear, it is possible to clearly determine using the lane information whether a no-entry sign corresponds to the lane in which the vehicle is traveling or whether it is a sign outside the lane (e.g., the opposite lane) that can be ignored. This provides the advantage of strengthening the function of recognizing no-entry signs and displaying them on the vehicle. In other words, there is an advantage of improving the marketability of sign recognition technology without increasing costs.

[0029] In addition, it has the advantage of enabling the delivery of more accurate information to drivers, thereby more clearly guiding them toward safe driving, which is the primary function of signs. Brief explanation of the drawing

[0031] FIG. 1 is a diagram showing an example of misrecognition when determining the location of a sign using a front camera according to the prior art. FIG. 2 is a configuration example diagram showing a sign position determination system using a front camera according to an embodiment of the present invention, and FIG. 3 is an exemplary diagram illustrating a process by a system for determining the location of a sign using a front camera and a method for determining the location according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating a method for determining the location of a sign using a front camera according to an embodiment of the present invention. Specific details for implementing the invention

[0032] The objects, features, and advantages of the present invention described above will become more apparent through the following embodiments in connection with the accompanying drawings. The specific structural or functional descriptions below are merely illustrative for the purpose of explaining embodiments according to the concept of the present invention, and embodiments according to the concept of the present invention may be implemented in various forms and should not be interpreted as being limited to the embodiments described in this specification or application. Since embodiments according to the concept of the present invention may be subject to various modifications and may take various forms, specific embodiments are illustrated in the drawings and described in detail in this specification or application. However, this is not intended to limit embodiments according to the concept of the present invention to specific disclosed forms, and should be understood to include all modifications, equivalents, and substitutions that fall within the spirit and scope of the present invention. Terms such as "first" and / or "second" may be used to describe various components, but said components are not limited to said terms. The above terms may be used solely for the purpose of distinguishing one component from other components, for example, without departing from the scope of rights according to the concept of the present invention, such that the first component may be named the second component, and similarly, the second component may be named the first component. When it is stated that a component is connected to or coupled with another component, it should be understood that it may be directly connected to or coupled with that other component, or that there may be other components in between. On the other hand, when it is stated that a component is directly connected to or directly coupled with another component, it should be understood that there are no other components in between. Other expressions used to describe the relationship between components, such as "between" and "exactly between," or "adjacent to" and "directly adjacent to," should be interpreted in the same way.The terms used in this specification are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. Terms such as "include" or "have" in this specification are intended to indicate the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should not be understood as precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification. The invention will be described in detail below by describing preferred embodiments of the invention with reference to the accompanying drawings. Identical reference numerals in each drawing indicate identical components.

[0033] Furthermore, a system refers to a set of components, including devices, mechanisms, and means, that are organized and interact regularly to perform necessary functions.

[0035] As shown in Fig. 1, the 'no entry' sign located on both sides of the road to convey that the vehicle is in an abnormal direction of travel (reverse direction) rather than a normal direction of travel, is positioned such that the front of the sign must be visible to the vehicle traveling in reverse, and thus the front of the sign faces in the opposite direction to the normal direction of travel of the vehicle.

[0036] In other words, this means that there is a possibility that a vehicle traveling in the opposite lane in the normal direction of travel can recognize the front of the no-entry sign.

[0037] Accordingly, in fact, as illustrated in FIGS. 1 and 3, when the road on which the vehicle is traveling is a curved road, the front of a no-entry sign located in the opposite lane, which does not need to be recognized by a vehicle traveling normally, is recognized as being located in the left / right area in front of the vehicle, and a driving warning is output accordingly.

[0038] Accordingly, the system for determining the location of a sign using a front camera and the method for determining the location thereof, according to one embodiment of the present invention, relates to a technology for determining whether a "no entry" sign corresponds to the lane in which the vehicle is traveling, or whether it is a sign that can be ignored outside the lane (e.g., the opposite lane), by utilizing lane information when the lane information of the lane in which the vehicle is traveling is relatively clear. Through this, it is possible to convey more accurate information to the driver, thereby having the advantage of more clearly inducing safe driving, which is the inherent function of a sign.

[0039] FIG. 2 is a configuration example diagram showing a sign position determination system using a front camera according to an embodiment of the present invention, and with reference to FIG. 2, a sign position determination system using a front camera according to an embodiment of the present invention will be described in detail.

[0040] A system for determining the location of a sign using a front camera according to one embodiment of the present invention includes a sign recognition unit (100), a lane analysis unit (200), a lane prediction unit (300), and a location determination unit (400), as illustrated in FIG. 2. Each component is included in a single computational processing means configured to include a computer, etc., or in a plurality of computational processing means, and performs operations.

[0041] Let's take a closer look at each component.

[0042] It is preferable for the sign recognition unit (100) to recognize a sign located in front by analyzing image data obtained through a front camera sensor mounted on the vehicle. Since the technology of recognizing a sign located in front by analyzing image data using a front camera sensor mounted on the vehicle is a conventional technology as various advanced functions (e.g., ISLA, Intelligent Speed ​​Limit Assist, Intelligent Speed ​​Limit Assist, etc.) are applied to vehicles recently, a detailed explanation of the technology of recognizing a sign by analyzing image data is omitted. However, in the present invention, it is preferable to recognize two or more signs displaying the same regulatory information located in front, and more specifically, it is preferable that they be no-entry signs.

[0043] Due to the nature of the "No Entry" sign, unlike other road traffic signs, there is a clear danger when driving against the flow of traffic; therefore, it is placed on both sides of the road as a clear "warning" rather than a "cautionary driving" sign to warn drivers. Accordingly, the sign recognition unit (100) recognizes two or more signs displaying the same regulatory information located ahead.

[0044] It is preferable for the lane analysis unit (200) to analyze information related to the lane the vehicle is currently driving on by using image data obtained through a front camera sensor mounted on the vehicle (vehicle).

[0045] In this case, regarding lane-related information, it is desirable to include information regarding the left lane and the right lane of the lane currently being driven, based on the vehicle.

[0046] More specifically, since lane-related information is analyzed using image data, the analysis is performed based on the camera position derived from the image data. Since the front camera sensor is typically located in the center of the vehicle's windshield, it is acceptable to interpret the camera position as the front center of the vehicle. Based on this, it is desirable to include at least one of the following as left lane-related information: distance information between the left lane and the camera (camera position) (C0_L), heading angle between the left lane and the camera (C1_L), curvature information of the left lane (C2_L), and rate of change of curvature of the left lane (C3_L). The same applies to right lane-related information. That is, it is desirable to include at least one of the following as right lane-related information: distance information between the right lane and the camera (C0_R), heading angle between the right lane and the camera (C1_R), curvature information of the right lane (C2_R), and rate of change of curvature of the right lane (C3_R). Analyzing the distance between the camera and the lane, the heading angle between the camera and the lane, and curvature can be measured through the lane area included in the image data.

[0047] It is preferable for the lane prediction unit (300) to analyze the lateral separation distance information of the currently driving lane located on the same line as the recognized sign, based on the distance information between the sign and the vehicle (X in FIG. 3) recognized through the sign recognition unit (100) and using the lane-related information analyzed by the lane analysis unit (200).

[0048] That is, the lateral separation distance information of the lane is analyzed based on the camera position, which is the pre-set front center point of the vehicle, through the lane prediction unit (300). As described above, since the analysis is performed using image data, the camera position that acquired the corresponding image data is interpreted as the front center of the vehicle and the analysis is performed.

[0049] In detail, the lane prediction unit (300) preferably analyzes the lateral separation distance information of the lane and the forward center point (camera position) located X m ahead of the lane currently being driven relative to the vehicle, which is located on the same line as the position of the recognized sign as illustrated in FIG. 3. To do this, lane-related information analyzed by the lane analysis unit (200) is used. To summarize briefly, the lateral separation distance of the lane X m ahead of the lane currently being driven is predicted, and the reference point for calculating the separation distance is preferably the forward center point X m ahead.

[0050] At this time, since the lane-related information includes left lane-related information and right lane-related information, it is desirable that the lane-lateral separation distance information also includes a first lateral separation distance information, which is the lateral separation distance information between the left lane and the front center point, and a second lateral separation distance information, which is the lateral separation distance information between the right lane and the front center point.

[0051] Based on the technical details described above, the first lateral separation distance information (y_L) and the second lateral separation distance information (y_R) can be analyzed using the following Equation 1. As such, calculating the separation distance between the lane ahead and the vehicle's front center point using the following Equation 1 is a matter of conventional technology. For example, in the case of an autonomous vehicle, since the vehicle's turning angle must be determined by identifying the condition of the lane ahead, it is necessary to analyze the lateral separation distance information of the lane ahead in this manner.

[0052]

[0053] It is preferable for the position determination unit (400) to determine whether the sign recognized by the sign recognition unit (100) is located in the lane where the vehicle is currently driving, or whether the sign recognized by the sign recognition unit (100) is located outside the lane where the vehicle is currently driving, by using the lane prediction unit (300) to determine the lateral separation distance information (y_L, y_R) of the lane.

[0054] To explain in detail, the sign recognition unit (100) determines whether the no-entry sign recognized is located in the current lane (i.e., whether the vehicle is driving in the wrong direction) or is located in the opposite lane (i.e., whether the vehicle is driving in the normal direction, but the sign is located in the opposite lane, and due to the characteristics of the no-entry sign to prevent driving in the wrong direction, the front is formed opposite to the normal direction, so the front of the sign is inevitably recognized).

[0055] To this end, the position determination unit (400) includes an additional analysis unit (410), a first determination unit (420), and a second determination unit (430) as shown in FIG. 2.

[0056] The additional analysis unit (410) analyzes image data obtained through a front camera sensor mounted on a vehicle, and if the front center point is located on the same line as the recognized sign, it is desirable to analyze the distance information between each sign and the front center point. That is, as shown in FIG. 3, the lateral distance information of the recognized sign is analyzed, and the reference point for calculating the distance is the front center point X in front. The distance information analyzed by the additional analysis unit (410) includes the lateral distance information (y2) of the sign corresponding to the left area relative to the front center point and the lateral distance information (y1) of the sign corresponding to the right area relative to the front center point.

[0057] The first judgment unit (420) uses two or more separation distance information (y1, y2) analyzed through the additional analysis unit (410) and the lane lateral separation distance information (y_L, y_R) analyzed through the lane prediction unit (300). If both of the analyzed separation distance information (y1, y2) are smaller than the first lateral separation distance information (y_L) by a predetermined value, it is desirable to determine that the sign recognized by the sign recognition unit (100) is located outside the lane in which the vehicle is currently driving. That is, if both of the separation distance information (y1, y2) are smaller than the first lateral separation distance information (y_L) by a predetermined value, it means that all of the sign recognized by the sign recognition unit (100) are located to the left of the lane in which the vehicle is currently driving, so it is acceptable to ignore the sign. In this case, it is preferable that the preset value be set using the distance between the lane the vehicle is currently driving in and the opposite lane, and this can also be set by analyzing image data acquired through the front camera sensor mounted on the vehicle.

[0058] The second judgment unit (430) uses two or more separation distance information (y1, y2) analyzed through the additional analysis unit (410) and the lane lateral separation distance information (y_L, y_R) analyzed through the lane prediction unit (300). If both of the analyzed separation distance information (y1, y2) are greater than a predetermined value set beforehand than the second lateral separation distance information (y_R), it is desirable to determine that the sign recognized by the sign recognition unit (100) is located outside the lane in which the vehicle is currently driving. That is, if both of the separation distance information (y1, y2) are greater than or equal to a predetermined value set beforehand than the second lateral separation distance information (y_R), it means that all of the sign recognized by the sign recognition unit (100) are located to the right of the lane in which the vehicle is currently driving, so it is acceptable to ignore the sign. In this case, it is desirable that the preset value is also set using the distance between the lane the vehicle is currently driving in and the opposite lane, and this can also be set by analyzing image data acquired through the front camera sensor mounted on the vehicle.

[0059] It is desirable for the location determination unit (400) to output a warning to the driver to indicate that the vehicle has entered a prohibited area when it is determined that the recognized sign is located in the lane in which the vehicle is currently driving, based on the result of the determination of the first determination unit (420) or the second determination unit (430). To this end, the location determination unit (400) further includes a warning output unit (440) as shown in FIG. 2.

[0060] It is preferable for the warning output unit (440) to output entry prohibition warning information through a display means mounted on the vehicle when, based on the judgment result of the first judgment unit (420) or the second judgment unit (430), it is determined that the recognized sign is located in the lane where the vehicle is currently driving.

[0061] The system for determining the position of a sign using a front camera according to one embodiment of the present invention can perform operations when lane-related information can be analyzed through the lane analysis unit (200), that is, when the lane information is relatively clear. However, in actual driving environments, there are many situations where it is difficult to analyze lane-related information depending on various road conditions.

[0062] In preparation for this, the sign position determination system using a front camera according to one embodiment of the present invention determines whether a sign recognized by the sign recognition unit (100) is located on each side based on the vehicle's front center point when it is impossible to analyze lane-related information by the lane analysis unit (200), that is, when it is difficult to analyze both left lane-related information and right lane-related information. In other words, it is preferable to determine whether a sign recognized by the sign recognition unit (100) is located in both the right area and the left area, and if it is located there, to output a no-entry warning information through a display means mounted on the vehicle.

[0063] However, in this case, since the analysis result is based on unclear information, it is advisable to output an entry prohibition caution rather than a clear entry prohibition warning. Furthermore, because the entry prohibition caution requires the driver to personally determine whether an actual situation of driving against the flow of traffic is occurring, it may be information that demands even greater caution; therefore, it is advisable to interpret it as a different type of warning rather than a warning of a lower grade.

[0065] FIG. 4 is a sequence example diagram illustrating a method for determining the location of a sign using a front camera according to an embodiment of the present invention. With reference to FIG. 4, the method for determining the location of a sign using a front camera according to an embodiment of the present invention includes a sign recognition step (S100), a lane analysis step (S200), a lane prediction step (S300), and a location determination step (S400). Each step is performed using a system for determining the location of a sign using a front camera, which is executed by a computational processing means configured to include a computer, etc.

[0066] Let's take a closer look at each step.

[0067] The sign recognition step (S100) involves the sign recognition unit (100) analyzing image data acquired through a front camera sensor mounted on the vehicle to recognize a sign located in front. Since the technology of recognizing a sign located in front by analyzing image data using a front camera sensor mounted on the vehicle is a conventional technology due to the recent application of various advanced functions (e.g., ISLA, Intelligent Speed ​​Limit Assist, etc.) to vehicles, a detailed explanation of the technology for recognizing a sign by analyzing image data is omitted. However, in the present invention, it is preferable to recognize two or more signs displaying the same regulatory information located in front, and more specifically, it is preferable that they be no-entry signs.

[0068] Due to their nature, unlike other road traffic signs, no-entry signs pose a clear danger when driving against the flow of traffic; therefore, they are placed on both sides of the road as a clear 'warning' rather than a 'cautionary driving' sign to warn drivers. Accordingly, the sign recognition step (S100) recognizes two or more signs displaying the same regulatory information located ahead.

[0069] The lane analysis step (S200) involves the lane analysis unit (200) analyzing information related to the lane in which the vehicle is currently driving by using image data obtained through a front camera sensor mounted on the vehicle (vehicle). At this time, it is preferable that the lane-related information includes information related to the left lane and information related to the right lane of the lane in which the vehicle is currently driving relative to the vehicle.

[0070] More specifically, since lane-related information is analyzed using image data, the analysis is performed based on the camera position derived from the image data. Since the front camera sensor is typically located in the center of the vehicle's windshield, it is acceptable to interpret the camera position as the front center of the vehicle. Based on this, it is desirable to include at least one of the following as left lane-related information: distance information between the left lane and the camera (camera position) (C0_L), heading angle between the left lane and the camera (C1_L), curvature information of the left lane (C2_L), and rate of change of curvature of the left lane (C3_L). The same applies to right lane-related information. That is, it is desirable to include at least one of the following as right lane-related information: distance information between the right lane and the camera (C0_R), heading angle between the right lane and the camera (C1_R), curvature information of the right lane (C2_R), and rate of change of curvature of the right lane (C3_R). Analyzing the distance between the camera and the lane, the heading angle between the camera and the lane, and curvature can be measured through the lane area included in the image data.

[0071] The lane prediction step (S300) analyzes the lateral separation distance information of the currently driving lane located on the same line as the recognized sign, based on the distance information (X in FIG. 3) between the sign and the vehicle recognized through the sign recognition step (S100) in the lane prediction unit (300), and using the lane-related information analyzed by the lane analysis step (S200).

[0072] That is, the lateral separation distance information of the lane is analyzed based on the camera position, which is the pre-set front center point of the vehicle. As described above, since the analysis is performed using image data, the camera position that acquired the corresponding image data is interpreted as the front center of the vehicle to perform the analysis.

[0073] In detail, the lane prediction step (S300) analyzes the lateral separation distance information of the lane and the forward center point (camera position) located X m ahead of the lane currently being driven relative to the vehicle, which is located on the same line as the position of the recognized sign as illustrated in FIG. 3. To do this, lane-related information analyzed by the lane analysis step (S200) is used. To summarize briefly, the lateral separation distance of the lane X m ahead of the lane currently being driven is predicted, and it is preferable that the reference point for calculating the separation distance is the forward center point X m ahead.

[0074] At this time, since the lane-related information includes left lane-related information and right lane-related information, it is desirable that the lane-lateral separation distance information also includes a first lateral separation distance information, which is the lateral separation distance information between the left lane and the front center point, and a second lateral separation distance information, which is the lateral separation distance information between the right lane and the front center point.

[0075] Based on the technical details described above, the first lateral separation distance information (y_L) and the second lateral separation distance information (y_R) can be analyzed using the above Equation 1. As such, calculating the separation distance between the lane ahead and the vehicle's front center point using the following Equation 1 is considered a standard technique. For example, in the case of an autonomous vehicle, since the vehicle's turning angle must be determined by identifying the condition of the lane ahead, it is necessary to analyze the lateral separation distance information of the lane ahead in this manner.

[0076] The position determination step (S400) uses the lateral separation distance information of the lane analyzed through the lane prediction step (S300) in the position determination unit (400) to determine whether the sign recognized through the sign recognition step (S100) is located in the lane where the vehicle is currently driving, or whether the sign recognized through the sign recognition step (S100) is located outside the lane where the vehicle is currently driving.

[0077] To explain in detail, it is determined whether the no-entry sign recognized in the sign recognition step (S100) is located in the current lane (i.e., whether the vehicle is driving in the wrong direction) or in the opposite lane (i.e., whether the vehicle is driving in the normal direction, but the sign is located in the opposite lane, and due to the characteristics of the no-entry sign designed to prevent driving in the wrong direction, the front of the sign is formed opposite to the normal direction, so the front of the sign is inevitably recognized).

[0078] To this end, the position determination step (S400) includes an additional analysis step (S410), a first determination step (S420), and a second determination step (S430), as illustrated in FIG. 4.

[0079] The additional analysis step (S410) analyzes image data obtained through a front camera sensor mounted on a vehicle, and if the front center point is located on the same line as the recognized sign, it analyzes the separation distance information between each sign and the front center point. That is, as illustrated in FIG. 3, the lateral separation distance information of the sign recognized by the sign recognition step (S100) is analyzed, and it is preferable that the reference point for calculating the separation distance is the front center point X in front. The separation distance information analyzed by the additional analysis step (S410) includes the lateral separation distance information (y2) of the sign corresponding to the left area relative to the front center point and the lateral separation distance information (y1) of the sign corresponding to the right area relative to the front center point.

[0080] The first judgment step (S420) uses two or more separation distance information analyzed by the additional analysis step (S410) and the lane lateral separation distance information (y_L, y_R) analyzed by the lane prediction step (S300). If both of the analyzed separation distance information (y1, y2) are smaller than the first lateral separation distance information (y_L) by a predetermined value, it is determined that the sign recognized by the sign recognition step (S100) is located outside the lane in which the vehicle is currently driving. That is, if both of the separation distance information (y1, y2) are smaller than the first lateral separation distance information (y_L) by a predetermined value, it means that all of the sign recognized by the sign recognition unit (100) are located to the left of the lane in which the vehicle is currently driving, and therefore it is acceptable to ignore the sign. In this case, it is preferable that the preset value be set using the distance between the lane the vehicle is currently driving in and the opposite lane, and this can also be set by analyzing image data acquired through the front camera sensor mounted on the vehicle.

[0081] The second judgment step (S430) uses two or more separation distance information analyzed by the additional analysis step (S410) and the lane lateral separation distance information (y_L, y_R) analyzed by the lane prediction step (S300). If both of the analyzed separation distance information (y1, y2) are greater than a predetermined value set beforehand than the second lateral separation distance information (y_R), it is determined that the sign recognized by the sign recognition unit (100) is located outside the lane in which the vehicle is currently driving. That is, if both of the separation distance information (y1, y2) are greater than or equal to a predetermined value set beforehand than the second lateral separation distance information (y_R), it means that all of the sign recognized by the sign recognition unit (100) are located to the right of the lane in which the vehicle is currently driving, and therefore it is acceptable to ignore the sign. In this case, it is desirable that the preset value is also set using the distance between the lane the vehicle is currently driving in and the opposite lane, and this can also be set by analyzing image data acquired through the front camera sensor mounted on the vehicle.

[0082] The position determination step (S400) further includes a warning output step (S440) as illustrated in FIG. 4.

[0083] The warning output step (S440) outputs entry prohibition warning information through a display means mounted on the vehicle when, depending on the result of the judgment of the first judgment step (S420) or the second judgment step (S430), it is determined that the sign recognized by the sign recognition step (S100) is located in the lane where the vehicle is currently driving.

[0084] In addition, the method for determining the location of a sign using a front camera according to one embodiment of the present invention does not output a warning to a display means mounted on a vehicle when, depending on the result of the first determination step (S420) or the second determination step (S430), it is determined that the sign recognized by the sign recognition step (S100) is located in the opposite lane of the lane in which the vehicle is currently driving.

[0085] The method for determining the location of a sign using a front camera according to one embodiment of the present invention can be performed when lane-related information can be analyzed through the lane analysis step (S200), that is, when the self-lane information is relatively clear. However, in actual driving environments, there are many situations where it is difficult to analyze lane-related information depending on various road conditions.

[0086] In preparation for this, a method for determining the location of a sign using a front camera according to one embodiment of the present invention further includes a sign analysis step (S10) and a warning output step (S20), as illustrated in FIG. 4.

[0087] The sign analysis step (S10) determines whether the sign recognized through the sign recognition step (S100) is located on each side relative to the vehicle's front center point when it is impossible to analyze lane-related information through the lane analysis step (S200), that is, when it is difficult to analyze both left lane-related information and right lane-related information. In other words, it determines whether the sign recognized through the sign recognition step (S100) is located in both the right area and the left area.

[0088] The warning output step (S20) outputs entry prohibition warning information through a display means mounted on the vehicle when, based on the analysis result of the sign analysis step (S10), signs are located in both the right and left lanes. More preferably, in this case, since the analysis result is output based on unclear information, it is desirable to output entry prohibition caution information that is different from the clear entry prohibition warning information. Furthermore, since the entry prohibition caution information requires the driver to personally determine whether or not there is an actual situation of driving against the flow of traffic, it may be information that requires even greater caution; therefore, it is desirable to interpret it as a different type of warning rather than a warning of a lower grade.

[0090] The above-described invention can be implemented as computer-readable code on a medium on which a program is recorded. A computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and also include implementation in the form of a carrier wave (e.g., transmission via the Internet). Furthermore, the computer may include a sign position determination system using the front camera of the present invention and a determination method.

[0092] Although preferred embodiments of the present invention have been described above, the embodiments disclosed in the present invention are intended only to illustrate, not to limit, the technical scope of the present invention. Accordingly, the technical scope of the present invention includes not only each disclosed embodiment but also combinations of the disclosed embodiments, and furthermore, the scope of the technical scope of the present invention is not limited by such embodiments. In addition, a person skilled in the art to which the present invention pertains can make numerous changes and modifications to the present invention without departing from the spirit and scope of the appended claims, and all such appropriate changes and modifications should be deemed to fall within the scope of the present invention as equivalents. Explanation of the symbols

[0094] 100 : Sign recognition unit 200 : Lane Analysis Department 300 : Lane prediction section 400 : Location determination unit 410 : Additional Analysis Section 420 : First Judgment Division 430 : Second Judgment Division 440 : Warning Output Section

Claims

Claim 1 A sign position determination system using a front camera, comprising: a sign recognition unit that recognizes a sign located in front by analyzing image data acquired through a front camera sensor mounted on a vehicle; a lane analysis unit that analyzes information related to the lane in which the vehicle is currently driving using the image data; a lane prediction unit that analyzes lateral separation distance information of the lane in which the vehicle is currently driving located on the same line as the recognized sign using the analyzed lane-related information based on distance information between the recognized sign and the vehicle; and a position determination unit that determines whether the recognized sign is located in the lane in which the vehicle is currently driving or is located outside the lane in which the vehicle is currently driving using the analyzed lateral separation distance information of the lane. Claim 2 In claim 1, the sign recognition unit recognizes two or more signs displaying the same regulatory information located in front, and the sign is a no-entry sign, a sign position determination system using a front camera. Claim 3 In paragraph 2, the lane-related information includes left lane-related information and right lane-related information of the lane currently being driven relative to the vehicle, a sign position determination system using a front camera. Claim 4 A system for determining the position of a sign using a front camera, wherein, in paragraph 3, the lateral separation distance information of the lane is analyzed based on a preset front center point of the vehicle, and includes a first lateral separation distance information which is lateral separation distance information between the left lane and the front center point, and a second lateral separation distance information which is lateral separation distance information between the right lane and the front center point. Claim 5 In claim 4, the position determination unit analyzes the image data and, if the front center point is located on the same line as the recognized sign, an additional analysis unit analyzes the separation distance information between each sign and the front center point; a first determination unit uses the two or more analyzed separation distance information and the analyzed lane's lateral separation distance information to determine that the recognized sign is located outside the lane in which the vehicle is currently driving if both of the two or more analyzed separation distance information are smaller than the first lateral separation distance information by a predetermined value; and a second determination unit uses the two or more analyzed separation distance information and the analyzed lane's lateral separation distance information to determine that the recognized sign is located outside the lane in which the vehicle is currently driving if both of the two or more analyzed separation distance information are larger than the second lateral separation distance information by a predetermined value. Claim 6 In claim 5, the position determination unit further comprises a warning output unit that outputs no-entry warning information through a display means mounted on the vehicle when the recognized sign is located in the lane in which the vehicle is currently driving, according to the determination result of the first determination unit or the second determination unit. Claim 7 A method for determining the position of a sign using a front camera, wherein each step is performed by a computational processing means, comprising: a sign recognition step (S100) for recognizing a sign located in front by analyzing image data acquired through a front camera sensor mounted on a vehicle; a lane analysis step (S200) for analyzing information related to the lane in which the vehicle is currently driving using the acquired image data; a lane prediction step (S300) for analyzing lateral separation distance information of the lane in which the vehicle is currently driving, which is located on the same line as the recognized sign, using the lane-related information analyzed by the lane analysis step (S200), based on the distance information between the sign recognized through the sign recognition step (S100) and the vehicle; and a position determination step (S400) for determining whether the recognized sign is located in the lane in which the vehicle is currently driving or is located outside the lane in which the vehicle is currently driving, using the lateral separation distance information of the lane analyzed through the lane prediction step (S300). Claim 8 In claim 7, the sign recognition step (S100) recognizes two or more signs indicating the same regulatory matters located in front, and the sign is a no-entry sign, a method for determining the location of a sign using a front camera. Claim 9 In claim 8, the lane analysis step (S200) includes left lane related information and right lane related information of the lane currently being driven relative to the vehicle as lane related information, a method for determining the position of a sign using a front camera. Claim 10 In claim 9, the lane prediction step (S300) analyzes lateral separation distance information of the lane based on a preset front center point of the vehicle, wherein the lateral separation distance information of the lane includes a first lateral separation distance information which is lateral separation distance information between the left lane and the front center point, and a second lateral separation distance information which is lateral separation distance information between the right lane and the front center point. A method for determining the position of a sign using a front camera. Claim 11 In claim 10, the position determination step (S400) analyzes the acquired image data and, if the front center point is located on the same line as the recognized sign, performs an additional analysis step (S410) for analyzing distance information between each sign and the front center point; and, if the two or more distance information analyzed by the additional analysis step (S410) and the lateral distance information of the lane analyzed by the lane prediction step (S300) are both smaller than the first lateral distance information by a predetermined value, determines that the recognized sign is located outside the lane in which the vehicle is currently driving. A method for determining the position of a sign using a front camera, comprising: a second determination step (S430) in which, using two or more separation distance information analyzed by the additional analysis step (S410) and lateral separation distance information of the lane analyzed by the lane prediction step (S300), if both of the two or more separation distance information are greater than a predetermined value set above than the second lateral separation distance information, the recognized sign is determined to be located outside the lane in which the vehicle is currently driving. Claim 12 A method for determining the location of a sign using a front camera, wherein, in the above-mentioned claim 11, the location determination step (S400) further comprises a warning output step (S440) which outputs no-entry warning information through a display means mounted on the vehicle when the recognized sign is located in the lane in which the vehicle is currently driving, depending on the result of the determination of the first determination step (S420) or the second determination step (S430). Claim 13 In the above 8th claim, the method for determining the location of a sign using the front camera further comprises: a sign analysis step (S10) in which, after performing the sign recognition step (S100), if it is difficult to analyze information related to the lane in which the vehicle is currently driving through the lane analysis step (S200), the acquired image data is analyzed to determine whether there are signboards in both the right lane and the left lane centered on the lane; and a warning output step (S20) in which, according to the analysis result of the sign analysis step (S10), if signboards are located in both the right lane and the left lane, no-entry warning information is output through a display means mounted on the vehicle; wherein the sign analysis step (S10) analyzes the acquired image data to determine whether each recognized signboard is located in the right area and the left area based on the front center point when the front center point is located on the same line as the recognized signboard.

Images