METHOD, SYSTEM AND COMPUTER PROGRAM PRODUCT FOR PROVIDING DRIVING ASSISTANCE

Abstract

A method for providing a driver assistance system, the method is implemented using a driver assistance system (1) comprising a processor (14) and an image acquisition unit (11), the method comprising the following steps a) Control of the image acquisition unit (11) by the processor (14) to continuously acquire images of the vehicle's surroundings; b) Performing an image processing procedure by the processor (14) on at least one of the images captured by the image acquisition unit (11); c) Determine, by the processor (14), whether a traffic sign displaying a message has been detected in at least one of the images; d) if the finding in step c) is positive, determine, by the processor (14), whether additional information relating to the traffic sign is detected that indicates a state associated with the message; e) Determine, by the processor (14), whether an alarm action needs to be performed based on at least the message displayed by the traffic sign and a current state associated with the vehicle, the current state comprising a current time, current weather, movement of the vehicle or any combination thereof, and f) if the determination in step e) is positive, execution of the alarm action by the processor (14), characterized by, that the procedure between steps d) and e) further includes: g) if in at least one of the images in step c) a flat surface comprising a group of traffic signs is detected and additional information associated with one of the traffic signs is detected in relation to one of the traffic signs in step d), selection of one of the traffic signs as a traffic sign to be processed; where step e) is carried out in relation to the traffic sign to be processed, and that step g) further includes: Select one of the traffic signs with the additional information as the primary traffic sign; Determine whether at least part of the current state matches at least part of the future state; If the assessment is positive, carry out step e) with regard to the primary traffic sign that serves as the traffic sign to be processed; and If the finding is negative, carry out step e) with regard to the other traffic sign that serves as the traffic sign to be processed.

Description

[0001] The disclosure relates to a method, a system, and a computer program product for providing driver assistance.

[0002] Conventionally, traffic signs are regularly erected at the edges of roads and / or across roads to provide various pieces of information or instructions to road users (drivers, pedestrians, etc.). Traffic signs can be categorized into different types; for example, Annex 1 of the Vienna Convention on Road Signs and Signals defines eight categories of signs. Different countries may adopt different types of traffic sign conventions.

[0003] It should be noted that reading the various types of traffic signs can be difficult and distracting for some road users, potentially increasing the risks for these users on the road. Methods for recognizing traffic signs are known from US 2014 / 0119605 A1, US 2017 / 0017849 A1 and DE 10 2017 215 138 A1.

[0004] One objective of the disclosure is to provide a process that makes driver assistance available.

[0005] According to the disclosure, the method is implemented using a driver assistance system comprising a processor and an image acquisition unit. The method includes: a) Control of the image acquisition unit by the processor to continuously capture images of the vehicle's surroundings; b) Performing an image processing procedure by the processor on at least one of the images captured by the image acquisition unit; c) Determine, by the processor (14), whether a traffic sign displaying a message has been detected in at least one of the images; d) if the finding in step c) is positive, determine, by the processor, whether additional information relating to the traffic sign is detected that indicates a state associated with the message; e) Determine, by the processor, whether an alarm action must be performed based on at least the message displayed by the traffic sign and a current state associated with the vehicle, the current state comprising a current time, current weather, movement of the vehicle, or any combination thereof; and f) If the determination in step e) is positive, the processor executes the alarm action. Between steps d) and e), the procedure further includes: g) if in at least one of the images in step c) a flat surface comprising a group of traffic signs is detected and additional information associated with one of the traffic signs is detected in relation to one of the traffic signs in step d), selection of one of the traffic signs as a traffic sign to be processed; where step e) is carried out in relation to the traffic sign to be processed, and that step g) further includes: Select one of the traffic signs with the additional information as the primary traffic sign; Determine whether at least part of the current state matches at least part of the future state; If the assessment is positive, carry out step e) with regard to the primary traffic sign that serves as the traffic sign to be processed; and If the finding is negative, carry out step e) with regard to the other traffic sign that serves as the traffic sign to be processed.

[0006] Another goal of the revelation is to provide a system capable of implementing the aforementioned procedure.

[0007] According to one embodiment of the disclosure, the driver assistance system comprises a processor and an image acquisition unit connected to the processor. The processor is programmed to: to control the image acquisition unit in order to continuously capture images of the vehicle's surroundings; to perform an image processing procedure on at least one of the images captured by the image acquisition unit; to determine whether a traffic sign displaying a message is recognized in at least one of the images; If the finding is positive, determine whether additional information relating to the traffic sign is detected that indicates a condition associated with the message; to determine whether an alarm action is to be performed, based on at least the message displayed by the traffic sign and a current state associated with the vehicle, the current state comprising a current time, current weather, vehicle movement, or any combination thereof; and if the determination is positive, to perform the alarm action. The processor is further programmed to perform the following actions before determining whether an alarm action should be performed: to select one of the traffic signs as a traffic sign to be processed if at least one of the images detects a flat surface encompassing a group of traffic signs and additional information associated with one of the traffic signs is detected in relation to that one of the traffic signs; wherein the determination of whether an alarm action is to be carried out is performed by the said processor (14) with regard to the traffic sign to be processed; where, when selecting one of the traffic signs as the traffic sign to be processed, the aforementioned processor (14) is further programmed to Select one of the traffic signs with the additional information as the primary traffic sign; to determine whether at least part of the current state matches at least part of the state; If the assessment is positive, to determine whether the alarm action should be carried out in relation to the primary traffic sign that serves as the traffic sign to be processed; and If the determination is negative, it must be determined whether the alarm action should be carried out with regard to the other traffic sign that serves as the traffic sign to be processed.

[0008] Another goal of the disclosure is to provide a computer program product capable of implementing the aforementioned procedure.

[0009] According to one embodiment of the disclosure, the computer program product comprises instructions which, when executed by a processor of an in-vehicle driver assistance system, cause the processor to perform steps of the above-mentioned procedure.

[0010] Further features and advantages of the disclosure will become clear in the following detailed description of the embodiments with reference to the accompanying drawings, of which: Fig. 1 is a block diagram representing a driver assistance system according to an embodiment of the disclosure; Fig. 2 is a flowchart illustrating the steps of a method for providing driver assistance according to one embodiment of the disclosure; Fig. 3 is a schematic representation of an exemplary traffic sign; Fig. 4 is a flowchart illustrating the steps of a method for providing driver assistance according to one embodiment of the disclosure; and Fig. 5 is a schematic representation of an exemplary set of traffic signs.

[0011] Before describing the revelation in more detail, it should be noted that, where deemed appropriate, reference signs or terminal parts of reference signs have been repeated between the figures to indicate corresponding or analogous elements which may optionally have similar characteristics.

[0012] Throughout the entire disclosure, the term “connected to” can refer to a direct connection between a multitude of electrical devices / equipment / accessories via an electrically conductive material (e.g., an electrical wire), an indirect connection between two electrical devices / equipment / accessories via one or more other devices / equipment / accessories, or wireless communication.

[0013] Fig. Figure 1 is a block diagram representing a driver assistance system 1 according to an embodiment of the disclosure. In this embodiment, the driver assistance system 1 is arranged in a vehicle (not shown in the drawings) and comprises an image acquisition unit 11, a data storage device 12, an output unit 13, a communication unit 15, and a processor 14, which is electrically connected to the image acquisition unit 11, the data storage device 12, the output unit 13, and the communication unit 15.

[0014] The image acquisition unit 11 can be embodied by a driving video recorder (DVR) with a wide-angle lens capable of covering a viewing angle of at least 120 degrees and arranged in the vehicle so that it faces outwards to capture images of the vehicle's surroundings in directions outside the vehicle. In some embodiments, the image acquisition unit 11 can comprise a plurality of DVRs facing in different directions and thus capable of covering the vehicle's surroundings in all directions. In some embodiments, the image acquisition unit 11 can be embodied by other types of digital image acquisition devices capable of capturing images, such as a still camera with a burst mode (also known as burst mode) to rapidly capture a large number of images in succession.

[0015] Data storage 12 can be implemented as flash memory, a hard disk, a solid-state drive (SSD), or other types of non-transitory storage media. Data storage 12 stores a software application, a neural network model for dynamic image recognition, and an optical character recognition (OCR) model. The neural network model for dynamic image recognition can be implemented as a convolutional neural network (CNN).

[0016] The software application contains instructions which, when executed by processor 14, cause processor 14 to perform a series of functions, as described in the following paragraphs. In some embodiments, the content stored in data memory 12 (i.e., the software application, the neural network model for dynamic image recognition, and the OCR model) may be packaged as a computer program product. Dynamic image recognition includes object identification.

[0017] During operation, the processor 14, which runs the software application, can be configured to identify a traffic sign in an image and to recognize one or more characters (e.g., word characters (e.g., Chinese characters), letters, numbers, symbols, etc.) within and / or near the traffic sign. Furthermore, the processor 14 can be configured to determine a traffic sign category based on, for example, the shape and one or more colors of the traffic sign, the characters within and / or near the traffic sign, etc.

[0018] It should be noted that the recognition of traffic signs and symbols using the neural network model for dynamic image recognition and the OCR model are well known in related technical fields, and the details of this are omitted here for the sake of brevity.

[0019] The output unit 13 can be represented by a screen, an audio output component, or a combination thereof.

[0020] The processor 14 can comprise, but is not limited to, a single-core processor, a multi-core processor, a dual-core mobile processor, a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), and / or a system-on-a-chip (SoC). In this embodiment, the data storage 12 and the processor 14 can be integrated in the form of an in-vehicle computer or a remote server that is in signal communication with the image acquisition unit 11 and the output unit 13.

[0021] The communication unit 15 may include at least one radiofrequency integrated circuit (RFIC), a short-range wireless communication module supporting a short-range wireless communication network using wireless technology such as Bluetooth® and / or Wi-Fi, etc., or a mobile communication module supporting telecommunications using Long-Term Evolution (LTE), third generation (3G) and / or fifth generation (5G) wireless mobile telecommunications technology and / or similar.

[0022] In some embodiments, the driver assistance system 1 can be embodied by an electronic device (e.g., a smartphone, a dashboard camera, etc.).

[0023] Fig. Figure 2 is a flowchart illustrating the steps of a method for providing driver assistance according to an embodiment of the disclosure. In this embodiment, the method is implemented by the driver assistance system 1, as shown in Figure 2. Fig. 1 described, implemented.

[0024] During operation, when the vehicle user (e.g., a driver) is driving the vehicle on the road, he / she can operate the driver assistance system 1 to run the software application. In response, the processor 14 is programmed to control the components of the driver assistance system 1 to perform the steps described below.

[0025] In step S11, the processor 14 controls the image acquisition unit 11 to continuously capture images of the vehicle's surroundings. It should be noted that in some embodiments, the image acquisition unit 11 can be controlled to continuously capture images regardless of the vehicle's state of motion (e.g., moving or idling).

[0026] In step S12, the processor 14 performs an image processing procedure on at least one of the images captured by the image acquisition unit 11. In particular, the processor 14 can obtain the images captured by the image acquisition unit 11 (the images can be extracted from a video recorded by the DVR and stored in the data storage 12) and perform dynamic image recognition on the images to identify objects in the images using the CNN model.

[0027] In this embodiment, the image acquisition unit 11 is capable of capturing images of the vehicle's surroundings from various directions, covering the left front of the vehicle, the front of the vehicle, and the right front of the vehicle. In other embodiments, dynamic image recognition can be performed with respect to different parts of the captured images. For example, in this embodiment, dynamic image recognition can be performed with respect to images of the surroundings on a side of the vehicle, that is, on the side of a front passenger seat (hereinafter referred to as the "passenger side").It should be noted that because in different geological regions of the world the front passenger seat may be located on different sides of the vehicle, dynamic image recognition may be performed with respect to different parts of the images, depending on the geological region in which this procedure is carried out.

[0028] In embodiments where the vehicle is driven on a road, one or more traffic signs may be located on the front or sides of the vehicle. Therefore, in step S13, the processor 14 determines whether at least one traffic sign is detected in any of the images, that is, whether the identified objects in step S12 exhibit a traffic sign. A traffic sign may be in the form of a sign, a light device, a screen, or the like, and may contain various types of information, instructions, rules, restrictions, or requirements, etc. In this embodiment, the traffic sign may be restrictive in nature, i.e., it may display a message such as a speed limit, a prohibited action, etc. In this embodiment, the processor 14 can perform step S13 by determining whether a flat surface is detected.The presence of a flat surface can indicate that a traffic sign has been recognized. In other embodiments, various methods known from the associated technology for traffic sign recognition can be used. Once it has been determined that a traffic sign has been recognized, the processor 14 can identify the content of the traffic sign (i.e., determine the message) using the OCR model or other commercially available recognition techniques. In one example, the type of traffic sign can be determined based on its shape and / or color (e.g., in Taiwan, a maximum speed limit sign might be circular with a red outer ring, and a minimum speed limit sign might be circular with a blue background).After recognizing the shape and / or color of the traffic sign, the processor 14 can further identify the characters on the traffic sign to determine the details contained within it. For example, the characters "70" on a circular traffic sign with a red outer ring can indicate that a maximum speed limit of 70 kilometers per hour is in effect.

[0029] It should be noted that in some embodiments, the processor 14 can be configured to perform dynamic image recognition with respect to a portion of the image. For example, for an image captured by the image acquisition unit 11, the processor 14 can be configured to divide the image into three parts (e.g., an upper part, a middle part, and a lower part) and perform dynamic image recognition with respect to the middle part of the image. In this way, the traffic sign recognition operation can be performed more efficiently, as the amount of data that the processor 14 needs to process is significantly reduced.

[0030] If the determination in step S13 is positive, the flow continues to step S14. Otherwise, the flow periodically returns to step S13 to determine whether at least one traffic sign is detected in any of the images subsequently acquired by the image acquisition unit 11. For example, in this embodiment, if no traffic sign is detected, the processor 14 can repeat the detection after a predetermined time interval (e.g., 500 milliseconds).

[0031] In step S14, processor 14 determines whether additional information indicating a state associated with the message is detected in relation to the traffic sign (e.g., the states can be in the traffic sign itself or in its vicinity). Essentially, step S14 serves to determine whether the traffic sign is a conditional traffic sign, meaning that the restriction, requirement, or information of the conditional traffic sign applies only when a specific condition, as described in (or near) the traffic sign, is met (e.g., the restriction / requirement / information applies only to a specific type of vehicle, such as buses, or the restriction / requirement / information applies only during a specific time period(s) of the day (from 6:00 a.m. to 8:00 p.m.)).

[0032] In this embodiment, the processor 14 is first configured to define a detection area in one of the images that is associated with the traffic sign. In this embodiment, the detection area can be in the form of a circle with a center point that coincides with the geometric center of the traffic sign in one of the images, wherein the size of the detection area is positively related to the size of the traffic sign in one of the images. For example, the size of the detection area can be twice the size of the traffic sign in one of the images.

[0033] The processor 14 then determines whether additional information is detected within the detection area. If the determination in step S14 is negative, the process continues with step S15. Otherwise, the process continues with step S16. It should be noted that in this embodiment, the processor 14 is not required to perform the detection with respect to all images captured by the image acquisition unit 11. In this way, the process of determining whether additional information is detected within the detection area can be carried out more efficiently, as the amount of data to be processed by the processor 14 is reduced.

[0034] In step S15, the processor 14 determines whether an alarm action should be performed, based on at least the message displayed by the traffic sign and a current state associated with the vehicle. It should be noted that the current state consists of one or more factors, such as the current time, the current weather, and the vehicle's movement (e.g., speed, direction of travel, lane the vehicle is in, etc.). If the determination in step S15 is positive, the processor 14 executes the alarm action. In this embodiment, the processor 14 controls the output unit 13 of the driver assistance system 1 to output an alarm (e.g., an alarm text, a warning tone, a speech that reads the alarm text aloud, etc.) associated with the message. Otherwise, the processor 14 may conclude that no action is required, and the procedure is terminated.In other embodiments, the method can be reimplemented after a predetermined time interval (e.g., 500 milliseconds).

[0035] In step S16, processor 14 identifies the state contained in the additional information and determines whether at least one part of the current state matches at least one part of the state. In this embodiment, processor 14 identifies the state contained in the additional information using the OCR model or other commercially available recognition techniques. If it is determined that at least one part of the current state matches at least one part of the state, the process continues to step S17. Otherwise, processor 14 may conclude that no action is required, and the process terminates. In other embodiments, the process can be reimplemented after a predetermined time interval (e.g., 500 milliseconds).

[0036] In step S17, processor 14 determines whether an alarm action should be performed based on at least the message displayed by the traffic sign and the current state associated with the vehicle.

[0037] In particular, the operations of step S17 include the processor 14 determining whether at least part of the current state violates at least part of the message. If it is determined that at least part of the current state violates at least part of the message, the processor 14 executes the alarm action. In this embodiment, the processor 14 controls the output unit 13 of the driver assistance system 1 to output an alarm (such as text, a warning tone, etc.) associated with the message. Otherwise, if it is determined that no part of the current state violates the message, the processor 14 can repeat the determination after a predetermined time interval (e.g., one second) or restart the procedure.

[0038] During operation, when the vehicle is in motion, a traffic sign 2, as in Fig. 3, depicted by the image acquisition unit 11. The traffic sign 2 can have a graphic part 21 that displays a message (e.g. "Left turns are prohibited"), and a text part 22 that indicates a condition associated with the message (e.g. the specified time periods 7:00 to 9:00 a.m. and 3:30 to 5:30 p.m., during which left turns are prohibited).

[0039] In this way, processor 14 determines in step S13 that a traffic sign is detected and determines the message (i.e., left turns are prohibited), and the sequence continues to step 14 to determine if any additional information is detected indicating a condition associated with the message in or near traffic sign 2. If text part 22 is detected, the sequence continues to step S16.

[0040] In step S16, processor 14 identifies the state contained in the additional information. Processor 14 then determines whether the current time falls within a specific time period defined in the state (i.e., 7:00 to 9:00 a.m. or 3:30 to 5:30 p.m.). If the determination is negative, this means that turning left is permitted at the current time and the procedure can be aborted. If the determination is positive, step S17 is performed to determine whether any movement of the vehicle indicates that the vehicle is performing the prohibited action (turning left), and to trigger the execution of the alarm action if the vehicle's movement violates the message (i.e., the vehicle turns left). In this embodiment, the determination can be made by checking various parameters from different components of the vehicle, such as...a steering wheel that is turned counter-clockwise by, for example, more than a preset angle, a left-turn direction light that is activated, a preset navigation route that indicates that a left turn is imminent, etc.

[0041] In other embodiments, different types of traffic signs can be recognized. For example, if a speed limit sign is recognized, the message may be a maximum and / or minimum speed limit, and the condition may be a vehicle classification (for example, the traffic sign may indicate speed limits prescribed for light vehicles). In this case, the processor 14 can first determine the vehicle classification (e.g., based on the original equipment manufacturer (OEM) data stored in data memory 12) to ascertain whether the vehicle is classified as a light vehicle.If the vehicle is determined to be classified as a light vehicle, the processor 14 can use the vehicle's speedometer to determine whether the vehicle is moving faster than the maximum speed limit or slower than the minimum speed limit, and initiate the execution of the alarm action if the vehicle's movement violates at least part of the message (moving faster than the maximum speed limit or slower than the minimum speed limit).

[0042] In another example, the condition could be a weather condition of the vehicle's surroundings (for example, the traffic sign could indicate a maximum speed limit prescribed in rain, fog, etc.). In this case, the processor 14 can first determine the current weather condition (e.g., control the communication unit 15 to communicate with a remote server to obtain the current weather condition) to determine whether the current weather condition is the weather condition indicated by the additional information relating to the traffic sign.If it is determined that the current weather condition corresponds to the weather condition specified by the additional information relating to the traffic sign, the processor 14 can determine, based on a speedometer of the vehicle, whether the vehicle is moving faster than the maximum speed limit and initiate the execution of the alarm action if the movement of the vehicle violates the message (i.e., moves faster than the maximum speed limit).

[0043] In some examples, the message may contain a size restriction (e.g., height, width, etc.) and a weight restriction. In such cases, processor 14 can access the OEM data stored in data memory 12 to determine whether the vehicle is within the size restrictions specified by the traffic sign.

[0044] Using the above procedure, the driver assistance system 1 can be configured to detect a traffic sign message while the vehicle is in motion and to determine a state associated with that traffic sign message. Subsequently, the driver assistance system 1 can be configured to determine whether the current state of the vehicle indicates that it may be performing a prohibited action and, accordingly, initiate an alarm action to inform the driver.

[0045] Fig. Figure 4 is a flowchart illustrating the steps of a method for providing driver assistance according to one embodiment of the disclosure. In this embodiment, the method is implemented by the driver assistance system 1, as shown in Fig. Figure 1 is shown and implemented. It should be noted that the steps of the procedure in this embodiment deal with the potential case where several traffic signs are recognized in one image.

[0046] In step S21, the processor 14 controls the image acquisition unit 11 to continuously capture images of the vehicle's surroundings.

[0047] In step S22, the processor 14 performs an image processing procedure on at least one of the images captured by the image acquisition unit 11. In particular, the processor 14 performs dynamic image recognition on at least one of the images to identify objects in the image(s) using the CNN model.

[0048] In step S23, the processor 14 determines whether a first traffic sign displaying an initial message has been detected. After determining that a first traffic sign has been detected, the processor 14 can further identify the content of the first traffic sign (i.e., determine the message) using the OCR model or other commercially available recognition techniques. If the determination in step S23 is positive, the process continues to step S24. Otherwise, the process periodically returns to step S23 to determine whether at least one traffic sign is detected in any of the images subsequently acquired by the image acquisition unit 11. For example, in this embodiment, if no traffic sign is detected, the processor 14 can repeat the detection after a predetermined time interval (e.g., 500 milliseconds).

[0049] In step S24, processor 14 determines whether additional information indicating a state associated with the message is detected in or near the first traffic sign, and whether a second traffic sign displaying a second message is detected. Based on the determination in step S24, one of steps S25 to S27, as described below, can be implemented. For example, the first traffic sign might display a maximum speed limit (i.e., the first message) with a state, and the second traffic sign might display a different maximum speed limit (i.e., the second message).

[0050] In this embodiment, the processor 14 is first configured to define a detection area in one of the images, which is associated with the first traffic sign. In this embodiment, the detection area can be circular with a center point that coincides with the geometric center of the first traffic sign in the image, with the size of the detection area being positively related to the size of the first traffic sign in the image. For example, the size of the detection area can be twice the size of the first traffic sign in the image. The processor 14 then determines whether the additional information and the second traffic sign are detected within the detection area.

[0051] If no additional information or traffic sign is detected, the process continues with step S25, in which processor 14 determines whether an alarm action is required, based on at least the message displayed by the traffic sign and a current state associated with the vehicle. It should be noted that in this case, the operations of step S25 can be performed by processor 14 in a similar manner to those of step S15 (see Fig. 2).

[0052] If additional information is detected but no additional traffic sign is detected, the process continues with step S26, in which processor 14 determines whether at least one part of the current state matches at least one part of the state. It should be noted that in this case, the operations of step S26 can be performed by processor 14 in a similar manner to the operations of steps S16 and S17 (see Fig. 2).

[0053] In the event that both additional information and the second traffic sign are recognized, the process continues with steps S27 and S28, in which processor 14 selects one of the first traffic sign and the second traffic sign as the traffic sign to be dealt with.

[0054] In particular, the identified traffic signs may be in the form of a group of traffic signs 3, as in Fig. Figure 5 illustrates the group of traffic signs 3, which comprises a first traffic sign 31A, a second traffic sign 31B, and an image 32. The first traffic sign 31A and the second traffic sign 31B can be associated with a first message (e.g., a maximum speed limit or a minimum speed limit) and, correspondingly, with a second message (e.g., another maximum speed limit or another minimum speed limit). The image 32 can contain graphic information representing a condition (e.g., a specific weather condition) and can be associated with one of the first traffic sign 31A and the second traffic sign 31B. In other embodiments, the image 32 can contain characters (e.g., text and numbers) that provide additional information.

[0055] It should be noted that in this example the first traffic sign 31A, the second traffic sign 31B and picture 32 are enclosed by broken lines (which may not actually be present on the traffic signs).

[0056] In this embodiment, in step s27, processor 14 selects one of the traffic signs that has the additional information (i.e., the first traffic sign 31A) as the primary traffic sign. The process then continues with step S28, in which processor 14 determines whether at least one part of the current state matches at least one part of the state shown. For example, in this case, where the group of traffic signs 3 is detected and the state indicates rain, processor 14 can access a remote server to obtain the current weather state in order to determine whether the current weather state is the weather state indicated by image 32 (i.e., rain).

[0057] If the current weather condition is determined to be that specified in Figure 32, the process proceeds to step S29, in which processor 14, based on the vehicle's speedometer, determines whether the vehicle is traveling faster than the maximum speed limit (110 km / h) indicated by the first traffic sign 31A. If the vehicle's movement violates the first message (the vehicle is traveling at a speed exceeding 110 km / h), the processor initiates the alarm action. In other words, if the determination in step S28 is positive, step S29 is executed with respect to the primary traffic sign, which serves as the traffic sign to be processed.

[0058] On the other hand, if step S28 determines that the current weather condition is not the weather condition indicated in Figure 32, the process continues with step S30, in which processor 14 uses the vehicle's speedometer to determine whether the vehicle is traveling faster than the maximum speed limit (130 km / h) indicated by the second traffic sign 31B. If the vehicle's movement violates the second message (i.e., the vehicle is traveling at a speed exceeding 130 km / h), the process triggers the alarm action. In other words, if the determination in step S28 is negative, then step S30 is executed with respect to the second traffic sign, which serves as the traffic sign to be processed.

[0059] In some examples, when processing the detected group of traffic signs 3, if it is determined that the messages of the first traffic sign 31A and the second traffic sign 31B are of the same type (e.g., both are maximum speed limits), and the condition typically indicates that the message in question would be more restrictive (e.g., typically a lower maximum speed limit applies in rainy or foggy weather than in good (normal) weather), the processor 14 can directly select the traffic sign associated with the more restrictive message (e.g., a lower maximum speed limit, which may be 110 kilometers per hour) as the primary traffic sign in step S27 and then execute steps S28 to S30 accordingly.

[0060] It should be noted that although in this embodiment the first traffic sign 31A and the second traffic sign 31B cannot be recognized simultaneously, the processor 14 is nevertheless able to correctly select one of the first traffic sign 31A and the second traffic sign 31B as a primary traffic sign, based on the current state, and to use one of the traffic signs applicable to the current state of the vehicle to determine whether the alarm action should be carried out. That is, under normal weather conditions (e.g., not rainy and not foggy), the processor 14 is programmed to determine whether the vehicle is traveling faster than 130 kilometers per hour, and on a rainy day, the processor 14 is programmed to determine whether the vehicle is traveling faster than 110 kilometers per hour.

[0061] In one embodiment, the operations of step S27, in which the processor 14 selects one of the first traffic sign and the second traffic sign as the primary traffic sign, can be performed in a different way than described above.

[0062] In particular, in such an embodiment, the processor 14 determines whether one of the traffic signs, together with the additional information, is enclosed by a visible boundary and the other traffic sign is not enclosed within the visible boundary. In such a case, the processor 14 can select the traffic sign that, together with the additional information, is enclosed by a visible boundary as the primary traffic sign. Using the group of traffic signs 3 as in Fig.Figure 5 is shown as an example where, if a visible boundary 33B, which includes the first traffic sign 31A and the image 32, and a visible boundary 33C, which includes the second traffic sign 31B, are detected, the processor 14 can select the first traffic sign 31A, which is assumed to be associated with the image 32 (and thus with the additional information), as the primary traffic sign.

[0063] According to one embodiment of the disclosure, a computer program product is provided which includes instructions that, when executed by a processor of an in-vehicle driver assistance system, cause the processor to perform steps of a method for providing driver assistance, as described in each of the embodiments mentioned above. The computer program product can be stored on a non-temporary, computer-readable storage medium (such as a flash drive, a compact disc, a cloud drive, etc.).

[0064] According to some embodiments, the driver assistance system 1 can be configured to obtain the current weather via one of several methods without communicating with external servers. For example, the processor 14 can be programmed to determine the current weather by executing an image processing procedure, which can be programmed to determine the current weather based on images captured by the image acquisition unit 11. Alternatively, the processor 14 can be programmed to determine the current weather by detecting the states of certain vehicle components, such as whether a windshield wiper is activated, whether there are raindrops on the windshield, etc. It should be noted that determining the current weather based on images may be available in the associated technology, so details are omitted here for the sake of brevity.

[0065] In another case, the determination in step S28 can initially be based on the current weather obtained from the remote server. If it is determined that the current weather matches the condition (i.e., it is raining, snowing, foggy, etc.), the process continues with step S29. On the other hand, if it is determined that the current weather does not match the condition (i.e., it is sunny, cloudy, etc.), processor 14 can further determine the current weather based on the states of some vehicle components and / or the result of the image processing procedure. If this further determination also indicates that the current weather does not match the condition, the process continues with step S30. Conversely, if this further determination indicates that the current weather matches the condition, the process continues with step S29.

[0066] In this way, the process of the above embodiment can serve to double-check the information received from the remote server in order to rule out the potential situation that the current weather received from the remote server is not the same weather actually experienced by the vehicle.

[0067] In these cases, the driver can manually switch to implement the detection method described above. In these cases, some operations in the procedure can be performed more efficiently and cost-effectively, as the need for external inputs (e.g., the current weather from remote servers) can be eliminated and the process can be implemented using components already present in the vehicle (i.e., no additional component needs to be installed).

[0068] In summary, the embodiments of the disclosure provide a method, a system, and a computer program product capable of capturing images of a traffic sign on the road, identifying the message and the relevant condition associated with the traffic sign, and determining whether the current state of the vehicle violates the message. If the determination is positive, an alarm action is executed, which may include the output of an alarming text, a warning tone, a speech that reads the alarming text aloud, etc., to alert the vehicle's user. In this way, the user can concentrate more on driving itself and is less distracted by the need to look at traffic signs.

[0069] The above description includes numerous specific details for explanatory purposes, in order to provide a comprehensive understanding of the embodiments. However, it will be obvious to the person skilled in the art that one or more other embodiments can be implemented without some of these specific details. It should also be noted that, throughout this specification, references to "an embodiment," "an embodiment with a reference numeral," and so on, mean that a particular feature, structure, or property may be incorporated into the exercise of the disclosure.It should also be acknowledged that in the description, sometimes different features in a single embodiment, figure or description thereof are grouped together for the purpose of streamlining the disclosure and aiding in the understanding of various inventive aspects, and that one or more features or specific details from one embodiment may be applied together with one or more features or specific details from another embodiment, where appropriate, in the exercise of the disclosure.

Claims

[1] A method for providing a driver assistance system, the method being implemented using a driver assistance system (1) comprising a processor (14) and an image acquisition unit (11), the method comprising the following steps a) Control of the image acquisition unit (11) by the processor (14) to continuously acquire images of the vehicle's surroundings; b) Performing an image processing procedure by the processor (14) on at least one of the images captured by the image acquisition unit (11); c) Determine, by the processor (14), whether a traffic sign displaying a message has been detected in at least one of the images; d) if the finding in step c) is positive, determine, by the processor (14), whether additional information relating to the traffic sign is detected that indicates a state associated with the message; e) Determine, by the processor (14), whether an alarm action needs to be performed based on at least the message displayed by the traffic sign and a current state associated with the vehicle, the current state comprising a current time, current weather, movement of the vehicle or any combination thereof, and f) if the finding in step e) is positive, the processor (14) executes the alarm action, characterized by , that the procedure between steps d) and e) further includes: g) if in at least one of the images in step c) a flat surface comprising a group of traffic signs is detected and additional information associated with one of the traffic signs is detected in relation to one of the traffic signs in step d), selection of one of the traffic signs as a traffic sign to be processed; where step e) is carried out in relation to the traffic sign to be processed, and that step g) further includes: Select one of the traffic signs with the additional information as the primary traffic sign; Determine whether at least part of the current state matches at least part of the future state; If the assessment is positive, carry out step e) with regard to the primary traffic sign that serves as the traffic sign to be processed; and If the finding is negative, carry out step e) with regard to the other traffic sign that serves as the traffic sign to be processed. [2] The method according to claim 1, characterized by , that: Step e) includes determining whether at least one part of the current state matches at least one part of the state, and if it is determined that at least one part of the current state matches at least one part of the state, it is determined whether at least one part of the current state violates at least one part of the message; and Step f) includes controlling an output unit (13) of the driver assistance system (1) to issue an alarm associated with the message. [3] The method of claim 1, wherein the messages for the traffic signs include speed limits, the additional information includes a specific weather condition, characterized by, that step g) is carried out by comparing the current weather with the specific weather condition and using one of the traffic signs with the additional information as the traffic sign to be processed if the current weather corresponds to the specific weather condition. [4] The method according to claim 1, characterized by , that step g) includes: If it is determined that one of the traffic signs, together with the additional information, is enclosed within a visible boundary and another of the traffic signs is not enclosed within the visible boundary, carry out step e) with respect to the one of the traffic signs that serves as the traffic sign to be processed. [5] A driver assistance system (1) comprising a processor (14) and an image acquisition unit (11) connected to the processor (14), wherein the processor (14) is programmed to to control the aforementioned image acquisition unit (11) in order to continuously acquire images of the vehicle's surroundings; to perform an image processing procedure on at least one of the images captured by the image acquisition unit (11); to determine whether a traffic sign displays a message in which at least one of the images is recognized; If the finding is positive, determine whether additional information relating to the traffic sign is detected that indicates a condition associated with the message; to determine whether an alarm action is to be performed, based on at least the message displayed by the traffic sign and a current condition associated with the vehicle, the current condition comprising a current time, current weather, movement of the vehicle, or any combination thereof; and if the determination is positive, to perform the alarm action, characterized by , that the processor (14) is further programmed to determine, before deciding whether to execute an alarm action: to select one of the traffic signs as a traffic sign to be processed if at least one of the images detects a flat surface encompassing a group of traffic signs and additional information associated with one of the traffic signs is detected in relation to that one of the traffic signs; wherein the determination of whether an alarm action is to be carried out is performed by the said processor (14) with regard to the traffic sign to be processed; where, when selecting one of the traffic signs as the traffic sign to be processed, the aforementioned processor (14) is further programmed to Select one of the traffic signs with the additional information as the primary traffic sign; to determine whether at least part of the current state matches at least part of the state; If the assessment is positive, to determine whether the alarm action should be carried out in relation to the primary traffic sign that serves as the traffic sign to be processed; and If the determination is negative, it must be determined whether the alarm action should be carried out with regard to the other traffic sign that serves as the traffic sign to be processed. [6] The driver assistance system (1) according to claim 5, further characterized by an output unit (13) wherein: When determining whether an alarm action should be executed, the processor (14) is programmed to determine whether at least one part of the current state matches at least one part of the state, and if it is determined that at least one part of the current state matches at least one part of the state, to determine whether at least one part of the current state violates at least one part of the message; and When the alarm action is executed, the processor (14) mentioned above is programmed to control the output unit (13) of the driver assistance system (1) to issue an alarm associated with the message. [7] The driver assistance system (1) according to claim 5, wherein the messages for the traffic signs include speed limits, the additional information includes a specific weather condition, characterized by , that when selecting one of the traffic signs as a traffic sign to be processed, the said processor (14) is programmed to compare the current weather with the specific weather condition and to use one of the traffic signs with the additional information as the traffic sign to be processed if the weather corresponds to the specific weather condition. [8] The driver assistance system (1) according to claim 5, characterized by, that when one of the traffic signs is selected as a traffic sign to be processed, the said processor (14) is programmed to determine whether an alarm action should be carried out with respect to the one of the traffic signs that serves as the traffic sign to be processed, if it is determined that one of the traffic signs together with the additional information is enclosed within a visible boundary and another of the traffic signs is not enclosed within the visible boundary. [9] A computer program product, characterized by Instructions which, when executed by a processor (14) of an in-vehicle driver assistance system (1), cause the processor (14) to perform steps of a method according to claim 1.

Images