Driving reminding method and device, equipment, storage medium and product

Abstract

The invention discloses a driving reminding method and device, equipment, a storage medium and a product. The method comprises the steps that driving image information of a vehicle is acquired; based on the driving image information of the vehicle, determining color information of a target signal lamp of a target traffic light group corresponding to the driving direction of the vehicle in the driving image information; determining first distance information between the vehicle and a stop line in the driving image information based on the driving image information under the condition that the color information of the target signal lamp represents that the vehicle is prohibited from passing; based on the first distance information and vehicle speed information of the vehicle, time information of the vehicle crossing the stop line with the vehicle speed information is determined; and when the time information is within the preset time range, sending reminding information. Thus, reminding information can be sent out in time, the unconscious red light running behavior caused by inattention or misjudgment of traffic light signals of a driver is effectively avoided, and therefore the driving safety is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle driving safety assistance, and in particular to a driving reminder method, device, equipment, storage medium and product. Background Art

[0002] Although current mobile navigation applications have provided traffic light reminder functions, these reminders mainly rely on information input from sensors such as high-precision maps and crowdsourced maps, but fail to fully consider the dynamic changes in road conditions during actual driving. Drivers still need to judge whether they can safely pass the stop line at the intersection while complying with traffic rules, which leads to insufficient accuracy of reminders. Therefore, in order to improve vehicle driving safety, it is particularly important to develop a real-time and highly accurate red light running reminder system. Summary of the invention

[0003] In view of this, the embodiments of the present application at least provide a driving reminder method, device, equipment, storage medium and product.

[0004] The technical solution of the embodiment of the present application is as follows:

[0005] In a first aspect, an embodiment of the present application provides a driving reminder method, which includes: obtaining driving image information of a vehicle; based on the driving image information of the vehicle, determining the color information of a target signal light of a target traffic light group corresponding to the driving direction of the vehicle in the driving image information; when the color information of the target signal light indicates that the vehicle is prohibited from passing, based on the driving image information, determining first distance information between the vehicle and a stop line in the driving image information; based on the first distance information and the vehicle's speed information, determining time information when the vehicle crosses the stop line at the speed information; and issuing a reminder message when the time information is within a preset time range.

[0006] In this way, the driving reminder device first obtains the driving image information of the vehicle, and determines the color information of the target signal light of the traffic light group corresponding to the driving direction of the vehicle based on this information. When the color information of the target signal light shows that the vehicle is prohibited from passing, the driving reminder device will further determine the first distance information between the vehicle and the stop line in the driving image. Secondly, the system will calculate the time information required to cross the stop line according to the current speed of the vehicle. If the time information falls within the preset time range, the system will issue a reminder message in time. By analyzing the surrounding environment information of the vehicle during driving and the driving data of the vehicle itself in real time, it can be accurately evaluated whether the vehicle can stop safely before the stop line when judging whether the time information required for the vehicle to cross the stop line is within the preset time range. When the time information is within the preset time range, it is determined that the vehicle can stop safely before the stop line. At this time, it is necessary to issue a reminder message in time to effectively avoid the driver's unconscious red light running behavior caused by inattention or misjudgment of the traffic light signal, thereby improving driving safety.

[0007] Furthermore, based on the vehicle's driving image information, the color information of the target signal light of the target traffic light group corresponding to the vehicle's driving direction in the driving image information is determined, including: determining the vehicle's driving direction and the target traffic light group based on the vehicle's driving image information; determining the color information of the target signal light in the target traffic light group based on the vehicle's driving direction.

[0008] In this way, the driving direction and the corresponding target traffic light group are determined by analyzing the driving image information of the vehicle, and the color information of the target signal light is further identified according to the driving direction of the vehicle. It can quickly respond to changes in the traffic environment and accurately identify the driving direction of the vehicle, thereby ensuring that the target traffic light group is accurately locked in a complex and changing traffic environment.

[0009] Furthermore, based on the vehicle's driving image information, the vehicle's driving direction is determined, including: determining all ground guide arrow image information in the vehicle's driving image information; determining, among all the ground guide arrow image information, the target ground guide arrow image information having the smallest lateral distance to the vehicle; and determining the direction indicated by the target ground guide arrow image information as the vehicle's driving direction.

[0010] In this way, the vehicle's driving direction is determined by determining all ground guidance arrow image information in the vehicle's driving image information and finding the target ground guidance arrow image information with the smallest lateral distance from the vehicle. It can quickly analyze and respond to changes in the traffic environment, not only ensuring driving safety, but also promoting compliance with traffic regulations.

[0011] Furthermore, based on the driving image information of the vehicle, the target traffic light group is determined, including: determining all the traffic light groups in the driving image information of the vehicle; determining the first traffic light group among all the traffic light groups whose angle with the vehicle is within a preset angle range; when the number of first traffic light groups is 1, determining the first traffic light group as the target traffic light group; when the number of first traffic light groups is greater than 1, the distance between at least two first traffic light groups is within a preset longitudinal distance range, and the color information of the signal lights of at least two first traffic light groups is the same, determining any first traffic light group among the at least two first traffic light groups as the target traffic light group.

[0012] In this way, by comprehensively considering the position, direction, distance and color information of the traffic light group, the target traffic light group can be identified and confirmed. It can flexibly adapt to different roads, avoid driving reminder errors caused by complex traffic conditions (multiple traffic lights are set at the intersection), ensure that the driving reminder device makes correct reminder decisions, and improve driving safety.

[0013] Further, based on the driving direction of the vehicle, color information of the target signal light is determined in the target traffic light group, including: determining the target signal light in the target traffic light group based on the direction guided by the ground guide arrow image; and acquiring the color information of the target signal light.

[0014] In this way, the color of the target signal light that is consistent with the driving direction is located in the target traffic light group through the driving direction guided by the ground guidance arrow image. This ensures that the driving reminder device can accurately identify and pay attention to the color of the signal light related to its own driving direction, avoiding erroneous reminder behaviors caused by observation errors or negligence, and improving driving safety.

[0015] Furthermore, the driving reminder method also includes: determining the minimum time threshold and the maximum time threshold for the vehicle to cross the stop line based on the preset driver reaction time information, the vehicle's driving speed information and the first distance information; setting the minimum time threshold and the maximum time threshold as the starting boundary value and the ending boundary value of the preset time range respectively.

[0016] In this way, based on the preset driver reaction time information, the vehicle's driving speed information and the first distance information, the shortest time threshold and the longest time threshold for the vehicle to cross the stop line are determined, and the two time thresholds are set as the starting boundary value and the ending boundary value of the preset time range, respectively. This allows the driver to more accurately grasp the timing of crossing the stop line, ensure that there is enough time to react to the traffic conditions ahead, avoid traffic accidents caused by vehicle violations, and improve driving safety.

[0017] In a second aspect, an embodiment of the present application provides a driving reminder device, which includes: an acquisition module for acquiring driving image information of a vehicle; a first determination module for determining, based on the driving image information of the vehicle, the color information of a target signal light of a target traffic light group corresponding to the driving direction of the vehicle in the driving image information; a second determination module for determining, based on the driving image information, first distance information between the vehicle and a stop line in the driving image information when the color information of the target signal light indicates that the vehicle is prohibited from passing; a third determination module for determining, based on the first distance information and the vehicle's speed information, time information when the vehicle crosses the stop line at the speed information; and an issuance module for issuing a reminder message when the time information is within a preset time range.

[0018] In a third aspect, an embodiment of the present application provides a driving reminder device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements some or all of the steps in the above method when executed by a processor.

[0020] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program or instructions, which implement some or all of the steps in the above method when executed by a processor.

[0021] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0023] Figure 1 A schematic diagram of the implementation process of a driving reminder method provided in an embodiment of the present application Figure 1 ;

[0024] Figure 2 A schematic diagram of the implementation process of a driving reminder method provided in an embodiment of the present application Figure 2 ;

[0025] Figure 3 A schematic diagram of the implementation process of a driving reminder method provided in an embodiment of the present application Figure 3 ;

[0026] Figure 4A scenario diagram of a driving reminder method provided in an embodiment of the present application Figure 1 ;

[0027] Figure 5 A schematic diagram of the implementation process of a driving reminder method provided in an embodiment of the present application Figure 4 ;

[0028] Figure 6 A scenario diagram of a driving reminder method provided in an embodiment of the present application Figure 2 ;

[0029] Figure 7 A schematic diagram of the implementation process of a driving reminder method provided in an embodiment of the present application Figure 5 ;

[0030] Figure 8 A schematic diagram of the implementation process of a driving reminder method provided in an embodiment of the present application Figure 6 ;

[0031] Fig. 9 A scenario diagram of a driving reminder method provided in an embodiment of the present application Figure 3 ;

[0032] Fig.10 A flowchart of a driving reminder method provided in an embodiment of the present application;

[0033] Fig.11 A schematic diagram of the structure of a driving reminder device provided in an embodiment of the present application;

[0034] Fig.12 A hardware entity schematic diagram of a driving reminder device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0036] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0037] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0038] The terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first / second / third" can be interchanged with a specific order or sequence where permitted so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this application and are not intended to limit this application.

[0040] Before describing the sequence processing method in the embodiment of the present application, the professional terms involved in the embodiment of the present application are first explained:

[0041] Traffic Sign Recognition (TSR): An intelligent traffic sign recognition technology used in the field of automobile safety.

[0042] China New Car Assessment Program (C-NCAP): Provides consumers with professional and authoritative automobile safety performance evaluation. The latest version of the China New Car Assessment Program is C-NCAP2024, which will be officially implemented on July 1, 2024.

[0043] With the increasing number of private cars in China, traffic lights have been added to cities to effectively control and guide traffic. Regardless of the length of a driver's driving experience, there may be times when they are distracted or negligent while driving. Although current mobile navigation maps such as Baidu and AutoNavi have provided speed recommendations and traffic light reminders for some green wave sections, drivers still need to judge the color, status, and distance of the traffic lights at each intersection to determine whether they can safely drive across the stop line while complying with traffic rules. Among them, the green wave section refers to the coordinated control of signal lights in traffic management, which allows vehicles to continuously pass through multiple intersections when driving along a specific route within a period of time without stopping and waiting for red lights.

[0044] In addition, China's new car assessment procedure C-NCAP2024 has added a number of new and optimized test scoring items for reminder functions, including traffic sign recognition-red light running warning: a new red light running warning scenario has been added, which uses traffic sign recognition technology to remind drivers to obey traffic rules and avoid running red lights.

[0045] This move profoundly reflects the high importance attached to driving safety. The existing technology can remind drivers of the status of traffic lights based on high-precision maps, but it does not combine the actual driving conditions to accurately judge the color of the corresponding traffic lights according to the current driving direction and issue red light running warnings in a timely manner.

[0046] In order to solve the problems of the related art, an embodiment of the present application provides a driving reminder method. Figure 1 A schematic diagram of the implementation process of a driving reminder method provided in an embodiment of the present application Figure 1 The driving reminder method in the embodiment of the present application can be executed by a driving reminder device running in a vehicle computer, such as Figure 1 As shown, the driving reminder method can be implemented through steps S101 to S104:

[0047] Step S101, obtaining driving image information of the vehicle.

[0048] The driving image information refers to the real-time image data captured by the vehicle camera or other image acquisition equipment during the driving process of the vehicle, which contains key information about the vehicle's surrounding environment. For example, the driving image information may include road conditions, traffic signs, other vehicles, pedestrians, and traffic lights.

[0049] In some embodiments, obtaining the driving image information of the vehicle may include obtaining the driving image information of the vehicle through a vehicle-mounted camera. The vehicle-mounted camera refers to a video camera installed on the vehicle, which is used to record, monitor or analyze image data of the vehicle's surrounding environment. Exemplarily, the vehicle-mounted camera includes a camera installed by the vehicle's original factory, such as a standard safety configuration of the car - a reversing image camera; and also includes a camera not installed by the vehicle's original factory, such as a driving recorder.

[0050] Step S102: based on the driving image information of the vehicle, determining the color information of the target signal light of the target traffic light group corresponding to the driving direction of the vehicle in the driving image information.

[0051] Among them, a traffic light group refers to a group of traffic lights set up at traffic intersections or specific sections of roads. They change colors to indicate when vehicles and pedestrians can pass, when they need to prepare to stop or wait. A traffic light group usually consists of multiple signal lights, each of which can change to a different color to affect the vehicle's driving decision. For example, the signal light can be displayed as a red light, a green light, and a yellow light. Each color of light has a specific traffic indication function. When the signal light is red, vehicles are prohibited from passing; when the signal light is yellow, vehicles should slow down and prepare to stop; when the signal light is green, vehicles can pass.

[0052] In some embodiments, the target traffic light group is a traffic light group related to the vehicle's driving direction and having an instructive meaning to the driver. The target signal light is a signal light related to the vehicle's current driving path and direction and affecting the vehicle's driving decision.

[0053] It is understandable that the driving image information reflects the real-time road conditions of the vehicle, including the signal light status of the traffic light group. Therefore, the driving reminder device can determine the color information of the signal light corresponding to the vehicle's driving direction based on the driving image information.

[0054] In some embodiments, the types of signal lights include circular signal lights and arrow signal lights. The circular signal light is a signal light without pattern and in a circular shape. The arrow signal light is a signal light with an arrow shape. Exemplarily, the shapes of the arrow signal lights include a circular straight arrow shape, a left turn arrow shape, a right turn arrow shape, a U-turn arrow shape, and a composite arrow straight left turn arrow shape, a straight right turn arrow shape, a left turn U-turn arrow shape, etc.

[0055] In some embodiments, determining the color information of a target signal light of a target traffic light group corresponding to the driving direction of the vehicle in the driving image information may include, when the type of the target signal light is a circular signal light, determining the color of the target signal light as the color information of the target signal light of the target traffic light group corresponding to the driving direction of the vehicle in the driving image information.

[0056] It is understandable that when the target signal light is a circular signal light, the signal light is used to indicate the passage of all vehicles at the current intersection. If the color of the circular signal light is identified as red (indicating no passage), all vehicles passing through the intersection will receive a no passage instruction without having to judge based on their respective driving directions.

[0057] In some embodiments, determining the color information of the target signal light of the target traffic light group corresponding to the driving direction of the vehicle in the driving image information may include, based on the vehicle driving direction and the driving image information, using color analysis technology to identify and determine the color information of the target signal light of the target traffic light group corresponding to the driving direction of the vehicle in the driving image information.

[0058] For example, the driving reminder device can determine the driving direction of the vehicle through the vehicle sensor, and determine the target signal light in the target traffic light group that is consistent with the driving direction. Then, the color analysis technology is used to identify and obtain the color of the target signal light.

[0059] Step S103 : when the color information of the target signal light indicates that the vehicle is prohibited from passing, determining first distance information between the vehicle and the stop line in the driving image information based on the driving image information.

[0060] The first distance information refers to the distance between the current position of the vehicle and the stop line in front of the lane where the vehicle is traveling.

[0061] It is understandable that the driving image is an image taken by a vehicle-mounted camera, showing the position of the stop line in the image from the camera's perspective. Through the geometric relationship between the camera's perspective and the actual road surface, the position of the stop line on the actual road surface can be derived, thereby calculating the longitudinal distance between the vehicle and the stop line.

[0062] In some embodiments, based on the driving image information, determining the first distance information between the vehicle and the stop line in the driving image information may include creating a coordinate axis, and analyzing the driving image information based on a deep learning algorithm and the coordinate axis to determine the first distance information between the vehicle and the stop line in the driving image information.

[0063] Exemplarily, the driving reminder device obtains the intrinsic parameters (such as focal length, principal point coordinates, etc.) and extrinsic parameters (such as camera position and direction) of the camera, and establishes a two-dimensional or three-dimensional coordinate system with a fixed point (such as the center point or the front end point) of the vehicle bumper as the origin. The direction and unit of the coordinate system can be set according to actual needs, for example, the direction of vehicle travel is the X-axis, and the direction perpendicular to the ground is the Z-axis. The driving reminder device uses perspective transformation or three-dimensional reconstruction algorithm to establish a relationship between the pixel coordinates in the image and the physical coordinates in the real world. For example, the target is located in the image through a known calibration plate or reference object size, and the internal and external parameter information of the camera is used for geometric transformation to calculate the coordinates of the target in the actual space.

[0064] Step S104: based on the first distance information and the vehicle speed information, determine the time information when the vehicle crosses the stop line at the vehicle speed information.

[0065] It is understandable that the driving reminder device needs to make decisions to cope with complex traffic environments. The first distance information of the current vehicle from the stop line and the vehicle speed information can reflect the relative position and driving speed between the vehicle and the stop line. Then the driving reminder device can predict the time when the vehicle reaches the stop line by combining the first distance information and the vehicle speed information, providing an objective and important basis for whether to issue a reminder message, thereby improving driving safety.

[0066] In some embodiments, based on the first distance information and the vehicle speed information, determining the time information when the vehicle crosses the stop line at the vehicle speed information may include taking the quotient of the first distance information and the vehicle speed information to determine the time information when the vehicle crosses the stop line at the vehicle speed information.

[0067] Step S105: When the time information is within a preset time range, a reminder message is issued.

[0068] It is understandable that the preset time range is used by the driving reminder device to determine whether the vehicle can safely stop before the stop line without violating traffic rules when the traffic light indicates that the vehicle is prohibited from passing. The minimum threshold of the preset time range indicates the shortest braking time of the vehicle without crossing the stop line. If the time information of the vehicle crossing the stop line with the speed information is less than the minimum threshold, it means that the driver may deliberately run a red light, or even if the driving reminder device issues a reminder, the driver still cannot avoid running a red light, so no reminder information is issued; the maximum threshold of the preset time range indicates the longest braking time of the vehicle without crossing the stop line. If the time information of the vehicle crossing the stop line with the speed information is greater than the maximum threshold, it means that the driver has enough time to take deceleration measures before the stop line, or in order to avoid cultivating the bad driving habit of the driver to frequently brake at the intersection, no reminder information is issued. When the time information of the vehicle crossing the stop line with the speed information is within the preset time range, the driving reminder device can timely remind the driver that there may be a red light running behavior, and remind the driver to brake before the stop line in time to achieve safe driving.

[0069] In some embodiments, issuing the reminder information includes issuing a voice reminder information. Exemplarily, the driving reminder device issues the voice reminder information through a vehicle-mounted audio system, where the vehicle-mounted audio system refers to an audio device installed on the vehicle, and is used to convert an audio signal into sound.

[0070] In some embodiments, issuing a reminder message further comprises issuing a vibration reminder message. Exemplarily, the driving reminder device vibrates the steering wheel or the seat to remind the driver that a red light running operation may be about to be issued.

[0071] In the embodiment of the present application, the driving reminder device first determines the color information of the target signal light of the traffic light group corresponding to the driving direction of the vehicle based on the driving image information of the vehicle. When the color information of the target signal light shows that the vehicle is prohibited from passing, the driving reminder device will further determine the first distance information between the vehicle and the stop line in the driving image. Secondly, the driving reminder device will determine the time information required to cross the stop line according to the current speed of the vehicle. If the time information falls within the preset time range, the system will issue a reminder message in time. In this way, by real-time analysis of the surrounding environment information of the vehicle during driving and the driving data of the vehicle itself, it is possible to accurately assess whether the vehicle can safely stop before the stop line by judging whether the time information required for the vehicle to cross the stop line is within the preset time range. When the time information is within the preset time range, it is determined that the vehicle can safely stop before the stop line, and at this time, it is necessary to issue a reminder message in time to effectively avoid the driver's unconscious red light running behavior caused by inattention or misjudgment of the traffic light signal, thereby improving driving safety.

[0072] In some embodiments, Figure 2 A schematic diagram of the implementation process of a driving reminder method provided in an embodiment of the present application Figure 2 ,like Figure 2 As shown, the step S102 of "determining the color information of the target signal light of the target traffic light group corresponding to the driving direction of the vehicle in the driving image information based on the driving image information of the vehicle" may include steps S201 and S202. Figure 2 The steps shown are explained.

[0073] Step S201, determining the driving direction of the vehicle and the target traffic light group based on the driving image information of the vehicle.

[0074] It is understandable that the driving image information may include road conditions, traffic signs, other vehicles, and traffic light groups, etc. By analyzing the driving image information of the vehicle, the driving direction of the vehicle and the target traffic light group can be determined.

[0075] In some embodiments, determining the vehicle's driving direction and target traffic light group based on the vehicle's driving image information may include analyzing ground guide arrow image information in the vehicle's driving image information through a target detection algorithm to determine the vehicle's driving direction and target traffic light group.

[0076] Exemplarily, the driving reminder device recognizes the ground guidance arrow pattern in the image through the target detection algorithm, determines the direction of the arrow, and judges the direction in which the vehicle will travel (such as straight or turn) according to the direction of the arrow. Further, when the vehicle is about to arrive at the intersection, if the ground arrow indicates straight, the vehicle needs to determine the image information of the traffic light group in the straight direction.

[0077] In some embodiments, determining the driving direction of the vehicle and the target traffic light group based on the driving image information of the vehicle may also include determining the driving direction of the vehicle and the target traffic light group by analyzing road sign information in the driving image information of the vehicle.

[0078] Exemplarily, the driving reminder device uses a target detection algorithm to identify road sign information in an image. When there is directional road sign information (such as left turn road sign information or right turn road sign information) in the image, the current driving direction of the vehicle can be inferred. Furthermore, when there is right turn road sign information in the image, the driving reminder device can infer that the vehicle will turn right. At this time, the position and state of the traffic light group will be associated with the turning direction of the vehicle, and the target traffic light group that the vehicle needs to pay attention to can be identified.

[0079] Step S202, determining the color information of the target signal light in the target traffic light group based on the driving direction of the vehicle.

[0080] It is understandable that different signal lights in a traffic light group are used to indicate different driving directions. When a vehicle is driving safely, it is necessary to determine the color information of the target signal light in the target traffic light group used to indicate the current driving direction according to the driving direction of the vehicle.

[0081] In the embodiment of the present application, the driving direction and the corresponding target traffic light group are determined by analyzing the driving image information of the vehicle, and the color information of the target signal light is further identified according to the driving direction of the vehicle. In this way, it is possible to quickly respond to changes in the traffic environment and accurately identify the driving direction of the vehicle, so that the target traffic light group can be accurately locked in a complex and changeable traffic environment.

[0082] In some embodiments, Figure 3 A schematic diagram of the implementation process of a driving reminder method provided in an embodiment of the present application Figure 3 ,like Figure 3 As shown, the step S201 of "determining the driving direction of the vehicle based on the driving image information of the vehicle" may include steps S301 to S303. Figure 3 The steps shown are explained.

[0083] Step S301, determining all ground guidance arrow image information in the vehicle's driving image information.

[0084] It is understandable that the ground guide arrow image information is often matched with the traffic light group, and the vehicle's driving decision must obtain the ground guide arrow image information. Therefore, the driving reminder device is crucial to ensure the accuracy and safety of driving decisions by determining all the ground guide arrow image information in the vehicle's driving image information.

[0085] In some embodiments, determining all ground guidance arrow image information in the vehicle's driving image information may include determining all ground guidance arrow image information in the vehicle's driving image information based on a deep learning algorithm.

[0086] Exemplarily, based on the deep learning algorithm, the image features in the driving image information are automatically extracted, the shape and direction presented by the ground guide arrow image information are identified and classified, and finally the position and direction of all ground guide arrows are marked to provide real-time driving instruction information for the vehicle.

[0087] Step S302: among all ground guidance arrow image information, determine the target ground guidance arrow image information having the smallest lateral distance to the vehicle.

[0088] In some embodiments, determining the target ground guidance arrow image information having the smallest lateral distance from the vehicle among all ground guidance arrow image information may include establishing a coordinate system, and determining the target ground guidance arrow image information having the smallest lateral distance from the vehicle among all ground guidance arrow image information based on a deep learning algorithm and coordinate axes.

[0089] In some embodiments, Figure 4 A scenario diagram of a driving reminder method provided in an embodiment of the present application Figure 1 ,like Figure 4 As shown, the driving scenarios of step S404 of "determining the target ground guide arrow image information with the smallest lateral distance from the vehicle among all ground guide arrow image information" may include Figure 4 , the following combination Figure 4 The scenario shown is described.

[0090] Exemplarily, the driving reminder device takes a fixed point (such as the center point or the front end point) of the vehicle bumper as the origin to establish a two-dimensional or three-dimensional coordinate system. The direction and unit of the coordinate system can be set according to actual needs, for example, the vehicle driving direction is the X axis, and the direction perpendicular to the vehicle driving direction is the Y axis. The driving reminder device uses a target detection algorithm to identify all ground guide arrow image information, obtains the center point position of all ground guide arrow image information, obtains the lateral distance between the center point position of each ground guide arrow image information and the vehicle through a radar sensor or a vehicle positioning system, such as the distance y1 between the vehicle and the center line 401 of the left turn arrow image, the distance y0 between the vehicle and the center line 402 of the straight arrow image, and the distance y2 between the vehicle and the center line 403 of the right turn arrow image, and determines the ground guide arrow image information corresponding to the minimum value among y0, y1 and y2 as the target ground guide arrow image information.

[0091] Step S303: determine the direction indicated by the target ground guide arrow image information as the driving direction of the vehicle.

[0092] It is understandable that the guiding arrow image information on the ground in different lanes is different, and the target ground arrow image information closest to the vehicle laterally indicates the driving direction of the lane the vehicle is in. Therefore, the driving reminder device can determine the current driving direction of the vehicle based on the ground arrow image information closest to the vehicle.

[0093] In some embodiments, after determining the driving direction of the vehicle, the vehicle drives in the direction indicated by the target ground guidance arrow image information.

[0094] In the embodiment of the present application, the vehicle's driving direction is determined by determining all ground guide arrow image information in the vehicle's driving image information and finding the target ground guide arrow image information with the smallest lateral distance from the vehicle. It can quickly analyze and respond to changes in the traffic environment, not only ensuring driving safety, but also promoting compliance with traffic regulations.

[0095] In some embodiments, Figure 5 A schematic diagram of the implementation process of a driving reminder method provided in an embodiment of the present application Figure 4 ,like Figure 5 As shown, the step S201 of "determining the target traffic light group based on the driving image information of the vehicle" may include steps S501 to S504. Figure 5 The steps shown are explained.

[0096] Step S501, determining all traffic light groups in the vehicle's driving image information.

[0097] It is understandable that at complex intersections or areas with multiple traffic lights, vehicles may face traffic light groups in different directions, and the driving reminder device must be able to identify and understand all traffic light groups in order to correctly determine which traffic lights have an impact on the current vehicle's driving path and decision.

[0098] In some embodiments, determining all traffic light groups in the vehicle's driving image information may include determining all traffic light groups in the vehicle's driving image information based on a deep learning algorithm.

[0099] Exemplarily, driving image information is automatically analyzed based on a deep learning algorithm, key features of traffic light groups in the image are extracted, the lighting status (red, yellow, green) of the traffic lights is distinguished through a classification network, and the exact location of all traffic light groups and image information of all traffic light groups are determined in combination with location information, providing real-time traffic signal instructions for vehicles.

[0100] Step S502, determining the first traffic light group among all traffic light groups, the first traffic light group having an angle with the vehicle facing within a preset angle range.

[0101] It is understandable that only the traffic light groups related to the current driving direction are used to indicate the current driving direction. Since there is a specific angle relationship between these traffic lights and the vehicle, the system can accurately determine which traffic light groups are related to the vehicle's driving direction by determining the traffic light groups whose angles with the vehicle are within a preset range, thereby avoiding misidentification of traffic light groups that are not related to the current driving direction.

[0102] In some embodiments, determining the first traffic light group among all traffic light groups whose angle with the vehicle is within a preset angle range may include determining the first traffic light group among all traffic light groups whose angle with the vehicle is within a preset angle range based on image processing technology.

[0103] Exemplarily, the driving reminder module uses image processing technology to locate the positions of all traffic light groups, and obtains the orientation angles of all traffic light groups and the vehicle, and screens out the traffic light groups whose orientation angles with the vehicle are within a preset angle range (such as setting the preset angle range to ±30 degrees). In this way, the traffic light group consistent with the vehicle's driving direction can be determined as the target traffic light group for indicating the vehicle's driving direction.

[0104] Step S503: when the number of the first traffic light group is 1, determine the first traffic light group as the target traffic light group.

[0105] Step S504, when the number of first traffic light groups is greater than 1, the distance between at least two first traffic light groups is within a preset longitudinal distance range, and the color information of the signal lights of at least two first traffic light groups is the same, any first traffic light group among the at least two first traffic light groups is determined as the target traffic light group.

[0106] It is understandable that at an intersection or in a complex road condition, there may be multiple traffic light groups. These traffic light groups with the same color information are fixed within a preset longitudinal distance range based on the urban road layout, and together indicate the traffic signal of an intersection. When the driving reminder device identifies at least two traffic light groups that meet the preset longitudinal distance range within the effective recognition range, and the signal lights of the traffic light groups display the same color, the driving reminder device can use any one of the at least two traffic light groups as the traffic light group indicating the current vehicle driving direction.

[0107] In some embodiments, when the number of first traffic light groups is greater than 1, when the distance between at least two first traffic light groups is not within a preset longitudinal distance range, or when the color information of the signal lights of at least two first traffic light groups is different, the first traffic light group closest to the longitudinal distance of the vehicle is determined as the target traffic light group.

[0108] In some embodiments, Figure 6 A scenario diagram of a driving reminder method provided in an embodiment of the present application Figure 2 ,like Figure 6 As shown, in step S504, "when the number of first traffic light groups is greater than 1, the distance between at least two first traffic light groups is within a preset longitudinal distance range, and the color information of the signal lights of at least two first traffic light groups is the same, any first traffic light group in at least two first traffic light groups is determined as the target traffic light group" A driving scenario may include Figure 6 , the following combination Figure 6 The scenario shown is described.

[0109] Figure 6 The diagram shows a scenario of a two-way six-lane or wider intersection, where the intersection is configured with a traffic light group 601 and a traffic light group 602, and the traffic light groups 601 and 602 are arranged in a road section with a longitudinal distance interval of Y.

[0110] In some embodiments, when the colors displayed by the traffic light groups 601 and 602 are the same and the longitudinal distance interval Y meets the preset longitudinal distance range, both the traffic light group 601 and the traffic light group 602 can be used as the target traffic light group. Exemplarily, the preset longitudinal distance is 0 meters to 20 meters. When the colors displayed by the traffic light group 601 and the traffic light group 602 are both red and the longitudinal distance interval is 15 meters, it can be determined that the traffic light group 601 and the traffic light group 602 jointly provide traffic instructions for the intersection where the current vehicle is traveling, that is, the traffic light group 601 and the traffic light group 602 can be used as the target traffic light group.

[0111] In some embodiments, when the colors displayed by the traffic light groups 601 and 602 are different, and the longitudinal distance interval Y satisfies the preset longitudinal distance range, the traffic light group 601 closest to the vehicle in longitudinal distance is determined as the target traffic light group. Exemplarily, when the preset longitudinal distance is 0 meters to 20 meters, when the color displayed by the traffic light group 601 is red, the color displayed by the traffic light group 602 is green, and the longitudinal distance interval is 15 meters, it can be determined that the traffic light groups 601 and 602 respectively provide traffic instructions for different intersections, and then the traffic light group 601 closest to the vehicle in longitudinal distance is determined as the target traffic light group.

[0112] In some embodiments, when the colors displayed by the traffic light groups 601 and 602 are the same, and the longitudinal distance interval Y does not meet the preset longitudinal distance range, the traffic light group 601 closest to the vehicle in longitudinal distance is determined as the target traffic light group. Exemplarily, the preset longitudinal distance is 0 meters to 20 meters. When the colors displayed by the traffic light groups 601 and 602 are both green, and the longitudinal distance intervals are 25 meters, it can be determined that the traffic light groups 601 and 602 respectively provide traffic instructions for different intersections, and then the traffic light group 601 closest to the vehicle in longitudinal distance is determined as the target traffic light group.

[0113] In some embodiments, the intersection may be configured with more than two traffic light groups to indicate traffic conditions in different directions or lanes.

[0114] In the embodiment of the present application, the target traffic light group is identified and confirmed by comprehensively considering the position, direction, distance and color information of the traffic light group. It can flexibly adapt to different roads, avoid driving reminder errors caused by complex traffic conditions (multiple traffic lights are set at the intersection), ensure that the driving reminder device makes correct reminder decisions, and improve driving safety.

[0115] In some embodiments, Figure 7 A schematic diagram of the implementation process of a driving reminder method provided in an embodiment of the present application Figure 5 ,like Figure 7 As shown, the step S202 of “determining the color information of the target signal light in the target traffic light group based on the driving direction of the vehicle” may include steps S701 and S702. Figure 7 The steps shown are explained.

[0116] Step S701, determining a target traffic light in a target traffic light group based on the direction guided by the ground guidance arrow image.

[0117] It can be understood that the arrows on the ground indicate the lane or direction in which the vehicle should travel, while the signals of the traffic light group control the traffic status of these lanes. By matching the indications of the arrows on the ground with the corresponding signal lights, it can ensure that vehicles pass through intersections or road sections safely at the right time, thereby avoiding traffic conflicts and improving traffic efficiency.

[0118] Step S702, obtaining color information of a target traffic light.

[0119] In the embodiment of the present application, the color of the target signal light that is consistent with the driving direction is located in the target traffic light group through the driving direction guided by the ground guide arrow image. This ensures that the driving reminder device can accurately identify and pay attention to the color of the signal light related to its own driving direction, avoiding erroneous reminder behaviors caused by observation errors or negligence, and improving driving safety.

[0120] In some embodiments, Figure 8 A schematic diagram of the implementation process of a driving reminder method provided in an embodiment of the present application Figure 6 ,like Figure 6 As shown, the driving reminder method may include step S801 and step S802. Figure 8 The steps shown are explained.

[0121] Step S801, based on preset driver reaction time information, vehicle speed information and first distance information, determine the shortest time threshold and the longest time threshold for the vehicle to cross the stop line.

[0122] The driver's reaction time information is an important parameter representing the time interval from when the driver perceives the stimulus to when he performs the driving operation.

[0123] The shortest time threshold is the lower limit of the fastest time to cross the stop line by combining the driver's reaction time information, the vehicle's driving speed information, the first distance information, etc. The shortest time threshold is the upper limit of the time to cross the stop line by combining the driver's reaction time information, the vehicle's driving speed information, the first distance information, etc.

[0124] It is understandable that if a car wants to stop before the stop line, it must not only consider the driver's reaction time, but also the vehicle's current speed and the distance from the stop line. Based on these factors, it can be determined how long it takes for the driver to brake suddenly and cross the stop line after receiving the reminder message, as well as how long it takes to decelerate evenly before crossing the stop line.

[0125] Step S802: setting the shortest time threshold and the longest time threshold as the starting boundary value and the ending boundary value of the preset time range respectively.

[0126] The start boundary value and the end boundary value are used to define the boundaries of the range so that the elements in the range have clear boundaries.

[0127] In this way, based on the preset driver reaction time information, the vehicle's driving speed information and the first distance information, the shortest time threshold and the longest time threshold for the vehicle to cross the stop line are determined, and the two time thresholds are set as the starting boundary value and the ending boundary value of the preset time range, respectively. This allows the driver to more accurately grasp the timing of crossing the stop line, ensure that there is enough time to react to the traffic conditions ahead, avoid traffic accidents caused by vehicle violations, and improve driving safety.

[0128] In some embodiments, Fig. 9 A scenario diagram of a driving reminder method provided in an embodiment of the present application Figure 3 ,like Fig. 9 As shown below, combined Fig. 9 The scenario shown is described.

[0129] It is understandable that since the time it takes for the visual system to recognize the ground stop line and ground guide arrow sign in front of the vehicle usually lags behind the recognition of the traffic light group, before the driving reminder device implements the driving reminder method, it is necessary to first confirm the situation of the traffic light group at the intersection where the vehicle is currently located.

[0130] If the longitudinal distance between the traffic light group and the vehicle is detected to be d1 within the identifiable range of the driving reminder device, then the longitudinal distance X from the traffic light group is set as the effective identification distance of the traffic light group, where d1 is less than or equal to X. In order to prevent the longitudinal distance d1 from changing dramatically due to a sudden change in the longitudinal distance, a hysteresis interval can be set within the range of (-X, X) to smooth the change in the longitudinal distance, thereby ensuring the stability of the identification process.

[0131] In the effective recognition range of the driving reminder device, when multiple traffic light groups are recognized, it is determined whether the signal lights of the multiple traffic light groups have the same color. If the colors are the same, any one of the multiple traffic light groups is determined as the target traffic light group for indicating the current driving direction. In the case where only one traffic light group is recognized, the traffic light group is determined as the target traffic light group for indicating the current driving direction. In the case where the signal lights of the two traffic light groups closest to the vehicle have different colors, the traffic light group closest to the vehicle is determined as the target traffic light group for indicating the current driving direction.

[0132] Through image training and other technologies, the type of target traffic light group and the shape of the signal light are identified, such as the type of traffic light group is a circular signal light group and an arrow signal light group, the shapes of the signal lights are a straight arrow shape, a left turn arrow shape, a right turn arrow shape, a U-turn arrow shape, and a composite arrow straight left turn arrow shape, a straight right turn arrow shape, a left turn U-turn arrow shape, and the corresponding colors of the signal lights.

[0133] With the vehicle bumper as the coordinate axis, a coordinate system is established. The image information of the visible traffic light group in front of the current road and the stop line in front of the vehicle are recognized by the visual sensor, and the coordinates of the traffic light group and the stop line are output to obtain the longitudinal coordinates of the traffic light group and the stop line, and the longitudinal distances d1 and d0 of the vehicle from the traffic light group and the stop line are obtained. At the same time, all ground guide arrow image information is recognized, and the center point position of all ground guide arrow image information is obtained. The lateral distance between the center point position of each ground guide arrow image information and the vehicle is obtained by the radar sensor or the vehicle positioning system, such as the distance y0 between the vehicle and the center line 901 of the left turn arrow image, the distance y1 between the vehicle and the center line 902 of the straight arrow image, and the distance y2 between the vehicle and the center line 903 of the right turn arrow image. The ground guide arrow image information corresponding to the minimum value among y0, y1 and y2 is determined as the target ground guide arrow image information, and the direction of the target ground guide arrow image information is determined as the guiding direction of the vehicle currently passing through the intersection, and is memorized and maintained during driving. After the vehicle passes an intersection, the memory retention is interrupted.

[0134] When the color of the traffic light group is red, calculate the time information when the front bumper of the vehicle is about to cross the stop line at the current speed Vego. The calculation method of the time information is TTC = d0 / Vego. Determine whether TTC is within the preset range time period (t0, t1). When the TTC size is within the range (t0, t1), it is considered that the vehicle has the possibility of running a red light. Among them, t0 and t1 are thresholds obtained through comprehensive training, taking into account multiple factors such as the distance d0 from the car to the stop line, the user's reaction time, the current speed of the vehicle, and the efficiency of the vehicle's braking (stopping). t0 is the shortest time that the user can safely stop before the stop line; t1 is the longest time that the user can safely stop before the stop line. When TTC is within the preset range time period, it is determined that the vehicle has a red light running behavior that can be stopped, and a red light running alarm is issued.

[0135] It is understandable that when TTC is less than t0, it means that the driver wants to pass the current red light intersection on his own initiative, or the driving reminder device believes that even if the driver is reminded of the risk of running a red light, the upcoming red light running behavior cannot be avoided. When TTC is greater than t1, it means that the driver has enough time to slow down before the traffic light group, or the driver should avoid the driving habit of braking suddenly to avoid the false alarm of the red light running reminder in the above situation.

[0136] After the vehicle passes the stop line, it is determined whether to issue a red light warning again based on the stop line of the intersection where the red light warning was issued last time. If the vehicle is detected to have run a red light again 50 meters after the vehicle passes the stop line, an alarm is issued. If the vehicle is detected to have run a red light again after the red light warning is issued without passing the stop line 50 meters, a red light warning is issued.

[0137] If the traffic light group has a countdown, you can get the countdown seconds when the traffic light is about to turn green. If the countdown seconds are less than t1, no reminder will be given.

[0138] In some embodiments, Fig.10 A flowchart of a driving reminder method provided in an embodiment of the present application is shown in FIG. Fig.10 As shown, the driving reminder method may include steps S1001 to S1013. Fig.10 The steps shown are explained.

[0139] Step S1001, obtaining driving image information.

[0140] Step S1002, determining whether there is traffic light group image information.

[0141] If it is determined in step S1002 that there is no traffic light group, the process returns to step S1001 to obtain driving image information; if it is determined in step S1002 that there is a traffic light group, the process executes step S1003.

[0142] Step S1003, determining whether the longitudinal distance of the traffic light groups meets a threshold.

[0143] If it is determined in step S1003 that the longitudinal distance of the traffic light group does not meet the threshold, return to step S1002 to determine whether there is a traffic light group; if it is determined in step S1003 that the longitudinal distance of the traffic light group meets the threshold, execute step S1004.

[0144] Step S1004, determining a semantically unique target traffic light group.

[0145] In some embodiments, virtual light positions representing different semantics are preset in the driving reminder device. Exemplarily, the virtual light positions include round lights, straight lights, left turn lights, right turn lights, and U-turn lights. The only definition requirement is that within a certain range, there is only one traffic light group of the same shape, or there are multiple traffic light groups of the same shape and color. When the uniqueness condition is met, the corresponding virtual light position is considered valid.

[0146] Step S1005, determining whether the target traffic light group is a circular signal light.

[0147] If it is determined in step S1005 that the target traffic light group is a circular signal light, step S1006 is executed; if it is determined in step S1005 that the target traffic light group is not a circular signal light, step S1008 is executed.

[0148] Step S1006, determining ground guidance arrow image information.

[0149] Exemplarily, when the longitudinal distance between the vehicle and the target traffic light group meets the range of 100 meters, the matching of the ground guide arrows begins. The ground guide arrows are matched by matching the driving direction of the vehicle through the center point of the ground arrow image information and the lateral distance of the vehicle. For ground arrows of non-compound guidance types such as straight, left turn, U-turn, right turn, etc., when a suitable guide arrow is matched, it is marked as valid and remains valid until the vehicle passes the stop line, zebra crossing or no-parking grid line, interrupting the current state and re-matching the ground guide arrows. During the maintenance period, if a new qualified ground arrow is matched, the current match is interrupted and updated to the new ground guide arrow.

[0150] Step S1007, determining whether the ground guidance arrow matches the target signal light.

[0151] If it is determined in step S1007 that the ground guide arrow matches the target signal light, execute step S1008; if it is determined in step S1007 that the ground guide arrow does not match the target signal light, return to execute step S1001 to obtain driving image information.

[0152] Step S1008, outputting the target traffic light color.

[0153] For example, when the traffic light group is a circular signal light group, the color of the target signal light of the target traffic light group is directly output as the color of the actual traffic light group of the current lane. When the traffic light group is an arrow signal light group, the arrow direction indicated by the currently memorized ground arrow image information is matched with the signal light of the target traffic light group of the virtual light position. If there is a situation where the valid light position is consistent with the current lane, the signal light color corresponding to the matched virtual valid light position is output as the color of the actual target signal light of the lane where the vehicle is currently located.

[0154] For example, if the composite arrow cannot clearly reflect the unique direction of the vehicle's driving path, then the color of the target signal light does not need to be output.

[0155] Step S1009, determining whether the target traffic light color indicates that passage is prohibited.

[0156] If it is determined in step S1009 that the target traffic light color indicates that passage is prohibited, step S1010 is executed; if it is determined in step S1009 that the target traffic light color does not indicate that passage is prohibited, the process returns to step S1008 and outputs the target traffic light color.

[0157] Step S1010, determining whether the time distance between the vehicle and the stop line meets a threshold.

[0158] The time distance is the time information required for the vehicle to travel from the current position to pass the stop line.

[0159] If it is determined in step S1010 that the time distance between the vehicle and the stop line meets the threshold, step S1011 is executed; if it is determined in step S1010 that the time distance between the vehicle and the stop line does not meet the threshold, the process returns to step S1009.

[0160] Step S1011, determining whether a reminder has been issued at the current intersection.

[0161] If it is determined in step S1011 that a reminder has been issued at the current intersection, step S1012 is executed; if it is determined in step S1011 that a reminder has not been issued at the current intersection, step S1013 is executed.

[0162] Step S1012, determining whether the vehicle exceeds a preset warning distance of the stop line.

[0163] If it is determined in step S1012 that the vehicle exceeds the preset warning distance of the stop line, step S1013 is executed; if it is determined in step S1012 that the vehicle does not exceed the preset warning distance of the stop line, the process returns to step S1001 to obtain driving image information.

[0164] Step S1013, sending a reminder message.

[0165] For example, when it is confirmed that the target traffic light in the lane where the vehicle is located is a no-passing signal, if there are no other vehicles, cyclists or pedestrians within 20 meters in front of the vehicle, the time distance is calculated based on the longitudinal distance from the vehicle to the stop line and the vehicle speed, and a determination is made as to whether the time distance triggers a red light running alarm.

[0166] Exemplarily, in this process, the boundary values ​​of the time interval threshold are set to 1.7 seconds and 4 seconds, respectively, and the driver's driving style is used to avoid false alarm triggering due to driving habits. The stepping of the accelerator pedal is used to determine whether the driver runs a red light unconsciously. For intersections with traffic light countdowns, if the remaining time of the red light is greater than 4 seconds and other conditions are met, the alarm is triggered according to the red light running reminder TTC requirements of China's new automobile evaluation rules CNCAP2024. To avoid triggering the alarm again after the vehicle passes the stop line, if the alarm has been triggered once or the vehicle is within 50 meters of the first stop line, a block time will be set to limit the alarm from being triggered again.

[0167] For example, when a double traffic light is set at an intersection, after a vehicle passes the first stop line, if it passes the second stop line in compliance with traffic rules, it is considered to have passed in compliance. However, when the intersections where the two stop lines are located are far apart, the two stop lines do not correspond to the same intersection. Therefore, after the vehicle passes the first stop line, if it is detected that the driver may have run a red light within a range of more than 50 meters, a red light running warning should be triggered in time to ensure driving safety and compliance with traffic rules.

[0168] At this point, a driving reminder method that can issue reminders in a timely manner and improve driving safety has been completed.

[0169] The present application relates to a vehicle driving safety assistance system, which involves obtaining road environment information through a vehicle-mounted visual sensor, including but not limited to the longitudinal distance between the traffic light group and the stop line of the lane where the vehicle is traveling, the target signal light status of the target signal light group (such as color, shape, countdown, etc.), the ground arrow guide signs, zebra crossings and stop lines of the current road, etc. By matching the relevant information, the actual correspondence between the current lane and the traffic light is judged, and combined with auxiliary information such as the intersection distance, the risk of running a red light is evaluated, and a corresponding reminder is issued.

[0170] The beneficial effects of this application include the ability to effectively identify red light signals in general scenarios through visual recognition and intelligent processing alone, without relying on high-precision maps, crowdsourced maps or other sensor inputs, thereby avoiding drivers' failure to notice red light scenes in time due to distraction, reducing the risk of traffic accidents, and helping to avoid traffic violation penalties.

[0171] The driving reminder method of the present application includes: 1) Traffic light color and shape recognition: The system can recognize the signal light color (red, yellow, green) and shape of the traffic light, including round lights, straight lights, left turn lights, right turn lights, U-turn lights, and composite arrow lights (such as straight left turn lights, straight right turn lights, left turn U-turn lights, etc.). Determine the lateral and longitudinal distance information between the traffic light group and the center point of the vehicle through the spatial coordinate relationship. 2) Road guidance arrow shape recognition: Identify the ground guidance arrows on the road in front of the vehicle, including straight arrows, left turn arrows, right turn arrows and composite arrows (such as straight left turn, straight right turn, left turn U-turn, straight left turn right turn, etc.). Through the relationship between the image coordinates and the vehicle coordinate system, output the lateral and longitudinal distance information between the center point of the ground arrow and the center point of the vehicle. 3) Stop line recognition and distance calculation: The stop line on the current road is identified by a visual sensor, and the lateral and longitudinal distance information between the center point of the stop line and the center point of the vehicle is calculated based on the conversion relationship between the image coordinates and the vehicle coordinate system. 4) Traffic light and lane information matching and red light running reminder: semantically match the traffic light information with the guide arrow information of the current lane to confirm the traffic light status that matches the direction of the vehicle. At the same time, based on the distance between the vehicle and the stop line and the vehicle speed, calculate the time distance of relative movement (i.e. the time required for the vehicle to travel from the current position to the stop line). Through time distance judgment, it can be confirmed whether the red light running reminder conditions are met, thereby triggering the corresponding warning.

[0172] Based on the same inventive concept, the embodiment of the present application further provides a driving reminder device, which runs in a vehicle computer. Fig.11 A schematic diagram of the structure of a driving reminder device is provided for an embodiment of the present application. Fig.11 As shown, the driving reminder device 1100 includes an acquisition module 1101, a first determination module 1102, a second determination module 1103, a third determination module 1104 and an issuing module 1105, wherein:

[0173] The acquisition module 1101 is used to acquire the driving image information of the vehicle.

[0174] The first determination module 1102 is used to determine, based on the driving image information of the vehicle, the color information of the target signal light of the target traffic light group corresponding to the driving direction of the vehicle in the driving image information.

[0175] The second determination module 1103 is configured to determine, based on the driving image information, first distance information between the vehicle and the stop line in the driving image information when the color information of the target signal light indicates that the vehicle is prohibited from passing.

[0176] The third determining module 1104 is configured to determine, based on the first distance information and the vehicle speed information, the time information at which the vehicle crosses the stop line at the vehicle speed information.

[0177] The sending module 1105 is used to send a reminder message when the time information is within a preset time range.

[0178] In some embodiments, the first determination module 1102 includes: a first determination unit, used to determine the vehicle's driving direction and the target traffic light group based on the vehicle's driving image information; a second determination unit, used to determine the color information of the target signal light in the target traffic light group based on the vehicle's driving direction.

[0179] In some embodiments, the first determination unit includes: a third determination unit, used to determine all ground guide arrow image information in the vehicle's driving image information; a fourth determination unit, used to determine the target ground guide arrow image information with the smallest lateral distance from the vehicle among all the ground guide arrow image information; and a fifth determination unit, used to determine the direction indicated by the target ground guide arrow image information as the vehicle's driving direction.

[0180] In some embodiments, the first determination unit also includes: a sixth determination unit, used to determine all traffic light groups in the vehicle's driving image information; a seventh determination unit, used to determine the first traffic light group among all traffic light groups whose angle with the vehicle is within a preset angle range; an eighth determination unit, used to determine the first traffic light group as the target traffic light group when the number of first traffic light groups is 1; and a ninth determination unit, used to determine any first traffic light group among at least two first traffic light groups as the target traffic light group when the number of first traffic light groups is greater than 1, the distance between at least two first traffic light groups is within a preset longitudinal distance range, and the color information of the signal lights of at least two first traffic light groups is the same.

[0181] In some embodiments, the second determination unit further includes: a tenth determination unit, used to determine the target traffic light in the target traffic light group based on the direction guided by the ground guide arrow image; and a first acquisition unit, used to acquire color information of the target traffic light.

[0182] In some embodiments, the driving reminder device 1100 also includes: a fourth determination module, used to determine the minimum time threshold and the maximum time threshold for the vehicle to cross the stop line based on the preset driver reaction time information, the vehicle's driving speed information and the first distance information; a setting module, used to set the shortest time threshold and the maximum time threshold to the starting boundary value and the ending boundary value of the preset time range respectively.

[0183] The above description of the driving reminder system embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the system provided by the embodiment of the present disclosure can be used to execute the method described in the above method embodiment. For technical details not disclosed in the system embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0184] It should be noted that in the embodiment of the present application, if the above-mentioned resource configuration method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.

[0185] Fig.12 The hardware entity diagram of a driving reminder device provided in an embodiment of the present application is shown in FIG. Fig.12 As shown, the driving reminder device 1200 can be the electronic device in the above embodiment. The driving reminder device 1200 may include a processor 1201, which can perform various appropriate steps and processes according to computer program instructions stored in a read-only memory 1202 or computer program instructions loaded from a memory 1208 to a random access memory (RAM) 1203. In RAM 1203, various programs and data required for the operation of the driving reminder device 1200 can also be stored. The processor 1201, ROM 1202 and RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0186] Multiple components in the driving reminder device 1200 are connected to the I / O interface 1205, including: an input unit 1206, such as a keyboard, a mouse, etc.; an output unit 1207, such as various types of displays, speakers, etc.; a memory 1208, such as a disk, an optical disk, etc.; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1209 allows the driving reminder device 1200 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0187] The processor 1201 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 1201 include, but are not limited to, a central processing unit, a graphics processor, various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The processor 1201 may execute the various methods and processes described above, such as executing the resource configuration method described above. For example, in some embodiments, the resource configuration method described above may be implemented as a computer software program, which is stored in a machine-readable medium, such as a memory 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on the driving reminder device 1200 via the ROM 1202 and / or the communication unit 1209. When the computer program is loaded into the RAM 1203 and executed by the processor 1201, one or more steps in the resource configuration method described above may be executed. Alternatively, in other implementations, the processor 1201 may be configured to execute one or more steps in the resource configuration method described above in any other appropriate manner (for example, by means of firmware).

[0188] It should be further noted that the present application may include methods, systems and / or computer program products. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present application.

[0189] The embodiment of the present application provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method can be implemented. The computer-readable storage medium can be transient or non-transient.

[0190] An embodiment of the present application provides a computer program, including computer-readable program instructions. When the computer-readable program instructions are executed in a computer device, a processor in the computer device executes some or all of the steps in the above method.

[0191] The present application embodiment provides a computer program product, including a computer program or an instruction, which, when executed by a processor, can implement some or all of the steps in the above method. The computer program product can be implemented specifically by hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK).

[0192] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM), a flash memory, a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof. The computer-readable storage medium used in this document is not to be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (for example, light pulses through a fiber optic cable), or an electrical signal transmitted through wires.

[0193] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or to a processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0194] The computer program instructions for performing the operation of the present application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as C language or similar programming languages. The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, by utilizing the state information of computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array, or a programmable logic array (PLA), the electronic circuit can execute the computer-readable program instructions to implement various aspects of the present application.

[0195] Various aspects of the present application are described herein with reference to the flow chart and / or step diagram of the method, device (system), and computer program product according to the exemplary embodiments of the present application. It should be understood that each step of the flow chart and / or step diagram and the combination of each step in the flow chart and / or step diagram can be implemented by computer-readable program instructions.

[0196] These computer-readable program instructions can be provided to a processor in a voice interaction device, a general-purpose computer, a special-purpose computer, or a processing unit of other programmable data processing devices, thereby producing a machine, so that when these instructions are executed by a processing unit of a computer or other programmable data processing device, a device that implements the functions / steps specified in one or more steps in a flowchart and / or a step diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions enable a computer, a programmable data processing device, and / or other devices to work in a specific manner, so that a computer-readable medium storing instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / steps specified in one or more steps in a flowchart and / or a step diagram.

[0197] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / steps specified in one or more steps in the flowchart and / or step diagram.

[0198] The flow chart and step diagram in the accompanying drawings show the possible architecture, function and operation of the equipment, method and computer program product according to multiple embodiments of the present application. In this regard, each step in the flow chart or step diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the steps can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each step in the step diagram and / or flow chart, and the combination of the steps in the step diagram and / or flow chart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0199] The above embodiments are only preferred embodiments for fully illustrating the present application, and the protection scope of the present application is not limited thereto. Any equivalent substitution or change made by a person skilled in the art based on the present application is within the protection scope of the present application.

Claims

1. A driving reminder method, characterized in that: The driving reminder method comprises: Acquire driving image information of the vehicle; Based on the driving image information of the vehicle, determining the color information of the target signal light of the target traffic light group corresponding to the driving direction of the vehicle in the driving image information; In a case where the color information of the target signal light indicates that vehicles are prohibited from passing, determining first distance information between the vehicle and a stop line in the driving image information based on the driving image information; Based on the first distance information and the vehicle speed information of the vehicle, determining the time information when the vehicle crosses the stop line at the vehicle speed information; When the time information is within a preset time range, a reminder message is issued.

2. The driving reminder method according to claim 1, characterized in that: The determining, based on the driving image information of the vehicle, the color information of the target signal light of the target traffic light group corresponding to the driving direction of the vehicle in the driving image information includes: Determining the driving direction of the vehicle and the target traffic light group based on the driving image information of the vehicle; Based on the driving direction of the vehicle, color information of the target signal light is determined in the target traffic light group.

3. The driving reminder method according to claim 2, characterized in that: Determining a driving direction of the vehicle based on driving image information of the vehicle includes: Determining all ground guide arrow image information in the driving image information of the vehicle; Determine, among all the ground guide arrow image information, target ground guide arrow image information having the smallest lateral distance from the vehicle; The direction indicated by the target ground guide arrow image information is determined as the driving direction of the vehicle.

4. The driving reminder method according to claim 2, characterized in that: Determining the target traffic light group based on the driving image information of the vehicle includes: Determining all traffic light groups in the driving image information of the vehicle; Determine a first traffic light group among all the traffic light groups whose angle with the vehicle is within a preset angle range; When the number of the first traffic light groups is 1, determining the first traffic light group as a target traffic light group; When the number of the first traffic light groups is greater than 1, the distance between at least two first traffic light groups is within a preset longitudinal distance range, and the color information of the signal lights of at least two first traffic light groups is the same, any first traffic light group among the at least two first traffic light groups is determined as the target traffic light group.

5. The driving reminder method according to claim 3, characterized in that: The step of determining the color information of the target signal light in the target traffic light group based on the driving direction of the vehicle includes: Determining the target signal light in the target traffic light group based on the direction guided by the ground guide arrow image; Obtain color information of the target signal light.

6. The driving reminder method according to any one of claims 1 to 5, characterized in that: The method further comprises: Determine a minimum time threshold and a maximum time threshold for the vehicle to cross the stop line based on preset driver reaction time information, the vehicle's travel speed information, and the first distance information; The shortest time threshold and the longest time threshold are respectively set as the starting boundary value and the ending boundary value of the preset time range.

7. A driving reminder device, characterized in that: include: An acquisition module, used for acquiring driving image information of a vehicle; A first determination module is used to determine, based on the driving image information of the vehicle, the color information of the target signal light of the target traffic light group corresponding to the driving direction of the vehicle in the driving image information; a second determining module, configured to determine, based on the driving image information, first distance information between the vehicle and a stop line in the driving image information when the color information of the target signal light indicates that the vehicle is prohibited from passing; A third determining module, configured to determine, based on the first distance information and the vehicle speed information, time information at which the vehicle crosses the stop line at the vehicle speed information; The sending module is used to send a reminder message when the time information is within a preset time range.

8. A driving reminder device, characterized in that: include: A memory for storing executable instructions; A processor, configured to implement the method according to any one of claims 1 to 6 when executing the executable instructions stored in the memory.

9. A computer-readable storage medium, characterized in that: The storage medium stores executable instructions, and when the executable instructions are executed by the processor, the steps in the driving reminder method described in any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images