Lane change warning method, device, equipment and storage medium

Abstract

The application discloses a lane changing warning method and device, equipment and a storage medium, and belongs to the technical field of vehicle management. The method comprises the following steps: determining the lane changing direction of a vehicle according to the opening signal of the turn signal of the vehicle; collecting the head image information of the driver of the vehicle, and obtaining the head deflection direction of the driver through the head image information of the driver; in the case that the head deflection direction of the driver matches the lane changing direction, determining the head deflection angle of the driver according to the head image information of the driver; in the case that the head deflection angle reaches the deflection angle threshold, determining the reference time length during which the head deflection angle of the driver reaches the deflection angle threshold; in the case that the reference time length reaches the time length threshold, determining the first lane changing warning information in combination with the merging assistance system, and performing lane changing warning prompt based on the first lane changing warning information. Through the lane changing warning prompt in combination with the merging assistance system, the safety of the vehicle driver is ensured, and the reliability and accuracy of the lane changing warning are improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of vehicle management, and in particular, relate to a lane changing warning method, device, equipment and storage medium. BACKGROUND

[0002] With the continuous development of vehicle management technology, driving safety problems caused by non-standard lane changing habits during driving gradually appear. If the driver does not observe the rear vehicle situation and directly changes lanes after turning on the turn signal, it is easy to cause traffic accidents. Therefore, it is necessary to perform lane changing warning during lane changing. SUMMARY

[0003] Embodiments of the present application provide a lane changing warning method, device, equipment and storage medium, which can be used for lane changing warning during lane changing. The technical solution is as follows:

[0004] In a first aspect, the embodiments of the present application provide a lane changing warning method, which comprises:

[0005] determining a lane changing direction of the vehicle according to an opening signal of a turn signal of the vehicle;

[0006] collecting head image information of a driver of the vehicle, and obtaining a head deflection direction of the driver through the head image information of the driver;

[0007] in a case where the head deflection direction of the driver matches the lane changing direction, determining a head deflection angle of the driver according to the head image information of the driver;

[0008] in a case where the head deflection angle reaches a deflection angle threshold, determining a reference time length for the head deflection angle of the driver to reach the deflection angle threshold;

[0009] in a case where the reference time length reaches a time length threshold, combining and collecting information of moving objects around the vehicle by a lane merging assistance system, the information of the moving objects including at least one of the number, speed, direction of the moving objects and the distance between the moving objects and the vehicle, the lane merging assistance system including at least one of a sensor installed on the vehicle, a camera installed on a rearview mirror of the vehicle and a radar installed on the vehicle;

[0010] determining first lane changing warning information according to the collected information of the moving objects, the first lane changing warning information being used to prompt attention to the moving objects;

[0011] performing lane changing warning prompt based on the first lane changing warning information.

[0012] In a second aspect, the embodiments of the present application provide a lane changing warning device, the device comprising:

[0013] a first determining module configured to determine a lane changing direction of the vehicle according to an opening signal of a turn signal of the vehicle;

[0014] a collecting module configured to collect head image information of a driver of the vehicle, and obtain a head deflection direction of the driver according to the head image information of the driver;

[0015] a second determining module configured to determine a head deflection angle of the driver according to the head image information of the driver in a case where the head deflection direction of the driver matches the lane changing direction;

[0016] a third determining module configured to determine a reference time length during which the head deflection angle of the driver reaches a deflection angle threshold in a case where the head deflection angle reaches the deflection angle threshold;

[0017] a warning module configured to collect information of moving objects around the vehicle in combination with a lane merging assistance system in a case where the reference time length reaches a time length threshold, the information of the moving objects comprising at least one of a number, a speed, a direction of the moving objects and a distance between the moving objects and the vehicle, the lane merging assistance system comprising at least one of a sensor installed on the vehicle, a camera installed on a rearview mirror of the vehicle and a radar installed on the vehicle; determine first lane changing warning information according to the collected information of the moving objects, and perform a lane changing warning prompt based on the first lane changing warning information, the first lane changing warning information being used to prompt attention to the moving objects.

[0018] In another aspect, a computer device is provided, the computer device comprising a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to enable the computer device to implement any of the lane changing warning methods described above.

[0019] In another aspect, a computer readable storage medium is also provided, the computer readable storage medium storing at least one computer program, the at least one computer program being loaded and executed by a processor to enable a computer to implement any of the lane changing warning methods described above.

[0020] In another aspect, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the lane changing warning method described above.

[0021] The technical solutions provided by the embodiments of the present application at least bring the following beneficial effects:

[0022] The technical solutions provided by the embodiments of the present application determine the head deflection direction of the driver and the head deflection angle of the driver by collecting the head image information of the driver, determine the first lane changing warning information in combination with the merging assistance system in the case that the head deflection direction of the driver matches the lane changing direction and the head deflection angle of the driver is maintained within the time threshold range and reaches the deflection angle threshold, and perform lane changing prompting based on the first lane changing warning information. The lane changing warning prompting is performed in combination with the merging assistance system, the safety of the driver of the vehicle is ensured, and the reliability and accuracy of the lane changing warning are improved. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0024] Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0025] Figure 2 is a method flowchart of a lane changing warning method provided by an embodiment of the present application;

[0026] Figure 3 is a schematic diagram of the position of a shooting device in a lane changing warning method provided by an embodiment of the present application;

[0027] Figure 4 is a schematic diagram of a lane changing warning process provided by an embodiment of the present application;

[0028] Figure 5 is a structural schematic diagram of a lane changing warning device provided by an embodiment of the present application;

[0029] Figure 6 is a structural schematic diagram of a server provided by an embodiment of the present application;

[0030] Figure 7is a structural schematic diagram of a terminal provided by an embodiment of the present application. DETAILED DESCRIPTION

[0031] For the purpose of making the purpose, technical solutions and advantages of the present application more clear, the following will make further detailed description to the embodiments of the present application in combination with the drawings.

[0032] It should be noted that the terms "first", "second" and the like (if any) in the specification of the present application are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application.

[0033] An embodiment of the present application provides a lane changing warning method, please refer to Figure 1 which shows a schematic diagram of a method implementation environment provided by an embodiment of the present application. The implementation environment can include a terminal 11 and a server 12.

[0034] Among them, the terminal 11 is located on a vehicle, and the vehicle has a camera or a radar detector capable of collecting vehicle surrounding information. When the camera or the radar detector needs to collect vehicle surrounding information, the method provided by the present application can be applied to operate. The server 12 can store the information to be displayed, and the terminal 11 can obtain the information to be operated from the server 12. Of course, the terminal 11 can also store the obtained information.

[0035] Optionally, the terminal 11 can obtain the information required in the method through a parallel line auxiliary system or other devices. The server 12 can be a vehicle server, or a server cluster composed of multiple vehicle servers, or a cloud computing service center. The terminal 11 and the server 12 establish a communication connection through a wired or wireless network.

[0036] Those skilled in the art should understand that the above terminal 11 and server 12 are only examples, and other existing or future terminal or server can be applicable to the present application, and should be included in the protection scope of the present application, and is hereby incorporated by reference.

[0037] Based on the above Figure 1 The implementation environment is shown, an embodiment of the present application provides a lane changing warning method, as shown in Figure 2 The method is applied to the terminal as an example, and the method includes steps 201-205.

[0038] Step 201, determining the lane changing direction of the vehicle according to the turn signal opening signal of the vehicle.

[0039] The turn signal opening signal refers to that the driver sends a signal by opening the left or right turn signal button of the vehicle, indicating that the lane changing or U-turn operation will be performed. The application does not limit the reason for the operation. No matter what reason the driver opens the left or right turn signal button, the opening of the turn signal button is regarded as the sending of the turn signal opening signal.

[0040] Based on the sending of the turn signal opening signal, the turn signal opening signal is received by the receiver of the vehicle. The receiver can be specially set up to receive the turn signal opening signal, or can receive the turn signal opening signal by means of the existing vehicle system. The application does not limit the receiving mode of the receiver receiving the turn signal opening signal. As long as the turn signal opening signal can be received and detected by the receiver itself or the vehicle system.

[0041] After the receiver receives the turn signal opening signal, it analyzes the received turn signal opening signal to determine whether the left or right turn signal is opened. The application does not limit the way the receiver analyzes the turn signal opening signal.

[0042] The specific left or right turn signal opening determined by the receiver can determine the lane changing direction of the driver driving the vehicle through the opening of the turn signal. The lane changing direction of the driver driving the vehicle is the same side as the opening side of the turn signal. The lane changing direction refers to the direction of the driver changing the driving lane of the vehicle. For example, if the receiver analyzes that the left turn signal is opened, it is determined that the lane changing direction of the driver driving the vehicle is left. If the receiver analyzes that the right turn signal is opened, it is determined that the lane changing direction of the driver driving the vehicle is right.

[0043] After determining the lane changing direction of the driver driving the vehicle, the receiver of the vehicle continues to receive the turn signal opening signal and determines the lane changing direction of the vehicle according to the turn signal opening signal, so as to react to the lane changing direction of the driver driving the vehicle in real time. For example, after the driver opens the left turn signal, the receiver determines that the driver will change the lane to the left. However, at this time, the driver closes the left turn signal based on road condition information or other information, and the receiver determines that the driver stops changing the lane to the left. If the driver opens the left turn signal, the receiver determines that the driver will change the lane to the left. However, at this time, the driver opens the right turn signal based on road condition information or other information, and the receiver determines that the driver changes the lane to the right.

[0044] In step 202, the head image information of the driver of the vehicle is collected, and the head deflection direction of the driver is obtained through the head image information of the driver.

[0045] The head image information of the driver refers to image information obtained by photographing or recording the head of the driver through a photographing device. The image information can include information such as facial expressions and eye movement trajectories of the driver, and also include data reflecting the head deflection direction of the driver. The type of photographing device is not limited in the embodiment of the application, for example, it can be photographed by an AI (artificial intelligence) camera, or it can be photographed by a normal camera, or it can be a combination of multiple photographing and information collecting devices. Regardless of any device, the head deflection direction information and the like that can be used in the embodiment of the application can be collected. In addition, the collection time is not limited in the embodiment of the application, which can be after the turn signal described in step 201 is obtained, or it can be collected for a long time without interruption after the vehicle starts. The head image information of the driver obtained after collection is used for subsequent analysis, and the collection time period used for subsequent analysis is after the turn signal described in step 201 is received.

[0046] The position of the image information collecting device is not limited in the embodiment of the application, and any position that can collect the head image information of the driver can be used. For example Figure 3 A schematic diagram of the position of the photographing device in the lane change warning method is shown in the figure, wherein 301 is the vehicle instrument panel, 302 is the image information collecting device, and 303 is the vehicle steering wheel, and the positional relationship is as shown in Figure 3 .

[0047] The information of the head deflection direction of the driver is obtained by analyzing the collected head image information of the driver, and the information of the head deflection direction of the driver can reflect whether the head deflection direction of the driver is to the left or to the right.

[0048] In one possible implementation, the head image information of the driver is photographed or recorded by an AI camera, and the point data of the driver's face in the image is obtained through the photographed or recorded image. The point data is a data operation on the facial information of the driver, for example, the driver's eyes are regarded as two point information. When the head of the driver is directly opposite the direction of travel of the vehicle, the positions of the two point information are recorded. In the case of left deflection of the head of the driver, compared with the driver directly opposite the direction of travel of the vehicle, the positions of the two points will move to the left. The way of determining the head deflection direction of the driver through the collected head image information of the driver is not limited in the embodiment of the application. Regardless of any way, the head deflection direction of the driver can be determined through the head image information of the driver.

[0049] The head deflection direction of the driver is compared with the lane change direction of the vehicle to determine whether the head deflection direction of the driver matches the lane change direction.

[0050] In a possible implementation, the determination result is divided into two cases, a case where the head deflection direction of the driver matches the lane change direction and a case where the head deflection direction of the driver does not match the lane change direction. For example, when the head deflection direction of the driver is left and the lane change direction is left, or when the head deflection direction of the driver is right and the lane change direction is right, it is considered that the head deflection direction of the driver matches the lane change direction. When the head deflection direction of the driver is left and the lane change direction is right, or when the head deflection direction of the driver is right and the lane change direction is left, it is considered that the head deflection direction of the driver does not match the lane change direction.

[0051] In the case where the head deflection direction of the driver matches the lane change direction, the head deflection angle of the driver is determined according to the head image information of the driver.

[0052] After determining whether the head deflection direction of the driver matches the lane change direction, different processing operations are performed for the two cases. If the head deflection direction of the driver matches the lane change direction, a subsequent operation of determining the head deflection angle is performed. If the head deflection direction of the driver does not match the lane change direction, second lane change warning information is generated to prompt the driver to pay attention to the lane change direction, for example, by playing the second lane change warning information through voice, or by displaying the second lane change warning information on the display screen of the vehicle, or by indicating the second lane change warning information through the indicator light on the vehicle. The way of prompting the driver to pay attention to the lane change direction is not limited in the embodiment of the application. No matter which way is used to prompt the driver to pay attention to the lane change direction, the effect of warning the driver can be achieved. Until the head deflection direction of the driver matches the lane change direction or the same direction side turn signal button is turned on. The purpose of the reminder in the step of the embodiment of the application is to prompt the driver to perform the operation, so as to standardize the lane change habit of the driver and improve the safety of the driver during driving.

[0053] In the case where the head deflection direction of the driver matches the lane change direction, the head deflection angle of the driver is determined according to the head image information of the driver. The head deflection angle of the driver refers to the rotation angle of the head of the driver relative to the center line of the vehicle in the reference direction of the front. It can reflect the current head position and posture of the driver. It usually includes the deflection angles in three directions of left and right swing, up and down tilt and left and right skew. The collected head deflection angle information is not limited in the embodiment of the application. No matter which direction of the deflection angle information or the combination of the deflection angle information in multiple directions can reflect the action of the driver and can be used for subsequent comparison and determination operations.

[0054] In a possible implementation, the collected image information is head front information of the driver, the determined head deflection angle information of the driver is a left-right swing angle of the head of the driver, the point position data of the face of the driver is obtained through the head image information of the driver, and the point position data is a data operation on the face information of the driver, for example, the two eyes of the driver are regarded as two point position information. When the head of the driver is directly opposite the driving direction of the vehicle, the positions of the two point position information are recorded. In the case of left deflection of the head of the driver, the positions of the two point position information change compared with the case that the head of the driver is directly opposite the driving direction of the vehicle. The manner of determining the head deflection angle of the driver through the collected head image information of the driver is not limited in the embodiment of the application. As long as the head deflection angle of the driver can be determined through the head image information of the driver, it is acceptable.

[0055] The determined head deflection angle of the driver is compared with a pre-set deflection angle threshold. The pre-set deflection angle threshold is determined according to the scene. In different environmental scenes or different vehicle information, a deflection angle threshold obtained through multiple simulations or specific experiments is pre-set. The deflection angle threshold is used to regulate the behaviors of the observation rearview mirror, the lane-keeping auxiliary radar, the streaming rear camera, the side camera, and other visual images that can reflect the rear or surrounding vehicle conditions in the A-pillar during the lane-changing process of the driver. The A-pillar refers to the longitudinal support rod on both sides of the front of the vehicle, which is located near the windshield glass of the front door of the vehicle. The A-pillar connects the roof and the bottom of the vehicle. Meanwhile, the embodiment of the application does not require the left deflection angle threshold of the driver to be associated with the right deflection angle threshold. The deflection angle thresholds on both sides can be the same or different. The deflection angle threshold range is not limited in the embodiment of the application. For example, the deflection angle threshold can be set as a value, and the comparison between the head deflection angle of the driver and the value is used for the judgment operation. The deflection angle threshold can also be set as a range, and the judgment operation is performed by comparing whether the head deflection angle of the driver is within the range of the deflection angle threshold.

[0056] Exemplarily, taking the case that the head deflection direction of the driver is left and the left deflection angle threshold is 20° as an example, if the head deflection angle of the driver is greater than 20°, the subsequent operation of determining the head deflection angle maintaining time of the driver is performed; if the head deflection angle of the driver is less than or equal to 20°, the third lane change warning information is generated to prompt the driver to pay attention to the oncoming vehicle, for example, the third lane change warning information is played through voice, or the third lane change warning information is displayed through the display screen of the vehicle, or the third lane change warning information is indicated through the indicator light on the vehicle. The way of prompting the driver to pay attention to the oncoming vehicle is not limited in the embodiment of the application. No matter which way is used to prompt the driver to pay attention to the oncoming vehicle, the effect of warning the driver can be achieved until the head deflection angle of the driver is adjusted to be greater than 20°. The purpose of the prompting in the step of the embodiment of the application is to prompt the driver to operate so as to regulate the lane change habit of the driver and improve the safety of the driver in the driving process.

[0057] Exemplarily, taking the case that the head deflection direction of the driver is right and the right deflection angle threshold is 30° as an example, if the head deflection angle of the driver is greater than 30°, the subsequent operation of determining the head deflection angle maintaining time of the driver is performed; if the head deflection angle of the driver is less than or equal to 30°, the third lane change warning information is generated to prompt the driver to pay attention to the oncoming vehicle, for example, the third lane change warning information is played through voice, or the third lane change warning information is displayed through the display screen of the vehicle, or the third lane change warning information is indicated through the indicator light on the vehicle. The way of prompting the driver to pay attention to the oncoming vehicle is not limited in the embodiment of the application. No matter which way is used to prompt the driver to pay attention to the oncoming vehicle, the effect of warning the driver can be achieved until the head deflection angle of the driver is adjusted to be greater than 30°. The purpose of the prompting in the step of the embodiment of the application is to prompt the driver to operate so as to regulate the lane change habit of the driver and improve the safety of the driver in the driving process.

[0058] In step 204, in the case that the head deflection angle reaches the deflection angle threshold, the reference time length during which the head deflection angle of the driver reaches the deflection angle threshold is determined.

[0059] According to the comparison and prompting operation in step 203, when the head deflection angle of the driver reaches the deflection angle threshold, the timer of the vehicle starts to collect the reference time length during which the head deflection angle of the driver is kept within the deflection angle threshold. The time during which the head deflection angle of the driver is kept within the deflection angle threshold refers to the time length during which the head of the driver is kept in a certain fixed posture, i.e., the head deflection angle, within a certain time period. At this time, the deflection angle must satisfy the condition that the head deflection angle of the driver is within the deflection angle threshold described in step 203. Generally, the time length is in seconds or milliseconds, which can reflect the attention and concentration of the driver to the current driving situation. In the embodiment of the application, the way of collecting the time is not limited, and any way of collecting information can reflect the time during which the head deflection angle of the driver is kept within the deflection angle threshold, so that the subsequent comparison and judgment operations can be performed.

[0060] In a possible implementation, when the head deflection angle of the driver reaches the deflection angle threshold for the first time, the timer starts counting from zero. When the head deflection angle of the driver leaves the deflection angle threshold range for the first time after reaching the deflection angle threshold, the timer stops counting, and the counting result is recorded.

[0061] The reference time length during which the head deflection angle of the driver is kept within the deflection angle threshold range is compared with a time length threshold. The time length threshold is determined according to the scene. In different environmental scenes or different vehicle information, a time length threshold obtained through multiple simulations or specific experiments is preset. The time length threshold is used to standardize the behavior of the driver in observing the rearview mirror, the lane change assist radar, the streaming media rear camera, the side camera or other visual images in the A-pillar that can reflect the situation of the rear or surrounding vehicles during the lane changing process. In the embodiment of the application, the left time length threshold and the right time length threshold are not required to be related. The time length thresholds on the left and right sides can be the same or different. The embodiment of the application does not limit the range of the time length threshold. For example, the time length threshold can be set as a value, and the judgment operation is performed by comparing the time during which the head deflection angle is kept within the deflection angle threshold range with the value. The time length threshold can also be set as a range, and the judgment operation is performed by comparing whether the time during which the head deflection angle of the driver is kept within the deflection angle threshold range is within the range of the time length threshold.

[0062] For example, if the driver's head deflection angle remains within the deflection angle threshold range for a reference time period greater than 3 seconds, the operation of determining the first lane change warning information by the merging assistant system is performed; if the driver's head deflection angle remains within the deflection angle threshold range for a reference time period less than or equal to 3 seconds, the fourth lane change warning information is generated to prompt the driver to pay attention to the oncoming vehicle, for example, by playing the fourth lane change warning information through voice, or by displaying the fourth lane change warning information on the vehicle's display screen, or by indicating the fourth lane change warning information through the indicator light on the vehicle. The way of prompting the driver to pay attention to the oncoming vehicle is not limited in the embodiment of the application. No matter which way is used to prompt the driver to pay attention to the oncoming vehicle, the effect of warning the driver can be achieved until the driver's head deflection angle remains within the deflection angle threshold range for a reference time period greater than 3 seconds. The purpose of the reminder in the step of the embodiment of the application is to prompt the driver to operate, so as to regulate the driver's lane change habit and improve the safety of the driver during driving.

[0063] In step 205, when the reference time period reaches the time period threshold, the merging assistant system collects information of moving objects around the vehicle, determines the first lane change warning information according to the collected information of moving objects, and performs lane change warning prompt based on the first lane change warning information.

[0064] According to the comparison and prompt operation in step 204, when the reference time period reaches the time period threshold, the operation of the merging assistant system is started.

[0065] The merging assistant system is a safety auxiliary device that can help the driver to change lanes more safely and conveniently. The system perceives the situation of the vehicles on both sides of the lane and provides relevant auxiliary information to assist the driver in completing the lane change operation. The merging assistant system includes at least one of a sensor installed on the vehicle, a camera installed on the rearview mirror of the vehicle, and a radar installed on the vehicle.

[0066] The merging assistant system determines the first lane change warning information. When the vehicle needs to perform merging operation, the vehicle sensor will immediately start scanning the surrounding environment of the vehicle to obtain information such as the state of the surrounding vehicles, the road conditions, and the traffic signs. If a dangerous situation is detected, such as a blue warning light or a prompt sound, the system will issue a corresponding timely warning to the driver to remind the driver to pay attention to traffic safety.

[0067] For example, after the information of the moving object is collected, it is determined according to the vehicle processor that the moving object is in the left rear lane of the vehicle and is approaching the vehicle, and the moving object in the left rear lane of the vehicle and approaching the vehicle is regarded as being able to affect the lane changing behavior of the vehicle, and a subsequent reminding operation is performed.

[0068] The first lane changing warning information is determined according to the collected information of the moving object, and the first lane changing warning information is used to prompt attention to the moving object. The driver is prompted to pay attention to the rear vehicle based on the first lane changing warning information. The first lane changing warning information can be played through voice, or the first lane changing warning information can be displayed through the display screen of the vehicle, or the first lane changing warning information can be indicated through the indicator light on the vehicle. The first warning information is generated until the driver receives the first lane changing warning information and completes the lane changing operation. The embodiment of the application does not limit the way of lane changing warning prompt. No matter which way is used to prompt the lane changing warning information, the driver can be prompted to pay attention to the rear vehicle, the driving habit of the driver can be standardized, and the driver can complete the lane changing operation more safely during the driving of the vehicle.

[0069] In summary, the lane changing warning method provided by the embodiment of the application determines the lane changing direction of the vehicle according to the opening signal of the turn signal of the vehicle; collects the head image information of the driver of the vehicle, and obtains the head deflection direction of the driver through the head image information of the driver; in the case that the head deflection direction of the driver matches the lane changing direction, the head deflection angle of the driver is determined according to the head image information of the driver; in the case that the head deflection angle reaches the deflection angle threshold, the reference time length during which the head deflection angle of the driver reaches the deflection angle threshold is determined; in the case that the reference time length reaches the time threshold, the first lane changing warning information is determined in combination with the lane merging assistance system, and the lane changing warning prompt is performed based on the first lane changing warning information. The lane changing warning prompt is performed in combination with the lane merging assistance system, the safety of the driver of the vehicle is ensured, and the reliability and accuracy of the lane changing warning are improved.

[0070] For ease of understanding, the method provided by the embodiment of the application is illustrated by taking the flow shown in Figure 4 The method provided by the embodiment of the application is illustrated by taking the flow shown in Figure 4As shown, the lane change warning process includes the following steps 410, steps 420-426, steps 430-436. Among them, the implementation of steps 410, steps 420, steps 430 can refer to the related description of step 201 above, the implementation of steps 421, steps 422, steps 431 and steps 432 can refer to the related description of step 202 above, the implementation of steps 423, steps 433, steps 426 and steps 436 can refer to the related description of step 203 above, the implementation of steps 424, steps 434, steps 426 and steps 436 can refer to the related description of step 204 above, the implementation of steps 425 and steps 435 can refer to the related description of step 205 above, which will not be repeated here.

[0071] Referring to Figure 5 The embodiment of the present application provides a lane change warning device, which comprises:

[0072] The first determination module 501 is configured to determine the lane change direction of the vehicle according to the opening signal of the turn signal of the vehicle.

[0073] The acquisition module 502 is configured to acquire the head image information of the driver of the vehicle, and obtain the head deflection direction of the driver through the head image information of the driver.

[0074] The second determination module 503 is configured to determine the head deflection angle of the driver according to the head image information of the driver in the case that the head deflection direction of the driver matches the lane change direction.

[0075] The third determination module 504 is configured to determine the reference time length for the head deflection angle of the driver to reach the deflection angle threshold in the case that the head deflection angle reaches the deflection angle threshold.

[0076] The warning module 505 is configured to, in the case that the reference time length reaches the time threshold, combine the information of the moving object around the vehicle collected by the lane merging assistance system, the information of the moving object including at least one of the number, speed, direction and distance of the moving object from the vehicle, and the lane merging assistance system including at least one of the sensor installed on the vehicle, the camera installed on the rearview mirror of the vehicle and the radar installed on the vehicle; determine the first lane change warning information according to the collected information of the moving object, and perform lane change warning prompt based on the first lane change warning information, the first lane change warning information being used to prompt attention to the moving object.

[0077] In a possible implementation, the warning module 505 is further configured to, in the case that the head deflection direction of the driver does not match the lane change direction, generate the second lane change warning information, and perform lane change warning prompt based on the second lane change warning information, the second lane change warning information being used to prompt the driver to pay attention to the lane change direction. In a possible implementation, the warning module 505 is further configured to, in the case that the head deflection direction of the driver does not match the lane change direction, generate the second lane change warning information, and perform lane change warning prompt based on the second lane change warning information, the second lane change warning information being used to prompt the driver to pay attention to the lane change direction.

[0078] In a possible implementation, the early warning module 505 is further configured to generate third lane change early warning information when the head deflection angle of the driver does not reach the deflection angle threshold, and perform a lane change early warning prompt based on the third lane change early warning information, the third lane change early warning information being used to prompt the driver to pay attention to the oncoming vehicle.

[0079] In a possible implementation, the early warning module 505 is further configured to generate fourth lane change early warning information when the reference time length during which the head deflection angle of the driver reaches the deflection angle threshold does not reach the time length threshold, and perform a lane change early warning prompt based on the fourth lane change early warning information, the fourth lane change early warning information being used to prompt the driver to pay attention to the oncoming vehicle.

[0080] In a possible implementation, the early warning module 505 is configured to play the first lane change early warning information through voice, or display the first lane change early warning information through a display screen of the vehicle, or indicate the first lane change early warning information through an indicator light on the vehicle.

[0081] It should be noted that the apparatus provided in the above embodiments is only used as an example for the division of the above functional modules in realizing the functions thereof, and in actual applications, the above functions can be completed by different functional modules according to the needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the above described functions. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0082] Figure 6 FIG. 1 is a structural schematic diagram of a server provided by an embodiment of the present application. The server can be quite different due to different configurations or performances, and can include one or more processors 1101 and one or more memories 1102, wherein the one or more memories 1102 store at least one computer program, the at least one computer program is loaded and executed by the one or more processors 1101, so that the server implements the lane change early warning method provided by each method embodiment described above. Of course, the server can also have a wired or wireless network interface, a keyboard, and an input and output interface, and other components for realizing the functions of the apparatus, which will not be described here.

[0083] Figure 7 FIG. 1 is a structural schematic diagram of a terminal provided by an embodiment of the present application. The terminal can be, for example, a smart phone, a tablet computer, a player, a notebook computer, or a desktop computer. The terminal can also be referred to as user equipment, a portable terminal, a laptop terminal, a desktop terminal, or other names.

[0084] Generally, the terminal includes a processor 1501 and a memory 1502.

[0085] The processor 1501 can include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 1501 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor 1501 can also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit), and the coprocessor being a low-power processor for processing data in a standby state. In some embodiments, the processor 1501 can be integrated with a GPU (Graphics Processing Unit) for rendering and drawing content required to be displayed by the display screen. In some embodiments, the processor 1501 can further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

[0086] The memory 1502 can include one or more computer-readable storage media that can be non-transitory. The memory 1502 can also include a high-speed random access memory, and a nonvolatile memory such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1502 is used to store at least one instruction for being executed by the processor 1501 to enable the terminal to implement the lane change warning method provided by the method embodiments in the present application.

[0087] In some embodiments, the terminal can also optionally include a peripheral device interface 1503 and at least one peripheral device. The processor 1501, the memory 1502, and the peripheral device interface 1503 can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 1503 through a bus, a signal line, or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 1504, a display screen 1505, a camera assembly 1506, an audio circuit 1507, and a power supply 1508.

[0088] The peripheral interface 1503 can be used to connect at least one I / O (Input / Output) related peripheral device to the processor 1501 and the memory 1502. In some embodiments, the processor 1501, the memory 1502 and the peripheral interface 1503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1501, the memory 1502 and the peripheral interface 1503 can be implemented on a separate chip or circuit board, and the present embodiments are not limited in this regard.

[0089] The radio frequency circuit 1504 is configured to receive and send RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1504 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 1504 converts electrical signals into electromagnetic signals for transmission, or converts electromagnetic signals received into electrical signals. Optionally, the radio frequency circuit 1504 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and the like. The radio frequency circuit 1504 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1504 can also include NFC (Near Field Communication) related circuitry, and the present application is not limited in this regard.

[0090] The display 1505 is configured to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display 1505 is a touch display, the display 1505 is further configured to collect touch signals on or above the surface of the display 1505. The touch signals can be input to the processor 1501 as control signals for processing. At this time, the display 1505 can also be configured to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display 1505 can be one, arranged on the front panel of the terminal; in other embodiments, the display 1505 can be at least two, arranged on different surfaces of the terminal or in a folding design; in other embodiments, the display 1505 can be a flexible display, arranged on a curved surface or a folding surface of the terminal. Even, the display 1505 can also be arranged in an irregular shape, that is, a special-shaped screen. The display 1505 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0091] The camera assembly 1506 is configured to capture images or videos. Optionally, the camera assembly 1506 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, the rear camera is at least two, which is any one of a main camera, a depth-of-field camera, a wide-angle camera, and a long-focus camera, to realize the background blur function of the main camera and the depth-of-field camera, the panoramic shooting and VR (Virtual Reality) shooting function of the main camera and the wide-angle camera, or other fusion shooting functions. In some embodiments, the camera assembly 1506 can also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. The dual-color temperature flash refers to the combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0092] The audio circuit 1507 can include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into an electrical signal input to the processor 1501 for processing, or input to the radio frequency circuit 1504 to realize voice communication. For the purpose of stereo sound collection or noise reduction, the microphone can be multiple, respectively arranged at different parts of the terminal. The microphone can also be an array microphone or an omnidirectional collection type microphone. The speaker is used to convert the electrical signal from the processor 1501 or the radio frequency circuit 1504 into sound waves. The speaker can be a traditional diaphragm speaker, or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only can the electrical signal be converted into a sound wave that humans can hear, but also can be converted into a sound wave that humans cannot hear for ranging purposes. In some embodiments, the audio circuit 1507 can also include a headphone jack.

[0093] The power supply 1508 is used to supply power to each component in the terminal. The power supply 1508 can be alternating current, direct current, disposable battery or rechargeable battery. When the power supply 1508 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0094] In some embodiments, the terminal also includes one or more sensors 1509. The one or more sensors 1509 include but are not limited to: an acceleration sensor 1510, a gyroscope sensor 1511, a pressure sensor 1512, an optical sensor 1513, and a proximity sensor 1514.

[0095] The acceleration sensor 1510 can detect the acceleration in three coordinate axes of the coordinate system established by the terminal. For example, the acceleration sensor 1510 can be used to detect the components of the gravitational acceleration in three coordinate axes. The processor 1501 can control the display screen 1505 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 1510. The acceleration sensor 1510 can also be used for game or user motion data collection.

[0096] The gyroscope sensor 1511 can detect the body direction and rotation angle of the terminal, and the gyroscope sensor 1511 can collect 3D actions of the user on the terminal in cooperation with the acceleration sensor 1510. The processor 1501 can realize the following functions according to the data collected by the gyroscope sensor 1511: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization when shooting, game control, and inertial navigation.

[0097] The pressure sensor 1512 can be disposed on the side bezel of the terminal and / or the lower layer of the display screen 1505. When the pressure sensor 1512 is disposed on the side bezel of the terminal, it can detect the user's grip signal on the terminal, and the processor 1501 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1512. When the pressure sensor 1512 is disposed on the lower layer of the display screen 1505, the processor 1501 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 1505. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0098] Optical sensor 1513 is used to collect ambient light intensity. In one embodiment, processor 1501 can control the display brightness of display screen 1505 based on the ambient light intensity collected by optical sensor 1513. Specifically, when the ambient light intensity is high, the display brightness of display screen 1505 is increased; when the ambient light intensity is low, the display brightness of display screen 1505 is decreased. In another embodiment, processor 1501 can also dynamically adjust the shooting parameters of camera assembly 1506 based on the ambient light intensity collected by optical sensor 1513.

[0099] The proximity sensor 1514, also known as a distance sensor, is typically installed on the front panel of the terminal. The proximity sensor 1514 is used to detect the distance between the user and the front of the terminal. In one embodiment, when the proximity sensor 1514 detects that the distance between the user and the front of the terminal is gradually decreasing, the processor 1501 controls the display screen 1505 to switch from a screen-on state to a screen-off state; when the proximity sensor 1514 detects that the distance between the user and the front of the terminal is gradually increasing, the processor 1501 controls the display screen 1505 to switch from a screen-off state to a screen-on state.

[0100] Those skilled in the art will understand that Figure 7 The structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0101] In an exemplary embodiment, a computer device is also provided, comprising a processor and a memory storing at least one computer program. The at least one computer program is loaded and executed by one or more processors to enable the computer device to implement any of the lane change warning methods described above.

[0102] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one computer program that is loaded and executed by a processor of a computer device to enable the computer to implement any of the lane change warning methods described above.

[0103] In a possible implementation manner, the computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0104] In the example embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device performs any one of the lane-changing warning methods described above.

[0105] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, the lane-changing warning information involved in the present application is obtained under full authorization.

[0106] It should be understood that "multiple" referred to in the present application refers to two or more. The "and / or" describes the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship.

[0107] The above only describes example embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A lane change warning method, characterized by, The method comprises: determining a lane-changing direction of the vehicle according to an opening signal of a turn signal of the vehicle; acquiring head image information of a driver of the vehicle, and determining whether a head deflection direction of the driver is left or right through the head image information of the driver; in a case where the head deflection direction of the driver matches the lane-changing direction, determining a head deflection angle of the driver according to the head image information of the driver, the head deflection angle including deflection angles in three directions of left-right swing, up-down tilt and left-right skew; in a case where the head deflection angle reaches a deflection angle threshold, determining a reference time length during which the head deflection angle of the driver reaches the deflection angle threshold; in a case where the reference time length reaches a time length threshold, combining information of a moving object around the vehicle collected by a lane merging assistance system, the information of the moving object including at least one of a quantity, a speed, a direction of the moving object and a distance between the moving object and the vehicle, the lane merging assistance system including at least one of a sensor installed on the vehicle, a camera installed on a rearview mirror of the vehicle and a radar installed on the vehicle; determining first lane-changing warning information according to the collected information of the moving object, the first lane-changing warning information being used to prompt attention to the moving object; performing lane-changing warning prompting based on the first lane-changing warning information. The method further comprises: in a case where the head deflection direction of the driver does not match the lane-changing direction, generating second lane-changing warning information, performing lane-changing warning prompting based on the second lane-changing warning information until the head deflection direction of the driver is adjusted to match the lane-changing direction or a same-direction turn signal button on the same side as the head deflection direction is opened, the second lane-changing warning information being used to prompt the driver to pay attention to the lane-changing direction.

2. The method of claim 1, wherein, The method further comprises: in a case where the head deflection angle of the driver does not reach the deflection angle threshold, generating third lane-changing warning information, performing lane-changing warning prompting based on the third lane-changing warning information, the third lane-changing warning information being used to prompt the driver to pay attention to a vehicle coming from behind.

3. The method of claim 1, wherein, The method further comprises: in a case where the reference time length during which the head deflection angle of the driver reaches the deflection angle threshold does not reach the time length threshold, generating fourth lane-changing warning information, performing lane-changing warning prompting based on the fourth lane-changing warning information, the fourth lane-changing warning information being used to prompt the driver to pay attention to the vehicle coming from behind.

4. The method according to any of claims 1 to 3, characterized in that, The lane-changing warning prompting based on the first lane-changing warning information comprises: playing the first lane-changing warning information through voice, or displaying the first lane-changing warning information through a display screen of the vehicle, or indicating the first lane-changing warning information through an indicator light on the vehicle.

5. A lane change warning device characterized by comprising: The device comprises: a first determining module configured to determine a lane-changing direction of the vehicle according to an opening signal of a turn signal of the vehicle; an acquiring module configured to acquire head image information of a driver of the vehicle, and determine whether a head deflection direction of the driver is left or right through the head image information of the driver; The second determining module is configured to determine a head deflection angle of the driver according to the head image information of the driver in a case where the head deflection direction of the driver matches the lane changing direction, the head deflection angle including deflection angles in three directions of left-right shaking, up-down tilting, and left-right tilting. The third determining module is configured to determine a reference time length during which the head deflection angle of the driver reaches the deflection angle threshold in a case where the head deflection angle reaches the deflection angle threshold. The warning module is configured to, in a case where the reference time length reaches a time threshold, acquire information of a moving object around the vehicle in combination with a lane merging assistance system, the information of the moving object including at least one of a number, a speed, a direction, and a distance of the moving object relative to the vehicle, the lane merging assistance system including at least one of a sensor installed on the vehicle, a camera installed on a rearview mirror of the vehicle, and a radar installed on the vehicle; determine first lane changing warning information according to the acquired information of the moving object, and perform lane changing warning prompting based on the first lane changing warning information, the first lane changing warning information being used to prompt attention to the moving object. The warning module is further configured to, in a case where the head deflection direction of the driver does not match the lane changing direction, generate second lane changing warning information, and perform lane changing warning prompting based on the second lane changing warning information until the head deflection direction of the driver matches the lane changing direction or a same-direction turn signal button is turned on, the second lane changing warning information being used to prompt the driver to pay attention to the lane changing direction.

6. The apparatus of claim 5, wherein, The warning module is further configured to, in a case where the head deflection angle of the driver does not reach the deflection angle threshold, generate third lane changing warning information, and perform lane changing warning prompting based on the third lane changing warning information, the third lane changing warning information being used to prompt the driver to pay attention to a vehicle behind.

7. A computer device, comprising: The computer device includes a processor and a memory, and the memory stores at least one computer program, the at least one computer program is loaded and executed by the processor, so that the computer device implements the lane changing warning method according to any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by the processor, so that the computer implements the lane changing warning method according to any one of claims 1 to 4.

Images