Virtual lane line-based vehicle early warning method, program product and equipment

Abstract

The invention discloses a vehicle early warning method based on a virtual lane line, a program product and equipment. The method comprises the following steps: acquiring a current image acquired in a current scene; acquiring driving state data of the vehicle equipment; drawing a virtual lane line based on the current image and the driving state data; performing target detection on the current image, and identifying a current target in the current image and position data of the current target; determining a position relationship between each current target and the virtual lane line based on the position data of each current target and the virtual lane line; and executing early warning based on the position relationship between each current target and the virtual lane line.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a vehicle early warning method based on virtual lane lines, a computer program product, and vehicle equipment. Background Art

[0002] With the rapid development of automobile technology, assisted driving systems have become an important part of modern automobiles. These systems are designed to improve driving safety, reduce the incidence of accidents, improve driving convenience, and increase driving happiness through the combination of various sensors and algorithms. The demand for assisted driving systems stems from multiple factors. First, the frequent occurrence of traffic accidents has made automobile safety a focus of attention. The system can significantly reduce the risk of accidents by providing real-time monitoring and early warning; secondly, with the increase in urban traffic pressure, the psychological and physical burden faced by drivers is also increasing. The emergence of this system can alleviate this burden to a certain extent, making driving easier and more comfortable. In addition, the intelligent functions of the assisted driving system, such as front vehicle reminders, pedestrian reminders, and animal reminders, not only improve driving safety, but also promote the further development of intelligent transportation.

[0003] In complex road environments, the clarity of lane lines may be insufficient or even non-existent on some roads, making it difficult for drivers to accurately judge the boundaries and positions of lanes, or the relative positions of targets and the front of the vehicle, which reduces the reliability of detection and makes it impossible to provide timely warnings. Summary of the invention

[0004] In order to solve the existing technical problems, the present invention provides a vehicle warning method based on virtual lane lines, a computer program product and a vehicle device, which can provide clear road boundaries and can perform early warning on dangerous targets in a timely manner.

[0005] In a first aspect, a vehicle warning method based on virtual lane lines is provided, comprising: obtaining a current image captured in a current scene; obtaining driving status data of the vehicle equipment; drawing virtual lane lines based on the current image and the driving status data; performing target detection on the current image to identify the current target in the current image and the position data of the current target; determining the positional relationship between each current target and the virtual lane line based on the position data of each current target and the virtual lane line; and executing a warning alert based on the positional relationship between each current target and the virtual lane line.

[0006] In a second aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the vehicle warning method based on virtual lane lines as described in any one of the first aspect of the present application is implemented.

[0007] In a third aspect, a vehicle device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes a vehicle warning method based on virtual lane lines as described in any one of the first aspects of the present application.

[0008] The embodiment of the present application obtains the current image captured in the current scene and obtains the driving status data of the vehicle equipment; based on the current image and the driving status data, a virtual lane line is drawn, so that when there is no lane line in the current scene or the real lane line is unclear, the virtual lane line can be used to enable the user to perceive the driving trajectory in real time during driving, and can provide the driver with a clear road boundary and motion trajectory, helping the driver to better keep the vehicle equipment driving in the center of the lane. This visual guidance can improve the driver's spatial perception ability, reduce the risk of lane deviation, and thus improve driving safety; target detection is performed on the current image, and the current target in the current image and the position data of the current target are identified, so that the spatial position relationship between each current target and the virtual lane line can be obtained. Since the virtual lane line indicates the lane area in the front area of ​​the vehicle equipment where the vehicle equipment is about to travel, according to the spatial position relationship between each current target and the virtual lane line, the potential dangerous target in front of the vehicle equipment can be discovered in time, thereby timely warning, improving driving safety, and by processing the current image, real-time and timely warning can be guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 Schematic diagram of the application environment of a vehicle warning method based on virtual lane lines in one embodiment;

[0010] Figure 2 is a schematic block diagram of a vehicle device in one embodiment;

[0011] Figure 3 Flow chart of a vehicle warning method based on virtual lane lines in one embodiment;

[0012] Figure 4 A schematic diagram of displaying a virtual lane line in an embodiment;

[0013] Figure 5 is a flow chart of a vehicle warning method based on virtual lane lines in another embodiment;

[0014] Figure 6 A schematic diagram of a vehicle warning device based on virtual lane lines in an embodiment;

[0015] Figure 7 Schematic diagram of vehicle equipment in one embodiment. DETAILED DESCRIPTION

[0016] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the technical field of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0018] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0019] See also Figure 1 , is an application environment diagram of a vehicle warning method based on virtual lane lines in an embodiment. The vehicle warning method based on virtual lane lines is applied to a vehicle device 10, and the vehicle device 10 includes a display terminal 11, a sensor module 12, and a controller 13. The sensor module 12 is used to collect data during the driving process of the vehicle device 10. The controller 13 draws virtual lane lines based on the image data collected by the sensor module 12 and the driving state data of the vehicle device 10, and detects the current target in the image data, and performs a warning alert based on the positional relationship between the current target and the virtual lane line.

[0020] The sensor module 12 may be a module integrating one or more sensors, and the sensor module 12 may be one or more, and may be installed at different positions of the vehicle equipment 10. The sensor module 12 includes but is not limited to a multi-spectral visual sensor, an environmental perception sensor, and a motion posture sensor. The multi-spectral sensor includes but is not limited to a combination of one or more of the following sensors: thermal imaging sensor, visible light image sensor, millimeter wave sensor, lidar sensor, infrared thermal imaging sensor, and depth sensor. The environmental perception sensor includes but is not limited to a combination of one or more of the following sensors, such as brightness sensor, temperature sensor, haze sensor, and other environmental sensors. The motion posture sensor includes but is not limited to a combination of one or more of the following: inertial sensor (I nert ia l Measurement Unit, I MU), velocity sensor, acceleration sensor, gyroscope sensor, geomagnetic sensor, rotation vector sensor, steering wheel angle sensor, level sensor, tilt sensor, vibration sensor, displacement sensor, gravity sensor, etc.

[0021] The screen is the surface on which the display terminal 11 displays images. The screen is arranged in front of the driver so that the driver can view the images. The display terminal 11 includes but is not limited to displays such as liquid crystal displays, projectors with projector screens, etc. The display terminal 11 can be a display located on the central control of the vehicle device 10, or a projector arranged at the rear of the vehicle device 10 to project images onto the projector screen.

[0022] The controller 13 may be one or more. When there are multiple controllers 13, the multiple controllers may be integrated on one chip or independently set on each chip. The vehicle device 10 is a device installed on any type of mobile body, such as a vehicle, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobile device, an airplane, a drone, a ship or a robot, etc. In some embodiments, the controller 13 and the display terminal 11 may be integrated.

[0023] like Figure 2 As shown, Figure 2 1 is a block diagram of a vehicle device in an embodiment. The vehicle device 10 includes a display terminal 11, a controller 13, an image acquisition device 14 and a motion posture sensor 15. The controller 13 is respectively connected to the display terminal 11, the image acquisition device 14 and the motion posture sensor 15 for communication. The image acquisition device 14 is used to acquire image data during the motion of the vehicle device 10, and the motion posture sensor 16 is used to acquire data such as speed, posture, acceleration, gravity, etc. during the motion of the vehicle device 10.

[0024] The image acquisition device 14 may be a combination of one or more sensor modules 12. The image acquisition device may be a monocular vision sensor or a multi-eye vision sensor. For example, it may be a combination of one or more sensors among a thermal imaging sensor, a visible light image sensor, a millimeter wave sensor, a laser radar sensor, an infrared thermal imaging sensor and a depth sensor. The motion posture sensor 15 includes but is not limited to a combination of one or more of the following: an inertial sensor (I nertia l Measurement Unit, I MU), a velocity sensor, an acceleration sensor, a gyroscope sensor, a geomagnetic sensor, a rotation vector sensor, a steering wheel angle sensor, a level sensor, a tilt sensor, a vibration sensor, a displacement sensor and a gravity sensor, etc.

[0025] See also Figure 3 , is a flow chart of a vehicle warning method based on virtual lane lines provided in an embodiment of the present application. The vehicle warning method based on virtual lane lines is applied to a vehicle device, and the vehicle warning method based on virtual lane lines includes the following steps:

[0026] S11, obtaining a current image captured in a current scene.

[0027] In this embodiment, the current image may be an infrared image, or an infrared image and a visible light image. The current image may be a current frame image, or a continuous frame image. By processing in a real-time frame image, the subsequent real-time performance and the timeliness of the early warning can be guaranteed. Traditional visible light cameras are widely used in assisted driving because they can capture light visible to the human eye, thereby providing rich visual information. However, traditional visible light cameras have some significant shortcomings. They are very sensitive to lighting conditions, and their performance is greatly reduced in strong light, backlight or low light environments, and images are prone to blur or distortion, affecting the reliability of the system; bad weather such as rainy days and haze will also significantly reduce the effect of visible light cameras and fail to provide accurate environmental perception. However, infrared cameras are not limited by visible light conditions and can effectively penetrate obstacles such as rain and haze to ensure normal operation in bad weather. Therefore, infrared image sensors have shown significant advantages in assisted driving systems. Infrared image sensors can work in the full spectrum range, especially in low light or night environments, and can still provide clear images. This makes infrared image sensors have higher accuracy and reliability in identifying vehicles, pedestrians, animals and other obstacles. In addition, infrared cameras are not limited by visible light conditions and can effectively penetrate obstacles such as rain, fog and haze to ensure normal operation even in bad weather. Therefore, the application of infrared images in the field of assisted driving, or the application of infrared image technology and visible light image technology in the field of assisted driving, can enhance environmental perception and improve driving safety.

[0028] In some embodiments, when the image acquisition device is started, it will start to acquire the current image within the range of the image acquisition device in real time. The current image is first preprocessed by image signal processing (ISP) to enhance the quality and details of the image so that the infrared display vision is more comfortable and the target detection is more accurate. The preprocessing includes but is not limited to denoising, contrast enhancement, brightness adjustment, histogram equalization, edge sharpening, and image size adjustment. The preprocessing operation can improve the image quality and enhance the details and edge information of the image. By adjusting the image size operation, the size of the infrared image and the visible light image is adjusted to the same.

[0029] The pre-processed images are processed in two ways: one for real-time display and the other for input to the program corresponding to the vehicle warning method based on virtual lane lines, that is, input to the target detection model for analysis and processing to extract effective information and features.

[0030] S12, obtaining driving status data of vehicle equipment.

[0031] In this embodiment, the driving state data indicates the movement of the vehicle during driving. The driving state data can be data collected by one or more sensors, or data calculated from data collected by one or more sensors. The driving state data includes but is not limited to angular velocity data, speed data, acceleration data, yaw angle, etc. The angular velocity data can be measured by an inertial sensor, the speed data can be measured by a speed sensor, the acceleration data can be measured by an acceleration sensor, and the yaw angle can be measured by a gyroscope sensor or an inertial sensor. Among them, the acceleration data can determine whether the vehicle equipment is stationary or driving, and at the same time, the speed and displacement information of the vehicle equipment can be calculated. The angular velocity data can determine the direction and posture of the vehicle equipment, and determine whether the vehicle is turning or going straight and changing lanes. The yaw angle is used to describe the rotation angle of the vehicle equipment relative to the reference direction.

[0032] S13: Draw virtual lane lines based on the current image and driving status data.

[0033] In this embodiment, the virtual lane line indicates the lane area in the front area of ​​the vehicle device where the vehicle device is about to travel. Then the lane area indicated by the virtual lane line also includes the predicted driving trajectory of the vehicle device. Based on the driving state data, the driving trajectory of the vehicle device is predicted using the trajectory prediction method, that is, the trajectory points of the vehicle device in the future period of time are obtained, and then the multiple trajectory points are converted to the image coordinate system, and the virtual lane line is drawn in the current image based on the image coordinates of the multiple trajectory points.

[0034] In this way, in the front area of ​​the vehicle equipment, even if there are no lane lines in the current scene, such as in rural roads, or when the real lane lines in the current image are not clear, such as in bad weather, the real lane lines in the current image are not clear, through the embodiments of the present application, virtual lane lines can be drawn in the current image, and the virtual lane lines can be used as a driving reference for the vehicle equipment.

[0035] S14: Perform target detection on the current image, identify current targets in the current image, and determine the positional relationship between each current target and the virtual lane line.

[0036] In this embodiment, input data of a pre-trained target detection model is formed based on the current image, a feature map is extracted from the input data by the target detection model, and recognition data is output based on the extracted feature map. The recognition data includes but is not limited to at least one of the following: position data of the current target in the current image, target category, and confidence of the current target. The position data includes but is not limited to the position data of the target box, the boundary coordinates of the target box, and the like. The targets are divided into at least one of the following according to different categories: pedestrians, cyclists, animals, buses, cars, and other types of vehicles, and the confidence of the current target represents the confidence score of the target detection model for each current target, reflecting the reliability of the recognition. The boundary coordinates indicate the position of the target box of the current target in the current image.

[0037] The target detection model is obtained based on the training data set. Each training sample in the training data set includes a training image and the corresponding label data in the training image. The label data includes the location data of the sample target and the category of the sample target. The sample targets include pedestrians, cyclists, animals, buses, cars and other types of vehicles.

[0038] In an optional method, a training image is obtained from a training data set, and sample input data of a target detection model in a current iteration is formed based on the training image; the target detection model in the current iteration extracts sample feature data from the sample input data, identifies the target data in the current iteration based on the sample feature data, and calculates the loss value between the target data in the current iteration and the label data corresponding to the training image based on the target data in the current iteration, the loss function and the label data corresponding to the training image, to obtain the loss value of the current iteration; based on the loss value of the current iteration, it is determined whether the current iteration satisfies the iteration termination condition; if the current iteration does not meet the iteration termination condition, training images continue to be obtained from the training data set, and the target detection model is iteratively trained; if the current iteration meets the iteration termination condition, the target detection model after stopping the iteration is used as the pre-trained target detection model.

[0039] S15. Based on the position data of each current target and the virtual lane line, determine the positional relationship between each current target and the virtual lane line.

[0040] In this embodiment, the position relationship indicates the relationship between the current target and the virtual lane area indicated by the virtual lane line, and the position relationship includes but is not limited to: the current target is completely located in the virtual lane area, the current target is partially located in the virtual lane area, and the current target is not located in the virtual lane area. Since the virtual lane line is drawn in the current image, the position data of the current target is also the position data of the target in the current image. Therefore, based on the position data of the current target and the position range of the virtual lane area, the position relationship between each current target and the virtual lane line can be determined.

[0041] S16. Execute an early warning based on the positional relationship between each current target and the virtual lane line.

[0042] In this embodiment, based on the positional relationship between each current target and the virtual lane line, potential dangerous targets can be screened out from the current targets, and early warning warnings can be issued based on the potential dangerous targets.

[0043] It can be understood that, in the above steps, steps S11 and S12 may be executed in any order, and steps S13 and S14 may be executed in any order.

[0044] In one application scenario, the combination of infrared imaging and virtual lane line virtual auxiliary display technology can not only improve the visualization of the image, but also make up for the lack of detail in infrared imaging to a certain extent. By providing more intuitive and clear lane information through virtual lane lines, this method can significantly improve driving safety and driver confidence, especially at night or in bad weather conditions, ensuring the safe driving of the vehicle.

[0045] In the above embodiment, the current image captured in the current scene and the driving status data of the vehicle device are obtained; based on the current image and the driving status data, a virtual lane line is drawn, so that when there is no lane line in the current scene or the real lane line is unclear, the virtual lane line can be used to enable the user to perceive the driving trajectory in real time during driving, and can provide the driver with a clear road boundary and motion trajectory, helping the driver to better keep the vehicle device driving in the center of the lane. This visual guidance can improve the driver's spatial perception ability, reduce the risk of lane deviation, and thus improve driving safety; target detection is performed on the current image, and the current target in the current image and the position data of the current target are identified, so that the spatial position relationship between each current target and the virtual lane line can be obtained. Since the virtual lane line indicates the lane area in the front area of ​​the vehicle device where the vehicle device is about to travel, according to the spatial position relationship between each current target and the virtual lane line, the potential dangerous target in front of the vehicle device can be discovered in time, thereby timely warning, thereby improving driving safety; and by processing the current image, real-time and timely warning can be guaranteed.

[0046] In some embodiments, drawing a virtual lane line based on the current image and the driving status data includes:

[0047] Acquire virtual lane line width data corresponding to the current scene;

[0048] Based on the driving state data, predicting the movement trajectory of the vehicle equipment;

[0049] The virtual lane line is drawn based on the virtual lane line width data, the motion trajectory and the current image.

[0050] In this implementation, the virtual lane width data corresponds to the road scene type, and different virtual lane width data can be configured for different road scene types, where the virtual lane width data can be a specific value or a range. In the urban road scene, the configuration range of the lane width of the urban road scene is 3.0m to 3.5m; in the high-speed highway scene, the configuration range of the lane width of the highway is 3.75m to 4m; in the rural road scene, the configuration range of the lane width of the rural road is 2.5m to 3.0m.

[0051] In this embodiment, based on the driving state data, the driving trajectory of the vehicle equipment is predicted using a trajectory prediction method, that is, the trajectory points of the vehicle equipment in the future period of time are obtained, and then multiple trajectory points are converted to the image coordinate system. Based on the image coordinates of the multiple trajectory points, the driving trajectory is determined, and the extension direction of the virtual lane line can be determined according to the extension direction of the driving trajectory; the forward direction along the center line of the vehicle equipment and pointing to the vehicle equipment is determined as the center line of the virtual lane line; based on the extension direction of the virtual lane line, the center line of the virtual lane line and the virtual lane line width data, the virtual lane line is drawn on the current image. Since the virtual lane line is an extension direction determined based on the predicted driving trajectory, compared with the real lane line, the virtual lane line can better represent the lane area where the vehicle equipment is about to travel. For example, when the vehicle equipment is about to turn according to the motion state data of the vehicle equipment, the virtual lane line when the vehicle equipment turns is also a curve. The extension direction of the virtual lane line is consistent with the extension direction of the predicted trajectory.

[0052] In the above embodiment, virtual lane line width data corresponding to the current scene can be obtained according to the road scene type of the current scene; the movement trajectory of the vehicle equipment is predicted based on the driving status data; and virtual lane lines are drawn in the current image based on the virtual lane line width data and the movement trajectory. Since the virtual lane lines are extended in directions determined based on the predicted driving trajectory, compared with real lane lines, virtual lane lines can better represent the lane area where the vehicle equipment is about to travel. The driver can more intuitively and accurately determine the driving trajectory of the vehicle equipment in different scenarios, so as to make better decisions and thus improve driving safety.

[0053] In some embodiments, the width of the virtual lane line can be configured by the driver on a user interface displayed on a display terminal, and a specification selection control is displayed on the user interface, through which the desired lane width can be selected. The acquisition of virtual lane line width data corresponding to the current scene includes:

[0054] Based on a trigger operation on a specification selection control displayed on a user interface, lane line width data corresponding to the trigger operation is acquired, and the lane line width data corresponding to the trigger operation is used as the virtual lane line width data.

[0055] In one implementation, the specification selection control provides a variety of road scene types, and displays the lane line width data corresponding to the road scene type through the specification selection control. The lane line width data corresponding to each road scene type is pre-configured, and the target road scene type corresponding to the trigger operation is obtained, and then the lane line width data corresponding to each pre-configured road scene type is obtained as the virtual lane line width data. The lane line width data can be a range or a specific value. For example, the lane line width data under three road scene types are pre-configured. In the urban road scene, the lane width of the urban road scene is 3.2m; in the high-speed highway scene, the lane width of the highway is 4m; in the rural road scene, the configuration range of the lane width of the rural road is 2.5m. When the user selects the urban road scene type through the specification selection control, the virtual lane line width data is 3.2m. In one implementation, lane line width data for a variety of different gears can be directly provided through a specification selection control. The user directly selects the lane line width data for the required gear and obtains the lane line width data corresponding to the trigger operation. For example, three gears can be directly displayed through the specification selection control, namely 3.0m to 3.5m, 3.75m to 4m, and 2.5m to 3.0m.

[0056] In this embodiment, the width specification of the virtual lane line is dynamically adjusted according to the lane selection function of the human-computer interaction interface. When the driver selects the type of road scene where the vehicle equipment is located (such as rural, urban and other roads), the corresponding type of virtual lane line data can be automatically provided according to the selected road scene type. Through this flexible setting, the driver can get more intuitive visual guidance in different scenarios, thereby improving driving safety. In addition, when selecting lane line specifications, users can customize settings according to different road conditions and personal driving habits, which not only increases driving flexibility, but also improves the user experience.

[0057] In an optional embodiment, the acquiring virtual lane line width data corresponding to the current scene includes:

[0058] According to the current image, identifying a current road scene type corresponding to the current scene;

[0059] According to the current road scene type, the lane line width data corresponding to the current road scene type is obtained from the pre-stored correspondence between the road scene type and the lane line width, and the lane line width data corresponding to the current road scene type is used as the virtual lane line width data.

[0060] In this embodiment, the current image can be input into a pre-trained scene detection model, and the current road scene type can be output. The scene detection model can be trained by a scene training data set. Each scene training data in the scene training data set includes a scene sample image and a road scene type label corresponding to the scene sample image. The scene training data is obtained from the scene training data set to train the scene detection model using an iterative training method. In this way, the scene detection model learns the features of various different scenes during the training process, and after the training is completed, it can automatically detect the road scene type of the current scene. The road scene types in the scene training data set include but are not limited to urban roads, rural roads, highways, etc. In this way, the vehicle equipment can automatically determine the current road scene type according to the current image collected during the driving process, and adaptively select the lane line width data that matches the current road scene type according to the current road scene type. When the road scene changes during the driving process, the lane line width data is automatically switched, and a virtual lane line matching the current road scene type is drawn.

[0061] In the above embodiment, based on the current image, the current road scene type corresponding to the current scene is identified, and the changes in the road scene type surrounding the vehicle equipment can be perceived in real time. The current road scene type corresponding to the current scene is analyzed by the collected current image, so as to select the road specification corresponding to the current road scene type, thereby automatically switching the virtual lane width data, realizing adaptive change of the width of the virtual lane line, making the virtual lane line more in line with the current driving environment of the vehicle equipment, thereby improving driving safety.

[0062] In some embodiments, the performing of early warning based on the positional relationship between each current target and the virtual lane line includes:

[0063] Determine a potential dangerous target based on a positional relationship between each current target and the virtual lane line;

[0064] Calculating the collision time between the potential dangerous target and the vehicle equipment;

[0065] When the collision time is less than the collision threshold, a warning alert is performed.

[0066] Optionally, the determining of the potential dangerous target based on the positional relationship between each current target and the virtual lane line includes at least one of the following:

[0067] When the current target is located in the virtual lane area indicated by the virtual lane line, determining that the current target is a potential dangerous target;

[0068] When the current target is partially located in the virtual lane area, and the portion located in the virtual lane area meets a preset condition, the current target is determined to be a potentially dangerous target.

[0069] In this embodiment, the potential dangerous target is determined based on the positional relationship between the current target and the virtual lane line. Since the virtual lane line indicates the lane area in the front area of ​​the vehicle device where the vehicle device is about to travel, the current target that is completely located in the virtual lane area is screened out from the current target, and / or the current target that is partially located in the virtual lane area and the part of the current target that meets the preset conditions in the virtual lane area is screened out as a potential dangerous target. The preset conditions include but are not limited to the ratio of the width of the part located in the virtual lane area to the width of the virtual lane area exceeding the preset width ratio. For example, if the preset width ratio is 50%, then the preset condition is that the width of the part located in the virtual lane area accounts for more than 50% of the width of the virtual lane area. Compared with the prior art, which only regards the target that appears in the front area of ​​the vehicle device as a potential dangerous target, the embodiment of the present application can more accurately screen out the potential dangerous target of the vehicle device. In an optional implementation, multiple different sensitivity levels are pre-configured, and different sensitivity levels correspond to different width ratios. The higher the sensitivity, the smaller the width ratio. For example, the sensitivity is divided into three levels: high, medium, and low. When the sensitivity level is high, the width ratio is greater than 30%, that is, the dangerous target is displayed. When the sensitivity is low, the width ratio is greater than 50%, and the dangerous target is displayed. A sensitivity selection control may be provided on the user interface, through which the user may select a desired target sensitivity level. Therefore, the selected target sensitivity level may be obtained based on the user interface, the corresponding width ratio may be obtained according to the target sensitivity level, and the width ratio corresponding to the target sensitivity level may be set as the preset width ratio.

[0070] In this embodiment, each current target detected will be tracked, and the driving status data of the vehicle equipment can also be obtained in real time. Therefore, according to the tracking data of the potential dangerous target, the moving speed of the potential dangerous target can be obtained; according to the tracking data of the potential dangerous target and the driving status data of the vehicle equipment, the lateral relative distance and longitudinal relative distance between the potential dangerous target and the vehicle equipment can be calculated. According to the driving speed of the vehicle equipment and the moving speed of the potential dangerous target, the relative moving speed can be calculated. According to the lateral relative distance, the longitudinal relative distance and the relative moving speed, the collision time between the potential dangerous target and the vehicle equipment can be calculated. The collision time corresponding to the potential dangerous target is compared with the warning threshold. When the collision time corresponding to the potential dangerous target is less than the collision threshold, the alarm information will be output in time to remind the driver to take corresponding measures. By calculating the information of the detected potential dangerous target and predicting the danger, when the collision time corresponding to the potential dangerous target is less than the collision threshold, the driver is reminded to reduce the occurrence of danger.

[0071] In the above embodiment, based on the positional relationship between each current target and the virtual lane line, a potential dangerous target is determined. Since the virtual lane line indicates the lane area in the front area of ​​the vehicle equipment where the vehicle equipment is about to travel, referring to the positional relationship between each current target and the virtual lane line, the potential dangerous targets of the vehicle equipment can be screened out more accurately, and the collision time corresponding to the potential dangerous target is compared with the warning threshold. When the collision time corresponding to the potential dangerous target is less than the collision threshold, an alarm message will be output in time to remind the driver to take corresponding measures, thereby improving driving safety.

[0072] In some embodiments, the performing of the early warning alert includes:

[0073] The potential dangerous target is highlighted on the current image, wherein the display mode of the potential dangerous target is different from the display mode of the non-potential dangerous target.

[0074] In this embodiment, some of the current targets identified in the current image may not be potential dangerous targets, that is, non-potential dangerous targets, and the potential dangerous targets may be displayed in a different display mode from the non-potential dangerous targets. For example, the potential dangerous targets are displayed in a red flashing frame, and the non-potential dangerous targets are only displayed in a non-red color, but not flashing.

[0075] In the above embodiment, the display mode of the potential dangerous target is different from the display mode of the non-potential dangerous target, which allows the driver to observe the potential dangerous target more intuitively, thereby improving driving safety.

[0076] In some embodiments, the method further comprises:

[0077] The virtual lane line is displayed in real time on the current image.

[0078] In this embodiment, the current image is a current frame image acquired in real time, so displaying virtual lane lines on the current image can allow the driver to more intuitively and accurately determine the driving trajectory of the vehicle equipment, such as Figure 4 As shown, Figure 4 Schematic diagram of display of virtual lane lines in an embodiment, in which virtual lane lines and pedestrian targets are displayed in a first display screen, and vehicle targets and virtual lane lines are displayed in a second display screen.

[0079] In the above embodiment, the virtual lane line is displayed in real time on the current image, which can provide the driver with clear road boundaries and motion trajectories so as to make better decisions, thereby improving driving safety.

[0080] In some embodiments, the method further comprises:

[0081] Get the target category of the current target;

[0082] According to the target category, a target frame of the current target is displayed in the current image with a target color corresponding to the target category.

[0083] In this embodiment, as described in the above embodiment, the target category of the current target can be output through the target detection model. In the current image, there are multiple target categories, and the target boxes corresponding to different target categories are displayed in different colors. For example, the pedestrian target box is displayed in yellow, the vehicle target box is displayed in blue, and so on.

[0084] In the above embodiment, different colors are used to represent different types of targets. This process not only improves visual intuitiveness, but also helps the driver to quickly judge potential risks, thereby improving driving safety.

[0085] In some embodiments, the method further comprises:

[0086] Extracting a road boundary of an actual road from the current image, and acquiring a positional relationship between the virtual lane line and the road boundary;

[0087] When the positional relationship between the virtual lane line and the road boundary meets a prompt condition, a prompt message is output, wherein the prompt condition indicates that the virtual lane line exceeds the road boundary or the distance between the virtual lane line and the road boundary is less than a preset distance value.

[0088] In this embodiment, the image processing method can be used to extract the boundary contour features of the actual road from the current image, and based on the boundary contour features, the road boundary in the current image is obtained. According to the position data of the virtual lane line and the position data of the road boundary, the positional relationship between the virtual lane line and the road boundary can be determined. When the virtual lane line exceeds the road boundary, it indicates that the vehicle device may travel beyond the road boundary, so a prompt message is output to prompt the driver to change the driving direction. When the distance between the virtual lane line and the road boundary is less than the preset distance value, it means that the virtual lane line is very close to the road boundary, and a prompt message is output to prompt the driver to change the driving direction.

[0089] In the above embodiment, the road boundary of the actual road is extracted from the current image, and when the positional relationship between the virtual lane line and the road boundary meets the prompt condition, prompt information is output to prompt the driver to pay attention to the driving direction and remind the driver to change the driving trajectory, thereby improving safety.

[0090] like Figure 5 As shown, Figure 5 Flowchart of a vehicle warning method based on virtual lane lines in one embodiment; the flowchart includes the following steps:

[0091] S21. Obtain a current image captured in a current scene.

[0092] S22, obtaining driving status data of vehicle equipment.

[0093] S23. Obtain virtual lane line width data corresponding to the current scene.

[0094] S24: Draw a virtual lane line based on the current image, the virtual lane line width data and the driving status data.

[0095] S25, performing target detection on the current image to determine whether there is a current target in the current image. When there is no current target in the current image, returning to continue executing S21; when there is a current target in the current image, executing S26.

[0096] S26. Based on the positional relationship between the current target and the virtual lane line, determine whether the current target is a potentially dangerous target. If the current target is a potentially dangerous target, execute S28. If the current target is not a potentially dangerous target, execute S27, increment the loop variable of the current target, and return to execute S26, i.e. process the next current target.

[0097] S28: Calculate the collision time between the potential dangerous target and the vehicle equipment.

[0098] S29, determine whether the collision time is less than the collision threshold. If the collision time is less than the collision threshold, execute S30; if the collision time is not less than the collision threshold, execute S27, the loop variable of the current target is incremented, and return to execute S26, that is, process the next current target.

[0099] S30: Execute early warning alert.

[0100] The above steps have been described in one or more of the above embodiments and will not be repeated here.

[0101] On the other hand, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the vehicle warning method based on virtual lane lines described in any embodiment of the present application.

[0102] Among them, in the computer program product, an optional implementation form of the program module architecture of the computer program that implements the various steps of the vehicle warning method based on virtual lane lines can be a vehicle warning device based on virtual lane lines.

[0103] See also Figure 6 An embodiment of the present application provides a vehicle warning device based on virtual lane lines, including: an acquisition module 61, used to acquire a current image captured in a current scene; the acquisition module 61 is also used to acquire driving status data of the vehicle equipment; a drawing module 62, used to draw virtual lane lines based on the current image and the driving status data; an identification module 63, used to perform target detection on the current image, identify the current target in the current image and the position data of the current target; a determination module 64, used to determine the position relationship between each current target and the virtual lane line based on the position data of each current target and the virtual lane line; an early warning module 65, used to perform early warning warnings based on the position relationship between each current target and the virtual lane line.

[0104] Optionally, the drawing module 62 is further used for:

[0105] Acquire virtual lane line width data corresponding to the current scene;

[0106] Based on the driving state data, predicting the movement trajectory of the vehicle equipment;

[0107] The virtual lane line is drawn based on the virtual lane line width data, the motion trajectory and the current image.

[0108] Optionally, the drawing module 62 is further used for:

[0109] Based on a trigger operation on a specification selection control displayed on a user interface, lane line width data corresponding to the trigger operation is acquired, and the lane line width data corresponding to the trigger operation is used as the virtual lane line width data.

[0110] Optionally, the drawing module 62 is further used for:

[0111] According to the current image, identifying a current road scene type corresponding to the current scene;

[0112] According to the current road scene type, the lane line width data corresponding to the current road scene type is obtained from the pre-stored correspondence between the road scene type and the lane line width data, and the lane line width data corresponding to the current road scene type is used as the virtual lane line width data.

[0113] Optionally, the early warning module 65 is further used for:

[0114] Determine a potential dangerous target based on a positional relationship between each current target and the virtual lane line;

[0115] Calculating the collision time between the potential dangerous target and the vehicle equipment;

[0116] When the collision time is less than the collision threshold, a warning alert is performed.

[0117] Optionally, the early warning module 65 is further used for:

[0118] When the current target is located in the virtual lane area indicated by the virtual lane line, determining that the current target is a potential dangerous target;

[0119] When the current target is partially located in the virtual lane area, and the portion located in the virtual lane area meets a preset condition, the current target is determined to be a potentially dangerous target.

[0120] Optionally, the early warning module 65 is further used for:

[0121] The potential dangerous target is highlighted on the current image, wherein the display mode of the potential dangerous target is different from the display mode of the non-potential dangerous target.

[0122] Optionally, the early warning module 65 is further used for:

[0123] The virtual lane line is displayed in real time on the current image.

[0124] Optionally, the early warning module 65 is further used for:

[0125] Get the target category of the current target;

[0126] According to the target category, a target frame of the current target is displayed in the current image with a target color corresponding to the target category.

[0127] Optionally, the early warning module 65 is further used for:

[0128] Extracting a road boundary of an actual road from the current image, and acquiring a positional relationship between the virtual lane line and the road boundary;

[0129] When the positional relationship between the virtual lane line and the road boundary meets a prompt condition, a prompt message is output, wherein the prompt condition indicates that the virtual lane line exceeds the road boundary or the distance between the virtual lane line and the road boundary is less than a preset distance value.

[0130] It can be understood by those skilled in the art that Figure 6 The structure of the vehicle warning device based on virtual lane lines does not constitute a limitation on the vehicle warning device based on virtual lane lines, and each module can be implemented in whole or in part by software, hardware, and a combination thereof. The above modules can be embedded in or independent of the controller in the device in the form of hardware, or can be stored in the memory in the device in the form of software, so that the controller can call and execute the operations corresponding to the above modules. In other embodiments, the vehicle warning device based on virtual lane lines may include more or fewer modules than those shown in the figure.

[0131] See also Figure 7 In another aspect of the embodiment of the present application, a vehicle device 10 is provided, including a memory 3011 and a processor 3012. The memory 3011 stores a computer program. When the computer program is executed by the processor, the processor 3012 executes the steps of the vehicle warning method based on virtual lane lines provided in any of the above embodiments of the present application. The vehicle device 10 is a device installed on any type of mobile body, such as a vehicle, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobile device, an airplane, a drone, a ship, or a robot, etc.

[0132] The processor 3012 is the control center, which uses various interfaces and lines to connect various parts of the entire device, and executes various functions of the device and processes data by running or executing software programs and / or modules stored in the memory 3011, and calling data stored in the memory 3011. Optionally, the processor 3012 may include one or more processing cores; preferably, the processor 3012 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 3012.

[0133] The memory 3011 can be used to store software programs and modules. The processor 3012 executes various functional applications and data processing by running the software programs and modules stored in the memory 3011. The memory 3011 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory 3011 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 3011 may also include a memory controller to provide the processor 3012 with access to the memory 3011.

[0134] On the other hand, an embodiment of the present application further provides a storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the vehicle warning method based on virtual lane lines provided in any of the above embodiments of the present application.

[0135] Those skilled in the art can understand that all or part of the processes in the methods provided in the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0136] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A vehicle warning method based on virtual lane lines, applied to vehicle equipment, characterized in that: include: Get the current image collected in the current scene; Acquiring driving status data of the vehicle equipment; Drawing a virtual lane line based on the current image and the driving status data; Performing target detection on the current image to identify the current target in the current image and the position data of the current target; Based on the position data of each current target and the virtual lane line, determining the position relationship between each current target and the virtual lane line; Based on the positional relationship between each current target and the virtual lane line, a warning alert is performed.

2. The vehicle warning method based on virtual lane lines as claimed in claim 1, characterized in that: Drawing a virtual lane line based on the current image and the driving state data includes: Acquire virtual lane line width data corresponding to the current scene; Based on the driving state data, predicting the movement trajectory of the vehicle equipment; The virtual lane line is drawn based on the virtual lane line width data, the motion trajectory and the current image.

3. The vehicle warning method based on virtual lane lines as claimed in claim 2, characterized in that: The acquiring of virtual lane line width data corresponding to the current scene comprises: Based on a trigger operation on a specification selection control displayed on a user interface, lane line width data corresponding to the trigger operation is acquired, and the lane line width data corresponding to the trigger operation is used as the virtual lane line width data.

4. The vehicle warning method based on virtual lane lines as claimed in claim 2, characterized in that: The acquiring of virtual lane line width data corresponding to the current scene comprises: According to the current image, identifying a current road scene type corresponding to the current scene; According to the current road scene type, the lane line width data corresponding to the current road scene type is obtained from the pre-stored correspondence between the road scene type and the lane line width data, and the lane line width data corresponding to the current road scene type is used as the virtual lane line width data.

5. The vehicle warning method based on virtual lane lines as claimed in claim 1, characterized in that: The performing of early warning based on the positional relationship between each current target and the virtual lane line includes: Determine a potential dangerous target based on a positional relationship between each current target and the virtual lane line; Calculating the collision time between the potential dangerous target and the vehicle equipment; When the collision time is less than the collision threshold, a warning alert is performed.

6. The vehicle warning method based on virtual lane lines as claimed in claim 5, characterized in that: The determining of the potential dangerous target based on the positional relationship between each current target and the virtual lane line includes at least one of the following: When the current target is located in the virtual lane area indicated by the virtual lane line, determining that the current target is a potential dangerous target; When the current target is partially located in the virtual lane area, and the portion located in the virtual lane area meets a preset condition, the current target is determined to be a potentially dangerous target.

7. The vehicle warning method based on virtual lane lines as claimed in claim 5, characterized in that: The execution early warning warning includes: The potential dangerous target is highlighted on the current image, wherein the display mode of the potential dangerous target is different from the display mode of the non-potential dangerous target.

8. The vehicle warning method based on virtual lane lines as claimed in claim 1, characterized in that: The method further comprises: The virtual lane line is displayed in real time on the current image.

9. The vehicle warning method based on virtual lane lines as claimed in claim 1, characterized in that: The method further comprises: Get the target category of the current target; According to the target category, a target frame of the current target is displayed in the current image with a target color corresponding to the target category.

10. The vehicle warning method based on virtual lane lines according to claim 1, characterized in that: The method further comprises: Extracting a road boundary of an actual road from the current image, and acquiring a positional relationship between the virtual lane line and the road boundary; When the positional relationship between the virtual lane line and the road boundary meets a prompt condition, a prompt message is output, wherein the prompt condition indicates that the virtual lane line exceeds the road boundary or the distance between the virtual lane line and the road boundary is less than a preset distance value.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the vehicle warning method based on virtual lane lines as described in any one of claims 1 to 10 is implemented.

12. A vehicle device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the vehicle warning method based on virtual lane lines as claimed in any one of claims 1 to 10.

Images