Lane driving state determination method and device, vehicle and medium
By processing the vehicle driving image and entering the early warning model, the accuracy of lane driving status judgment is solved, and the performance of assisted driving system in bad weather is improved.
Patent Information
- Application Number
- CN202310643929.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2024-12-03
AI Technical Summary
The existing lane driving state determination methods have low accuracy, which affects the effect of assisted driving, especially in severe weather conditions, which reduces the accuracy of lane line detection, resulting in false alarms and inaccurate judgment of driving state.
By acquiring the current driving image of the target vehicle, the driving data is obtained, including the offset distance and the offset speed, and input it into the pre-trained deviation warning model to determine the driving state of the vehicle. This method combines image enhancement processing to improve image clarity, improve lane line detection accuracy and driving state judgment accuracy.
Accurate and real-time judgment of the vehicle's driving status is achieved, and the reliability and safety of the assisted driving system is improved, especially in severe weather conditions such as heavy fog and heavy rain.
Smart Images

Figure CN119058719A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of autonomous driving, and in particular, to a method, device, vehicle and medium for determining a lane driving state. Background Art
[0002] With the popularization of automobiles, traffic accidents have gradually increased. In order to reduce the occurrence of accidents, the application of assisted driving has increased. Assisted driving needs to be judged based on the driving state of the vehicle in the driving lane. To judge the driving state of the vehicle, it is necessary to first judge the driving trend of the vehicle. The accuracy of the existing method for determining the driving trend of the vehicle is low, which affects the effect of assisted driving. Summary of the invention
[0003] The main purpose of the present disclosure is to provide a lane driving state determination method, device, vehicle and medium, aiming to solve the technical problem of low accuracy of existing lane driving state determination methods.
[0004] To achieve the above object, the present disclosure provides a method for determining a lane driving state, comprising:
[0005] Obtain the current driving image of the target vehicle;
[0006] Processing the current driving image to obtain driving data of the target vehicle; wherein the driving data includes an offset distance and an offset speed of the target vehicle relative to a lane line of a current driving lane;
[0007] The driving state of the target vehicle is determined according to the driving data and a pre-trained deviation warning model.
[0008] Optionally, processing the current driving image to obtain driving data of the target vehicle includes:
[0009] Recognize the current driving image to obtain lane line feature points of the current driving lane;
[0010] The driving data is determined according to the coordinates of the lane line feature points and the target position points on the target vehicle in the image coordinate system.
[0011] Optionally, the target position point is a point on the centerline of the target vehicle.
[0012] Optionally, determining the driving data according to the coordinates of the lane line feature point and the target position point on the target vehicle in an image coordinate system includes:
[0013] Determine a lane line equation of the current driving lane in the image coordinate system according to the lane line feature point;
[0014] The driving data is determined according to the lane line equation and the coordinates of the target position point on the target vehicle in the image coordinate system.
[0015] Optionally, determining the driving data according to the lane line equation and the coordinates of the target position point on the target vehicle in the image coordinate system includes:
[0016] Determine the target distance according to the lane line equation and the coordinates of the target position point; wherein the target distance is the lateral distance between the center of the target vehicle and the lane line of the current driving lane, or the lateral distance between the edge of the target vehicle and the lane line of the current driving lane;
[0017] The offset speed and / or the offset distance are determined according to the target distance.
[0018] Optionally, determining the offset speed according to the target distance includes:
[0019] The offset speed is determined according to the target distance corresponding to the current driving image and the target distance corresponding to each frame of a plurality of consecutive frames of historical driving images before the current driving image.
[0020] Optionally, determining the offset distance according to the target distance includes:
[0021] The offset distance is determined according to the target distance corresponding to the current driving image and the target distance corresponding to the first frame of the historical driving image among a plurality of consecutive frames of historical driving images before the current driving image.
[0022] Optionally, processing the current driving image to obtain the driving data of the target vehicle further includes:
[0023] Before recognizing the current driving image, image enhancement processing is performed on the current driving image.
[0024] Optionally, the lane markings include left lane markings and right lane markings;
[0025] The offset speed includes a left offset speed and a right offset speed, wherein the left offset speed is used to characterize the speed at which the target vehicle deviates toward the left lane line, and the right offset speed is used to characterize the speed at which the target vehicle deviates toward the right lane line;
[0026] The offset distance includes a left offset distance and a right offset distance, wherein the left offset distance is used to characterize the distance that the target vehicle moves laterally toward the left lane line, and the right offset distance is used to characterize the distance that the target vehicle moves laterally toward the right lane line.
[0027] Optionally, the deviation warning model includes an input layer, a plurality of hidden layers and an output layer;
[0028] The input layer is used to receive the driving data, and the driving data includes a left-side deviation speed, a right-side deviation speed, a left-side deviation distance, and a right-side deviation distance;
[0029] The hidden layer is used to process the driving data to obtain the driving state, wherein the driving state includes a vehicle left deviation, a vehicle no deviation, and a vehicle right deviation;
[0030] The output layer is used to output the driving status.
[0031] In addition, to achieve the above-mentioned purpose, the present disclosure also provides a lane driving state determination device, comprising:
[0032] An image acquisition module is used to acquire the current driving image of the target vehicle;
[0033] A driving data acquisition module, used to process the current driving image to obtain the driving data of the target vehicle; wherein the driving data includes the offset distance and offset speed of the target vehicle relative to the lane line of the current driving lane;
[0034] The driving state determination module is used to determine the driving state of the target vehicle according to the driving data and a pre-trained deviation warning model.
[0035] In addition, to achieve the above-mentioned purpose, the present disclosure also provides a non-temporary computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the lane driving state determination method provided by the present disclosure are implemented.
[0036] In addition, to achieve the above objectives, the present disclosure also provides a vehicle, comprising:
[0037] a memory having a computer program stored thereon;
[0038] A processor is used to execute the computer program in the memory to implement the steps of the lane driving state determination method provided by the present disclosure.
[0039] The above technical solution proposed in the embodiment of the present disclosure processes the current driving image of the target vehicle to obtain the offset distance and offset speed of the target vehicle relative to the lane line of the current driving lane, and inputs the offset distance and offset speed into a pre-trained deviation warning model to determine the driving state of the target vehicle. The offset distance and offset speed can accurately reflect the driving trend of the target vehicle, and then the driving state of the target vehicle can be accurately and real-time determined through the deviation warning model.
[0040] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:
[0042] Figure 1 A schematic diagram of a flow chart of a lane driving state determination method provided by an embodiment of the present disclosure;
[0043] Figure 2 A schematic diagram of the relationship between the lane line and the distance between the target vehicle in a lane driving state determination method provided in an embodiment of the present disclosure;
[0044] Figure 3 A current driving image in a foggy scene provided by an embodiment of the present disclosure;
[0045] Figure 4 A current driving image after image enhancement processing in a foggy scene provided by an embodiment of the present disclosure;
[0046] Figure 5 A current driving image in a heavy rain scene provided by an embodiment of the present disclosure;
[0047] Figure 6 A current driving image after image enhancement processing in a heavy rain scene provided by an embodiment of the present disclosure;
[0048] Figure 7 A schematic diagram of the structure of a deviation warning model provided by an embodiment of the present disclosure;
[0049] Figure 8 A schematic diagram of the structure of a lane driving state determination device provided by an embodiment of the present disclosure;
[0050] Fig. 9 It is a schematic structural diagram of a vehicle involved in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0051] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.
[0052] In the present disclosure, unless otherwise stated, directional words such as "front, rear, left, right" are generally determined based on the driving direction of the target vehicle.
[0053] Assisted driving needs to be judged based on the driving state of the vehicle in the driving lane, and the driving state needs to be based on the determination of the lane line. The inventor has found that the existing lane line detection method relies on obvious lane line markings and only maintains a high detection accuracy in specific environments. When it rains or fogs, the existing lane line detection method cannot quickly and accurately detect the lane line; at the same time, the existing assisted driving system can usually warn before danger occurs, but due to the existence of erroneous detection of single-frame driving images, or the instantaneous change of angle caused by the swing of the vehicle, false alarms are caused, affecting the driver's safe driving.
[0054] To this end, the present disclosure provides a solution, which processes the current driving image of the target vehicle to obtain the offset distance and offset speed of the target vehicle relative to the lane line of the current driving lane, and inputs the offset distance and offset speed into a pre-trained deviation warning model to determine the driving state of the target vehicle. The offset distance and offset speed can accurately reflect the driving trend of the target vehicle, and then the driving state of the target vehicle can be accurately and real-time determined through the deviation warning model; and by performing image enhancement processing on the current driving image, the clarity and contrast of the image can be improved, which is conducive to improving the lane line detection accuracy and the subsequent driving state judgment accuracy.
[0055] Figure 1 A flow chart of a method for determining lane driving status provided by an embodiment of the present disclosure. Figure 1 As shown, the method may include the following steps:
[0056] S101: Acquire the current driving image of the target vehicle.
[0057] In one embodiment of the present disclosure, an image acquisition device for acquiring road conditions during driving is installed on the target vehicle, and the current driving image is acquired through the image acquisition device. For example, the image acquisition device can be a monocular camera, and the acquired current driving image is a color image with three channels of red, green and blue (RGB).
[0058] S102: Processing the current driving image to obtain driving data of the target vehicle; wherein the driving data includes an offset distance and an offset speed of the target vehicle relative to a lane line of a current driving lane.
[0059] In one embodiment of the present disclosure, the driving data of the target vehicle is obtained by processing the current driving image. The driving data refers to the offset distance and offset speed of the target vehicle relative to the lane line of the current driving lane, that is, the distance and speed of the lateral movement of the vehicle position of the target vehicle after driving for a period of time compared with the driving trajectory corresponding to the original driving direction. The driving data can reflect the driving information of the target vehicle, can judge the driving status of the target vehicle in the current lane, and is the data basis of the assisted driving system.
[0060] S103: Determine the driving state of the target vehicle according to the driving data and a pre-trained deviation warning model.
[0061] In one embodiment of the present disclosure, driving data is input into a pre-trained deviation warning model to output the driving state of the target vehicle. The driving data may be driving data of the target vehicle over a period of time. The deviation warning model has a high accuracy, and its algorithm speed can reach more than 100 frames per second. It can output the driving state of the target vehicle in real time. Moreover, the driving data can reflect the driving trend of the target vehicle. The driving data at the current moment is used as input data. The deviation warning model can predict the driving state of the target vehicle and has predictiveness.
[0062] In one embodiment, the driving state includes vehicle left deviation, vehicle no deviation and vehicle right deviation, which are used to determine the deviation state of the target vehicle relative to the current driving lane. The driving data may include left deviation speed, right deviation speed, left deviation distance and right deviation distance.
[0063] During the model training phase, driving data for a period of time can be collected and labeled. The label information is the driving state corresponding to each set of driving data. A set of driving data and the corresponding label form a training data. The deviation warning model is trained with the training data until the preset training cutoff condition is met, thereby obtaining the deviation warning model.
[0064] As an optional implementation, S102 may further include: identifying the current driving image to obtain lane line feature points of the current driving lane; determining the driving data based on the coordinates of the lane line feature points and the target position points on the target vehicle in the image coordinate system.
[0065] In one embodiment of the present disclosure, a deep learning model is used to identify the current driving image to obtain the lane line feature points of the current driving lane where the target vehicle is located. The deep learning model has high accuracy and its algorithm speed is fast, and it can detect the lane line of the target vehicle's driving lane in real time. In the image coordinate system, a point is selected on the target vehicle as the target position point, and the driving data of the target vehicle is calculated based on the coordinates of the target position point and the coordinates of the lane line feature points.
[0066] As an optional implementation, the target position point is a point on the center line of the target vehicle.
[0067] In one embodiment of the present disclosure, the target position point refers to a point on the center line of the target vehicle, and the specific position of the target vehicle and the distance between the target vehicle and the lane line can be determined based on the coordinates of the point and the width of the target vehicle.
[0068] like Figure 2 As shown, the origin of the image coordinate system is the intersection of the optical axis of the image acquisition device and the imaging plane, which is also the center point of the current driving image; the solid square in the figure represents the target vehicle, and the width is W v Point A is the target position point, and the image width of the current driving image is W. After calibration, the front of the vehicle is parallel to the optical axis direction of the image acquisition device, that is, the center line of the current driving image is the center line of the target vehicle, so the horizontal coordinate of point A is W / 2.
[0069] As an optional implementation, the driving data is determined based on the coordinates of the lane line feature points and the target position points on the target vehicle in the image coordinate system, including: determining the lane line equation of the current driving lane in the image coordinate system based on the lane line feature points; determining the driving data based on the lane line equation and the coordinates of the target position points on the target vehicle in the image coordinate system.
[0070] In one embodiment of the present disclosure, segmented fitting or other data processing is performed on the identified lane feature points to obtain the lane equation of the current driving lane in the image coordinate system. In the image coordinate system, the driving data of the target vehicle is calculated based on the coordinates of the target position point and the lane equation.
[0071] In one embodiment, the current driving image is recognized using a BézierLaneNet deep learning model. The BézierLaneNet deep learning model can realize lane line detection based on polynomial curves. A third-order Bezier curve is used in the deep learning model, that is, after inputting the current driving image, four control points (i.e., lane line feature points) can be obtained, and a Bezier curve result, i.e., the lane line in the current driving image, is obtained according to the control points.
[0072] The image acquisition device and the vehicle are calibrated, and the spatial coordinates of the target vehicle and the image acquisition device are converted based on the calibration data so that the front of the vehicle is parallel to the optical axis of the camera. After obtaining the lane feature points of the lane, the lane equation of the lane in the image coordinate system is determined according to the coordinates of the lane feature points. Finally, the driving data of the target vehicle is calculated based on the lane equation and the coordinates of the target position point.
[0073] As an optional implementation, the driving data is determined according to the lane line equation and the coordinates of the target position point on the target vehicle in the image coordinate system, including: determining the target distance according to the lane line equation and the coordinates of the target position point; wherein the target distance is the lateral distance between the center of the target vehicle and the lane line of the current driving lane, or the lateral distance between the edge of the target vehicle and the lane line of the current driving lane; and determining the offset speed and / or the offset distance according to the target distance.
[0074] In one embodiment of the present disclosure, the target distance includes the lateral distance between the center of the target vehicle and the lane line of the current driving lane, that is, the difference between the coordinates of the target position point and the coordinates of the intersection of the extension line of the front edge of the target vehicle and the lane line, which is used to calculate the offset speed of the target vehicle; it also includes the lateral distance between the edge of the target vehicle and the lane line of the current driving lane, that is, the difference between the coordinates of the intersection of the side edge of the target vehicle and the extension line of the front edge of the target vehicle and the lane line, which can also be expressed by the difference between the lateral distance between the center of the target vehicle and the lane line of the current driving lane and the width of the target vehicle, which is used to calculate the offset distance of the target vehicle.
[0075] As an optional implementation, determining the offset speed according to the target distance includes: determining the offset speed according to the target distance corresponding to the current driving image and the target distance corresponding to each frame of historical driving images in a plurality of consecutive frames of historical driving images before the current driving image.
[0076] In one embodiment of the present disclosure, the image acquisition device can obtain multiple frames of continuous driving images, including the current driving image and multiple frames of historical driving images. According to the above method, the target distance corresponding to multiple moments can be obtained, that is, the lateral distance between the center of the target vehicle and the lane line of the current driving lane. Knowing the time and distance, it is easy to know the deviation speed of the target vehicle in this time period.
[0077] For example, obtain n first distances D corresponding to n consecutive frames of images and store them in the array data, where data[0] indicates that it stores the target distance D corresponding to the frame of the n consecutive images that is farthest from the current image in time, and data[n-1] indicates that it stores the target distance D corresponding to the frame of the n consecutive images that is closest to the current image in time (i.e., the current driving image). Calculate the offset speed according to the following relationship:
[0078]
[0079] Among them, i represents the sequence number of the element in the array data, n is the number of images, and v h is the offset speed.
[0080] As an optional implementation, the offset distance is determined according to the target distance, including: determining the offset distance according to the target distance corresponding to the current driving image and the second distance corresponding to the first frame of historical driving image in a plurality of consecutive frames of historical driving images before the current driving image.
[0081] In one embodiment of the present disclosure, the image acquisition device can acquire multiple frames of continuous driving images, including the current driving image and multiple frames of historical driving images, and according to the above method, the target distances corresponding to multiple moments can be obtained, that is, the lateral distance between the edge of the target vehicle and the lane line of the current driving lane. According to the target distance corresponding to the current driving image and the target distance corresponding to the frame of historical driving image farthest from the current one in time (that is, the first frame of historical driving image), the offset distance of the target vehicle within this time period can be determined.
[0082] For example, obtain n target distances d corresponding to n consecutive frames of images, and calculate the offset distance according to the following relationship:
[0083] d h =d0-d1
[0084] Among them, d0 is the target distance corresponding to the first frame of historical driving image, d1 is the target distance corresponding to the current driving image, and d h is the offset distance.
[0085] As an optional implementation, the lane line includes a left lane line and a right lane line; the offset speed includes a left offset speed and a right offset speed, the left offset speed is used to characterize the speed at which the target vehicle deviates toward the left lane line, and the right offset speed is used to characterize the speed at which the target vehicle deviates toward the right lane line; the offset distance includes a left offset distance and a right offset distance, the left offset distance is used to characterize the distance that the target vehicle moves laterally toward the left lane line, and the right offset distance is used to characterize the distance that the target vehicle moves laterally toward the right lane line.
[0086] In one embodiment of the present disclosure, Figure 2 As shown, L l is the left lane line of the current lane, L r is the right lane line of the current driving lane, and the intersection coordinates of the left lane line and the extended line of the front edge of the target vehicle are (pl x ,pl y ), the intersection coordinates of the right lane line and the extended line of the front edge of the target vehicle are (pr x ,pr y ), the target vehicle width is W v ;D l Indicates the lateral distance between the center of the target vehicle and the left lane line of the current driving lane, that is, the first left target distance, D r Indicates the lateral distance between the center of the target vehicle and the right lane line of the current driving lane, that is, the first right target distance. d l Indicates the lateral distance between the left edge of the target vehicle and the left lane line of the current lane, that is, the second left target distance. d r Indicates the lateral distance between the right edge of the target vehicle and the right lane line of the current driving lane, that is, the second right target distance.
[0087] Calculate the left offset speed according to the following relationship:
[0088]
[0089] Among them, dataL[i] stores the first left target distance D of multiple consecutive frames of driving images l , i represents the sequence number of the element in the array dataL, n is the number of driving images, v hl is the left offset speed.
[0090] Calculate the right offset speed according to the following relationship:
[0091]
[0092] Among them, dataR[i] stores the first right target distance D of multiple consecutive frames of driving images r , i represents the sequence number of the element in the array dataR, n is the number of driving images, v jr is the right offset speed.
[0093] Calculate the left offset distance according to the following relationship:
[0094] d hl =d l0 -d l1
[0095] Among them, d l0 is the distance of the second left target corresponding to the first frame of historical driving image, d l1 is the distance of the second left target corresponding to the current driving image, d hl is the left offset distance.
[0096] Calculate the right offset distance according to the following relationship:
[0097] d hr =d r0 -d r1
[0098] Among them, d r0 is the distance of the second right target corresponding to the first frame of historical driving image, d r1 is the distance of the second right target corresponding to the current driving image, d hr is the right offset distance.
[0099] As an optional implementation, the processing of the current driving image to obtain the driving data of the target vehicle further includes: before identifying the current driving image, performing image enhancement processing on the current driving image.
[0100] In one embodiment of the present disclosure, Figure 3-Figure 6 As shown, the current driving image obtained by the image acquisition device is enhanced through image enhancement processing to improve clarity and contrast before recognition, which can enhance image quality, reduce the impact of weather such as rain and fog on the current driving image, and provide an accurate data basis for subsequent lane line detection.
[0101] In this embodiment, the image data is processed based on the MSRCP (multi-scale Retinex with chromaticity preservation, MSRCP) algorithm, and the original image is color enhanced by adjusting the color intensity of different channels, so that the processed current driving image has both the color characteristics of the original image and the enhanced elements. After the current driving image in blurry scenes such as heavy rain and fog is processed by the MSRCP algorithm, the lane line detection effect is better.
[0102] As an optional implementation, the deviation warning model includes an input layer, multiple hidden layers and an output layer; wherein the input layer is used to receive the driving data, and the driving data includes a left deviation speed, a right deviation speed, a left deviation distance and a right deviation distance; the hidden layer is used to process the driving data to obtain the driving status, and the driving status includes a vehicle left deviation, a vehicle no deviation and a vehicle right deviation; the output layer is used to output the driving status.
[0103] In one embodiment of the present disclosure, the deviation warning model can be a fully connected neural network with multiple hidden layers, and its structure is as follows: Figure 7 As shown, the input layer has 4 nodes, corresponding to 4 driving data, namely, the left offset speed, the right offset speed, the left offset distance and the right offset distance respectively; the hidden layer has two layers, of which the first hidden layer has 6 nodes and the second hidden layer has 3 nodes; the output layer has 3 nodes, corresponding to 3 driving states, namely, the vehicle is offset to the left, the vehicle has no offset and the vehicle is offset to the right. Among them, each output node in the hidden layer is normalized and activated after calculating the weighted sum, and finally outputs the information of each node, namely the driving state. The deviation warning model can, for example, use the LeakyReLU activation function, which is expressed by the following relationship:
[0104]
[0105] Among them, x is the input data of the neuron, and α is the model hyperparameter, which is set to 0.01 by default.
[0106] The deviation warning model uses the Softmax loss function, which is expressed by the following relationship:
[0107]
[0108] Among them, x iis the ith driving data among the k driving data, k is the number of driving data, in the present disclosure, there are four driving data, namely, left-side offset speed, right-side offset speed, left-side offset distance and right-side offset distance, so k can be, for example, 4; c is the cth driving state among the m driving states, m is the number of driving state types, in the present disclosure, there are three driving states, namely, vehicle left offset, vehicle no offset and vehicle right offset, so m can be, for example, 3; y ic is a symbolic function. If the driving state to which the ith driving data belongs is the cth driving state, then y ic =1, otherwise y ic =0.
[0109] Figure 8 Schematic diagram of the structure of a lane driving state determination device provided by an embodiment of the present disclosure. Figure 8 As shown, the lane driving state determination device 200 may include an image acquisition module 201 , a driving data acquisition module 202 and a driving state determination module 203 .
[0110] An image acquisition module 201 is used to acquire a current driving image of a target vehicle;
[0111] A driving data acquisition module 202 is used to process the current driving image to obtain the driving data of the target vehicle; wherein the driving data includes an offset distance and an offset speed of the target vehicle relative to a lane line of a current driving lane;
[0112] The driving state determination module 203 is used to determine the driving state of the target vehicle according to the driving data and a pre-trained deviation warning model.
[0113] Optionally, the driving data acquisition module 202 includes:
[0114] An image recognition submodule, used to recognize the current driving image and obtain lane line feature points of the current driving lane;
[0115] The first driving data determination submodule is used to determine the driving data according to the coordinates of the lane line feature points and the target position points on the target vehicle in the image coordinate system.
[0116] Optionally, the first driving data determination submodule includes:
[0117] A lane line equation determination submodule, used to determine the lane line equation of the current driving lane in the image coordinate system according to the lane line feature points;
[0118] The second driving data determination submodule is used to determine the driving data according to the lane line equation and the coordinates of the target position point on the target vehicle in the image coordinate system.
[0119] Optionally, the second driving data determination submodule includes:
[0120] A target distance determination submodule, for determining a target distance according to the lane line equation and the coordinates of the target position point; wherein the target distance is the lateral distance between the center of the target vehicle and the lane line of the current driving lane, or the lateral distance between the edge of the target vehicle and the lane line of the current driving lane;
[0121] The third driving data determination submodule is configured to determine the offset speed and / or the offset distance according to the target distance.
[0122] Optionally, the third driving data determination submodule is used to determine the offset speed according to the target distance corresponding to the current driving image and the target distance corresponding to each frame of historical driving images in a plurality of consecutive frames of historical driving images before the current driving image.
[0123] Optionally, the third driving data determination submodule is used to determine the offset distance according to the target distance corresponding to the current driving image and the target distance corresponding to the first frame of historical driving image in multiple consecutive frames of historical driving images before the current driving image.
[0124] Optionally, the driving data acquisition module 202 includes:
[0125] The image enhancement submodule is used for the image recognition submodule to perform image enhancement processing on the current driving image before recognizing the current driving image.
[0126] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0127] It is not difficult to find from the above description that this embodiment processes the current driving image of the target vehicle to obtain the offset distance and offset speed of the target vehicle relative to the lane line of the current driving lane, and inputs the offset distance and offset speed into the pre-trained deviation warning model to determine the driving state of the target vehicle. The offset distance and offset speed can accurately reflect the driving trend of the target vehicle, and then the driving state of the target vehicle can be accurately and real-time determined through the deviation warning model; and by performing image enhancement processing on the current driving image, the clarity and contrast of the image can be improved, which is conducive to improving the lane line detection accuracy and the subsequent driving state judgment accuracy.
[0128] The present disclosure also provides a non-temporary computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the lane driving state determination method provided by the present disclosure.
[0129] Fig. 9 FIG. 1 is a schematic diagram of the structure of a vehicle involved in an embodiment of the present disclosure. Fig. 9 As shown, the vehicle 300 may include: a processor 301 , a memory 302 . The vehicle 300 may also include one or more of a multimedia component 303 , an input / output (I / O) interface 304 , and a communication component 305 .
[0130] The processor 301 is used to control the overall operation of the vehicle 300 to complete all or part of the steps in the lane driving state determination method described above. The memory 302 is used to store various types of data to support the operation of the vehicle 300, and these data may include, for example, instructions for any application or method used to operate on the vehicle 300, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 303 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 302 or sent through the communication component 305. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 304 provides an interface between the processor 301 and other interface modules, and the above-mentioned other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 305 is used for wired or wireless communication between the vehicle 300 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 305 may include: Wi-Fi module, Bluetooth module, NFC module, etc.
[0131] In some embodiments, the vehicle 300 may be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned lane driving state determination method.
[0132] In addition, in one embodiment, the embodiment of the present disclosure further provides a computer-readable storage medium including program instructions, and when the program instructions are executed by the processor, the steps of the lane driving state determination method described above are implemented. For example, the computer-readable storage medium may be the memory 302 including the program instructions, and the program instructions may be executed by the processor 301 of the vehicle 300 to complete the lane driving state determination method described above.
[0133] In addition, in one embodiment, an embodiment of the present disclosure also provides a computer program product, which is a computer program that can be executed by a programmable device, and the computer program has a code portion for executing the above-mentioned lane driving state determination method when executed by the programmable device.
[0134] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.
[0135] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.
[0136] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.
Claims
1. A method for determining a lane driving state, characterized in that: include: Obtain the current driving image of the target vehicle; Processing the current driving image to obtain driving data of the target vehicle; wherein the driving data includes an offset distance and an offset speed of the target vehicle relative to a lane line of a current driving lane; The driving state of the target vehicle is determined according to the driving data and a pre-trained deviation warning model.
2. The lane driving state determination method according to claim 1, characterized in that: Processing the current driving image to obtain driving data of the target vehicle includes: Recognize the current driving image to obtain lane line feature points of the current driving lane; The driving data is determined according to the coordinates of the lane line feature points and the target position points on the target vehicle in the image coordinate system.
3. The lane driving state determination method according to claim 2, characterized in that: The target position point is a point on the center line of the target vehicle.
4. The lane driving state determination method according to claim 2 or 3, characterized in that: Determining the driving data according to the coordinates of the lane line feature points and the target position points on the target vehicle in the image coordinate system includes: Determine a lane line equation of the current driving lane in the image coordinate system according to the lane line feature point; The driving data is determined according to the lane line equation and the coordinates of the target position point on the target vehicle in the image coordinate system.
5. The lane driving state determination method according to claim 4, characterized in that: Determining the driving data according to the lane line equation and the coordinates of the target position point on the target vehicle in the image coordinate system includes: Determine the target distance according to the lane line equation and the coordinates of the target position point; wherein the target distance is the lateral distance between the center of the target vehicle and the lane line of the current driving lane, or the lateral distance between the edge of the target vehicle and the lane line of the current driving lane; The offset speed and / or the offset distance are determined according to the target distance.
6. The lane driving state determination method according to claim 5, characterized in that: Determining the offset speed according to the target distance includes: The offset speed is determined according to the target distance corresponding to the current driving image and the target distance corresponding to each frame of a plurality of consecutive frames of historical driving images before the current driving image.
7. The lane driving state determination method according to claim 5, characterized in that: Determining the offset distance according to the target distance includes: The offset distance is determined according to the target distance corresponding to the current driving image and the target distance corresponding to the first frame of the historical driving image among a plurality of consecutive frames of historical driving images before the current driving image.
8. The lane driving state determination method according to claim 2, characterized in that: Processing the current driving image to obtain the driving data of the target vehicle also includes: Before recognizing the current driving image, image enhancement processing is performed on the current driving image.
9. The lane driving state determination method according to claim 5, characterized in that: The lane markings include a left lane marking and a right lane marking; The offset speed includes a left offset speed and a right offset speed, wherein the left offset speed is used to characterize the speed at which the target vehicle deviates toward the left lane line, and the right offset speed is used to characterize the speed at which the target vehicle deviates toward the right lane line; The offset distance includes a left offset distance and a right offset distance, wherein the left offset distance is used to characterize the distance that the target vehicle moves laterally toward the left lane line, and the right offset distance is used to characterize the distance that the target vehicle moves laterally toward the right lane line.
10. The lane driving state determination method according to claim 1, characterized in that: The deviation warning model includes an input layer, a plurality of hidden layers and an output layer; The input layer is used to receive the driving data, and the driving data includes a left-side deviation speed, a right-side deviation speed, a left-side deviation distance, and a right-side deviation distance; The hidden layer is used to process the driving data to obtain the driving state, wherein the driving state includes a vehicle left deviation, a vehicle no deviation and a vehicle right deviation; The output layer is used to output the driving status.
11. A lane driving state determination device, characterized in that: include: An image acquisition module is used to acquire the current driving image of the target vehicle; A driving data acquisition module, used to process the current driving image to obtain the driving data of the target vehicle; wherein the driving data includes the offset distance and offset speed of the target vehicle relative to the lane line of the current driving lane; The driving state determination module is used to determine the driving state of the target vehicle according to the driving data and a pre-trained deviation warning model.
12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 10 are implemented.
13. A vehicle, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 10.
Citation Information
Patent Citations
Lane departure early warning method and device, electronic equipment and storage medium
CN115303295A
Early warning method and system for vehicle lane departure in night scene
CN115880658A
Commercial vehicle multi-camera lane line departure early warning method and system
CN116080675A
Lane departure early warning method, device, equipment and storage medium
CN116118727A
Vehicle state determination method and device, electronic equipment and readable storage medium
CN116152756A
Cited By
Road traffic behavior anomaly detection method and system based on computer vision
CN119636790A
