Vehicle driving state detection method and device, storage medium and electronic equipment

Abstract

The application discloses a vehicle driving state detection method and device, a storage medium and an electronic device, and relates to the technical field of vehicle driving state detection. The method comprises the following steps: acquiring a first image around a first vehicle by using a vehicle-mounted camera on the first vehicle; performing lane line detection and identification on the first image to acquire lane line information; performing vehicle detection and identification based on the lane line information to acquire vehicle information of at least one vehicle around the first vehicle; and judging a driving state of the at least one vehicle based on the lane line information and the vehicle information of the at least one vehicle around the first vehicle. The application can solve the technical problem that the image acquired by using the vehicle-mounted camera cannot be used to detect the driving state of the surrounding vehicles.

Description

Technical Field

[0001] This invention relates to computer vision processing technology, and more specifically, to a method, apparatus, storage medium, and electronic device for detecting vehicle driving status. Background Technology

[0002] Driving against the flow of traffic is a serious violation of traffic regulations and an extremely dangerous driving behavior. As people's demands for driving safety increase, effectively detecting vehicles driving against the flow of traffic is a crucial driver assistance function. Some solutions relying on surveillance cameras in road monitoring and security systems typically identify vehicles' movement trajectories based on multiple frames of images. These solutions are limited in coverage due to the number and location of the cameras; furthermore, the need for multiple frames makes detecting short-duration reverse driving (e.g., crossing a solid yellow line to overtake or turn) difficult. Other solutions utilizing vehicle-mounted cameras can only detect reverse driving of the vehicle itself (the vehicle where the camera is located), and cannot detect other vehicles driving against the flow of traffic in the surrounding area.

[0003] Therefore, it is necessary to propose a vehicle driving status detection method that can use images acquired by an onboard camera to detect the driving status of surrounding vehicles. Summary of the Invention

[0004] This invention provides a vehicle driving status detection method, device, storage medium, and electronic device to at least solve the technical problem of not being able to use images acquired by an onboard camera to detect the driving status of surrounding vehicles.

[0005] According to one aspect of the present invention, a vehicle driving state detection method is provided, comprising: acquiring a first image of the area surrounding the first vehicle using an onboard camera on a first vehicle; performing lane line detection and recognition on the first image to obtain lane line information; performing vehicle detection and recognition based on the lane line information to obtain vehicle information of at least one vehicle surrounding the first vehicle; and determining the driving state of at least one vehicle based on the lane line information and the vehicle information of at least one vehicle surrounding the first vehicle.

[0006] Optionally, performing lane line detection and recognition on the first image to obtain lane line information includes: performing lane line detection on the first image to obtain lane line detection results; recognizing lane lines based on the lane line detection results to obtain lane line recognition results; and using the lane line detection results and lane line recognition results as lane line information.

[0007] Optionally, vehicle detection and recognition based on lane line information to obtain vehicle information of at least one vehicle around the first vehicle includes: performing vehicle detection on a first image containing lane lines to obtain vehicle detection results; recognizing at least one vehicle around the first vehicle based on the vehicle detection results to obtain vehicle recognition results; and using the vehicle detection results and vehicle recognition results as vehicle information.

[0008] Optionally, the vehicle driving state detection method uses a first network model to perform lane line detection on the first image and obtains lane line detection results, wherein the lane line detection results include filtering out the first image containing lane lines and obtaining the position information of the lane lines.

[0009] Optionally, the first network model is a U-shaped convolutional neural network, including N1 first convolutional layer groups, a first downsampling group consisting of downsampling layers between every two first convolutional layer groups, N1 second convolutional layer groups, and a second upsampling group consisting of upsampling layers between every two second convolutional layer groups.

[0010] Optionally, the vehicle driving state detection method uses a second network model to identify lane lines based on the lane line detection results to obtain lane line identification results, wherein the lane line identification results include lane line shape and lane line color.

[0011] Optionally, the second network model is a multi-branch convolutional neural network, including N2 third convolutional layer groups, a second downsampling group consisting of downsampling layers between every two third convolutional layer groups, and a first fully connected layer group connected to the output of the third convolutional layer groups, wherein the first fully connected layer group contains N3 parallel fully connected layers.

[0012] Optionally, the vehicle driving state detection method uses a third network model to perform vehicle detection on a first image containing lane lines to obtain vehicle detection results, wherein the vehicle detection results are vehicle detection boxes containing vehicle position information.

[0013] Optionally, the third network model is a convolutional neural network with a detection head, including N3 fourth convolutional layer groups, a third downsampling group consisting of downsampling layers between every two fourth convolutional layer groups, a fully connected layer connected to the output of the fourth convolutional layer group, and a non-maximum suppression layer connected to the output of the fully connected layer.

[0014] Optionally, the vehicle driving state detection method uses a fourth network model to identify at least one vehicle around a first vehicle in a first image containing lane lines based on the vehicle detection results, and obtains vehicle identification results, wherein the vehicle identification results include the orientation of at least one vehicle.

[0015] Optionally, the fourth network model is a convolutional neural network, including N4 fifth convolutional layer groups, a fourth downsampling group consisting of downsampling layers between every two fifth convolutional layer groups, and a fully connected layer connected to the output of the fifth convolutional layer group.

[0016] Optionally, the vehicle-mounted camera is a forward-facing camera installed on the first vehicle.

[0017] According to another aspect of the present invention, a vehicle driving state detection device is provided, comprising: an image acquisition unit, installed on a first vehicle, for acquiring a first image of the area surrounding the first vehicle; a lane line information acquisition unit, for detecting and recognizing lane lines in the first image to acquire lane line information; a vehicle information acquisition unit, for detecting and recognizing vehicles based on the lane line information to acquire vehicle information of at least one vehicle surrounding the first vehicle; and a driving state determination unit, for determining the driving state of at least one vehicle based on the lane line information and the vehicle information of at least one vehicle surrounding the first vehicle.

[0018] Optionally, the lane line information acquisition unit includes: a lane line detection unit, used to perform lane line detection on the first image and obtain lane line detection results; a lane line recognition unit, used to recognize lane lines based on the lane line detection results and obtain lane line recognition results; the lane line detection results and the lane line recognition results are used as lane line information.

[0019] Optionally, the vehicle information acquisition unit includes: a vehicle detection unit, used to perform vehicle detection on a first image containing lane lines and obtain vehicle detection results; a vehicle recognition unit, used to recognize at least one vehicle around a first vehicle in the first image containing lane lines based on the vehicle detection results and obtain vehicle recognition results; the vehicle detection results and the vehicle recognition results are used as vehicle information.

[0020] According to another aspect of the present invention, a storage medium is provided, including a stored program, wherein, when the program is running, the device where the storage medium is located executes the vehicle driving state detection method described above.

[0021] According to another aspect of the present invention, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the vehicle driving state detection method of any one of the above-described embodiments by executing the executable instructions.

[0022] This invention proposes a vehicle driving status detection method that only requires a single image from an onboard camera to obtain lane line information and vehicle information of at least one vehicle surrounding the first vehicle, thereby determining the driving status of at least one vehicle around the first vehicle. This method does not rely on cameras installed on the road or require acquiring multiple frames of images, making it fast, flexible, and widely applicable. In particular, it can provide early warnings about the driving status of surrounding vehicles within a short period, assisting drivers in safe driving. Furthermore, since the number of lane lines on the road is far less than the number of vehicles, performing lane line detection and recognition first, followed by vehicle detection and recognition, significantly improves the efficiency of vehicle driving status detection. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0024] Figure 1 This is a flowchart of an optional vehicle driving status detection method according to an embodiment of the present invention;

[0025] Figure 2 This is a flowchart of another optional vehicle driving state detection method according to an embodiment of the present invention;

[0026] Figure 3 This is a structural block diagram of an optional first network model according to an embodiment of the present invention;

[0027] Figure 4 This is a structural block diagram of an optional second network model according to an embodiment of the present invention;

[0028] Figure 5 This is a structural block diagram of an optional third network model according to an embodiment of the present invention;

[0029] Figure 6 This is a structural block diagram of an optional fourth network model according to an embodiment of the present invention;

[0030] Figure 7 This is a structural block diagram of a vehicle driving status detection device according to an embodiment of the present invention;

[0031] Figure 8 This is a structural block diagram of another vehicle driving status detection device according to an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such orders can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] refer to Figure 1 This is a flowchart of an optional vehicle driving state detection method according to an embodiment of the present invention.

[0035] like Figure 1 As shown, the vehicle driving status detection method 10 includes:

[0036] Step S100: Use the onboard camera on the first vehicle to acquire a first image of the area around the first vehicle;

[0037] In one alternative embodiment, the vehicle-mounted camera refers to a forward-facing camera on the first vehicle, distinct from a surveillance camera. Specifically, the forward-facing camera may be installed in areas such as the hood, windshield, or front bumper.

[0038] Step S102: Perform lane line detection and recognition on the first image to obtain lane line information;

[0039] In one alternative embodiment, a neural network model can be used to detect and identify lane lines in the first image. The lane line information may include filtering out the first image containing lane lines, obtaining lane line location information and attribute information, etc.

[0040] Step S104: Detect and identify vehicles based on lane line information, and obtain vehicle information of at least one vehicle around the first vehicle;

[0041] In one alternative embodiment, a neural network model can be used to detect and identify vehicles based on lane line information, wherein vehicle information of at least one vehicle around the first vehicle includes vehicle position information and orientation information.

[0042] Step S106: Based on lane line information and vehicle information of at least one vehicle around the first vehicle, determine the driving status of at least one vehicle around the first vehicle.

[0043] The steps S100 to S106 described above only require a single first image acquired by the vehicle-mounted camera to obtain lane line information and vehicle information of at least one vehicle surrounding the first vehicle, thereby determining the driving status of at least one vehicle surrounding the first vehicle. This method does not rely on cameras installed on the road and does not require acquiring multiple frames of images, making the vehicle driving status detection method fast, flexible, and widely applicable. In particular, it can provide early warnings of the driving status of surrounding vehicles in a short period of time, assisting drivers in safe driving. Furthermore, since the number of lane lines on the road is far less than the number of vehicles, by first detecting and recognizing lane lines and then detecting and recognizing vehicles, the efficiency of vehicle driving status detection can be greatly improved.

[0044] In one optional specific embodiment, refer to Figure 2 The vehicle driving status detection method 20 includes:

[0045] Step S200: Use the onboard camera on the first vehicle to acquire a first image of the area around the vehicle;

[0046] Step S202: Perform lane line detection on the first image and obtain the lane line detection results;

[0047] In one optional embodiment, a first network model is used to detect lane lines in the first image to obtain lane line detection results. These results include filtering out first images containing lane lines and obtaining the position information of the lane lines. Figure 3As shown, the first network model is a U-shaped convolutional neural network with N1 first convolutional layer groups, a first downsampling group consisting of downsampling layers between every two first convolutional layer groups, and N1 second convolutional layer groups, a second upsampling group consisting of upsampling layers between every two second convolutional layer groups. Specifically, the first image first undergoes feature extraction through three first convolutional layer groups to obtain feature maps. A downsampling layer is added between every two first convolutional layer groups to downsample the feature maps, reducing their size to obtain a larger receptive field. Then, the feature maps are reconstructed through another three second convolutional layer groups, with an upsampling layer added between every two second convolutional layer groups to gradually restore the feature map size to the original input size. To improve the accuracy of lane line detection, short connections can be added between feature maps of the same size. If no lane lines are detected in this step, the vehicle driving state detection ends; otherwise, the next step is performed.

[0048] Step S204: Based on the lane line detection results, identify the lane lines to obtain the lane line identification results.

[0049] In one optional embodiment, a second network model is used to identify lane lines based on the lane line detection results to obtain lane line identification results. These results include lane line attributes, such as different attribute categories like lane line type and lane line color. Figure 4 As shown, the second network model is a multi-branch convolutional neural network with N2 groups of third convolutional layers, a second downsampling group consisting of downsampling layers between every two groups of third convolutional layers, and a first fully connected layer group connected to the output of the third convolutional layer group. The first fully connected layer group contains N3 parallel fully connected layers. The number of fully connected layers N3 can be less than or equal to the number of category information contained in the lane line attributes. When both are equal, higher recognition accuracy is achieved for different attribute categories. Specifically, the first image containing the lane lines undergoes image feature extraction through four groups of third convolutional layers to obtain feature maps. A downsampling layer is added between every two groups of third convolutional layers to downsample the feature maps, reducing their size to obtain a larger receptive field. Then, two parallel fully connected layers are used to recognize the lane line shape and color, respectively. To improve the accuracy of lane line recognition, short connections can be added between every two groups of third convolutional layers to fuse the feature information from the upper and lower layers. The system considers the attribute categories of lane lines (e.g., lane line type and lane line color) to determine if no target lane line (e.g., solid yellow line, including single and double yellow lines) is identified at this step. Otherwise, the vehicle driving status detection ends; otherwise, the next step is performed. In an optional embodiment, the first image containing lane lines can be cropped before being input into the third convolutional layer group. Cropping can narrow the recognition range to the image containing each lane line.

[0050] Step S206: Perform vehicle detection on the first image containing lane lines and obtain the vehicle detection results;

[0051] In one optional embodiment, a third network model is used to perform vehicle detection on a first image containing lane lines to obtain vehicle detection results, wherein the vehicle detection results may be vehicle detection boxes containing vehicle location information, etc. Figure 5 As shown, the third network model is a convolutional neural network with a detection head, having N3 groups of fourth convolutional layers. A third downsampling group consists of downsampling layers between every two groups of fourth convolutional layers, fully connected layers connected to the outputs of the fourth convolutional layers, and non-maximum suppression layers connected to the outputs of the fully connected layers. Specifically, the first image containing lane lines undergoes image feature extraction through four groups of fourth convolutional layers to obtain feature maps. A downsampling layer is added between every two groups of fourth convolutional layers to downsample the feature maps, reducing their size to obtain a larger receptive field. The fully connected layers identify the position coordinates of at least one vehicle and the confidence level of each coordinate. The non-maximum suppression layer filters the position coordinates of at least one vehicle based on their confidence level, retaining only the position coordinates with the highest confidence level in adjacent regions. Adjacent regions are defined as the union of any two detection boxes with an intersection-union ratio greater than a preset threshold. This effectively reduces false positives in vehicle detection. To improve the accuracy of vehicle detection, short connections can be added between every two groups of fourth convolutional layers to fuse feature information from upper and lower layers. If no vehicle is detected in this step, the vehicle driving status detection ends; otherwise, proceed to the next step.

[0052] Step S208: Based on the vehicle detection results, identify at least one vehicle around the first vehicle in the first image containing lane lines to obtain vehicle identification results;

[0053] In one optional embodiment, a fourth network model is used to identify at least one vehicle around the first vehicle based on the vehicle detection results, obtaining vehicle identification results, which include the orientation of at least one vehicle. Figure 6As shown, the fourth network model is a convolutional neural network with N4 fifth convolutional layer groups. A fourth downsampling group consists of downsampling layers between every two fifth convolutional layer groups, and fully connected layers connect to the outputs of the fifth convolutional layer groups. Specifically, the first image containing lane lines undergoes image feature extraction through four fifth convolutional layer groups to obtain feature maps. A downsampling layer is added between every two fifth convolutional layer groups to downsample the feature maps, reducing their size to obtain a larger receptive field. The fully connected layer then identifies the vehicle's orientation. To improve vehicle recognition accuracy, short connections can be added between every two fifth convolutional layer groups to fuse feature information from upper and lower layers. In an optional embodiment, the first image containing lane lines can be cropped based on the vehicle detection results before being input into the fifth convolutional layer groups. Cropping can narrow the recognition range to the area of ​​each vehicle.

[0054] Step S210: Based on the lane line detection results, lane line recognition results, vehicle detection results, and vehicle recognition results, determine the driving status of at least one vehicle around the first vehicle.

[0055] It should be noted that N1 to N4 are positive integers.

[0056] The steps S200 to S210 described above only require a single first image acquired by the vehicle-mounted camera to obtain lane line detection results and lane line recognition results as lane line information, and vehicle detection results and vehicle recognition results as vehicle information for at least one vehicle surrounding the first vehicle, thereby determining the driving status of at least one vehicle surrounding the first vehicle. This method does not rely on cameras installed on the road, nor does it require acquiring multiple frames of images. As a result, this vehicle driving status detection method is fast, flexible, and has a wide range of applications. In particular, it can determine and warn of the driving status of at least one vehicle surrounding the first vehicle within a short period of time, assisting the driver in safe driving. Furthermore, since the number of lane lines on the road is far less than the number of vehicles, by performing lane line detection and recognition first, and then vehicle detection and recognition, the efficiency of vehicle driving status detection can be greatly improved.

[0057] Taking the determination of vehicles traveling in the wrong direction within China as an example, the default orientation of the first vehicle is forward travel. When the lane line recognition result is a solid yellow line, if the recognition result of at least one vehicle around the first vehicle also indicates forward travel, and the lane line position information in the lane line detection result and the vehicle position information in the vehicle detection result indicate that at least one vehicle around the first vehicle is located to the left of the solid yellow line, or if the recognition result of at least one vehicle around the first vehicle indicates reverse travel, and the lane line position information in the lane line detection result and the vehicle position information in the vehicle detection result indicate that at least one vehicle around the first vehicle is located to the right of the solid yellow line, then it is determined that at least one vehicle around the first vehicle is traveling in the wrong direction.

[0058] In one optional embodiment, the vehicle driving status detection method may further include obstacle detection, pedestrian detection, traffic signal detection, and sign detection. For example, after determining the driving status of at least one vehicle, the method can further combine the obstacle detection results and transmit them to a display device for display or to an alarm device for alarm reminder. As another example, after determining the driving status of at least one vehicle, the method can further determine whether the driving status constitutes a traffic violation. If it is a traffic violation, the vehicle's location information and license plate information can be reported to the relevant regulatory authorities to reduce traffic violations and improve driving safety.

[0059] Those skilled in the art will recognize that the above-mentioned vehicle driving status detection method can also be extended to other scenarios with similar imaging characteristics (height, position) to the above-mentioned vehicle-mounted camera, such as non-motorized vehicle scenarios.

[0060] refer to Figure 7 The diagram below shows a structural block diagram of a vehicle driving state detection device according to an embodiment of the present invention. In this embodiment, the vehicle driving state detection device 70 includes an image acquisition unit 700, a lane line information acquisition unit 702, a vehicle information acquisition unit 704, and a driving state judgment unit 706.

[0061] An image acquisition unit 700 is installed on the first vehicle to acquire a first image of the area surrounding the first vehicle. In an optional embodiment, the image acquisition unit 700 refers to a front-view camera on the first vehicle, distinct from a surveillance camera, and can be installed in areas such as the hood, windshield, and front bumper of the first vehicle.

[0062] The lane line information acquisition unit 702 is used to detect and recognize lane lines in the first image to acquire lane line information. In an optional embodiment, the lane line information acquisition unit 702 uses a neural network model to detect and recognize lane lines in the first image, and the lane line information may include filtering out the first image containing lane lines, acquiring lane line location information and attribute information, etc.

[0063] The vehicle information acquisition unit 704 is used to detect and identify vehicles based on lane line information, and to acquire vehicle information of at least one vehicle surrounding the first vehicle. In an optional embodiment, the vehicle information acquisition unit 704 can use a neural network model to detect and identify vehicles based on lane line information, and the vehicle information of at least one vehicle surrounding the first vehicle includes vehicle position information and orientation information, etc.

[0064] The driving status determination unit 706 is used to determine the driving status of at least one vehicle based on lane line information and vehicle information of at least one vehicle around the first vehicle.

[0065] In optional embodiments, the image acquisition unit 700, the lane line information acquisition unit 702, the vehicle information acquisition unit 704, and the driving state judgment unit 706 can be discrete components; or partially integrated components; or fully integrated components.

[0066] In one optional specific embodiment, refer to Figure 8 The vehicle driving status detection device 80 includes an image acquisition unit 800, a lane line detection unit 802, a lane line recognition unit 804, a vehicle detection unit 806, a vehicle recognition unit 808, and a driving status judgment unit 810.

[0067] Image acquisition unit 800 is installed on the first vehicle to acquire a first image of the area surrounding the first vehicle. In an optional embodiment, image acquisition unit 700 refers to a front-view camera on the first vehicle, distinct from a surveillance camera, and can be installed in areas such as the engine hood, windshield, and front bumper of the first vehicle.

[0068] Lane detection unit 802 performs lane detection on the first image and obtains lane detection results. In an optional embodiment, lane detection unit 802 uses a first network model to perform lane detection on the first image and obtains lane detection results, wherein the lane detection results include filtering out the first image containing lane lines and obtaining the position information of the lane lines.

[0069] The lane line recognition unit 804 is used to recognize lane lines based on lane line detection results to obtain lane line recognition results. In an optional embodiment, the lane line recognition unit 804 uses a second network model to recognize lane lines based on lane line detection results to obtain lane line recognition results, wherein the lane line recognition results include lane line attributes, such as lane line type, lane line color, and other different categories of information.

[0070] The vehicle detection unit 806 is used to perform vehicle detection on a first image containing lane lines and obtain vehicle detection results. In an optional embodiment, the vehicle detection unit 806 uses a third network model to perform vehicle detection on the first image containing lane lines and obtain vehicle detection results, wherein the vehicle detection results may include vehicle detection boxes containing vehicle location information, etc.

[0071] The vehicle recognition unit 808 is configured to identify at least one vehicle surrounding a first vehicle in a first image containing lane lines based on vehicle detection results, thereby obtaining vehicle recognition results. In an optional embodiment, the vehicle recognition unit 808 uses a fourth network model to identify at least one vehicle surrounding a first vehicle in a first image containing lane lines based on vehicle detection results, thereby obtaining vehicle recognition results, wherein the vehicle recognition results include the orientation of at least one vehicle.

[0072] The driving status determination unit 810 is used to determine the driving status of at least one vehicle around the first vehicle based on the lane line detection result, lane line recognition result, vehicle detection result, and vehicle recognition result.

[0073] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0074] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0075] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0076] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0077] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0078] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0079] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting the driving status of a vehicle, comprising: A first image of the area around the first vehicle is acquired using an onboard camera on the first vehicle. The first image is a single image and includes lane lines. The first image is subjected to lane line detection and recognition to obtain lane line information, which includes lane line location information and attribute information. Based on the lane line information, vehicle detection and identification are performed to obtain vehicle information of at least one vehicle around the first vehicle, including vehicle position information and orientation information. Based on the lane line information and the vehicle information of at least one vehicle around the first vehicle, the driving status of at least one vehicle around the first vehicle is determined, including: when the first vehicle is driving in the forward direction and the lane line recognition result is a solid yellow line, if the recognition result of at least one vehicle around the first vehicle shows that it is driving in the forward direction, and the at least one vehicle around the first vehicle is determined to be to the left of the solid yellow line based on the lane line position information in the lane line detection result and the vehicle position information in the vehicle detection result, or if the recognition result of at least one vehicle around the first vehicle shows that it is driving in the reverse direction, and the at least one vehicle around the first vehicle is determined to be to the right of the solid yellow line based on the lane line position information in the lane line detection result and the vehicle position information in the vehicle detection result, then the at least one vehicle around the first vehicle is determined to be driving in the reverse direction.

2. The vehicle driving status detection method according to claim 1, characterized in that, The step of detecting and recognizing lane lines in the first image to obtain lane line information includes: Lane line detection is performed on the first image to obtain the lane line detection result; Based on the lane line detection results, the lane lines are identified to obtain lane line identification results; The lane line detection result and the lane line recognition result are used as the lane line information.

3. The vehicle driving status detection method according to claim 1, characterized in that, The process of detecting and identifying vehicles based on the lane line information, and obtaining vehicle information of at least one vehicle surrounding the first vehicle, includes: Vehicle detection is performed on the first image containing lane lines to obtain vehicle detection results; Based on the vehicle detection results, at least one vehicle around the first vehicle in the first image containing lane lines is identified to obtain vehicle identification results. The vehicle detection result and the vehicle identification result are used as the vehicle information.

4. The vehicle driving status detection method according to claim 2, characterized in that, The first image is used to perform lane line detection using a first network model to obtain the lane line detection result, wherein the lane line detection result includes filtering out the first image containing lane lines and obtaining the position information of the lane lines.

5. The vehicle driving status detection method according to claim 4, characterized in that, The first network model is a U-shaped convolutional neural network, including N1 first convolutional layer groups, a first downsampling group consisting of downsampling layers between every two first convolutional layer groups, N1 second convolutional layer groups, and a second upsampling group consisting of upsampling layers between every two second convolutional layer groups.

6. The vehicle driving status detection method according to claim 2, characterized in that, The second network model is used to identify the lane lines based on the lane line detection results to obtain lane line identification results, wherein the lane line identification results include lane line shape and lane line color.

7. The vehicle driving status detection method according to claim 6, characterized in that, The second network model is a multi-branch convolutional neural network, including N2 third convolutional layer groups, a second downsampling group consisting of downsampling layers between every two third convolutional layer groups, and a first fully connected layer group connected to the output of the third convolutional layer groups, wherein the first fully connected layer group contains N3 parallel fully connected layers.

8. The vehicle driving status detection method according to claim 3, characterized in that, A third network model is used to perform vehicle detection on the first image containing lane lines to obtain the vehicle detection result, wherein the vehicle detection result includes vehicle detection boxes containing vehicle location information.

9. The vehicle driving status detection method according to claim 8, characterized in that, The third network model is a convolutional neural network with a detection head, including N3 fourth convolutional layer groups, a third downsampling group consisting of downsampling layers between every two fourth convolutional layer groups, a fully connected layer connected to the output of the fourth convolutional layer group, and a non-maximum suppression layer connected to the output of the fully connected layer.

10. The vehicle driving status detection method according to claim 3, characterized in that, Using a fourth network model, at least one vehicle around the first vehicle in the first image containing lane lines is identified based on the vehicle detection results, and the vehicle identification results include the orientation of the at least one vehicle.

11. The vehicle driving status detection method according to claim 10, characterized in that, The fourth network model is a convolutional neural network, including N4 fifth convolutional layer groups, a fourth downsampling group consisting of downsampling layers between every two fifth convolutional layer groups, and a fully connected layer connected to the output of the fifth convolutional layer group.

12. The vehicle driving status detection method according to claim 1, characterized in that, The vehicle-mounted camera is a forward-facing camera installed on the first vehicle.

13. A vehicle driving status detection device, comprising: An image acquisition unit is installed on a first vehicle to acquire a first image of the area surrounding the first vehicle. The first image is a single image and includes lane lines. The lane line information acquisition unit is used to detect and recognize lane lines in the first image and acquire lane line information, which includes lane line position information and attribute information. The vehicle information acquisition unit is used to detect and identify vehicles based on the lane line information, and to acquire vehicle information of at least one vehicle around the first vehicle, the vehicle information including vehicle position information and orientation information; The driving state determination unit is used to determine the driving state of the at least one vehicle based on the lane line information and the vehicle information of at least one vehicle around the first vehicle, including: when the first vehicle is driving in the forward direction and the lane line recognition result is a solid yellow line, if the recognition result of the at least one vehicle around the first vehicle shows that it is driving in the forward direction, and it is determined that the at least one vehicle around the first vehicle is located to the left of the solid yellow line based on the lane line position information in the lane line detection result and the vehicle position information in the vehicle detection result; or if the recognition result of the at least one vehicle around the first vehicle shows that it is driving in the reverse direction, and it is determined that the at least one vehicle around the first vehicle is located to the right of the solid yellow line based on the lane line position information in the lane line detection result and the vehicle position information in the vehicle detection result, then it is determined that the at least one vehicle around the first vehicle is in a reverse driving state.

14. The vehicle driving status detection device according to claim 13, characterized in that, The lane line information acquisition unit includes: A lane line detection unit is used to perform lane line detection on the first image and obtain the lane line detection result; The lane line recognition unit is used to recognize the lane line based on the lane line detection result and obtain the lane line recognition result; The lane line detection result and the lane line recognition result serve as the lane line information.

15. The vehicle driving status detection device according to claim 13, characterized in that, The vehicle information acquisition unit includes: The vehicle detection unit is used to perform vehicle detection on the first image containing lane lines and obtain vehicle detection results. A vehicle recognition unit is configured to identify at least one vehicle surrounding the first vehicle in the first image containing lane lines based on the vehicle detection result, and obtain a vehicle recognition result. The vehicle detection result and the vehicle identification result serve as the vehicle information.

16. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the storage medium to perform the vehicle driving state detection method according to any one of claims 1 to 12.

17. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the vehicle driving state detection method according to any one of claims 1 to 12 by executing the executable instructions.

Images