Lane congestion detection method, device, electronic device and storage medium

Abstract

The embodiments of the present application disclose a lane congestion detection method, device, electronic device, and storage medium, wherein the method specifically includes: obtaining real-time video stream data in a target monitoring scenario, and determining a driving road focus area map in the target monitoring scenario based on the video stream data; determining an optical flow matrix corresponding to the driving road focus area map based on the video stream data and the driving road focus area map, in combination with a pre-trained optical flow extraction neural network model; determining a segmentation result feature map of the road idle area and the road occupied area in the target monitoring scenario based on the video stream data, in combination with a pre-trained semantic segmentation neural network model; and determining a road congestion detection result based on the optical flow matrix and the segmentation result feature map using a pre-trained congestion detection model. The embodiments of the present application detect road congestion results based on two feature indicators, the optical flow matrix and the segmentation result feature map, thereby improving the accuracy of the road congestion detection results.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation, and in particular to a lane congestion detection method, device, electronic device and storage medium. Background Art

[0002] With the improvement of my country's economic development level, more and more families choose to buy cars for travel. With the increase in car sales, road congestion has also become a headache for drivers. Therefore, how to detect congested road sections is a technical problem that needs to be solved urgently.

[0003] Currently, the commonly used method for detecting road congestion is as follows: camera calibration is performed on the road scene, and then based on the traffic congestion visual detection system, the ratio of stationary vehicles on the road to all vehicles is calculated, and the road congestion is determined based on the ratio.

[0004] However, this method has certain shortcomings: using only a single indicator, the ratio of stationary vehicles to all vehicles, to determine whether a road is congested results in low accuracy. Summary of the Invention

[0005] The embodiments of the present application provide a lane congestion detection method, device, electronic device, and storage medium to achieve the purpose of improving the accuracy of road congestion detection.

[0006] In a first aspect, an embodiment of the present application provides a lane congestion detection method, the method comprising:

[0007] Acquire real-time video stream data of the target monitoring scene, and determine the driving road focus area map of the target monitoring scene based on the video stream data;

[0008] Based on the video stream data and the road area of ​​interest map, and combined with a pre-trained optical flow extraction neural network model, the optical flow matrix corresponding to the road area of ​​interest map is determined;

[0009] Based on the video stream data and combined with the pre-trained semantic segmentation neural network model, the segmentation result feature map of the road idle area and the road occupied area in the target monitoring scene is determined;

[0010] Based on the optical flow matrix and the segmentation result feature map, the road congestion detection result is determined using the pre-trained congestion detection model.

[0011] In a second aspect, an embodiment of the present application provides a lane congestion detection device, comprising:

[0012] The data acquisition and processing module is used to obtain real-time video stream data in the target monitoring scene and determine the driving road focus area map in the target monitoring scene based on the video stream data;

[0013] An optical flow matrix extraction module is used to determine the optical flow matrix corresponding to the road area of ​​interest map based on the video stream data and the road area of ​​interest map, in combination with a pre-trained optical flow extraction neural network model;

[0014] The segmentation module is used to determine the segmentation result feature map of the road idle area and road occupied area in the target monitoring scene based on the video stream data and combined with the pre-trained semantic segmentation neural network model;

[0015] The congestion detection module is used to determine the road congestion detection result based on the optical flow matrix and the segmentation result feature map using a pre-trained congestion detection model.

[0016] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0017] one or more processors;

[0018] a storage device for storing one or more programs,

[0019] When one or more programs are executed by one or more processors, the one or more processors implement the lane congestion detection method as in any embodiment of the present application.

[0020] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements a lane congestion detection method as in any embodiment of the present application.

[0021] In an embodiment of the present application, based on the two feature indicators of the acquired optical flow matrix and the segmentation result feature map, and combined with the congestion detection model, the detection of road congestion results is achieved. Compared with the existing technology of detecting road congestion status through a single indicator, the accuracy of the road congestion detection results is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flowchart of the lane congestion detection method in the first embodiment of the present application;

[0023] Figure 2 is a flowchart of a lane congestion detection method in the second embodiment of the present application;

[0024] Figure 3 is a schematic structural diagram of a lane congestion detection device in a third embodiment of the present application;

[0025] Figure 4 It is a structural diagram of an electronic device for implementing a lane congestion detection method in the fourth embodiment of the present application. DETAILED DESCRIPTION

[0026] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the present application and are not intended to limit the present application. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present application, not all structures.

[0027] Figure 1 This is a flowchart of the lane congestion detection method of the first embodiment of the present application. This embodiment can be applied to a server or other computer device to detect whether a road is congested based on the video stream data obtained in any road monitoring scenario. The method can be executed by a lane congestion detection device, which can be implemented in software and / or hardware and can be integrated into an electronic device, such as a server or computer device integrated into a monitoring system.

[0028] like Figure 1 As shown in FIG, the lane congestion detection method specifically includes the following processes:

[0029] S101: Acquire real-time video stream data in a target monitoring scene, and determine a driving road focus area map in the target monitoring scene based on the video stream data.

[0030] In the embodiments of the present application, the target monitoring scene is exemplified as a city road monitoring scene or a highway monitoring scene, but other monitoring scenes are also possible and are not specifically limited herein. A camera is installed in the target monitoring scene to capture real-time video stream data from the scene, and a backend server or computer device in the monitoring system can obtain the video stream data captured by the camera in real time. The video stream data is composed of still images, which are referred to as video frames.

[0031] In the embodiments of the present application, the road area of ​​interest map refers to an image of the road in the target monitoring scene. When determining the road area of ​​interest map in the target monitoring scene based on video stream data, the road area of ​​interest map can optionally be determined based on the motion trajectories of moving vehicles contained in the video stream data using image recognition technology. The resulting road area of ​​interest map can optionally be a binary image.

[0032] S102: Determine an optical flow matrix corresponding to the driving road area of ​​interest map based on the video stream data and the driving road area of ​​interest map and in combination with a pre-trained optical flow extraction neural network model.

[0033] In the embodiments of this application, optical flow is the instantaneous speed of pixel motion of a spatially moving object on the observation imaging plane. It utilizes the temporal changes in pixels in an image sequence and the correlation between adjacent frames to find the correspondence between the previous and current frames, thereby calculating the motion information of the object between adjacent frames. Generally speaking, optical flow is generated by the movement of the foreground object itself in the scene, the movement of the camera, or both.

[0034] In the embodiments of the present application, determining the optical flow matrix corresponding to the road area of ​​interest map is to determine the motion information between vehicles within the road area of ​​interest map. Optionally, to improve the efficiency of obtaining the optical flow matrix, after obtaining the video stream data and the road area of ​​interest map, a pre-trained optical flow extraction neural network model can be used to extract the optical flow matrix corresponding to the road area of ​​interest map. The optical flow extraction neural network model can optionally be a FlowNet optical flow extraction network.

[0035] In an optional embodiment, the optical flow matrix corresponding to the driving road attention area map is determined based on the video stream data and the driving road attention area map, in combination with a pre-trained optical flow extraction neural network model, including: taking every two adjacent frames of images in the video stream data as model input data, inputting them into the optical flow extraction neural network model in sequence, and obtaining multiple initial optical flow matrices according to the model output; superimposing the multiple initial optical flow matrices, and normalizing the elements in the superimposed optical flow matrices to obtain a target optical flow matrix; superimposing the target optical flow matrix and the driving road attention area map to obtain the optical flow matrix corresponding to the driving road attention area map.

[0036] For example, two adjacent video frames in the video stream data are input into the optical flow extraction neural network model, and the optical flow matrix O is obtained according to the output. i1 Repeat the above steps and input the other two adjacent video frames in the video stream data into the optical flow extraction neural network model to obtain the optical flow matrix O i2 , O i3 ......O in , where O i1 , O i2 , O i3 ......O in is the initial optical flow matrix. Each element of the initial optical flow matrix carries direction information. The direction of the road can be determined based on the direction information. The value of n is equal to the number of repetitions and can be determined according to actual needs.

[0037] When multiple initial optical flow matrices are superimposed, the following formula can be used:

[0038] O avg =(O i1 +Oi2 +……+O in ) / n; where O avg Represents the superimposed target optical flow matrix.

[0039] When normalizing the elements in the superimposed optical flow matrix, O avg Each element in is normalized according to the following formula: 0<i<H,0<j<W;wherein, is the optical flow vector element located at row i and column j in the target optical flow matrix, and H and W are the height and width of the video frame image respectively.

[0040] When superimposing the target optical flow matrix and the driving road attention area map, optionally, since the driving road attention area map is a binary image, the target optical flow matrix and the driving road attention area map can be directly dot-multiplied to achieve the superposition operation, and then the optical flow matrix corresponding to the driving road attention area map is obtained according to the dot-multiplication result.

[0041] S103: Determine a segmentation result feature map of the road idle area and the road occupied area in the target monitoring scene based on the video stream data and in combination with a pre-trained semantic segmentation neural network model.

[0042] Among them, the semantic segmentation neural network model can output a semantic image marked with different instance object information based on the input image. In an embodiment of the present application, in the trained semantic segmentation neural network model, each lane area is divided and labeled according to occupation and vacancy and a training set is constructed, that is, the sample images in the training set are divided into images containing vehicles (lane areas are occupied) and images not containing vehicles (lane areas are idle); then training is performed based on the samples in the training set, and the loss function used in the training process is as follows: L = [y.log(p) + (1-y) · log(1-p)]; wherein y represents the label of the sample, the positive class (for example, the lane area is occupied) is 1, the negative class (for example, the lane area is idle) is 0, and p represents the probability that the sample is predicted to be a positive class.

[0043] In specific implementation, the video stream data can be input frame by frame into the trained semantic segmentation neural network model, and the segmentation result feature map of the marked road idle area and road occupied area can be obtained according to the output of the model, wherein the segmentation result feature map can optionally be a binary image.

[0044] S104: Based on the optical flow matrix and the segmentation result feature map, a pre-trained congestion detection model is used to determine a road congestion detection result.

[0045] In an optional embodiment, based on the optical flow matrix and the segmentation result feature map, a pre-trained congestion detection model is used to determine the road congestion detection result, including: fusing the optical flow matrix and the segmentation result feature map, for example, splicing the optical flow matrix and the segmentation result feature map, and inputting the fused feature data into the pre-trained congestion detection model; and determining whether the road is congested based on the output result of the congestion detection model.

[0046] It should be noted here that the process of training the congestion detection model includes: obtaining first video data when a congestion event occurs in the target monitoring scenario and second video data when no congestion event occurs; constructing a congestion classification sample set based on the first video data and the second video data, wherein the congestion classification sample set includes the optical flow matrix corresponding to the driving road attention area map corresponding to the congested and non-congested situations, and the segmentation result feature map of the road idle area and the road occupied area; and the specific process of obtaining the optical flow matrix corresponding to the driving road attention area map, and the segmentation result feature map of the road idle area and the road occupied area can be found in the above content and will not be repeated here; based on the congestion classification sample set and the preset calculation function, the congestion detection model is trained.

[0047] In an embodiment of the present application, based on the two feature indicators of the acquired optical flow matrix and the segmentation result feature map, and combined with the congestion detection model, the detection of road congestion results is achieved. Compared with the existing technology of detecting road congestion status through a single indicator, the accuracy of the road congestion detection results is improved.

[0048] Figure 2 This is a flow chart of the lane congestion detection method according to the second embodiment of the present application. This embodiment is optimized based on the above embodiment. Figure 2 , the method comprising:

[0049] S201: Acquire real-time video stream data in a target monitoring scenario.

[0050] In the embodiment of the present application, the video stream data is composed of still images, which can be called video frames.

[0051] S202: Extracting a preset number of foreground images from the video stream data based on background modeling technology.

[0052] In the embodiment of the present application, a Gaussian mixture model is exemplarily selected for background modeling. Background modeling techniques may also be employed, which are not specifically limited herein. Specifically, when extracting a foreground image, after each video frame is acquired in S101, the Gaussian mixture model is used to extract the foreground image from the video frame, thereby obtaining multiple foreground images, wherein the foreground image may optionally be a binary image including a vehicle.

[0053] S203: Superimpose a preset number of foreground images to obtain a driving road focus area map.

[0054] In the embodiment of the present application, the preset number of frames can be 75, but other values ​​are also possible and are not specifically limited herein. The superposition of the preset number of foreground images is equivalent to fusing the preset number of foreground images. Since vehicles can only travel on roads, a road area of ​​interest map can be obtained based on the vehicle's position trajectory in the fused images.

[0055] In an embodiment of the present application, to ensure the effective overlay of foreground images, adaptive dilation processing may be performed on the extracted foreground images before a preset number of foreground images are overlaid. Optionally, after obtaining the preset number of foreground images, adaptive dilation processing may be uniformly performed on each foreground image; alternatively, adaptive dilation processing may be performed directly on each extracted foreground image. It should be noted that adaptive dilation processing is performed because, in a foreground image, vehicles or roads closer to the camera appear larger, while vehicles or roads farther from the camera appear smaller. If dilation processing is performed according to a uniform standard, the display effect will be poor.

[0056] Furthermore, the resulting road area of ​​interest map may contain noise or glitches. To ensure its quality, it needs to be preprocessed. Preprocessing includes filtering (e.g., median filtering) and smoothing (e.g., edge smoothing). It should be noted that preprocessing can also include other specified operations, which can be set according to user needs.

[0057] S204 : Determine an optical flow matrix corresponding to the driving road area of ​​interest map based on the video stream data and the driving road area of ​​interest map, in combination with a pre-trained optical flow extraction neural network model.

[0058] S205 : Determine a segmentation result feature map of the road idle area and the road occupied area in the target monitoring scenario based on the video stream data and in combination with a pre-trained semantic segmentation neural network model.

[0059] S206 : Based on the optical flow matrix and the segmentation result feature map, a pre-trained congestion detection model is used to determine a road congestion detection result.

[0060] In an embodiment of the present application, a driving road area of ​​interest map is obtained by superimposing a preset number of foreground images, thereby improving the efficiency of obtaining the driving road area of ​​interest map; and by adaptively dilating the foreground image and filtering and smoothing the driving road area of ​​interest map, the quality of the driving road area of ​​interest map is ensured.

[0061] Figure 3 This is a schematic diagram of the structure of the lane congestion detection device of the third embodiment of the present application. This embodiment can be applied to a server or other computer device to detect whether a road is congested based on the acquired monitoring video stream data in any scenario. Figure 3 , the device comprises:

[0062] The data acquisition and processing module 301 is used to acquire real-time video stream data of the target monitoring scene and determine the driving road focus area map of the target monitoring scene based on the video stream data;

[0063] An optical flow matrix extraction module 302 is configured to determine an optical flow matrix corresponding to the road area of ​​interest map based on the video stream data and the road area of ​​interest map, in combination with a pre-trained optical flow extraction neural network model;

[0064] The segmentation module 303 is used to determine the segmentation result feature map of the road idle area and the road occupied area in the target monitoring scene based on the video stream data and in combination with the pre-trained semantic segmentation neural network model;

[0065] The congestion detection module 304 is configured to determine a road congestion detection result based on the optical flow matrix and the segmentation result feature map using a pre-trained congestion detection model.

[0066] In an embodiment of the present application, based on the two feature indicators of the acquired optical flow matrix and the segmentation result feature map, and combined with the congestion detection model, the detection of road congestion results is achieved. Compared with the existing technology of detecting road congestion status through a single indicator, the accuracy of the road congestion detection results is improved.

[0067] Based on the above embodiment, optionally, the data acquisition and processing module includes:

[0068] A foreground image acquisition unit is used to extract a preset number of foreground images from the video stream data based on background modeling technology;

[0069] The foreground superposition unit is used to superimpose a preset number of foreground images to obtain a driving road focus area map.

[0070] Based on the above embodiment, optionally, the data acquisition and processing module further includes:

[0071] The expansion processing unit is used to perform adaptive expansion processing on the extracted foreground images before superimposing a preset number of foreground images.

[0072] Based on the above embodiment, optionally, after obtaining the driving road area of ​​interest map, the data acquisition and processing module further includes:

[0073] The preprocessing unit is used to preprocess the driving road area of ​​interest map after obtaining the driving road area of ​​interest map, wherein the preprocessing includes filtering processing and smoothing processing.

[0074] Based on the above embodiment, optionally, the optical flow matrix extraction module includes:

[0075] The first matrix acquisition unit is used to input each two adjacent frames of image in the video stream data as model input data into the optical flow extraction neural network model in sequence, and obtain multiple initial optical flow matrices according to the model output;

[0076] The second matrix acquisition unit is used to superimpose multiple initial optical flow matrices and normalize the elements in the superimposed optical flow matrices to obtain a target optical flow matrix;

[0077] The third matrix acquisition unit is used to superimpose the target optical flow matrix and the driving road attention area map to obtain the optical flow matrix corresponding to the driving road attention area map.

[0078] Based on the above embodiment, optionally, the congestion detection module includes:

[0079] The fusion input unit is used to fuse the optical flow matrix and the segmentation result feature map, and input the fused feature data into the pre-trained congestion detection model;

[0080] The detection result output unit is used to determine whether the road is congested according to the output result of the congestion detection model.

[0081] Based on the above embodiment, the device may optionally further include a model training module for training a congestion detection model, specifically including:

[0082] a data acquisition unit, configured to acquire first video data when a congestion event occurs in a target monitoring scene and second video data when no congestion event occurs;

[0083] a sample set construction unit, configured to construct a congestion classification sample set based on the first video data and the second video data, wherein the congestion classification sample set includes an optical flow matrix corresponding to a driving road focus area map corresponding to each of congested and non-congested situations, and a segmentation result feature map of a road idle area and a road occupied area;

[0084] The training unit is used to train the congestion detection model based on the congestion classification sample set.

[0085] The lane congestion detection device provided in the embodiments of the present application can execute the lane congestion detection method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0086] Figure 4 Schematic diagram of the structure of an electronic device provided in the fourth embodiment of the present application. Figure 4 The electronic device provided in the embodiment of the present application includes: one or more processors 402 and a memory 401; the processor 402 in the electronic device can be one or more, Figure 4 In the figure, a processor 402 is taken as an example; the memory 401 is used to store one or more programs; the one or more programs are executed by one or more processors 402, so that the one or more processors 402 implement the lane congestion detection method as any one of the embodiments of the present application.

[0087] The electronic device may further include: an input device 403 and an output device 404 .

[0088] The processor 402, memory 401, input device 403 and output device 404 in the electronic device can be connected through a bus or other means. Figure 4 The bus connection is taken as an example.

[0089] The memory 401 in the electronic device serves as a computer-readable storage medium and can be used to store one or more programs, such as software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the lane congestion detection method provided in the embodiments of this application. The processor 402 executes the software programs, instructions, and modules stored in the memory 401 to execute various functional applications and data processing of the electronic device, thereby implementing the lane congestion detection method in the above-mentioned method embodiments.

[0090] The memory 401 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 401 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 401 may further include a memory remotely located relative to the processor 402, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0091] The input device 403 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 404 may include a display device such as a display screen.

[0092] Furthermore, when one or more programs included in the electronic device are executed by the one or more processors 402, the programs perform the following operations:

[0093] Acquire real-time video stream data of the target monitoring scene, and determine the driving road focus area map of the target monitoring scene based on the video stream data;

[0094] Based on the video stream data and the road area of ​​interest map, and combined with a pre-trained optical flow extraction neural network model, the optical flow matrix corresponding to the road area of ​​interest map is determined;

[0095] Based on the video stream data and combined with the pre-trained semantic segmentation neural network model, the segmentation result feature map of the road idle area and the road occupied area in the target monitoring scene is determined;

[0096] Based on the optical flow matrix and the segmentation result feature map, the road congestion detection result is determined using the pre-trained congestion detection model.

[0097] Of course, those skilled in the art will understand that when one or more programs included in the above-mentioned electronic device are executed by one or more processors 402, the program can also perform relevant operations in the lane congestion detection method provided in any embodiment of the present application.

[0098] In one embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the program is used to perform a lane congestion detection method, the method comprising:

[0099] Acquire real-time video stream data of the target monitoring scene, and determine the driving road focus area map of the target monitoring scene based on the video stream data;

[0100] Based on the video stream data and the road area of ​​interest map, and combined with a pre-trained optical flow extraction neural network model, the optical flow matrix corresponding to the road area of ​​interest map is determined;

[0101] Based on the video stream data and combined with the pre-trained semantic segmentation neural network model, the segmentation result feature map of the road idle area and the road occupied area in the target monitoring scene is determined;

[0102] Based on the optical flow matrix and the segmentation result feature map, the road congestion detection result is determined using the pre-trained congestion detection model.

[0103] Optionally, when the program is executed by a processor, it can also be used to execute the method provided in any embodiment of the present application.

[0104] The computer storage medium of the embodiment of the present application can adopt any combination of one or more computer-readable media.Computer-readable media can be computer-readable signal media or computer-readable storage media.Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination thereof.More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM), flash memories, optical fibers, portable CD-ROMs, optical storage devices, magnetic storage devices, or any suitable combination thereof.Computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0105] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0106] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.

[0107] The computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network (for example, including a local area network (LAN) or a wide area network (WAN)), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0108] Note that the above are only preferred embodiments of the present application and the technical principles employed. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present application. The scope of the present application is determined by the scope of the appended claims.

Claims

1. A lane congestion detection method, characterized in that: include: Acquire real-time video stream data of a target monitoring scene, and determine a driving road focus area map of the target monitoring scene based on the video stream data; Determine an optical flow matrix corresponding to the road area of ​​interest map based on the video stream data and the road area of ​​interest map, in combination with a pre-trained optical flow extraction neural network model; Determine, based on the video stream data and in combination with a pre-trained semantic segmentation neural network model, a segmentation result feature map of a road idle area and a road occupied area in the target monitoring scene; Determining a road congestion detection result based on the optical flow matrix and the segmentation result feature map using a pre-trained congestion detection model includes: Fusing the optical flow matrix and the segmentation result feature map, and inputting the fused feature data into a pre-trained congestion detection model; Whether the road is congested is determined according to the output result of the congestion detection model.

2. The method according to claim 1, characterized in that Determining a driving road area of ​​interest map in the target monitoring scenario according to the video stream data includes: Extracting a preset number of foreground images from the video stream data based on background modeling technology; A preset number of foreground images are superimposed to obtain the driving road focus area map.

3. The method according to claim 2, characterized in that Before superimposing the preset number of foreground images, the method further includes: Adaptively dilate the extracted foreground image.

4. The method according to claim 2, characterized in that After obtaining the driving road area of ​​interest map, the method further includes: The driving road area of ​​interest map is preprocessed, wherein the preprocessing includes filtering and smoothing.

5. The method according to claim 1, wherein Determining an optical flow matrix corresponding to the road area of ​​interest map based on the video stream data and the road area of ​​interest map in combination with a pre-trained optical flow extraction neural network model includes: Each two adjacent frames of images in the video stream data are used as model input data, and are sequentially input into an optical flow extraction neural network model, and a plurality of initial optical flow matrices are obtained according to the model output; Superimpose multiple initial optical flow matrices and normalize the elements in the superimposed optical flow matrix to obtain the target optical flow matrix; The target optical flow matrix and the driving road area of ​​interest map are superimposed to obtain an optical flow matrix corresponding to the driving road area of ​​interest map.

6. The method according to claim 1, characterized in that The process of training the congestion detection model includes: Acquire first video data when a congestion event occurs in the target monitoring scene and second video data when no congestion event occurs; Constructing a congestion classification sample set based on the first video data and the second video data, wherein the congestion classification sample set includes an optical flow matrix corresponding to a driving road focus area map corresponding to each of congested and non-congested situations, and a segmentation result feature map of a road idle area and a road occupied area; A congestion detection model is trained based on the congestion classification sample set.

7. A lane congestion detection device, characterized in that: include: A data acquisition and processing module is used to acquire real-time video stream data in a target monitoring scene, and determine a driving road focus area map in the target monitoring scene based on the video stream data; an optical flow matrix extraction module, configured to determine an optical flow matrix corresponding to the road area of ​​interest map based on the video stream data and the road area of ​​interest map, in combination with a pre-trained optical flow extraction neural network model; a segmentation module for determining, based on the video stream data and in combination with a pre-trained semantic segmentation neural network model, a segmentation result feature map of a road idle area and a road occupied area in the target monitoring scenario; a congestion detection module, configured to determine a road congestion detection result based on the optical flow matrix and the segmentation result feature map using a pre-trained congestion detection model; The congestion detection module includes: The fusion input unit is used to fuse the optical flow matrix and the segmentation result feature map, and input the fused feature data into the pre-trained congestion detection model; The detection result output unit is used to determine whether the road is congested according to the output result of the congestion detection model.

8. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the lane congestion detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the lane congestion detection method according to any one of claims 1 to 6 is implemented.

Images