Traffic incident detection method, device, electronic device and readable storage medium

Abstract

The embodiments of the present invention provide a traffic incident detection method, device, electronic device and readable storage medium, and relate to the field of computer technology. The embodiments of the present invention obtain a road surface image when a vehicle is traveling, extract features from the road surface image, obtain vehicle detection information, road line detection information and sign detection information, and after obtaining the vehicle detection information, determine the road congestion information based on the vehicle detection information, determine the road accident information based on the road line detection information and the sign detection information, and then obtain the road traffic incident detection result based on the road congestion information and the accident information. In this way, by combining the vehicle detection information, the road line detection information and the sign information to detect traffic incidents on the road, the detection accuracy of traffic incidents can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a traffic incident detection method, device, electronic device and readable storage medium. Background Art

[0002] Traffic incident detection is one of the most important components of intelligent transportation systems, and can provide important decision-making basis for road selection, traffic diversion, etc. Vehicle accidents, breakdowns, and congestion events are common events in urban traffic. These events are highly random and difficult to predict, and they pose a safety hazard to other people driving on urban roads.

[0003] At present, the detection of traffic incidents is prone to misjudgment, and the accuracy of traffic incident detection needs to be improved. Summary of the invention

[0004] Based on this, the embodiments of the present invention provide a traffic event detection method, device, electronic device and readable storage medium, which can effectively improve the detection accuracy of traffic events.

[0005] The embodiments of the present invention can be implemented in the following ways:

[0006] In a first aspect, an embodiment of the present invention provides a traffic incident detection method, the method comprising:

[0007] Acquire road surface images when the vehicle is traveling;

[0008] Extracting features from the road surface image to obtain road surface information; wherein the road surface information includes vehicle detection information, road line detection information, and sign detection information, the vehicle detection information includes a vehicle detection frame, the road line detection information includes lane line information of the road surface, and the sign detection information includes a sign detection result and the position information of the sign;

[0009] Determine the congestion information of the road surface according to the vehicle detection information, and determine the accident information of the road surface according to the road line detection information and the sign detection information;

[0010] A traffic incident detection result of the road surface is obtained according to the congestion information and the accident information.

[0011] In an optional implementation manner, determining the accident information of the road surface according to the road line detection information and the sign detection information includes:

[0012] Determine a road surface area in the road surface image according to the lane line information of the road surface in the road line detection information;

[0013] Determining whether the sign detection result in the sign detection information includes sign information;

[0014] If the sign information is included, detecting whether the sign is located in the road area according to the position information of the sign in the sign detection information;

[0015] If it is located in the road surface area, it is determined that there is a traffic accident on the road surface.

[0016] In an optional implementation, the road surface information further includes pedestrian detection information and signal light detection information. After determining whether the sign detection result in the sign detection information includes the sign information, the method further includes:

[0017] If the sign information is not included, judging and analyzing the pedestrian detection information to determine whether there is a target pedestrian;

[0018] If the target pedestrian exists, detecting whether the road surface is congested according to the vehicle detection information and the signal light detection information, and if no congestion occurs, determining that a traffic accident exists on the road surface;

[0019] If the target pedestrian does not exist, it is determined that no traffic accident has occurred on the road surface.

[0020] In an optional implementation, the determining the road surface area in the road surface image according to the lane line information of the road surface in the road line detection information includes:

[0021] Determine two target lane lines that are farthest apart from each other according to the lane line coordinates in the road line detection information;

[0022] Determine the road surface area according to the two target lane lines;

[0023] The location information of the sign includes coordinates of a sign detection frame, and detecting whether the sign is located in the road surface area according to the location information of the sign in the sign detection information includes:

[0024] Determine whether the coordinates of the sign detection frame are between the coordinates of the two target lane lines;

[0025] If so, it is determined that the sign is located in the road surface area.

[0026] In an optional implementation manner, the road surface information further includes signal light detection information; and determining the road congestion information according to the vehicle detection information includes:

[0027] Analyze the vehicle detection frame included in the vehicle detection information to obtain a target detection frame; wherein the target detection frame is the vehicle detection frame that is closest to the center line of the road surface image and has the largest area;

[0028] Detecting whether the area ratio of the target detection frame to the road surface image is greater than a first preset threshold;

[0029] If the area ratio of the target detection frame to the road image is not greater than the first preset threshold, it is determined that the road is not congested;

[0030] If the area ratio of the target detection frame to the road surface image is greater than the first preset threshold, the speed information of the vehicle is acquired, and whether congestion occurs on the road surface is detected based on the traffic light detection information and the speed information.

[0031] In an optional implementation manner, the detecting whether congestion occurs on the road according to the signal light detection information and the speed information includes:

[0032] Detecting, according to the speed information, whether the speed of the vehicle is greater than a second preset threshold;

[0033] If the speed of the vehicle is greater than the second preset threshold, it is determined that the road is not congested;

[0034] If the speed of the vehicle is not greater than the second preset threshold, analyzing the signal light information included in the signal light detection information to detect whether the signal light information matches the target information;

[0035] If the traffic light information matches the target information, it is determined that the road is not congested;

[0036] If the traffic light information does not match the target information, it is determined that congestion occurs on the road.

[0037] In an optional implementation manner, extracting features from the road surface image to obtain road surface information includes:

[0038] Performing target detection on the road surface image according to a preset target detection model to obtain vehicle detection information, sign detection information, pedestrian detection information, and signal light detection information;

[0039] Lane line detection is performed on the road surface image according to a preset lane line detection model to obtain the road line detection information.

[0040] In an optional implementation, the performing target detection on the road surface image according to a preset target detection model to obtain the vehicle detection information, sign detection information, pedestrian detection information, and signal light detection information includes:

[0041] Inputting the road surface image into the target detection model, performing feature extraction on the road surface image through the feature extraction layer of the target detection model to obtain a feature map;

[0042] Performing feature processing on the feature map through the feature prediction layer of the target detection model, and outputting a detection frame of the target object and the coordinates of the detection frame; wherein the target object includes at least one of a vehicle, a sign, a pedestrian, and a traffic light;

[0043] The vehicle detection information, sign detection information, pedestrian detection information and signal light detection information are obtained according to the detection frame of the target object and the coordinates of the detection frame.

[0044] In an optional implementation, performing lane line detection on the road surface image according to a preset lane line detection model to obtain the road line detection information includes:

[0045] Inputting the road surface image into the lane line detection model, and segmenting the pixels belonging to the lane line and the pixels belonging to the background area through the segmentation branch of the lane line detection model;

[0046] By using the vector branch of the lane line detection model, pixels belonging to the lane line are vectorized to obtain a pixel feature vector belonging to the lane line;

[0047] Clustering the pixel feature vectors belonging to the lane line through the clustering layer of the lane line detection model to obtain a set of pixels belonging to the same lane line;

[0048] The pixel sets of different lane lines are fitted to obtain each lane line and the coordinates of each lane line to obtain road line detection information.

[0049] In a second aspect, an embodiment of the present invention provides a traffic incident detection device, comprising:

[0050] An image acquisition module, used to acquire road surface images when the vehicle is traveling;

[0051] An image analysis module, configured to extract features from the road surface image to obtain road surface information; wherein the road surface information includes vehicle detection information, road line detection information, and sign detection information, wherein the vehicle detection information includes a vehicle detection frame, the road line detection information includes lane line information of the road surface, and the sign detection information includes a sign detection result and the position information of the sign;

[0052] The event detection module is used to determine the congestion information of the road surface based on the vehicle detection information, and to determine the accident information of the road surface based on the road line detection information and the sign detection information, and to obtain the traffic event detection result of the road surface based on the congestion information and the accident information.

[0053] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the traffic event detection method described in any of the aforementioned embodiments when executing the program.

[0054] In a fourth aspect, an embodiment of the present invention provides a readable storage medium, wherein the readable storage medium includes a computer program, and when the computer program is executed, the electronic device where the readable storage medium is located is controlled to execute the traffic event detection method described in any of the aforementioned embodiments.

[0055] The traffic incident detection method, device, electronic device and readable storage medium provided by the embodiment of the present invention obtains the road surface image when the vehicle is traveling, extracts the features of the road surface image, obtains the vehicle detection information, the road line detection information and the sign detection information, after obtaining the vehicle detection information, the road line detection information and the sign detection information, determines the road congestion information according to the vehicle detection information, determines the road accident information according to the road line detection information and the sign detection information, and then obtains the road traffic incident detection result according to the road congestion information and the accident information. In this way, by combining the vehicle detection information, the road line detection information and the sign information to detect the traffic incident on the road, the detection accuracy of the traffic incident can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The technical solutions and other beneficial effects of the present invention will be made apparent by describing in detail the specific embodiments of the present invention in conjunction with the accompanying drawings.

[0057] Figure 1 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0058] Figure 2 A schematic diagram of a flow chart of a traffic incident detection method provided by an embodiment of the present invention.

[0059] Figure 3 A schematic diagram of the structure of a target detection model provided in an embodiment of the present invention.

[0060] Figure 4 A schematic diagram of the structure of a lane detection model provided in an embodiment of the present invention.

[0061] Figure 5 A lane schematic diagram provided by an embodiment of the present invention.

[0062] Figure 6 A block diagram of a traffic incident detection device provided by an embodiment of the present invention.

[0063] Icon: 100 - electronic device; 10 - traffic incident detection device; 11 - image acquisition module; 12 - image analysis module; 13 - incident detection module; 20 - memory; 30 - processor; 40 - communication unit. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0065] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0066] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or mutual communication; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0067] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0068] The disclosure below provides many different embodiments or examples to realize different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplicity and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides various specific examples of processes and materials, but those of ordinary skill in the art can be aware of the application of other processes and / or the use of other materials.

[0069] As described in the background technology, traffic incident detection is one of the most important components of intelligent transportation systems, and can provide important decision-making basis for road selection, traffic diversion and other tasks. Vehicle accidents, breakdowns and congestion incidents are common events in urban traffic. These events are highly random and difficult to predict, and they pose a safety hazard to other people driving on urban roads. At present, the detection of traffic incidents is prone to misjudgment, so the accuracy of traffic incident detection needs to be improved.

[0070] Based on the above research, an embodiment of the present invention provides a traffic event detection method, device, electronic device and readable storage medium. By analyzing the actual scene of the traffic event, after obtaining the road surface image when the vehicle is driving, the road surface image is feature extracted to obtain vehicle detection information, road line detection information, and sign detection information related to the traffic event in the road scene. After obtaining the vehicle detection information, road line detection information and sign detection information, the congestion information of the road surface is determined based on the vehicle detection information, and the accident information of the road surface is determined based on the road line detection information and the sign detection information. Then, the traffic event detection result of the road surface is obtained based on the congestion information and the accident information of the road surface. In this way, by combining the vehicle detection information, the road line detection information and the sign information to detect the traffic event on the road surface, the detection accuracy of the traffic event can be effectively improved.

[0071] See also Figure 1 , Figure 1 The block diagram is a structural block diagram of an electronic device 100 provided in this embodiment. The electronic device 100 includes a traffic event detection device 10, a memory 20, a processor 30 and a communication unit 40. The memory 20 stores machine-readable instructions executable by the processor 30. When the electronic device 100 is running, the processor 30 communicates with the memory 20 via a bus, and the processor 30 executes the machine-readable instructions and executes the traffic event detection method.

[0072] The memory 20, the processor 30 and the communication unit 40 are electrically connected to each other directly or indirectly to achieve signal transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The traffic event detection device 10 includes at least one software function module that can be stored in the memory 20 in the form of software or firmware. The processor 30 is used to execute the executable module stored in the memory 20 (for example, the software function module or computer program included in the traffic event detection device 10).

[0073] Among them, the memory 20 can be, but is not limited to, random access memory (Random Access memory, RAM), read only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0074] In some embodiments, the processor 30 is used to perform one or more functions described in the present embodiment. In some embodiments, the processor 30 may include one or more processing cores (e.g., a single-core processor (S) or a multi-core processor (S)). By way of example only, the processor 30 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC) or a microprocessor, or any combination thereof.

[0075] In this embodiment, the memory 20 is used to store programs, and the processor 30 is used to execute the programs after receiving execution instructions. The process definition method disclosed in any implementation of this embodiment can be applied to the processor 30 or implemented by the processor 30.

[0076] The communication unit 40 is used to establish a communication connection between the electronic device 100 and other devices through a network, and to send and receive data through the network.

[0077] In some embodiments, the network may be any type of wired or wireless network, or a combination thereof. By way of example only, the network may include a wired network, a wireless network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, or a near field communication (NFC) network, or any combination thereof.

[0078] In this embodiment, the electronic device 100 can be, but is not limited to, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a physical server, or other electronic devices. This embodiment does not impose any restrictions on the specific type of the electronic device.

[0079] Understandably, Figure 1 The structure shown is for illustration only. The electronic device 100 may also have Figure 1 More or fewer components than shown, or with Figure 1 Different configurations are shown. Figure 1 The components shown may be implemented in hardware, software or a combination thereof.

[0080] based on Figure 1 The present embodiment provides a traffic incident detection method. Figure 1 The electronic device shown in the figure performs the following detailed description of the steps of the traffic incident detection method provided in this embodiment. Figure 2 The traffic event detection method provided in this embodiment includes steps S101 to S104.

[0081] Step S101: Acquire a road surface image when a vehicle is traveling.

[0082] Among them, an image acquisition device can be installed on the vehicle, and the image acquisition device is used to collect the road surface image when the vehicle is driving. In this embodiment, the road surface image when the vehicle is driving is the image directly in front of the vehicle.

[0083] Optionally, the image acquisition device can acquire road images in real time when the vehicle is traveling, and then send the acquired road images to an electronic device for processing. The image acquisition device can also acquire road images according to a preset period, and then send the acquired road images to an electronic device. For example, a road image can be acquired once a minute, or once every 30 seconds. The specific setting can be based on actual needs, and this embodiment does not impose any restrictions.

[0084] Optionally, a video capture device may be installed on the vehicle to capture video of the vehicle while it is driving, and then the video is transmitted back to an electronic device, which performs frame extraction processing on the video to obtain a road surface image.

[0085] Step S102: extracting features from the road surface image to obtain road surface information.

[0086] After the road surface image is obtained, the road surface image can be analyzed to extract features from the road surface image.

[0087] In this embodiment, through the analysis of actual scenes of traffic events, it can be known that in most cases, signs (such as triangular warning signs and triangular warning stands) are placed in scenes where vehicle accidents and breakdowns occur, and in a small number of scenes, special groups such as traffic police exist. As for road traffic, vehicles, road lines, and traffic lights are fixed scenes. Therefore, after obtaining the road surface image, this embodiment performs feature extraction on the road surface image to obtain road surface information such as vehicle detection information, road line detection information, sign detection information, pedestrian detection information, and traffic light detection information.

[0088] Among them, vehicle detection information includes a vehicle detection frame, road line detection information includes lane line information on the road surface, sign detection information includes sign detection results and sign location information, pedestrian detection information includes detection results of whether the target pedestrian is detected, and signal light detection information includes detection results of whether the signal is detected and the type of signal light.

[0089] Optionally, in this embodiment, a deep learning method can be used to perform feature extraction on a road surface image based on a model obtained through sample training to obtain road surface information. Traditional image extraction methods can also be used to perform feature extraction on a road surface image to obtain road surface information. For example, scale-invariant feature transformation, directional gradient histogram, etc. can be used to perform feature extraction on a road surface image to obtain road surface information.

[0090] Step S103: determining the congestion information of the road surface according to the vehicle detection information, and determining the accident information of the road surface according to the road line detection information and the sign detection information.

[0091] After obtaining the road surface information, the vehicle detection information, road line detection information and sign detection information can be integrated to detect traffic events on the road surface.

[0092] In this embodiment, traffic events on the road may include congestion events and accident events, wherein accident information on the road may be determined based on road line detection information and signboard detection information, and congestion information on the road may be determined based on vehicle detection information.

[0093] In this embodiment, the vehicle detection frame information included in the vehicle detection information can reflect the driving conditions of the vehicle on the road. Therefore, after obtaining the vehicle detection information, the congestion information of the road can be determined based on the vehicle detection information. The road line detection information and the sign information can also reflect whether there are signs on the road. When a sign appears on the road, it means that a traffic accident has occurred on the road. Therefore, after obtaining the road line detection information and the sign detection information, the accident information of the road can be determined based on the road line detection information and the sign detection information.

[0094] Step S104: Obtaining a traffic incident detection result on the road surface according to the congestion information and the accident information.

[0095] In this embodiment, the congestion information includes the detection result of whether there is congestion information, and the accident information includes the detection result of whether an accident has occurred. When the traffic event detection result of the road surface is obtained based on the congestion information and the accident information of the road surface, if the congestion information of the road surface reflects the occurrence of congestion, and the accident information of the road surface reflects the occurrence of an accident, then the traffic event detection result of the road surface is that congestion and an accident have occurred; if the congestion information of the road surface reflects the occurrence of congestion, and the accident information of the road surface does not reflect the occurrence of an accident, then the traffic event detection result of the road surface is that congestion has occurred; if the congestion information of the road surface does not reflect the occurrence of congestion, and the accident information of the road surface reflects the occurrence of an accident, then the traffic event detection result of the road surface is that an accident has occurred.

[0096] The traffic incident detection method provided in this embodiment obtains the road surface image when the vehicle is traveling, extracts features from the road surface image, obtains vehicle detection information, road line detection information, and sign detection information related to the traffic incident in the road scene, determines the road congestion information based on the vehicle detection information, determines the road accident information based on the road line detection information and the sign detection information, and then obtains the road traffic incident detection result based on the road congestion information and the accident information. In this way, by combining the vehicle detection information, the road line detection information, and the sign information, the traffic incident on the road is detected, which can effectively improve the detection accuracy of the traffic incident.

[0097] In order to improve the detection accuracy of traffic incidents and ensure the accuracy of road surface information extraction, in this embodiment, a deep learning target detection model and a semantic segmentation model can be used to extract road surface information related to traffic incidents in road surface images.

[0098] Optionally, in this embodiment, the step of extracting features from the road surface image to obtain road surface information may include:

[0099] The road image is subjected to target detection according to the preset target detection model to obtain vehicle detection information, sign detection information, pedestrian detection information and signal light detection information.

[0100] Lane line detection is performed on the road surface image according to the preset lane line detection model to obtain road line detection information.

[0101] Among them, the preset target detection model can be obtained by training a neural network for target detection, and the preset lane line detection model can be obtained by training a neural network for semantic segmentation. In this embodiment, the training of the target detection model and the lane line detection model adopts a supervised training method, that is, an optimal model is obtained by training through sample data with known true values, and then all inputs are mapped to corresponding outputs using this model, and the output is judged to achieve the purpose of prediction or classification. For example, for the target detection model, the information such as vehicles, signs, pedestrians, and traffic lights in multiple road images can be annotated to obtain a sample set, and then the sample set is input into the neural network of target detection for training to obtain the target detection model. For another example, for the lane line detection model, the lane lines in multiple road images can be annotated to obtain a sample set, and then the sample set is input into the neural network of semantic segmentation for training to obtain the lane line detection model.

[0102] After obtaining the target detection model and the lane line detection model, the target detection model can be used to perform target detection on the road image to obtain vehicle detection information, sign detection information, pedestrian detection information, and signal light detection information. The lane line detection model can be used to perform lane line detection on the road image to obtain road line detection information.

[0103] In order to improve the accuracy of feature extraction, in this embodiment, the target detection model can be a yolov4 detection model. The yolov4 detection model retains the head part of yolov3, modifies the backbone network to CSPDarknet53, and adopts the idea of ​​spatial pyramid pooling (SPP) to expand the receptive field. PANet is used as the neck part. By deepening the network layer, the accuracy of feature information detection is improved. Its structure can be as follows Figure 3As shown in the figure, CSPDarknet53 is the backbone network and the entrance of the entire network. The input image will first pass through the CSPDarknet53 network, then through the SPP layer and the PANet layer, and finally output the result through the Yolov3 Head. The output result includes the location information of the initial prediction box and the score information of the detected target.

[0104] Based on this, in this embodiment, the step of performing target detection on the road image according to the preset target detection model to obtain vehicle detection information, sign detection information, pedestrian detection information and signal light detection information may include:

[0105] The road surface image is input into the target detection model, and features of the road surface image are extracted through the feature extraction layer of the target detection model to obtain a feature map.

[0106] The feature map is processed by the feature prediction layer of the target detection model to output the detection frame of the target object and the coordinates of the detection frame; wherein the target object includes at least one of a vehicle, a sign, a pedestrian and a traffic light.

[0107] According to the detection frame of the target object and the coordinates of the detection frame, vehicle detection information, sign detection information, pedestrian detection information and signal light detection information are obtained.

[0108] Among them, the feature extraction layer of the target detection model includes CSPDarknet53, SPP layer and PANet layer, and the feature prediction layer of the target detection model includes the Head part of Yolov. The road image is first extracted through the CSPDarknet53 network, and then further processed through the SPP layer and PANet layer to obtain the feature map of the road image. After obtaining the feature map, the feature map is processed through the Head part of Yolov3, and the detection frame of the target object predicted in the road image and the coordinates of the detection frame are output.

[0109] In this embodiment, the target object includes at least one of a vehicle, a sign, a pedestrian, and a signal light. After obtaining the detection frame of the target object and the coordinates of the detection frame, vehicle detection information, sign detection information, pedestrian detection information, and signal light detection information can be obtained according to the detection frame of the target object and the coordinates of the detection frame.

[0110] For example, if the target objects include vehicles, signboards, pedestrians and traffic lights, the vehicle detection information includes the vehicle detection frame and the coordinates of the vehicle detection frame, the signboard detection information includes the signboard detection frame and the coordinates of the signboard detection frame, the pedestrian detection information includes the pedestrian detection frame and the coordinates of the detection frame, and the traffic light detection information includes the traffic light detection frame and the coordinates of the traffic light detection frame.

[0111] For another example, the target objects include vehicles and signboards, then the vehicle detection information includes the vehicle detection frame and the coordinates of the vehicle detection frame, the signboard detection information includes the signboard detection frame and the coordinates of the signboard detection frame, the pedestrian detection information includes the detection results of no pedestrians being detected, and the signal light detection information includes the detection results of no signal lights being detected.

[0112] The traffic incident detection method provided in this embodiment performs target detection on road surface images through the yolov4 detection model, which can effectively improve the accuracy of detecting target objects.

[0113] In order to accurately segment the road lines in the road surface image, in this embodiment, the lane line detection model can use the LaneNet segmentation model, which includes a segmentation branch, a vector branch, and a clustering layer. Figure 4 As shown in the figure, the segmentation branch of the LaneNet segmentation model and the vector branch Embedding branch share the encoding layer (encoder layer), where the segmentation branch is a conventional semantic segmentation, which is used to segment the lane line and the background, and implements binary classification. The vector branch Embedding branch of the LaneNet segmentation model is used for pixel vectorization, and each pixel is represented by a high-dimensional vector to distinguish which lane line each pixel belongs to. After completing the segmentation and vectorization, the vectorized pixels are clustered to obtain pixels belonging to the same lane line. The clustering of pixels occurs in the prediction part, which is based on the Mean-Shift algorithm. The results of the two branches are clustered to obtain the segmentation result, and the pixel set of each lane line is output.

[0114] Based on this, in this embodiment, the step of performing lane line detection on the road surface image according to the preset lane line detection model to obtain road line detection information may include:

[0115] The road surface image is input into the lane detection model, and the segmentation branch of the lane detection model is used to segment the pixels belonging to the lane and the pixels belonging to the background area.

[0116] Through the vector branch of the lane line detection model, the pixels belonging to the lane line are vectorized to obtain the pixel feature vector belonging to the lane line;

[0117] The pixel feature vectors belonging to the lane line are clustered through the clustering layer of the lane line detection model to obtain a set of pixels belonging to the same lane line;

[0118] The pixel sets of different lane lines are fitted to obtain each lane line and the coordinates of each lane line to obtain road line detection information.

[0119] Among them, after obtaining the road image, the road image is input into the lane line detection model, and the segmentation branch performs binary classification on the pixels of the road image to determine whether each pixel in the road image belongs to the lane line or the background, thereby segmenting the pixels belonging to the lane line and the pixels belonging to the background area. Then, through the vector branch, the pixels belonging to the lane line are vectorized, that is, each pixel is represented by a high-dimensional vector, the image features are represented in the embedding space, and the relationship between the features is mapped in the embedding space to obtain the pixel feature vector belonging to the lane line. After obtaining the pixel feature vector belonging to the lane line, the pixel feature vector belonging to the lane line is clustered by the clustering layer using the mean shift algorithm, so that the cluster center moves along the direction of increasing density to prevent outliers from being selected into the same cluster. Then, the pixel vector is divided until all lane line pixels are assigned to the corresponding lanes, and a set of pixels belonging to the same lane line is obtained.

[0120] Since the output of LaneNet is a pixel set of each lane line, in order to obtain the lane line information, in this embodiment, after obtaining the pixel set of each lane line, it is necessary to fit the pixel set of each lane line through a curve fitting algorithm and mark the coordinates, so as to obtain each lane line and the coordinates of each lane line. After obtaining each lane line and the coordinates of each lane line, the road line detection information is obtained, that is, in this embodiment, the road line detection information includes each lane line detected and the coordinates of each lane line.

[0121] The traffic incident detection method provided in this embodiment performs lane line detection through the LaneNet segmentation model, which can effectively improve the accuracy of road line segmentation.

[0122] After obtaining road surface information such as vehicle detection information, sign detection information, pedestrian detection information, signal light detection information, and road line detection information through a preset target detection model and a preset lane line detection model, the road surface traffic incident can be detected in combination with the road surface information such as vehicle detection information, sign detection information, pedestrian detection information, signal light detection information, and road line detection information. Optionally, in this embodiment, the step of determining to obtain the road surface accident information based on the road line detection information and the sign detection information may include:

[0123] According to the lane line information of the road surface in the road line detection information, a road surface area in the road surface image is determined.

[0124] It is determined whether the sign detection result in the sign detection information includes the sign information.

[0125] If the sign information is included, whether the sign is located in the road surface area is detected according to the position information of the sign in the sign detection information.

[0126] If it is located in the road surface area, it is determined that there is a traffic accident on the road surface.

[0127] The road surface area represents the area on the road where vehicles travel. In real life, most roads where vehicles travel are provided with lane lines. Therefore, after obtaining the road line detection information, the road surface area, that is, the area on the road where vehicles travel, can be determined according to the lane line information in the road line detection information.

[0128] In view of the fact that there are multiple lane lines on the road in actual applications, in order to accurately divide the road surface area, in this embodiment, the step of determining the road surface area in the road surface image according to the lane line information of the road surface in the road line detection information may include:

[0129] According to the lane line coordinates in the lane line information, the two target lane lines that are farthest apart are determined.

[0130] According to the two target lane lines, the road surface area is determined.

[0131] Each lane line in the road line detection information has coordinates. After obtaining the road line detection information, the coordinates of each lane line in the lane line information can be analyzed to determine the two target lane lines that are farthest apart, that is, to find the two target lane lines that are segmented and located on both sides (the leftmost and the rightmost), and then the middle area divided by these two target lane lines is the road surface area. Figure 5 As shown, Figure 5 A road diagram. Figure 5 A schematic diagram of a two-lane road. Figure 5 There are three lanes in the road, namely L1, L2 and L3, where the horizontal coordinates of L1, L2 and L3 are X1, X2 and X3 respectively. By analyzing the coordinates of lanes L1, L2 and L3, it can be found that lanes L1 and L3 are the farthest apart, and lanes L1 and L3 are located on both sides of the road. Lane L1 is the leftmost lane of the road, and lane L2 is the rightmost lane of the road. After determining that lanes L1 and L3 are located on both sides, the middle area divided by lanes L1 and L3 can be the road surface area, that is, Figure 5 The a area in .

[0132] In this embodiment, through the analysis of real scenes of traffic events, scenes of vehicle accidents and breakdowns, that is, scenes of traffic accidents, in most cases there are signs (such as triangular warning signs). Therefore, after determining the road surface area, it can be determined whether the sign detection result in the sign detection information includes the sign information.

[0133] If the sign detection result includes sign information, that is, the sign detection result includes a detection frame of the sign, when the sign detection result includes sign information, it indicates that there is a sign on the road surface. Then, based on the position information of the sign in the sign detection information, detect whether the sign is located in the road surface area. If it is located in the road surface area, it is determined that there is a traffic accident on the road surface. If it is not located in the road surface area, it is determined that there is no traffic accident on the road surface.

[0134] In this embodiment, the location information of the sign includes the coordinates of the sign detection frame. According to the location information of the sign in the sign detection information, the step of detecting whether the sign is located in the road surface area may include:

[0135] Determine whether the coordinates of the sign detection frame are between the coordinates of the two target lane lines.

[0136] If it is located between the coordinates of the two target lane lines, it is determined that the sign is located in the road surface area; if it is not located between the coordinates of the two target lane lines, it is determined that the sign is not located in the road surface area.

[0137] Among them, if the object detection model detects that there is a sign in the road image, it will predict the detection frame corresponding to the sign (ie, the sign detection frame) and the coordinates of the sign detection frame.

[0138] When judging whether the sign is located in the road surface area, it can be judged whether the coordinates of the sign detection frame are located between the coordinates of the two target lane lines based on the coordinates of the sign detection frame and the coordinates of the two target lane lines. If the coordinates of the sign detection frame are located between the coordinates of the two target lane lines, it is judged that the sign is located in the road surface area and there is a traffic accident on the road surface. If the coordinates of the sign detection frame are not located between the coordinates of the two target lane lines, it is judged that the sign is not located in the road surface area and there is no traffic accident on the road surface.

[0139] It should be noted that, in this embodiment, the coordinates of each object in the road surface image belong to the same coordinate system. Optionally, an XY coordinate system can be established at any point in the road surface image. For example, an XY coordinate system can be established at the center point in the road surface image, or at the vertex in the upper left corner of the road surface image. Specifically, this embodiment does not limit this. After the coordinate system is established, the coordinates of each detection frame output by the target detection model and the coordinates of the lane line output by the lane line detection model are all located at the same coordinates. In this way, the coordinates of the sign detection frame and the two target lane lines can be compared to determine whether the coordinates of the sign detection frame are located between the coordinates of the two target lane lines.

[0140] In this embodiment, through the analysis of the real scene of traffic incidents, in most cases, there are signs (such as triangular warning signs) in scenes where vehicle accidents and breakdowns occur, that is, scenes where traffic accidents occur, while in a small number of scenes where there are no signs, there are traffic accident handling personnel, such as traffic police, medical staff, etc. Based on this, in this embodiment, if it is determined that the sign detection result in the sign detection information does not include the sign information, it is necessary to further analyze whether there is a traffic incident on the road surface based on the pedestrian detection information and the signal light detection information.

[0141] Therefore, in this embodiment, after determining that the sign detection result does not include the sign information, the traffic event detection method provided in this embodiment may further include the following steps:

[0142] Judge and analyze pedestrian detection information to determine whether there is a target pedestrian.

[0143] If there is a target pedestrian, whether the road is congested is detected based on the vehicle detection information and the traffic light detection information. If there is no congestion, it is determined that there is a traffic accident on the road.

[0144] If there is no target pedestrian, it is determined that no traffic accident has occurred on the road.

[0145] Among them, after obtaining the pedestrian detection information, the pedestrian detection information is analyzed to determine whether the pedestrian detection information includes the detection result of detecting the pedestrian. If it does not include the detection result of detecting the pedestrian, and the sign detection result does not include the sign information, it is determined that no traffic accident has occurred on the road.

[0146] If it is determined that the pedestrian detection information includes a detection result of a pedestrian, it is necessary to analyze the pedestrian included in the pedestrian detection information to determine whether it is a target pedestrian, that is, to determine whether there is a target pedestrian in the pedestrian detection information.

[0147] In this embodiment, the target pedestrian may be a traffic accident handling personnel, such as a traffic policeman. If the pedestrian detection information does not contain the target pedestrian, and the sign detection information does not include a detection result of the existence of a sign, it is determined that no traffic accident has occurred on the road.

[0148] In actual applications, when congestion occurs on the road, there will also be traffic police to clear the vehicle congestion. Therefore, in this embodiment, when it is determined that there is a target pedestrian in the pedestrian detection information, it is also necessary to detect whether congestion occurs on the road based on the vehicle detection information and the traffic light detection information. If no congestion occurs, it can be determined that there is a traffic accident on the road. If congestion occurs on the road, it means that the target pedestrian is dealing with the congestion and there is no traffic accident on the road.

[0149] In this embodiment, the target detection model may be trained so that the target detection model can directly output a detection result of whether a target pedestrian exists.

[0150] When training the target detection model, the target pedestrians in the sample set can be labeled, and the sample set with the labeled target pedestrians can be input into the target detection model. The target detection model can be trained to enable the target detection model to learn the characteristics of the target pedestrians. In this way, the trained target detection model can be used to detect the target pedestrians. When the target pedestrian exists in the road image, after the road image is input into the target detection model, the target detection model can directly output the detection frame of the target pedestrian.

[0151] The traffic incident detection method provided in this embodiment utilizes a target detection model and a lane line detection model to extract vehicle detection information, road line detection information, sign detection information, pedestrian detection information, and signal light detection information from a road surface image, and detects traffic incidents on the road surface based on the vehicle detection information, road line detection information, sign detection information, pedestrian detection information, and signal light detection information. This method can effectively detect traffic accidents occurring on the road surface, such as vehicle breakdowns and vehicle accidents, with a high accuracy rate.

[0152] In view of the fact that traffic congestion may occur in actual road traffic, in order to accurately identify whether traffic congestion occurs on the road surface, in this embodiment, whether traffic congestion occurs on the road surface can be detected based on vehicle detection information.

[0153] Optionally, the step of determining to obtain road congestion information according to vehicle detection information may include:

[0154] The vehicle detection frame included in the vehicle detection information is analyzed to obtain a target detection frame; wherein the target detection frame is a vehicle detection frame that is closest to the center line of the road surface image and has the largest area.

[0155] It is detected whether the area ratio of the target detection frame to the road surface image is greater than a first preset threshold.

[0156] If the area ratio of the target detection frame to the road surface image is not greater than the first preset threshold, it is determined that there is no congestion on the road surface.

[0157] If the area ratio of the target detection frame to the road surface image is greater than a first preset threshold, the speed information of the vehicle is obtained, and whether the road surface is congested is detected based on the traffic light detection information and the speed information.

[0158] There may be multiple vehicles in the collected road image. In order to accurately detect whether the road is congested, this embodiment selects vehicles in the same lane for evaluation, that is, the current vehicle and the vehicle directly in front of the current vehicle are selected to evaluate whether the road is congested. Therefore, in this embodiment, after collecting the road image and obtaining the vehicle detection information in the road image, the vehicle detection frame in the vehicle detection information is analyzed to find the vehicle detection frame closest to the center line of the road image and with the largest area, and the vehicle detection frame closest to the center line of the road image and with the largest area is used as the target detection frame.

[0159] Among them, for each vehicle detection frame in the vehicle detection information, if the vehicle detection frame is closest to the center line of the road image and the area of ​​the vehicle detection frame is the largest, it means that the vehicle corresponding to the vehicle detection frame is located directly in front of the current vehicle. Therefore, the vehicle detection frame is used as the target detection frame.

[0160] After obtaining the target detection frame, the area of ​​the target detection frame is compared with the area of ​​the road image to determine whether the area ratio of the target detection frame to the road image is greater than a first preset threshold. If the area ratio of the target detection frame to the road image is greater than the first preset threshold, it means that the distance between the vehicle corresponding to the target detection frame and the current vehicle is too close, and the current road may be congested and further analysis is required. If the area ratio of the target detection frame to the road image is not greater than the first preset threshold, it means that the distance between the vehicle corresponding to the target detection frame and the current vehicle is too far, and the current road is not congested.

[0161] Optionally, the first preset threshold can be set according to actual conditions, and this embodiment does not make a specific limitation. Optionally, in this embodiment, the first preset threshold can be 0.05. When the area ratio of the target detection frame to the road image is greater than 0.05, it means that the distance between the vehicle corresponding to the target detection frame and the current vehicle is too close, and the current road may be congested, requiring further analysis. When the area ratio of the target detection frame to the road image is not greater than 0.05, it means that the distance between the vehicle corresponding to the target detection frame and the current vehicle is too far, and the current road is not congested.

[0162] In this embodiment, when the area ratio of the target detection frame to the road surface image is greater than a first preset threshold, the speed information of the vehicle can be obtained, and whether the road surface is congested can be further detected based on the traffic light detection information and the speed information.

[0163] In this embodiment, the step of detecting whether a road is congested according to the signal light detection information and the speed information may include:

[0164] According to the speed information, it is detected whether the speed of the vehicle is greater than a second preset threshold.

[0165] If the speed of the vehicle is greater than the second preset threshold, it is determined that there is no road congestion.

[0166] If the speed of the vehicle is not greater than the second preset threshold, the signal light information included in the signal light detection information is analyzed to detect whether the signal light information matches the target information.

[0167] If the traffic light information matches the target information, it is determined that there is no traffic congestion on the road.

[0168] If the traffic light information does not match the target information, it is determined that the road is congested.

[0169] When the area ratio of the target detection frame to the road surface image is greater than a first preset threshold, the speed information of the vehicle is obtained, and whether congestion occurs on the road surface is detected based on the speed information of the vehicle.

[0170] In this embodiment, the speed information of the vehicle can be uploaded to the electronic device along with the road surface image. The vehicle is equipped with a sensor for collecting the vehicle speed, and after the vehicle speed information is acquired, it can be uploaded to the electronic device along with the road surface image.

[0171] After obtaining the vehicle speed information, it can be determined whether the vehicle speed is greater than the second preset threshold. If the vehicle speed is greater than the second preset threshold, it means that the vehicle speed is normal and the vehicle is in a normal driving process, and there is no congestion on the road. If the vehicle speed is not greater than the second preset threshold, it means that the vehicle speed is too low and the vehicle is in a low-speed driving process, and there may be congestion on the road.

[0172] In this embodiment, the second preset threshold value can be set according to actual conditions, and this embodiment does not make specific limitations. Optionally, in this embodiment, the second preset threshold value can be 5km / h. When the speed of the vehicle is greater than 5km / h, it means that the speed of the vehicle is normal, and it is in a normal driving process, and there is no congestion on the road. If the speed of the vehicle is not greater than 5km / h, it means that the speed of the vehicle is too low, and it is in a low-speed driving process, and there may be congestion on the road.

[0173] When the speed of the vehicle is not greater than the second preset threshold, it is also possible that the vehicle temporarily stops due to changes in the traffic light. Therefore, when the speed of the vehicle is not greater than the second preset threshold, the signal light information included in the signal light detection information is analyzed to determine whether the signal light information matches the target information.

[0174] In this embodiment, the signal light may be detection information of a traffic light, and the target information may be red light information. After analyzing the signal light information included in the signal light detection information, if the signal light information matches the target information, that is, the signal light information is red light information, it means that the current vehicle is waiting for a red light, and the speed of the current vehicle is not greater than the second preset threshold, and it is further determined that there is no congestion on the current road. If the signal light information does not match the target information, that is, the signal light information is not red light information, it means that the current vehicle is not waiting for a red light, and the speed of the current vehicle is not greater than the second preset threshold, which is not caused by waiting for the signal light, and it can be determined that there is congestion on the current road.

[0175] It should be noted that after analyzing the signal light detection information, if the signal light detection information includes a detection result that no signal light is detected, it means that the vehicle speed is not greater than the second preset threshold and is not caused by waiting for the signal light, and it can be determined that congestion occurs on the current road.

[0176] The traffic incident detection method provided in this embodiment can effectively detect congestion on the road by comprehensively judging whether the current road surface is congested through vehicle detection information, traffic light detection information and speed information, with a high accuracy rate.

[0177] In an optional implementation, after a traffic accident is detected on the road and congestion is detected on the road, the detection results of the traffic accident and congestion can be reported so that the subsequent traffic department can perform traffic control based on the detection results to reduce safety hazards.

[0178] The traffic incident detection method provided in this embodiment, through the analysis of the actual scene of the traffic incident, after obtaining the road surface image when the vehicle is driving, the road surface image is feature extracted to obtain the vehicle detection information, road line detection information, sign detection information, pedestrian detection information and signal light detection information related to the traffic incident in the road scene, and the traffic incident on the road is detected by combining the vehicle detection information, road line detection information, sign detection information, pedestrian detection information and signal light detection information, which can effectively improve the detection accuracy of the traffic incident. Among them, by using the real traffic incidents screened by humans for testing, the recall rate of the fault for traffic accidents, that is, vehicle accidents, is 80.65%, and the recall rate for congestion scenes reaches 98%.

[0179] The traffic event detection method provided in this embodiment can effectively identify the events and location information being sent on the current road through real-time feedback data with high accuracy.

[0180] Based on the same inventive concept, please refer to Figure 6 This embodiment provides a traffic incident detection device 10, which is applied to Figure 1 The electronic equipment shown, such as Figure 6 As shown, the traffic event detection device 10 provided in this embodiment includes an image acquisition module 11 , an image analysis module 12 and an event detection module 13 .

[0181] The image acquisition module 11 is used to acquire the road surface image when the vehicle is traveling.

[0182] The image analysis module 12 is used to extract features from the road surface image to obtain road surface information; wherein the road surface information includes vehicle detection information, road line detection information and sign detection information, the vehicle detection information includes a vehicle detection frame, the road line detection information includes lane line information of the road surface, and the sign detection information includes sign detection results and sign location information.

[0183] The event detection module 13 is used to determine the congestion information of the road according to the vehicle detection information, determine the accident information of the road according to the road line detection information and the sign detection information; and obtain the traffic event detection result of the road according to the congestion information and the accident information.

[0184] In an optional implementation, the event detection module 13 is used to:

[0185] According to the lane line information of the road surface in the road line detection information, a road surface area in the road surface image is determined.

[0186] It is determined whether the sign detection result in the sign detection information includes the sign information.

[0187] If there is a signboard, it is detected whether the signboard is located in the road surface area according to the position information of the signboard in the signboard detection information.

[0188] If it is located in the road surface area, it is determined that there is a traffic accident on the road surface.

[0189] In an optional implementation manner, the event detection module 13 is further configured to:

[0190] If the sign information is not included, the pedestrian detection information is judged and analyzed to determine whether the target pedestrian exists.

[0191] If there is a target pedestrian, whether the road is congested is detected based on the vehicle detection information and the traffic light detection information. If there is no congestion, it is determined that there is a traffic accident on the road.

[0192] If there is no target pedestrian, it is determined that no traffic accident has occurred on the road.

[0193] In an optional implementation, the event detection module 13 is used to:

[0194] According to the coordinates of the lane lines in the lane line information, the two target lane lines that are farthest apart are determined.

[0195] According to the two target lane lines, the road surface area is determined.

[0196] The location information of the sign includes the coordinates of the sign detection frame. The event detection module 13 detects whether the sign is located in the road area according to the location information of the sign in the sign detection information, including:

[0197] Determine whether the coordinates of the sign detection frame are between the coordinates of the two target lane lines.

[0198] If so, it is determined that the sign is located in the road surface area.

[0199] In an optional implementation, the event detection module 13 is used to:

[0200] The vehicle detection frame included in the vehicle detection information is analyzed to obtain a target detection frame; wherein the target detection frame is a vehicle detection frame that is closest to the center line of the road surface image and has the largest area.

[0201] It is detected whether the area ratio of the target detection frame to the road surface image is greater than a first preset threshold.

[0202] If the area ratio of the target detection frame to the road surface image is not greater than a first preset threshold, it is determined that the road surface is not congested.

[0203] If the area ratio of the target detection frame to the road surface image is greater than a first preset threshold, the speed information of the vehicle is obtained, and whether the road surface is congested is detected based on the traffic light detection information and the speed information.

[0204] In an optional implementation, the event detection module 13 is used to:

[0205] According to the speed information, it is detected whether the speed of the vehicle is greater than a second preset threshold.

[0206] If the speed of the vehicle is greater than the second preset threshold, it is determined that there is no road congestion.

[0207] If the speed of the vehicle is not greater than the second preset threshold, the signal light information included in the signal light detection information is analyzed to detect whether the signal light information matches the target information.

[0208] If the traffic light information matches the target information, it is determined that there is no traffic congestion on the road.

[0209] If the traffic light information does not match the target information, it is determined that the road is congested.

[0210] In an optional embodiment, the image analysis module 12 is used to:

[0211] The road image is subjected to target detection according to the preset target detection model to obtain vehicle detection information, sign detection information, pedestrian detection information and signal light detection information.

[0212] Lane line detection is performed on the road surface image according to the preset lane line detection model to obtain road line detection information.

[0213] In an optional embodiment, the image analysis module 12 is used to:

[0214] The road surface image is input into the target detection model, and features of the road surface image are extracted through the feature extraction layer of the target detection model to obtain a feature map.

[0215] The feature map is processed by the feature prediction layer of the target detection model to output the detection frame of the target object and the coordinates of the detection frame; wherein the target object includes at least one of a vehicle, a sign, a pedestrian and a traffic light.

[0216] According to the detection frame of the target object and the coordinates of the detection frame, vehicle detection information, sign detection information, pedestrian detection information and signal light detection information are obtained.

[0217] In an optional embodiment, the image analysis module 12 is used to:

[0218] The road surface image is input into the lane detection model, and the segmentation branch of the lane detection model is used to segment the pixels belonging to the lane and the pixels belonging to the background area.

[0219] Through the vector branch of the lane line detection model, the pixels belonging to the lane line are vectorized to obtain the pixel feature vector belonging to the lane line.

[0220] The pixel feature vectors belonging to the lane line are clustered through the clustering layer of the lane line detection model to obtain a set of pixels belonging to the same lane line.

[0221] The pixel sets of different lane lines are fitted to obtain each lane line and the coordinates of each lane line to obtain road line detection information.

[0222] The traffic event detection device provided in this embodiment, through analysis of the actual scene of the traffic event, obtains the road surface image when the vehicle is traveling, and then extracts features from the road surface image to obtain vehicle detection information, road line detection information, sign detection information, pedestrian detection information and signal light detection information related to the traffic event in the road scene. By combining the vehicle detection information, road line detection information, sign detection information, pedestrian detection information and signal light detection information to detect traffic events on the road, the detection accuracy of traffic events can be effectively improved.

[0223] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the traffic event detection device 10 described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0224] Based on the above, this embodiment further provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the traffic event detection method described in any of the aforementioned embodiments is implemented.

[0225] The readable storage medium may be, but is not limited to, a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0226] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the readable storage medium described above can refer to the corresponding process in the aforementioned method, and will not be elaborated here.

[0227] In summary, the traffic event detection method, device, electronic device and readable storage medium provided by the embodiments of the present invention, through analysis of the actual scene of the traffic event, after acquiring the road surface image when the vehicle is traveling, perform feature extraction on the road surface image, and obtain vehicle detection information, road line detection information, sign detection information, pedestrian detection information and signal light detection information related to the traffic event in the road scene, and combine the vehicle detection information, road line detection information, sign detection information, pedestrian detection information and signal light detection information to detect traffic events such as road congestion and accidents, which can effectively improve the detection accuracy of traffic events.

[0228] In several implementations provided in the embodiments of the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device and method implementations described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple implementations of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0229] In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0230] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention is essentially or partly contributed to the prior art or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. It should be noted that, in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0231] The above are only various embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A traffic incident detection method, characterized in that: The method comprises: Acquire road surface images when the vehicle is traveling; Extracting features from the road surface image to obtain road surface information; wherein the road surface information includes vehicle detection information, road line detection information and sign detection information, the vehicle detection information includes a vehicle detection frame, the road line detection information includes lane line information of the road surface, the sign detection information includes a sign detection result and the position information of the sign; the road surface information also includes pedestrian detection information and signal light detection information; Determine the congestion information of the road surface according to the vehicle detection information, and determine the accident information of the road surface according to the road line detection information and the sign detection information; Obtaining a traffic incident detection result on the road surface according to the congestion information and the accident information; The determining, based on the road line detection information and the sign detection information, to obtain the accident information on the road surface includes: Determine a road surface area in the road surface image according to the lane line information of the road surface in the road line detection information; Determining whether the sign detection result in the sign detection information includes sign information; If the sign information is included, detecting whether the sign is located in the road area according to the position information of the sign in the sign detection information; If it is located in the road surface area, it is determined that there is a traffic accident on the road surface; If the sign information is not included, judging and analyzing the pedestrian detection information to determine whether there is a target pedestrian; If the target pedestrian exists, detecting whether the road surface is congested according to the vehicle detection information and the signal light detection information, and if no congestion occurs, determining that a traffic accident exists on the road surface; If the target pedestrian does not exist, it is determined that no traffic accident has occurred on the road surface.

2. The traffic incident detection method according to claim 1, characterized in that: The determining of a road surface area in the road surface image according to the road lane line information in the road line detection information includes: Determine two target lane lines that are farthest apart from each other according to the lane line coordinates in the lane line information; Determine the road surface area according to the two target lane lines; The location information of the sign includes coordinates of a sign detection frame, and detecting whether the sign is located in the road surface area according to the location information of the sign in the sign detection information includes: Determine whether the coordinates of the sign detection frame are between the coordinates of the two target lane lines; If so, it is determined that the sign is located in the road surface area.

3. The traffic incident detection method according to any one of claims 1 to 2, characterized in that: The road surface information also includes signal light detection information; and determining the road congestion information according to the vehicle detection information includes: Analyze the vehicle detection frame included in the vehicle detection information to obtain a target detection frame; wherein the target detection frame is the vehicle detection frame that is closest to the center line of the road surface image and has the largest area; Detecting whether the area ratio of the target detection frame to the road surface image is greater than a first preset threshold; If the area ratio of the target detection frame to the road image is not greater than the first preset threshold, it is determined that the road is not congested; If the area ratio of the target detection frame to the road surface image is greater than the first preset threshold, the speed information of the vehicle is acquired, and whether congestion occurs on the road surface is detected based on the traffic light detection information and the speed information.

4. The traffic incident detection method according to claim 3, characterized in that: The detecting whether the road surface is congested according to the signal light detection information and the speed information includes: Detecting, according to the speed information, whether the speed of the vehicle is greater than a second preset threshold; If the speed of the vehicle is greater than the second preset threshold, it is determined that the road is not congested; If the speed of the vehicle is not greater than the second preset threshold, analyzing the signal light information included in the signal light detection information to detect whether the signal light information matches the target information; If the traffic light information matches the target information, it is determined that the road is not congested; If the traffic light information does not match the target information, it is determined that congestion occurs on the road.

5. The traffic incident detection method according to claim 1, characterized in that: The step of extracting features from the road surface image to obtain road surface information includes: Performing target detection on the road surface image according to a preset target detection model to obtain vehicle detection information, sign detection information, pedestrian detection information, and signal light detection information; Lane line detection is performed on the road surface image according to a preset lane line detection model to obtain the road line detection information.

6. The traffic incident detection method according to claim 5, characterized in that: The performing target detection on the road surface image according to a preset target detection model to obtain the vehicle detection information, sign detection information, pedestrian detection information and signal light detection information includes: Inputting the road surface image into the target detection model, performing feature extraction on the road surface image through the feature extraction layer of the target detection model to obtain a feature map; Performing feature processing on the feature map through the feature prediction layer of the target detection model, and outputting a detection frame of the target object and the coordinates of the detection frame; wherein the target object includes at least one of a vehicle, a sign, a pedestrian, and a traffic light; The vehicle detection information, sign detection information, pedestrian detection information and signal light detection information are obtained according to the detection frame of the target object and the coordinates of the detection frame.

7. The traffic incident detection method according to claim 5, characterized in that: The performing lane line detection on the road surface image according to a preset lane line detection model to obtain the road line detection information includes: Inputting the road surface image into the lane line detection model, and segmenting the pixels belonging to the lane line and the pixels belonging to the background area through the segmentation branch of the lane line detection model; By using the vector branch of the lane line detection model, pixels belonging to the lane line are vectorized to obtain a pixel feature vector belonging to the lane line; Clustering the pixel feature vectors belonging to the lane line through the clustering layer of the lane line detection model to obtain a set of pixels belonging to the same lane line; The pixel sets of different lane lines are fitted to obtain each lane line and the coordinates of each lane line to obtain road line detection information.

8. A traffic incident detection device, characterized in that: include: An image acquisition module, used to acquire road surface images when the vehicle is traveling; An image analysis module, configured to extract features from the road surface image to obtain road surface information; wherein the road surface information includes vehicle detection information, road line detection information, and sign detection information, wherein the vehicle detection information includes a vehicle detection frame, the road line detection information includes lane line information of the road surface, and the sign detection information includes a sign detection result and the position information of the sign; the road surface information also includes pedestrian detection information and signal light detection information; the road surface information also includes pedestrian detection information and signal light detection information; An event detection module, used to determine the congestion information of the road surface according to the vehicle detection information, determine the accident information of the road surface according to the road line detection information and the sign detection information, and obtain the traffic event detection result of the road surface according to the congestion information and the accident information; The event detection module is used to: The determining, based on the road line detection information and the sign detection information, to obtain the accident information on the road surface includes: Determine a road surface area in the road surface image according to the lane line information of the road surface in the road line detection information; Determining whether the sign detection result in the sign detection information includes sign information; If the sign information is included, detecting whether the sign is located in the road area according to the position information of the sign in the sign detection information; If it is located in the road surface area, it is determined that there is a traffic accident on the road surface; If the sign information is not included, judging and analyzing the pedestrian detection information to determine whether there is a target pedestrian; If the target pedestrian exists, detecting whether the road surface is congested according to the vehicle detection information and the signal light detection information, and if no congestion occurs, determining that a traffic accident exists on the road surface; If the target pedestrian does not exist, it is determined that no traffic accident has occurred on the road surface.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the traffic event detection method according to any one of claims 1 to 7 when executing the program.

10. A readable storage medium, characterized in that: The readable storage medium includes a computer program, and when the computer program is executed, the electronic device where the readable storage medium is located is controlled to execute the traffic event detection method according to any one of claims 1 to 7.

Images