A traffic event detection method, device, equipment and storage medium

Abstract

The application discloses a traffic event detection method, device and equipment and a storage medium. A ball camera is controlled to sequentially acquire video frames of each local field divided by a global field of view of the ball camera according to preset polling parameters. The video frames of all local fields acquired by the ball camera are rearranged to obtain a video sequence set of the global field of view. Multi-dimensional convolution operation is performed on each video sequence unit in the video sequence set to obtain input features of the global field of view at each moment. The obtained input features are input into a convolutional neural network to extract visual features at each moment. The obtained visual features are input into a recurrent neural network in time sequence for analysis to obtain occurrence probabilities of each event at each moment. Only one variable-focus rotating ball camera is deployed, the global field of view to be monitored is divided into a plurality of local fields, the hardware cost and the deployment and maintenance cost are reduced, the ball camera has the characteristics of light weight and low cost, and the real-time traffic event fine and high-precision discrimination and management capability is improved.

Description

Technical Field

[0001] This invention relates to the field of traffic incident detection technology, specifically to a traffic incident detection method, apparatus, equipment, and storage medium. Background Technology

[0002] Traffic incidents refer to illegal or abnormal events that occur in real time on roads and affect traffic flow. They mainly include three types: First, events related to various traffic participants, such as vehicles driving in the wrong direction, illegal parking, suspected accidents, non-motorized vehicles entering the road unauthorizedly, and pedestrians entering the road unauthorizedly; second, man-made or natural events affecting road traffic, such as spilled debris, fog, and fireworks; and third, the state of traffic itself, such as slow-moving traffic or congestion. Traffic incident detection utilizes computing resources to analyze and detect video footage collected by surveillance cameras deployed on roads in real time, issuing warnings when a traffic incident is detected.

[0003] Current technologies typically employ a combined bullet and PTZ camera approach for traffic incident detection. The bullet camera provides a panoramic view, and upon detecting a traffic incident, it drives the PTZ camera to focus on the target for tracking and analysis. This approach requires two video acquisition devices, leading to high hardware costs. Furthermore, the dual-camera setup necessitates calibration and adjustment of both the bullet and PTZ cameras during deployment, which can introduce accumulated errors and potentially cause focusing failure, resulting in the inability to extract detailed information. Additionally, the high imaging resolution results in high image transmission bandwidth and computation time. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes a traffic incident detection method, apparatus, equipment, and storage medium, which features lightweight design and low cost, thereby improving the ability to accurately and precisely identify and manage real-time traffic incidents.

[0005] This invention provides a traffic incident detection method, the method comprising:

[0006] The PTZ camera is controlled to sequentially acquire video frames of each local field of view divided by the global field of view of the PTZ camera according to preset polling parameters;

[0007] The video frames of all local fields of view acquired by the PTZ camera are rearranged to obtain the video sequence set of the global field of view;

[0008] Multidimensional convolution operation is performed on each video sequence unit in the video sequence set to obtain the input features of the global field of view at each time step;

[0009] The obtained input features are fed into a convolutional neural network to extract visual features at each time step;

[0010] The obtained visual features are input into a recurrent neural network in chronological order for analysis to obtain the probability of each event occurring at each time step.

[0011] Preferably, the polling parameter determination process specifically includes:

[0012] The global field of view to be monitored is divided into several local fields of view according to preset rules;

[0013] The field of view parameters for each local field of view are determined based on the viewing angle of each local field of view and the installation position of the PTZ camera. The field of view parameters include the field of view angle and the field of view focal length.

[0014] The polling parameters of the PTZ camera are determined based on the preset polling order, preset transformation period, and field of view parameters of each local field of view for all local fields of view.

[0015] As a preferred embodiment, the step of rearranging all the video frames of the local field of view acquired by the PTZ camera to obtain the video sequence set of the global field of view specifically includes:

[0016] Obtain each local view R j The video frames are arranged sequentially according to a fixed period.

[0017] The global view video sequence is obtained by rearranging the video frames acquired from all local views.

[0018]

[0019]

[0020] ...

[0021] for:

[0022] At the same time, one video frame is taken from each of the local fields of view and rearranged to obtain a video sequence unit covering the global field of view.

[0023] For all acquired video sequence units Arranged in chronological order, the video sequence set W of the global field of view is obtained;

[0024] in, k represents the kth complete cycle of the current video frame, k = 0, 1, ..., S, where S represents the number of complete cycles; i represents the i-th frame of the current complete cycle of the current video frame, i = 0, 1, ..., N, where N represents the number of image frames in each complete cycle; j represents the j-th local field of view of the current video frame, j = 0, 1, ..., M, where M represents the number of local fields of view, and S, M, N > 0.

[0025] Preferably, the step of performing multidimensional convolution operations on each video sequence unit in the video sequence set to obtain the input features of the global field of view at each time step specifically includes:

[0026] Obtain each video sequence unit Included video frames

[0027] For all video frames Perform multidimensional convolution operations to obtain the input features of the global view at each time step.

[0028] in, k represents the k-th complete period of the current video frame, k = 0, 1, ..., S, where S represents the number of complete periods; i represents the i-th frame of the current complete period, i = 0, 1, ..., N, where N represents the number of image frames in each complete period; j represents the j-th local field of view of the current video frame, j = 0, 1, ..., M, where M represents the number of local fields of view, S, M, N > 0; d represents the time d, at which time i and j are known and correspond to the current time d; q represents the q-th channel video frame of the current video frame, q = 0, 1, ..., c, where c represents the number of channels in each video frame; conv represents the convolution operation. This indicates that a concatenation operation is performed on x.

[0029] Preferably, the step of inputting the obtained input features into a convolutional neural network to extract visual features at each time step specifically includes:

[0030] The input features I at each time step are extracted using a convolutional neural network. d Visual features F d ;

[0031] Among them, F d =Net(I d ), where Net is a deep learning network and d is the d-th time step.

[0032] Preferably, the step of inputting the obtained visual features into a recurrent neural network in chronological order for analysis to obtain the probability of occurrence of each event at each time step specifically includes:

[0033] A long short-term memory recurrent neural network is used to extract temporal information from visual features, and the probability of various traffic events occurring at the current moment is analyzed [Y]. 0,d ,…,Y p,d ];

[0034] Among them, [Y 0,d ,…,Y p,d ] = LSTM(F d-1 ,F d), where d represents time d, Y 0,d Let F represent the probability of the x-th traffic event occurring at time d, where x = 0, 1, ..., p, p > 0. d F represents the visual features at time d. d-1 This represents the visual features at time d-1.

[0035] As a preferred embodiment, the method further includes;

[0036] Determine the probability of each event occurring at each moment;

[0037] When an event occurs with a probability not less than a preset threshold, the time and probability of the event are recorded, and an alarm message is generated.

[0038] This invention also provides a traffic incident detection device, the device comprising:

[0039] The image acquisition module is used to control the PTZ camera to sequentially acquire video frames of each local field of view divided by the global field of view of the PTZ camera according to preset polling parameters;

[0040] The video sequence set acquisition module is used to rearrange all the video frames of the local field of view acquired by the PTZ camera to obtain the video sequence set of the global field of view;

[0041] The input feature calculation module is used to perform multidimensional convolution operations on each video sequence unit in the video sequence set to obtain the input features of the global field of view at each time step.

[0042] The visual feature calculation module is used to input the obtained input features into the convolutional neural network to extract the visual features at each time step.

[0043] The probability calculation module is used to input the obtained visual features into the recurrent neural network in chronological order for analysis, and to obtain the probability of occurrence of each event at each time step.

[0044] Preferably, the process by which the image acquisition module determines the polling parameters specifically includes:

[0045] The global field of view to be monitored is divided into several local fields of view according to preset rules;

[0046] The field of view parameters for each local field of view are determined based on the viewing angle of each local field of view and the installation position of the PTZ camera. The field of view parameters include the field of view angle and the field of view focal length.

[0047] The polling parameters of the PTZ camera are determined based on the preset polling order, preset transformation period, and field of view parameters of each local field of view for all local fields of view.

[0048] As a preferred embodiment, the video sequence set acquisition module is specifically used for:

[0049] Obtain each local view R j The video frames are arranged sequentially according to a fixed period.

[0050] The global view video sequence is obtained by rearranging the video frames acquired from all local views.

[0051]

[0052]

[0053] ...

[0054] for:

[0055] At the same time, one video frame is taken from each of the local fields of view and rearranged to obtain a video sequence unit covering the global field of view.

[0056] For all acquired video sequence units Arranged in chronological order, the video sequence set W of the global field of view is obtained;

[0057] in, k represents the kth complete cycle of the current video frame, k = 0, 1, ..., S, where S represents the number of complete cycles; i represents the i-th frame of the current complete cycle of the current video frame, i = 0, 1, ..., N, where N represents the number of image frames in each complete cycle; j represents the j-th local field of view of the current video frame, j = 0, 1, ..., M, where M represents the number of local fields of view, and S, M, N > 0.

[0058] Preferably, the input feature calculation module is specifically used for:

[0059] Obtain each video sequence unit Included video frames

[0060] For all video frames Perform multidimensional convolution operations to obtain the input features of the global view at each time step.

[0061] in, k represents the k-th complete period of the current video frame, k = 0, 1, ..., S, where S represents the number of complete periods; i represents the i-th frame of the current complete period, i = 0, 1, ..., N, where N represents the number of image frames in each complete period; j represents the j-th local field of view of the current video frame, j = 0, 1, ..., M, where M represents the number of local fields of view, S, M, N > 0; d represents the time d, at which time i and j are known and correspond to the current time d; q represents the q-th channel video frame of the current video frame, q = 0, 1, ..., c, where c represents the number of channels in each video frame; conv represents the convolution operation. This indicates that a concatenation operation is performed on x.

[0062] Preferably, the visual feature calculation module is specifically used for:

[0063] The input features I at each time step are extracted using a convolutional neural network. d Visual features F d ;

[0064] Among them, F d =Net(I d ), where Net is a deep learning network and d is the d-th time step.

[0065] Preferably, the probability calculation module is specifically used for:

[0066] A long short-term memory recurrent neural network is used to extract temporal information from visual features, and the probability of various traffic events occurring at the current moment is analyzed [Y]. 0,d ,…,Y p,d ];

[0067] Among them, [Y 0,d ,…,Y p,d ] = LSTM(F d-1 ,F d ), where d represents time d, Y 0,d Let F represent the probability of the x-th traffic event occurring at time d, where x = 0, 1, ..., p, p > 0. d F represents the visual features at time d. d-1 This represents the visual features at time d-1.

[0068] As a preferred embodiment, the device further includes an early warning module;

[0069] The early warning module is specifically used for:

[0070] Determine the probability of each event occurring at each moment;

[0071] When an event occurs with a probability not less than a preset threshold, the time and probability of the event are recorded, and an alarm message is generated.

[0072] This invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a traffic event detection method as described in any of the above embodiments.

[0073] This invention also provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform a traffic incident detection method as described in any of the above embodiments.

[0074] This invention provides a traffic incident detection method, apparatus, device, and storage medium. It involves controlling a PTZ camera to sequentially acquire video frames from each local field of view (PDV), divided by the PTZ camera's global field of view, according to preset polling parameters. The acquired PPV camera's PPV PDR video frames from all local PPV PDR video frames are rearranged to obtain a video sequence set of the global field of view. Multidimensional convolution operations are performed on each video sequence unit in the video sequence set to obtain the input features of the global field of view at each moment. The obtained input features are then input into a convolutional neural network to extract visual features at each moment. Finally, the obtained visual features are input into a recurrent neural network in chronological order for analysis to obtain the probability of occurrence of each event at each moment. This method deploys only a single zoomable PTZ camera, dividing the global field of view into several local PPV PDRs, reducing hardware and deployment / maintenance costs. It features lightweight design and low cost, improving the ability to accurately and precisely identify and manage real-time traffic incidents. Attached Figure Description

[0075] Figure 1 This is a schematic flowchart of a traffic incident detection method provided in an embodiment of the present invention;

[0076] Figure 2 This is a schematic diagram of a global field of view partitioning provided in an embodiment of the present invention;

[0077] Figure 3 This is a schematic diagram of the structure of a video sequence from a global perspective provided in an embodiment of the present invention;

[0078] Figure 4 This is a schematic flowchart of a traffic incident detection method provided in another embodiment of the present invention;

[0079] Figure 5 This is a schematic diagram of the structure of a traffic incident detection device provided in an embodiment of the present invention;

[0080] Figure 6 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present invention. Detailed Implementation

[0081] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0082] See Figure 1 This is a flowchart illustrating a traffic incident detection method provided in an embodiment of the present invention, the method comprising steps S1 to S5;

[0083] S1, control the PTZ camera to sequentially acquire video frames of each local field of view divided by the global field of view of the PTZ camera according to preset polling parameters;

[0084] S2, rearrange all the video frames of the local field of view acquired by the PTZ camera to obtain the video sequence set of the global field of view;

[0085] S3, perform multidimensional convolution operation on each video sequence unit in the video sequence set to obtain the input features of the global field of view at each time step;

[0086] S4, the obtained input features are fed into the convolutional neural network to extract the visual features at each time step;

[0087] S5. The obtained visual features are input into the recurrent neural network in chronological order for analysis to obtain the probability of each event at each time step.

[0088] In this specific implementation, a PTZ camera is installed at a high point in the area to be monitored, and video frames for detection are acquired through the PTZ camera.

[0089] The field of view that needs to be monitored is generally large. When the PTZ camera directly monitors the entire field of view, the image quality is poor, which is not effective for subsequent time detection.

[0090] In this embodiment, the PTZ camera is controlled to sequentially acquire video frames of each local field of view in the global field of view according to the set polling parameters. The obtained periodically transformed local field of view video sequences are used to detect traffic events such as congestion and accidents.

[0091] The video frames of each local field of view obtained through polling parameters are rearranged. By arranging the video frame data of each local field of view, a video sequence unit containing the overall field of view can be obtained. Then, by following the time sequence, a video sequence set of the global field of view is obtained by stitching and transforming the local fields of view. The video sequence set includes the global field of view image obtained by stitching video frames from different local fields of view at different times. By acquiring the images of the local fields of view in sequence, the image clarity is ensured and the amount of data transmitted at the same time is reduced.

[0092] Multidimensional convolution operations are performed on each video sequence unit in the global field of view of the acquired video sequence units. The input features of the global field of view at each time step are extracted through multidimensional convolution and connection operations.

[0093] The obtained input features are used as input to the convolutional neural network, and the visual features corresponding to the input features at each time step are extracted by the CNN.

[0094] The obtained visual features are input into a recurrent neural network in chronological order for analysis, and different events, such as normal traffic events, congestion, and accidents, are predicted to obtain the probability of occurrence of different events.

[0095] By deploying only one zoomable PTZ camera, the global field of view to be controlled is divided into several local fields of view, which greatly reduces the resources required and significantly lowers hardware costs and deployment and maintenance costs.

[0096] In another embodiment provided by the present invention, the polling parameter determination process specifically includes:

[0097] The global field of view to be monitored is divided into several local fields of view according to preset rules;

[0098] The field of view parameters for each local field of view are determined based on the viewing angle of each local field of view and the installation position of the PTZ camera. The field of view parameters include the field of view angle and the field of view focal length.

[0099] The polling parameters of the PTZ camera are determined based on the preset polling order, preset transformation period, and field of view parameters of each local field of view for all local fields of view.

[0100] In the specific implementation of this embodiment, please refer to Figure 2 This is a schematic diagram of a global field of view partitioning provided in an embodiment of the present invention;

[0101] The maximum field of view that the installed PTZ camera can monitor is defined as the global field of view. The global field of view to be monitored is divided into multiple local fields of view. The global field of view can be divided into 9 local fields of view of the same size in a 3*3 pattern.

[0102] The field of view parameters for each local field of view are determined based on the position of each local field of view within the overall field of view to be monitored and the installation position of the PTZ camera. These field of view parameters are used to control the PTZ camera to adjust the overall field of view to match the current local field of view. The field of view parameters include the field of view angle and the field of view focal length.

[0103] The polling parameters of the PTZ camera are determined based on the preset polling order, preset transformation period, and field of view parameters of each local field of view. The polling order can be determined as rotating the positions of the 3*3 local fields of view in a sequential order from left to right and from top to bottom. The set transformation period can be specifically set as 1 second for each local field of view. That is, the video frame of the first local field of view is acquired in the first second, the video frame of the second local field of view is acquired in the second second, and so on. After acquiring the video frame of the ninth local field of view, the video frame of the first local field of view is acquired in a loop.

[0104] The PTZ camera is controlled to automatically operate and transform according to the determined polling parameters, acquiring video frames for each local field of view.

[0105] It should be noted that in this embodiment, the method for determining the polling rule is illustrated by taking a preset rule of 3*3, a polling order of left to right and top to bottom, and a transformation period of 1 second as an example. In other embodiments, the segmentation rule, polling order, and transformation period can all be set to other parameters, which can increase the number of segments in the local field of view and shorten the transformation period, thereby increasing the amount of information obtained from the frame data and improving the prediction accuracy of the final traffic event; however, this will also increase the operating burden.

[0106] In another embodiment provided by the present invention, step S2 specifically includes:

[0107] Obtain each local view R j The video frames are arranged sequentially according to a fixed period.

[0108] The global view video sequence is obtained by rearranging the video frames acquired from all local views.

[0109]

[0110]

[0111] ...

[0112] for:

[0113] At the same time, one video frame is taken from each of the local fields of view and rearranged to obtain a video sequence unit covering the global field of view.

[0114] For all acquired video sequence units Arranged in chronological order, the video sequence set W of the global field of view is obtained;

[0115] in, k represents the kth complete cycle of the current video frame, k = 0, 1, ..., S, where S represents the number of complete cycles; i represents the i-th frame of the current complete cycle of the current video frame, i = 0, 1, ..., N, where N represents the number of image frames in each complete cycle; j represents the j-th local field of view of the current video frame, j = 0, 1, ..., M, where M represents the number of local fields of view, and S, M, N > 0.

[0116] In this specific implementation, the local field of view uses R. j This means that, under normal circumstances, nine local fields of view are used to cover the global field of view, that is, each local field of view corresponds to R0 to R8;

[0117] The proposal transforms local fields of view sequentially according to a fixed cycle. This means that a video sequence corresponding to a local field of view can be obtained within a single transformation cycle. For example, if a complete cycle consists of N frames, then N video frames corresponding to the local field of view can be obtained within one complete cycle. All local fields of view can be cycled through once every 9 cycles. Here, we use... This represents a video frame. k indicates that the current video frame is the kth complete cycle, k = 0, 1, ..., S, where S represents the number of complete cycles; i indicates that the current video frame is the i-th frame of the current complete cycle, i = 0, 1, ..., N, where N represents the number of image frames in each complete cycle; j indicates that the current video frame is the j-th local field of view, j = 0, 1, ..., M, where M represents the number of local fields of view, and S, M, N > 0.

[0118] After rearranging all video frames acquired from local views, the full [image / frame] is obtained in S+1 complete cycles.

[0119]

[0120]

[0121] ...

[0122] The video sequence with local perspective is as follows:

[0123] See Figure 3 This is a schematic diagram of the structure of a video sequence from a global perspective provided in an embodiment of the present invention; in the figure, the structure of the acquired video sequence is illustrated with a local field of view of 9.

[0124] Figure 3This represents the video sequence structure within a complete period. Each local view captures video frames within the current complete period, with a sampling period of 1 second. When 9 frames are captured in each complete period, the first local view captures video frames T within one complete period. 0,0 T 1,0 ,…,T N,0 The second local field of view acquires video frames T within a complete cycle. 0,1 T 1,1 ,…,T N,1 This allows access to all video frames within the same complete cycle.

[0125] Each complete cycle yields the same number of video frames for each local field of view. By taking one frame from each local field of view and rearranging them, a video sequence unit covering the global field of view is obtained. After recombination, within a complete cycle, the video sequence composed of local views after periodic transformation is as follows: That is, each local field of view samples N+1 frames per complete cycle, so one complete cycle can reassemble N+1 video sequence units of the global field of view.

[0126] For all acquired video sequence units Arranged chronologically, the video sequence set of the global field of view is obtained.

[0127] In another embodiment provided by the present invention, step S3 specifically includes:

[0128] Obtain each video sequence unit Included video frames

[0129] For all video frames Perform multidimensional convolution operations to obtain the input features of the global view at each time step.

[0130] in, k represents the k-th complete period of the current video frame, k = 0, 1, ..., S, where S represents the number of complete periods; i represents the i-th frame of the current complete period, i = 0, 1, ..., N, where N represents the number of image frames in each complete period; j represents the j-th local field of view of the current video frame, j = 0, 1, ..., M, where M represents the number of local fields of view, S, M, N > 0; d represents the time d, at which time i and j are known and correspond to the current time d; q represents the q-th channel video frame of the current video frame, q = 0, 1, ..., c, where c represents the number of channels in each video frame; conv represents the convolution operation. This indicates that a concatenation operation is performed on x.

[0131] In this specific implementation, acquiring video frames involves multiple different acquisition channels;

[0132] Obtain each video sequence unit Included video frames q indicates that the current video frame is the video frame of the qth channel, q = 1, ..., c, where c represents the number of channels contained in each video frame;

[0133] If a video frame contains 3 channels, then a multidimensional convolution operation is used to transform a video sequence unit. Transform into single-time input features I d It also contains 3 channels.

[0134] That is I d Mid-channel can be represented as That is, a single channel of a video frame is represented as

[0135] In practical implementation, a single-channel example will be used for explanation. See [link / reference]. Figure 4 This is a flowchart illustrating a traffic incident detection method provided in another embodiment of the present invention, which applies to all video sequence units U0 to U10. d video frames Perform n-dimensional convolution to obtain the input features I0~I0 of the global view at each time step. d ;

[0136] Single I d It contains information from all local fields of view, thus covering the entire field of view, in calculating a single I. d At the current time, i and j are known and correspond to the current time d; conv represents the convolution operation; This indicates the concat operation, which performs a concatenation operation.

[0137] In another embodiment provided by the present invention, step S4 specifically includes:

[0138] The input features I at each time step are extracted using a convolutional neural network. d Visual features F d ;

[0139] Among them, F d =Net(I d ), where Net is a deep learning network and d is the d-th time step.

[0140] In the specific implementation of this embodiment, please refer to Figure 4 In visual feature extraction, the input I at a single time step d Both methods extract visual features F using a CNN network.d That is, F d =Net(I d );

[0141] Where d represents the time step d, and Net can use common deep learning networks; in this embodiment, ResNet50 is used.

[0142] In another embodiment provided by the present invention, step S5 specifically includes:

[0143] A long short-term memory recurrent neural network is used to extract temporal information from visual features, and the probability of various traffic events occurring at the current moment is analyzed [Y]. 0,d ,…,Y p,d ];

[0144] Among them, [Y 0,d ,…,Y p,d ] = LSTM(F d-1 ,F d ), where d represents time d, Y 0,d Let F represent the probability of the x-th traffic event occurring at time d, where x = 0, 1, ..., p, p > 0. d F represents the visual features at time d. d-1 This represents the visual features at time d-1.

[0145] In the specific implementation of this embodiment, see [link / reference]. Figure 4 In temporal learning, a long short-term memory recurrent neural network is used. That is, for the visual features extracted at each time step, LSTM is used to extract temporal information and analyze the probability of each event. At each time step, the output of LSTM is a one-dimensional array, which stores the probabilities of events 0 to p in order from left to right.

[0146] By using LSTM to calculate the visual features F0 at time 0, the probability of events 0 to p occurring at that time is obtained [Y]. 0,0 ,…,Y p,0 ];

[0147] By using LSTM to calculate the visual features F1 at time 1, the occurrence probabilities [Y] of events 0 to p at that time are obtained. 0,1 ,…,Y p,1 ];

[0148] Similarly, by analyzing the visual features F at time d... d Using LSTM calculation, the probability of events 0 to p occurring at that time is obtained [Y]. 0,d ,…,Y p,d ];

[0149] Based on deep learning and computer vision theories and methods, and utilizing the characteristics of multidimensional convolution and temporal analysis, this paper proposes a multi-traffic event fusion and discrimination algorithm architecture based on video sequences with periodic transformation local vision. While increasing the types of traffic events that can be discriminated, it generates almost no additional computational resources, facilitating the continuous expansion of the algorithm's functions.

[0150] In another embodiment provided by the present invention, the method further includes:

[0151] Determine the probability of each event occurring at each moment;

[0152] When an event occurs with a probability not less than a preset threshold, the time and probability of the event are recorded, and an alarm message is generated.

[0153] In the specific implementation of this embodiment, the probability Y of each event occurring at each time moment is calculated. x,d The comparison and judgment are made with the preset threshold probability;

[0154] When an event occurs with a probability not less than a preset threshold, the time and probability of the event are recorded, and an alarm message is generated.

[0155] For example, if the probability of a congestion event at time 5 is 80%, which is greater than the set threshold probability of 75%, then the probability of a congestion event at time 5 is recorded as 80%, and a corresponding alarm message is generated so that managers can handle this event.

[0156] The traffic incident detection method provided by this invention can support the fusion judgment of multiple traffic incidents, and has the characteristics of end-to-end, lightweight, and low cost. It can be deployed rapidly on a large scale, greatly improving the ability to make refined and high-precision judgments and management of real-time traffic incidents.

[0157] In yet another embodiment provided by the present invention, see Figure 5 This is a schematic diagram of the structure of a traffic incident detection device provided in an embodiment of the present invention. The device includes: an image acquisition module, a video sequence set acquisition module, an input feature calculation module, a visual feature calculation module, and a probability calculation module.

[0158] The image acquisition module is used to control the PTZ camera to sequentially acquire video frames of each local field of view divided by the global field of view of the PTZ camera according to preset polling parameters;

[0159] The video sequence set acquisition module is used to rearrange all the video frames of the local field of view acquired by the PTZ camera to obtain the video sequence set of the global field of view;

[0160] The input feature calculation module is used to perform multidimensional convolution operations on each video sequence unit in the video sequence set to obtain the input features of the global field of view at each time step.

[0161] The visual feature calculation module is used to input the obtained input features into the convolutional neural network to extract the visual features at each time step.

[0162] The probability calculation module is used to input the obtained visual features into the recurrent neural network in chronological order for analysis, and to obtain the probability of occurrence of each event at each time step.

[0163] The traffic incident detection device provided in this embodiment can perform all the steps and functions of the traffic incident detection method provided in any of the above embodiments. The specific functions of the device will not be described in detail here.

[0164] See Figure 6 This is a schematic diagram of a terminal device provided in an embodiment of the present invention. The terminal device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a traffic incident detection program. When the processor executes the computer program, it implements the steps in the various embodiments of the traffic incident detection method described above, for example... Figure 1 The steps S1 to S5 are shown. Alternatively, when the processor executes the computer program, it implements the functions of each module in the above-described device embodiments.

[0165] For example, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the traffic incident detection device. For example, the computer program can be divided into various modules, the specific functions of which have been described in detail in the traffic incident detection method provided in any of the above embodiments; therefore, the specific functions of the device will not be repeated here.

[0166] The traffic incident detection device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The traffic incident detection device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the schematic diagram is merely an example of a traffic incident detection device and does not constitute a limitation on the device. It may include more or fewer components than illustrated, or combine certain components, or use different components. For example, the traffic incident detection device may also include input / output devices, network access devices, buses, etc.

[0167] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. This processor is the control center of the traffic incident detection device, connecting all parts of the device via various interfaces and lines.

[0168] The memory can be used to store the computer program and / or modules. The processor implements various functions of the traffic incident detection device by running or executing the computer program and / or modules stored in the memory and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0169] If the integrated module of the traffic incident detection device is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0170] It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered to be within the scope of protection of this invention.

Claims

1. A method for detecting traffic incidents, characterized in that, The method includes: The PTZ camera is controlled to sequentially acquire video frames of each local field of view divided by the global field of view of the PTZ camera according to preset polling parameters; The video frames of all local fields of view acquired by the PTZ camera are rearranged to obtain the video sequence set of the global field of view; Multidimensional convolution operation is performed on each video sequence unit in the video sequence set to obtain the input features of the global field of view at each time step; The obtained input features are fed into a convolutional neural network to extract visual features at each time step; The obtained visual features are input into a recurrent neural network in chronological order for analysis to obtain the probability of occurrence of each event at each time step. The step of rearranging all the video frames of the local field of view acquired by the PTZ camera to obtain the video sequence set of the global field of view specifically includes: Acquire each local field of view The video frames are arranged sequentially according to a fixed period. ; The video sequence of the global field of view obtained by rearranging the video frames acquired from all local fields of view is as follows: ; At the same time, one video frame is taken from each of the local fields of view and rearranged to obtain a video sequence unit covering the global field of view. ; For all acquired video sequence units Arranged in chronological order, the video sequence set W of the global field of view is obtained; in, k indicates that the current video frame is the kth complete cycle, k=0,1,…,S, where S represents the number of complete cycles; i indicates that the current video frame is the i-th frame of the current complete cycle, i=0,1,…,N, where N represents the number of image frames in each complete cycle; j indicates that the current video frame is the j-th local field of view, j=0,1,…,M, where M represents the number of local fields of view, and S, M, N>

0.

2. The traffic incident detection method according to claim 1, characterized in that, The polling parameter determination process specifically includes: The global field of view to be monitored is divided into several local fields of view according to preset rules; The field of view parameters for each local field of view are determined based on the viewing angle of each local field of view and the installation position of the PTZ camera. The field of view parameters include the field of view angle and the field of view focal length. The polling parameters of the PTZ camera are determined based on the preset polling order, preset transformation period, and field of view parameters of each local field of view for all local fields of view.

3. The traffic incident detection method according to claim 1, characterized in that, The step of performing multidimensional convolution operations on each video sequence unit in the video sequence set to obtain the input features of the global field of view at each time step specifically includes: Obtain each video sequence unit Included video frames ; For all video frames Perform multidimensional convolution operations to obtain the input features of the global view at each time step. ; in, Let k represent the current video frame as the kth complete cycle, k = 0, 1, ..., S, where S represents the number of complete cycles; i represent the current video frame as the i-th frame of the current complete cycle, i = 0, 1, ..., N, where N represents the number of image frames in each complete cycle; j represent the current video frame as the j-th local field of view, j = 0, 1, ..., M, where M represents the number of local fields of view, and S, M, N > 0; d represents the time d, at which time i and j are known and correspond to the current time d; q represents the current video frame as the q-th channel video frame, q = 0, 1, ..., c, where c represents the number of channels contained in each video frame. This represents the convolution operation. Indicates to x Perform the splicing operation.

4. The traffic incident detection method according to claim 1, characterized in that, The process of inputting the obtained input features into a convolutional neural network to extract visual features at each time step specifically includes: Extracting input features at each time step using a convolutional neural network visual features ; in, Net represents a deep learning network, and d represents the d-th time step.

5. The traffic incident detection method according to claim 1, characterized in that, The process involves inputting the obtained visual features into a recurrent neural network in chronological order for analysis to obtain the probability of occurrence of each event at each time step. Specifically, this includes: A long short-term memory recurrent neural network is used to extract temporal information from visual features, and the probability of various traffic events occurring at the current moment is analyzed. ; in, d represents time d. Let x represent the probability of the x-th traffic event occurring at time d, where x = 0, 1, ..., p, p >

0. Represents the visual features at time d. This represents the visual features at time d-1.

6. The traffic incident detection method according to claim 1, characterized in that, The method further includes; Determine the probability of each event occurring at each moment; When an event occurs with a probability not less than a preset threshold, the time and probability of the event are recorded, and an alarm message is generated.

7. A traffic incident detection device, characterized in that, For implementing the traffic incident detection method as described in any one of claims 1 to 6, the apparatus comprises: The image acquisition module is used to control the PTZ camera to sequentially acquire video frames of each local field of view divided by the global field of view of the PTZ camera according to preset polling parameters; The video sequence set acquisition module is used to rearrange all the video frames of the local field of view acquired by the PTZ camera to obtain the video sequence set of the global field of view; The input feature calculation module is used to perform multidimensional convolution operations on each video sequence unit in the video sequence set to obtain the input features of the global field of view at each time step. The visual feature calculation module is used to input the obtained input features into the convolutional neural network to extract the visual features at each time step. The probability calculation module is used to input the obtained visual features into the recurrent neural network in chronological order for analysis, and to obtain the probability of occurrence of each event at each time step. The video sequence set acquisition module is specifically used to rearrange all video frames of the local field of view acquired by the PTZ camera to obtain the video sequence set of the global field of view, specifically including: Acquire each local field of view The video frames are arranged sequentially according to a fixed period. ; The video sequence of the global field of view obtained by rearranging the video frames acquired from all local fields of view is as follows: ; At the same time, one video frame is taken from each of the local fields of view and rearranged to obtain a video sequence unit covering the global field of view. ; For all acquired video sequence units Arranged in chronological order, the video sequence set W of the global field of view is obtained; in, k indicates that the current video frame is the kth complete cycle, k=0,1,…,S, where S represents the number of complete cycles; i indicates that the current video frame is the i-th frame of the current complete cycle, i=0,1,…,N, where N represents the number of image frames in each complete cycle; j indicates that the current video frame is the j-th local field of view, j=0,1,…,M, where M represents the number of local fields of view, and S, M, N>

0.

8. A terminal device, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the traffic incident detection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the traffic incident detection method as described in any one of claims 1 to 6.

Images