A kind of intelligent traffic police robot control system and control method thereof

Abstract

The present invention relates to the technical field of traffic accident determination, and discloses an intelligent traffic police robot control system and a control method thereof, including: a data acquisition module, which is used to collect image data, vehicle data, and environmental data; a data preprocessing module, which is used to preprocess the image data and vehicle data to obtain first feature data and second feature data respectively; an accident preliminary judgment module, which is used to construct an accident preliminary judgment model, and output the probability of a traffic accident occurring at an intersection within a first preset time period; a site verification module, which is used to start the site verification program of the traffic police robot when a traffic accident is suspected to occur at an intersection, and manually confirm the scene image of the intersection through the traffic police robot; an emergency handling module, which is used to start emergency measures when it is determined that a traffic accident occurs at an intersection. The present invention realizes the high efficiency, accuracy and reliability of traffic accident determination, while reducing manpower and time costs.

Description

Technical Field

[0001] The invention belongs to the technical field of traffic accident determination, and specifically relates to an intelligent traffic police robot control system and a control method thereof. Background Art

[0002] In order to improve the efficiency of traffic accident determination, many cities have deployed surveillance cameras at major intersections to monitor traffic flow and vehicle status in real time through video images. However, the existing accident determination technology based on intersection cameras still has many shortcomings, which restricts the actual application effect of the system.

[0003] First, the fixed viewing angle of the intersection camera results in a limited monitoring range, making it difficult to fully cover all areas of the intersection. Especially after an accident, the detailed information of the vehicle (such as dents on the vehicle body, collision marks) or the distribution of debris at the scene often cannot be accurately identified. Secondly, in complex traffic scenarios, such as abnormal detention or emergency parking of multiple vehicles, it is difficult to identify the accident vehicle and its cause only through the remote image of the camera. In this case, it is often necessary for humans to arrive at the scene for review, which increases the labor cost and time cost. Summary of the invention

[0004] The present invention provides an intelligent traffic police robot control system and a control method thereof, which solve the technical problems of high labor cost and low time efficiency in related technologies.

[0005] The present invention provides an intelligent traffic police robot control system, comprising:

[0006] A data acquisition module, which is used to perform multi-source data acquisition tasks within a preset time period, and the multi-source data acquisition tasks include: collecting image data through intersection cameras at preset time intervals within the preset time period, collecting vehicle data through radar equipment, and collecting environmental data, the vehicle data includes: position, speed, acceleration and vehicle status, and the environmental data includes: signal light status and weather status;

[0007] The data preprocessing module is used to preprocess the image data and the vehicle data to obtain the first feature data and the second feature data respectively. The specific steps include:

[0008] Step S201, use the image data Processing, in which represents the processed pixel value, x represents the current pixel position, represents the filter window centered on x, and y represents the other pixel positions within the filter window. , and Respectively represent the spatial weight parameter, grayscale weight parameter and edge directionality weight parameter, and Represents the grayscale value of the pixel at the x position and the pixel at the y position respectively, Represents the spatial distance between the pixel at the x position and the pixel at the y position, and represents the value of the pixel at the x position and the pixel at the y position, exp represents the exponential function, represents pixel gradient;

[0009] Step S202, performing histogram equalization processing on the image data processed in step S201 to obtain first feature data;

[0010] Step S203, extracting the emergency brake impact index feature according to the vehicle data and splicing it after the vehicle data. The calculation formula of the emergency brake impact index feature is: ,in, Indicates the characteristics of the emergency brake impact index. and Indicates the start and end time of the monitoring period. represents the first derivative of acceleration, i.e. jerk, The vehicle at time The acceleration of is the rate of change of velocity with time, is the second derivative of acceleration, , and is the weight coefficient;

[0011] Step S204: extract the deviation stability coefficient feature according to the vehicle data and splice it after the vehicle data. The calculation formula of the deviation stability coefficient feature is: ,in, represents the deviation stability coefficient characteristic, N represents the number of acquisition time points within the preset time period, i represents the number index of acquisition time points, represents the y-axis position of the vehicle at the i-th acquisition time point, Indicates the y-axis position of the center line of the vehicle's current lane, Indicates the distance the vehicle deviates from the center of the lane. The vehicle is The speed of each acquisition time point, is the maximum speed allowed on the road, The vehicle is The acceleration at each acquisition time point, The lane is in The curvature of each acquisition time point, , and is the weight coefficient;

[0012] Step S205: extract potential collision probability features based on vehicle data and concatenate them with the vehicle data to obtain second feature data. The calculation formula of potential collision probability features is: ,in, Represents the potential collision probability feature, with a value range of 0 to 1. represents the horizontal coordinate of the vehicle at time t, represents the ordinate of the vehicle at time t, represents the horizontal coordinate of the preceding vehicle at time t, represents the ordinate of the preceding vehicle at time t, represents the distance attenuation coefficient;

[0013] An accident preliminary judgment module is used to construct an accident preliminary judgment model based on the first characteristic data, the second characteristic data and the environmental data. The accident preliminary judgment model uses the first characteristic data, the second characteristic data and the environmental data as inputs, and uses the probability of a traffic accident occurring at an intersection within a first preset time period as output; and performs graded judgment on the probability of a traffic accident occurring. When the probability of a traffic accident occurring is higher than a first preset threshold, it is determined that a traffic accident has occurred at the intersection; when the probability of a traffic accident occurring is between the second preset threshold and the first preset threshold, it is determined that a traffic accident is suspected to have occurred at the intersection; when the probability of a traffic accident occurring is lower than the second preset threshold, it is determined that no traffic accident has occurred at the intersection;

[0014] The on-site verification module is used to start the on-site verification procedure of the traffic police robot when a traffic accident is suspected to have occurred at the intersection; the on-site verification procedure includes: the traffic police robot obtains the on-site image of the intersection through the camera equipped with it, and transmits the on-site image of the intersection to the traffic control center, and the personnel of the traffic control center manually confirm the on-site image of the intersection;

[0015] The emergency handling module is used to initiate emergency measures when it is determined that a traffic accident occurs at an intersection.

[0016] Furthermore, a two-dimensional rectangular coordinate system is established with the radar as the origin, the x-axis represents the direction in front of the radar, the y-axis represents the direction to the side of the radar, and the position is the coordinate of the vehicle in the coordinate system;

[0017] The vehicle states include: acceleration state, driving state, deceleration state and stationary state. The mathematical expressions of the four vehicle states include:

[0018] ;

[0019] ;

[0020] ;

[0021] ;

[0022] in, , , and They represent the acceleration state, driving state, deceleration state and stationary state respectively, v and a represent the speed and acceleration of the vehicle respectively, and Indicates the first preset speed and the second preset speed, , and represent the first preset acceleration, the second preset acceleration and the third preset acceleration respectively, Indicates the absolute value of vehicle acceleration;

[0023] The signal light status includes: red light, yellow light and green light, and the signal light status is represented by one-hot coding. The weather status includes: sunny day, cloudy day, rainy day, foggy day and snowy day, and the weather status is represented by one-hot coding.

[0024] Furthermore, the accident preliminary judgment model includes: an image feature extraction unit, a vehicle feature extraction unit, an environment feature extraction unit, a mosaic unit and a first classifier;

[0025] The image feature extraction unit is used to input first feature data and output image features;

[0026] The vehicle feature extraction unit is used to input the second feature data and output the vehicle feature;

[0027] The environmental feature extraction unit is used to splice the signal light status and weather status of the environmental data to obtain the environmental features;

[0028] The mosaic unit is used for stacking the image features, the vehicle features and the environment features, and mosaicking them to obtain mosaic features;

[0029] The chimeric features are expanded into chimeric vectors and input into a first classifier, and the classification space of the first classifier represents the probability of a traffic accident occurring at the intersection within a first preset time period.

[0030] Furthermore, an image feature extraction unit is constructed based on a convolutional neural network to extract image features; and the image feature extraction unit is pre-trained before training the accident preliminary judgment model. During the pre-training process, the image features output by the image feature extraction unit are expanded into vectors and then input into the second classifier. The classification space of the second classifier represents whether the vehicle is deformed, and the classification labels of the second classifier are manually labeled.

[0031] Furthermore, the vehicle feature extraction unit is constructed based on an improved gated recurrent unit. The structure of the improved gated recurrent unit includes: an update gate, a reset gate, a candidate hidden state, a hidden state update and a global memory unit. The improvements include: adding time-sensitive adjustment and multi-head feature fusion in the update gate, introducing a multi-head memory mechanism in the reset gate, introducing an attention mechanism in the candidate hidden state, and fusing the multi-head memory, global memory and the current candidate state in the hidden state update stage.

[0032] Furthermore, the image features, vehicle features and environmental features are stacked to obtain a multimodal feature matrix. The specific process is: the image features are expanded into image feature vectors, and the image feature vectors, vehicle features and environmental features are stacked. The mathematical expression of the multimodal feature matrix is: , represents the multimodal feature matrix, represents the image feature vector, Indicates vehicle characteristics, Represents environmental features; the multimodal feature matrix is ​​embedded to obtain the embedded features, and the embedded calculation formula is:

[0033] ;

[0034] in, Indicates chimeric features, , , and They represent query weight parameter, key weight parameter, global weight parameter and value weight parameter respectively, o represents the row index of the multimodal feature matrix, It represents the selection of all columns of the o-th row of the multimodal feature matrix, d represents the number of columns of the multimodal feature matrix, and T represents the transposition operation.

[0035] The present invention provides a method for controlling an intelligent traffic police robot, comprising the following steps:

[0036] Step S301, executing a multi-source data collection task within a preset time period, the multi-source data collection task including: collecting image data through a road intersection camera at a preset time interval within the preset time period, collecting vehicle data through a radar device, and collecting environmental data;

[0037] Step S302, preprocessing the image data and the vehicle data to obtain first feature data and second feature data respectively;

[0038] Step S303, constructing an accident preliminary judgment model based on the first characteristic data, the second characteristic data and the environmental data, outputting the probability of a traffic accident occurring at the intersection within the first preset time period; and performing a graded judgment on the probability of a traffic accident occurring;

[0039] Step S304, when a traffic accident is suspected to have occurred at the intersection, the on-site verification program of the traffic police robot is started to manually confirm the on-site image of the intersection;

[0040] Step S305, when it is determined that a traffic accident occurs at the intersection, emergency measures are initiated.

[0041] The beneficial effects of the present invention are as follows: the present invention constructs an accident preliminary judgment model by integrating image features, vehicle features and environmental features. Through the complementarity of multi-source data, it reduces the misjudgment or missed judgment caused by a single data source, and significantly improves the judgment accuracy of traffic accidents;

[0042] The present invention introduces a traffic police robot for on-site verification, takes close-up photos of suspected accidents and conducts secondary confirmation, thus solving the problem of blind spots in the fixed camera’s viewing angle. At the same time, this method reduces the possibility of suspected accidents being misjudged as actual accidents, thus providing a guarantee for the reliability of the system.

[0043] The present invention significantly reduces the frequency and intensity of manual review through intelligent collection and automatic judgment of multi-source data, especially in complex weather conditions or during peak hours. The system's automation capabilities can effectively reduce the manpower cost of traffic management and improve overall operational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a module schematic diagram of an intelligent traffic police robot control system of the present invention;

[0045] Figure 2 It is a flow chart of preprocessing image data and vehicle data of the present invention.

[0046] In the figure: a data acquisition module 101, a data pre-processing module 102, an accident preliminary judgment module 103, an on-site verification module 104 and an emergency processing module 105. DETAILED DESCRIPTION

[0047] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the contents of this specification. Each example may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0048] like Figure 1-Figure 2 As shown, an intelligent traffic police robot control system and a control method thereof, comprising:

[0049] The data acquisition module 101 is used to perform a multi-source data acquisition task within a preset time period, and the multi-source data acquisition task includes: collecting image data through a road intersection camera at a preset time interval within the preset time period, collecting vehicle data through a radar device, and collecting environmental data, wherein the vehicle data includes: position, speed, acceleration and vehicle status, and the environmental data includes: signal light status and weather status;

[0050] The data preprocessing module 102 is used to preprocess the image data and the vehicle data to obtain the first feature data and the second feature data respectively. The specific steps include:

[0051] Step S201, use the image data Processing, in which represents the processed pixel value, x represents the current pixel position, represents the filter window centered on x. Preferably, the filter window is set to a 5×5 square window. y represents other pixel positions within the filter window. , and Respectively represent the spatial weight parameter, grayscale weight parameter and edge directionality weight parameter, and Represents the grayscale value of the pixel at the x position and the pixel at the y position respectively, Represents the spatial distance between the pixel at the x position and the pixel at the y position, and represents the value of the pixel at the x position and the pixel at the y position, exp represents the exponential function, Represents the pixel gradient, that is, the changing trend of the image at the x position;

[0052] Step S202, performing histogram equalization processing on the image data processed in step S201 to obtain first feature data, so as to enhance contrast and make brightness distribution more uniform;

[0053] Step S203, extracting the emergency brake impact index feature according to the vehicle data and splicing it after the vehicle data. The calculation formula of the emergency brake impact index feature is: ,in, Indicates the characteristics of the sudden brake impact index. and Indicates the start and end time of the monitoring period. represents the first derivative of acceleration, i.e. jerk, The vehicle at time The acceleration of is the rate of change of velocity with time, is the second derivative of acceleration, , and is the weight coefficient;

[0054] Step S204: extract the deviation stability coefficient feature according to the vehicle data and splice it after the vehicle data. The calculation formula of the deviation stability coefficient feature is: ,in, represents the deviation stability coefficient characteristic, N represents the number of acquisition time points within the preset time period, i represents the number index of acquisition time points, represents the y-axis position of the vehicle at the i-th acquisition time point, Indicates the y-axis position of the center line of the vehicle's current lane, Indicates the distance the vehicle deviates from the center of the lane. The vehicle is The speed of each acquisition time point, is the maximum speed allowed on the road, The vehicle is The acceleration at each acquisition time point, The lane is in The curvature of each acquisition time point, , and is the weight coefficient;

[0055] Step S205: extract potential collision probability features based on vehicle data and concatenate them with the vehicle data to obtain second feature data. The calculation formula of potential collision probability features is: ,in, Represents the potential collision probability feature, with a value range of 0 to 1. represents the horizontal coordinate of the vehicle at time t, represents the ordinate of the vehicle at time t, represents the horizontal coordinate of the preceding vehicle at time t, represents the ordinate of the preceding vehicle at time t, It represents the distance attenuation coefficient. The larger the value of the potential collision probability feature is, the higher the risk of collision is.

[0056] The accident preliminary judgment module 103 is used to construct an accident preliminary judgment model based on the first characteristic data, the second characteristic data and the environmental data. The accident preliminary judgment model uses the first characteristic data, the second characteristic data and the environmental data as inputs, and uses the probability of a traffic accident occurring at an intersection within a first preset time period as output; and performs graded judgment on the probability of a traffic accident occurring. When the probability of a traffic accident occurring is higher than a first preset threshold, it is determined that a traffic accident has occurred at the intersection; when the probability of a traffic accident occurring is between the second preset threshold and the first preset threshold, it is determined that a traffic accident is suspected to have occurred at the intersection; when the probability of a traffic accident occurring is lower than the second preset threshold, it is determined that no traffic accident has occurred at the intersection;

[0057] The on-site verification module 104 is used to start the on-site verification procedure of the traffic police robot when a traffic accident is suspected to have occurred at the intersection; the on-site verification procedure includes: the traffic police robot obtains the on-site image of the intersection through the camera equipped with it, and transmits the on-site image of the intersection to the traffic control center, and the personnel of the traffic control center manually confirm the on-site image of the intersection;

[0058] The emergency processing module 105 is used to initiate emergency measures when it is determined that a traffic accident occurs at an intersection.

[0059] In one embodiment of the present invention, a traffic police robot is connected to an urban traffic management system to obtain image data and traffic light status at preset time intervals at intersections through a real-time streaming protocol; a laser radar is fixed on a signal pole at the intersection to obtain the position, speed, acceleration and status of the vehicle at preset time intervals; and the traffic police robot is connected to a weather service interface to obtain the weather status at the intersection.

[0060] In one embodiment of the present invention, a two-dimensional rectangular coordinate system is established with the radar as the origin, the x-axis represents the direction directly in front of the radar, the y-axis represents the direction directly to the side of the radar, and the position is the coordinate of the vehicle in the coordinate system;

[0061] The vehicle states include: acceleration state, driving state, deceleration state and stationary state. The mathematical expressions of the four vehicle states include:

[0062] ;

[0063] ;

[0064] ;

[0065] ;

[0066] in, , , and They represent the acceleration state, driving state, deceleration state and stationary state respectively, v and a represent the speed and acceleration of the vehicle respectively, and Indicates the first preset speed and the second preset speed, , and represent the first preset acceleration, the second preset acceleration and the third preset acceleration respectively, represents the absolute value of the vehicle acceleration, preferably, and Set to 5 respectively and 2 , , and Set to 2 respectively , -2 and 0.5 ;

[0067] The signal light status includes: red light, yellow light and green light, and the signal light status is represented by one-hot coding. The weather status includes: sunny day, cloudy day, rainy day, foggy day and snowy day, and the weather status is represented by one-hot coding.

[0068] In one embodiment of the present invention, the environmental data is normalized by a maximum-minimum normalization method.

[0069] In one embodiment of the present invention, the accident preliminary judgment model includes: an image feature extraction unit, a vehicle feature extraction unit, an environment feature extraction unit, a chimera unit and a first classifier;

[0070] The image feature extraction unit is used to input first feature data and output image features;

[0071] The vehicle feature extraction unit is used to input the second feature data and output the vehicle feature;

[0072] The environmental feature extraction unit is used to splice the signal light status and weather status of the environmental data to obtain the environmental features;

[0073] The mosaic unit is used for stacking the image features, the vehicle features and the environment features, and mosaicking them to obtain mosaic features;

[0074] The chimeric features are expanded into chimeric vectors and input into a first classifier, and the classification space of the first classifier represents the probability of a traffic accident occurring at the intersection within a first preset time period.

[0075] In one embodiment of the present invention, an image feature extraction unit is constructed based on a convolutional neural network and is used to extract image features. The image feature extraction unit is pre-trained before training the accident initial judgment model. During the pre-training process, the image features output by the image feature extraction unit are expanded into vectors and then input into a second classifier. The classification space of the second classifier represents whether the vehicle is deformed, and the classification labels of the second classifier are manually labeled.

[0076] Through the above pre-training, the image feature extraction unit already has a certain feature extraction capability, and the training process of the accident initial judgment model only needs fine-tuning, reducing training time and data requirements; the image feature extraction unit and the accident initial judgment model are designed separately to improve the flexibility and scalability of the system. For example, the image feature extraction unit can be reused in other scenarios. In addition, the objectives of the second classifier can be flexibly changed.

[0077] In one embodiment of the present invention, a vehicle feature extraction unit is constructed based on an improved gated recurrent unit. The structure of the improved gated recurrent unit includes: an update gate, a reset gate, a candidate hidden state, a hidden state update and a global memory unit. The improvements include: adding time-sensitive adjustment and multi-head feature fusion in the update gate, introducing a multi-head memory mechanism in the reset gate, introducing an attention mechanism in the candidate hidden state, and fusing the multi-head memory, global memory and the current candidate state in the hidden state update stage.

[0078] In one embodiment of the present invention, the calculation formula of the improved gated recurrent unit includes:

[0079] The calculation formula of the update gate includes:

[0080] ;

[0081] The calculation formula for the reset gate includes:

[0082] ;

[0083] The calculation formula for candidate hidden states includes:

[0084] ;

[0085] ;

[0086] The calculation formula for hidden state update includes:

[0087] ;

[0088] The calculation formula of the global memory unit includes:

[0089] ;

[0090] ;

[0091] in, represents the input of the vehicle feature extraction unit at the nth time point, that is, the data representation of the second feature data at the nth time point, represents the output of the vehicle feature extraction unit at the nth time point, and the output of the vehicle feature extraction unit at the Nth time point is the vehicle characteristic, , , , , and They represent the update gate, the reset gate of the k-th memory head, the candidate hidden state, the candidate hidden state of the k-th memory head, the global memory unit, and the global memory update gate, respectively. , , and Respectively represent the first weight parameter, the second weight parameter, the third weight parameter and the bias parameter of the update gate, represents the output of the vehicle feature extraction unit at the n-1th time point, represents the time attenuation coefficient, which attenuates the information weight of long time steps, sigmoid represents the sigmoid activation function, , and represents the first weight parameter, the second weight parameter and the bias parameter of the reset gate of the kth memory head, Q represents the number of memory heads, k and j represent the index of the memory head, exp represents the exponential function, represents the attention weight coefficient, , and They represent the first weight parameter, the second weight parameter and the bias parameter of the candidate hidden state of the kth memory head respectively, represents the candidate hidden state of the jth memory head, T represents the transposition operation, represents point-wise multiplication, represents the tanh activation function, represents the global memory unit at the n-1th time point, and Respectively represent the first weight parameter and the second weight parameter of the global memory unit, , and They respectively represent the first weight parameter, second weight parameter and bias parameter of the global memory update gate.

[0092] In the improved gated recurrent unit, by introducing a multi-head memory mechanism, multiple candidate hidden states are used to capture information of different time scales; the introduction of a time decay coefficient can strengthen recent features; the update gate focuses on short-term information and determines the degree of updating of the hidden state, and the global memory update gate focuses on long-term information and determines the update strategy of the global memory unit.

[0093] In one embodiment of the present invention, the image features, vehicle features and environmental features are stacked to obtain a multimodal feature matrix. The specific process is: the image features are expanded into image feature vectors, and the image feature vectors, vehicle features and environmental features are stacked, wherein the mathematical expression of the multimodal feature matrix is: , each row of the multimodal feature matrix represents a feature vector, and the number of rows represents the type of feature. Specifically, represents the multimodal feature matrix, represents the image feature vector, Indicates vehicle characteristics, Represents environmental features; the multimodal feature matrix is ​​embedded to obtain the embedded features, and the embedded calculation formula is:

[0094] ;

[0095] in, Indicates chimeric features, , , and They represent query weight parameter, key weight parameter, global weight parameter and value weight parameter respectively, o represents the row index of the multimodal feature matrix, It represents the selection of all columns of the o-th row of the multimodal feature matrix, d represents the number of columns of the multimodal feature matrix, and T represents the transposition operation.

[0096] In one embodiment of the present invention, a cross entropy loss function is used as the loss function of the accident initial judgment model.

[0097] In one embodiment of the present invention, the probability of a traffic accident occurring at an intersection obtained by the accident preliminary judgment model is divided into three risk levels. When the probability of a traffic accident occurring at an intersection is greater than 0.9, the risk level is high, indicating that the possibility of a traffic accident is very high. The emergency measures for the high risk level are: turning on the intersection warning lights to alert pedestrians and vehicles, adjusting the status of traffic lights on surrounding sections, diverting traffic that may be congested, and sending the accident location and time to the traffic control center; when the probability of a traffic accident occurring at an intersection is between 0.5 and 0.9, the risk level is medium, indicating that a traffic accident is suspected to have occurred. The emergency measures for the medium risk level are: dispatching a traffic police robot to the suspected accident point, using the camera on the robot's head to take close-up photos of the vehicle status and road status, and transmitting the captured images to the traffic control center, which will be manually reviewed by the control center staff to confirm whether an accident has occurred, and continuously monitor the suspected road sections; when the probability of a traffic accident occurring at an intersection is less than 0.5, the risk level is low, indicating that traffic is normal and no traffic accident has occurred.

[0098] In one embodiment of the present invention, since cameras at intersections are usually installed in fixed positions and cannot cover all details of the vehicle (such as dents on the vehicle body and ground debris), it may not be possible to fully determine that a traffic accident has occurred at the intersection. Therefore, for medium risk levels, a secondary confirmation method is used to reduce the situation where a "suspected accident" is misjudged as an "actual accident."

[0099] In one embodiment of the present invention, the present invention provides a method for controlling an intelligent traffic police robot, comprising the following steps:

[0100] Step S301, executing a multi-source data collection task within a preset time period, the multi-source data collection task including: collecting image data through a road intersection camera at a preset time interval within the preset time period, collecting vehicle data through a radar device, and collecting environmental data;

[0101] Step S302, preprocessing the image data and the vehicle data to obtain first feature data and second feature data respectively;

[0102] Step S303, constructing an accident preliminary judgment model based on the first characteristic data, the second characteristic data and the environmental data, outputting the probability of a traffic accident occurring at the intersection within the first preset time period; and performing a graded judgment on the probability of a traffic accident occurring;

[0103] Step S304, when a traffic accident is suspected to have occurred at the intersection, the on-site verification program of the traffic police robot is started to manually confirm the on-site image of the intersection;

[0104] Step S305, when it is determined that a traffic accident occurs at the intersection, emergency measures are initiated.

[0105] The above describes an embodiment of the present embodiment, but the present embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present embodiment, ordinary technicians in this field can also make many forms, all of which are within the protection of the present embodiment.

Claims

1. An intelligent traffic police robot control system, characterized in that: include: A data acquisition module, which is used to perform multi-source data acquisition tasks within a preset time period, and the multi-source data acquisition tasks include: collecting image data through intersection cameras at preset time intervals within the preset time period, collecting vehicle data through radar equipment, and collecting environmental data, the vehicle data includes: position, speed, acceleration and vehicle status, and the environmental data includes: signal light status and weather status; The data preprocessing module is used to preprocess the image data and the vehicle data to obtain the first feature data and the second feature data respectively. The specific steps include: Step S201, use the image data Processing, in which represents the processed pixel value, x represents the current pixel position, represents the filter window centered on x, and y represents the other pixel positions within the filter window. , and Respectively represent the spatial weight parameter, grayscale weight parameter and edge directionality weight parameter, and Represents the grayscale value of the pixel at the x position and the pixel at the y position respectively, Represents the spatial distance between the pixel at the x position and the pixel at the y position, and represents the value of the pixel at the x position and the pixel at the y position, exp represents the exponential function, represents pixel gradient; Step S202, performing histogram equalization processing on the image data processed in step S201 to obtain first feature data; Step S203, extracting the emergency brake impact index feature according to the vehicle data and splicing it after the vehicle data. The calculation formula of the emergency brake impact index feature is: ,in, Indicates the characteristics of the emergency brake impact index. and Indicates the start and end time of the monitoring period. represents the first derivative of acceleration, i.e. jerk, The vehicle at time The acceleration of is the rate of change of velocity with time, is the second derivative of acceleration, , and is the weight coefficient; Step S204: extract the deviation stability coefficient feature according to the vehicle data and splice it after the vehicle data. The calculation formula of the deviation stability coefficient feature is: ,in, represents the deviation stability coefficient characteristic, N represents the number of acquisition time points within the preset time period, i represents the number index of acquisition time points, represents the y-axis position of the vehicle at the i-th acquisition time point, Indicates the y-axis position of the center line of the vehicle's current lane, Indicates the distance the vehicle deviates from the center of the lane. The vehicle is The speed of each acquisition time point, is the maximum speed allowed on the road, The vehicle is The acceleration at each acquisition time point, The lane is in The curvature of each acquisition time point, , and is the weight coefficient; Step S205: extract potential collision probability features based on vehicle data and concatenate them with the vehicle data to obtain second feature data. The calculation formula of potential collision probability features is: ,in, Represents the potential collision probability feature, with a value range of 0 to 1. represents the horizontal coordinate of the vehicle at time t, represents the ordinate of the vehicle at time t, represents the horizontal coordinate of the preceding vehicle at time t, represents the ordinate of the preceding vehicle at time t, represents the distance attenuation coefficient; An accident preliminary judgment module is used to construct an accident preliminary judgment model based on the first characteristic data, the second characteristic data and the environmental data. The accident preliminary judgment model uses the first characteristic data, the second characteristic data and the environmental data as inputs, and uses the probability of a traffic accident occurring at an intersection within a first preset time period as output; and performs graded judgment on the probability of a traffic accident occurring. When the probability of a traffic accident occurring is higher than a first preset threshold, it is determined that a traffic accident has occurred at the intersection; when the probability of a traffic accident occurring is between the second preset threshold and the first preset threshold, it is determined that a traffic accident is suspected to have occurred at the intersection; when the probability of a traffic accident occurring is lower than the second preset threshold, it is determined that no traffic accident has occurred at the intersection; The on-site verification module is used to start the on-site verification procedure of the traffic police robot when a traffic accident is suspected to have occurred at the intersection; the on-site verification procedure includes: the traffic police robot obtains the on-site image of the intersection through the camera equipped with it, and transmits the on-site image of the intersection to the traffic control center, and the personnel of the traffic control center manually confirm the on-site image of the intersection; The emergency handling module is used to initiate emergency measures when it is determined that a traffic accident occurs at an intersection.

2. The intelligent traffic police robot control system according to claim 1 is characterized in that: With the radar as the origin, a two-dimensional rectangular coordinate system is established, where the x-axis represents the direction directly in front of the radar, the y-axis represents the direction directly to the side of the radar, and the position is the coordinate of the vehicle in the coordinate system; The vehicle states include: acceleration state, driving state, deceleration state and stationary state. The mathematical expressions of the four vehicle states include: ； ； ； ； in, , , and They represent the acceleration state, driving state, deceleration state and stationary state respectively, v and a represent the speed and acceleration of the vehicle respectively, and represents the first preset speed and the second preset speed, , and represent the first preset acceleration, the second preset acceleration and the third preset acceleration respectively, Indicates the absolute value of vehicle acceleration; The signal light status includes: red light, yellow light and green light, and the signal light status is represented by one-hot coding. The weather status includes: sunny day, cloudy day, rainy day, foggy day and snowy day, and the weather status is represented by one-hot coding.

3. The intelligent traffic police robot control system according to claim 1 is characterized in that: The accident preliminary judgment model includes: an image feature extraction unit, a vehicle feature extraction unit, an environment feature extraction unit, a mosaic unit and a first classifier; The image feature extraction unit is used to input first feature data and output image features; The vehicle feature extraction unit is used to input the second feature data and output the vehicle feature; The environmental feature extraction unit is used to splice the signal light status and weather status of the environmental data to obtain the environmental features; The mosaic unit is used for stacking the image features, the vehicle features and the environment features, and mosaicking them to obtain mosaic features; The chimeric features are expanded into chimeric vectors and input into a first classifier, and the classification space of the first classifier represents the probability of a traffic accident occurring at the intersection within a first preset time period.

4. The intelligent traffic police robot control system according to claim 3 is characterized in that: The image feature extraction unit is constructed based on a convolutional neural network and is used to extract image features. The image feature extraction unit is pre-trained before training the accident preliminary judgment model. During the pre-training process, the image features output by the image feature extraction unit are expanded into vectors and then input into the second classifier. The classification space of the second classifier indicates whether the vehicle is deformed, and the classification labels of the second classifier are manually labeled.

5. The intelligent traffic police robot control system according to claim 4 is characterized in that: The vehicle feature extraction unit is built based on the improved gated recurrent unit. The structure of the improved gated recurrent unit includes: update gate, reset gate, candidate hidden state, hidden state update and global memory unit. The improvements include: adding time-sensitive adjustment and multi-head feature fusion in the update gate, introducing a multi-head memory mechanism in the reset gate, introducing an attention mechanism in the candidate hidden state, and fusing multi-head memory, global memory and the current candidate state in the hidden state update stage.

6. The intelligent traffic police robot control system according to claim 3 is characterized in that: The image features, vehicle features and environmental features are stacked to obtain a multimodal feature matrix. The specific process is: the image features are expanded into image feature vectors, and the image feature vectors, vehicle features and environmental features are stacked. The mathematical expression of the multimodal feature matrix is: , represents the multimodal feature matrix, represents the image feature vector, Indicates vehicle characteristics, Represents environmental features; the multimodal feature matrix is ​​embedded to obtain the embedded features, and the embedded calculation formula is: ； in, Indicates chimeric features, , , and They represent query weight parameter, key weight parameter, global weight parameter and value weight parameter respectively, o represents the row index of the multimodal feature matrix, It represents the selection of all columns of the o-th row of the multimodal feature matrix, d represents the number of columns of the multimodal feature matrix, and T represents the transposition operation.

7. A method for controlling an intelligent traffic police robot, characterized in that: The intelligent traffic police robot control system according to any one of claims 1 to 6 comprises the following steps: Step S301, executing a multi-source data collection task within a preset time period, the multi-source data collection task including: collecting image data through a road intersection camera at a preset time interval within the preset time period, collecting vehicle data through a radar device, and collecting environmental data; Step S302, preprocessing the image data and the vehicle data to obtain first feature data and second feature data respectively; Step S303, constructing an accident preliminary judgment model based on the first characteristic data, the second characteristic data and the environmental data, outputting the probability of a traffic accident occurring at the intersection within the first preset time period; and performing a graded judgment on the probability of a traffic accident occurring; Step S304, when a traffic accident is suspected to have occurred at the intersection, the on-site verification program of the traffic police robot is started to manually confirm the on-site image of the intersection; Step S305, when it is determined that a traffic accident occurs at the intersection, emergency measures are initiated.

Images