A traffic scene risk identification method and system based on hybrid data learning

Abstract

The present invention relates to a method and system for identifying risks at a traffic scene based on hybrid data learning. In the present invention, a first image acquisition device and a second image acquisition device that can acquire both visible light images and infrared images are used to acquire road conditions, and the acquired visible light images and infrared images are filtered by Gaussian filtering to obtain filtered images; for the filtered images, a normalized image is obtained by range normalization, and the normalized image is used to extract a risk feature vector containing hybrid data; the risk feature vector is transmitted to a risk learning model for risk prediction. The data processing method and optimized neural network model based on the present application can realize the identification of risks at the traffic scene, which helps to improve the efficiency of traffic management, detect risks early, and protect the safety of people's property; the normalized image is used to extract the risk feature vector, which reduces the dimension of the data, reduces the amount of data transmission, and improves the learning performance of the model.

Description

Technical Field

[0001] The present invention belongs to the field of smart transportation, and in particular, relates to a traffic scene risk identification method and system based on hybrid data learning. Background Art

[0002] With the rapid development of artificial intelligence and big data technologies, smart transportation applications are playing an increasingly important role in urban traffic management. With the rapid development of the social economy and science and technology, the number of motor vehicles has increased rapidly, and urban traffic conditions have become increasingly complex, with traffic congestion and traffic accidents occurring frequently. Smart transportation, based on new technologies such as artificial intelligence and the Internet of Things, monitors traffic conditions in real time to detect anomalies and risks, allowing for timely resolution and improving the transportation system's ability to respond to risks and anomalies.

[0003] Existing methods for monitoring road traffic conditions typically rely on image processing and recognition, but these methods suffer from poor accuracy and lack intelligence. With the development of artificial intelligence, there are also technologies using neural network models for image recognition to assess traffic risks and conditions. However, current surveillance cameras are typically located in the middle of a road section, facing a single direction to monitor a specific length, or at intersections to monitor conditions at that intersection. In other words, the images captured by the cameras are solely for a specific road section, without considering interconnected sections. Furthermore, existing systems typically upload captured images to a server, where neural networks perform recognition and assessment. This places high demands on communication bandwidth. Current communication bandwidths are insufficient for uploading large quantities of high-definition images in real time, forcing the entire system to sacrifice image quantity or quality to ensure uncongested upload channels and a timely and accurate response. Furthermore, common neural network models offer poor accuracy in risk warnings for roads, especially complex ones.

[0004] Therefore, a system is needed that can identify traffic risks in real time, efficiently and accurately. Summary of the Invention

[0005] In order to solve one or more of the above technical problems, as well as the technical problems mentioned in the embodiments, the present invention proposes a traffic scene risk identification method and system based on hybrid data learning.

[0006] In a first aspect, the present invention provides a method for identifying traffic scene risks based on hybrid data learning, comprising:

[0007] Step 1: Collection and preprocessing of mixed image data:

[0008] A first image acquisition device and a second image acquisition device are used to simultaneously acquire visible light images and infrared images of road conditions; the first image acquisition device is positioned within a road section and faces a set direction; the second image acquisition devices are positioned at both ends of the road section, with two second image acquisition devices positioned at each end; the visible light images and infrared images captured by the first and second image acquisition devices have the same resolution;

[0009] For infrared images and visible light images, the corresponding Gaussian functions are used for convolution calculation to obtain the filtered image I′; the Gaussian window size of the Gaussian function corresponding to the visible light image is ∈1, For any visible light image, calculate the number of pixels corresponding to each grayscale in the image, denoted as c1, c2, ..., c 256 ; Take the three gray levels with the largest number of corresponding pixels, and get c in descending order of number i 、c j 、c k ;For infrared images, the Gaussian window size of the corresponding Gaussian function is ∈2, ∈2=15;

[0010] Step 2: Normalization of mixed image data: The filtered image I′ is normalized to obtain a normalized image.

[0011] Step 3: Extraction of on-site risk feature vector based on normalized mixed image data: For normalized image Divide it into S×S size sub-images in space; for the normalized image of the visible light image For each sub-image, find the maximum value of its pixel value, recorded as s max , for the normalized image of the infrared image For each sub-image, find the median value of its pixel value, recorded as s med , assuming that s(i,j) is the value of the pixel with coordinates (i,j) in sub-image s, calculate

[0012]

[0013]

[0014] is the normalized image of the visible light image The first eigenvector corresponding to the subgraph s; is the normalized image of the infrared image The second eigenvector corresponding to the sub-image s; normalized image The first eigenvector or the second eigenvector of all sub-images constitutes the normalized image Identification features of all normalized images The identification features of the site risk feature vector;

[0015] Step 4: Traffic scene risk identification based on scene risk feature vector:

[0016] Expand the on-site risk feature vector obtained in step 3 into a matrix. Each row of the matrix corresponds to the identification features extracted from all sub-images of an image. The number of rows in the matrix is (2D+8), which corresponds to the normalized images corresponding to the four visible light images and four infrared images collected by the four second image acquisition devices at both ends of the road section. The identification features are placed in the last 8 rows of the matrix, and the expanded matrix is recorded as Will As the input data of the neural network input layer; the hidden layer includes five layers; the output layer node y of the neural network z Corresponding to the probability of the existence of the risk at the traffic scene to be identified, when y z = 0, indicating that the risk does not exist. z =1, it means the risk exists.

[0017] Furthermore: In step 4, the dimension of the expanded matrix is 3×N S ; The number of rows in the matrix is (2D+8), corresponding to D+4 visible light images and D+4 infrared images collected each time, N S The number of sub-images contained in each acquired image.

[0018] Furthermore: the Gaussian function corresponding to the visible light image in step 1 is:

[0019]

[0020] The Gaussian function corresponding to the infrared image in step 1 is:

[0021]

[0022] Furthermore: in step 3, S=8.

[0023] Furthermore, the resolutions of the visible light image and the infrared image captured by the first image capturing device and the second image capturing device are consistent.

[0024] In the second aspect, the present invention provides a traffic scene risk identification system, which implements the traffic scene risk identification method based on hybrid data learning, including: a first image acquisition device, a second image acquisition device and a cloud server, wherein the first image acquisition device and the second image acquisition device are connected to the cloud server via a communication network.

[0025] Furthermore, the first image acquisition device and the second image acquisition device are provided with an intelligent processing module for filtering and normalizing the visible light image and infrared image acquired by the first image acquisition device and the second image acquisition device to obtain a normalized image. And use the normalized image Extract the on-site risk feature vector.

[0026] Furthermore: the cloud server is used to send the on-site risk feature vector into the neural network for identification, thereby determining the risk of the road section. When the cloud server determines that a risk occurs, it sends it to the command center for forecast or warning.

[0027] Furthermore: the optical axes of the lenses of the first image acquisition device and the second image acquisition device are directed toward the road and form an angle of 30 degrees with the road, and the field of view angle is not less than 120 degrees.

[0028] Furthermore: there are one or more first image acquisition devices.

[0029] The invention and technical effects of the present invention are as follows:

[0030] 1. Use the first image acquisition device deployed at both ends of the road section where the risk is to be identified and the second image acquisition device on the road section to simultaneously collect visible light images and infrared images. Not only the road conditions of the road section to be identified are collected, but also the related road sections are collected as auxiliary basis for judging the occurrence of risks. This makes the possible risk judgment of the road section to be tested more accurate and improves the performance of risk identification.

[0031] 2. Visible light images and infrared images are used as data sources, and special and different filtering methods are designed for different types of images, which helps to improve the method's ability to resist interference from external environmental influences and make recognition more accurate.

[0032] 3. By extracting on-site risk feature vectors in the intelligent processing module, the transmission of images to the cloud server is avoided, reducing the communication burden while ensuring the accuracy of recognition.

[0033] 4. Through the design of a special neural network structure, it is more suitable for processing on-site risk feature vectors derived from visible light images and infrared images, and can cooperate with image data filtering, value range normalization and on-site risk feature vector extraction methods to achieve risk feature-oriented learning and recognition. Compared with classical methods, it can effectively identify traffic scene risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention. In the drawings:

[0035] Figure 1A flow chart of a traffic scene risk identification method based on hybrid data learning provided by an embodiment of the present invention;

[0036] Figure 2 A schematic diagram of a traffic scene risk identification system provided by an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of the arrangement of a first image acquisition device and a second image acquisition device on a road section in a traffic scene risk identification system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0039] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0040] Example 1

[0041] See Figure 1 As shown, the embodiment of the present application provides a traffic scene risk identification method based on hybrid data learning, including:

[0042] Step 1: Acquisition and filtering of hybrid image data.

[0043] Visible light images in the visible light band and infrared images in the infrared band are collected by a first image acquisition device and a second image acquisition device on the road section to be identified, and the resolution of the visible light images and the infrared images collected by the first image acquisition device and the second image acquisition device are the same; and filtering is performed using the intelligent processing modules of the first image acquisition device and the second image acquisition device.

[0044] Assume that the four second image acquisition devices at both ends of the road section are numbered a, b, c, and d, and the first image acquisition devices in the middle of the road section are numbered 1, 2, ..., D in the direction from a, b to c, d. Assume that the visible light image of each device is marked as α and the infrared image is marked as β. The images collected by each device are recorded as

[0045] For any of the aforementioned visible light images, the following method is used for filtering.

[0046] The visible light image pixel brightness value is discretized and divided into T grayscales, that is, discrete values of pixel brightness, usually T=256.

[0047] For any visible light image, calculate the number of pixels corresponding to each gray level in the visible light image, and record them as c1, c2, ..., c 256 . Take the three gray levels with the largest number of corresponding pixels and record them as c in descending order. i 、c j 、c k .

[0048] calculate

[0049]

[0050] Calculation results Equal to c i 、c j 、c k The maximum absolute value of the three differences is used, with max representing the maximum value. The three largest grayscales are selected and the maximum value is taken to better fit the variance of the image pixel grayscale (grayscale value). This can reduce the impact of noise compared to directly taking the two largest grayscales.

[0051] The Gaussian function corresponding to the visible light image is:

[0052] in, It is obtained by calculation from formula (1), where G represents the Gaussian window in two-dimensional space, i, j represent the spatial relative coordinates in each dimension of the Gaussian window, and exp represents the exponential function. The spatial size of the Gaussian window is ∈1.

[0053] Visible light image I α (u,v) is convolved with the corresponding Gaussian function to obtain the filtered image I′(u,v):

[0054]

[0055] Where G(i,j) represents the Gaussian function obtained by equation (2), and the symbol Represents the convolution operation.

[0056] In the above steps, by calculating the statistical value of the number of pixels in the visible light image, a corresponding Gaussian function is determined as a filter; and by applying this filter to the original collected visible light image, a filtered image is obtained. This effectively removes noise from the visible light image and improves the risk identification performance of the present invention.

[0057] For all infrared images, the following method is used for filtering.

[0058] The Gaussian function corresponding to the infrared image is:

[0059]

[0060] As a preferred option, take ∈2=15

[0061] Infrared Image I β The filtered image I′(u,v) of (u,v):

[0062]

[0063] In visible light images, the distribution of grayscale values (pixel values) is relatively dispersed, so using pixel statistics to estimate the Gaussian window size is effective. Infrared images typically have a more concentrated pixel distribution, making statistical methods more susceptible to noise. Therefore, using an empirical value of 15 can achieve good results. By filtering visible light and infrared images separately, we can better utilize the statistical characteristics of each type of image, specifically improve the quality of the collected data, and achieve better overall recognition results.

[0064] Step 2: Normalization of mixed image data.

[0065] According to the filtered image I′ outputted in step 1, a normalization operation is performed on it using an intelligent processing module.

[0066] For the input filtered image I′, the pixel value at the coordinate is I′(u,v), let:

[0067]

[0068]

[0069] Where size(I′) represents the number of pixels in the filtered image I′, ∑ u,v I′(u,v) represents the sum of all pixel values in the filtered image I′, μ represents the average value of all pixel values in the filtered image I′, ∑ u,v (I′(u,v)-μ) 2It represents the sum of the squares of the differences between all pixel values in the filtered image I′ and the average value μ.

[0070] Furthermore, the normalized image of the filtered image I′ is calculated using the following formula:

[0071]

[0072] in, is the pixel at (u, v) in the normalized image.

[0073] The original visible light image or infrared image is filtered to obtain the filtered image I′, which is then further normalized to obtain the normalized image Its purpose is to unify the values of all image pixels so that each image pixel is statistically consistent. The above normalization can normalize visible light and infrared images at the same time and maintain the statistical characteristics of both. It is simpler than applying the classical normalization method separately and does not affect the performance of subsequent steps.

[0074] Step 3: Traffic scene risk feature extraction based on normalized mixed image data.

[0075] Using the obtained normalized image Perform on-site risk feature extraction; the risk feature extraction refers to establishing a feature vector with high responsiveness to specific traffic scene risks based on the input mixed image data at the acquisition device end, which is used for risk identification in subsequent steps.

[0076] According to the arrangement method of the image acquisition devices described in step 1, each of the D first image acquisition devices and the four second image acquisition devices is equipped with a visible light imaging element and an infrared imaging element. At a certain moment, one visible light image and one infrared image can be acquired simultaneously. Accordingly, at that moment, a total of D+4 visible light images corresponding to the road section to be identified are acquired. and D+4 infrared images According to the methods of step 1 and step 2, a filtered image I′ of the visible light image and the infrared image is obtained, and further a range-normalized image corresponding to the filtered image I′ is obtained.

[0077] For the normalized D+4 visible light images corresponding to the normalized images The following method is used to extract features.

[0078] For any normalized image corresponding to a visible light image, divide it into S×S sub-images in space, where S≥4. As a preferred method, S=8. For each sub-image, find the maximum value s of its pixel value. max,calculate

[0079]

[0080]

[0081] Among them, s(i,j) is the value of the pixel with coordinates (i,j) in sub-image s, ∑ i,j (f) means traversing the values of function f corresponding to all pixels in the sub-image and summing the values of function f.

[0082] Get the first eigenvector of the sub-image of the normalized image corresponding to the visible light image Normalized image corresponding to the visible light image The first eigenvectors of all its subgraphs constitute its identification features.

[0083] For the normalized D+4 infrared images corresponding to the normalized image The following method is used to extract features.

[0084] For any normalized image corresponding to an infrared image, spatially divide it into S×S sub-images, where S≥4, and the segmentation size is the same as that set for the normalized image corresponding to the visible light image; as a preferred method, S=8. For each sub-image of the normalized image corresponding to the infrared image, calculate the median value s of its pixel values. med ,calculate

[0085]

[0086]

[0087] Among them, s(i,j) is the value of the pixel with coordinates (i,j) in sub-image s, ∑ i,j (f) means traversing the values of function f corresponding to all pixels in the sub-image and summing the values of function f.

[0088] Get the second eigenvector of the sub-image of the normalized image corresponding to the infrared image For the normalized image corresponding to the infrared image The second eigenvectors of all its subgraphs constitute its identification features.

[0089] Since visible light and infrared images have the same spatial dimensions, the dimensions of their normalized recognition features are also the same. Separately extracting the recognition features of different image types and employing different extraction methods can better extract scene-related information from both visible light and infrared images, improving recognition performance.

[0090] The identification features of all normalized images constitute the scene risk feature vector.

[0091] By reducing the dimensionality of high-dimensional visible light and infrared images to generate identification features, and then forming a site risk feature vector based on the identification features, inputting the site risk feature vector into the neural network can greatly reduce the amount of data input to the neural network in subsequent steps and improve computational efficiency. Completing the feature vector creation on the acquisition device side and transmitting only low-dimensional vectors during communication can reduce the communication load and improve transmission efficiency.

[0092] Step 4: Traffic scene risk identification based on scene risk feature vector.

[0093] According to the on-site risk feature vector obtained in step 3, a risk learning and identification model is established to realize traffic scene risk identification based on the on-site risk feature vector.

[0094] A neural network-based risk identification model is established on the cloud server. The neural network consists of an input layer, a hidden layer, and an output layer. The input layer inputs the on-site risk feature vector obtained in step 3. The output layer represents the probability of the risk to be identified. Each output dimension corresponds to a certain risk, with a value of 0 indicating the absence of risk and a value of 1 indicating the presence of risk. The hidden layer represents the mapping from the input layer to the output layer, consisting of several connections represented by weights. The connection structure between the hidden layer, input layer, and output layer is determined during neural network establishment, but the connection weights are not. Therefore, training with sample data is required to determine the connection weights between the hidden layer, input layer, and output layer. The hidden layers include the first hidden layer, the second hidden layer, the third hidden layer, the fourth hidden layer, and the fifth hidden layer.

[0095] The input layer expands the on-site risk feature vector into a matrix, where each row corresponds to a normalized image The recognition feature of S , N S For each normalized image The number of sub-images included; the number of rows in the matrix is (2D+8), corresponding to D+4 visible light images and D+4 infrared images collected each time; and the normalized images corresponding to the four visible light images and four infrared images taken by the four second image acquisition devices at both ends of the road section The identification features are placed in the last eight rows of the matrix. The input layer is recorded as

[0096] The connections of the first hidden layer of the neural network are defined as follows:

[0097]

[0098] Where, Represents a one-dimensional convolution window centered at (u,v) in the u direction, represents a one-dimensional convolution window centered at (u, v) in the v direction, p and q represent the coordinates of the relative positions in the two convolution windows, respectively. The window size is 13, The window size is 5. The values of the input layer matrix at coordinates (p, v) and (u, q) are respectively. b0 is the linear bias. σ(x) is a nonlinear activation function:

[0099]

[0100] arctan(x) represents the inverse tangent trigonometric function. The parameter δ creates a discontinuity at x = 0. This improves recognition performance compared to directly applying the inverse tangent trigonometric function. max represents the larger of the two, controlling the excitation function's range within the permitted sample values. δ = 0.003 is preferred.

[0101] Since the physical meanings of the rows and columns of the input layer are relatively independent, the connection between the first hidden layer and the rows and columns in the input layer is defined as two independent dimensions, which greatly reduces the connection complexity of the hidden layer and improves the computational efficiency of the model. The calculation method of the third hidden layer and the fifth hidden layer in the following text is similar. In the first hidden layer of the hidden layer, the parameters determined by training are weights. and a linear bias b0. The first hidden layer of the hidden layer calculates the first 2D rows and the last 8 rows of the scene risk feature vector separately, so that the parameters of the first hidden layer can fit the difference between the images captured by the first image acquisition device and the second image acquisition device.

[0102] The connections of the second hidden layer are defined as follows:

[0103]

[0104] Where, Indicates the maximum value of all elements in the window (p,q) with a reference of (2u,2v) in the first hidden layer of the hidden layer, where p and q are {0,1}, and the size of the second hidden layer of the hidden layer is half of the first hidden layer; represents the node with coordinates (2u+p, 2v+q) in the first hidden layer. b1 is the linear bias. σ(x) is defined by Equation (10).

[0105] The connections of the third hidden layer are defined as follows:

[0106]

[0107] Where, Represents a one-dimensional convolution window centered at (u,v) in the u direction, represents a one-dimensional convolution window centered at (u, v) in the v direction, p and q represent the coordinates of the relative positions in the two convolution windows, respectively. The window size is 11, The window size is 5. The values of the matrix of the second hidden layer at coordinates (p, v) and (u, q) are respectively. b2 is the linear bias. In the third hidden layer of the hidden layer, the parameter to be determined is the weight and the linear bias b2. σ(x) is defined by equation (10).

[0108] The connections of the fourth hidden layer are defined as follows:

[0109]

[0110] Where, It represents the maximum value of all elements in the window (p,q) with a reference of (2u,2v) in the third hidden layer of the hidden layer, where p and q are {0,1}. It can be seen that the size of the fourth hidden layer of the hidden layer is half of the third hidden layer; represents the node with coordinates (2u+p, 2v+q) in the third hidden layer. b3 is the linear bias. σ(x) is defined by Equation (10).

[0111] The connections of the fifth hidden layer are defined as follows:

[0112]

[0113] Where, Represents a one-dimensional convolution window centered at (u,v) in the u direction, represents a one-dimensional convolution window centered at (u, v) in the v direction, p and q represent the coordinates of the relative positions in the two convolution windows, respectively. The window size is 9, The window size is 3. The values of the matrix of the fourth hidden layer at coordinates (p, v) and (u, q) are respectively. b4 is the linear bias. In the fifth hidden layer of the hidden layer, the parameter to be determined is the weight and the linear bias b4. σ(x) is defined by equation (10).

[0114] The node connection method of the output layer is defined as:

[0115]

[0116] Where, For a node in the fifth hidden layer, p and q define the position of the node. Representation node and the output layer node y z The connection weights between all y z The vector composed of the sum of is the output layer, each dimension y z represents the probability of a risk existing, and its value range is [0,1]. b5 is the linear bias. σ(x) is defined by equation (10).

[0117] The output layer node y of the neural network z Corresponding to the probability of existence of a certain traffic scene risk to be identified, when y z = 0, indicating that the risk does not exist. z =1, indicating that the risk exists. Each dimension can be set to represent a specific risk, such as traffic jams, driving accidents, or crowds. The number of nodes in the output layer is equal to the number of independent risks to be identified.

[0118] The neural network model training phase involves collecting visible light and infrared image data and filtering and normalizing them according to the methods described in steps 1 and 2. Then, according to the method in step 3, a site risk feature vector is constructed as the neural network input. The corresponding risk value (1 or 0) is annotated to indicate whether the input is present. A set of image data is collected at each moment, and after processing, it corresponds to a set of samples. After collecting samples, the neural network is trained based on the sample data to produce a risk identification model.

[0119] During training, the neural network is used to calculate the output of a given input sample Determine the cost function with the true value y of the manual annotation:

[0120]

[0121] Among them, y represents the true value of manual annotation, Indicates that the neural network calculates the output of a given input sample, y z 、 They are vector y, 0<λ<0.01 is a control coefficient that helps improve the model's robustness to noise. Optimally, λ=0.007 is used.

[0122] The back propagation method is used to find the values of the parameters of the neural network to be determined when the cost function is an extreme value, and the training of the neural network is completed.

[0123] When using the risk identification model obtained through neural network training to assess traffic scene risks, according to the methods described in steps 1 and 2, control the first image acquisition device and the second image acquisition device to simultaneously acquire the required visible light and infrared image data, and perform filtering and normalization. According to the method of step 3, establish a scene risk feature vector and input it into the risk identification model; the risk identification model will output an estimated value of the probability of risk occurrence. The closer the output value is to 1, the greater the possibility of the existence of this type of risk.

[0124]

[0125] The present invention is applied to the automatic identification of various types of traffic risks in regular urban road environments under daily circumstances. It has a high degree of intelligence and helps to improve the efficiency of traffic management, detect risks early, and protect the safety of people and property. On the one hand, the present invention uses a first image acquisition device and a second image acquisition device to synchronously capture visible light images and infrared images of the road, obtaining more information at the same time, and at the same time being able to fully utilize the features contained in images of different bands, thereby improving the accuracy of recognition. By filtering and normalizing the image, a unique on-site risk feature vector is established, which reduces the dimension of the data and improves the learning performance of the model; at the same time, an optimized neural network structure is established, so compared with the classic deep convolutional network model, the same risk recognition performance is achieved with relatively fewer network layers and higher computational efficiency. As shown in the table above, the present invention has better real-time performance and higher accuracy than the classic recognition method using a convolutional neural network model.

[0126] Example 2

[0127] See also Figure 2 and Figure 3 As shown, an embodiment of the present invention provides a traffic scene risk identification system, including a first image acquisition device and a second image acquisition device set at a road section to be identified, the first image acquisition device and the second image acquisition device are connected to a cloud server via a communication network; the first image acquisition device and the second image acquisition device are provided with an intelligent processing module for filtering and normalizing the visible light image and infrared image acquired by the first image acquisition device and the second image acquisition device to obtain a normalized image And use the normalized image Extracting on-site risk feature vectors. Each first and second image acquisition device is connected to a cloud server via a wireless or wired communication network. The cloud server is configured with a neural network-based risk identification model. When the cloud server identifies a road section with a risk, it issues a forecast or warning to the command center.

[0128] Specifically, the first image capture device is positioned within the road section, with the optical axis of the lens of the first image capture device facing the road at a 30-degree angle. At least one first image capture device is provided within the road section, and the number of first image capture devices is specifically configured based on the actual length of the road section.

[0129] The second image acquisition device is located at either end of the road section, typically at an intersection. The optical axis of the lens of the first image acquisition device faces the road at a 30-degree angle. The second image acquisition device can capture visible light and infrared images of a larger area associated with the road section.

[0130] During specific implementation, in an urban street environment, the installation height of the first image acquisition device and the second image acquisition device shall not be less than 3 meters, and the feasible installation height range verified on site is 3-8 meters. During specific implementation. As a preferred option, the installation height of the first image acquisition device is 4-6 meters from the road surface, and the weather environment is daytime and clear, and the interval between the first image acquisition devices is 280-320 meters. Under the conditions of lowering the installation height and poor visibility caused by the weather environment, the distance between adjacent first image acquisition devices should be reduced accordingly, so that the image data collected by the first image acquisition device can fully contain the amount of information required for analysis.

[0131] During the specific implementation process, when the risk identification model of the cloud server determines that a risk occurs on a road section, it notifies the traffic command center to make a forecast or warning.

[0132] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0134] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0135] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is intended to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A traffic scene risk identification method based on hybrid data learning, characterized in that: include: Step 1: Acquisition and filtering of mixed image data: A first image acquisition device and a second image acquisition device are used to simultaneously acquire visible light images and infrared images of road conditions; the first image acquisition device is positioned within a road section and faces a set direction; the second image acquisition devices are positioned at both ends of the road section, with two second image acquisition devices positioned at each end; the visible light images and infrared images captured by the first and second image acquisition devices have the same resolution; For infrared images and visible light images, the corresponding Gaussian functions are used for convolution calculation to obtain the filtered images. ; The Gaussian window size of the Gaussian function corresponding to the visible light image is , , for any visible light image, calculate the number of pixels corresponding to each grayscale in the image, recorded as 、 、…、 ; Take the three gray levels with the largest number of corresponding pixels, and get them in descending order 、 、 ; For infrared images, the Gaussian window size of the corresponding Gaussian function is ; Step 2: Normalization of mixed image data: Filtering image After range normalization, the normalized image is obtained ; Step 3: Extraction of on-site risk feature vector based on normalized mixed image data: For normalized image , which is divided into Sub-images of different sizes; normalized images for visible light images For each sub-image of , find the maximum value of its pixel value, recorded as , for the normalized image of the infrared image For each sub-image of , find the median value of its pixel value, recorded as , The coordinates in subgraph s are The pixel value is calculated is the normalized image of the visible light image The first eigenvector corresponding to the subgraph s; is the normalized image of the infrared image The second eigenvector corresponding to the sub-image s; normalized image The first eigenvector or the second eigenvector of all sub-images constitutes the normalized image Identification features of all normalized images The identification features of the site risk feature vector; Step 4: Traffic scene risk identification based on scene risk feature vector: Expand the on-site risk feature vector obtained in step 3 into a matrix. Each row of the matrix corresponds to the identification features extracted from all sub-images of an image. The number of rows in the matrix is , D first image acquisition devices collect D infrared images and D visible light images; corresponding to the normalized images of 4 visible light images and 4 infrared images collected by the four second image acquisition devices at both ends of the road section The identification features are placed in the last 8 rows of the matrix, and the expanded matrix is recorded as ;Will As the input data of the neural network input layer; the hidden layer includes five layers; the output layer nodes of the neural network Corresponding to the probability of the existence of the risk at the traffic scene to be identified, when When , it means that the risk does not exist. , it indicates that the risk exists.

2. The method according to claim 1, wherein: In step 4, the dimension of the expanded matrix is ; The number of rows in the matrix is , corresponding to D+4 visible light images and D+4 infrared images collected each time, The number of sub-images contained in each acquired image.

3. The method according to claim 1, wherein: The Gaussian function corresponding to the visible light image in step 1 is: ； G represents a Gaussian window in two-dimensional space, i and j represent the relative spatial coordinates in each dimension of the Gaussian window, and exp represents the exponential function; The Gaussian function corresponding to the infrared image in step 1 is: 。 4. The method according to claim 1, wherein: In step 3, S=8.

5. The method according to claim 1, wherein: The resolutions of the visible light image and the infrared image captured by the first image capture device and the second image capture device are consistent.

6. A traffic scene risk identification system, implementing the traffic scene risk identification method based on hybrid data learning according to any one of claims 1 to 5, characterized in that: include: A first image acquisition device, a second image acquisition device and a cloud server, wherein the first image acquisition device and the second image acquisition device are connected to the cloud server via a communication network.

7. The system according to claim 6, wherein: The first image acquisition device and the second image acquisition device are provided with an intelligent processing module for filtering and normalizing the visible light image and infrared image acquired by the first image acquisition device and the second image acquisition device to obtain a normalized image. , and use the normalized image Extract the on-site risk feature vector.

8. The system according to claim 6, wherein: The cloud server is used to send the on-site risk feature vector into the neural network for identification, thereby determining the risk of the road section. When the cloud server determines that a risk occurs, it sends it to the command center for forecast or warning.

9. The system according to claim 6, wherein: The optical axes of the lenses of the first image acquisition device and the second image acquisition device are oriented toward the road and form an angle of 30 degrees with the road, and the field of view angle is not less than 120 degrees.

10. The system according to claim 6, wherein: There are one or more first image acquisition devices.

Images