Traffic state recognition method, computer device and storage medium

Abstract

The application discloses a traffic state recognition method, a computer device and a storage medium. The method comprises the following steps: acquiring a vehicle image set of a preset area, the vehicle image set comprising at least two frames of vehicle images, and the preset area comprising at least one lane; acquiring a connection graph of a target vehicle in each frame of vehicle image, wherein the connection graph comprises at least one of a connection relationship of the target vehicle in the same lane and a connection relationship of the target vehicle in different lanes; and obtaining a traffic state classification to which the preset area belongs by using the target vehicle in each frame of vehicle image and the connection graph. The above scheme can improve the applicability of traffic state recognition.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and in particular to a method for identifying traffic conditions, a computer device, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of the transportation sector, it is necessary to understand the current road traffic conditions in order to better manage traffic and dispatch emergency resources, so as to help managers make decisions.

[0003] Currently, it is typically necessary to deploy a large number of electronic surveillance devices to capture images, and then use the captured information to analyze traffic conditions such as normal traffic flow, slow traffic, traffic accidents, and traffic congestion. This method requires the joint analysis of multiple electronic surveillance devices to comprehensively identify traffic conditions, resulting in low applicability to traffic condition identification. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a traffic condition identification method, computer equipment, and storage medium that can improve the applicability of traffic condition identification.

[0005] To address the aforementioned issues, the first aspect of this application provides a method for identifying traffic conditions. The method includes: acquiring a set of vehicle images of a preset area, the set of vehicle images containing at least two frames of vehicle images, and the preset area containing at least one divided lane; acquiring a connection graph of target vehicles in each frame of vehicle images, wherein the connection graph contains at least one of the connection relationships between target vehicles in the same lane and the connection relationships between target vehicles in different lanes; and using the target vehicles and the connection graph of each frame of vehicle images to obtain a traffic condition classification for the preset area.

[0006] To address the aforementioned problems, a second aspect of this application provides a computer device comprising a memory and a processor coupled to each other, the memory storing program data and the processor executing the program data to implement any step of the aforementioned traffic state identification method.

[0007] To address the aforementioned problems, a third aspect of this application provides a computer-readable storage medium storing program data executable by a processor, the program data being used to implement any step of the traffic state identification method described above.

[0008] The above scheme acquires a set of vehicle images of a preset area, which contains at least two frames of vehicle images. It then obtains a connection map of the target vehicles in each frame of the vehicle image. The preset area contains at least one lane, and the connection map contains at least one of the connection relationships between target vehicles in the same lane and the connection relationships between target vehicles in different lanes. Using the target vehicles and connection map of each frame of the vehicle image, the traffic state classification of the preset area is obtained. This scheme can analyze the traffic state of a preset area using vehicle images collected from a single preset area, without the need to construct a complex road network, thus improving the applicability and efficiency of traffic state recognition. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in this application, the accompanying drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Among them:

[0010] Figure 1 This is a flowchart illustrating the first embodiment of the traffic status identification method of this application;

[0011] Figure 2 This application Figure 1 A flowchart illustrating an embodiment of step S12;

[0012] Figure 3 This is an example diagram of a previous embodiment of the target frame image sequence number mapping in this application;

[0013] Figure 4 This is an example illustration of an embodiment of the target frame image sequence number mapping in this application;

[0014] Figure 5 This is the link corresponding to the target frame image in this application. Figure 1 Example diagram of the embodiment;

[0015] Figure 6 This application Figure 1 A flowchart illustrating an embodiment of step S13;

[0016] Figure 7 This is a link of three vehicle images from this application. Figure 1 Example diagram of the embodiment;

[0017] Figure 8 This is a schematic diagram of the structure of an embodiment of the spatiotemporal graph convolutional network of this application;

[0018] Figure 9 This is a flowchart illustrating the second embodiment of the traffic status identification method of this application;

[0019] Figure 10 This is a schematic diagram of the structure of an embodiment of the traffic status identification device of this application;

[0020] Figure 11 This is a schematic diagram of the structure of an embodiment of the computer device of this application;

[0021] Figure 12 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] The terms "first" and "second" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0024] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0025] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0026] This application provides the following embodiments, and each embodiment is described in detail below.

[0027] Please see Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the traffic status identification method of this application. The method may include the following steps:

[0028] S11: Obtain a set of vehicle images of a preset area, the set of vehicle images containing at least two frames of vehicle images, and the preset area containing at least one lane.

[0029] The preset area can be an area where traffic status recognition is required, such as a lane, road, intersection, or junction. This application does not impose any restrictions on this.

[0030] Electronic capture devices, such as video cameras or cameras, can be deployed in a preset area to capture images of vehicles in the preset area, obtain vehicle videos or multiple frames of vehicle images, and obtain a set of vehicle images of the preset area based on the vehicle videos or multiple frames of vehicle images, with at least two frames of vehicle images in the set of vehicle images.

[0031] In some implementations, vehicle video can be acquired in real time. The vehicle video is decomposed into a set of vehicle image frames I = {I1, I2, I3, ..., I...} t}, where t represents the number of vehicle images in the vehicle image set, and t is a positive integer greater than or equal to 2.

[0032] In some implementations, after acquiring a set of vehicle images of a preset area, target detection and tracking can be performed on each frame of the vehicle image to detect information such as the position of the vehicle in each vehicle image.

[0033] For single-frame vehicle images, convolutional neural networks (CNNs) can be used for detection and tracking. Examples of CNNs include YOLO, Faster R-CNN, and DETR. These CNNs can detect bounding boxes of target vehicles (such as motor vehicles) and output a unique identifier (ID) for each target vehicle. In some applications, after detecting the bounding boxes of target vehicles, tracking algorithms such as Kalman filtering can be used to track the target and output the ID of each target vehicle.

[0034] In some implementations, the preset area may include at least one divided lane, where one lane may correspond to a line of target vehicles (e.g., multiple target vehicles parallel to the direction of travel of the lane), or one lane may correspond to one or more lane areas. The preset area may be divided into K lanes, where K is a positive integer.

[0035] As an example, a preset area in a vehicle image contains multiple columns of target vehicles. The multiple target vehicles in each column can be arranged in a straight line or a curve. Lane lines can be determined according to the positions of the multiple columns of target vehicles. That is, lane lines can be straight lines or curves, so that the area where each column of target vehicles or multiple columns (more than one column) of target vehicles are located can be divided into lanes according to the lane lines.

[0036] As an example, a preset area in a vehicle image contains multiple lane areas, each of which can be distinguished by the markings of the preset area. Lane lines can be determined according to the lane areas to divide the preset area into lanes.

[0037] As an example, a preset area can be divided into a lane. For instance, in a one-way traffic lane where only one target vehicle can pass through the preset area, the preset area can be divided into a lane.

[0038] In some implementations, the position or initial key point of each target vehicle in a preset area can be obtained first. The center point of the detection frame of the target vehicle can be used as the initial key point. The lane lines can be divided using the initial key point so that the multiple target vehicles contained in each column can be individually assigned to each lane.

[0039] In some implementations, deep learning can be used for adaptive extraction to determine lanes within a preset area. For example, a semantic segmentation algorithm can be used to separate lane lines first, followed by polynomial fitting to fit the lane lines, which are then used to divide the lanes.

[0040] In some implementations, a vehicle image can be determined from a set of vehicle images as the target frame image, and lanes can be divided into lanes in a preset area of ​​the target frame image. Reference frame images after the target frame image can then be divided into the same lanes as the target frame image.

[0041] The lanes defined in the preset area can be used to analyze the changes in target vehicles in the sub-areas of the preset area. This application does not limit the method of dividing lanes in the preset area or the size of the area.

[0042] S12: Obtain the connection graph of the target vehicles in each frame of vehicle image, wherein the connection graph contains at least one of the connection relationships of target vehicles in the same lane and the connection relationships of target vehicles in different lanes.

[0043] After performing target detection on each frame of the vehicle image, the detection bounding boxes and corresponding IDs of each target vehicle in each frame are obtained. Based on the positions of the detection bounding boxes of each target vehicle, a connection graph of the target vehicles in each frame can be obtained. This can be achieved by connecting target vehicles in each lane, or by obtaining a pre-defined topology graph of connection relationships, to obtain the connection graph of the target vehicles in each frame. The connection graph includes at least one of the connection relationships between target vehicles in the same lane and the connection relationships between target vehicles in different lanes.

[0044] In some embodiments, please refer to Figure 2 This embodiment can further extend step S12 of the above embodiment. Obtaining the connection map of the target vehicles in each frame of the vehicle image can include the following steps:

[0045] S121: Obtain the target key points corresponding to each frame of vehicle image, where the target key points correspond to each target vehicle.

[0046] After performing target detection on each frame of the vehicle image to obtain the detection bounding box and corresponding ID of each target vehicle, the initial keypoints of each frame can be determined using the positions of the detection bounding boxes of each target vehicle in the vehicle image. For example, the center point of each detection bounding box can be used as the initial keypoint of each target vehicle, and the initial keypoints can represent each target vehicle.

[0047] Then, among the initial keypoints of the vehicle image in each frame, a first number M target keypoints corresponding to each frame of the vehicle image are determined; where M is a positive integer, and the target keypoints include the initial keypoints of the target vehicles in at least one lane. Target keypoints can be uniformly selected from the initial keypoints of the target vehicles in each lane; this application does not restrict the selection method.

[0048] In the case where the preset area contains multiple lanes, each lane contains at least one target vehicle, that is, each lane can contain at least one initial keypoint.

[0049] For each lane, a second number N of target key points are determined from the initial key points of the vehicle image according to a preset selection rule. The second number N is less than the first number M, and N is a positive integer. The preset selection rule can be uniform selection according to lanes, interval selection, etc., and this application does not impose any restrictions on this. For example, the second number can be expressed as N = M / K. When selecting target key points, multiple frames of vehicle images need to be considered, such as T frames of vehicle images, where T is a positive integer greater than 1. All target key points exist in the vehicle images of a third number of frames. Based on the second number of target key points for each lane, a first number of target key points corresponding to each vehicle image can be obtained.

[0050] In some implementations, common initial keypoints for all vehicle images can be determined first. For example, a vehicle image is selected as the target frame image from the set of vehicle images, and vehicle images of a predetermined number of frames following the target frame image are used as reference frame images. Common initial keypoints for all lanes are determined from the target frame image and the reference frame images; these are the initial keypoints belonging to the same target vehicle ID. Then, for each lane in the target frame image, a second number N target keypoints are determined from the common initial keypoints identified in the vehicle images according to a predetermined selection rule, so that all target keypoints exist in a third number of vehicle images. This step of filtering initial keypoints can yield more representative keypoints, thereby improving the accuracy of the subsequently acquired connectivity maps.

[0051] Using the above method, if M target key points are selected from each frame of vehicle image, a total of T×M target key points of the target vehicle can be obtained.

[0052] In some implementations, the detection bounding box of the target vehicle can be detected frame by frame to determine initial key points, and then the target key points of each frame of the vehicle image can be determined sequentially. For example, a first number of M target key points of the target frame image can be determined according to a preset selection rule. This process can select M target key points from the initial key points located in the middle region of a preset area, minimizing the selection of target key points at the edge positions to increase the probability that the target key points still exist in subsequent reference frame images. The M target key points are then selected as target key points in subsequent reference frame images. This application does not limit the method of selecting target key points.

[0053] S122: Determine a vehicle image as the target frame image, map the target key points corresponding to the target frame image, and obtain the connection graph corresponding to the target frame image.

[0054] A vehicle image is selected as the target frame image. The target frame image can be the first vehicle image in a set of vehicle images, or it can be a first vehicle image that meets preset conditions, such as clarity conditions, vehicle quantity conditions, and completeness conditions, to select the target frame image as a benchmark. Vehicle images in a preset number of frames after the target frame image are used as reference frame images. Alternatively, the target frame image can be a vehicle image in the current frame, in which case vehicle images acquired in subsequent frames can be used as reference frame images. This application does not restrict the selection of the target frame image and the reference frame image.

[0055] The target key points corresponding to the target frame image are mapped. These key points can also be represented by the IDs of the target vehicles. This process can remap the identifier sequence number to which the target vehicle ID belongs. Based on the connection relationships of the mapped key points, the connection graph corresponding to the target frame image is determined.

[0056] The initial identifier number of each target key point corresponding to the target frame image can be obtained. The target key point can represent the target vehicle, and the initial identifier number can identify each target vehicle. The initial identifier number of each target key point is mapped. This process can be carried out by renumbering according to the position of each initial identifier number. For example, renumbering according to the lane from left to right and from top to bottom can obtain the current identifier number corresponding to each target key point.

[0057] Please see Figures 3 to 4 For example, a preset area contains three lanes, and each lane has three target key points corresponding to target vehicles. The initial identifier numbers of the three lanes in the target frame image are 9-1-7, 6-2-3, and 4-5-8. By mapping the initial identifier numbers of each lane in order from left to right and from top to bottom, the current identifier numbers corresponding to each target key point in the target frame image can be obtained. That is, the current identifier numbers of the three lanes in the target frame image are 1-2-3, 4-5-6, and 7-8-9.

[0058] In subsequent reference frame images, the current identifier sequence number corresponding to each target vehicle (or target key point) can continue the current identifier sequence number of the target frame image. Alternatively, it can be understood that the target key points in the reference frame image are mapped to the same current identifier sequence number as the corresponding target key points in the target frame image. For example, if the current identifier sequence number corresponding to the target key point of target vehicle 1 in the target frame image is 1, then the current identifier sequence number of target vehicle 1 in the reference frame image is also 1.

[0059] Therefore, based on the current identifier number corresponding to each target keypoint, the connection relationships between each target keypoint can be obtained, thus determining the connection graph corresponding to the target frame image. The connection relationships in the connection graph include at least one of the following: a first connection relationship between target keypoints corresponding to target vehicles in the same lane, and a second connection relationship between target keypoints corresponding to target vehicles in different lanes. The first connection relationship can be an adjacency relationship, where adjacent target keypoints are connected. The second connection relationship can be an adjacency relationship between preset keypoints.

[0060] In some implementations, where the preset area includes multiple lanes, a first connection relationship of target key points in the same lane can be determined, and / or a second connection relationship of target key points in different lanes can be determined.

[0061] For the same lane: The first connection relationship of the target key points within the lane is determined by sorting them according to their preset positions based on the current identifier numbers. This first connection relationship includes sequential connections. The preset position sorting can be any sorting of the current target key points according to the target vehicle's driving direction, the distance between the target vehicle and the data collection location, etc. The target key points within the lane are then connected sequentially according to the pre-sorted adjacency relationship to determine the first connection relationship.

[0062] For different lanes: Preset key points are obtained from each target key point in each lane, and the preset key points of adjacent lanes are connected to determine a second connection relationship. The preset key points can be the first target key point in each lane ordered according to a preset position. Alternatively, each target key point can be determined as a preset key point to connect the target key points of different lanes. This application does not impose any limitations on this.

[0063] Please see Figure 5 As an example, the current identifier numbers corresponding to the three lanes in the target frame image are 1-2-3, 4-5-6, and 7-8-9, respectively. Based on the current identifier numbers of each target keypoint, the connection relationships between the target keypoints are obtained. These connection relationships can be represented as: 1→2→3, 4→5→6, 7→8→9, 1→4→7, to obtain the connection diagram corresponding to the target frame image.

[0064] In some implementations, the connection graph can be a directed connection graph or an undirected connection graph. After the connection graph of the target frame image is determined, the connection method of subsequent reference frame images is consistent with that of the target frame image.

[0065] In some implementations, during the construction of the connectivity graph of the target key points, this connectivity graph can be represented by an adjacency matrix A, which serves as the input to a subsequent pre-defined convolutional model. The adjacency matrix A indicates whether two target key points are connected, with 1 indicating connectivity and 0 indicating non-connectivity. For directed graph connections, the adjacency matrix is ​​asymmetric; for undirected graph connections, the adjacency matrix is ​​symmetric.

[0066] The connection diagram of each vehicle image can be determined according to the specific application scenario. This application does not restrict the connection method and the proximity matrix of the connected images.

[0067] S123: Using the connection graph corresponding to the target frame image, determine the connection graph corresponding to the reference frame image to obtain the connection graph of the target vehicle in each frame of vehicle image; wherein, the reference frame image is the vehicle image of a preset number of frames after the target frame image.

[0068] The target key points in the target frame image can be used to determine the corresponding target key points in the reference frame image. Based on the current identifier sequence number of the target key points in the target frame image, the current identifier sequence number of the target key points in the reference frame image is determined. After determining the connection graph of the target frame image, the connection method of subsequent reference frame images remains consistent with that of the target frame image.

[0069] For example, if the connection diagram corresponding to each target key point (1-9) in the target frame image is determined, and the connection relationship of the connection diagram is: 1→2→3, 4→5→6, 7→8→9, 1→4→7, then according to the connection diagram of the target frame image, the connection diagram corresponding to each target key point in the reference frame image can also be determined as: 1→2→3, 4→5→6, 7→8→9, 1→4→7.

[0070] Using the above method, the connection graph corresponding to the target key points of the target vehicle in each frame of the vehicle image can be determined.

[0071] S13: Using the target vehicles and connection graph of each frame's vehicle image, obtain the traffic state classification of the preset area.

[0072] The feature information of the target vehicle in each vehicle image is obtained, and the changes in the connection graph of the target vehicle in the vehicle image are judged by combining the connection graph, so as to determine the traffic state classification of the preset area based on the changes in each frame of vehicle image.

[0073] In some embodiments, please refer to Figure 6 This embodiment can further extend step S13 of the above embodiment. Using the target vehicles and connectivity map of each frame's vehicle image, a traffic state classification for a preset area is obtained. This embodiment may include the following steps:

[0074] S131: Obtain the vehicle features of the target key points corresponding to the target vehicle in each frame of the vehicle image, wherein the vehicle features include at least one of the following: coordinate information of the target key points and optical flow information of the target key points.

[0075] Using the above method, target key points of T×M target vehicles can be obtained, and then vehicle features of each target key point can be extracted. The vehicle features include at least one of the coordinate information of the target key point and the optical flow information of the target key point.

[0076] For example, the optical flow information f of each target key point can be obtained, combined with the coordinate information (x, y) of each target key point, where x represents the horizontal coordinate and y represents the vertical coordinate. Then, the target key point of each target vehicle can be represented as (x, y, f), with a size of T×M×3, which is the vehicle feature.

[0077] In some implementations, the vehicle features include at least one of the following: coordinate information of target key points, optical flow information of target key points, number of recognition frames, batch size, etc. The number of recognition frames is the total number of frames in the target frame image and the reference frame image. For example, the vehicle features of each vehicle image can be represented as Input_Key(B,3,T,M), where B is the batch size, T is the number of recognition frames, and M is the target key point.

[0078] In some implementations, the vehicle feature Input_Key can be viewed as a 2D image, with its horizontal axis representing the target keypoint dimension or spatial dimension, and its vertical axis representing the time dimension. The vehicle feature Input_Key can be represented as:

[0079]

[0080] in, This represents the vehicle features of the M target key points corresponding to the vehicle image in the first frame. Let M represent the vehicle features of the M target key points corresponding to the vehicle image in frame t, where t represents the number of recognition frames, which is the total number of frames of the target frame image and the reference frame image.

[0081] S132: Use a pre-defined convolutional model to process the vehicle features and connectivity graph of each frame to obtain the traffic state classification of the pre-defined region.

[0082] The vehicle feature Input_Key and the adjacency matrix A of the connectivity graph can be input into a preset convolutional model. The preset convolutional model is used to process the vehicle features and connectivity graph of each frame to obtain the traffic state classification of the preset region.

[0083] In some implementations, a pre-defined convolutional model can be used to process the vehicle features and connectivity graph of each frame. This allows for the acquisition of changes in target vehicles across multiple frames of vehicle images, or analysis of changes in target vehicles within each connectivity graph. The analysis of these changes helps determine the traffic state classification of a pre-defined region. Traffic state classifications include normal traffic flow, slow traffic, traffic accidents, and traffic congestion, among others; this application does not impose any limitations on these classifications.

[0084] In some implementations, a pre-defined convolutional model can be used to process the vehicle features and connectivity graph of each frame to obtain the changes between the target key points corresponding to each target vehicle. The changes include at least one of the changes between the target key points corresponding to each target vehicle in the same lane and the changes between the target key points corresponding to each target vehicle in different lanes. Based on the changes between the target key points, the traffic state classification of the pre-defined area can be determined.

[0085] Please see Figure 7 As an example, consider obtaining a connection graph of three vehicle images. The connection relationships in the connection graph are: 1→2→3, 4→5→6, 7→8→9, 1→4→7. In one frame, the target key points (i.e., target vehicles) in the connection graphs of each lane are aligned. In the second frame of the vehicle image, the forward distance of target key point 4→5→6 in the second lane of the connection graph is less than that of 1→2→3, 7→8→9. In the third frame of the vehicle image, target key point 6 in the second lane of the connection graph moves to the third lane. It can be analyzed that the target vehicle corresponding to target key point 4 in the second lane is abnormal, and the second lane is in a slow-moving state.

[0086] In some implementations, the pre-defined convolutional model may include a Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition (Grad-CAM).

[0087] Please see Figure 8 Spatiotemporal graph convolutional networks can include spatial graph convolutional networks (GCNs) and temporal graph convolutional networks (TCNs), with spatial graph convolution being the core component. A single spatial graph convolution plus a single temporal convolution constitutes one layer. A spatiotemporal graph convolutional network can contain N layers, meaning it can include N cascaded spatial graph convolutional networks (GCNs) and temporal graph convolutional networks (TCNs).

[0088] The vehicle feature Input_Key and the adjacency matrix A of the connection graph are input into the spatial graph convolutional network GCN for processing to obtain spatial convolution features. Then, they are input into the temporal graph convolutional network TCN for processing to obtain temporal convolution features. Thus, the input of one layer is obtained. After further processing by the spatial graph convolutional network GCN and the temporal graph convolutional network TCN, the traffic state classification of the preset area is finally output.

[0089] In some implementations, spatiotemporal graph convolutional networks can be represented by the following formula:

[0090] f(Input_Key,A)=TCN(GCN(Input_Key,A)).

[0091] Where f(Input_Key,A) represents a spatiotemporal graph convolutional network, whose input parameters are the vehicle feature Input_Key and the adjacency matrix A of the connection graph. GCN is spatial graph convolution, and TCN is temporal graph convolution.

[0092] In some implementations, the aforementioned spatiotemporal graph convolutional network can be trained to perform traffic state recognition. The traffic state recognition method described above is used to process the vehicle features (Input_Key) of the sample vehicle image set and the adjacency matrix A of the connectivity graph to extract features. Then, the loss is calculated using the cross-entropy formula, and backpropagation is performed to obtain the traffic state classification, thus training the spatiotemporal graph convolutional network. The spatiotemporal graph convolutional network also includes auxiliary networks, such as Residual, which can be used during the training process to assist in training.

[0093] In this embodiment, a set of vehicle images of a preset area is acquired, containing at least two frames of vehicle images. A connection map of the target vehicles in each frame is obtained. The preset area includes at least one lane, and the connection map contains at least one of the connection relationships between target vehicles in the same lane or between target vehicles in different lanes. Using the target vehicles and connection map of each frame, the traffic state classification of the preset area is obtained. This allows for the analysis of the traffic state of a preset area using vehicle images collected from a single preset area, eliminating the need to construct a complex road network and improving the applicability of traffic state recognition. For example, vehicle images collected from an electronic capture device at a single intersection can be used to analyze the traffic state of that intersection in detail, thus avoiding the need to construct complex road networks and improving the applicability and efficiency of traffic state recognition.

[0094] Please see Figure 9 , Figure 9 This is a flowchart illustrating a second embodiment of the traffic status identification method of this application. The method may include the following steps:

[0095] S21: Determine a salient region by using the traffic state classification of the preset region, wherein the salient region corresponds to at least one of the vehicle image, target key point, and time information that are classified as traffic state preset.

[0096] After obtaining the traffic state classification of the preset area using the target vehicles and connection map of the vehicle images in each frame, any step of this embodiment can be performed.

[0097] In this embodiment, relevant information on the identified traffic conditions can be reported to provide alerts to managers or users.

[0098] In this step, when the traffic condition is classified into a preset state, such as when the traffic condition is abnormal (e.g., a traffic accident, abnormal slowdown), visualization techniques like Grad-CAM can be used to identify salient areas. This process extracts salient areas from a T×M two-dimensional image. The (t,m) coordinates corresponding to the salient area represent the time of the anomaly and the target key point. The coordinates (x,y) of the target key point represent the anomaly location. Finally, the anomaly time and location are reported. Additionally, vehicle images in preset states can also be identified and reported.

[0099] S22: Report the status information of traffic conditions classified into preset states; or, report the status information of significant areas.

[0100] Different reporting methods can be determined based on different application scenarios or traffic conditions. Traffic conditions can be classified into preset states for reporting, such as traffic accidents or abnormal slow traffic. Status information of significant areas can be reported, such as abnormal time, abnormal location, and abnormal vehicle images.

[0101] In this embodiment, by utilizing the traffic state classification of a preset area, a significant area is determined, and the status information of the traffic state classified as a preset state is reported, or the status information of the significant area is reported, so that the traffic state of the preset area can be understood in a timely manner, providing managers with assistance in making decisions about the traffic state.

[0102] Regarding the above embodiments, this application provides a computer device; please refer to [link / reference]. Figure 10 , Figure 10 This is a schematic diagram of an embodiment of the traffic state identification device of this application. The traffic state identification device can be used to implement the above-described traffic state identification method.

[0103] The traffic condition recognition device 30 includes an image module 31, a connection module 32, and a classification module 33. The image module 31, the connection module 32, and the classification module 33 are interconnected.

[0104] The image module 31 is used to acquire a set of vehicle images of a preset area. The set of vehicle images contains at least two frames of vehicle images, and the preset area contains at least one lane.

[0105] The connection module 32 is used to acquire the connection map of the target vehicles in each frame of vehicle image, wherein the connection map includes at least one of the connection relationship of target vehicles in the same lane and the connection relationship of target vehicles in different lanes.

[0106] The classification module 33 is used to obtain the traffic state classification of a preset area by using the target vehicle and connection map of the vehicle image of each frame.

[0107] The specific implementation of this embodiment can be referred to the implementation process of the above embodiments, and will not be repeated here.

[0108] Regarding the above embodiments, this application provides a computer device; please refer to [link / reference]. Figure 11 , Figure 11 This is a schematic diagram of the structure of a computer device according to an embodiment of the present application. The computer device 40 includes a memory 41 and a processor 42, wherein the memory 41 and the processor 42 are coupled to each other. The memory 41 stores program data, and the processor 42 is used to execute the program data to implement the steps of any embodiment of the traffic state identification method described above.

[0109] In this embodiment, processor 42 can also be referred to as a CPU (Central Processing Unit). Processor 42 may be an integrated circuit chip with signal processing capabilities. Processor 42 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The general-purpose processor can be a microprocessor, or processor 42 can be any conventional processor.

[0110] The methods described in the above embodiments can be implemented as computer programs; therefore, this application proposes a computer-readable storage medium. Please refer to [link to relevant documentation]. Figure 12 , Figure 12 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 50 stores program data 51 that can be executed by a processor. The program data 51 can be executed by the processor to implement the steps of any embodiment of the traffic state identification method described above.

[0111] In this embodiment, the computer-readable storage medium 50 can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or a medium that can store program data 51. Alternatively, it can be a server that stores the program data 51. The server can send the stored program data 51 to other devices for execution, or it can run the stored program data 51 itself.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0114] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0115] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application.

[0116] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, and thus stored in a computer-readable storage medium for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Therefore, this application is not limited to any particular hardware and software combination.

[0117] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for identifying traffic conditions, characterized in that, The method includes: Obtain a set of vehicle images of a preset area, wherein the set of vehicle images contains at least two frames of vehicle images, and the preset area contains at least one lane. Obtain a connection graph of target vehicles in each frame of vehicle image, wherein the connection graph includes at least one of the connection relationships of target vehicles in the same lane and the connection relationships of target vehicles in different lanes, including: Obtain the target key points corresponding to each frame of vehicle image, wherein the target key points correspond to each target vehicle; A vehicle image is identified as a target frame image. The target key points corresponding to the target frame image are mapped. Based on the sequential connection relationship of the target key points in the same lane after mapping and the connection relationship between preset key points in each target key point in adjacent lanes, the connection diagram corresponding to the target frame image is determined. Using the connection graph corresponding to the target frame image, the connection graph corresponding to the reference frame image is determined to obtain the connection graph of the target vehicle in each frame of vehicle image; wherein, the reference frame image is a vehicle image of a preset number of frames after the target frame image; Using the target vehicles in each frame of the vehicle image and the connectivity graph, the traffic state classification of the preset region is obtained, including: Obtain the vehicle features of the target key points corresponding to the target vehicle in each frame of the vehicle image; The vehicle features and the connectivity graph of each frame are processed using a preset convolutional model to obtain the traffic state classification of the preset region.

2. The method according to claim 1, characterized in that, The acquisition of target key points corresponding to each frame of vehicle image includes: Using the positions of each target vehicle in each frame of the vehicle image, initial key points are determined for each frame of the vehicle image; wherein, the initial key points are used to represent each target vehicle. In the initial keypoints of the vehicle image in each frame, a first number of target keypoints corresponding to the vehicle image in each frame are determined; wherein the target keypoints include the initial keypoints of target vehicles in at least one lane.

3. The method according to claim 2, characterized in that, The preset area includes multiple lanes, and each lane contains at least one target vehicle; The step of determining a first number of target key points corresponding to each frame of vehicle image from the initial key points includes: For each lane, a second number of target key points are determined from the initial key points of the vehicle image according to a preset selection rule; Based on the second number of target key points for each lane, a first number of target key points corresponding to the vehicle image are obtained; wherein, the second number is less than the first number; all the target key points exist in the vehicle image of a third number of frames.

4. The method according to claim 1, characterized in that, The step of mapping the target key points corresponding to the target frame image, and determining the connection map corresponding to the target frame image based on the sequential connection relationship of the target key points in the same lane after mapping and the connection relationship between preset key points in each target key point in adjacent lanes, includes: Obtain the initial identifier number of each target key point corresponding to the target frame image; The initial identifiers of each target key point are mapped to obtain the current identifiers corresponding to each target key point; According to the current identifier number corresponding to each target key point, the connection relationship between each target key point is obtained to determine the connection graph corresponding to the target frame image; wherein, the connection relationship of the connection graph includes at least one of the following: a first connection relationship between target key points corresponding to target vehicles in the same lane, and a second connection relationship between target key points corresponding to target vehicles in different lanes.

5. The method according to claim 4, characterized in that, The preset area includes multiple lanes. The step of obtaining the connection relationships between the target key points according to their current identifier numbers includes: For the same lane: the first connection relationship of the target key points contained in the lane is determined by sorting them according to the preset position of the current identifier number corresponding to each target key point in the lane. The first connection relationship includes sequential connection relationships. For different lanes: obtain the preset key points in each target key point of each lane, and connect the preset key points of adjacent lanes to determine the second connection relationship.

6. The method according to claim 1, characterized in that, The vehicle features include at least one of the following: the coordinate information of the target key point and the optical flow information of the target key point.

7. The method according to claim 1, characterized in that, The process of using a preset convolutional model to process the vehicle features and the connectivity graph of each frame to obtain the traffic state classification of the preset region includes: The vehicle features and the connectivity graph of each frame are processed using a preset convolutional model to obtain the changes between target key points corresponding to each target vehicle; wherein, the changes include at least one of the changes between target key points corresponding to each target vehicle in the same lane and the changes between target key points corresponding to each target vehicle in different lanes. Based on the changes between the key target points, the traffic state classification of the preset area is determined.

8. The method according to claim 1, characterized in that, After obtaining the traffic state classification of the preset region using the target vehicle and the connectivity graph of each frame of the vehicle image, the method further includes: Using the traffic state classification to which the preset area belongs, a salient area is determined, wherein the salient area corresponds to at least one of the vehicle image, target key point, and time information of the traffic state classification as the preset state; and / or; The traffic conditions are classified into preset conditions and reported; or, the conditions of significant areas are reported.

9. A computer device, characterized in that, The method includes a memory and a processor coupled to each other, the memory storing program data and the processor executing the program data to implement the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The system stores program data that can be executed by a processor, the program data being used to implement the steps of the method according to any one of claims 1 to 8.

Images