An image anomaly matching detection method based on a multi-layer graph neural network

Abstract

The application relates to an image abnormality matching detection method based on a multilayer graph neural network, and relates to the technical field of image recognition, and comprises the following steps: acquiring continuous image frames collected by a fixed-view monitoring camera at night, detecting a vehicle target, and physically decomposing appearance features into light stable features and light sensitive features; cross-frame matching is established based on the similarity of light stable features of front and rear frames, a vehicle time sequence correlation graph and a spatial relationship graph are constructed; a space-time model formed by inputting a multilayer graph neural network is subjected to space-time coding, and predicted node features and predicted edge features are output; an abnormality score is calculated in combination with a matching relationship, and if the abnormality score exceeds a preset threshold, it is determined that the matching is abnormal. The application solves the technical problem that in night monitoring, vehicle light sudden change leads to dramatic change of appearance features, and it is difficult to correctly determine whether the vehicle is consistent during cross-frame tracking.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to an image anomaly matching detection method based on a multilayer graph neural network. Background Technology

[0002] In the field of intelligent traffic monitoring, fixed traffic checkpoint cameras are widely deployed on gantries or roadside poles on urban expressways and main roads to continuously capture images of passing vehicles and recognize license plates from a fixed perspective. By continuously tracking and matching the same vehicle across multiple frames, functions such as vehicle trajectory reconstruction, traffic flow statistics, and suspect vehicle tracking can be achieved. However, at night, when vehicles drive with their headlights on, the headlights become the primary light source for cameras to capture vehicle features, and the appearance of the vehicle target is highly dependent on the direction and brightness of the headlights. When a vehicle switches between high and low beams within the monitored area, applies the brakes causing the taillights to illuminate, or changes the direction of the headlights abruptly when turning, the vehicle's appearance in consecutive image frames will change drastically and instantaneously, including significant jumps in brightness and color distribution.

[0003] Currently, cross-frame vehicle matching primarily relies on appearance-based matching algorithms, which determine whether vehicles in adjacent frames belong to the same entity by comparing the similarity of features such as color, texture, and shape. However, this appearance-based approach has significant limitations in nighttime scenarios with sudden changes in lighting. The feature jumps caused by lighting changes and the feature differences caused by changes in vehicle identity are highly similar in appearance. Matching algorithms can easily misclassify the same vehicle as different vehicles due to feature deviations caused by sudden lighting changes, or even swap the identities of two visually similar vehicles at the moment of the lighting change. Such matching errors can directly lead to broken or confused vehicle trajectories, affecting the accuracy of subsequent traffic behavior analysis, illegal evidence collection, and suspect vehicle control. This makes continuous vehicle tracking under complex nighttime lighting conditions a challenging problem in intelligent traffic monitoring. Summary of the Invention

[0004] This application provides an image anomaly matching and detection method based on multi-layer graph neural networks, which improves the technical problem in the prior art that when facing sudden changes in light at night, the instantaneous appearance fluctuations cannot be effectively distinguished from normal changes and anomalies by relying solely on image features, resulting in unstable vehicle trajectory reconstruction.

[0005] This application discloses the following technical solution: This application provides an image anomaly matching detection method based on a multilayer graph neural network, the method comprising: Acquire continuous image frames captured by a fixed-view surveillance camera during nighttime, detect vehicle targets in each frame, and physically decompose the appearance features of each vehicle target into light-stable features and light-sensitive features. Based on the similarity between the light stabilization features of each vehicle target in the current frame and the light stabilization features of each tracked vehicle target in the previous frame, cross-frame vehicle matching relationships are established, and vehicle temporal association graphs and spatial relationship graphs are constructed. The vehicle temporal correlation graph and spatial relationship graph of multiple consecutive frames are input into the spatiotemporal graph neural network model constructed based on multi-layer graph neural network. The spatiotemporal graph neural network model performs spatiotemporal encoding on the light stability features and spatial relationships of each vehicle target, and outputs the predicted node features and predicted edge features of each vehicle target in the current frame. Based on the cross-frame vehicle matching relationship of the current frame, the predicted node features, and the predicted edge features, calculate the matching anomaly score for each vehicle target; When the matching anomaly score of a vehicle target exceeds a preset anomaly threshold, it is determined that the cross-frame vehicle matching relationship of the vehicle target is abnormal.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages: The technical solution of this application provides an image anomaly matching detection method based on a multi-layer graph neural network. First, by physically decomposing the appearance features of the vehicle target in each frame of the image into light-stable features that are not easily affected by changes in lighting and light-sensitive features that are easily affected, the separation of the vehicle's own structural attributes and external lighting effects is achieved. This provides a stable identity identifier that is resistant to lighting interference for subsequent matching, and solves the problem that the single feature description is unreliable and it is difficult to distinguish between the vehicle's own changes and the lighting effects when the appearance of the vehicle changes suddenly at night.

[0007] Furthermore, by establishing cross-frame vehicle matching relationships based on lighting stability features and constructing a spatial relationship graph within the frame that reflects the spatial positions and relationships between vehicles, a dual description of individual vehicle motion trajectories and group spatial topology is achieved. This provides a structured input basis for subsequent spatiotemporal consistency verification and solves the problem that traditional matching methods rely solely on individual feature similarity and lack spatial position verification methods.

[0008] Furthermore, by inputting the vehicle temporal correlation graph and spatial relationship graph of multiple consecutive frames into the spatiotemporal graph neural network model, the vehicle features and spatial relationships of the current frame are predicted by utilizing historical temporal patterns. This achieves temporal consistency modeling of the appearance and spatial position that a vehicle should have under normal driving conditions, and solves the problem that it is impossible to judge whether feature changes are reasonable by simply relying on the information of the current frame.

[0009] Furthermore, by quantifying the deviation between the actual extracted stable light features and spatial relationships and the expected values ​​predicted by the model, and comprehensively calculating the matching anomaly score, multi-dimensional cross-validation of the matching state is achieved, solving the problem that it is difficult to distinguish between normal light mutations and matching errors in a single dimension.

[0010] Finally, by backtracking the stable features of normal frames for correction matching when abnormal scores exceed the limit, the automatic repair of abnormal matching relationships is realized, solving the problems of identity confusion and trajectory breakage caused by sudden changes in lighting.

[0011] In summary, this application addresses the cross-frame matching anomaly caused by sudden changes in vehicle lights during nighttime traffic monitoring. By separating the stable and sensitive features of vehicle exterior lighting and utilizing a spatiotemporal graph neural network to perform temporal consistency verification of vehicle motion trajectories and spatial relationships, it achieves accurate detection and automatic correction of matching anomalies, effectively improving the technical problems of feature misjudgment and vehicle identity confusion caused by sudden changes in lighting in nighttime scenarios. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating an image anomaly matching detection method based on a multilayer graph neural network, provided in an embodiment of this application.

[0014] Figure 2 A flowchart illustrating a light change perception method for image anomaly matching detection based on a multi-layer graph neural network, provided in this application embodiment. Detailed Implementation

[0015] This application provides an image anomaly matching detection method based on a multi-layer graph neural network, which solves the technical problem in the prior art that when continuously tracking vehicles at night, it is difficult to accurately identify whether cross-frame matching has failed due to sudden changes in brightness caused by headlight switching or braking.

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. It should be noted that the numerical values ​​in the embodiments are for illustrative purposes only and do not constitute a limitation on this application.

[0017] Example 1, as shown in the appendix Figure 1 As shown, this application provides an image anomaly matching detection method based on a multilayer graph neural network, the method comprising the following steps: S100: Acquire continuous image frames captured by a fixed-view surveillance camera during nighttime, detect vehicle targets in each frame, and physically decompose the appearance features of each vehicle target into light-stable features and light-sensitive features. In this embodiment, in a nighttime fixed-view surveillance image, the appearance of a vehicle target changes drastically between frames due to the influence of headlight illumination. To support subsequent stable matching and anomaly detection, this step requires decomposing the appearance features of the vehicle target into stable parts unaffected by headlight changes and sensitive parts susceptible to them. This provides stable features resistant to headlight interference for matching, while retaining headlight-sensitive features to detect sudden changes.

[0018] Step S100 in the method provided in this application embodiment includes: Continuous image frames captured by a fixed-view surveillance camera during nighttime are acquired. Each image frame is input into a vehicle detection network, which outputs a bounding box for each vehicle target in each frame. The bounding box contains the vehicle target's position coordinates, width, and height within the image. For each vehicle target, the corresponding vehicle image region is cropped from the image frame based on the bounding box. A detailed explanation follows: In this embodiment, the vehicle detection network refers to a common deep learning-based target detection model in the field of computer vision, such as the YOLO series or Faster R-CNN. This network can automatically identify and locate each vehicle target from the input monitoring image frame, and output a bounding box containing its position coordinates, width, and height. The bounding box refers to the rectangular box output by the vehicle detection network that identifies the position and size of the vehicle target in the image.

[0019] In this step, firstly, to accurately locate all vehicle targets in the nighttime surveillance images, each image frame needs to be input into a general vehicle detection network. The network outputs the bounding box of each vehicle target in each frame, thereby determining the precise location and size range of each vehicle in the image. Then, to separate each vehicle target from the background for independent analysis, the corresponding vehicle image region needs to be cropped from the original image frame based on the coordinates of each bounding box, serving as input for subsequent appearance feature extraction.

[0020] Step S100 in the method provided in this application embodiment further includes: For each vehicle target's vehicle image region, the vehicle structure contour is extracted using an edge detection algorithm. Combined with prior knowledge of vehicle shape, the window region and roof contour region are extracted. The vehicle structure contour, the window region, and the roof contour region are collectively used as the light stabilization region. The bright areas in the vehicle image region whose brightness values ​​exceed a preset brightness threshold are extracted as the headlight self-illuminating region. The regions in the vehicle image region other than the light stabilization region and the headlight self-illuminating region are used as the vehicle body reflection region. The headlight self-illuminating region and the vehicle body reflection region are collectively used as the light sensitive region. An directional gradient histogram is extracted from the vehicle structure contour in the stable lighting region. This histogram is used as a vehicle structure shape descriptor. The normalized relative position of the window region center coordinates and the vehicle bounding box in the stable lighting region is calculated as the window relative position code. The vehicle structure shape descriptor and the window relative position code are used together as the lighting stable feature vector. A luminance distribution histogram and a color histogram are statistically analyzed for the lighting sensitive region. These two histograms are used together as the lighting sensitive feature vector. The inter-frame change between the lighting sensitive feature vector of each vehicle target in the current frame and the lighting sensitive feature vector of the same vehicle target in the previous frame is calculated. When the inter-frame change exceeds a preset abrupt change threshold, the vehicle target is determined to be at a lighting abrupt change moment. Detailed explanation follows: In this embodiment, the stable light region refers to the area of ​​the vehicle's exterior that maintains structural stability under changes in light intensity, including the vehicle's structural outline, windows, and roof—areas that are not easily altered by changes in light. The light-sensitive region refers to the area of ​​the vehicle's exterior that is significantly affected by changes in light intensity, including the self-illuminating areas of the headlights and areas of the vehicle body affected by light reflection. The histogram of directional gradients is a feature descriptor used to describe the local texture and shape of an image. It characterizes the edge and shape information of an object by statistically analyzing the gradient direction histogram of local image regions. The preset brightness threshold is a critical value used to distinguish between the self-illuminating areas of the headlights and the reflective areas of the vehicle body. It is determined by statistically analyzing the brightness distribution of the headlight area in nighttime vehicle images and taking its low quantile. When the image brightness ranges from 0 to 255, a typical value is set to 200. Pixels with brightness exceeding this threshold are marked as self-illuminating headlight areas, and the rest are classified as reflective areas of the vehicle body.

[0021] The relative position encoding of the vehicle window refers to the normalized coordinates of the center point of the window area within the vehicle's bounding box, used to describe the relative position of the window within the overall vehicle. This feature is unaffected by changes in lighting conditions. Inter-frame variation refers to the measure of the difference in light-sensitive feature vectors between adjacent frames, using histogram distance or vector distance to determine if a sudden change in lighting has occurred. Preliminary inter-frame association refers to, before establishing a formal cross-frame matching relationship, using the physical characteristic that the vehicle target's motion displacement is small between consecutive frames from a fixed viewpoint, and utilizing the spatial intersection-over-union ratio of the bounding box or the Euclidean distance of the center point, matching the target in the current frame with the nearest vehicle target in the previous frame to establish a temporary target correspondence.

[0022] A sudden change in vehicle headlight status refers to a moment when the headlight state changes abruptly and drastically, such as switching between high and low beams or the brake lights illuminating. At this time, the headlight sensitivity characteristics will exhibit a significant jump. The preset sudden change threshold is obtained through statistical calibration based on historical normal traffic flow data, or empirically set according to the physical characteristics of vehicle headlight changes. Specifically, the 95th percentile of the statistical distribution of inter-frame changes under normal driving conditions is taken as the preset sudden change threshold, or an empirical value is directly set as the boundary between normal headlight fluctuations and true sudden changes. In this embodiment, the preset sudden change threshold is preferably 0.5. The headlight sudden change determination process in this step is as follows: Figure 2 As shown.

[0023] In this step, firstly, to separate structural regions unaffected by lighting changes from the vehicle image area, edge detection algorithms are used to extract the vehicle's contour lines. Combined with prior knowledge of vehicle shape, the window and roof contour regions are identified, and these regions are collectively labeled as lighting-stable areas. Edge detection extracts the boundary lines between the vehicle body and windows, while shape priors assist in accurately locating the window positions under occlusion or low-contrast conditions. Next, to extract regions affected by lighting changes, bright areas in the vehicle image area with brightness values ​​exceeding a preset brightness threshold are extracted as headlight self-illuminating areas. The areas outside the lighting-stable and headlight self-illuminating regions are labeled as vehicle body reflection areas. These two areas together constitute the lighting-sensitive areas.

[0024] Furthermore, to generate vehicle identity features robust to changes in lighting conditions, feature extraction is required for lighting-stable regions. An oriented gradient histogram is extracted from the vehicle's structural contour region as a vehicle structural shape descriptor. This descriptor primarily captures the geometric shape information of the vehicle contour and is unaffected by changes in illumination intensity. Simultaneously, the normalized relative position of the window's center point within the vehicle's bounding box is calculated as the window's relative position code, reflecting the window's relative layout within the overall vehicle. The vehicle structural shape descriptor and the window's relative position code are concatenated to form a lighting-stable feature vector, serving as a stable identity identifier for the vehicle under nighttime lighting variations.

[0025] Furthermore, to capture appearance fluctuations caused by changes in lighting, it is necessary to statistically analyze the luminance distribution histogram and color histogram of the light-sensitive area. These two histograms together constitute the light-sensitive feature vector. The luminance histogram reflects the distribution of reflected light from the vehicle body and the luminous intensity of the headlights, while the color histogram reflects the vehicle's color performance under specific lighting conditions. When the lighting conditions change abruptly, these histograms will show significant differences between frames.

[0026] Finally, to detect whether a vehicle is experiencing a sudden change in headlights, a preliminary inter-frame association operation is required: leveraging the characteristic that the vehicle's motion displacement between adjacent frames is relatively small when monitored from a fixed viewpoint, based on the spatial intersection-over-union ratio or Euclidean distance of the vehicle target bounding box, each vehicle target in the current frame is initially matched with the nearest target in the previous frame. Then, the inter-frame change between the headlight-sensitive feature vector of each vehicle target in the current frame and the headlight-sensitive feature vector of the corresponding vehicle target in the previous frame obtained from the preliminary matching is calculated. When the inter-frame change exceeds a preset abrupt change threshold, the vehicle target is determined to be at the moment of a headlight change. The determination of the headlight change moment is used to dynamically adjust the weight of node feature deviation in subsequent matching anomaly detection, appropriately reducing the sensitivity to changes in headlight-sensitive features during headlight changes, and avoiding misjudging normal headlight switching as a matching anomaly.

[0027] In summary, this step, through vehicle detection and physical decomposition of appearance features, separates the appearance of nighttime vehicle targets into light-stable features unaffected by light changes and light-sensitive features affected by light changes, while also identifying the moment of light abrupt changes. Compared with existing technologies, this step has the following advantages: First, it physically separates the structural attributes of the vehicle appearance that are unrelated to light changes from the illumination attributes that are related to light changes, allowing the subsequent matching process to be performed in a stable feature space resistant to light interference, fundamentally reducing the interference of light abrupt changes on the matching; second, it preserves the ability to perceive light-sensitive features and the moment of abrupt changes, providing crucial state information for dynamically adjusting the sensitivity of anomaly detection at the moment of light abrupt changes, avoiding false detections caused by normal light switching.

[0028] S200: Based on the similarity between the light stabilization features of each vehicle target in the current frame and the light stabilization features of each tracked vehicle target in the previous frame, establish cross-frame vehicle matching relationships and construct vehicle temporal association graphs and spatial relationship graphs. In this embodiment, after obtaining the stable lighting features of vehicle targets in each frame, it is necessary to establish the vehicle identity correspondence between adjacent frames based on these stable features resistant to lighting interference. Simultaneously, a relationship graph describing the spatial layout of vehicles within the frame is constructed to provide motion associations of individual vehicles and spatial topological information of the group for subsequent spatiotemporal graph neural networks. This step achieves robust association of vehicle identities between frames through similarity matching of stable lighting features and Hungarian global optimization; a spatial relationship graph is constructed through joint constraints of spatial distance and azimuth angle, encoding the relative positional relationships between vehicles on the road surface as graph structure edge features.

[0029] Step S200 in the method provided in this application embodiment includes: Calculate the cosine similarity between the stable light feature vector of each vehicle target in the current frame and the stable light feature vector of each tracked vehicle target in the previous frame. For each vehicle target in the current frame, select the tracked vehicle target with the highest cosine similarity as the matching target, and establish a cross-frame vehicle matching relationship between the vehicle target and the matching target in the current frame. When multiple vehicle targets match the same tracked vehicle target, use a global optimal matching algorithm to globally optimize the matching results, eliminate matching conflicts, and obtain the final cross-frame vehicle matching relationship. Treat each vehicle target in each frame as a graph node, and use the cross-frame vehicle matching relationship as a temporal edge to connect the corresponding graph nodes of the same vehicle target in the preceding and following frames, forming a vehicle temporal association graph. For all vehicle targets in each frame, calculate the spatial distance and relative azimuth angle between any two vehicle targets, and use the spatial distance and relative azimuth angle as spatial edge features to form a spatial relationship graph. Detailed explanation is as follows: In this embodiment, cross-frame vehicle matching refers to the identity correspondence established based on feature similarity between vehicle targets detected in the current frame and tracked vehicle targets in the previous frame, which is the basis for the continuity of vehicle trajectories. Cosine similarity refers to the cosine value of the angle between two vectors in the feature space, ranging from -1 to 1. The closer to 1, the more consistent the directions of the two stable light feature vectors, and the greater the probability that the vehicles share the same identity. The vehicle temporal association graph refers to a cross-frame graph structure constructed with vehicle targets in each frame as nodes and cross-frame matching relationships as edges. Edges connect nodes of the same vehicle in different frames, reflecting the motion trajectory of individual vehicles. The spatial relationship graph refers to an intra-frame graph structure constructed with vehicle targets as nodes and spatial proximity relationships between vehicles as edges within the same frame. Edge features include the spatial distance and relative azimuth angle between two vehicles, reflecting the spatial topology of the vehicle group.

[0030] In this step, firstly, to quantify the similarity of vehicle appearance between the current frame and the previous frame, the cosine similarity of the stable feature vectors of all vehicle target lights between the two frames needs to be calculated to construct a similarity matrix. Then, for each vehicle target in the current frame, the tracked vehicle target with the largest cosine similarity in the previous frame is selected as a candidate matching object to establish a preliminary cross-frame vehicle matching relationship.

[0031] Furthermore, when multiple vehicle targets point to the same candidate match, a matching conflict arises, requiring a globally optimal matching algorithm to optimize the entire matching scheme. The similarity matrix is ​​transformed into the input of an optimization problem, with the objective of maximizing the sum of similarities for all matching pairs. The Hungarian algorithm is used to solve for the globally optimal matching scheme, ensuring that each vehicle target corresponds one-to-one with the unique tracked vehicle target in the previous frame. When the number of vehicles differs between two frames, the matrix can be expanded into a square matrix by adding rows or columns with zero values. The matching results corresponding to the added rows or columns are considered as newly appearing or disappeared vehicles.

[0032] Furthermore, in order to record the motion trajectory of individual vehicles, all vehicle targets in each frame need to be used as graph nodes, and cross-frame matching relationships need to be used as temporal edges to connect the nodes of the same vehicle in consecutive frames, thereby constructing a vehicle temporal association graph.

[0033] Finally, to describe the spatial layout relationship between vehicles within the same frame, the spatial distance and relative azimuth angle of each pair of vehicle targets in each frame need to be calculated. When the spatial distance between two vehicles is less than or equal to a preset adjacency distance threshold, a spatial edge is established between them. The spatial distance and relative azimuth angle are used as spatial edge features. All spatial edges between vehicle pairs that meet the threshold conditions are summarized to form the spatial relationship graph of that frame.

[0034] Step S200 in the method provided in this application embodiment further includes: A similarity matrix is ​​constructed by comparing the cosine similarities between all vehicle targets in the current frame and all tracked vehicle targets in the previous frame. With the optimization objective of maximizing the sum of the cosine similarities of all matching pairs, the Hungarian algorithm is used to solve for the global optimal matching of this similarity matrix, resulting in a one-to-one cross-frame vehicle matching relationship between each vehicle target in the current frame and the tracked vehicle targets in the previous frame. A detailed explanation follows: In this embodiment, the Hungarian algorithm is a combinatorial optimization algorithm for solving the assignment problem, capable of finding a matching scheme that maximizes the total weight in a bipartite graph in polynomial time. In this application scenario, the vehicle target in the current frame is taken as one side node, and the vehicle target tracked in the previous frame is taken as the other side node. Each element in the similarity matrix is ​​taken as the matching weight between the corresponding nodes. The Hungarian algorithm iteratively optimizes the matching scheme by finding augmenting paths, and finally outputs the globally optimal matching result that maximizes the total similarity.

[0035] For example, suppose vehicle targets C1, C2, and C3 are detected in the current frame, and vehicle targets P1, P2, and P3 were tracked in the previous frame. Calculating the similarity of the light stabilization features yields: C1 has the highest similarity to P1 (0.95), C2 also has a relatively high similarity to P1 (0.85), and C3 has the highest similarity to P2 (0.90). If we simply take the maximum value for each row, both C1 and C2 will match P1, resulting in a conflict. Using the Hungarian algorithm for global optimization, the algorithm exhaustively searches all matching combinations and ultimately determines the optimal matching scheme as: C1 matches P1 (0.95), C2 matches P3 (0.35), and C3 matches P2 (0.90), with a total similarity of 2.20, higher than any other combination. This scheme successfully eliminates the matching conflict between C1 and C2, achieving globally optimal cross-frame matching.

[0036] In this step, a cosine similarity matrix is ​​first constructed between all vehicle targets in the current frame and the tracked vehicle targets in the previous frame. Then, with the optimization objective of maximizing the sum of the cosine similarities of all matching pairs, the Hungarian algorithm is used to solve for the globally optimal matching of the similarity matrix. When the number of vehicle targets differs between two frames, zero values ​​are filled into the matrix to expand it into a square matrix; the filled zero values ​​correspond to newly appearing or disappearing vehicles. After the solution is completed, the optimal cross-frame matching relationship for all vehicle targets between two frames is obtained.

[0037] Step S200 in the method provided in this application embodiment further includes: Obtain the coordinates of the center points of the bounding boxes of two vehicle targets, calculate the Euclidean distance between the two center point coordinates, and use it as the spatial distance between the two vehicle targets; calculate the direction angle of the line connecting the two center point coordinates, and use it as the relative azimuth angle between the two vehicle targets; when the spatial distance is less than or equal to a preset adjacency distance threshold, establish a spatial edge between the two vehicle targets, and use the spatial distance and the relative azimuth angle as the spatial edge features of the spatial edge; combine the spatial edges between all pairs of vehicle targets that satisfy the preset adjacency distance threshold to form a spatial relationship graph. Detailed explanation follows: In this embodiment, spatial distance refers to the Euclidean distance between the center points of the bounding boxes of two vehicle targets within the same frame, reflecting the distance between the two vehicles in the image space. Relative azimuth angle refers to the angle between the straight line connecting the center points of the two vehicles and the horizontal direction of the image, reflecting the relative directional relationship between the two vehicles. Spatial edge refers to the edge connecting the two vehicle target nodes in the spatial relationship graph when the spatial distance between the two vehicles meets a preset adjacency distance threshold; its edge features are composed of both spatial distance and relative azimuth angle.

[0038] In this step, firstly, to quantify the spatial relationship between two vehicle targets within the same frame, the coordinates of the center points of the bounding boxes of the two vehicles are obtained, and the Euclidean distance between the two points is calculated as the spatial distance. Simultaneously, the angle between the line connecting the two points and the horizontal direction of the image is calculated as the relative azimuth angle. Then, the spatial distance is compared with a preset adjacency distance threshold. When the spatial distance is less than or equal to the threshold, a spatial edge is established between the two vehicles, and the spatial distance and relative azimuth angle are used as the feature vector of the spatial edge. Finally, all vehicle target pairs within the frame are traversed, and all spatial edges that meet the conditions are summarized to form the spatial relationship graph of the current frame.

[0039] For example, consider a segment of three consecutive frames from a nighttime urban expressway checkpoint monitoring scenario. The current frame is frame t, and the previous frame is frame t. 1 frame. The t-th frame. In frame 1, vehicles A, B, and C have been tracked. In frame t, vehicles D, E, and F have been detected. Calculate the relationship between the three vehicles in frame t and the data in frame t. The cosine similarity of the stable light features of the three vehicles in one frame is used to form a 3x3 similarity matrix. The Hungarian algorithm is used to solve the global optimal matching problem on the similarity matrix. The matching results are: D matches A, E matches B, and F matches C. All matching pairs have the maximum total similarity and there are no matching conflicts.

[0040] The cross-frame matching relationship of the three vehicles is used as a temporal edge, connecting the t-th frame. The corresponding vehicle nodes in frame 1 and frame t constitute a vehicle temporal relationship graph. Within frame t, the spatial distance and relative azimuth angle between each pair of the three vehicles are calculated, with a preset adjacency distance threshold set to one-third of the image width. If the spatial distance between vehicles D and E is less than the threshold, a spatial edge DE is created, with edge features being a two-dimensional vector of spatial distance and relative azimuth angle. If the spatial distance between E and F is also less than the threshold, a spatial edge EF is created. If the distance between D and F exceeds the threshold, no spatial edge is created. The two spatial edges and their corresponding edge features are summarized to form the spatial relationship graph for frame t.

[0041] In summary, this step establishes a robust association of vehicle identities between frames through cosine similarity matching of stable lighting features and Hungarian global optimization, solving the problem of local optimal matching conflicts caused by the decrease in appearance similarity when lighting changes abruptly. At the same time, an intra-frame spatial relationship graph is constructed through spatial distance and azimuth constraints, encoding the geometric positional relationships between vehicles as graph structure edge features, providing dual inputs of individual trajectory and group spatial topology for subsequent spatiotemporal graph neural networks.

[0042] S300: Input the vehicle temporal correlation graph and spatial relationship graph of multiple consecutive frames into the spatiotemporal graph neural network model constructed based on the multi-layer graph neural network. The spatiotemporal graph neural network model performs spatiotemporal encoding on the light stability features and spatial relationships of each vehicle target, and outputs the predicted node features and predicted edge features of each vehicle target in the current frame. In this embodiment, after constructing a vehicle temporal association graph and spatial relationship graph for multiple consecutive frames, it is necessary to use the vehicle appearance and spatial topology patterns of historical frames to predict the appearance features that each vehicle should have in future frames and the spatial relationships that should exist between vehicles. This establishes a reference benchmark for the features that should be presented under normal conditions, providing a basis for subsequent comparison of deviations with the actual extracted features. This step uses a spatial graph convolutional layer to encode the interactive features of adjacent vehicles within a frame, and a temporal graph convolutional layer to model the temporal evolution of individual vehicle features. This allows the predicted node features and predicted edge features output by the model to integrate the vehicle's own historical motion patterns and the mutual influence of surrounding vehicles.

[0043] Step S300 in the method provided in this application embodiment includes: The temporal correlation graph and spatial relationship graph of vehicles in consecutive frames preceding the current frame are input into a spatiotemporal graph neural network model constructed based on a multi-layer graph neural network. The spatial graph convolutional layer of the spatiotemporal graph neural network model performs graph convolution operations on the spatial relationship graph of each frame. The spatial graph convolutional layer performs bidirectional feature aggregation on two vehicle targets with spatial edges, aggregating the lighting stability features of spatially adjacent vehicle targets of each vehicle target to that vehicle target, and simultaneously aggregating the lighting stability features of that vehicle target to the spatially adjacent vehicle targets, outputting the spatial context feature vector of each vehicle target. The temporal graph convolutional layer of the spatiotemporal graph neural network model performs temporal convolution operations on the spatial context feature vectors of the same vehicle target in consecutive frames, outputting the predicted node features and predicted edge features of each vehicle target in the current frame. Detailed explanation follows: In this embodiment, the spatiotemporal graph neural network model refers to a multi-layer graph neural network composed of stacked spatial graph convolutional layers and temporal graph convolutional layers. The spatial graph convolutional layer is responsible for performing feature interaction encoding on spatially adjacent vehicle targets within the same frame, while the temporal graph convolutional layer is responsible for temporal evolution modeling of the feature sequences of the same vehicle target across multiple consecutive frames. The spatial graph convolutional layer refers to a network layer that performs graph convolution operations on a spatial relationship graph, aggregating the features of spatially adjacent nodes of each vehicle target to integrate local spatial topological information into the node features. The temporal graph convolutional layer refers to a network layer that performs convolution operations along the time dimension on a vehicle temporal relationship graph, capturing the feature evolution patterns during vehicle motion by temporally encoding the spatial context feature vector sequences of the same vehicle target at consecutive time steps.

[0044] Spatial context feature vector refers to the vehicle target feature vector after aggregating the features of adjacent vehicle targets through a spatial graph convolutional layer. It includes the vehicle's own appearance features and its relative spatial relationship with surrounding vehicles. Predicted node features refer to the predicted light stability feature vectors that each vehicle target should present in the current frame, based on historical information from multiple consecutive frames prior to the current frame, provided by the spatiotemporal graph neural network model. Predicted edge features refer to the predicted spatial distance and relative azimuth angles that should be presented between each pair of vehicle targets with spatial edges in the current frame, based on the spatial relationship graph of historical frames, provided by the spatiotemporal graph neural network model.

[0045] In this step, firstly, to input vehicle motion information and spatial topology information from multiple consecutive frames into the spatiotemporal graph neural network model, the vehicle temporal association graph and spatial relationship graph from the previous multiple frames are used as input to the spatiotemporal graph neural network model. Then, within each frame, the spatial graph convolutional layer performs bidirectional feature aggregation on all vehicle target pairs with spatial edges in the spatial relationship graph. For adjacent vehicle targets, their headlight stabilization features are bidirectionally transmitted and aggregated along the spatial edges, so that the feature vector of each vehicle target incorporates the spatial context information of the surrounding vehicles, resulting in a spatial context feature vector. The specific operation of bidirectional aggregation can employ mean aggregation, maximum aggregation, or weighted aggregation based on attention weights.

[0046] Furthermore, to capture the motion trajectory and feature evolution patterns of vehicle targets across multiple consecutive frames, a temporal graph convolutional layer is needed to perform temporal convolution operations on the spatial context feature vector sequence corresponding to the same vehicle target across multiple consecutive frames. The temporal convolutional layer performs one-dimensional convolution along the time axis or uses gated recurrent units for temporal encoding, outputting the predicted node features of each vehicle target in the current frame, as well as the predicted edge features between each pair of vehicle targets with spatial edges in the current frame. The predicted node features reflect the stable lighting characteristics that the vehicle target should have in the current frame under normal driving conditions, while the predicted edge features reflect the spatial distance and relative azimuth relationship between adjacent vehicle pairs under normal driving conditions.

[0047] Step S300 in the method provided in this application embodiment further includes: A historical video dataset of nighttime traffic monitoring is acquired. Vehicle target detection and physical decomposition of appearance features are performed on consecutive image frames in the historical video dataset to construct a historical vehicle temporal correlation graph and a historical spatial relationship graph. For each frame in the historical video dataset, the stable light feature vector and spatial edge features of each vehicle target in that frame under normal matching conditions are obtained to form a training sample label set. Based on a multi-layer graph neural network, an initial spatiotemporal graph neural network model is constructed, which includes spatial graph convolutional layers and temporal graph convolutional layers. Using the historical vehicle temporal correlation graph and the historical spatial relationship graph as input, and the stable light feature vector and spatial edge features of each vehicle target in the next frame from the training sample label set as output labels, the initial spatiotemporal graph neural network model is supervisedly trained with the training objective of minimizing the weighted sum of the node feature prediction error between the predicted node features and the stable light feature vector, and the edge feature prediction error between the predicted edge features and the spatial edge features. This training continues until convergence is verified, resulting in a trained spatiotemporal graph neural network model. Detailed explanation follows: In this embodiment, the historical vehicle temporal correlation graph and historical spatial relationship graph refer to the vehicle temporal correlation graph and spatial relationship graph extracted and constructed from historical video data of nighttime traffic monitoring in the manner described in the preceding steps, which serve as input data for training the spatiotemporal graph neural network model. The node feature prediction error refers to the deviation between the predicted node features output by the spatiotemporal graph neural network model and the actual extracted light stability feature vector of the corresponding vehicle target in that frame, calculated using cosine distance or mean square error. The edge feature prediction error refers to the deviation between the predicted edge features output by the spatiotemporal graph neural network model and the actual spatial edge features between the corresponding vehicle pairs, calculated using Euclidean distance or mean square error.

[0048] In this step, firstly, to train the spatiotemporal graph neural network model, training data needs to be extracted from the historical video dataset of nighttime traffic monitoring. For consecutive image frames in the historical video, vehicle target detection and physical decomposition of appearance features are performed as described in the previous steps. Similarly, a historical vehicle temporal association graph and a historical spatial relationship graph are constructed as described in the previous steps. Frames that are manually labeled or whose matching relationships have been verified to be normal are used as training samples. The stable light feature vector and spatial edge features of the normal matching state in each frame are used as the training sample label set.

[0049] Furthermore, to construct the initial spatiotemporal graph neural network model, a multi-layer graph neural network structure containing spatial graph convolutional layers and temporal graph convolutional layers needs to be built. The number of spatial graph convolutional layers and the output dimension of each layer are determined based on the node feature dimension, while the temporal graph convolutional layers can be implemented using a temporal convolutional network or a gated recurrent unit.

[0050] Furthermore, to train the spatiotemporal graph neural network model to learn the vehicle appearance and spatial relationship patterns under normal matching conditions, it is necessary to use historical vehicle temporal association maps and historical spatial relationship maps from multiple consecutive frames as input, and the actual extracted light stability feature vector and spatial edge features from the next frame as output supervision labels. The training objective is to minimize the weighted sum of the node feature prediction error between the predicted node features and the actual light stability feature vector, and the edge feature prediction error between the predicted edge features and the actual spatial edge features. The spatiotemporal graph neural network model parameters are iteratively updated through gradient backpropagation. Training stops when the total loss on the validation set converges to a preset level or no longer decreases after several consecutive rounds, thus obtaining the trained spatiotemporal graph neural network model.

[0051] For example, the pre-training process of the spatiotemporal graph neural network model is as follows: From historical videos collected by multiple nighttime traffic checkpoint surveillance cameras, 200 video segments with normal traffic flow periods and no abnormal matching records are selected, each segment containing 50 consecutive frames. For each frame, vehicle targets are extracted and a temporal correlation graph and spatial relationship graph are constructed using the aforementioned steps, generating approximately 10,000 labeled training sample frames. These are then divided into a training set of approximately 8,000 frames, a validation set of approximately 1,000 frames, and a test set of approximately 1,000 frames, in an 8:1:1 ratio.

[0052] The spatial graph convolutional layer employs two graph convolutional layers, each with an output dimension of 64, and uses the mean aggregation function. The temporal graph convolutional layer uses two gated recurrent units with a hidden dimension of 32. Cosine distance is used for node feature prediction error, and mean squared error is used for edge feature prediction error, with a weight ratio of 1:0.5. The optimizer is Adam, with an initial learning rate of 0.001, a batch size of 8 video segments, and a maximum training epoch of 100 epochs. Early termination occurs if the validation loss does not decrease for 10 consecutive epochs. In actual training, early termination is triggered after 68 epochs. On the test set, the mean cosine distance for node feature prediction is approximately 0.08, and the mean squared error for edge feature prediction is approximately 0.015, indicating that the spatiotemporal graph neural network model can accurately predict vehicle features and spatial relationships under normal matching conditions.

[0053] For example, consider a segment of three consecutive frames from a nighttime urban expressway checkpoint monitoring scenario. Let the t-th frame... 2 frames, t The temporal correlation graph and spatial relationship graph of vehicles in frame 1 are input into the trained spatiotemporal graph neural network model. The spatial graph convolutional layer is used for frames t-2 and t-th. The spatial relationship graph of frame 1 performs graph convolution operation. Vehicle D and its spatially adjacent vehicle E undergo bidirectional feature aggregation. The lighting stability features of vehicle D are integrated into the spatial context features of E, and the lighting stability features of E are also integrated into the spatial context features of D. Bidirectional aggregation is also performed between E and F.

[0054] After spatial graph convolution processing, the spatial context feature vectors of each vehicle target are obtained. The temporal graph convolutional layer processes vehicle D from the t-th... Frame 2 to t The spatial context feature vector sequence of a frame is temporally convolutionally encoded to output the predicted node features of vehicle D in frame t and the predicted edge features between vehicle D and its neighboring vehicles. For vehicle D, its predicted node features represent the stable lighting features that should be presented under the current normal driving condition; its predicted edge features with vehicle E represent the spatial distance and relative azimuth angle that the two should maintain under the current normal driving condition.

[0055] In summary, this step uses spatial graph convolutional layers to encode the features of spatially adjacent vehicles within a frame, and temporal graph convolutional layers to model the feature evolution of individual vehicles along the time axis. This enables the spatiotemporal graph neural network model to output predicted node features and predicted edge features of the current frame based on historical normal driving patterns. Compared with existing technologies, this step has the following advantages: First, it simplifies vehicle tracking into a regression problem of predicting the current state from historical trends, eliminating the need for direct training on abnormal samples in complex lighting change scenarios. It only needs to learn the evolution patterns under normal conditions to establish a prediction benchmark. Second, the bidirectional aggregation mechanism of the spatial graph convolutional layers allows the spatiotemporal graph neural network model to utilize the motion information of surrounding vehicles to assist in prediction. When the appearance features of the vehicle fluctuate drastically due to lighting changes, the features of spatially adjacent vehicles can provide additional spatial context constraints, improving the robustness and accuracy of the prediction.

[0056] S400: Based on the cross-frame vehicle matching relationship of the current frame, the predicted node features, and the predicted edge features, calculate the matching anomaly score for each vehicle target; In this embodiment of the application, after obtaining the cross-frame matching relationship and the node and edge features predicted by the spatiotemporal graph neural network model, it is necessary to compare the actual extracted light stability features and spatial relationships with the expected values ​​predicted by the model based on historical patterns, and to determine whether the current matching relationship deviates from the performance that should be under normal driving conditions through the quantitative fusion of multi-dimensional deviations.

[0057] Step S400 in the method provided in this application embodiment includes: For each vehicle target in the current frame, the stable light feature vector of the vehicle target and the predicted node feature output by the spatiotemporal graph neural network model are obtained. The cosine distance between the stable light feature vector and the predicted node feature is calculated as the node feature bias. For each pair of vehicle targets with spatial edges in the current frame, the spatial edge feature between them and the predicted edge feature output by the spatiotemporal graph neural network model are obtained. The Euclidean distance between the spatial edge feature and the predicted edge feature is calculated as the edge feature bias. The edge feature biases of all spatial edges associated with each vehicle target are accumulated to obtain the accumulated edge bias value of the vehicle target. The node feature bias and the accumulated edge bias value are weighted and summed to obtain the matching anomaly score of the vehicle target. When the vehicle target is at a moment of sudden light change, the weight of the node feature bias is reduced. Detailed explanation is as follows: In this embodiment, the matching anomaly score refers to a comprehensive quantitative score of the reliability of the cross-frame matching relationship in the current frame. The higher the score, the further the current matching relationship deviates from the normal driving state, and the greater the possibility of matching anomalies. Node feature deviation refers to the cosine distance between the actual stable feature vector of the vehicle target's lights and the predicted node features predicted by the spatiotemporal graph neural network model, reflecting the degree of deviation between the vehicle's own appearance features and the expected state. Edge feature deviation refers to the Euclidean distance between the actual spatial edge features between two vehicles and the predicted edge features predicted by the model, reflecting the degree of deviation between the spatial relationship between the vehicle pair and the expected state. The cumulative edge deviation value refers to the cumulative deviation obtained by summing the edge feature deviations of all spatial edges associated with the current vehicle target, comprehensively reflecting the overall degree of anomaly in the spatial relationship between the vehicle and all surrounding adjacent vehicles.

[0058] In this step, firstly, to assess the reliability of the matching from the perspective of the vehicle's own appearance, it is necessary to calculate the node feature bias. The stable feature vector of the vehicle's lights in the current frame and the predicted node features output by the model are obtained, and the cosine distance between the two is calculated as the node feature bias. The cosine distance ranges from zero to two; the smaller the value, the more consistent the directions of the two vectors are, and the more the vehicle's appearance features match expectations.

[0059] Furthermore, to assess the reliability of the matching from the perspective of spatial relationships, it is necessary to calculate the edge feature bias. For vehicle pairs with spatial edges, the actual spatial edge features between them and the predicted edge features output by the model are obtained, and the Euclidean distance between them is calculated as the edge feature bias. The smaller the Euclidean distance, the more the spatial distance and relative orientation between the two vehicles conform to normal driving patterns.

[0060] Furthermore, in order to integrate the spatial anomaly information of this vehicle with all its neighboring vehicles, the edge feature deviations of all spatial edges associated with the current vehicle target need to be accumulated to obtain the accumulated edge deviation value. When the vehicle target has no spatially adjacent vehicles, the accumulated edge deviation value is zero.

[0061] Finally, the weighted sum of node feature bias and edge bias is calculated to obtain the anomaly score. When the vehicle target is at a moment of sudden light change, although the stable light features are theoretically unaffected by the light change, the matching relationship is unstable at the moment of sudden change. Therefore, the weight of node feature bias in anomaly judgment should be appropriately reduced, and more reliance should be placed on spatial relationships for cross-validation to avoid misjudgment due to subtle fluctuations in feature extraction at the moment of sudden light change.

[0062] For example, taking a three-frame image segment from a nighttime urban expressway checkpoint monitoring scenario as an example, continuing the previous example. The current frame is frame t. Vehicle D matches A, E matches B, and F matches C. The cosine distance between the stable light feature vector of vehicle D and the model's predicted node features is 0.08, and the Euclidean distance between the actual spatial edge features and the predicted edge features of its neighboring vehicle E is 1.70. Vehicle D is not at a moment of sudden light change. The node bias weight is 0.6, the edge bias weight is 0.4, and the matching anomaly score is 0.6 × 0.08. 0.4×1.70 0.728. When vehicle E is at the moment of sudden light change, the node bias weight decreases to 0.2, the edge bias weight increases to 0.8, the node feature bias is 0.12, and the associated spatial edges DE and EF accumulate to 1.70. 2.15 = 3.85, the outlier score is 0.2 × 0.12 0.8×3.85 3.104. The vehicle's F-node feature deviation is 0.06, the cumulative edge deviation is 2.15, it is not at the moment of sudden light change, and the matching anomaly score is 0.6 × 0.06. 0.4×2.15 0.896. The matching anomaly scores for the three vehicles are output for subsequent anomaly detection.

[0063] In summary, this step quantifies the deviation between actual features and the model's predicted expected values ​​in both node and edge dimensions, achieving a comprehensive assessment of the degree of matching anomalies. Compared with existing technologies, this step has the following advantages: First, it expands anomaly detection from a single feature similarity comparison to a dual cross-validation of individual appearance and group spatial relationships, allowing feature fluctuations caused by normal light changes to be eliminated through spatial relationship consistency; second, it dynamically reduces node deviation weights at the moment of light change, relying on spatial relationships for judgment, avoiding false reporting of normal light switching as matching anomalies.

[0064] S500: When the matching anomaly score of the vehicle target exceeds the preset anomaly threshold, it is determined that the cross-frame vehicle matching relationship of the vehicle target is abnormal.

[0065] In this embodiment of the application, after calculating the matching anomaly score of each vehicle target, it is necessary to compare the anomaly score with a preset threshold to determine whether the cross-frame matching relationship of the current frame is abnormal, and to use the stable features of historical normal frames for correction and recovery when abnormal.

[0066] Step S500 in the method provided in this application embodiment includes: The matching anomaly score of each vehicle target is compared with a preset anomaly threshold. When the matching anomaly score of a vehicle target is greater than the preset anomaly threshold, it is determined that the cross-frame vehicle matching relationship of that vehicle target is abnormal. The headlight stability feature vector of that vehicle target in the previous normal frame of the abnormal frame is obtained. The headlight stability feature vector of the previous normal frame is used as the correction headlight stability feature vector, and the headlight stability feature vector of the vehicle target in the abnormal frame is replaced with the correction headlight stability feature vector to re-establish the cross-frame vehicle matching relationship. When the matching anomaly score of a vehicle target is less than or equal to the preset anomaly threshold, it is determined that the cross-frame vehicle matching relationship of that vehicle target is normal. Detailed explanation is as follows: In this embodiment, the preset anomaly threshold refers to the threshold value used to distinguish between normal and abnormal matching states by statistically analyzing the distribution of anomaly scores for each vehicle target in historical normal matching frames. In this embodiment, the mean of the anomaly scores in normal matching frames is approximately 1.0, and the standard deviation is approximately 0.5. Adding three times the standard deviation to the mean yields 2.5, which covers 99.7% of normal samples. The corrected headlight stability feature vector refers to the headlight stability feature vector extracted from the most recent normal frame that did not trigger anomaly determination when the matching anomaly is determined, and used to replace the distorted features in the current abnormal frame.

[0067] In this step, the anomaly score of each vehicle target is first compared with a preset anomaly threshold. When the anomaly score is greater than the preset anomaly threshold, the cross-frame matching relationship of the vehicle target in the current frame is determined to be abnormal; when the anomaly score is less than or equal to the preset anomaly threshold, the matching relationship is determined to be normal. For the vehicle target determined to be abnormal, the most recent normal frame that did not trigger the anomaly is searched forward, and the light stability feature vector of the vehicle in that frame is extracted as a correction light stability feature vector. This correction vector replaces the light stability feature vector in the abnormal frame, and the cross-frame matching calculation is re-executed to restore the normal identity association. Continuing the previous example, the anomaly score of vehicle E, 3.104, exceeds the preset anomaly threshold of 2.5, and is determined to be abnormal. The process is then backtracked to the t-th frame. The light stability feature vector of vehicle E in frame 1 is obtained as the correction light stability feature vector. After replacing the light stability feature vector of vehicle E in frame t, the matching is re-matched and the identity association is restored to normal.

[0068] In summary, this step achieves automatic detection and recovery of matching anomalies by using threshold determination of abnormal scores and backtracking correction based on historical normal features, thus solving the problem of cross-frame matching errors caused by feature jumps in nighttime lighting abrupt changes.

[0069] The embodiments of this application, through the specific implementation methods described above, achieve the following technical effects: This application proposes an image anomaly matching and detection method based on multi-layer graph neural networks. It realizes physical decomposition of the appearance features of nighttime vehicle targets, cross-frame matching and graph structure construction, spatiotemporal graph neural network prediction, multi-dimensional anomaly score calculation, and anomaly judgment and correction. Through vehicle detection and light stability feature extraction, Hungarian global optimal matching and intra-frame spatial relationship graph construction, spatial graph convolutional bidirectional aggregation and temporal graph convolutional evolutionary coding, node bias and edge bias weighted fusion and backtracking correction, it effectively improves the technical problem in nighttime traffic monitoring where sudden changes in vehicle light cause drastic jumps in appearance features, and traditional matching methods are unable to distinguish between normal light changes and matching anomalies.

[0070] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

Claims

1. An image anomaly matching detection method based on a multi-layer graph neural network, characterized in that, The method includes: Acquire continuous image frames captured by a fixed-view surveillance camera during nighttime, detect vehicle targets in each frame, and physically decompose the appearance features of each vehicle target into light-stable features and light-sensitive features. Based on the similarity between the light stabilization features of each vehicle target in the current frame and the light stabilization features of each tracked vehicle target in the previous frame, cross-frame vehicle matching relationships are established, and vehicle temporal association graphs and spatial relationship graphs are constructed. The vehicle temporal correlation graph and spatial relationship graph of multiple consecutive frames are input into the spatiotemporal graph neural network model constructed based on multi-layer graph neural network. The spatiotemporal graph neural network model performs spatiotemporal encoding on the light stability features and spatial relationships of each vehicle target, and outputs the predicted node features and predicted edge features of each vehicle target in the current frame. Based on the cross-frame vehicle matching relationship of the current frame, the predicted node features, and the predicted edge features, calculate the matching anomaly score for each vehicle target; When the matching anomaly score of a vehicle target exceeds a preset anomaly threshold, it is determined that the cross-frame vehicle matching relationship of the vehicle target is abnormal.

2. The image anomaly matching detection method based on a multi-layer graph neural network according to claim 1, characterized in that, Acquire continuous image frames from a fixed-view surveillance camera during nighttime, and detect vehicle targets in each frame, including: A series of image frames are acquired from a fixed-view surveillance camera during nighttime. Each image frame is input into a vehicle detection network, which outputs the bounding box of each vehicle target in each image frame. The bounding box contains the position coordinates, width, and height of the vehicle target in the image. For each vehicle target, the corresponding vehicle image region is cropped from the image frame based on the bounding box.

3. The image anomaly matching detection method based on a multi-layer graph neural network according to claim 2, characterized in that, The physical decomposition of the appearance features of each vehicle target into light-stabilized features and light-sensitive features includes: For each vehicle target's vehicle image region, the vehicle structure contour is extracted using an edge detection algorithm, and the window region and roof contour region are extracted by combining prior knowledge of the vehicle shape. The vehicle structure contour, the window region, and the roof contour region are collectively used as the lighting stabilization region. The bright areas in the vehicle image area whose brightness values ​​exceed a preset brightness threshold are extracted as the headlight self-illuminating areas. The areas in the vehicle image area other than the light stable areas and the headlight self-illuminating areas are taken as the vehicle body reflection areas. The headlight self-illuminating areas and the vehicle body reflection areas are taken as the light sensitive areas. A directional gradient histogram is extracted from the vehicle structure contour in the light-stabilized region. The directional gradient histogram is used as the vehicle structure shape descriptor. The normalized relative position of the center coordinates of the window region in the light-stabilized region and the vehicle bounding box is calculated as the window relative position code. The vehicle structure shape descriptor and the window relative position code are used together as the light-stabilized feature vector. A luminance distribution histogram and a color histogram are statistically analyzed for the light-sensitive area, and the luminance distribution histogram and the color histogram are used together as a light-sensitive feature vector; Calculate the inter-frame change between the light-sensitive feature vector of each vehicle target in the current frame and the light-sensitive feature vector of the same vehicle target in the previous frame. When the inter-frame change exceeds a preset abrupt change threshold, it is determined that the vehicle target is at a moment of light abrupt change.

4. The image anomaly matching detection method based on a multi-layer graph neural network according to claim 1, characterized in that, Based on the similarity between the light stabilization features of each vehicle target in the current frame and the light stabilization features of each tracked vehicle target in the previous frame, cross-frame vehicle matching relationships are established, and vehicle temporal association graphs and spatial relationship graphs are constructed, including: Calculate the cosine similarity between the light-stabilized feature vector of each vehicle target in the current frame and the light-stabilized feature vector of each vehicle target that has been tracked in the previous frame; For each vehicle target in the current frame, the tracked vehicle target with the highest cosine similarity is selected as the matching target, and a cross-frame vehicle matching relationship is established between the vehicle target and the matching target in the current frame. When multiple vehicle targets match the same tracked vehicle target, a global optimal matching algorithm is used to globally optimize the matching results, eliminate matching conflicts, and obtain the final cross-frame vehicle matching relationship. Each vehicle target in each frame is used as a graph node, and the cross-frame vehicle matching relationship is used as a temporal edge to connect the graph nodes corresponding to the same vehicle target in the previous and next frames, thus forming a vehicle temporal association graph. For all vehicle targets in each frame, calculate the spatial distance and relative azimuth between any two vehicle targets, and use the spatial distance and relative azimuth as spatial edge features to construct a spatial relationship graph.

5. The image anomaly matching detection method based on a multi-layer graph neural network according to claim 4, characterized in that, When multiple vehicle targets match the same tracked vehicle target, a global optimal matching algorithm is used to globally optimize the matching results, eliminate matching conflicts, and obtain the final cross-frame vehicle matching relationship, including: The cosine similarity between all vehicle targets in the current frame and all tracked vehicle targets in the previous frame is used to construct a similarity matrix; With the goal of maximizing the sum of cosine similarities of all matching pairs, the Hungarian algorithm is used to solve the global optimal matching problem on the similarity matrix, thereby obtaining the one-to-one cross-frame vehicle matching relationship between each vehicle target in the current frame and the tracked vehicle target in the previous frame.

6. The image anomaly matching detection method based on a multi-layer graph neural network according to claim 4, characterized in that, For all vehicle targets within each frame, calculate the spatial distance and relative azimuth between any two vehicle targets. Use the spatial distance and relative azimuth as spatial edge features to construct a spatial relationship graph, including: Obtain the coordinates of the center points of the bounding boxes of two vehicle targets, and calculate the Euclidean distance between the two center point coordinates as the spatial distance between the two vehicle targets; Calculate the direction angle of the line connecting the coordinates of the two center points, and use it as the relative azimuth angle between the two vehicle targets; When the spatial distance is less than or equal to a preset adjacency distance threshold, a spatial edge is established between the two vehicle targets. The spatial distance and the relative azimuth angle are used as the spatial edge features of the spatial edge. The spatial edges between all vehicle target pairs that satisfy the preset adjacency distance threshold are combined to form a spatial relationship diagram.

7. The image anomaly matching detection method based on a multi-layer graph neural network according to claim 1, characterized in that, The temporal correlation graph and spatial relationship graph of vehicles from multiple consecutive frames are input into a spatiotemporal graph neural network model constructed based on a multi-layer graph neural network. The spatiotemporal graph neural network model performs spatiotemporal encoding on the lighting stability features and spatial relationships of each vehicle target, and outputs the predicted node features and predicted edge features of each vehicle target in the current frame, including: Input the vehicle temporal correlation graph and spatial relationship graph of the consecutive frames before the current frame into the spatiotemporal graph neural network model constructed based on a multi-layer graph neural network; The spatial graph convolutional layer of the spatiotemporal graph neural network model performs graph convolution operation on the spatial relationship graph of each frame. The spatial graph convolutional layer performs bidirectional feature aggregation on two vehicle targets with spatial edges, aggregating the light stability features of spatially adjacent vehicle targets of each vehicle target to the vehicle target, and simultaneously aggregating the light stability features of the vehicle target to the spatially adjacent vehicle targets, and outputting the spatial context feature vector of each vehicle target. The temporal graph convolutional layer of the spatiotemporal graph neural network model performs temporal convolution operations on the spatial context feature vectors of the same vehicle target in multiple consecutive frames, and outputs the predicted node features and predicted edge features of each vehicle target in the current frame.

8. The image anomaly matching detection method based on a multi-layer graph neural network according to claim 7, characterized in that, The pre-construction process of the spatiotemporal graph neural network model includes: Acquire a historical video dataset of nighttime traffic monitoring, perform vehicle target detection and physical decomposition of appearance features on continuous image frames in the historical video dataset, and construct a historical vehicle temporal association graph and a historical spatial relationship graph. For each frame in the aforementioned historical video dataset of nighttime traffic monitoring, the stable light feature vector and spatial edge feature of each vehicle target in the frame under normal matching conditions are obtained to form a training sample label set; Based on a multi-layer graph neural network, an initial spatiotemporal graph neural network model is constructed, which includes a spatial graph convolutional layer and a temporal graph convolutional layer. Using the historical vehicle temporal correlation graph and the historical spatial relationship graph as input, and the light stability feature vector and spatial edge feature of each vehicle target in the next frame in the training sample label set as output labels, the training objective is to minimize the weighted sum of the node feature prediction error between the predicted node feature and the light stability feature vector and the edge feature prediction error between the predicted edge feature and the spatial edge feature. The initial spatiotemporal graph neural network model is subjected to supervised training until the verification convergence is obtained, thus obtaining the trained spatiotemporal graph neural network model.

9. The image anomaly matching detection method based on a multi-layer graph neural network according to claim 1, characterized in that, Based on the cross-frame vehicle matching relationship of the current frame, the predicted node features, and the predicted edge features, a matching anomaly score for each vehicle target is calculated, including: For each vehicle target in the current frame, obtain the stable light feature vector of the vehicle target and the predicted node features output by the spatiotemporal graph neural network model, and calculate the cosine distance between the stable light feature vector and the predicted node features as the node feature deviation. For each pair of vehicle targets with spatial edges in the current frame, the spatial edge features between them and the predicted edge features output by the spatiotemporal graph neural network model are obtained. The Euclidean distance between the spatial edge features and the predicted edge features is calculated as the edge feature deviation. The edge feature deviations of all spatial edges associated with each vehicle target are accumulated to obtain the accumulated edge deviation value of the vehicle target. The node feature deviations and the accumulated edge deviation value are weighted and summed to obtain the matching anomaly score of the vehicle target. When the vehicle target is at a moment of sudden change in lighting, the weight of the node feature deviations is reduced.

10. The image anomaly matching detection method based on a multi-layer graph neural network according to claim 1, characterized in that, When the matching anomaly score of a vehicle target exceeds a preset anomaly threshold, it is determined that the cross-frame vehicle matching relationship of the vehicle target is abnormal, including: The anomaly score of each vehicle target is compared with a preset anomaly threshold; When the matching anomaly score of a vehicle target is greater than the preset anomaly threshold, it is determined that the cross-frame vehicle matching relationship of the vehicle target is abnormal. The light stability feature vector of the vehicle target in the previous normal frame of the abnormal frame is obtained. The light stability feature vector of the previous normal frame is used as the correction light stability feature vector. The correction light stability feature vector is used to replace the light stability feature vector of the vehicle target in the abnormal frame, and the cross-frame vehicle matching relationship is re-established. When the matching anomaly score of a vehicle target is less than or equal to the preset anomaly threshold, the cross-frame vehicle matching relationship of the vehicle target is determined to be normal.

Images