A method for identifying whether a vehicle is pressing a line based on a graph neural network

Abstract

The application discloses a kind of methods for identifying whether vehicle is pressed line based on graph neural network, collect a large number of various traffic scenes under various angle shooting image, vehicle and lane line solid line, and pressed line condition mark;Based on image and mark result training image instance segmentation algorithm and graph neural network algorithm;Using the trained image instance segmentation algorithm and the graph neural network algorithm in deep learning identifies whether vehicle is pressed line, wherein, image instance segmentation algorithm is used for the detection and identification of vehicle and lane line solid line, graph neural network outputs the probability of vehicle pressed line, when the probability of vehicle pressed line exceeds preset threshold, output pressed line vehicle node and lane line node.The application can improve the accuracy of line detection, high robustness, avoids the determination mode of traditional rule setting according to different scenes based on coordinate point, more intelligent.

Description

Technical Field

[0001] This invention belongs to the field of vehicle line crossing detection technology, specifically relating to a method for identifying whether a vehicle has crossed a line based on a graph neural network. Background Technology

[0002] Currently, the mainstream method for determining vehicle lane crossing behavior is through images and videos. Manually locating whether a vehicle has crossed the lane from massive amounts of video data is extremely time-consuming and labor-intensive. Therefore, the current solution to locating vehicle lane crossings from massive amounts of video often employs computer vision technology for lane crossing event detection. Existing solutions using computer vision technology for vehicle lane crossing event detection include:

[0003] Solution 1: By automatically detecting key points of the vehicle (such as tire points, headlight and taillight points, etc.) and lane lines in the image, the algorithm can determine the positional relationship between the vehicle chassis and the lane lines, thereby determining whether the vehicle is crossing the lines. Examples include invention patents with publication numbers CN111402329A and CN113962249A.

[0004] Solution 2: By fixing the position of the shooting device and the position of the lane lines in the image, the algorithm can perform semantic segmentation of the vehicle. Then, through estimation and fitting, the vehicle chassis position is obtained based on the semantic segmentation results. Finally, the estimated vehicle chassis position and the preset lane line position are used to calculate whether the vehicle has crossed the lane line. See the invention patent with publication number CN111814765A.

[0005] The above solution has the following problems:

[0006] Question 1: In Solution 1, the invention patent with publication number CN111402329A locates key points of the vehicle's wheels by using the contour texture information of the tires and wheels, thereby determining the vehicle's chassis position. This method belongs to traditional image processing methods and is greatly affected by lighting, shooting angle, obstacle occlusion, vehicle proximity, and vehicle type, thus causing significant errors in the final line-crossing detection results. Although the invention patent with publication number CN113962249A uses a large amount of key point image data to learn and predict neural networks for key points such as tire points and headlight / rear headlight points, this solution does not clearly define the hierarchical relationship of key points of multiple vehicles when dealing with key points of other vehicles in the vehicle area when the vehicle is close to the vehicle. This can easily lead to key point information detected in a vehicle area belonging to multiple vehicles, resulting in errors in detecting line crossings.

[0007] Problem 2: In Solution 2, although the position and size of vehicles in the image are located through semantic segmentation, the semantic segmentation method will cause the segmentation results of different vehicles to be connected, making it impossible to distinguish the specific vehicle crossing the line. In addition, the arithmetic estimation of the vehicle chassis using the semantic segmentation results is easily affected by the vehicle type and traffic flow. Furthermore, the fixed shooting equipment and fixed lane lines are easily affected by vibration and video quality fluctuations, all of which will lead to errors in detecting crossing the line. At the same time, this method is no longer applicable when the angle of the shooting equipment changes, and has low robustness.

[0008] Question 3: In Scheme 1 and Scheme 2, lane line detection and vehicle detection are two independent workflows. The semantic context relationship between lane lines and vehicles in the images is not captured by the model. This problem will cause lane line detection errors when vehicles obscure the lane lines.

[0009] Question 4: In Schemes 1 and 2, the determination of lane lines and whether a vehicle has crossed the lines relies solely on manually set rules between coordinate points. These rules are not universally applicable across different scenarios and require manual adjustment of the rule parameters based on the specific scene or camera angle. This method of judgment is not intelligent enough. Summary of the Invention

[0010] The technical problem this invention aims to solve is to address the shortcomings of existing technologies by providing a method for identifying whether a vehicle is crossing lane lines based on a graph neural network. This method employs customized data annotation and a customized algorithm to identify lane line violations. The customized algorithm refers to an end-to-end algorithm that fuses a customized graph neural network and a segmentation network. This invention can segment each vehicle and each solid lane line instance in an image by learning from a large amount of customized labeled data, and then predict the pairwise relationship (crossing or not crossing) between each vehicle and each lane line using a graph neural network. This avoids lane line detection failures caused by errors in keypoint estimation, and also avoids the problem of insufficient robustness to environmental factors in traditional algorithms. Furthermore, using the output probability results of the graph neural network to determine lane line violations avoids the traditional coordinate point-based rule-based method, making it more intelligent. In addition, since this algorithm is an end-to-end algorithm that fuses a graph neural network and a segmentation network, the optimization process of the graph neural network helps the segmentation network learn the contextual semantic relationship between vehicles and lane lines more accurately, thus improving the performance of the segmentation network.

[0011] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0012] A method for identifying whether a vehicle has crossed a lane line based on a graph neural network includes:

[0013] Step 1: Collect a large number of images taken from various angles in various traffic scenarios, and mark the vehicles and lane lines, as well as the instances of vehicles crossing the lines.

[0014] Step 2: Train the image instance segmentation algorithm and the graph neural network algorithm based on the image and annotation results from Step 1;

[0015] Step 3: Use the trained image instance segmentation algorithm and the graph neural network algorithm in deep learning to identify whether a vehicle is crossing the line. The image instance segmentation algorithm is used to detect and identify vehicles and solid lane lines. The graph neural network outputs the probability of a vehicle crossing the line. When the probability of a vehicle crossing the line exceeds a preset threshold, it outputs the vehicle node and lane line node that are crossing the line.

[0016] To optimize the above technical solution, the specific measures also include:

[0017] Step one above includes:

[0018] S1. Collect a large number of images taken from various angles in various traffic scenarios, including highways, tunnels, elevated roads, national highways, provincial highways, county roads, and urban road scenarios;

[0019] S2. Label each visible vehicle and each visible lane line in the image collected in S1, and label the pairwise line crossing relationship between each labeled vehicle and each labeled lane line.

[0020] S3. Store the annotation results from S2 and their corresponding images in a fixed format for use in training subsequent algorithms.

[0021] Step two above includes:

[0022] S1. Read the labeled data and corresponding images;

[0023] S2. Input the image into the backbone network of the instance segmentation algorithm model to obtain the feature map;

[0024] S3. Input the feature map into the object detection branch to predict the vehicle target box and the solid line target box of the lane line;

[0025] S4. Calculate the error between the predicted target boxes of vehicles and solid lane lines in S3 and the labeled target boxes in the data read in S1, denoted as L. obj and L cls , where represent the bounding box loss function and the bounding box category loss function, respectively;

[0026] S5. Input the feature map output by the target detection branch into the segmentation branch to perform position instance segmentation of vehicles and lanes in the target box, and obtain the segmentation prediction results of vehicles and lane lines in the target box.

[0027] S6. Calculate the error between the instance segmentation prediction results of vehicles and solid lane lines in S5 and the labeled mask read in S1, denoted as L. mask , represents the loss function for instance segmentation prediction results;

[0028] S7. Construct an adjacency matrix based on the labeled data in S1. The value of its adjacency matrix This represents the line overlap relationship between the i-th vehicle or lane line and the j-th vehicle or lane line;

[0029] S8. Aggregate all feature vectors within the mask region of the feature map in S2 corresponding to the mask markings of vehicle and lane line positions in S1 to obtain feature vector matrix E, containing N. car +N line E eigenvectors i E i Prediction map of vehicle and lane line crossing Nodes in;

[0030] S9, N in S8 car +N line The feature vectors are input to the graph neural network F(θ,E), where θ represents the parameters of the graph neural network, and the input is the feature vector matrix E obtained in S8. The output of the graph neural network is denoted as E. out ;

[0031] S10. Transpose the output of the graph neural network obtained in S9 to obtain...

[0032] S11, replace E in S9 out and S10 Perform matrix multiplication: Obtain vehicle and lane line crossing prediction map adjacency matrix

[0033] S12, Calculate the predicted adjacency matrix in S11. Adjacency matrix in S7 Calculate the loss function, denoted as L. graph ;

[0034] S13. Calculate the total loss function L total =L graph +L mask +L obj +L cls With the total loss function L totalThe minimum parameter θ in the target-updating graph neural network F(θ,E), as well as the parameters in the segmentation branch, the target detection branch, and the backbone network, are updated.

[0035] S14. Store the optimized model structure and parameters from S13 for use in algorithm prediction.

[0036] Specifically, S7 above involves constructing an adjacency matrix based on the labeled data in S1, and first converting the relational information of instances in the labels into an undirected acyclic graph. In this graph, connected nodes represent nodes with overlapping lines; an adjacency matrix is ​​then constructed from this undirected acyclic graph, denoted as . The size of this adjacency matrix is ​​(N car +N line )*(N car +N line );

[0037] Where, N car N represents the number of vehicles labeled in the labeled data. line This indicates the number of solid lane lines marked in the labeled data;

[0038] And the value of this adjacency matrix This represents the line overlap relationship between the i-th vehicle or lane line and the j-th vehicle or lane line;

[0039] The S8 mentioned above specifically refers to:

[0040] Aggregate all feature vectors within the mask region of the feature map in S2, corresponding to the masks that mark the positions of vehicles and lane lines in S1. Each mask that marks a vehicle or lane line corresponds to a feature vector E. i Finally, the eigenvector matrix E is obtained;

[0041] Wherein, the feature vector E i For prediction graph The nodes in the diagram represent the predicted feature vectors corresponding to the i-th vehicle or lane line at the labeled location, totaling N. car +N line indivual;

[0042] In S13 above, the total loss function L total With the goal of minimizing, the gradient descent method is used to update the parameters θ in the graph neural network F(θ,E), as well as the parameters in the segmentation branch, object detection branch, and backbone network.

[0043] Step three above specifically includes:

[0044] S1. Load the trained image instance segmentation algorithm and the graph neural network algorithm in deep learning;

[0045] S2. Input the image into the backbone network of the instance segmentation algorithm model to obtain the predicted feature map;

[0046] S3. Input the predicted feature map into the target detection branch to predict the vehicle target box and the solid line target box of the lane line;

[0047] S4. Input the feature map output by the target detection branch into the segmentation branch of the instance segmentation algorithm model to perform position instance segmentation of vehicles and lanes in the target box, and obtain the segmentation prediction results of vehicles and lane lines in the target box.

[0048] S5. Aggregate all feature vectors within the mask region of the feature map in S2 corresponding to the segmentation results of the predicted vehicle and lane line positions in S4. For each labeled vehicle or lane line in the segmentation prediction results, a feature vector is obtained corresponding to the mask. This represents the feature vector corresponding to the i-th vehicle or lane line in the predicted mask region, totaling [number missing]. indivual;

[0049] S6, from S5 The feature vectors are input into the graph neural network F(θ,E), and the output of the graph neural network is calculated and denoted as E. out ;

[0050] S7. Transpose the output of the graph neural network obtained in S6 to obtain...

[0051] S8, change E in S6 out And in S7 Perform matrix multiplication Obtain the prediction matrix of the relationship between the vehicle and the lane line.

[0052] S9, in Iterate through all vehicle nodes and obtain the probability of crossing the lane line between each vehicle node and the lane line nodes connected to it.

[0053] S10. Determine whether each vehicle has crossed the line based on the probability of crossing the line and a set threshold.

[0054] S11. If the probability of crossing the line exceeds the preset threshold, output the vehicle node and lane line node that are crossing the line.

[0055] The present invention has the following beneficial effects:

[0056] 1. When multiple vehicles are close together and occlude each other in the image, the line-crossing detection method of the present invention can avoid the failure of line-crossing detection due to unclear key point relationships and key point detection errors, thereby improving the accuracy of line-crossing detection.

[0057] 2. This invention has no requirements for image shooting angle or fixed camera, and has strong adaptability to environmental factors such as image vibration, light changes, and occlusion. The method of this invention has high robustness.

[0058] 3. During algorithm training, the detection and recognition of lane lines and vehicles in this invention are both carried out in a "segment-to-end" workflow. This approach can capture the semantic contextual relationship between lane lines and vehicles in the image, thereby improving the accuracy of lane line segmentation and vehicle segmentation.

[0059] 4. The output probability results of the graph neural network are used to determine whether the line is crossed, which avoids the traditional method of setting rules based on coordinate points according to different scenarios, making it more intelligent. Attached Figure Description

[0060] Figure 1 This is a customized data annotation flowchart for the present invention;

[0061] Figure 2 This is a flowchart of the algorithm training part in the customized algorithm of this invention;

[0062] Figure 3 This is a flowchart of the algorithm prediction part in the customized algorithm of this invention;

[0063] Figure 4 This is a schematic diagram of the mask annotation method of the present invention;

[0064] Figure 5 This is a schematic diagram of the target bounding box annotation method of the present invention;

[0065] Figure 6 This is a schematic diagram illustrating the mask labeling storage content of an example of the present invention;

[0066] Figure 7 This is a schematic diagram illustrating the storage content of the target box annotation in this invention;

[0067] Figure 8 This is a schematic diagram illustrating the storage content of the pressure line relationship between various embodiments of the present invention;

[0068] Figure 9 This is a schematic diagram of the training steps of the algorithm of the present invention;

[0069] Figure 10 This is a schematic diagram of the prediction steps of the algorithm of the present invention;

[0070] Figure 11This is the flowchart of the method for identifying whether a vehicle is crossing the line based on a graph neural network according to the present invention. Detailed Implementation

[0071] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0072] This invention employs customized data annotation and a customized algorithm to identify whether a vehicle is crossing a lane line. The customized data annotation provides training data samples for the customized algorithm; the customized algorithm, through learning and training on a large amount of labeled data, develops the ability to automatically detect lane line violations in real-world environments. The customized algorithms mainly include image instance segmentation algorithms from the field of computer vision and graph neural network algorithms from deep learning. For details, see [link to documentation]. Figure 1-11 The present invention discloses a method for identifying whether a vehicle has crossed a lane line based on a graph neural network, comprising the following steps:

[0073] Step 1: Collect a large number of images taken from various angles in various traffic scenarios, and mark the vehicles and lane lines, as well as the instances of vehicles crossing the lines.

[0074] Step 2: Train the image instance segmentation algorithm and the graph neural network algorithm based on the image and annotation results from Step 1;

[0075] Step 3: Use the trained image instance segmentation algorithm and the graph neural network algorithm in deep learning to identify whether a vehicle is crossing the line. The image instance segmentation algorithm is used to detect and identify vehicles and solid lane lines. The graph neural network outputs the probability of a vehicle crossing the line. When the probability of a vehicle crossing the line exceeds a preset threshold, it outputs the vehicle node and lane line node that are crossing the line.

[0076] In a specific embodiment, the customized data annotation, i.e., step one, includes the following steps, which can be referred to in the accompanying drawings. Figure 1 Step-by-step flowchart:

[0077] S1. Collect a large number of images taken from various angles in various traffic scenarios, including but not limited to highways, tunnels, elevated roads, national highways, provincial highways, county roads, and urban roads.

[0078] S2. Manually label each visible vehicle and each visible lane line solid line in the image collected in S1 using labeling software running on a microcomputer, but not limited to a microcomputer, and label the pairwise line crossing relationship between each labeled vehicle and each labeled lane line solid line.

[0079] S3. Store the annotation results from S2 and their corresponding images in a fixed format on a storage device for use in training the customized algorithm.

[0080] In a specific embodiment, the customized algorithm includes algorithm training and algorithm prediction. The steps for algorithm training and algorithm prediction are as follows, which can be referred to separately. Figure 2 and Figure 3 Step-by-step flowchart:

[0081] In a specific embodiment, the algorithm training step in the customized algorithm, i.e., step two, is as follows, which can be referred to. Figure 2 Step-by-step flowchart:

[0082] S1. Read the annotation data and corresponding images from the storage device;

[0083] S2. Input the image into the backbone network of the instance segmentation algorithm model to obtain the calculation result of the backbone network, which is called the feature map;

[0084] S3. Input the feature map into the object detection branch to predict the vehicle target box and the solid line target box of the lane line;

[0085] S4. Calculate the error between the predicted target boxes of vehicles and solid lane lines in S3 and the labeled target boxes in the data read in S1, denoted as L. obj and L cls , where represent the bounding box loss function and the bounding box category loss function, respectively;

[0086] S5. Input the feature map output by the target detection branch into the branch part of the instance segmentation algorithm model to perform position instance segmentation of vehicles and lanes in the target box, and obtain the segmentation prediction results of vehicles and lane lines in the target box.

[0087] S6. Calculate the error between the instance segmentation prediction results of vehicles and solid lane lines in S5 and the labeled mask read in S1, denoted as L. mask , represents the loss function for instance segmentation prediction results;

[0088] S7. Construct an adjacency matrix based on the labeled data in S1. First, convert the relationship information of the instances in the labels into an undirected acyclic graph. The connected nodes in this diagram indicate that they are connected by a line.

[0089] Then construct an adjacency matrix from this undirected acyclic graph, denoted as . The size of this adjacency matrix is ​​(N car +N line )*(N car +N line ),;

[0090] Where, N car N represents the number of vehicles marked in S1. lineThis indicates the number of solid lane lines marked in S1;

[0091] And the value of this adjacency matrix This represents the line overlap relationship between the i-th vehicle or lane line and the j-th vehicle or lane line.

[0092] S8. Aggregate all feature vectors within the mask region of the feature map in S2, corresponding to the masks that mark the positions of vehicles and lane lines in S1. Each mask that marks a vehicle or lane line corresponds to a feature vector E. i Finally, the eigenvector matrix E is obtained;

[0093] Wherein, the feature vector E i For prediction graph The nodes in the diagram represent the predicted feature vectors corresponding to the i-th vehicle or lane line at the labeled location, totaling N. car +N line indivual;

[0094] S9. The N mentioned in S8 car +N line The feature vectors are input to the graph neural network F(θ,E), where θ represents the parameters of the graph neural network, and the input is the feature vector matrix E obtained in S8. The output of the graph neural network is denoted as E. out ;

[0095] S10. Transpose the output of the graph neural network obtained in S9 to obtain...

[0096] S11, replace E in S9 out and S10 Perform matrix multiplication: Obtain vehicle and lane line crossing prediction map adjacency matrix

[0097] S12, Calculate the predicted adjacency matrix in S11. Adjacency matrix in S7 Calculate the loss function, denoted as L. graph ;

[0098] S13. Calculate the total loss function L total =L graph +L mask +L obj +L cls With the total loss function L total With the goal of minimizing, gradient descent is used to update the parameters θ in F(θ,E), as well as the parameters in the segmentation branch, object detection branch, and Mask-R-CNN backbone network.

[0099] S14. Store the optimized model structure and model parameters from S13 to a storage device for use in algorithm prediction.

[0100] In a specific embodiment, the algorithm prediction step, specifically step three, is as follows, which can be referenced. Figure 3 Step-by-step flowchart:

[0101] S1. Load the model structure and model parameters stored in S14 of the algorithm training section to the storage device;

[0102] S2. Input the image into the backbone network of the instance segmentation algorithm model to obtain the predicted feature map;

[0103] S3. Input the predicted feature map into the target detection branch to predict the vehicle target box and the solid line target box of the lane line;

[0104] S4. Input the feature map output by the target detection branch into the segmentation branch of the instance segmentation algorithm model to perform position instance segmentation of vehicles and lanes in the target box, and obtain the segmentation prediction results of vehicles and lane lines in the target box.

[0105] S5. Aggregate all feature vectors within the mask region of the feature map in S2 corresponding to the segmentation results of the predicted vehicle and lane line positions in S4. For each labeled vehicle or lane line in the segmentation prediction results, a feature vector is obtained corresponding to the mask. This represents the feature vector corresponding to the i-th vehicle or lane line in the predicted mask region, totaling [number missing]. indivual;

[0106] S6, from S5 The feature vectors are input into the graph neural network F(θ,E), and the output of the graph neural network is calculated and denoted as E. out ;

[0107] S7. Transpose the output of the graph neural network obtained in S6 to obtain...

[0108] S8, change E in S6 out And in S7 Perform matrix multiplication Obtain the prediction matrix of the relationship between the vehicle and the lane line.

[0109] S9, in Iterate through all vehicle nodes and obtain the probability of crossing the lane line between each vehicle node and the lane line nodes connected to it.

[0110] S10. Determine whether each vehicle has crossed the line based on the probability of crossing the line and a set threshold.

[0111] S11. If the probability of crossing the line exceeds the preset threshold, output the vehicle node and lane line node that are crossing the line.

[0112] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for identifying whether a vehicle has crossed a lane line based on a graph neural network, characterized in that, include: Step 1: Collect a large number of images taken from various angles in various traffic scenarios, and mark the vehicles and lane lines, as well as the instances of vehicles crossing the lines. Step 2: Based on the images and annotation results from Step 1, train the image instance segmentation algorithm and the graph neural network algorithm, including: S1. Read the annotation data and corresponding images; S2. Input the image into the backbone network of the instance segmentation algorithm model to obtain the feature map; S3. Input the feature map into the object detection branch to predict the vehicle target box and the solid line target box of the lane line; S4. Calculate the error between the predicted target boxes of vehicles and solid lane lines in S3 and the labeled target boxes in the data read in S1, denoted as L. obj and L cls , where represent the bounding box loss function and the bounding box category loss function, respectively; S5. Input the feature map output by the target detection branch into the segmentation branch to perform position instance segmentation of vehicles and lanes in the target box, and obtain the segmentation prediction results of vehicles and lane lines in the target box. S6. Calculate the error between the instance segmentation prediction results of vehicles and solid lane lines in S5 and the labeled mask read in S1, denoted as L. mask , represents the loss function for instance segmentation prediction results; S7. Construct an adjacency matrix based on the labeled data in S1. The value of its adjacency matrix This represents the line overlap relationship between the i-th vehicle or lane line and the j-th vehicle or lane line; S8. Aggregate all feature vectors within the mask region of the feature map in S2 corresponding to the mask markings of vehicle and lane line positions in S1 to obtain feature vector matrix E, containing N. car +N line E eigenvectors i E i Prediction map of vehicle and lane line crossing The nodes in, where N car N represents the number of vehicles labeled in the labeled data. line This indicates the number of solid lane lines marked in the labeled data; S9, N in S8 car +N line The feature vectors are input to the graph neural network F(θ,E), where θ represents the parameters of the graph neural network, and the input is the feature vector matrix E obtained in S8. The output of the graph neural network is denoted as E. out ; S10. Transpose the output of the graph neural network obtained in S9 to obtain... S11, replace E in S9 out and S10 Perform matrix multiplication: Obtain vehicle and lane line crossing prediction map adjacency matrix S12, Calculate the predicted adjacency matrix in S11. Adjacency matrix in S7 Calculate the loss function, denoted as L. grap h; S13. Calculate the total loss function L total =L grap h+L mask +L obj +L cls With the total loss function L total The minimum parameter θ in the target-updating graph neural network F(θ,E), as well as the parameters in the segmentation branch, the target detection branch, and the backbone network, are updated. S14. Store the optimized model structure and parameters from S13 for use in algorithm prediction; Step 3: Use the trained image instance segmentation algorithm and the graph neural network algorithm in deep learning to identify whether a vehicle is crossing the line. The image instance segmentation algorithm is used to detect and identify vehicles and solid lane lines. The graph neural network outputs the probability of a vehicle crossing the line. When the probability of a vehicle crossing the line exceeds a preset threshold, it outputs the vehicle node and lane line node that are crossing the line.

2. The method for identifying whether a vehicle has crossed a lane based on a graph neural network according to claim 1, characterized in that, Step one includes: S1. Collect a large number of images taken from various angles in various traffic scenarios, including highways, tunnels, elevated roads, national highways, provincial highways, county roads, and urban road scenarios; S2. Label each visible vehicle and each visible lane line in the image collected in S1, and label the pairwise line crossing relationship between each labeled vehicle and each labeled lane line. S3. Store the annotation results from S2 and their corresponding images in a fixed format for use in training subsequent algorithms.

3. The method for identifying whether a vehicle has crossed a lane based on a graph neural network according to claim 1, characterized in that, Specifically, S7 involves: constructing an adjacency matrix based on the labeled data in S1, and first converting the relationship information of instances in the labels into an undirected acyclic graph. In this graph, connected nodes represent nodes with overlapping lines; an adjacency matrix is ​​then constructed from this undirected acyclic graph, denoted as . The size of this adjacency matrix is ​​(N car +N line )*(N car +N line ); Where, N car N represents the number of vehicles labeled in the labeled data. line This indicates the number of solid lane lines marked in the labeled data; And the value of this adjacency matrix This represents the line overlap relationship between the i-th vehicle or lane line and the j-th vehicle or lane line.

4. The method for identifying whether a vehicle has crossed a lane based on a graph neural network according to claim 1, characterized in that, Specifically, S8 is: Aggregate all feature vectors within the mask region of the feature map in S2, corresponding to the masks that mark the positions of vehicles and lane lines in S1. Each mask that marks a vehicle or lane line corresponds to a feature vector E. i Finally, the eigenvector matrix E is obtained; Wherein, the feature vector E i For prediction graph The nodes in the diagram represent the predicted feature vectors corresponding to the i-th vehicle or lane line at the labeled location, totaling N. car +N line indivual.

5. The method for identifying whether a vehicle has crossed a lane based on a graph neural network according to claim 1, characterized in that, In S13, the total loss function L total With the goal of minimizing, the gradient descent method is used to update the parameters θ in the graph neural network F(θ,E), as well as the parameters in the segmentation branch, object detection branch, and backbone network.

6. The method for identifying whether a vehicle has crossed a lane based on a graph neural network according to claim 1, characterized in that, Step three specifically includes: S1. Load the trained image instance segmentation algorithm and the graph neural network algorithm in deep learning; S2. Input the image into the backbone network of the instance segmentation algorithm model to obtain the predicted feature map; S3. Input the predicted feature map into the target detection branch to predict the vehicle target box and the solid line target box of the lane line; S4. Input the feature map output by the target detection branch into the segmentation branch of the instance segmentation algorithm model to perform position instance segmentation of vehicles and lanes in the target box, and obtain the segmentation prediction results of vehicles and lane lines in the target box. S5. Aggregate all feature vectors within the mask region of the feature map in S2 corresponding to the segmentation results of the predicted vehicle and lane line positions in S4. For each labeled vehicle or lane line in the segmentation prediction results, a feature vector is obtained corresponding to the mask. This represents the feature vector corresponding to the i-th vehicle or lane line in the predicted mask region, totaling [number]. indivual; S6, from S5 The feature vectors are input into the graph neural network F(θ,E), and the output of the graph neural network is calculated and denoted as E. out ; S7. Transpose the output of the graph neural network obtained in S6 to obtain... S8, change E in S6 out And in S7 Perform matrix multiplication Obtain the prediction matrix of the relationship between the vehicle and the lane line. S9, in Iterate through all vehicle nodes and obtain the probability of crossing the lane line between each vehicle node and the lane line nodes connected to it. S10. Determine whether each vehicle has crossed the line based on the probability of crossing the line and a set threshold. S11. If the probability of crossing the line exceeds the preset threshold, output the vehicle node and lane line node that are crossing the line.

Images