Vehicle matching relationship determination method and device based on high-bit video monitoring

Abstract

The application discloses a vehicle matching relationship judgment method and device based on high-bit video monitoring. The method comprises the following steps: constructing a real relationship matrix of a vehicle and a matching object and an adjacency matrix of each detection frame according to a plurality of video frame images, a vehicle category, a detection frame coordinate position, a detection frame identification number, a matching object category, a detection frame coordinate position and a detection frame identification number; transforming the image according to the vehicle detection frame coordinate position and the matching object detection frame coordinate position to obtain a detection frame image; splicing the feature of each detection frame image with a distance feature to obtain a spliced feature, performing feature transformation on the spliced feature to obtain a transformed feature; inputting each transformed feature and the adjacency matrix of each detection frame into a graph convolutional neural network to output a predicted relationship matrix of the vehicle and the matching object; constructing a loss function, and adjusting the parameters of the graph convolutional neural network according to the loss function to obtain a trained graph convolutional neural network.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a method and device for determining vehicle matching relationships based on high-position video surveillance. Background Art

[0002] With the development of high-position video technology, it is now possible to detect and capture actions such as roadside parking and vehicle violations, recording the entire process in the form of images and videos. The parking or violation evidence data chain formed by high-position video technology has greatly alleviated the phenomenon of arbitrary roadside parking charges. During the management process, it is necessary to detect the vehicle target and identify the corresponding matching object in the captured video frame image. The matching object is then matched with the corresponding vehicle to determine the matching relationship between the matching object and the vehicle. Once the matching relationship is correctly established, the parking or violation order is sent to the user. The matching object of the vehicle can be a license plate or a parking space, etc.

[0003] Traditional vehicle matching relationship determination methods use logical judgment conditions based on the positional relationship and intersection-in-union (IUN) between the two matching objects to determine their relationship. However, when using this traditional method, errors in coordinate position can lead to errors in the IUN calculation, which in turn can lead to errors in the determination of the relationship between the vehicle and the matching object, resulting in low matching accuracy.

[0004] Application Contents

[0005] The purpose of this application is to solve the technical problem of low matching accuracy in traditional vehicle matching relationship judgment methods. To achieve the above purpose, this application provides a vehicle matching relationship judgment method and device based on high-level video monitoring.

[0006] This application provides a vehicle matching relationship determination method based on high-position video surveillance, comprising:

[0007] Acquire multiple video frame images, and obtain, based on the multiple video frame images, a vehicle category, a vehicle detection frame coordinate position, a vehicle detection frame identification number, a matching object category, a matching object detection frame coordinate position, and a matching object detection frame identification number corresponding to each of the video frame images;

[0008] Constructing a true relationship matrix between the vehicle and the matching object and an adjacency matrix for each detection frame based on the vehicle category, the coordinate position of the vehicle detection frame, the identification number of the vehicle detection frame, the matching object category, the coordinate position of the matching object detection frame, and the identification number of the matching object detection frame; wherein each detection frame is a vehicle detection frame or a matching object detection frame;

[0009] Dividing and resizing each of the video frame images according to the coordinate position of the vehicle detection frame and the coordinate position of the matching object detection frame to obtain a plurality of detection frame images;

[0010] Splicing the features of each detection frame image with the distance features to obtain a plurality of splicing features, and performing feature transformation on the plurality of splicing features to obtain a plurality of transformed features;

[0011] Inputting the adjacency matrix of each of the transformed features and each of the detection boxes into a graph convolutional neural network, and outputting a predicted relationship matrix between the vehicle and the matching object;

[0012] Constructing a loss function based on the real relationship matrix between the vehicle and the matching object and the predicted relationship matrix between the vehicle and the matching object, and adjusting the parameters of the graph convolutional neural network based on the loss function to obtain a trained graph convolutional neural network;

[0013] Based on the trained graph convolutional neural network, the video frame image to be tested is predicted to obtain a matching relationship between the vehicle and the matching object.

[0014] In one embodiment, the step of splicing the features of each detection frame image with the distance features to obtain a plurality of spliced ​​features, and performing feature transformation on the plurality of spliced ​​features to obtain a plurality of transformed features includes:

[0015] Obtaining 3*H*W dimensional features of each of the detection frame images;

[0016] Set a 4*H*W dimensional distance feature, and perform feature splicing on the channel dimension of the 3*H*W dimensional feature of each detection frame image and the 4*H*W dimensional distance feature to obtain multiple 7*H*W dimensional splicing features;

[0017] Where H*W represents the width and height of the detection box, and the number of channels 7 includes R channel, G channel, B channel, X coordinate channel, Y coordinate channel, W width channel, and H height channel.

[0018] In one embodiment, the step of splicing the features of each detection frame image with the distance features to obtain a plurality of spliced ​​features, and performing feature transformation on the plurality of spliced ​​features to obtain a plurality of transformed features further includes:

[0019] Inputting the multiple concatenated features into multiple convolutional layers and outputting multiple transformed features;

[0020] The multiple conversion features are input into a fully connected layer network, and the multiple transformation features are output.

[0021] In one embodiment, inputting the adjacency matrix of each of the transformed features and each of the detection boxes into a graph convolutional neural network and outputting a predicted relationship matrix between the vehicle and the matching object includes:

[0022] Construct the graph convolutional neural network, which is:

[0023]

[0024]

[0025] Wherein, X represents each of the transformation features, A represents the relationship matrix of each of the detection frames, represents the adjacency matrix of each detection frame, I represents the identity matrix of each detection frame, Represents the degree matrix of each detection box, W (0) With W (1) Represents the parameters of the graph convolutional neural network, F1 represents the nonlinear activation function, and F2 represents the normalization function;

[0026] According to the graph convolutional neural network, a predicted relationship matrix between the vehicle and the matching object is output.

[0027] In one embodiment, in constructing the graph convolutional neural network, the diagonal eigenvalue of the degree matrix of each detection box is added by one to obtain a new degree matrix of each detection box. The graph convolutional neural network is:

[0028]

[0029]

[0030] in, represents the new adjacency matrix of each detection box, Represents the newness matrix of each detection box.

[0031] In one embodiment, constructing a loss function based on the real relationship matrix between the vehicle and the matching object and the predicted relationship matrix between the vehicle and the matching object, and adjusting the parameters of the graph convolutional neural network based on the loss function to obtain a trained graph convolutional neural network includes:

[0032] Construct a loss function, which is:

[0033]

[0034] in, represents the predicted relationship matrix between the vehicle and the matching object corresponding to the i-th training sample, y (i)Represents the true relationship matrix between the vehicle and the matching object corresponding to the i-th training sample;

[0035] The parameters of the graph convolutional neural network are adjusted to minimize the loss function, thereby obtaining the trained graph convolutional neural network.

[0036] In one embodiment, the matching object includes a license plate or a parking space, the real relationship matrix between the vehicle and the matching object includes a real subordinate relationship matrix between the vehicle and the license plate or a real occupancy relationship matrix between the vehicle and the parking space, the predicted relationship matrix between the vehicle and the matching object includes a predicted subordinate relationship matrix between the vehicle and the license plate or a predicted occupancy relationship matrix between the vehicle and the parking space, and the matching relationship between the vehicle and the matching object includes a subordinate relationship between the vehicle and the license plate or an occupancy relationship between the vehicle and the parking space.

[0037] In one embodiment, the present application provides a vehicle matching relationship determination device based on high-position video surveillance, comprising:

[0038] a data acquisition module, configured to acquire a plurality of video frame images and, based on the plurality of video frame images, obtain a vehicle category, a vehicle detection frame coordinate position, a vehicle detection frame identification number, a matching object category, a matching object detection frame coordinate position, and a matching object detection frame identification number corresponding to each of the video frame images;

[0039] a relationship matrix acquisition module, configured to construct a true relationship matrix between the vehicle and the matching object and an adjacency matrix for each detection frame based on the vehicle category, the coordinate position of the vehicle detection frame, the identification number of the vehicle detection frame, the matching object category, the coordinate position of the matching object detection frame, and the identification number of the matching object detection frame; wherein each detection frame is a vehicle detection frame or a matching object detection frame;

[0040] a detection frame image acquisition module, configured to divide and resize each of the video frame images according to the coordinate position of the vehicle detection frame and the coordinate position of the matching object detection frame to obtain a plurality of detection frame images;

[0041] a transformation feature acquisition module, configured to splice the features of each of the detection frame images with the distance features to obtain a plurality of spliced ​​features, and perform feature transformation on the plurality of spliced ​​features to obtain a plurality of transformed features;

[0042] A graph convolutional neural network module, configured to input the adjacency matrix of each of the transformed features and each of the detection boxes into a graph convolutional neural network, and output a predicted relationship matrix between the vehicle and the matching object;

[0043] A training module, configured to construct a loss function based on the real relationship matrix between the vehicle and the matching object and the predicted relationship matrix between the vehicle and the matching object, and adjust the parameters of the graph convolutional neural network based on the loss function to obtain a trained graph convolutional neural network;

[0044] The vehicle and matching object relationship determination module is used to predict the video frame image to be tested based on the trained graph convolutional neural network to obtain the matching relationship between the vehicle and the matching object.

[0045] In one embodiment, the transformation feature acquisition module includes:

[0046] A first dimensional feature acquisition module, configured to acquire 3*H*W dimensional features of each of the detection frame images;

[0047] A splicing feature acquisition module is used to set a 4*H*W dimensional distance feature and perform feature splicing on the channel dimension of the 3*H*W dimensional feature of each detection frame image with the 4*H*W dimensional distance feature to obtain multiple 7*H*W dimensional splicing features;

[0048] Where H*W represents the width and height of the detection box, and the number of channels 7 includes R channel, G channel, B channel, X coordinate channel, Y coordinate channel, W width channel, and H height channel.

[0049] In one embodiment, the transformation feature acquisition module further includes:

[0050] A convolutional layer module, configured to input the plurality of concatenated features into a plurality of convolutional layers and output a plurality of transformed features;

[0051] The fully connected layer network module is used to input the multiple conversion features into the fully connected layer network and output the multiple transformation features.

[0052] In one embodiment, the graph convolutional neural network module includes:

[0053] A construction module is used to construct the graph convolutional neural network, which is:

[0054]

[0055]

[0056] Wherein, X represents each of the transformation features, A represents the relationship matrix of each of the detection frames, represents the adjacency matrix of each detection frame, I represents the identity matrix of each detection frame, Represents the degree matrix of each detection box, W (0) With W (1)Represents the parameters of the graph convolutional neural network, F1 represents the nonlinear activation function, and F2 represents the normalization function;

[0057] A prediction module is used to output a predicted relationship matrix between the vehicle and the matching object based on the graph convolutional neural network.

[0058] In one embodiment, in the construction module, the diagonal eigenvalue of the degree matrix of each detection box is added by one to obtain a new degree matrix of each detection box, and the graph convolutional neural network is:

[0059]

[0060]

[0061] in, represents the new adjacency matrix of each detection box, Represents the newness matrix of each detection box.

[0062] In one embodiment, the training module includes:

[0063] The loss function construction module is used to construct the loss function, which is:

[0064]

[0065] in, represents the predicted relationship matrix between the vehicle and the matching object corresponding to the i-th training sample, y (i) Represents the true relationship matrix between the vehicle and the matching object corresponding to the i-th training sample;

[0066] A parameter optimization module is used to adjust the parameters of the graph convolutional neural network so as to minimize the loss function and obtain the trained graph convolutional neural network.

[0067] In the above-mentioned vehicle matching relationship determination method based on high-position video surveillance, a vehicle-matching relationship matrix and an adjacency matrix for each detection frame are formed by obtaining the vehicle category, vehicle detection frame coordinate position, vehicle detection frame identification number, matching object category, matching object detection frame coordinate position, and matching object detection frame identification number. The relationship between the vehicle and the matching object is constructed as graph structure data. The graph structure data represents the relationship between each detection frame. By using the graph structure data formed by the vehicle and the matching object and leveraging the powerful graph structure data learning capabilities of the graph convolutional neural network, the relationship between the vehicle and the matching object is learned, achieving end-to-end vehicle and matching object relationship determination. Furthermore, distance features are added to the features of each detection frame image to form a splicing feature. The distance features are further added to train the graph convolutional neural network, thereby further enhancing the learning of the matching relationship between the vehicle and the matching object, and facilitating the acquisition of a more stable training model. Therefore, the vehicle matching relationship determination method based on high-position video surveillance of this application can more accurately determine the matching relationship between the vehicle and the matching object, solving the problem of low matching accuracy in traditional methods and improving the accuracy of determining the matching relationship between the vehicle and the matching object. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 This is a flowchart of the steps of the vehicle matching relationship judgment method based on high-position video surveillance provided by this application.

[0069] Figure 2 It is a schematic diagram of the overall structure of the vehicle matching relationship judgment device based on high-position video monitoring provided by this application. DETAILED DESCRIPTION

[0070] The technical solution of the present application is further described in detail below through the accompanying drawings and examples.

[0071] See Figure 1 The present application provides a vehicle matching relationship determination method based on high-position video surveillance, comprising:

[0072] S10, acquiring a plurality of video frame images, and obtaining, based on the plurality of video frame images, a vehicle category, a vehicle detection frame coordinate position, a vehicle detection frame identification number, a matching object category, a matching object detection frame coordinate position, and a matching object detection frame identification number corresponding to each video frame image;

[0073] S20, constructing a true relationship matrix between the vehicle and the matching object and an adjacency matrix for each detection frame based on the vehicle category, the vehicle detection frame coordinate position, the vehicle detection frame identification number, the matching object category, the matching object detection frame coordinate position, and the matching object detection frame identification number; wherein each detection frame is either a vehicle detection frame or a matching object detection frame;

[0074] S30, dividing and resizing each video frame image according to the coordinate position of the vehicle detection frame and the coordinate position of the matching object detection frame to obtain multiple detection frame images;

[0075] S40, splicing the features of each detection frame image with the distance features to obtain a plurality of spliced ​​features, and performing feature transformation on the plurality of spliced ​​features to obtain a plurality of transformed features;

[0076] S50, inputting the adjacency matrix of each transformed feature and each detection box into a graph convolutional neural network, and outputting a predicted relationship matrix between the vehicle and the matching object;

[0077] S60, constructing a loss function based on the actual relationship matrix between the vehicle and the matching object and the predicted relationship matrix between the vehicle and the matching object, and adjusting the parameters of the graph convolutional neural network based on the loss function to obtain a trained graph convolutional neural network;

[0078] S70: Predict the video frame image to be tested based on the trained graph convolutional neural network to obtain a matching relationship between the vehicle and the matching object. The matching object includes a license plate or a parking space, the real relationship matrix between the vehicle and the matching object includes a real subordination relationship matrix between the vehicle and the license plate or a real occupancy relationship matrix between the vehicle and the parking space, the predicted relationship matrix between the vehicle and the matching object includes a predicted subordination relationship matrix between the vehicle and the license plate or a predicted occupancy relationship matrix between the vehicle and the parking space, and the matching relationship between the vehicle and the matching object includes a subordination relationship between the vehicle and the license plate or an occupancy relationship between the vehicle and the parking space.

[0079] In S10, the multiple video frame images are video frame images at different positions, angles, and time periods. The multiple video frame images can be obtained by extracting frames from video data captured by a high-position video camera and saved as video frame images. Each video frame image includes target information such as vehicles and matching objects. Furthermore, multiple vehicle categories, multiple vehicle detection frame coordinate positions, multiple vehicle detection frame identification numbers, multiple matching object categories, multiple matching object detection frame coordinate positions, and multiple matching object detection frame identification numbers can be obtained from the multiple video frame images. The categories in each video frame image can be divided into three categories: vehicle, license plate, and parking space. The matching object category includes the license plate category or the parking space category. During detection, whether the category is the parking space category or the license plate category can be detected. The vehicle detection frame coordinate position includes the center point coordinate information of the vehicle detection frame and the width and height information of the vehicle detection frame. The matching object detection frame coordinate position includes the center point coordinate information of the matching object detection frame and the width and height information of the matching object detection frame. The center point coordinate information of the detection frame and the width and height information of the detection frame can be expressed as X, Y, W, and H, respectively. The vehicle detection frame identification number can be the ID number of the vehicle detection frame, which can be understood as the identification serial number of the vehicle detection frame. The matching object detection frame identification number can be the ID number of the matching object detection frame, which can be understood as the identification serial number of the matching object detection frame. Each vehicle detection frame identification number and matching object detection frame identification number are unique and different from each other. The vehicle category, vehicle detection frame coordinate position, vehicle detection frame identification number, matching object category, matching object detection frame coordinate position, and matching object detection frame identification number are stored in files in formats such as txt, json, and xml.

[0080] In S20, the vehicle bounding box identification number and the matching object bounding box identification number are obtained according to the steps in S10. Each vehicle bounding box identification number and each matching object bounding box identification number is a unique representation, so that each vehicle and each matching object has a unique bounding box ID number for representation. It can be understood that one vehicle bounding box identification number corresponds to one vehicle bounding box, and one matching object bounding box identification number corresponds to one matching object bounding box, and each bounding box has a unique identification number. According to the plurality of vehicle bounding box identification numbers and the plurality of matching object bounding box identification numbers, the identification numbers of the vehicle and the matching object that meet the matching relationship are taken as a matching pair to represent that the vehicle and the matching object meet the matching relationship. According to the matching relationship formed by the vehicle and the matching object, the label data of the matching relationship is constructed to form an N*N dimension matrix. The N*N dimension matrix is the real relationship matrix of the vehicle and the matching object. Wherein, N represents the number of bounding boxes. The vehicle bounding box and the matching object bounding box can be collectively referred to as a bounding box. One bounding box corresponds to one bounding box identification number. The N*N dimension feature represents the matching relationship between each bounding box and other N-1 bounding boxes. When a bounding box has a matching relationship with another bounding box, the coordinate value representing the matching relationship of the two bounding boxes in the real relationship matrix is 1, otherwise it is 0. The relationship between each bounding box and itself in the real relationship matrix is represented as 0. Thus, according to the vehicle category, the vehicle bounding box coordinate position, the vehicle bounding box identification number, the matching object category, the matching object bounding box coordinate position, and the matching object bounding box identification number, the N*N dimension real relationship matrix of the vehicle and the matching object can be constructed. According to the vehicle bounding box coordinate position, the vehicle bounding box identification number, the matching object bounding box coordinate position, and the matching object bounding box identification number, the relationship matrix of each bounding box can be obtained. The relationship matrix of each bounding box represents the relationship matrix between each bounding box and other bounding boxes. Each bounding box has a matching relationship with other bounding boxes, and the value in the matrix is 1, otherwise it is 0. Further, according to the relationship matrix of each bounding box, the adjacency matrix of each bounding box can be obtained. The adjacency matrix represents the adjacency relationship between the bounding boxes, which is an n-order square matrix. Each bounding box can be a vehicle bounding box or a matching object bounding box.

[0081] In S30, the vehicle detection frame coordinate position represents the specific position of the vehicle detection frame. According to the vehicle detection frame coordinate position, the video frame image can be cropped, and the corresponding vehicle detection frame image can be obtained from the original image. The image of the corresponding vehicle detection frame obtained is resized to obtain a detection frame image of the same size. The matching object detection frame coordinate position represents the specific position of the matching object detection frame. According to the matching object detection frame coordinate position, the video frame image can be cropped, and the corresponding matching object detection frame image can be obtained from the original image. The image of the corresponding matching object detection frame obtained is resized to obtain a detection frame image of the same size. The multiple detection frame images include vehicle detection frame images and matching object detection frame images. Through division and resizing, the multiple images are converted into detection frame images of the same size, which facilitates subsequent image processing.

[0082] In S40, the distance feature characterizes the distance features of adjacent detection frames. For video frame images, the probability of a matching relationship between a vehicle detection frame and a matching object detection frame is high. Furthermore, the distance feature is added to the features of each detection frame image for splicing to obtain a spliced ​​feature. By adding the distance feature and incorporating the distance of the detection frame into the spliced ​​feature, the learning of the matching relationship between the vehicle and the matching object can be enhanced, which is beneficial for model training and a more stable training model. By performing feature transformation on the spliced ​​feature, a transformed feature is obtained, which is convenient for calculation in graph convolutional neural networks.

[0083] In S50, the graph convolutional neural network includes multiple convolutional layers, nonlinear activation layers, and other layers. Graph convolutional neural networks can be used to process unstructured data. Each transformed feature serves as an input feature vector and the adjacency matrix of each detection box, which serves as the input to the graph convolutional neural network for feature learning. The predicted relationship matrix between the vehicle and the matching object is graph-structured data and serves as the output of the graph convolutional neural network.

[0084] In S60, a loss function is constructed based on the actual relationship between the vehicle and the matching object and the predicted relationship between the vehicle and the matching object, and the parameters of the graph convolutional neural network are optimized. Finally, a trained graph convolutional neural network is obtained for prediction in S70.

[0085] In S70, the trained graph convolutional neural network is used to determine the matching relationship between the vehicle and the matching object in the video frame image to be tested, so as to obtain an accurate matching relationship between the vehicle and the matching object.

[0086] Through S10 to S70, the vehicle category, vehicle detection frame coordinate position, vehicle detection frame identification number, matching object category, matching object detection frame coordinate position, and matching object detection frame identification number are obtained to form a real relationship matrix between the vehicle and the matching object and an adjacency matrix for each detection frame, and the matching relationship between the vehicle and the matching object is constructed as graph structure data. The graph structure data represents the relationship between each detection frame. Through the graph structure data formed by the vehicle and the matching object, and utilizing the powerful graph structure data learning ability of the graph convolutional neural network, the relationship between the vehicle and the matching object is learned, and end-to-end relationship judgment between the vehicle and the matching object is achieved. In addition, the distance feature is added to the features of each detection frame image to form a splicing feature, and the distance feature is further added to train the graph convolutional neural network, thereby further enhancing the learning of the relationship between the vehicle and the matching object, which is conducive to obtaining a more stable training model. Therefore, through the vehicle matching relationship judgment method based on high-position video surveillance of the present application, the matching relationship between the vehicle and the matching object can be more accurately judged, solving the problem of low matching accuracy of traditional methods and improving the accuracy of judging the matching relationship between the vehicle and the matching object.

[0087] In one embodiment, the matching object is a license plate. The real relationship matrix between the vehicle and the matching object is the real subordination matrix between the vehicle and the license plate. The predicted relationship matrix between the vehicle and the matching object is the predicted subordination matrix between the vehicle and the license plate. The matching relationship between the vehicle and the matching object is the subordination relationship between the vehicle and the license plate. The vehicle matching relationship determination method based on high-position video surveillance includes:

[0088] Acquire multiple video frame images, and obtain, based on the multiple video frame images, a vehicle category, a vehicle detection frame coordinate position, a vehicle detection frame identification number, a license plate category, a license plate detection frame coordinate position, and a license plate detection frame identification number corresponding to each video frame image;

[0089] Based on the vehicle category, the coordinate position of the vehicle detection frame, the identification number of the vehicle detection frame, the license plate category, the coordinate position of the license plate detection frame, and the identification number of the license plate detection frame, a true subordination matrix of the vehicle and the license plate and an adjacency matrix of each detection frame are constructed; wherein each detection frame is a vehicle detection frame or a license plate detection frame;

[0090] According to the coordinate position of the vehicle detection frame and the coordinate position of the license plate detection frame, each video frame image is divided and resized to obtain multiple detection frame images;

[0091] The features of each detection frame image are spliced ​​with the distance features to obtain multiple spliced ​​features, and the multiple spliced ​​features are transformed to obtain multiple transformed features;

[0092] The adjacency matrix of each transformed feature and each detection box is input into the graph convolutional neural network, and the predicted affiliation matrix of the vehicle and license plate is output;

[0093] According to the real affiliation matrix between vehicles and license plates and the predicted affiliation matrix between vehicles and license plates, a loss function is constructed, and the parameters of the graph convolutional neural network are adjusted according to the loss function to obtain a trained graph convolutional neural network.

[0094] Based on the trained graph convolutional neural network, the video frame image to be tested is predicted to obtain the subordinate relationship between the vehicle and the license plate.

[0095] In this embodiment, through the vehicle matching relationship judgment method based on high-position video surveillance, the subordinate relationship between the vehicle and the license plate can be judged, and a correct matching relationship can be formed. This can provide a basis for users to send parking or violation orders, thereby helping to solve vehicle management problems such as roadside parking and vehicle violations.

[0096] In one embodiment, the matching object is a parking space, the actual relationship matrix between the vehicle and the matching object is the actual occupancy relationship matrix between the vehicle and the parking space, the predicted relationship matrix between the vehicle and the matching object is the predicted occupancy relationship matrix between the vehicle and the parking space, and the matching relationship between the vehicle and the matching object is the occupancy relationship between the vehicle and the parking space. The vehicle matching relationship determination method based on high-position video surveillance includes:

[0097] Acquire multiple video frame images, and obtain, based on the multiple video frame images, a vehicle category, a vehicle detection frame coordinate position, a vehicle detection frame identification number, a berth category, a berth detection frame coordinate position, and a berth detection frame identification number corresponding to each video frame image;

[0098] Based on the vehicle category, vehicle detection frame coordinate position, vehicle detection frame identification number, berth category, berth detection frame coordinate position, and berth detection frame identification number, a real occupancy relationship matrix between vehicles and berths and an adjacency matrix for each detection frame are constructed; wherein each detection frame is either a vehicle detection frame or a berth detection frame;

[0099] Divide and resize each video frame image according to the coordinate position of the vehicle detection frame and the coordinate position of the berth detection frame to obtain multiple detection frame images;

[0100] The features of each detection frame image are spliced ​​with the distance features to obtain multiple spliced ​​features, and the multiple spliced ​​features are transformed to obtain multiple transformed features;

[0101] The adjacency matrix of each transformed feature and each detection box is input into the graph convolutional neural network, and the predicted occupancy relationship matrix of the vehicle and the parking space is output;

[0102] Based on the actual occupancy relationship matrix between vehicles and parking spaces and the predicted occupancy relationship matrix between vehicles and parking spaces, a loss function is constructed. Based on the loss function, the parameters of the graph convolutional neural network are adjusted to obtain a trained graph convolutional neural network.

[0103] Based on the trained graph convolutional neural network, the video frame image to be tested is predicted to obtain the occupancy relationship between the vehicle and the parking space.

[0104] In this embodiment, through the vehicle matching relationship judgment method based on high-position video surveillance, it is possible to judge the occupancy relationship between vehicles and parking spaces, form a correct matching relationship, and then make accurate judgments on whether a parking space is occupied or released, thereby sending the user the accurate parking time and performing parking fee management, which is conducive to solving vehicle management problems such as roadside parking and vehicle violations.

[0105] In one embodiment, in S10, based on multiple video frame images, the vehicle category, vehicle detection frame coordinate position, vehicle detection frame identification number, matching object category, matching object detection frame coordinate position and matching object detection frame identification number corresponding to each video frame image are obtained, and based on the target detection model, the vehicle category, vehicle detection frame coordinate position, vehicle detection frame identification number, matching object category, matching object detection frame coordinate position and matching object detection frame identification number corresponding to each video frame image are obtained.

[0106] In this embodiment, the target detection model includes but is not limited to a target detection model obtained by using deep learning algorithms, machine learning algorithms and other algorithms to perform target detection on each video frame image.

[0107] In one embodiment, when the matching object is a berth, when marking the berth, the coordinate positions of the berth corner points in the video frame image are calibrated, and the coordinates of the minimum circumscribed rectangular frame of the berth are calculated to obtain the coordinate position information of the berth detection frame.

[0108] In this embodiment, the coordinates of each berth's four corner points within the video frame image are determined by calibrating the coordinate positions of the berth's four corner points. When the high-position video camera is free from external interference, the position of each berth remains unchanged within the video frame image captured by the camera. Based on the obtained coordinates of the four corner points, the coordinates of the minimum bounding rectangle of the quadrilateral can be obtained using a minimum bounding rectangle algorithm. The minimum bounding rectangle algorithm finds the rectangle that circumscribes the polygon and has the smallest area.

[0109] In one embodiment, when the minimum bounding rectangle of a berth is obtained by calculation, background features of the berth may be added, which is beneficial to further assist in determining the occupancy relationship between the vehicle and the berth.

[0110] In one embodiment, the matching object category can be set to 1, and the vehicle category can be set to 2. The specific settings can be set according to actual application conditions.

[0111] In one embodiment, S40, the features of each detection frame image are spliced ​​with the distance features to obtain a plurality of spliced ​​features, and the plurality of spliced ​​features are subjected to feature transformation to obtain a plurality of transformed features, including:

[0112] S410, obtaining 3*H*W dimensional features of each detection frame image;

[0113] S420: Set a 4*H*W dimensional distance feature, and perform feature splicing on the channel dimension of the 3*H*W dimensional feature and the 4*H*W dimensional distance feature of each detection frame image to obtain multiple 7*H*W dimensional splicing features;

[0114] Where H*W represents the width and height of the detection box, and the number of channels 7 includes R channel, G channel, B channel, X coordinate channel, Y coordinate channel, W width channel, and H height channel.

[0115] In this embodiment, in the 3*H*W dimensional features of each detection frame image, 3 represents the number of channels, and H*W represents the width and height information. In the 4*H*W dimensional distance features, 4 represents the number of channels, which respectively represent the X, Y, W, and H information of the detection frame coordinate position, and H*W represents the width and height information. In this embodiment, the width and height information in the features of each detection frame image are the same as the width and height information in the distance features. In multiple 7*H*W dimensional splicing features, the number of channels represents the RGB channels, the X and Y coordinate information of the center point of the detection frame, and the W and H width and height information of the detection frame, respectively. The features representing the distance between the detection frames are added to the features of each detection frame image, and the distance features are fully integrated. By setting the distance features, the distance between the detection frames is taken into account in the features of each detection frame image. By adding the distance features, the matching relationship between vehicles and matching objects that are close to each other can be easily detected, which can further enhance the learning of the matching relationship between vehicles and matching objects, and improve the accuracy of the matching relationship judgment between vehicles and matching objects.

[0116] In one embodiment, S40, stitching the features of each detection frame image with the distance features to obtain a plurality of stitching features, and performing feature transformation on the plurality of stitching features to obtain a plurality of transformed features, further includes:

[0117] S440, inputting the multiple concatenated features into multiple convolutional layers, and outputting multiple transformed features;

[0118] S450: Input the multiple conversion features into a fully connected layer network and output the multiple transformation features.

[0119] In this embodiment, through S440, the spliced ​​features of 7*H*W dimensions are input into multiple convolutional layers to obtain image features after feature transformation. The dimension of the transformed features is expressed as 128*H*W. Through S450, the transformed feature data is sent to the fully connected layer network for calculation to obtain a transformed feature with a feature dimension of 128*1*1 as the input feature vector of the graph convolutional neural network. Through the fully connected layer network, the learned spliced ​​features are mapped to the sample label space through distributed feature representation for further calculation of the graph convolutional neural network.

[0120] In one embodiment, S50, the adjacency matrix between each transformed feature and each detection box is input into a graph convolutional neural network, and a predicted relationship matrix between the vehicle and the matching object is output, including:

[0121] S510, construct a graph convolutional neural network, which is:

[0122]

[0123]

[0124] Among them, X represents each transformation feature, A represents the relationship matrix of each detection box, represents the adjacency matrix of each detection box, I represents the identity matrix of each detection box, Represents the degree matrix of each detection box, W (0) With W (1) Represents the parameters of the graph convolutional neural network, F1 represents the nonlinear activation function, and F2 represents the normalization function;

[0125] S520: Output a predicted relationship matrix between the vehicle and the matching object based on the graph convolutional neural network.

[0126] In S510, the graph convolutional neural network Z includes two network layers, and the output feature dimension is N*N. X represents each transformation feature. Each transformation feature serves as the input feature vector of the first layer of the graph convolutional neural network, integrating the detection frame image features and distance features. The adjacency matrix of each transformation feature and each detection frame serves as the graph structure data input of the first layer of the graph convolutional neural network. A represents the relationship matrix of each detection frame, which can be understood as the matrix formed by the relationship between each detection frame and other detection frames. If each detection frame has a relationship with a certain detection frame, the value displayed in the matrix is ​​1, otherwise it is 0. I represents the unit matrix of each detection frame. By adding the unit matrix, the feature representation of each detection frame itself can be increased. The adjacency matrix of each detection frame The adjacency matrix is ​​obtained by combining the relationship matrix of each detection box with the identity matrix and degree matrix of each detection box. The adjacency matrix represents the adjacent relationship matrix between each detection box and is an n-order square matrix. Represents the degree matrix of each detection box. The degree matrix represents the degree of each detection box as a node, which can also be understood as the number of detection boxes that each detection box has a connection with. Represents the first layer of graph convolutional neural network, and the output feature dimension is N*N. The adjacency matrix of each detection box The feature dimension of W is N*N. (0) W represents the learnable parameters of the first-layer graph convolutional neural network, with a feature dimension of 128*N. The output of the first-layer graph convolutional neural network serves as the input of the second-layer graph convolutional neural network. (1) Represents the learnable parameters of the second-layer graph convolutional neural network, with a feature dimension of 128*N.

[0127] In S520, based on a graph convolutional neural network (GCNN), the adjacency matrix of each transformed feature and each detection box is used as graph structure data input to the GCNN, and the corresponding output is a predicted relationship matrix between the vehicle and the matching object. The relationship between the vehicle and the matching object is constructed as graph structure data and input into the GCNN for learning. By learning the relationship between the vehicle and the matching object, end-to-end matching relationship judgment between the vehicle and the matching object can be achieved. In addition, each transformed feature integrates the original image features of the video frame and the distance features, and further training is performed based on the spliced ​​features, further enhancing the learning of the matching relationship between the vehicle and the matching object, which is conducive to obtaining a more stable training model.

[0128] The graph convolutional neural network constructed by S510 and S520 can more accurately determine the relationship between the vehicle and the matching object, solving the problem of low matching accuracy in traditional methods and improving the accuracy of determining the matching relationship between the vehicle and the matching object.

[0129] In one embodiment, F1 is the Rectified Linear Unit (ReLU) nonlinear activation function. F2 is the softmax normalization function. The graph convolutional neural network is represented as

[0130] In this embodiment, a ReLU nonlinear activation function and a softmax normalization function are used in the graph convolutional neural network. The ReLU function has stronger expressive power and does not suffer from the vanishing gradient problem, which maintains the model's convergence speed at a stable state. The model, after sparsification through the ReLU function, can better explore the relationship characteristics between vehicles and matching objects and fit the training data. The softmax normalization function can reduce the difficulty of model training, making multi-classification problems more easily converged, and is more suitable for the judgment of the matching relationship between vehicles and matching objects in the present application method.

[0131] In one embodiment, in S510, in constructing a graph convolutional neural network, the diagonal eigenvalues ​​of the degree matrix of each detection box are added by one to obtain a new degree matrix for each detection box. The graph convolutional neural network is:

[0132]

[0133]

[0134] in, represents the new adjacency matrix for each detection box, Represents the newness matrix of each detection box.

[0135] In this embodiment, the degree matrix Increasing the diagonal eigenvalues ​​of the degree matrix by one can avoid the problem of being unable to invert the degree matrix when a vehicle's matching object is occluded and cannot be matched with any other matching objects. This can then lead to a new degree matrix that can be further applied to the graph convolutional neural network for matching relationship judgment and prediction. By adding one to the diagonal eigenvalues ​​of the degree matrix in this embodiment, the problem of mismatching caused by occlusion of matching objects in traditional methods can be solved.

[0136] In one embodiment, S60 constructs a loss function based on the actual relationship matrix between the vehicle and the matching object and the predicted relationship matrix between the vehicle and the matching object, and adjusts the parameters of the graph convolutional neural network based on the loss function to obtain a trained graph convolutional neural network, including:

[0137] S610, construct a loss function, the loss function is:

[0138]

[0139] in, Represents the predicted relationship matrix between the vehicle and the matching object corresponding to the i-th training sample, y (i) Represents the true relationship matrix between the vehicle and the matching object corresponding to the i-th training sample;

[0140] S620: Adjust the parameters of the graph convolutional neural network to minimize the loss function and obtain a trained graph convolutional neural network.

[0141] In S610, the i-th training sample can also be understood as the training sample data corresponding to the i-th detection box. The model is trained using the L2 norm loss function. The L2 norm loss function is also called the lowest square error loss function. In the formula, m represents the number of samples in the training sample set.

[0142] In S620, the objective of the loss function is to minimize the sum of squares. By adjusting the parameters of the graph convolutional neural network, the loss function is minimized, ultimately obtaining the optimal network parameters. Based on the optimal network parameters, a trained graph convolutional neural network can be obtained. Using the graph convolutional neural network, the matching relationship between the vehicle and the matching object in the video frame image to be tested can be determined, thereby accurately obtaining the matching relationship between the vehicle and the matching object.

[0143] In one embodiment, S70, predicting the video frame image to be tested based on the trained graph convolutional neural network to obtain a matching relationship between the vehicle and the matching object includes:

[0144] Obtain the video frame image to be tested, and obtain the corresponding vehicle category, vehicle detection frame coordinate position, vehicle detection frame identification number, matching object category, matching object detection frame coordinate position and matching object detection frame identification number according to the video frame image to be tested;

[0145] According to the vehicle category, the coordinate position of the vehicle detection frame, the identification number of the vehicle detection frame, the category of the matching object, the coordinate position of the matching object detection frame, and the identification number of the matching object detection frame, the adjacency matrix of each detection frame is retrieved and calculated from the real relationship matrix between the vehicle and the matching object;

[0146] Divide and resize each video frame image according to the coordinate position of the vehicle detection frame and the coordinate position of the matching object detection frame to obtain multiple detection frame images;

[0147] The features of each detection frame image are spliced ​​with the distance features to obtain multiple spliced ​​features, and the multiple spliced ​​features are transformed to obtain multiple transformed features;

[0148] The adjacency matrix of each transformed feature and each detection box is input into the trained graph convolutional neural network, and the matching relationship between the vehicle and the matching object is output.

[0149] See Figure 2 In one embodiment, the present application provides a vehicle matching relationship judgment device 100 based on high-position video surveillance, including a data acquisition module 10, a relationship matrix acquisition module 20, a detection frame image acquisition module 30, a transformation feature acquisition module 40, a graph convolutional neural network module 50, a training module 60, and a vehicle and matching object relationship determination module 70.

[0150] The data acquisition module 10 is configured to acquire a plurality of video frame images, and obtain, according to the plurality of video frame images, a vehicle category corresponding to each video frame image, a vehicle bounding box coordinate position, a vehicle bounding box identification number, a matching object category, a matching object bounding box coordinate position, and a matching object bounding box identification number. The relationship matrix acquisition module 20 is configured to construct a real relationship matrix of the vehicle and the matching object and an adjacency matrix of each bounding box according to the vehicle category, the vehicle bounding box coordinate position, the vehicle bounding box identification number, the matching object category, the matching object bounding box coordinate position, and the matching object bounding box identification number. Each bounding box is a vehicle bounding box or a matching object bounding box.

[0151] The bounding box image acquisition module 30 is configured to divide and transform the size of each video frame image according to the vehicle bounding box coordinate position and the matching object bounding box coordinate position, to obtain a plurality of bounding box images. The transformed feature acquisition module 40 is configured to splice a feature of each bounding box image with a distance feature to obtain a plurality of spliced features, and perform feature transformation on the plurality of spliced features to obtain a plurality of transformed features. The graph convolutional neural network module 50 is configured to input each transformed feature and the adjacency matrix of each bounding box into a graph convolutional neural network, and output a predicted relationship matrix of the vehicle and the matching object.

[0152] The training module 60 is configured to construct a loss function according to the real relationship matrix of the vehicle and the matching object and the predicted relationship matrix of the vehicle and the matching object, and adjust parameters of the graph convolutional neural network according to the loss function to obtain a trained graph convolutional neural network. The vehicle and matching object relationship determination module 70 is configured to predict a to-be-tested video frame image according to the trained graph convolutional neural network to obtain a vehicle and matching object relationship.

[0153] In this embodiment, the related description of the data acquisition module 10 can refer to the related description of S10 in the above embodiment. The related description of the relationship matrix acquisition module 20 can refer to the related description of S20 in the above embodiment. The related description of the bounding box image acquisition module 30 can refer to the related description of S30 in the above embodiment. The related description of the transformed feature acquisition module 40 can refer to the related description of S40 in the above embodiment. The related description of the graph convolutional neural network module 50 can refer to the related description of S50 in the above embodiment. The related description of the training module 60 can refer to the related description of S60 in the above embodiment. The related description of the vehicle and matching object relationship determination module 70 can refer to the related description of S70 in the above embodiment.

[0154] In one embodiment, the transformation feature acquisition module 40 includes a first-dimensional feature acquisition module (not shown) and a splicing feature acquisition module (not shown). The first-dimensional feature acquisition module is used to acquire 3*H*W dimensional features for each detection frame image. The splicing feature acquisition module is used to set 4*H*W dimensional distance features and perform feature splicing on the channel dimension of the 3*H*W dimensional features and the 4*H*W dimensional distance features of each detection frame image to obtain multiple 7*H*W dimensional splicing features.

[0155] Where H*W represents the width and height of the detection box, and the number of channels 7 includes R channel, G channel, B channel, X coordinate channel, Y coordinate channel, W width channel, and H height channel.

[0156] In this embodiment, the description of the first dimension feature acquisition module can refer to the description of S410 in the above embodiment. The description of the splicing feature acquisition module can refer to the description of S420 in the above embodiment.

[0157] In one embodiment, the transformation feature acquisition module 40 further includes a convolutional layer module (not shown) and a fully connected layer network module (not shown). The convolutional layer module is used to input multiple convolutional features into multiple convolutional layers and output multiple transformation features. The fully connected layer network module is used to input multiple transformation features into a fully connected layer network and output multiple transformation features.

[0158] In this embodiment, the description of the convolutional layer module can refer to the description of S430 in the above embodiment. The description of the splicing feature acquisition module can refer to the description of S440 in the above embodiment.

[0159] In one embodiment, the graph convolutional neural network module 50 includes a construction module (not shown in the figure) and a prediction module (not shown in the figure). The construction module is used to construct a graph convolutional neural network, which is:

[0160]

[0161]

[0162] Among them, X represents each transformation feature, A represents the relationship matrix of each detection box, represents the adjacency matrix of each detection box, I represents the identity matrix of each detection box, Represents the degree matrix of each detection box, W (0) With W (1) Represents the parameters of the graph convolutional neural network, F1 represents the nonlinear activation function, and F2 represents the normalization function;

[0163] The prediction module is used to output the predicted relationship matrix between vehicles and matching objects based on the graph convolutional neural network.

[0164] In this embodiment, the description of the construction module can refer to the description of S510 in the above embodiment. The description of the prediction module can refer to the description of S520 in the above embodiment.

[0165] In one embodiment, the graph convolutional neural network in the building block is:

[0166] In this embodiment, the relevant description can refer to the relevant description in the above embodiment.

[0167] In one embodiment, in the construction module, the diagonal eigenvalues ​​of the degree matrix of each detection box are added by one to obtain a new degree matrix for each detection box. The graph convolutional neural network is:

[0168]

[0169]

[0170] in, represents the new adjacency matrix for each detection box, Represents the newness matrix of each detection box.

[0171] In this embodiment, relevant descriptions may refer to the description of the newness matrix in the above embodiment.

[0172] In one embodiment, the training module 60 includes a loss function construction module (not shown in the figure) and a parameter optimization module (not shown in the figure). The loss function construction module is used to construct a loss function, and the loss function is:

[0173]

[0174] in, Represents the predicted relationship matrix between the vehicle and the matching object corresponding to the i-th training sample, y (i) Represents the true relationship matrix between the vehicle and the matching object corresponding to the i-th training sample;

[0175] The parameter optimization module is used to adjust the parameters of the graph convolutional neural network to minimize the loss function and obtain the trained graph convolutional neural network.

[0176] In this embodiment, the description of the loss function construction module can refer to the description of S610 in the above embodiment. The description of the parameter optimization module can refer to the description of S620 in the above embodiment.

[0177] In one embodiment, the matching object in the data acquisition module 10 includes a license plate or a parking space, the real relationship matrix between the vehicle and the matching object in the relationship matrix acquisition module 20 includes a real subordination relationship matrix between the vehicle and the license plate or a real occupancy relationship matrix between the vehicle and the parking space, the predicted relationship matrix between the vehicle and the matching object in the graph convolutional neural network module 50 includes a predicted subordination relationship matrix between the vehicle and the license plate or a predicted occupancy relationship matrix between the vehicle and the parking space, and in the vehicle and matching object relationship determination module 70, the matching relationship between the vehicle and the matching object includes the subordination relationship between the vehicle and the license plate or the occupancy relationship between the vehicle and the parking space.

[0178] For the relevant description in this embodiment, reference may be made to the relevant description in the above embodiment.

[0179] In the various embodiments described above, the specific order or hierarchy of steps in the disclosed processes is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of this disclosure. The accompanying method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0180] Those skilled in the art will also appreciate that the various illustrative logical blocks, units, and steps listed in the embodiments of the present application can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present application.

[0181] The various illustrative logic blocks described in the embodiments of the present application, or units can be implemented or operated by the design of a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination thereof. The general-purpose processor can be a microprocessor, alternatively, the general-purpose processor can also be any traditional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration to implement.

[0182] The steps of the methods or algorithms described in the embodiments of the present application can be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium can also be integrated into the processor. The processor and the storage medium can be provided in an ASIC, which can be provided in a user terminal. Alternatively, the processor and the storage medium can also be provided in different components in the user terminal.

[0183] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of this application. It should be understood that the above description is only the specific implementation methods of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A vehicle matching relationship judgment method based on high-position video monitoring, characterized in that: include: Acquire multiple video frame images, and obtain, based on the multiple video frame images, a vehicle category, a vehicle detection frame coordinate position, a vehicle detection frame identification number, a matching object category, a matching object detection frame coordinate position, and a matching object detection frame identification number corresponding to each of the video frame images; Constructing a true relationship matrix between the vehicle and the matching object and an adjacency matrix for each detection frame based on the vehicle category, the coordinate position of the vehicle detection frame, the identification number of the vehicle detection frame, the matching object category, the coordinate position of the matching object detection frame, and the identification number of the matching object detection frame; wherein each detection frame is a vehicle detection frame or a matching object detection frame; Dividing and resizing each of the video frame images according to the coordinate position of the vehicle detection frame and the coordinate position of the matching object detection frame to obtain a plurality of detection frame images; Splicing the features of each detection frame image with the distance features to obtain a plurality of splicing features, and performing feature transformation on the plurality of splicing features to obtain a plurality of transformed features; Inputting the adjacency matrix of each of the transformed features and each of the detection boxes into a graph convolutional neural network, and outputting a predicted relationship matrix between the vehicle and the matching object; Inputting the adjacency matrix of each of the transformed features and each of the detection boxes into a graph convolutional neural network and outputting a predicted relationship matrix between the vehicle and the matching object includes: Construct the graph convolutional neural network, which is: Wherein, X represents each of the transformation features, A represents the relationship matrix of each of the detection frames, represents the adjacency matrix of each detection frame, I represents the identity matrix of each detection frame, Represents the degree matrix of each detection box, W (0) With W (1) Represents the parameters of the graph convolutional neural network, F1 represents the nonlinear activation function, and F2 represents the normalization function; Outputting a predicted relationship matrix between the vehicle and the matching object according to the graph convolutional neural network; Constructing a loss function based on the real relationship matrix between the vehicle and the matching object and the predicted relationship matrix between the vehicle and the matching object, and adjusting the parameters of the graph convolutional neural network based on the loss function to obtain a trained graph convolutional neural network; Based on the trained graph convolutional neural network, the video frame image to be tested is predicted to obtain a matching relationship between the vehicle and the matching object.

2. The vehicle matching relationship determination method based on high-position video monitoring according to claim 1 is characterized in that: The step of splicing the features of each detection frame image with the distance features to obtain a plurality of spliced ​​features, and performing feature transformation on the plurality of spliced ​​features to obtain a plurality of transformed features includes: Obtaining 3*H*W dimensional features of each of the detection frame images; Set a 4*H*W dimensional distance feature, and perform feature splicing on the channel dimension of the 3*H*W dimensional feature of each detection frame image and the 4*H*W dimensional distance feature to obtain multiple 7*H*W dimensional splicing features; Where H*W represents the width and height of the detection box, and the number of channels 7 includes R channel, G channel, B channel, X coordinate channel, Y coordinate channel, W width channel, and H height channel.

3. The vehicle matching relationship determination method based on high-position video monitoring according to claim 2 is characterized in that: The step of splicing the features of each detection frame image with the distance features to obtain a plurality of spliced ​​features, and performing feature transformation on the plurality of spliced ​​features to obtain a plurality of transformed features, further includes: Inputting the multiple concatenated features into multiple convolutional layers and outputting multiple transformed features; The multiple conversion features are input into a fully connected layer network, and the multiple transformation features are output.

4. The vehicle matching relationship determination method based on high-position video monitoring according to claim 3 is characterized in that: In constructing the graph convolutional neural network, the diagonal eigenvalue of the degree matrix of each detection frame is added by one to obtain a new degree matrix of each detection frame. The graph convolutional neural network is: in, represents the new adjacency matrix of each detection box, Represents the newness matrix of each detection box.

5. The vehicle matching relationship determination method based on high-position video monitoring according to claim 1 is characterized in that: The step of constructing a loss function based on the real relationship matrix between the vehicle and the matching object and the predicted relationship matrix between the vehicle and the matching object, and adjusting the parameters of the graph convolutional neural network based on the loss function to obtain a trained graph convolutional neural network includes: Construct a loss function, which is: in, represents the predicted relationship matrix between the vehicle and the matching object corresponding to the i-th training sample, y (i) represents the true relationship matrix between the vehicle and the matching object corresponding to the i-th training sample; The parameters of the graph convolutional neural network are adjusted to minimize the loss function, thereby obtaining the trained graph convolutional neural network.

6. The vehicle matching relationship determination method based on high-position video monitoring according to claim 1 is characterized in that: The matching object includes a license plate or a parking space, the real relationship matrix between the vehicle and the matching object includes a real subordinate relationship matrix between the vehicle and the license plate or a real occupancy relationship matrix between the vehicle and the parking space, the predicted relationship matrix between the vehicle and the matching object includes a predicted subordinate relationship matrix between the vehicle and the license plate or a predicted occupancy relationship matrix between the vehicle and the parking space, and the matching relationship between the vehicle and the matching object includes a subordinate relationship between the vehicle and the license plate or an occupancy relationship between the vehicle and the parking space.

7. A vehicle matching relationship judgment device based on high-position video monitoring, characterized in that: include: a data acquisition module, configured to acquire a plurality of video frame images and, based on the plurality of video frame images, obtain a vehicle category, a vehicle detection frame coordinate position, a vehicle detection frame identification number, a matching object category, a matching object detection frame coordinate position, and a matching object detection frame identification number corresponding to each of the video frame images; a relationship matrix acquisition module, configured to construct a true relationship matrix between the vehicle and the matching object and an adjacency matrix for each detection frame based on the vehicle category, the coordinate position of the vehicle detection frame, the identification number of the vehicle detection frame, the matching object category, the coordinate position of the matching object detection frame, and the identification number of the matching object detection frame; wherein each detection frame is a vehicle detection frame or a matching object detection frame; a detection frame image acquisition module, configured to divide and resize each of the video frame images according to the coordinate position of the vehicle detection frame and the coordinate position of the matching object detection frame to obtain a plurality of detection frame images; a transformation feature acquisition module, configured to splice the features of each of the detection frame images with the distance features to obtain a plurality of spliced ​​features, and perform feature transformation on the plurality of spliced ​​features to obtain a plurality of transformed features; A graph convolutional neural network module is used to input the adjacency matrix of each of the transformed features and each of the detection boxes into a graph convolutional neural network, and output a predicted relationship matrix between the vehicle and the matching object; A training module, configured to construct a loss function based on the real relationship matrix between the vehicle and the matching object and the predicted relationship matrix between the vehicle and the matching object, and adjust the parameters of the graph convolutional neural network based on the loss function to obtain a trained graph convolutional neural network; A vehicle-matching object relationship determination module is used to predict the video frame image to be tested based on the trained graph convolutional neural network to obtain the matching relationship between the vehicle and the matching object; The graph convolutional neural network module includes: A construction module is used to construct the graph convolutional neural network, which is: Wherein, X represents each of the transformation features, A represents the relationship matrix of each of the detection frames, represents the adjacency matrix of each detection frame, I represents the identity matrix of each detection frame, Represents the degree matrix of each detection box, W (0) With W (1) Represents the parameters of the graph convolutional neural network, F1 represents the nonlinear activation function, and F2 represents the normalization function; A prediction module is used to output a predicted relationship matrix between the vehicle and the matching object based on the graph convolutional neural network.

8. The vehicle matching relationship judgment device based on high-position video monitoring according to claim 7 is characterized in that: The transformation feature acquisition module includes: A first dimensional feature acquisition module, configured to acquire 3*H*W dimensional features of each of the detection frame images; The splicing feature acquisition module is used to set the 4*H*W dimensional distance feature and perform feature splicing on the channel dimension of the 3*H*W dimensional feature of each detection frame image with the 4*H*W dimensional distance feature to obtain multiple 7*H*W dimensional splicing features.

9. The vehicle matching relationship judgment device based on high-position video monitoring according to claim 8 is characterized in that: The transformation feature acquisition module also includes: A convolutional layer module, configured to input the plurality of concatenated features into a plurality of convolutional layers and output a plurality of transformed features; The fully connected layer network module is used to input the multiple conversion features into the fully connected layer network and output the multiple transformation features.

10. The vehicle matching relationship judgment device based on high-position video monitoring according to claim 9 is characterized in that: In the construction module, the diagonal eigenvalues ​​of the degree matrix of each detection box are added by one to obtain a new degree matrix of each detection box. The graph convolutional neural network is: in, represents the new adjacency matrix of each detection box, Represents the newness matrix of each detection box.

11. The vehicle matching relationship judgment device based on high-position video monitoring according to claim 7, characterized in that: The training module includes: The loss function construction module is used to construct the loss function, which is: in, represents the predicted relationship matrix between the vehicle and the matching object corresponding to the i-th training sample, y (i) represents the true relationship matrix between the vehicle and the matching object corresponding to the i-th training sample; A parameter optimization module is used to adjust the parameters of the graph convolutional neural network so as to minimize the loss function and obtain the trained graph convolutional neural network.

12. The vehicle matching relationship judgment device based on high-position video monitoring according to claim 7, characterized in that: The matching object in the data acquisition module includes a license plate or a parking space, the real relationship matrix between the vehicle and the matching object in the relationship matrix acquisition module includes a real subordination relationship matrix between the vehicle and the license plate or a real occupancy relationship matrix between the vehicle and the parking space, the predicted relationship matrix between the vehicle and the matching object in the graph convolutional neural network module includes a predicted subordination relationship matrix between the vehicle and the license plate or a predicted occupancy relationship matrix between the vehicle and the parking space, and in the vehicle and matching object relationship determination module, the matching relationship between the vehicle and the matching object includes a subordination relationship between the vehicle and the license plate or an occupancy relationship between the vehicle and the parking space.

Images