A TARD electric vehicle passenger violation intelligent detection method and device

Abstract

The present invention discloses a TARD electric vehicle passenger violation intelligent detection method and device, which is suitable for the intelligent detection of electric vehicle passenger violation. The method comprises the following steps: using a custom module MSF and a custom module RMSFE to construct a backbone network of the model; using a custom module FFM, a custom module IPR_FFM and a custom feature focusing diffusion mechanism to construct a model neck structure, and obtain a custom model TotalAggResDiffNet, i.e., TARD; using a public remote sensing data set SIMD to train the TARD model to determine the model parameters. Compared with the existing neural network model, the detection method proposed in the present invention has achieved a 4.6% mAP50 improvement and a 4.2% accuracy improvement in the electric vehicle passenger monitoring application while ensuring the detection rate, and can be applied to the construction of smart cities to improve the level of urban traffic management.

Description

Technical Field

[0001] The present invention relates to the field of computer vision target detection, and in particular to a TARD electric vehicle manned violation intelligent detection method and device. Background Art

[0002] With the rapid development of urban transportation, electric vehicles have become one of the important means of transportation for citizens. However, the illegal phenomenon of electric vehicles carrying passengers is becoming more and more serious, which has brought great hidden dangers to urban traffic order and public safety. Traditional methods for detecting illegal electric vehicles carrying passengers mostly rely on manual inspections and video surveillance, which have problems such as low efficiency and poor real-time performance.

[0003] Target detection technology has become one of the research hotspots in the field of computer vision, especially the YOLO series of algorithms are widely used due to their high efficiency and accuracy. However, the current target detection technology, including the YOLO series of models, still has low accuracy in the detection and recognition of small target objects at low resolution, such as the recognition of illegal electric vehicle carrying passengers. In order to solve this problem, it is usually adopted to increase the number of model parameters and the model calculation complexity in exchange for improved accuracy. However, the application scope of such models with too large volume will also be narrowed, especially difficult to apply to vehicle-mounted edge computing platforms. How to further improve the low-resolution recognition accuracy of small target objects without increasing the calculation complexity and the number of parameters, so as to meet the needs of real-time monitoring of illegal electric vehicle carrying passengers, is a direction worthy of further research. Summary of the invention

[0004] Purpose of the invention: In view of the problems raised in the background technology, the present invention provides a TARD electric vehicle manned violation intelligent detection method and device, by constructing an MSF&RMSFE backbone network; custom FFM&IPR_FFM combined with a feature focus diffusion mechanism to construct a model neck structure, and obtain a custom model TotalAggResDiffNet (TARD), which solves the current problems of slow model detection rate and poor detection accuracy under low resolution, and meets the needs of real-time monitoring of electric vehicle manned violation behaviors.

[0005] Technical solution:

[0006] The present invention provides a TARD electric vehicle passenger violation intelligent detection method, the method comprising the following steps:

[0007] S1 defines a traffic monitoring video stream data set, extracts key frames from the traffic monitoring video stream data set, and obtains a traffic monitoring video stream key frame data set;

[0008] S2 builds the TARD detection model;

[0009] S2.1 connects MSF module and RMSFE module in series to build the backbone network;

[0010] S2.2 Concatenate FFM module, IPR_FFM module and custom feature focusing diffusion mechanism to build the model neck structure;

[0011] S2.3 uses the public remote sensing data set SIMD to train the Origin-TARD model to obtain the trained TARD detection model;

[0012] S3 uses the trained TARD detection model to detect the traffic monitoring video stream key frame enhancement dataset described in step 1 to obtain a prediction data set containing all key frame detection results in the traffic monitoring video stream key frame enhancement dataset;

[0013] S4 defines a confidence threshold, and determines whether a key frame of a video stream in a prediction data set has a violation of passenger traffic according to the confidence threshold, thereby obtaining a key frame data set with a violation of passenger traffic and a key frame data set without a violation of passenger traffic.

[0014] Furthermore, the traffic monitoring video stream key frame dataset described in S1 is enhanced by using image rotation, brightness enhancement and Gaussian blur methods to obtain a traffic monitoring video stream key frame enhanced dataset.

[0015] Furthermore, the MSF module structure described in S2.1 includes:

[0016] Define the feature map Feature_maps of the traffic monitoring video stream key frame enhanced dataset described in step 1 = {{X1, C1, Stage1}, {X2, C2, Stage2}, …, {Xs, Cs, Stages}, …, {XQ, CQ, StageQ}}, s∈[1,Q], Q is a natural number, where XQ is the feature map of the Qth stage of the network model, CQ is the number of channels of XQ, and StageQ is the Qth stage of the network model;

[0017] The Split module is defined to split the feature map X1 of Stage 1 into two equal parts along the channel dimension;

[0018] The MConv module is defined as the feature map X3 of the Stage 3 stage is passed through a 3×3 convolution module to obtain the feature map X4 of the Stage 4 stage, and then connected in series with a 3×3 convolution module;

[0019] Define the Identity module to pass the feature map X3 of Stage 3 directly to the subsequent layers without any transformation;

[0020] The MSF module is defined as the feature map X1 from the Stage 1 stage connected in series with a Split module to obtain the feature map X2 of the Stage 2 stage and the feature map X3 of the Stage 3 stage. The feature map X3 of the Stage 3 stage is connected in parallel with an MConv module, a 7×7 convolution module and an Identity module, and then a bitwise addition operation is performed, and then a 1×1 convolution is connected in series to obtain the feature map X5 of the Stage 5 stage. The feature maps X2, X3, X4 and X5 from the four stages of Stage 2, Stage 3, Stage 4 and Stage 5 are channel-fused and connected in series with a 1×1 convolution module, and then a bitwise addition operation is performed with the feature map X1 of the Stage 1 stage.

[0021] Furthermore, the RMSFE module structure described in S2.1 is as follows:

[0022] The RMSFE module is defined as a 1×1 convolution module that reduces the number of channels C6 of the feature map X6 of the Stage 6 stage by one half to obtain the feature map X7 of the Stage 7 stage with the number of channels C7. It is then connected in series with an MSF module and bitwise added to the feature map X7 of the Stage 7 stage. A 1×1 convolution module is then used to double the number of channels C7 and bitwise added to the feature map X6 of the Stage 6 stage to obtain the feature map X8 of the Stage 8 stage. A 1×1 convolution module is then used to double the number of channels C8 of the feature map X8 of the Stage 8 stage.

[0023] Furthermore, the FFM module structure described in S2.2 is as follows:

[0024] The MSFM module is defined as a 5×5, 7×7, 9×9, and 11×11 channel-by-channel convolution Depthwise Convolution module connected in parallel, then connected in parallel with an Identity module, followed by a bitwise addition operation, and then connected in series with a 1×1 convolution module;

[0025] The FFM module is defined as the feature maps X9, X10 and X11 from Stage9, Stage10 and Stage11 are input into the FFM module. The feature map X9 of Stage9 is connected in series with a downsampling module, the feature map X10 of Stage10 is connected in series with a 1×1 convolution module, and the feature map X11 of Stage11 is connected in series with an upsampling module and a 1×1 convolution module. The feature maps X9, X10 and X11 from Stage9, Stage10 and Stage11 are channel fused and then connected in parallel with an MSFM module and an Identity module for bitwise addition.

[0026] Furthermore, the IPR_FFM module structure described in S2.2 is as follows:

[0027] The IPR_FFM module is defined as follows: the feature maps X12, X13 and X14 from Stage 12, Stage 13 and Stage 14 are input into the IPR_FFM module. The feature maps X12, X13 and X14 from Stage 12, Stage 13 and Stage 14 are connected in series with an MSF module respectively to obtain the feature maps X15, X16 and X17 of Stage 15, Stage 16 and Stage 17; the feature map X15 of Stage 15 is connected in series with a 1×1 convolution and a channel-by-channel convolution DepthwiseConvolution module to obtain the feature map X18 of Stage 18, which is then connected in series with a 1×1 convolution and an upsampling module, and then bitwise added with the feature map X15 of Stage 15, and then connected in series with the FFM module; the feature map X16 of Stage 16 is bitwise added with the feature map X18 of Stage 18, and then connected in series with a 1×1 convolution and a channel-by-channel convolution Depthwise The Convolution modules are connected in series to obtain the feature map X19 of the Stage 19 stage, which is then connected in series with a 1×1 convolution and an upsampling module, and then bitwise added to the feature map X16 of the Stage 16 stage, and then connected in series with the FFM module; the feature map X17 of the Stage 17 stage is bitwise added to the feature map X19 of the Stage 19 stage, and then multiplied with the feature map X17 of the Stage 17 stage and connected in series with the FFM module.

[0028] Furthermore, the custom feature focusing diffusion mechanism described in S2.2 optimizes the neck feature fusion of the model, and the structure is as follows:

[0029] The custom feature focusing diffusion mechanism optimizes Stage 6 to Stage 17, and diffuses the feature maps after focusing fusion of the IPR_FFM module according to three different detection scales: large, medium and small. The three feature maps generated by the diffusion mechanism are further focused and fused through the IPR_FFM module, and the three feature maps after focusing fusion are diffused to the detection head of the corresponding size.

[0030] Furthermore, the public remote sensing dataset SIMD described in S2.3 trains the original TARD model to obtain the trained TARD detection model and the best training weight best.pt. The validation set of the public remote sensing dataset SIMD is validated using best.pt, and then the model performance is evaluated. The performance evaluation includes:

[0031] Precision(P):

[0032]

[0033] Recall (R):

[0034]

[0035] Mean Average Precision (mAP50):

[0036]

[0037]

[0038] Among them, TP refers to the number of samples that are positive and predicted as positive, FP refers to the number of samples that are negative and predicted as positive, FN refers to the number of samples that are positive and predicted as negative, n is the number of target categories detected, AP i is the AP of the ith target class.

[0039] Furthermore, the present invention discloses a TARD electric vehicle passenger violation intelligent detection device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, any step of the above-mentioned TARD electric vehicle passenger violation intelligent detection method is implemented.

[0040] Beneficial effects:

[0041] 1. The present invention uses the custom module MSF and the custom module RMSFE to construct the backbone network of the model, and achieves an average accuracy improvement of 1.2% in the electric vehicle passenger monitoring application;

[0042] 2. The present invention uses the custom module FFM, the custom module IPR_FFM and the custom feature focusing diffusion mechanism to construct the model neck structure, and achieves an average accuracy improvement of 2.1% in the electric vehicle passenger monitoring application;

[0043] 3. The present invention uses the MSF module, RMSFE module, FFM module, IPR_FFM module and feature focus diffusion mechanism to construct a model to obtain the detection model Origin-TARD, and uses the public remote sensing data set SIMD to train the Origin-TARD model to obtain the trained model TARD, which improves the accuracy of model detection and recognition while ensuring the test rate; in the electric vehicle passenger monitoring application, a 4.6% mAP50 improvement and a 4.2% accuracy improvement were achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is the overall flow chart of the present invention;

[0045] Figure 2This is a structural diagram of the TARD model of the present invention;

[0046] Figure 3 It is the MSF structure diagram of the present invention;

[0047] Figure 4 This is the structure diagram of the RMSFE of the present invention;

[0048] Figure 5 This is the structural diagram of the FFM of the present invention;

[0049] Figure 6 This is the IPR_FFM structure diagram of the present invention;

[0050] Figure 7 This is a comparative example diagram of the detection effect of the present invention;

[0051] Figure 8 This is a performance parameter comparison chart between the present invention and YOLOv8n;

[0052] Fig. 9 This is a comparison chart of the loss rate between the present invention and YOLOv8n. DETAILED DESCRIPTION

[0053] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in each embodiment of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0054] This embodiment provides a TARD electric vehicle passenger violation intelligent detection method and device. The method includes obtaining an image or video stream to be identified, inputting the image or video stream to be identified into the TARD network model, such as Figure 1 shown.

[0055] In order to overcome the shortcomings of the current technology, the TARD model is used to solve the problem of missed detection and false detection of electric vehicle passenger violations at low resolution. In the present invention, in order to solve the problem of low detection and recognition accuracy of small target objects at low resolution, a customized MSF module, a customized RMSFE module, a customized FFM module, a customized IPR_FFM module and a customized feature focus diffusion mechanism are used to build an electric vehicle passenger violation intelligent detection model TARD, that is, Figure 3 to Figure 6 shown.

[0056] The overall architecture of the model in this embodiment is divided into four modules, including the Input terminal, the Backbone network, the Neck structure and the Head output layer. In order to solve the problem of false detection or missed detection of electric vehicle passenger violation detection under low resolution, a TARD network model is constructed. Figure 2shown.

[0057] The network structure parameters are shown in Table 1. The backbone network is constructed using a custom MSF module and a custom RMSFE module, i.e., layers 0 to 5. The neck network structure is constructed using a custom FFM module and a custom IPR_FFM module, and the feature fusion method of the neck is optimized using a custom feature diffusion mechanism, i.e., layers 6 to 17. The feature maps after the focus fusion of the IPR_FFM module are diffused according to three different detection scales: large, medium, and small. The three feature maps generated by the diffusion mechanism are further feature focused and fused through the IPR_FFM module, and the three feature maps after the focus fusion are diffused to the detection head of the corresponding size.

[0058] Table 1

[0059]

[0060]

[0061] In order to determine the optimal model parameters of the TARD model for the traffic monitoring video stream data problem, an embodiment is used to determine the optimal model parameters. The embodiment provides a training set and a validation set for training the model, as well as a training method for the model.

[0062] In the embodiment, a traffic monitoring video stream dataset G1 is used to extract key frames from the traffic monitoring video stream dataset G1 to obtain a traffic monitoring video stream key frame dataset G2. The traffic monitoring video stream key frame dataset G2 is enhanced by using image rotation, brightness enhancement and Gaussian blur methods to obtain a traffic monitoring video stream key frame enhanced dataset G3. The traffic monitoring video stream key frame enhanced dataset G3 is annotated by the LabelImg annotation tool, and a category Class is defined. Class = {0: notoverload, 1: overload}, where 0: notoverload represents a violation of number and no passenger, and 1: overload represents a violation of number and no passenger, respectively. d represents the number and passenger violation respectively; each target object in each key frame image is annotated with a label box and assigned a specific classification label. Each riding electric vehicle in each image will have a passenger violation category label and a set of corresponding position coordinates. The labeling information of the riding electric vehicle in each image is saved as a txt file with the same name as the image, and the label data set L1 is obtained. Let L1 = {id, Ox, Oy, width, high}, where id, Ox, Oy, width, high are the category number, the x-axis coordinate of the center point, the y-axis coordinate of the center point, the width, and the height respectively; the data set G3 is divided into a training set V1 and a validation set V2 in a ratio of 8:2.

[0063] The training set V1 and the validation set V2 are input into the TARD and YOLOv8n networks for training and testing, and the weight files with the best model parameters of TARD and YOLOv8n are obtained during the training process. The weight file with the best model parameters of TARD is P1, and the weight file with the best model parameters of YOLOv8n is P2. They are saved separately for comparative experiments and verification. The number of model training rounds is 200, the batch-size is 24, the initial learning rate is 0.01, the minimum learning rate is 0.01, and the weight decay coefficient is 0.0005. Figure 7 shown.

[0064] In order to better illustrate the application effect, the evaluation of the model obtained in the embodiment is included, and the evaluation is as follows:

[0065] The evaluation indicators of model performance include two aspects: detection speed and detection accuracy. The embodiment adopts evaluation indicators to comprehensively evaluate the performance, including precision P, recall R, mAP50, and mAP50-95. The indicators have different focuses: precision P predicts the number of positive samples that meet the positive sample conditions, and recall R is the number of accurate predictions in the positive category. Average precision AP is the average value of the accuracy at different confidence thresholds. It uses the overlap between the detected target and the true target and is measured using the IoU intersection-over-union ratio. AP uses the precision-recall curve. Average precision mAP is the average value of the AP of the two categories of unmanned violations and manned violations, and is an important indicator for evaluating the performance of the target detection system. mAP is calculated at a confidence threshold. For example, mAP50 represents the mAP value when the confidence threshold is 50%. Calculate the mAP value with confidence thresholds ranging from 50% to 95% to evaluate the robustness and accuracy of the model.

[0066] Precision(P):

[0067]

[0068] Recall (R):

[0069]

[0070] Mean Average Precision (mAP50):

[0071]

[0072] Among them, TP refers to the number of samples that are positive and predicted to be positive, FP refers to the number of samples that are negative and predicted to be positive, FN refers to the number of samples that are positive and predicted to be negative, n is the number of detected target categories, and is the AP of the sth target class.

[0073] The embodiment conducts comparative experiments to verify the performance of the customized MSF module and the customized RMSFE model, the optimization effect of the customized FFM module, the customized IPR_FFM module and the customized feature focusing diffusion mechanism, and further evaluates the performance of the TARD model.

[0074] Compared with YOLOv8n, the proposed method improves the accuracy of the algorithm for detecting small target objects at low resolution while ensuring the detection speed, reduces the model loss rate, and achieves a 4.6% increase in mAP50 and a 4.2% increase in accuracy in the application of monitoring violations of electric vehicles. Figure 8 and Fig. 9 shown.

[0075] Use weight P1 to predict test set V2, use predicted test set V2 for end-to-end detection, input 3-channel RGB images of any size, and normalize the images to 640*640 as model input; define output box Class_boxes = {id, Ox, Oy, width, high}, where id, Ox, Oy, width, high represent category number, center point x-axis coordinate, center point y-axis coordinate, output box width, and output box height, respectively; use non-maximum suppression Non-MaximumSuppression to filter out prediction boxes during TARD network model detection, and obtain a prediction data set R1 containing all key frame detection results in the traffic monitoring video stream key frame enhancement data set G3. Let R1 = {{classes1, class_boxes1, scores1}, {classes2, class_boxes2, scores2}, …, {classes k , class_boxes k ,scores k},…,{classes N , class_boxes N ,scores N}}, k∈[1,N], N is a natural number, among which classes N , class_boxes N , scores N Respectively represent the Nth group of category information, output box, and confidence; define confidence thresholds Confident1 and Confident2, which represent the confidence thresholds for manned violation and unmanned violation, respectively;

[0076] The specific algorithm steps of the final TARD model are as follows:

[0077] Step 1.1: Define a loop variable i for traversing the prediction data set R1, i∈[1,N], N is a natural number, i is initialized to 1, imagei is the i-th video stream key frame in R1; define a feature list feature={id,label}, where id and label are the sequence number and feature label of imagei respectively; define a manned violation target to record the feature list of manned violation targets, set object={feature}; define a manned violation target feature Object to record manned violation targets, set Object={object1,object 2,…,objectf,…,objectY}, f∈[1,Y], Y is a natural number, and 0≤Y≤N, where objectY is the Y-th manned violation target;

[0078] Step 1.2: traverse the prediction data set R1. If i≤N, jump to step 1.3. Otherwise, end the traversal of the prediction data set R1 and jump to step 1.12.

[0079] Step 1.3: Define the unmanned violation detection classifier notoverload_detection = {{classes1, classes2, …, classest, …, classesE}, t∈[1,E], E is a natural number, and 0≤E≤N, where classesE is the E-th group of category information, cyclically detect imagei, and judge whether it is an unmanned violation or a manned violation;

[0080] Step 1.4: Define the key frame data set R2 for manned violations = {{scores1}, {scores2}, …, {scoresb}, …, {scoresM}}, b∈[1,M], M is a natural number, and 0≤M≤N, where scoresM is the Mth group of confidences; define the key frame data set R3 for unmanned violations = {{scores1}, {scores2}, …, {scoresc}, …, {scoresP}}, c∈[1,P], P is a natural number, and 0≤P≤N, P+M=N, where scoresP is the Pth group of confidences;

[0081] Step 1.5: Load the unmanned violation detection classifier notoverload_detection. If classest in imagei is 0, jump to step 1.6, otherwise jump to step 1.9;

[0082] Step 1.6: If the key frame data set R2 of the manned violation already has a manned violation target object, jump to step 1.7, otherwise jump to step 1.8;

[0083] Step 1.7: R2 = R2∪{};

[0084] Step 1.8: Create a new manned violation target objectf, objectf = {feature}; create a new feature list of manned violation target features Object = Object∪{objectf}, and judge according to the confidence threshold Confident1. If scoresb is greater than Confident1, then R2 = R2∪{imagei}, otherwise R2 = R2∪{};

[0085] Step 1.9: If the key frame data set R3 of the unmanned violation already has an unmanned violation target object, jump to step 1.10, otherwise jump to step 1.11;

[0086] Step 1.10: R3 = R3∪{};

[0087] Step 1.11: According to the confidence threshold Confident2, if scoresb is greater than Confident2, then R3 = R3∪{imagei}, otherwise R3 = R3∪{};

[0088] Step 1.12: Obtain the final key frame data set R2 for manned violations and the key frame data set R3 for unmanned violations.

[0089] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A TARD electric vehicle passenger violation intelligent detection method, characterized in that: The method comprises the following steps: S1 defines a traffic monitoring video stream data set, extracts key frames from the traffic monitoring video stream data set, and obtains a traffic monitoring video stream key frame data set; S2 builds the TARD detection model; S2.1 connects MSF module and RMSFE module in series to build the backbone network; S2.2 Concatenate FFM module, IPR_FFM module and custom feature focusing diffusion mechanism to build the model neck structure; The IPR_FFM module structure is as follows: The IPR_FFM module is defined as follows: the feature maps X12, X13 and X14 from Stage 12, Stage 13 and Stage 14 are input into the IPR_FFM module. The feature maps X12, X13 and X14 from Stage 12, Stage 13 and Stage 14 are connected in series with an MSF module respectively to obtain the feature maps X15, X16 and X17 of Stage 15, Stage 16 and Stage 17; the feature map X15 of Stage 15 is connected in series with a 1×1 convolution and a channel-by-channel convolution DepthwiseConvolution module to obtain the feature map X18 of Stage 18, which is then connected in series with a 1×1 convolution and an upsampling module, and then bitwise added with the feature map X15 of Stage 15, and then connected in series with the FFM module; the feature map X16 of Stage 16 is bitwise added with the feature map X18 of Stage 18, and then connected in series with a 1×1 convolution and a channel-by-channel convolution Depthwise The Convolution modules are connected in series to obtain the feature map X19 of the Stage 19 stage, which is then connected in series with a 1×1 convolution and an upsampling module, and then bitwise added with the feature map X16 of the Stage 16 stage, and then connected in series with the FFM module; the feature map X17 of the Stage 17 stage is bitwise added with the feature map X19 of the Stage 19 stage, and then multiplied with the feature map X17 of the Stage 17 stage, and then connected in series with the FFM module; S2.3 uses the public remote sensing data set SIMD to train the Origin-TARD model to obtain the trained TARD detection model; S3 uses the trained TARD detection model to detect the traffic monitoring video stream key frame enhancement dataset described in step 1 to obtain a prediction data set containing all key frame detection results in the traffic monitoring video stream key frame enhancement dataset; S4 defines a confidence threshold, and determines whether a key frame of a video stream in a prediction data set has a violation of passenger traffic according to the confidence threshold, thereby obtaining a key frame data set with a violation of passenger traffic and a key frame data set without a violation of passenger traffic.

2. The TARD electric vehicle passenger violation intelligent detection method according to claim 1 is characterized in that: The traffic monitoring video stream key frame data set described in S1 is subjected to image enhancement by using image rotation, brightness enhancement and Gaussian blur methods to obtain a traffic monitoring video stream key frame enhanced data set.

3. The TARD electric vehicle passenger violation intelligent detection method according to claim 1 is characterized in that: The MSF module structure described in S2.1 includes: Define the feature map Feature_maps of the traffic monitoring video stream key frame enhanced dataset described in step 1 = {{X1, C1, Stage1}, {X2, C2, Stage2}, …, {Xs, Cs, Stages}, …, {XQ, CQ, StageQ}}, s∈[1,Q], Q is a natural number, where XQ is the feature map of the Qth stage of the network model, CQ is the number of channels of XQ, and StageQ is the Qth stage of the network model; The Split module is defined to split the feature map X1 of Stage 1 into two equal parts along the channel dimension; The MConv module is defined as the feature map X3 of the Stage 3 stage is passed through a 3×3 convolution module to obtain the feature map X4 of the Stage 4 stage, and then connected in series with a 3×3 convolution module; Define the Identity module to pass the feature map X3 of Stage 3 directly to the subsequent layers without any transformation; The MSF module is defined as the feature map X1 from the Stage 1 stage connected in series with a Split module to obtain the feature map X2 of the Stage 2 stage and the feature map X3 of the Stage 3 stage. The feature map X3 of the Stage 3 stage is connected in parallel with an MConv module, a 7×7 convolution module and an Identity module, and then a bitwise addition operation is performed, and then a 1×1 convolution is connected in series to obtain the feature map X5 of the Stage 5 stage. The feature maps X2, X3, X4 and X5 from the four stages of Stage 2, Stage 3, Stage 4 and Stage 5 are channel-fused and connected in series with a 1×1 convolution module, and then a bitwise addition operation is performed with the feature map X1 of the Stage 1 stage.

4. The TARD electric vehicle passenger violation intelligent detection method according to claim 1 is characterized in that: The RMSFE module structure described in S2.1 is as follows: The RMSFE module is defined as a 1×1 convolution module that reduces the number of channels C6 of the feature map X6 of the Stage 6 stage by one half to obtain the feature map X7 of the Stage 7 stage with the number of channels C7. It is then connected in series with an MSF module and bitwise added to the feature map X7 of the Stage 7 stage. A 1×1 convolution module is then used to double the number of channels C7 and bitwise added to the feature map X6 of the Stage 6 stage to obtain the feature map X8 of the Stage 8 stage. A 1×1 convolution module is then used to double the number of channels C8 of the feature map X8 of the Stage 8 stage.

5. The TARD electric vehicle passenger violation intelligent detection method according to claim 1 is characterized in that: The FFM module structure described in S2.2 is as follows: The MSFM module is defined as a 5×5, 7×7, 9×9, and 11×11 channel-by-channel convolution Depthwise Convolution module connected in parallel, then connected in parallel with an Identity module, followed by a bitwise addition operation, and then connected in series with a 1×1 convolution module; The FFM module is defined as the feature maps X9, X10 and X11 from Stage9, Stage10 and Stage11 are input into the FFM module. The feature map X9 of Stage9 is connected in series with a downsampling module, the feature map X10 of Stage10 is connected in series with a 1×1 convolution module, and the feature map X11 of Stage11 is connected in series with an upsampling module and a 1×1 convolution module. The feature maps X9, X10 and X11 from Stage9, Stage10 and Stage11 are channel fused and then connected in parallel with an MSFM module and an Identity module for bitwise addition.

6. The TARD electric vehicle passenger violation intelligent detection method according to claim 1 is characterized in that: The custom feature focusing diffusion mechanism described in S2.2 optimizes the neck feature fusion of the model, and the structure is as follows: The custom feature focusing diffusion mechanism optimizes Stage 6 to Stage 17, and diffuses the feature maps after focusing fusion of the IPR_FFM module according to three different detection scales: large, medium and small. The three feature maps generated by the diffusion mechanism are further focused and fused through the IPR_FFM module, and the three feature maps after focusing fusion are diffused to the detection head of the corresponding size.

7. The TARD electric vehicle passenger violation intelligent detection method according to claim 1 is characterized in that: The original TARD model is trained with the SIMD public remote sensing dataset described in S2.3 to obtain the trained TARD detection model and the best training weight best.pt. The validation set of the SIMD public remote sensing dataset is validated using best.pt, and then the model performance is evaluated. The performance evaluation includes: Precision (P): Recall (R): Average Mean Accuracy (mAP): Among them, TP refers to the number of samples that are positive and predicted as positive, FP refers to the number of samples that are negative and predicted as positive, FN refers to the number of samples that are positive and predicted as negative, n is the number of target categories detected, AP i is the AP of the ith target class.

8. A TARD electric vehicle passenger violation intelligent detection device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images