Terminal vehicle type lightweight identification method and device based on digital retina, computer equipment and storage medium

Abstract

The embodiment of the invention relates to the field of artificial intelligence, and provides a terminal vehicle type lightweight identification method and device based on a digital retina, computer equipment and a storage medium, and the method comprises the steps: obtaining a to-be-identified vehicle image frame, and carrying out the preprocessing of the vehicle image frame based on a digital retina technology; generating a region of interest of the vehicle candidate frame based on a lightweight detector, and cutting to obtain a single-frame cut image and a short-time window cut sequence; performing feature extraction on the single-frame cut image based on a static appearance flow technology, and performing feature extraction on the short-time window cut sequence based on a dynamic motion flow technology; performing structure key point detection and proportion feature calculation on the single-frame cut image; performing feature fusion to obtain a joint feature vector; and determining an identification result associated with the vehicle image frame according to the joint feature vector. By adopting the method, the accuracy of vehicle identification can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a lightweight method, apparatus, computer device, and storage medium for terminal vehicle type recognition based on digital retina. Background Technology

[0002] In the field of artificial intelligence technology, urban traffic monitoring systems are now widely deployed in scenarios such as roads, parking lots, checkpoints, and toll stations. Automatic identification of different types of vehicles has become an important part of traffic management.

[0003] Although some intelligent front-ends have acquired recognition capabilities in recent years, traditional single-frame appearance recognition is susceptible to the effects of lighting, angle, and occlusion, and has a weak ability to distinguish similar car models. It also suffers from insufficient recognition robustness, which can easily lead to misidentification and affect the practicality of the system.

[0004] In summary, traditional vehicle recognition technologies suffer from low recognition accuracy. Summary of the Invention

[0005] This application provides a lightweight vehicle type identification method, device, computer equipment, and computer storage medium based on digital retina, which can improve the accuracy of vehicle identification.

[0006] In a first aspect, embodiments of this application provide a lightweight vehicle type identification method based on a digital retina, including:

[0007] A vehicle image frame to be identified is acquired, and the vehicle image frame is preprocessed based on digital retina technology to obtain preprocessed data; the preprocessed data includes digital retina feature maps.

[0008] Based on the lightweight detector, the region of interest (ROI) of the vehicle candidate box corresponding to the preprocessed data is generated. For any ROI, a single-frame cropped image and a short-time window cropping sequence are obtained by cropping.

[0009] The single-frame cropped image is subjected to feature extraction based on static appearance flow technology to obtain appearance feature vector, and the short-time window cropping sequence is subjected to feature extraction based on dynamic motion flow technology to obtain motion feature vector.

[0010] Structural key point detection and proportional feature calculation are performed on the single-frame cropped image to obtain a geometric proportional feature vector;

[0011] The digital retina feature map, the appearance feature vector, the motion feature vector, and the geometric proportion feature vector are fused to obtain a joint feature vector;

[0012] The recognition result associated with the vehicle image frame is determined based on the joint feature vector.

[0013] Secondly, embodiments of this application provide a lightweight vehicle type recognition device based on a digital retina, which has the function of implementing the lightweight vehicle type recognition method based on a digital retina provided in the first aspect above. The function can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above function, and the modules can be software and / or hardware.

[0014] In one possible design, the device includes:

[0015] The data acquisition and preprocessing module is used to acquire vehicle image frames to be identified, and preprocess the vehicle image frames based on digital retina technology to obtain preprocessed data; the preprocessed data includes digital retina feature maps.

[0016] The cropping module is used to generate the region of interest of vehicle candidate boxes corresponding to the preprocessed data based on the lightweight detector, and crop to obtain a single-frame cropped image and a short-time window cropping sequence for any region of interest.

[0017] The feature extraction module is used to extract features from the single-frame cropped image based on static appearance flow technology to obtain appearance feature vectors, and to extract features from the short-time window cropping sequence based on dynamic motion flow technology to obtain motion feature vectors.

[0018] The proportional feature calculation module is used to perform structural key point detection and proportional feature calculation on the single-frame cropped image to obtain a geometric proportional feature vector.

[0019] The feature fusion module is used to fuse the digital retina feature map, the appearance feature vector, the motion feature vector, and the geometric proportion feature vector to obtain a joint feature vector;

[0020] The recognition module is used to determine the recognition result associated with the vehicle image frame based on the joint feature vector.

[0021] In another aspect, this application provides a computer device including at least one connected processor and a memory, wherein the memory is used to store program code, and the processor is used to invoke the program code in the memory to execute the methods described in the above aspects.

[0022] In another aspect, this application provides a computer storage medium including instructions that, when executed on a computer, cause the computer to perform the methods described in the above aspects.

[0023] In another aspect, this application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the methods described in the above aspects.

[0024] Compared to traditional vehicle recognition technologies, the technical solution of this application introduces a digital retinal feature stream to achieve multimodal feature perception of vehicle images at different scales and in different regions, forming a feature vector stream. Combined with a dual-stream convolutional neural network, it simultaneously extracts static appearance features and dynamic temporal features, and then fuses the structural keypoint method to extract geometric and topological information of significant parts. This achieves highly robust vehicle type recognition, improves the ability to distinguish similar vehicle models, reduces the frequency of false recognition, improves the practicality of the system, and enhances the accuracy of vehicle recognition, thereby improving the effect of vehicle recognition. Attached Figure Description

[0025] Figure 1 This is an application environment diagram from one embodiment;

[0026] Figure 2 This is a flowchart illustrating a lightweight vehicle type identification method based on digital retina in one embodiment.

[0027] Figure 3 This is a schematic diagram of the overall technical solution flow in one embodiment;

[0028] Figure 4 This is a schematic diagram of the application task flow in one embodiment;

[0029] Figure 5 This is a structural block diagram of a lightweight vehicle type recognition device based on a digital retina in one embodiment.

[0030] Figure 6 This is an internal structural diagram of a computer device in one embodiment;

[0031] Figure 7 This is a diagram of the internal structure of a computer device in another embodiment. Detailed Implementation

[0032] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The division of modules in the embodiments of this application is merely a logical division; in actual applications, there may be other division methods. For example, multiple modules may be combined into or integrated into another system, or some features may be ignored or not performed. Additionally, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interface, and the indirect coupling or communication connection between modules may be electrical or other similar forms, none of which are limited in the embodiments of this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed among multiple circuit modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of this application.

[0033] Figure 1 As shown in the application environment diagram of one embodiment, this application provides a lightweight vehicle type recognition method based on digital retina, which can be applied to, for example... Figure 1 In the application scenario shown, terminal 102 communicates with server 104 via a network.

[0034] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0035] It should be noted that the terminal 102 involved in the embodiments of this application can be a wired terminal or a wireless terminal, and can be a device that provides voice and / or data connectivity to a user, a handheld device with wireless connectivity, or other processing devices connected to a wireless modem. The wireless terminal can communicate with one or more core networks via a wireless access network, and the wireless terminal can be a mobile terminal, such as a mobile phone or a computer with a mobile terminal.

[0036] Figure 2This is a flowchart illustrating a lightweight vehicle type recognition method based on digital retina in one embodiment. Figure 2 As shown, the lightweight vehicle type identification method based on digital retina provided in this application embodiment, which can be simply referred to as the identification method provided in this application, includes:

[0037] S201, acquire the vehicle image frame to be identified, and preprocess the vehicle image frame based on digital retina technology to obtain preprocessed data.

[0038] Among them, the vehicle image frame is the target image to be identified as the vehicle type, and the data exists in the form of image frames.

[0039] The preprocessed data includes digital retinal feature maps, which can be denoted as f_DR.

[0040] S202: Based on the lightweight detector, generate the region of interest (ROI) of the vehicle candidate box corresponding to the preprocessed data. For any ROI, crop to obtain a single-frame cropped image and a short-time window cropping sequence.

[0041] Among them, the region of interest of the vehicle candidate box refers to the vehicle candidate box ROI (Region of Interest), the single-frame cropped image A(t) is used for appearance flow, and the short-time window cropping sequence A(tn...t+n) is used for optical flow calculation.

[0042] S203, based on static appearance flow technology, feature extraction is performed on a single-frame cropped image to obtain an appearance feature vector; and based on dynamic motion flow technology, feature extraction is performed on a short-time window cropping sequence to obtain a motion feature vector.

[0043] In particular, step S203 is the process of dual-stream feature extraction provided in this application. Dual-stream refers to static appearance stream and dynamic motion stream. The appearance feature vector can be denoted as f_app, and the motion feature vector can be denoted as f_mov.

[0044] S204 performs structural key point detection and proportional feature calculation on a single-frame cropped image to obtain a geometric proportional feature vector.

[0045] Among them, the geometric proportion feature vector is the feature obtained after structural key point detection and proportion feature calculation, denoted as f_struct.

[0046] S205 performs feature fusion on the digital retina feature map, appearance feature vector, motion feature vector, and geometric proportion feature vector to obtain a joint feature vector.

[0047] The joint feature vector can be denoted as f, and its expression can be:

[0048] f = concat( w1·f_app, w2·f_mov, w3·f_DR, w4·f_struct ); where w1 to w4 are the corresponding weights.

[0049] S206, determine the recognition result associated with the vehicle image frame based on the joint feature vector.

[0050] The recognition result is the final recognition result of the image frame of the vehicle to be recognized, which may be the information corresponding to the vehicle type.

[0051] Compared to traditional vehicle recognition technologies, this embodiment first acquires the vehicle image frame to be recognized, then preprocesses the vehicle image frame based on digital retina technology, and then crops the region of interest (ROI) of the vehicle candidate box based on a lightweight detector. Simultaneously, feature extraction is performed based on static appearance flow and dynamic motion flow technologies, and structural keypoint detection and proportional feature calculation are performed on the cropped single-frame image. Finally, feature fusion is performed on the digital retina feature map, appearance feature vector, motion feature vector, and geometric proportional feature vector to determine the recognition result associated with the vehicle image frame. The technical solution of this embodiment, by introducing digital retina feature flow, enables multimodal feature perception of vehicle images at different scales and in different regions to form a feature vector flow. Combined with a dual-stream convolutional neural network, it simultaneously extracts static appearance features and dynamic temporal features, and then fuses the structural keypoint method to extract geometric and topological information of significant parts. This achieves highly robust vehicle type recognition, improves the ability to distinguish similar vehicle models, reduces the frequency of false recognition, improves the system's practicality, and enhances the accuracy of vehicle recognition, thereby improving the overall vehicle recognition effect.

[0052] Optionally, in some embodiments of this application, obtaining the vehicle image frame to be identified includes: continuously acquiring images based on a short-time window to obtain vehicle image frames for time-series processing and optical flow calculation.

[0053] The short-term window refers to the window that saves 8 to 16 frames of the image.

[0054] Among them, time-series processing is used to analyze the changing patterns of consecutive frames. For example, the movement trajectories and vibration amplitudes of trucks and cars are different. Time-series processing can capture these differences, making it easier to distinguish similar vehicle models.

[0055] Optical flow calculation refers to calculating the direction and speed of pixel movement. For example, for the same type of vehicle, stationary and moving vehicles can be distinguished by optical flow, and it can also determine whether the vehicle is changing lanes or turning, making the recognition more accurate.

[0056] For example, the identification method provided in this application includes step S301: data acquisition and buffering, specifically including: continuously acquiring image frames I(t-n...t...t+n), and saving a short window (e.g., 8 to 16 frames) for time-series processing and optical flow calculation.

[0057] Optionally, in some embodiments of this application, vehicle image frames are preprocessed based on digital retina technology to obtain preprocessed data, including: performing local contrast enhancement on the vehicle image frames to obtain enhanced data; performing multi-scale edge extraction on the enhanced data to obtain an edge response map; and performing temporal nonlinear suppression processing on the edge response map to obtain preprocessed data.

[0058] Local contrast enhancement refers to adaptive contrast stretching of a local window, and the enhanced data is the data corresponding to the vehicle image frame obtained after local contrast enhancement.

[0059] Among them, multi-scale edge extraction refers to the process of obtaining edge response maps using a biomimetic filter bank.

[0060] Among them, temporal nonlinear suppression processing refers to pulse filtering of the response of consecutive frames to remove static texture interference and highlight the edges of moving targets; then, the local statistics of DR are calculated as DR feature vectors to obtain the preprocessed data.

[0061] Digital Retina is a biomimetic vision technology that mimics the image perception and processing mechanisms of the human retina. Its key feature is that high-resolution image acquisition and initial feature extraction are performed on the edge, primarily using vectorized features of reported events; core deep learning and intelligent analysis are conducted on the cloud, achieving a low-power, high-efficiency visual processing architecture with edge-cloud collaboration. This technology balances rapid front-end response with global cloud optimization, making it suitable for intelligent monitoring and real-time decision-making in complex and dynamic environments.

[0062] For example, the recognition method provided in this application includes step S302: digital retina preprocessing, specifically including:

[0063] Perform the following operation on each frame:

[0064] Local Contrast Enhancement (LCE): Adaptive contrast stretching is performed on a local window.

[0065] Multi-scale edge extraction: Edge response map E_s(scale) is obtained using a biomimetic filter bank (high-pass combination, center-periphery suppression);

[0066] Temporal nonlinearity suppression: Pulse filtering is applied to the response of consecutive frames to remove static texture interference and highlight the edges of moving targets;

[0067] The final output is the DR (Digital Retina) feature map F_DR (usually a sparse, low-rank matrix) corresponding to each ROI, and the local statistics of DR (such as edge density, orientation histogram, etc.) are calculated as the DR feature vector.

[0068] In another embodiment, in step S202, the region of interest of the vehicle candidate box is the vehicle candidate box ROI.

[0069] For example, the recognition method provided in this application includes step S303: candidate box generation and cropping, specifically including: generating vehicle candidate boxes (ROIs) (Regions of Interest) using a lightweight detector (such as Tiny-Detector or based on connected components after image segmentation). For each ROI, the following are simultaneously cropped to obtain: a single-frame cropped image A(t) (for appearance flow), a short-time window cropping sequence A(tn...t+n) (for optical flow calculation), and the corresponding region F_DR_ROI of the DR feature map.

[0070] In another embodiment, in step S203, feature extraction is performed on a single-frame cropped image based on static appearance flow technology to obtain an appearance feature vector.

[0071] The static appearance flow technique can be implemented based on an Appearance-CNN network, such as MobileNetV3 or a custom Light-CNN. More specifically, the network can be composed of the following layers: convolution, depthwise separable convolution, SE / CBAM attention, global pool, and 128-d vector.

[0072] The resulting appearance feature vector can be denoted as f_app.

[0073] Optionally, in some embodiments of this application, feature extraction is performed on the short-term window cropping sequence based on dynamic motion flow technology to obtain motion feature vectors, including: calculating the optical flow of the short-term window cropping sequence to obtain motion feature vectors; and / or, inputting the short-term window cropping sequence into a lightweight convolutional neural network based on frame difference stack as input to output motion feature vectors.

[0074] The dynamic motion flow technology can be implemented by directly calculating optical flow, or by using a lightweight convolutional neural network (such as Motion-CNN), or by implementing it in batches using either of the above methods.

[0075] The motion feature vector can be denoted as f_mov.

[0076] For example, the identification method provided in this application includes step S304: dual-stream feature extraction, specifically including:

[0077] Static Appearance Flow (Appearance-CNN): Takes A(t) as input to a lightweight CNN (e.g., MobileNetV3 or a custom Light-CNN) and outputs an appearance feature vector f_app (size 128 or 256). Layers: Convolution -> Depthwise Separable Convolution -> SE / CBAM Attention -> Global Pool -> 128-d Vector.

[0078] Motion-CNN: Calculates the optical flow (sparse / dense) of A(tn...t+n), or directly uses the frame difference stack as input to another lightweight CNN, and outputs a motion feature vector f_mov (size 64 to 128).

[0079] The goal of feature extraction based on dynamic motion flow is to capture temporal information such as vehicle acceleration / deceleration, vibration, and contour changes, and to distinguish the differences in acceleration curves of vehicles such as trucks, cars, and buses.

[0080] Optionally, in some embodiments of this application, structural key point detection and proportional feature calculation are performed on a single-frame cropped image to obtain a geometric proportional feature vector, including: using a key point detection network to detect structural key points in the single-frame cropped image to obtain a preset number of structural key point coordinates; and calculating proportional features based on the structural key point coordinates to obtain a geometric proportional feature vector.

[0081] Here, a single frame cropped image is denoted as A(t), and the preset number of structural key point coordinates refers to the N structural key point coordinates P = {p1...pN}.

[0082] The geometric proportion eigenvector is denoted as f_struct.

[0083] Optionally, in some embodiments of this application, the geometric proportion feature vector is obtained by calculating the proportion features based on the coordinates of the structural key points, including: in the case of partial missing key points, calculating the initial geometric proportion data and confidence level using the available key points, and determining the geometric proportion feature vector based on the initial geometric proportion data and confidence level.

[0084] The initial geometric scale data is preliminary geometric scale-related data calculated using available key points. The credibility, also known as the confidence level, is used to evaluate the validity or usability of the initial geometric scale data.

[0085] The process of calculating the coordinates of key points can either use complete key points directly for settlement, or use available key points to calculate credibility when some key points are missing, and then weight them according to credibility in subsequent fusion.

[0086] For example, the identification method provided in this application includes step S305: structural key point detection and proportional feature calculation, specifically including:

[0087] For A(t), use a keypoint detection network (a lightweight version of KeypointNet) to obtain N structural keypoint coordinates P = {p1...pN}. The structural keypoint coordinates include at least the coordinates of the left and right front lights, the left and right rear lights, the center of the left / right front wheels, the center of the left / right rear wheels, the front and rear points of the roof, and the center of the license plate.

[0088] Calculate the geometric proportion feature vector f_struct, such as: vehicle length / width ratio L / W; wheelbase to vehicle length ratio; roof height ratio (roof height / body width); headlight spacing / body width; and the relative position of the license plate within the vehicle height.

[0089] When key points are partially missing (occluded), the credibility is calculated using the available key points and weighted according to credibility in subsequent fusion.

[0090] Optionally, in some embodiments of this application, determining the recognition result associated with the vehicle image frame based on the joint feature vector includes: determining the vehicle category probability vector corresponding to the joint feature vector based on the classification head; and, based on the vehicle category probability vector, performing intra-time window voting and confidence fusion for continuous regions of interest on the same trajectory to obtain the recognition result.

[0091] The classification head is used to input the joint feature vector into a small MLP or lightweight Transformer classifier and output a vehicle class probability vector, which can be denoted as p.

[0092] For example, the identification method provided in this application includes step S306: feature fusion strategy, specifically including:

[0093] Feature-Level Fusion: Constructing the joint feature vector as follows:

[0094] f = concat( w1·f_app, w2·f_mov, w3·f_DR, w4·f_struct ), where the weights w1 to w4 can be fixed or adaptively adjusted based on the scene (see step S308 for details).

[0095] In the feature fusion process in step S306, an execution unit equipped with an attention / gating module and a classification head is used. Specifically, the attention / gating module is used to introduce a gating network (GatingNet) to automatically adjust the weights of each subflow based on DR sparse activation, illumination estimation, and occlusion confidence. The classification head (Classifier) ​​is used to input f into a small MLP or lightweight Transformer classifier and output the vehicle class probability vector p = softmax(MLP(f)).

[0096] For example, the recognition method provided in this application includes step S307: post-decision processing and multi-frame voting, specifically including:

[0097] For consecutive ROIs of the same trajectory (associated through trackers such as SORT / DeepSORT), a time window voting and confidence fusion is performed to finally output the vehicle type label and confidence score at the trajectory level.

[0098] Additionally, when the confidence level is below a threshold (e.g., < 0.6), the system outputs "Undetermined / Requires Review" and uploads the keyframes to the backend for manual or larger model review.

[0099] For example, the recognition method provided in this application includes step S308: scene adaptive weights and strategies, specifically including:

[0100] The sub-stream weights are dynamically adjusted based on external scene information. In bright light / nighttime conditions, the weights of DR and structural keypoints are increased, while the weights of color and texture are decreased. In high-speed scenes, the weights of motion flow are enhanced. In addressing severe occlusion issues, GatingNet reduces the contribution of damaged sub-streams by relying on the sparse contours of DR and existing keypoints.

[0101] Additionally, it should be noted that the weight adjustment strategy can be based on simple rules or online learning (such as using a small policy network to continuously fine-tune at the edge).

[0102] The technical research process and other technical details of this application are described below with reference to a specific embodiment.

[0103] This application provides a lightweight vehicle recognition method based on the digital retina mechanism, which overcomes the shortcomings of existing vehicle type recognition methods that rely on a single deep learning model (such as the YOLO series) for target detection. It also solves the problems of insufficient vehicle recognition accuracy and poor generalization ability in complex scenarios (such as changes in lighting, partial occlusion, and similar vehicle appearances).

[0104] This application introduces digital retinal feature flow to achieve multimodal feature perception of vehicle images at different scales and in different regions, forming a feature vector flow; it combines a two-stream convolutional neural network (CNN) to extract both static appearance features and dynamic temporal features; and it further integrates the structural keypoint method to extract geometric and topological information of significant parts such as headlights, windows, and contour edges, thereby achieving highly robust recognition of vehicle types.

[0105] This application can effectively improve the recognition accuracy and stability of vehicles in multiple scenarios, multiple angles, and complex backgrounds, and has strong practical value and promotion significance.

[0106] To overcome the problems of traditional single-frame appearance recognition being affected by lighting, angle, and occlusion, and having a weak ability to distinguish similar vehicle models, this application proposes to combine three types of information streams for vehicle type discrimination on edge terminals: (1) Digital Retina Feature Stream (DR-Stream), i.e., a sparse structured visual feature vector stream generated by biomimetic preprocessing; (2) Two-Stream Convolutional Network (Two-StreamCNN), i.e., static appearance stream and dynamic motion stream; (3) Structural Keypoints, i.e., vehicle keypoint detection and geometric proportion features. The above three types of features are fused in multiple stages at the feature layer and decision layer, thereby achieving highly robust and low-latency vehicle type recognition on embedded / edge devices.

[0107] Figure 3 This is a schematic diagram of the overall technical solution flow in one embodiment. Combining steps S301 to S308 in the above embodiments, the overall flow of the identification method provided in this application is as follows: Figure 3 As shown.

[0108] In addition, this embodiment also provides a model training method applied to the recognition method, the training details and data preparation process of which specifically include:

[0109] For the dataset, vehicle images and video clips from multiple scenes, lighting conditions, and angles were collected. The annotation process used bounding boxes, vehicle model labels, keypoint coordinates, and inter-frame tracking IDs. It is recommended to have ≥50k vehicle instances (including various vehicle models and occluded / nighttime samples).

[0110] Regarding parameterized training of the DR module, DR is a non-learning filter and temporal suppression, but if it contains learnable parameters (such as learnable center-periphery kernels), it can be fine-tuned end-to-end using small-scale data.

[0111] In terms of training the dual-subnet, Appearance-CNN (for classification tasks) and Motion-CNN (for trajectory classification or auxiliary tasks) are pre-trained and converged separately; then, joint fine-tuning (joint loss) is performed in the fusion layer: L = L_cls + λ1·L_keypoint + λ2·L_temporal_consistency; finally, the keypoint loss is L2 coordinate regression loss or heatmap cross-entropy loss.

[0112] Data augmentation techniques include random cropping, bionic noise, lighting perturbation, occlusion synthesis (patch occlusion), and optical flow perturbation.

[0113] In terms of distillation / quantization, to adapt to edge devices, knowledge distillation is used after training to obtain a small model, which is then quantized using INT8 / 16 and the accuracy degradation is tested.

[0114] In terms of evaluation metrics, Accuracy, mAP (by vehicle type), trajectory consistency, runtime latency (ms / frame), model size (MB), and power consumption (W) are used for evaluation.

[0115] Figure 4 As a schematic diagram of the application task flow in one embodiment, the recognition method provided in this application further includes: S401: The traffic management center issues tasks (such as vehicle flow statistics and illegal parking detection). S402: The edge device loads the corresponding lightweight recognition model according to the task. S403: The recognition module runs and outputs real-time vehicle classification results. S404: The edge device summarizes the results and reports them to the traffic management platform. S405: The platform receives the recognition data and generates traffic flow reports or alarm information.

[0116] In addition, this embodiment also provides an identification system for implementing the identification method, which consists of the following modules:

[0117] Image Acquisition Module: Front-end camera (visible light / infrared), outputting continuous video frames. Digital Retina Preprocessing Module (DR Module): Performs local contrast enhancement and retina-style multi-scale edge / temporal suppression on frames (resulting in sparse activation maps and feature maps). Target Candidate Generation Module: Generates vehicle candidate boxes (ROIs) based on a lightweight detector or traditional segmentation / background modeling. Dual-Stream Feature Extraction Module: Includes a static appearance subnet (Appearance-CNN) and a dynamic motion subnet (Motion-CNN, inputting optical flow or temporal difference). Keypoint Detection Module: Outputs several vehicle structural keypoints based on a lightweight keypoint detector (e.g., headlight center, taillight center, wheel center, license plate center, front and rear points of the roof, etc.). Feature Fusion and Discrimination Module: Fuses DR features, dual-stream CNN features, and structural keypoint proportion features according to a strategy and inputs them into a classifier to output vehicle model labels and confidence scores. Result Output and Reporting Module: Provides local storage, a reporting interface, and a scene-adaptive strategy (reporting only labels and metadata in low-bandwidth scenarios).

[0118] In a specific embodiment, such as deployment at an urban checkpoint, the edge system of this method is deployed at an intersection in City A, with the camera operating at 30fps and a short window of 8 frames. Regarding the DR module parameter settings, the local window is 32×32, the temporal impulse threshold is 0.2; the Appearance-CNN output is 128-d, the Motion-CNN output is 64-d, and the key point detection output is 10 points. In terms of testing, after two weeks of testing, the overall vehicle model recognition accuracy improved from 87% (baseline YOLO+Classifier) ​​to 94%; the improvement was even more significant at night, increasing from 72% to 91%.

[0119] The key technical features of the identification method provided in this application are as follows.

[0120] (1) Digital Retina Prior and Sparse Feature Flow: Using biologically inspired “center-periphery + temporal suppression” as a pre-module, it outputs sparse structured visual activation, which is significantly different from traditional image enhancement or histogram methods. (2) Dual Flow + Structural Keypoint Combination: Unlike using only a single-frame YOLO + classifier, this scheme uses three complementary information types—static appearance, short-term motion pattern, and vehicle structural geometry—as the discrimination criteria, and performs adaptive weighting through gating attention, fundamentally improving the ability to distinguish similar vehicle types (such as SUV vs MPV, van vs small truck). (3) Scene Adaptive Weight Strategy: Based on the “sparseness / edge quality” of DR output and external illumination estimation, the sub-flow weights are dynamically adjusted to solve the problem of sub-flow failure caused by illumination / occlusion. (4) Trajectory-Level Voting and Confidence Management: The frame-level classification is upgraded to trajectory-level output, and a low-confidence sample automatic back-to-kernel mechanism is implemented to reduce the misjudgment rate and improve the long-term application robustness.

[0121] The technical effects of the identification method provided in this application are as follows.

[0122] (1) Vehicle outline information can be recovered in low light / strong reflective scenes, thereby improving the reliability of subsequent key point and appearance recognition. (2) Improved real-time performance: Reduced image uploading and back-end calculation, recognition is completed on the edge, and the latency is significantly reduced. (3) Improved recognition accuracy: Improved robustness of vehicle recognition in complex environments. (4) High deployment flexibility: Supports rapid application in multiple scenarios such as roads, parking lots, and toll stations, reducing construction and maintenance costs.

[0123] In addition, the recognition method provided in this application can be applied to the fields of intelligent transportation, computer vision and edge computing. Due to its high deployment flexibility, it may also be applied to scenarios such as roads, parking lots, checkpoints, and toll stations in the future to reduce construction and maintenance costs and overcome the problems of traditional single-frame appearance recognition being affected by lighting, angle and occlusion and having a weak ability to distinguish similar vehicle models.

[0124] The recognition method provided in this application has the main advantage over traditional technologies in that it introduces a digital retina feature vector stream and fuses it with a dual-stream CNN (static + dynamic) and structural keypoint method in multiple layers to form a multimodal discrimination framework for vehicle type identification. This framework differs from existing single appearance recognition methods based on YOLO in terms of algorithm layer (feature source and fusion strategy), module implementation (DR preprocessing, gating fusion), and deployment method (edge ​​adaptive weight, trajectory-level voting), demonstrating clear inventiveness and industrial feasibility.

[0125] The main innovation of the recognition method provided in this application compared with traditional technologies lies in the combination of digital retina bionic feature extraction with dual-stream CNN architecture and structural key point detection, and the lightweight optimization to adapt it to edge devices, thereby achieving efficient and low-power vehicle type recognition in complex environments. This solution breaks through the limitation of existing vehicle type recognition methods that rely solely on a single depth model.

[0126] Figures 1 to 4 Any technical feature in the embodiments corresponding to any of the above items is also applicable to the embodiments of this application. Figures 5 to 7 The corresponding implementation examples will not be repeated hereafter.

[0127] The above describes a lightweight vehicle type identification method based on digital retina in the embodiments of this application. The following describes the apparatus or device for performing the above method.

[0128] Figure 5 This is a structural block diagram of a lightweight vehicle type recognition device based on a digital retina in one embodiment, with reference to... Figure 5 The device includes:

[0129] The data acquisition and preprocessing module 501 is used to acquire vehicle image frames to be identified, preprocess the vehicle image frames based on digital retina technology, and obtain preprocessed data; the preprocessed data includes digital retina feature maps.

[0130] The cropping module 502 is used to generate the region of interest of the vehicle candidate box corresponding to the preprocessed data based on the lightweight detector, and crop to obtain a single-frame cropped image and a short-time window cropping sequence for any region of interest.

[0131] The feature extraction module 503 is used to extract features from a single-frame cropped image based on static appearance flow technology to obtain an appearance feature vector, and to extract features from a short-time window cropping sequence based on dynamic motion flow technology to obtain a motion feature vector.

[0132] The proportional feature calculation module 504 is used to perform structural key point detection and proportional feature calculation on a single frame cropped image to obtain a geometric proportional feature vector.

[0133] The feature fusion module 505 is used to fuse the digital retina feature map, appearance feature vector, motion feature vector and geometric proportion feature vector to obtain a joint feature vector;

[0134] The recognition module 506 is used to determine the recognition result associated with the vehicle image frame based on the joint feature vector.

[0135] In this embodiment of the application, based on, as follows Figure 5The connections between the modules shown in the diagram demonstrate how the cooperation between these modules can improve the accuracy of vehicle recognition.

[0136] In another embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, it includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores relevant data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the various methods described in the above embodiments.

[0137] In yet another embodiment, a computer device is provided, such as a terminal, whose internal structure diagram may be as follows: Figure 7 As shown, it includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements the various methods described in the above embodiments.

[0138] Those skilled in the art will understand that Figure 6 and Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. It may also include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements, in order to achieve functions such as a terminal or server.

[0139] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0140] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the systems, devices, equipment, modules or units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0141] In the embodiments provided in this application, it should be understood that the disclosed systems, devices, equipment, or methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or modules may be electrical, mechanical, or other forms.

[0142] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0144] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0145] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0146] The technical solutions provided in the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. A lightweight vehicle type recognition method based on digital retina, characterized in that, The method includes: A vehicle image frame to be identified is acquired, and the vehicle image frame is preprocessed based on digital retina technology to obtain preprocessed data; the preprocessed data includes digital retina feature maps. Based on the lightweight detector, the region of interest (ROI) of the vehicle candidate box corresponding to the preprocessed data is generated. For any ROI, a single-frame cropped image and a short-time window cropping sequence are obtained by cropping. The single-frame cropped image is subjected to feature extraction based on static appearance flow technology to obtain appearance feature vector, and the short-time window cropping sequence is subjected to feature extraction based on dynamic motion flow technology to obtain motion feature vector. Structural key point detection and proportional feature calculation are performed on the single-frame cropped image to obtain a geometric proportional feature vector; The digital retina feature map, the appearance feature vector, the motion feature vector, and the geometric proportion feature vector are fused to obtain a joint feature vector; The recognition result associated with the vehicle image frame is determined based on the joint feature vector.

2. The method according to claim 1, characterized in that, The process of acquiring the vehicle image frame to be identified includes: Continuous acquisition is performed by saving short-time windows to obtain vehicle image frames for time-series processing and optical flow calculation.

3. The method according to claim 1, characterized in that, The preprocessing of the vehicle image frames based on digital retina technology to obtain preprocessed data includes: The vehicle image frame is subjected to local contrast enhancement to obtain enhanced data; Multi-scale edge extraction is performed on the enhanced data to obtain an edge response map; The edge response map is subjected to temporal nonlinear suppression processing to obtain the preprocessed data.

4. The method according to claim 1, characterized in that, The feature extraction of the short-window cropping sequence based on dynamic motion flow technology yields a motion feature vector, including: Calculate the optical flow of the short-window clipping sequence to obtain the motion feature vector; And / or, input the short-window cropping sequence into a lightweight convolutional neural network based on a frame difference stack as input to output the motion feature vector.

5. The method according to claim 1, characterized in that, The step of performing structural key point detection and proportional feature calculation on the single-frame cropped image to obtain a geometric proportional feature vector includes: The key point detection network is used to detect structural key points in the single-frame cropped image to obtain a preset number of structural key point coordinates. The geometric proportion feature vector is obtained by calculating the proportional features based on the coordinates of the key points of the structure.

6. The method according to claim 5, characterized in that, The step of calculating the geometric proportion feature vector based on the coordinates of the key structural points includes: In the case of missing key points, the initial geometric scale data and confidence level are calculated using the available key points, and the geometric scale feature vector is determined based on the initial geometric scale data and the confidence level.

7. The method according to claim 1, characterized in that, The step of determining the recognition result associated with the vehicle image frame based on the joint feature vector includes: The vehicle category probability vector corresponding to the joint feature vector is determined based on the classification head; Based on the vehicle category probability vector, voting and confidence fusion are performed within a time window for consecutive regions of interest along the same trajectory to obtain the recognition result.

8. A lightweight vehicle type recognition device based on digital retina, characterized in that, The device includes: The data acquisition and preprocessing module is used to acquire vehicle image frames to be identified, and preprocess the vehicle image frames based on digital retina technology to obtain preprocessed data; the preprocessed data includes digital retina feature maps. The cropping module is used to generate the region of interest of vehicle candidate boxes corresponding to the preprocessed data based on the lightweight detector, and crop to obtain a single-frame cropped image and a short-time window cropping sequence for any region of interest. The feature extraction module is used to extract features from the single-frame cropped image based on static appearance flow technology to obtain appearance feature vectors, and to extract features from the short-time window cropping sequence based on dynamic motion flow technology to obtain motion feature vectors. The proportional feature calculation module is used to perform structural key point detection and proportional feature calculation on the single-frame cropped image to obtain a geometric proportional feature vector. The feature fusion module is used to fuse the digital retina feature map, the appearance feature vector, the motion feature vector, and the geometric proportion feature vector to obtain a joint feature vector; The recognition module is used to determine the recognition result associated with the vehicle image frame based on the joint feature vector.

9. A computer device, characterized in that, The computer device includes: At least one processor and memory; The memory is used to store program code, and the processor is used to call the program code stored in the memory to execute the method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, It includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 7.

Images