Vehicle identification method, device and edge device

Abstract

Embodiments of the present application provide a vehicle identification method, device and edge equipment, and relate to the technical field of vehicle analysis. When a reliable license plate number exists for a vehicle leaving a field, the license plate number is used to match and associate the vehicle leaving the field with a vehicle having entered the field. When a reliable license plate number does not exist for the vehicle leaving the field, vehicle fingerprint of the vehicle leaving the field is generated according to vehicle color, vehicle type, vehicle appearance normalization information and vehicle bounding box information of the vehicle leaving the field, and at least one of the vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information and vehicle bounding box information is used to perform step-by-step matching with the vehicle having entered the field. In actual scenarios such as no license plate or license plate blur, the vehicle identification is no longer dependent on a single license plate number, but is supported by structured appearance features and multi-source attributes, so that the matching is not interrupted due to failure of a single feature, thereby improving continuity and success rate of vehicle identification.

Description

Technical Field

[0001] This invention relates to the field of vehicle analysis technology, and more specifically, to a vehicle identification method, apparatus, and edge device. Background Technology

[0002] In practical applications such as smart parking and vehicle access control, vehicle recognition systems need to accurately determine whether multiple images of the same vehicle collected at different times and under different monitoring devices belong to the same entity. This core recognition capability is the technical foundation for realizing business functions such as automatic billing, reverse vehicle location, and evasion prevention.

[0003] Currently, the mainstream solutions in the industry mainly fall into two technical paths: one is a detection method based on license plate recognition, but this method has an inherent blind spot for vehicles without license plates. The other is a comparison scheme based on vehicle brand characteristics, but due to insufficient stability of feature recognition, the false positive rate for vehicle matching remains high. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a vehicle identification method, apparatus and edge device that can improve the continuity and success rate of vehicle identification.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, the present invention provides a vehicle identification method, the method comprising: The first license plate number information of the departing vehicle is generated based on the departure image of the departing vehicle; If the first license plate number information indicates that the departing vehicle has an identifiable license plate number, determine whether there is a license plate number in the license plate index database that matches the first license plate number information; the license plate index database is used to store the license plate numbers of vehicles that have entered the venue. If a license plate number matching the first license plate number exists in the license plate index database, the vehicle corresponding to the matching license plate number will be associated with the departing vehicle as the same vehicle. If the first license plate number information indicates that the departing vehicle does not have an identifiable license plate number or that there is no license plate number in the license plate index database that matches the first license plate number information, then the second license plate number information, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicle are generated based on the departure image of the departing vehicle; the vehicle attribute information includes vehicle color and vehicle type. The vehicle fingerprint of the departing vehicle is generated based on the vehicle color, vehicle type, vehicle appearance normalization information, and vehicle bounding box information. The vehicle is matched with the vehicle that has entered the venue based on at least one of the following: the second license plate number information of the departing vehicle, the vehicle fingerprint, the vehicle attribute information, the vehicle appearance feature vector, the vehicle appearance normalization information, and the vehicle bounding box information. The matched vehicle is then associated with the departing vehicle as the same vehicle.

[0006] In an optional implementation, generating the first license plate number information of the departing vehicle based on the departure image of the departing vehicle includes: Using a local vehicle detection model, vehicle detection is performed based on the departure image to obtain a first detection box; If the first detection box indicates that the departing vehicle has been detected, the area corresponding to the first detection box in the departing image is determined as the target image, and the license plate number is identified based on the target image to obtain the first license plate number information; If the first detection box indicates that the departing vehicle has not been detected, the first license plate number in the first license plate number information is set to an invalid value.

[0007] In an optional implementation, generating the second license plate number information, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicle based on the departure image of the departing vehicle includes: Vehicle detection is performed based on the departure image using a multimodal large model in the cloud, resulting in a second detection box, vehicle attribute information, and vehicle appearance information; If the first detection box indicates that the departing vehicle has been detected, a vehicle appearance feature vector is generated based on the first detection box and the departure image, and the first detection box is determined as vehicle bounding box information. If the first detection box indicates that the departing vehicle has not been detected, a second license plate number and a vehicle appearance feature vector are generated based on the second detection box and the departing image, and the second detection box is determined as the vehicle bounding box information. The vehicle appearance information is normalized according to a preset normalization rule to obtain the normalized vehicle appearance information.

[0008] In an optional implementation, generating the vehicle fingerprint of the departing vehicle based on its vehicle color, vehicle type, vehicle appearance normalization information, and vehicle bounding box information includes: The aspect ratio of the departing vehicle is determined based on the vehicle boundary box information; The target interval label is determined based on the aspect ratio of the departing vehicle; the target interval label is used to distinguish the vehicle type of the departing vehicle. The identifier of the departing vehicle is obtained by hash calculation based on the vehicle color, vehicle type, normalized vehicle appearance information and target interval label of the departing vehicle. The vehicle fingerprint of the departing vehicle is generated based on the vehicle's identifier.

[0009] In an optional implementation, the step of matching the departing vehicle with an already entered vehicle based on at least one of the second license plate number information, vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, and associating the matched vehicle with the departing vehicle as the same vehicle, includes: If the second license plate number information indicates that the departing vehicle has an identifiable license plate number, determine whether there is a license plate number in the license plate index database that matches the second license plate number information; If a license plate number matching the second license plate number exists in the license plate index database, the vehicle corresponding to the matching license plate number will be associated with the departing vehicle as the same vehicle. If the second license plate number information indicates that the departing vehicle does not have an identifiable license plate number or that there is no license plate number in the license plate index database that matches the second license plate number information, then the departing vehicle is matched with the already entered vehicles based on at least one of the vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, and the matched vehicle is associated with the departing vehicle as the same vehicle.

[0010] In an optional implementation, the step of matching the departing vehicle with an already entered vehicle based on at least one of the vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, and associating the matched vehicle with the departing vehicle as the same vehicle, includes: If a vehicle fingerprint that matches the fingerprint of the departing vehicle exists in the fingerprint database, the vehicle corresponding to the matching fingerprint is associated with the departing vehicle as the same vehicle; the fingerprint database is used to store the vehicle fingerprints of vehicles that have entered the parking lot. If there is no matching vehicle fingerprint in the fingerprint database, an index vector of the departing vehicle is generated based on the vehicle appearance feature vector and the vehicle appearance normalization information. The index vector with the highest similarity to the index vector of the departing vehicle in the vector index library is determined as the target index vector; the vector index library is used to store the index vectors of vehicles that have entered the venue. If the similarity between the target index vector and the index vector of the departing vehicle exceeds a preset index similarity threshold, the vehicle corresponding to the target index vector and the departing vehicle will be associated as the same vehicle. If the similarity between the target index vector and the index vector of the departing vehicle does not exceed a preset index similarity threshold, the departing vehicle is matched with the already entered vehicle based on at least one of the vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, and the matched vehicle is associated with the departing vehicle as the same vehicle.

[0011] In an optional implementation, the step of matching the departing vehicle with the already entered vehicles based on at least one of the vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, and associating the matched vehicle with the departing vehicle as the same vehicle, includes: The vehicle appearance feature vector with the highest similarity to the departing vehicle in the vehicle information database is determined as the departing vehicle appearance feature vector; the vehicle information database is used to store vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information and vehicle bounding box information of vehicles that have entered the site. If the similarity between the appearance feature vector of the departing vehicle and the appearance feature vector of the departing vehicle exceeds a preset appearance similarity threshold, the vehicle corresponding to the appearance feature vector of the departing vehicle will be associated with the departing vehicle as the same vehicle. If the similarity between the appearance feature vector of the departing vehicle and the appearance feature vector of the departing vehicle does not exceed a preset appearance similarity threshold, the vehicle corresponding to the target index vector is determined as a candidate vehicle. The attribute similarity is determined based on the vehicle attribute information, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicles, as well as the vehicle attribute information, vehicle appearance normalization information, and vehicle bounding box information of the candidate vehicles. The overall similarity is determined based on the similarity between the target index vector and the index vector of the departing vehicle, as well as the attribute similarity. If the overall similarity exceeds the overall similarity threshold, the candidate vehicle and the departing vehicle will be associated as the same vehicle.

[0012] In an optional implementation, the method further includes: Determine whether the entering vehicle has a recognizable license plate number based on the entering vehicle's image; If an identifiable license plate number exists and the license plate index database does not record the license plate number of the entering vehicle, the license plate number of the entering vehicle shall be stored in the license plate index database. If no identifiable license plate number exists, the vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information of the entering vehicle are generated based on the entering image of the entering vehicle. The target vehicle is determined by the vehicle attribute information and vehicle appearance feature vector stored in the vehicle information database within a preset time period before the current time. Calculate the similarity between the vehicle appearance feature vector of each target vehicle and the vehicle appearance feature vector of the entering vehicle, and determine the target vehicle with the largest similarity and not less than the preset appearance similarity threshold as the best candidate vehicle. The attribute score is determined based on the vehicle attribute information of the best candidate vehicle and the vehicle attribute information of the entering vehicle. If the attribute score exceeds a preset score threshold, the vehicle fingerprint of the best candidate vehicle is determined as the vehicle fingerprint of the entering vehicle. If the similarity between the vehicle appearance feature vector of each target vehicle and the vehicle appearance feature vector of the entering vehicle is less than a preset appearance similarity threshold or the attribute score does not exceed a preset score threshold, the vehicle fingerprint of the entering vehicle is generated based on the vehicle color, vehicle type, vehicle appearance normalization information and vehicle bounding box information of the entering vehicle.

[0013] In a second aspect, the present invention provides a vehicle identification device, the device comprising: The processing module is used to generate the first license plate number information of the departing vehicle based on the departure image of the departing vehicle; The processing module is further configured to determine whether there is a license plate number matching the first license plate number information in the license plate index library if the first license plate number information indicates that the departing vehicle has an identifiable license plate number; the license plate index library is used to store the license plate numbers of vehicles that have entered the parking lot. The matching module is used to associate the vehicle corresponding to the matching license plate number with the departing vehicle if there is a license plate number in the license plate index library that matches the first license plate number information. The processing module is further configured to, if the first license plate number information indicates that the departing vehicle does not have an identifiable license plate number or that there is no license plate number in the license plate index database that matches the first license plate number information, generate the second license plate number information, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicle based on the departure image of the departing vehicle; the vehicle attribute information includes vehicle color and vehicle type; and generate the vehicle fingerprint of the departing vehicle based on the vehicle color, vehicle type, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicle. The matching module is also used to match the departing vehicle with the already entered vehicle based on at least one of the second license plate number information, vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information and vehicle bounding box information, and associate the matched vehicle with the departing vehicle as the same vehicle.

[0014] Thirdly, the present invention provides an edge device including a processor and a memory, the memory storing a computer program executable by the processor, the processor being able to execute the computer program to implement the vehicle identification method described in any of the foregoing embodiments.

[0015] Compared to existing technologies, the vehicle recognition method, apparatus, and edge device provided in this invention, when a departing vehicle has a reliable license plate number, uses the license plate number to match and associate the departing vehicle with the vehicles that have entered the parking area. When a departing vehicle does not have a reliable license plate number, a vehicle fingerprint of the departing vehicle is generated based on the vehicle color, vehicle type, vehicle appearance normalization information, and vehicle bounding box information. Based on at least one of the vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, a step-by-step matching is performed with the vehicles that have entered the parking area. In real-world scenarios such as no license plate or blurred license plate, vehicle recognition no longer relies on a single license plate number. Instead, it uses structured appearance features and multi-source attributes to collaboratively support vehicle recognition, avoiding matching interruptions due to the failure of a single feature. This makes the vehicle association process more adaptable and fault-tolerant, thereby improving the continuity and success rate of vehicle recognition.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic flowchart of a vehicle identification method provided by an embodiment of the present invention is shown.

[0019] Figure 2 This diagram illustrates another flow chart of the vehicle identification method provided in an embodiment of the present invention.

[0020] Figure 3 A block diagram of a vehicle identification device provided in an embodiment of the present invention is shown.

[0021] Figure 4 A block diagram of a vehicle identification device provided in an embodiment of the present invention is shown. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0023] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0024] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0025] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0026] Please refer to Figure 1 , Figure 1 A schematic flowchart of a vehicle identification method provided by an embodiment of the present invention is shown. The method includes the following steps: Step S10: Generate the first license plate number information of the departing vehicle based on the departure image of the departing vehicle.

[0027] In this embodiment of the invention, a departing vehicle is identified as a departing vehicle, and a corresponding departure image is captured for the departing license plate. When a departing vehicle leaves the site, it is necessary to determine whether the departing vehicle is the same vehicle as a previously registered vehicle that has entered the site, thereby completing subsequent operations such as billing and release.

[0028] In practical applications, the license plate number is first initially identified based on the position image of the departing vehicle to obtain the first license plate number information of the departing vehicle. Next, based on the first license plate number information, it is determined whether there are any other vehicles that have entered the venue and belong to the same group as the departing vehicle.

[0029] Step S20: If the first license plate number information indicates that the departing vehicle has an identifiable license plate number, determine whether there is a license plate number in the license plate index database that matches the first license plate number information; the license plate index database is used to store the license plate numbers of vehicles that have entered the venue.

[0030] In this embodiment of the invention, the license plate index database stores valid license plate numbers of vehicles that have successfully registered upon entry. A valid license plate number refers to a license plate number with a confidence level not lower than a preset license plate confidence threshold and a clear and complete license plate number. The first license plate number information includes the first license plate number and the first license plate confidence level.

[0031] If the first license plate number of the departing vehicle is not invalid and the confidence level of the first license plate is not lower than the preset license plate confidence threshold, then it is determined that the departing vehicle has an identifiable license plate number (i.e., a valid license plate number). Then, it is queried in the vehicle index database whether there is an already entered vehicle with the same license plate number as the first license plate number of the departing vehicle.

[0032] Step S30: If a license plate number matching the first license plate number exists in the license plate index database, the vehicle corresponding to the matching license plate number is associated with the departing vehicle as the same vehicle.

[0033] In this embodiment of the invention, if there is an already entered vehicle in the vehicle index database with the same license plate number as the first license plate number of the departing vehicle, the already entered vehicle corresponding to the first license plate number is associated with the departing vehicle as the same vehicle.

[0034] Step S40: If the first license plate number information indicates that the departing vehicle does not have an identifiable license plate number or there is no license plate number in the license plate index database that matches the first license plate number information, the second license plate number information, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicle are generated based on the departure image of the departing vehicle; the vehicle attribute information includes vehicle color and vehicle type.

[0035] In this embodiment of the invention, if the first license plate number is invalid or the confidence level of the first license plate is lower than a preset license plate confidence threshold, the departing vehicle is determined to be a vehicle without a license plate or with an unclear license plate number, i.e., the departing vehicle does not have an identifiable license plate number. If the departing vehicle does not have an identifiable license plate number or the first license plate number is not found in the license plate index database, then instead of relying on a single license plate number, multi-source information of the departing vehicle is generated based on the departing image, including but not limited to second license plate number information, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information.

[0036] Among them, the vehicle appearance feature vector is used to capture the visual features of the departing vehicle, the vehicle appearance normalization information is used to characterize the semantic features of the departing vehicle, and the vehicle attribute information also includes the vehicle brand and vehicle model.

[0037] Step S50: Generate the vehicle fingerprint of the departing vehicle based on the vehicle color, vehicle type, vehicle appearance normalization information, and vehicle bounding box information.

[0038] In this embodiment of the invention, when generating vehicle fingerprints, vehicle brand and vehicle model are not used. By avoiding unreliable attributes, the accuracy and uniqueness of vehicle fingerprint generation are improved.

[0039] Step S60: Match the departing vehicle with the entering vehicle based on at least one of the following: the second license plate number information, vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, and associate the matched vehicle with the departing vehicle as the same vehicle.

[0040] In this embodiment of the invention, the second license plate number information of the departing vehicle, vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information are used to flexibly match with the vehicles that have entered the parking lot. If a matching vehicle that has entered the parking lot is found, the matching vehicle is associated with the departing vehicle as the same vehicle. If no matching vehicle that has entered the parking lot is found, staff are notified to perform manual identification.

[0041] In summary, the vehicle recognition method provided by this invention, when a reliable license plate number exists for a departing vehicle, uses the license plate number to match and associate the departing vehicle with the vehicles that have already entered the parking area. When a reliable license plate number does not exist for a departing vehicle, a vehicle fingerprint is generated based on the departing vehicle's color, vehicle type, normalized vehicle appearance information, and vehicle bounding box information. Then, based on at least one of the vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, normalized vehicle appearance information, and vehicle bounding box information, a step-by-step matching process is performed with the vehicles that have already entered the parking area. In real-world scenarios such as no license plate or an unclear license plate, the method no longer relies on a single license plate number. Instead, it uses structured appearance features and multi-source attributes to collaboratively support vehicle recognition, avoiding matching interruptions due to the failure of a single feature. This makes the vehicle association process more adaptable and fault-tolerant, thereby improving the continuity and success rate of vehicle recognition.

[0042] Optionally, the following is a possible implementation method for preliminary identification of the license plate number of departing vehicles. Figure 1 The sub-steps of step S10 may include: Step S100: Use the local vehicle detection model to detect vehicles based on the departure image to obtain the first detection box.

[0043] In this embodiment of the invention, the local vehicle detection model is a lightweight object detection model deployed on an edge device, including but not limited to the You Only Look Once (YOLO) model, such as the YOLOv8 model.

[0044] The departing vehicle is input into the local vehicle detection model for vehicle detection. One or more detection boxes are output, along with the confidence level of each detection box. To suppress false detections in the background and duplicate detections of the same vehicle, the detection box with the largest retained area is normalized to obtain the first detection box.

[0045] Step S110: If the first detection box indicates that a departing vehicle has been detected, the area corresponding to the first detection box in the departing image is determined as the target image, and the license plate number is identified based on the target image to obtain the first license plate number information.

[0046] In this embodiment of the invention, if the first detection box contains a complete departing vehicle, it is determined that the first detection box represents a detected departing vehicle. The region corresponding to the first detection box is cropped from the departing image to obtain the target image. The license plate recognition model is then used to identify the license plate number based on the target image to obtain the first license plate number information.

[0047] The first license plate information includes the first license plate number and its confidence level. License plate recognition models include, but are not limited to, the Hyper Lightweight Plate Recognition 3 (HyperLPR3) model. HyperLPR3 is a lightweight optical character recognition (OCR) model based on YOLOv8-cls, capable of high-precision license plate recognition.

[0048] If the departing vehicle is without a license plate, the first license plate number output by the license plate recognition model is invalid. If the departing vehicle has a license plate, the first license plate number output by the license plate recognition model is valid, but whether the first license plate number is reliable depends on the confidence level of the first license plate.

[0049] Step S120: If the first detection box indicates that no departing vehicle has been detected, the first license plate number in the first license plate number information is set to an invalid value.

[0050] In this embodiment of the invention, if the first detection frame does not contain a complete departing vehicle, it is determined that the first detection frame indicates that no departing vehicle has been detected, and the first license plate number in the first license plate number information is set to an invalid value.

[0051] Alternatively, the following is a possible implementation method for refining the vehicle information of departing vehicles. Figure 1 The sub-steps of step S40 may include: Step S400: Utilize the multimodal large model in the cloud to perform vehicle detection based on the off-field image, and obtain the second detection box, vehicle attribute information, and vehicle appearance information.

[0052] In this embodiment of the invention, when the local vehicle detection model detects the license plate number of a departing vehicle, it directly uses the license plate number to perform vehicle recognition. When the local vehicle detection model cannot detect the license plate number of a departing vehicle, it calls the multimodal large model deployed on the cloud server as needed, instead of calling the entire model frame by frame. The average inference latency is reduced from about 2 seconds for pure remote calls to about 300 milliseconds, significantly reducing cloud transmission bandwidth and reducing the risk of data leakage, thus improving data security.

[0053] The departure image is uploaded to a remote location, and a multimodal large model is used to detect the vehicle color, vehicle type, vehicle boundary, and vehicle appearance in the departure image, resulting in a second detection box, vehicle attribute information, vehicle appearance information, and corresponding confidence scores.

[0054] The second detection box is the normalized vehicle bounding box. Vehicle attribute information includes vehicle color and vehicle type, which includes, but is not limited to, sedans, sport utility vehicles (SUVs), and multi-purpose vehicles (MPVs). Vehicle exterior information includes, but is not limited to, vehicle brand, headlights, grille, and body shape.

[0055] It should be noted that if the second detection frame does not detect a complete departing vehicle, the vehicle recognition process will exit, a new departing image will be acquired, and the vehicle recognition process will be restarted based on the new departing image.

[0056] Step S410: If the first detection box indicates that a departing vehicle has been detected, a vehicle appearance feature vector is generated based on the first detection box and the departing image, and the first detection box is determined as the vehicle bounding box information.

[0057] In this embodiment of the invention, if the position of the departing vehicle in the departure image can be determined by the first detection box, the region corresponding to the first detection box is cropped from the departure image to obtain the target image. The appearance features of the departing vehicle in the target image are extracted using an appearance feature extraction model to obtain the vehicle appearance feature vector. The appearance feature extraction model includes, but is not limited to, the DINOv2 model and a Contrastive Language-Image Pre-training (CLIP) image encoder.

[0058] Step S420: If the first detection box indicates that no departing vehicle has been detected, generate the second license plate number information and the vehicle appearance feature vector based on the second detection box and the departing image, and determine the second detection box as the vehicle bounding box information.

[0059] In this embodiment of the invention, if the position of the departing vehicle in the departure image cannot be determined by the first detection box, the area corresponding to the second detection box is determined in the departure image, and the area corresponding to the second detection box is expanded according to a preset expansion rule (e.g., expansion by 10%), and the expanded area is cropped out of the departure image to obtain the expanded image.

[0060] A license plate recognition model is used to identify license plate numbers based on the extended image, obtaining second license plate information. This second license plate information includes the second license plate number and its confidence level. An appearance feature extraction model is then used to extract the appearance features of departing vehicles from the extended image, resulting in vehicle appearance feature vectors.

[0061] Step S430: Normalize the vehicle appearance information according to the preset normalization rules to obtain normalized vehicle appearance information.

[0062] In this embodiment of the invention, it is assumed that the vehicle's appearance information is "a brand A logo in the center, bright white light on the left, yellow headlights on the right, horizontal grille, and a square body." The preset normalization rules include, but are not limited to, rules for masking vehicle brands, standardization rules for light descriptions, standardization rules for grille descriptions, standardization rules for body shape, and normalization and recombination rules.

[0063] According to the rules for blocking vehicle brands, keywords for blocking vehicle brands can be used. By utilizing a keyword database covering both domestic and foreign car brands, the preset vehicle brands in the keyword database can be removed from the vehicle's exterior information. The updated vehicle exterior information will be "a logo in the center, bright white light on the left, yellow light on the right, and a horizontal grille".

[0064] According to the standardization rules for light description, regular mapping is performed on the light description to unify different expressions such as "left side bright white light / left light white / white light left side" into "left light white", different expressions such as "right side bright yellow light / right light yellow" into "right light yellow", different expressions such as "round halogen lamp / round headlight" into "round headlight", and different expressions such as "rectangular headlight / square headlight / strip headlight" into "rectangular headlight".

[0065] According to the standardization rules for grid description, the grid description is standardized, and different expressions such as "horizontal strip grid / horizontal stripe grid / horizontal stripe grid" are uniformly converted into "horizontal stripe grid", and different expressions such as "mesh grid / honeycomb grid / honeycomb grid" are uniformly converted into "mesh grid".

[0066] Standardize vehicle body shape rules to unify different expressions such as "square body / flat roof / relatively square" into "square body", and unify different expressions such as "streamlined / rounded body" into "streamlined body".

[0067] The standardized vehicle appearance information is "a logo in the center, a white left headlight, a yellow right headlight, and a horizontal grille". Based on preset structural feature keywords and their order (e.g., body | grille | lights | bumper | roof rack | sliding door), the standardized vehicle appearance information is normalized and recombined, resulting in "a square body | a horizontal grille | a white left headlight | a yellow right headlight", ensuring that the normalized vehicle appearance information generated from different frame images for the same vehicle is completely consistent.

[0068] For example, suppose the vehicle's exterior information in the first frame image is "Brand A logo in the center, white left headlight, yellow right headlight, horizontal grille, and boxy body," and the vehicle's exterior information in the second frame image is "Brand B logo in the center, white left headlight, yellow right headlight, horizontal grille, and boxy body." The normalized vehicle exterior information in the first frame image is "boxy body | horizontal grille | white left headlight | yellow right headlight," and the normalized vehicle exterior information in the second frame image is "boxy body | horizontal grille | white left headlight | yellow right headlight." This can eliminate the interference caused by inconsistent brand recognition in determining vehicle identity.

[0069] Alternatively, the following is one possible implementation for generating vehicle fingerprints for unlicensed vehicles. Figure 1 The sub-steps of step S50 may include: Step S500: Determine the aspect ratio of the departing vehicle based on the vehicle boundary box information.

[0070] In this embodiment of the invention, the aspect ratio is calculated based on the coordinates [x1, y1, x2, y2] in the vehicle bounding box information, i.e., (x2-x1) / (y2-y1). The aspect ratio of a vehicle is relatively stable; the aspect ratio fluctuation of the same vehicle between different frames is usually less than ±0.02. However, the aspect ratio difference between different vehicle types such as MPVs, sedans, and SUVs is generally greater than 0.15. Therefore, a high aspect ratio has high cross-frame stability and the ability to distinguish vehicle types.

[0071] Step S510: Determine the target section label based on the aspect ratio of the departing vehicle; the target section label is used to distinguish the vehicle type of the departing vehicle.

[0072] In this embodiment of the invention, the aspect ratio is quantized according to a preset interval division step size (e.g., 0.25) to ensure that vehicles of the same type can still stably fall into the same interval label under real-world interference such as changes in lighting and shifts in shooting angle. For example, if the aspect ratio is 1.23, the interval label is ar_1.25. If the aspect ratio is 1.48, the interval label is ar_1.50.

[0073] The interval division step size not only tolerates lighting / angle deviations but is also sufficiently detailed to distinguish vehicle types. For example, minivans or MPVs typically fall into the 1.0~1.5 range, sedans typically fall into the 1.5~2.0 range, and SUVs typically fall into the 1.3~1.8 range, thus providing a basis for judging the vehicle profile of unlicensed vehicles independently of brand and model.

[0074] Step S520: Perform hash calculation based on the vehicle color, vehicle type, vehicle appearance normalization information and target interval label of the departing vehicle to obtain the identifier of the departing vehicle.

[0075] In this embodiment of the invention, for vehicles without license plates (i.e., vehicles without license plates) or whose license plate confidence is lower than the license plate confidence threshold, the vehicle color and vehicle type are standardized. The vehicle color is standardized and mapped according to a preset color alias mapping table (e.g., unifying "black / black / BLACK" as "black", "pearl white / white" as "white", and "silver" as "silver", supporting case conversion and automatic bracket filtering).

[0076] Vehicle types are standardized and mapped according to a preset vehicle type alias mapping table (e.g., unifying "off-road vehicle / SUV" as "SUV", unifying "commercial vehicle / van / mpv" as "MPV", supporting case conversion and automatic bracket filtering).

[0077] The system selects features with a confidence level greater than a preset threshold (e.g., 0.5) from standardized vehicle colors, standardized vehicle types, normalized vehicle appearance information, and target interval labels. These features are then hashed (e.g., using MD5 hashing) to obtain the identifiers of departing vehicles. The confidence level of the normalized vehicle appearance information is the same as the confidence level of the vehicle appearance information.

[0078] For example, if the confidence scores of the standardized vehicle color, standardized vehicle type, normalized vehicle appearance information, and target interval label are all greater than the preset filtering threshold, the string "c:{standardized vehicle color}|t:{standardized vehicle type}|marks: normalized vehicle appearance information|ar: target interval label" is generated. The hash value of the string is calculated using the MD5 hash algorithm, and the first eight bits of the hash value are taken as the identifier (e.g., ID) of the departing vehicle.

[0079] Step S530: Generate the vehicle fingerprint of the departing vehicle based on the vehicle's identifier.

[0080] In this embodiment of the invention, the vehicle fingerprint is obtained by combining the identifier of the departing vehicle with the date code corresponding to the current time. The date code is an eight-digit number (YYYYMMDD) corresponding to the current day, ultimately generating a vehicle fingerprint in a format such as "fd_{eight-bit hash}_{eight-bit date code}". This avoids vehicle fingerprint shifts caused by inconsistencies in brand recognition between frames, significantly outperforming traditional schemes that rely on brand and model numbers to generate vehicle fingerprints.

[0081] For vehicles with license plates, if the confidence level of the license plate number is not lower than the license plate confidence threshold, an identifier for the departing vehicle can be generated based on the license plate number. For example, the string "plate: license plate number" can be formed based on the license plate number, then a hash calculation can be performed on the string, and the first eight digits of the hash value can be used as the identifier for the departing vehicle. This is then combined with an eight-digit date code to generate a vehicle fingerprint.

[0082] Alternatively, one possible implementation method for how to associate vehicles is provided below. Figure 1 The sub-steps of step S60 may include: Step S600: If the second license plate number information indicates that the departing vehicle has an identifiable license plate number, determine whether there is a license plate number in the license plate index database that matches the second license plate number information.

[0083] Step S610: If a license plate number matching the second license plate number exists in the license plate index database, associate the vehicle corresponding to the matching license plate number with the departing vehicle as the same vehicle.

[0084] In this embodiment of the invention, if the second license plate number is valid, clearly identifiable, and complete, the license plate index in the SQLite database is directly invoked to precisely query whether the second license plate number exists. This query is highly efficient, taking less than 1 millisecond, and has a confidence level of 1. If the second license plate number is found in the license plate index, the corresponding entered and exited vehicles are associated as the same vehicle.

[0085] Step S620: If the second license plate number information indicates that the departing vehicle does not have an identifiable license plate number or there is no license plate number in the license plate index database that matches the second license plate number information, match the departing vehicle with the already entered vehicle based on at least one of the vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information and vehicle bounding box information, and associate the matched vehicle with the departing vehicle as the same vehicle.

[0086] In this embodiment of the invention, if a legal, clear, and complete second license plate number is not identified, or if a second license plate number is not found in the license plate index database, the conditions are relaxed, and a no-license-plate matching process is executed.

[0087] Optionally, regarding how to associate unlicensed vehicles, the following is a possible implementation method. Sub-steps of step S620 may include: Step S621: If a vehicle fingerprint that matches the vehicle fingerprint of the departing vehicle exists in the fingerprint database, the vehicle corresponding to the matching vehicle fingerprint is associated with the departing vehicle as the same vehicle; the fingerprint database is used to store the vehicle fingerprints of vehicles that have entered the site.

[0088] In this embodiment of the invention, when a match cannot be made using the license plate number (i.e., the first stage of matching), a vehicle fingerprint matching process (i.e., the second stage of matching) is initiated. Based on the fingerprint of the departing vehicle, the fingerprint database is searched for the fingerprints of all entering vehicles. If a fingerprint is found that completely matches the identifier (first eight bits of the hash value) of the departing vehicle, the match is considered successful, and the corresponding entering and departing vehicles are associated as the same vehicle. This matching is a precise comparison; once a match is found, the result is returned directly, and the third stage of matching is not initiated.

[0089] Step S622: If there is no vehicle fingerprint in the fingerprint database that matches the vehicle fingerprint of the departing vehicle, generate an index vector of the departing vehicle based on the vehicle appearance feature vector and the vehicle appearance normalization information.

[0090] In this embodiment of the invention, if vehicle fingerprint matching fails, a hybrid vector matching (i.e., third-stage matching) process is initiated. Vehicle appearance feature vectors (e.g., 384-dimensional vectors) and vehicle appearance normalized information (e.g., 512-dimensional vectors) are concatenated according to a preset ratio (e.g., 7:3) to form a new 896-dimensional vector. This new 896-dimensional vector is then subjected to L2 normalization to obtain the index vector of the departing vehicle. The weight of the vehicle appearance feature vector is greater than the weight of the vehicle appearance normalized information to avoid inaccurate hybrid vector matching due to semantic illusion.

[0091] Step S623: The index vector with the highest similarity to the index vector of the departing vehicle in the vector index library is determined as the target index vector; the vector index library is used to store the index vectors of the vehicles that have entered the venue.

[0092] In this embodiment of the invention, the vector index library can be constructed using the FAISS tool. Since the length of each index vector is 1, the similarity between two index vectors can be obtained by calculating the inner product of the two index vectors. When the FAISS tool is available, it quickly finds the index vectors of the top 10 vehicles that have entered the parking lot that are most similar to the index vector of the departing vehicle from the vector index library, and determines the index vector with the highest similarity as the target index vector.

[0093] It should be noted that when the FAISS tool is unavailable, a preset number (e.g., 500) of the index vectors recently stored in the vector index library are selected as the search vectors. The similarity between each search vector and the index vector of the departing vehicle is calculated, and the search vector with the highest similarity is selected as the target index vector.

[0094] Step S624: If the similarity between the target index vector and the index vector of the departing vehicle exceeds a preset index similarity threshold, the vehicle corresponding to the target index vector and the departing vehicle are associated as the same vehicle.

[0095] In this embodiment of the invention, once the similarity between the target index vector and the index vector of the departing vehicle exceeds a preset index similarity threshold (e.g., 0.85), a match is found, the fourth stage matching process is not initiated, and the vehicles that have entered and departed corresponding to the target index vector are directly associated as the same vehicle.

[0096] Step S625: If the similarity between the target index vector and the index vector of the departing vehicle does not exceed the preset index similarity threshold, match the departing vehicle with the already entered vehicle based on at least one of the vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information and vehicle bounding box information, and associate the matched vehicle with the departing vehicle as the same vehicle.

[0097] In this embodiment of the invention, if hybrid vector matching fails, the appearance vector matching (i.e., fourth-stage matching) process is initiated.

[0098] Alternatively, another possible implementation for associating unlicensed vehicles is provided below. The sub-steps of step S625 may include: Step S625-1: The vehicle appearance feature vector with the highest similarity to the vehicle appearance feature vector of the departing vehicle in the vehicle information database is determined as the departing vehicle appearance feature vector; the vehicle information database is used to store the vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information and vehicle bounding box information of the vehicles that have entered the site.

[0099] In this embodiment of the invention, the similarity between the vehicle appearance feature vector of each vehicle that has entered the vehicle information database and the vehicle appearance feature vector of each vehicle that has left the vehicle is calculated (e.g., cosine matching is performed), and the vehicle appearance feature vector with the highest similarity is determined as the appearance feature vector of the vehicle that has left the vehicle.

[0100] Step S625-2: If the similarity between the appearance feature vector of the departing vehicle and the appearance feature vector of the departing vehicle exceeds a preset appearance similarity threshold, the vehicle corresponding to the appearance feature vector of the departing vehicle is associated with the departing vehicle as the same vehicle.

[0101] In this embodiment of the invention, once the similarity between the appearance feature vector of the departing vehicle and the appearance feature vector of the departing vehicle exceeds a preset appearance similarity threshold (e.g., 0.8), a match is found, the fifth stage matching process is not initiated, and the vehicles that have entered the venue and the departing vehicle corresponding to the appearance feature vector of the departing vehicle are directly associated as the same vehicle.

[0102] Based on testing and verification, the appearance feature vectors of departing vehicles have strong anti-interference capabilities. Even if the multimodal large model identifies the same vehicle as a different brand, the cosine similarity of the two generated vehicle appearance feature vectors is usually greater than 0.85.

[0103] Step S625-3: If the similarity between the appearance feature vector of the departing vehicle and the appearance feature vector of the departing vehicle does not exceed the preset appearance similarity threshold, the vehicle corresponding to the target index vector is determined as a candidate vehicle.

[0104] In this embodiment of the invention, if the similarity between the appearance feature vector of a departing vehicle and the appearance feature vector of another departing vehicle does not exceed a preset appearance similarity threshold, then appearance vector matching fails. If appearance vector matching fails, then the attribute and vector hybrid matching (i.e., fifth-stage matching) process is initiated, and the vehicles that have entered the venue corresponding to the target index vector are identified as candidate vehicles.

[0105] Step S625-4: Determine the attribute similarity based on the vehicle attribute information, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicle, as well as the vehicle attribute information, vehicle appearance normalization information, and vehicle bounding box information of the candidate vehicle.

[0106] In this embodiment of the invention, for each attribute in the vehicle attribute information, the similarity between the attribute of the departing vehicle and the attribute corresponding to the candidate vehicle is calculated, and the similarity of each attribute in the vehicle attribute information is weighted and summed to obtain a first similarity. Taking vehicle color as an example, if the colors are the same, the similarity is set to 1; if the colors are similar (e.g., white and beige), the similarity is set to 0.3; and if the colors are not similar and are not the same, the similarity is set to 0.

[0107] Calculate the second similarity between the normalized appearance information of the departing vehicle and the normalized appearance information of the candidate vehicle. If the text of the two normalized appearance information is completely identical, the second similarity is set to 1.0. If the text of the two normalized appearance information is inconsistent, extract Bigram features from the normalized appearance information, calculate the Jaccard coefficient and overlap coefficient based on the Bigram features, and take the maximum value of the Jaccard coefficient and overlap coefficient as the second similarity. The Jaccard coefficient is the ratio of the intersection of the Bigram features of the candidate vehicle and the departing vehicle to the union of the Bigram features. The overlap coefficient is the ratio of the intersection of the Bigram features of the candidate vehicle and the departing vehicle to the minimum value of the two Bigram feature values.

[0108] The geometric features of departing vehicles are determined based on their vehicle bounding box information. These features include aspect ratio, center coordinates, and area. The aspect ratio, center coordinates, and area of ​​candidate vehicles are determined based on their vehicle bounding box information. The area refers to the area of ​​the vehicle's bounding box.

[0109] It should be understood that the geometric features are generated directly from the vehicle bounding box information, and highly stable discriminative features can be obtained without training and inference, achieving zero additional training cost.

[0110] The third similarity is obtained by weighting and summing the absolute value of the difference between the aspect ratio of the departing vehicle and the aspect ratio of the candidate vehicle (e.g., weighted at 0.5), the Euclidean distance between the center coordinates of the departing vehicle and the center coordinates of the candidate vehicle (e.g., weighted at 0.3), and the absolute value of the difference between the area of ​​the departing vehicle and the area of ​​the candidate vehicle (e.g., weighted at 0.2).

[0111] For example, if the weighted sum is no greater than 0.05, the third similarity is set to 1.0 (full score). If the weighted sum is greater than 0.05 but no greater than 0.1, the third similarity is set to 0.7. If the weighted sum is greater than 0.1 but no greater than 0.15, the third similarity is set to 0.3. If the weighted sum is greater than 0.15, the third similarity is set to 0 (elimination).

[0112] As one possible implementation, the weights used for vehicles with and without license plates are shown in Table 1. If the departing vehicle has a license plate, the first similarity is calculated using the license plate weight, and then the attribute similarity is calculated using the first, second, and third similarities and the license plate weight. Similarly, if the departing vehicle does not have a license plate, the no-license-plate weight is used when calculating the attribute similarity; adaptive weighting improves robustness.

[0113] Table 1

[0114] Step S625-5: Determine the comprehensive similarity based on the similarity between the target index vector and the index vector of the departing vehicle, as well as the attribute similarity.

[0115] Step S625-6: If the overall similarity exceeds the overall similarity threshold, the candidate vehicle and the departing vehicle are associated as the same vehicle.

[0116] In this embodiment of the invention, a weighted sum of the similarity between the target index vector and the index vector of the departing vehicle, as well as the attribute similarity, is performed to obtain a comprehensive similarity score. The comprehensive similarity thresholds include a license plate threshold (e.g., 0.7) and an unlicensed vehicle threshold (e.g., 0.55). If the departing vehicle has a license plate, and the comprehensive similarity exceeds the license plate threshold, the candidate vehicle is associated with the departing vehicle as the same vehicle. If the departing vehicle does not have a license plate, and the comprehensive similarity exceeds the unlicensed vehicle threshold, the candidate vehicle is associated with the departing vehicle as the same vehicle. If the comprehensive similarity does not exceed the comprehensive similarity threshold, a new departure image of the departing vehicle is acquired, and vehicle identification is performed based on the new departure image.

[0117] As can be seen, the embodiments of the present invention significantly improve the accuracy and robustness of associating vehicles without license plates through a five-stage cascaded matching process. For vehicles with identifiable license plates, precise matching is achieved using the license plate number, resulting in millisecond-level high-confidence matching. For vehicles without license plates or with unidentifiable license plates, a structurally stable vehicle fingerprint is first generated, ensuring approximately 95% consistency in identification among vehicles of the same vehicle across multiple frames. Then, matching conditions are gradually relaxed based on hybrid vectors and pure appearance vectors, effectively avoiding matching failures caused by inconsistent brand identification. Finally, matching is performed using attribute and appearance vectors, providing a fallback decision. The overall matching accuracy is improved from approximately 60% to approximately 90%.

[0118] Alternatively, another possible implementation for identifying entering vehicles is provided below. Please refer to... Figure 2 The vehicle identification method also includes the following steps: Step S700: Determine whether the vehicle entering the venue has a recognizable license plate number based on the entry image of the vehicle entering the venue.

[0119] Step S710: If a recognizable license plate number exists and the license plate index database does not record the license plate number of the entering vehicle, store the license plate number of the entering vehicle in the license plate index database.

[0120] Step S720: If there is no identifiable license plate number, generate vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information and vehicle bounding box information of the entering vehicle based on the entering vehicle's image.

[0121] Step S730: The vehicle corresponding to the vehicle attribute information and vehicle appearance feature vector stored in the vehicle information database within the preset time period before the current time is identified as the target vehicle.

[0122] In this embodiment of the invention, if the vehicle entering the venue has no license plate number or the license plate number cannot be identified, the vehicle attribute information and the vehicle appearance feature vector recorded in the vehicle information database within the most recent period (e.g., within 600 seconds from the current time) are used as the target vehicles to determine whether there is an entering vehicle among the target vehicles.

[0123] Step S740: Calculate the similarity between the vehicle appearance feature vector of each target vehicle and the vehicle appearance feature vector of the entering vehicle, and determine the target vehicle with the largest similarity and not less than the preset appearance similarity threshold as the best candidate vehicle.

[0124] Step S750: Determine the attribute score based on the vehicle attribute information of the best candidate vehicle and the vehicle attribute information of the entering vehicle.

[0125] Step S760: If the attribute score exceeds the preset score threshold, the vehicle fingerprint of the best candidate vehicle is determined as the vehicle fingerprint of the entering vehicle.

[0126] In this embodiment of the invention, if the record information of an entering vehicle is found in the vehicle information database, cross-frame fingerprint merging is performed. Additionally, information in the vehicle information database can be periodically batch-aggregated, merging fingerprints of the same vehicle in the fingerprint database. Through triple protection of appearance similarity, time window, and attribute verification, cross-frame fingerprint drift is automatically corrected.

[0127] Step S770: If the similarity between the vehicle appearance feature vector of each target vehicle and the vehicle appearance feature vector of the entering vehicle is less than a preset appearance similarity threshold or the attribute score does not exceed a preset score threshold, generate the vehicle fingerprint of the entering vehicle based on the vehicle color, vehicle type, vehicle appearance normalization information and vehicle bounding box information of the entering vehicle.

[0128] In this embodiment of the invention, if no record information for an entering vehicle is found, a vehicle fingerprint of the entering vehicle is generated and saved in a fingerprint database. The vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information of the entering vehicle are also saved in a vehicle information database. An index vector is generated based on the vehicle appearance feature vector and vehicle appearance normalization information of the entering vehicle and saved in a vector index database.

[0129] Based on the same inventive concept, the basic principle and technical effects of the vehicle recognition device provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments.

[0130] Please refer to Figure 3 , Figure 3 This is a block diagram of a vehicle identification device 400 provided in an embodiment of the present invention. The vehicle identification device 400 includes a processing module 410 and a matching module 420.

[0131] Processing module 410 is used to generate the first license plate number information of the departing vehicle based on the departure image of the departing vehicle; The processing module 410 is also used to determine whether there is a license plate number in the license plate index library that matches the first license plate number information if the first license plate number information indicates that the departing vehicle has an identifiable license plate number; the license plate index library is used to store the license plate numbers of vehicles that have entered the venue. The matching module 420 is used to associate the vehicle corresponding to the matching license plate number with the departing vehicle if there is a license plate number in the license plate index library that matches the first license plate number information. The processing module 410 is further configured to, if the first license plate number information indicates that the departing vehicle does not have an identifiable license plate number or there is no license plate number in the license plate index database that matches the first license plate number information, generate the second license plate number information, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicle based on the departure image of the departing vehicle; the vehicle attribute information includes vehicle color and vehicle type; and generate the vehicle fingerprint of the departing vehicle based on the vehicle color, vehicle type, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicle. The matching module 420 is also used to match the departing vehicle with the already entered vehicle based on at least one of the second license plate number information, vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information and vehicle bounding box information, and associate the matched vehicle with the departing vehicle as the same vehicle.

[0132] In summary, the vehicle recognition device provided in this embodiment of the invention, when a departing vehicle has a reliable license plate number, uses the license plate number to match and associate the departing vehicle with the vehicles that have entered the parking area. When a departing vehicle does not have a reliable license plate number, a vehicle fingerprint of the departing vehicle is generated based on the vehicle color, vehicle type, vehicle appearance normalization information, and vehicle bounding box information. Based on at least one of the vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, a step-by-step matching is performed with the vehicles that have entered the parking area. In practical scenarios such as no license plate or an unclear license plate, vehicle recognition no longer relies on a single license plate number, but rather uses structured appearance features and multi-source attributes to collaboratively support vehicle recognition. This avoids matching interruptions due to the failure of a single feature, making the vehicle association process more adaptable and fault-tolerant, thereby improving the continuity and success rate of vehicle recognition.

[0133] Optionally, the processing module 410 is used to perform vehicle detection based on the departure image using a local vehicle detection model to obtain a first detection box; if the first detection box indicates that a departing vehicle has been detected, the region corresponding to the first detection box in the departure image is determined as the target image, and the license plate number is identified based on the target image to obtain the first license plate number information; if the first detection box indicates that no departing vehicle has been detected, the first license plate number in the first license plate number information is set to an invalid value.

[0134] Optionally, the processing module 410 is used to perform vehicle detection based on the departure image using a multimodal large model in the cloud, and obtain a second detection box, vehicle attribute information and vehicle appearance information; if the first detection box indicates that a departing vehicle has been detected, a vehicle appearance feature vector is generated based on the first detection box and the departure image, and the first detection box is determined as vehicle bounding box information; if the first detection box indicates that no departing vehicle has been detected, a second license plate number and a vehicle appearance feature vector are generated based on the second detection box and the departure image, and the second detection box is determined as vehicle bounding box information; the vehicle appearance information is normalized according to a preset normalization rule to obtain normalized vehicle appearance information.

[0135] Optionally, the processing module 410 is used to determine the aspect ratio of the departing vehicle based on the vehicle bounding box information; determine the target interval label based on the aspect ratio of the departing vehicle; the target interval label is used to distinguish the vehicle type of the departing vehicle; perform hash calculation based on the vehicle color, vehicle type, vehicle appearance normalization information and target interval label of the departing vehicle to obtain the identifier of the departing vehicle; and generate the vehicle fingerprint of the departing vehicle based on the identifier of the departing vehicle.

[0136] Optionally, the matching module 420 is used to determine whether there is a license plate number in the license plate index library that matches the second license plate number if the second license plate number information indicates that the departing vehicle has an identifiable license plate number; if there is a license plate number in the license plate index library that matches the second license plate number information, the vehicle corresponding to the matched license plate number is associated with the departing vehicle as the same vehicle; if the second license plate number information indicates that the departing vehicle does not have an identifiable license plate number or there is no license plate number in the license plate index library that matches the second license plate number information, the departing vehicle is matched with the already entered vehicles based on at least one of the vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, and the matched vehicle is associated with the departing vehicle as the same vehicle.

[0137] Optionally, the matching module 420 is configured to: if a vehicle fingerprint matching the fingerprint of a departing vehicle exists in the fingerprint database, associate the vehicle corresponding to the matched fingerprint with the departing vehicle as the same vehicle; the fingerprint database stores the vehicle fingerprints of vehicles that have entered the parking lot; if no vehicle fingerprint matching the fingerprint of a departing vehicle exists in the fingerprint database, generate an index vector for the departing vehicle based on the vehicle appearance feature vector and vehicle appearance normalization information; determine the index vector with the highest similarity to the index vector of the departing vehicle in the vector index database as the target index vector; the vector index database stores the index vectors of vehicles that have entered the parking lot; if the similarity between the target index vector and the index vector of the departing vehicle exceeds a preset index similarity threshold, associate the vehicle corresponding to the target index vector with the departing vehicle as the same vehicle; if the similarity between the target index vector and the index vector of the departing vehicle does not exceed the preset index similarity threshold, match the departing vehicle with the vehicles that have entered the parking lot based on at least one of the vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, and associate the matched vehicle with the departing vehicle as the same vehicle.

[0138] Optionally, the matching module 420 is used to determine the vehicle appearance feature vector with the highest similarity to the vehicle appearance feature vector of the departing vehicle in the vehicle information database as the departing vehicle appearance feature vector; the vehicle information database is used to store the vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information of the vehicles that have entered the parking lot; if the similarity between the departing vehicle appearance feature vector and the vehicle appearance feature vector of the departing vehicle exceeds a preset appearance similarity threshold, the vehicle corresponding to the departing vehicle appearance feature vector is associated with the departing vehicle as the same vehicle; if the similarity between the departing vehicle appearance feature vector and the vehicle appearance feature vector of the departing vehicle does not exceed the preset appearance similarity threshold, the vehicle corresponding to the target index vector is determined as a candidate vehicle; the attribute similarity is determined based on the vehicle attribute information, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicle, as well as the vehicle attribute information, vehicle appearance normalization information, and vehicle bounding box information of the candidate vehicle; the comprehensive similarity is determined based on the similarity between the target index vector and the index vector of the departing vehicle, and the attribute similarity; if the comprehensive similarity exceeds the comprehensive similarity threshold, the candidate vehicle is associated with the departing vehicle as the same vehicle.

[0139] Optionally, the processing module 410 is further configured to determine whether an entering vehicle has a recognizable license plate number based on the entering image of the entering vehicle; if a recognizable license plate number exists and the license plate number of the entering vehicle is not recorded in the license plate index database, the license plate number of the entering vehicle is stored in the license plate index database; if no recognizable license plate number exists, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information of the entering vehicle are generated based on the entering image of the entering vehicle; the vehicles corresponding to the vehicle attribute information and vehicle appearance feature vectors stored in the vehicle information database within a preset time period before the current time are identified as target vehicles; and the vehicle appearance feature vectors of each target vehicle are calculated and compared with those of the entering vehicle. The similarity of the appearance feature vectors of the vehicles is calculated, and the target vehicle with the highest similarity and not less than a preset appearance similarity threshold is determined as the best candidate vehicle. The attribute score is determined based on the vehicle attribute information of the best candidate vehicle and the vehicle attribute information of the entering vehicle. If the attribute score exceeds the preset score threshold, the vehicle fingerprint of the best candidate vehicle is determined as the vehicle fingerprint of the entering vehicle. If the similarity between the appearance feature vectors of each target vehicle and the appearance feature vectors of the entering vehicle is less than the preset appearance similarity threshold or the attribute score does not exceed the preset score threshold, the vehicle fingerprint of the entering vehicle is generated based on the vehicle color, vehicle type, vehicle appearance normalization information and vehicle bounding box information of the entering vehicle.

[0140] Please refer to Figure 4 This is a block diagram illustrating an edge device 500 provided in an embodiment of the present invention. The edge device 500 includes a memory 510, a processor 520, and a communication module 530. The memory 510, processor 520, and communication module 530 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0141] The memory 510 is used to store programs or data. The memory 510 may be, but is not limited to, random access memory, read-only memory, programmable read-only memory, erasable read-only memory, electrically erasable read-only memory, etc.

[0142] The processor 520 is used to read / write data or programs stored in the memory 510 and perform corresponding functions. For example, when a computer program stored in the memory 510 is executed by the processor 520, the vehicle identification method disclosed in the above embodiments can be implemented.

[0143] The communication module 530 is used to establish a communication connection between the edge device 500 and other communication terminals via the network, and to send and receive data via the network.

[0144] It should be understood that, Figure 4The structure shown is only a schematic diagram of the edge device 500. The edge device 500 may also include components such as... Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown. Figure 4 The components shown can be implemented using hardware, software, or a combination thereof.

[0145] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0146] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0147] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0148] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A vehicle identification method, characterized in that, The method includes: The first license plate number information of the departing vehicle is generated based on the departure image of the departing vehicle; If the first license plate number information indicates that the departing vehicle has an identifiable license plate number, determine whether there is a license plate number in the license plate index database that matches the first license plate number information; the license plate index database is used to store the license plate numbers of vehicles that have entered the venue. If a license plate number matching the first license plate number exists in the license plate index database, the vehicle corresponding to the matching license plate number will be associated with the departing vehicle as the same vehicle. If the first license plate number information indicates that the departing vehicle does not have an identifiable license plate number or that there is no license plate number in the license plate index database that matches the first license plate number information, then the second license plate number information, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicle are generated based on the departure image of the departing vehicle; the vehicle attribute information includes vehicle color and vehicle type. The vehicle fingerprint of the departing vehicle is generated based on the vehicle color, vehicle type, vehicle appearance normalization information, and vehicle bounding box information. The vehicle is matched with the vehicle that has entered the venue based on at least one of the following: the second license plate number information of the departing vehicle, the vehicle fingerprint, the vehicle attribute information, the vehicle appearance feature vector, the vehicle appearance normalization information, and the vehicle bounding box information. The matched vehicle is then associated with the departing vehicle as the same vehicle.

2. The vehicle identification method according to claim 1, characterized in that, The step of generating the first license plate number information of the departing vehicle based on the departure image of the departing vehicle includes: Using a local vehicle detection model, vehicle detection is performed based on the departure image to obtain a first detection box; If the first detection box indicates that the departing vehicle has been detected, the area corresponding to the first detection box in the departing image is determined as the target image, and the license plate number is identified based on the target image to obtain the first license plate number information; If the first detection box indicates that the departing vehicle has not been detected, the first license plate number in the first license plate number information is set to an invalid value.

3. The vehicle identification method according to claim 2, characterized in that, The step of generating the second license plate number information, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicle based on the departure image of the departing vehicle includes: Vehicle detection is performed based on the departure image using a multimodal large model in the cloud, resulting in a second detection box, vehicle attribute information, and vehicle appearance information; If the first detection box indicates that the departing vehicle has been detected, a vehicle appearance feature vector is generated based on the first detection box and the departure image, and the first detection box is determined as vehicle bounding box information. If the first detection box indicates that the departing vehicle has not been detected, a second license plate number and a vehicle appearance feature vector are generated based on the second detection box and the departing image, and the second detection box is determined as the vehicle bounding box information. The vehicle appearance information is normalized according to a preset normalization rule to obtain the normalized vehicle appearance information.

4. The vehicle identification method according to claim 1, characterized in that, The step of generating the vehicle fingerprint of the departing vehicle based on its vehicle color, vehicle type, normalized appearance information, and vehicle bounding box information includes: The aspect ratio of the departing vehicle is determined based on the vehicle boundary box information; The target interval label is determined based on the aspect ratio of the departing vehicle; the target interval label is used to distinguish the vehicle type of the departing vehicle. The identifier of the departing vehicle is obtained by hash calculation based on the vehicle color, vehicle type, normalized vehicle appearance information and target interval label of the departing vehicle. The vehicle fingerprint of the departing vehicle is generated based on the vehicle's identifier.

5. The vehicle identification method according to claim 1, characterized in that, The step of matching the departing vehicle with an already entered vehicle based on at least one of the following: the second license plate number information, vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, and associating the matched vehicle with the departing vehicle as the same vehicle, includes: If the second license plate number information indicates that the departing vehicle has an identifiable license plate number, determine whether there is a license plate number in the license plate index database that matches the second license plate number information; If a license plate number matching the second license plate number exists in the license plate index database, the vehicle corresponding to the matching license plate number will be associated with the departing vehicle as the same vehicle. If the second license plate number information indicates that the departing vehicle does not have an identifiable license plate number or that there is no license plate number in the license plate index database that matches the second license plate number information, then the departing vehicle is matched with the already entered vehicles based on at least one of the vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, and the matched vehicle is associated with the departing vehicle as the same vehicle.

6. The vehicle identification method according to claim 5, characterized in that, The step of matching the departing vehicle with an already entered vehicle based on at least one of the vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, and associating the matched vehicle with the departing vehicle as the same vehicle, includes: If a vehicle fingerprint that matches the fingerprint of the departing vehicle exists in the fingerprint database, the vehicle corresponding to the matching fingerprint is associated with the departing vehicle as the same vehicle; the fingerprint database is used to store the vehicle fingerprints of vehicles that have entered the parking lot. If there is no matching vehicle fingerprint in the fingerprint database, an index vector of the departing vehicle is generated based on the vehicle appearance feature vector and the vehicle appearance normalization information. The index vector with the highest similarity to the index vector of the departing vehicle in the vector index library is determined as the target index vector; the vector index library is used to store the index vectors of vehicles that have entered the parking lot. If the similarity between the target index vector and the index vector of the departing vehicle exceeds a preset index similarity threshold, the vehicle corresponding to the target index vector and the departing vehicle will be associated as the same vehicle. If the similarity between the target index vector and the index vector of the departing vehicle does not exceed a preset index similarity threshold, the departing vehicle is matched with the already entered vehicle based on at least one of the vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, and the matched vehicle is associated with the departing vehicle as the same vehicle.

7. The vehicle identification method according to claim 6, characterized in that, The step of matching the departing vehicle with the already entered vehicles based on at least one of the vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information, and associating the matched vehicle with the departing vehicle as the same vehicle, includes: The vehicle appearance feature vector with the highest similarity to the departing vehicle in the vehicle information database is determined as the departing vehicle appearance feature vector; the vehicle information database is used to store vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information and vehicle bounding box information of vehicles that have entered the site. If the similarity between the appearance feature vector of the departing vehicle and the appearance feature vector of the departing vehicle exceeds a preset appearance similarity threshold, the vehicle corresponding to the appearance feature vector of the departing vehicle will be associated with the departing vehicle as the same vehicle. If the similarity between the appearance feature vector of the departing vehicle and the appearance feature vector of the departing vehicle does not exceed a preset appearance similarity threshold, the vehicle corresponding to the target index vector is determined as a candidate vehicle. The attribute similarity is determined based on the vehicle attribute information, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicle, as well as the vehicle attribute information, vehicle appearance normalization information, and vehicle bounding box information of the candidate vehicle. The overall similarity is determined based on the similarity between the target index vector and the index vector of the departing vehicle, as well as the attribute similarity. If the overall similarity exceeds the overall similarity threshold, the candidate vehicle and the departing vehicle will be associated as the same vehicle.

8. The vehicle identification method according to claim 1, characterized in that, The method further includes: Determine whether the vehicle has a recognizable license plate number based on the entry image of the vehicle. If an identifiable license plate number exists and the license plate index database does not record the license plate number of the entering vehicle, the license plate number of the entering vehicle will be stored in the license plate index database. If no identifiable license plate number exists, the vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information of the entering vehicle are generated based on the entering image of the entering vehicle. The target vehicle is determined by the vehicle attribute information and vehicle appearance feature vector stored in the vehicle information database within a preset time period before the current time. Calculate the similarity between the vehicle appearance feature vector of each target vehicle and the vehicle appearance feature vector of the entering vehicle, and determine the target vehicle with the largest similarity and not less than the preset appearance similarity threshold as the best candidate vehicle. The attribute score is determined based on the vehicle attribute information of the best candidate vehicle and the vehicle attribute information of the entering vehicle. If the similarity between the vehicle appearance feature vector of each target vehicle and the vehicle appearance feature vector of the entering vehicle is less than a preset appearance similarity threshold or the attribute score exceeds a preset score threshold, the vehicle fingerprint of the best candidate vehicle is determined as the vehicle fingerprint of the entering vehicle. If the attribute score does not exceed a preset score threshold, the vehicle fingerprint of the entering vehicle is generated based on the vehicle color, vehicle type, vehicle appearance normalization information, and vehicle bounding box information of the entering vehicle.

9. A vehicle identification device, characterized in that, The device includes: The processing module is used to generate the first license plate number information of the departing vehicle based on the departure image of the departing vehicle; The processing module is further configured to determine whether there is a license plate number matching the first license plate number information in the license plate index library if the first license plate number information indicates that the departing vehicle has an identifiable license plate number; the license plate index library is used to store the license plate numbers of vehicles that have entered the parking lot. The matching module is used to associate the vehicle corresponding to the matching license plate number with the departing vehicle if there is a license plate number in the license plate index library that matches the first license plate number information. The processing module is further configured to, if the first license plate number information indicates that the departing vehicle does not have an identifiable license plate number or that there is no license plate number in the license plate index database that matches the first license plate number information, generate the second license plate number information, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicle based on the departure image of the departing vehicle; the vehicle attribute information includes vehicle color and vehicle type; and generate the vehicle fingerprint of the departing vehicle based on the vehicle color, vehicle type, vehicle appearance normalization information, and vehicle bounding box information of the departing vehicle. The matching module is also used to match the departing vehicle with the already entered vehicle based on at least one of the second license plate number information, vehicle fingerprint, vehicle attribute information, vehicle appearance feature vector, vehicle appearance normalization information and vehicle bounding box information, and associate the matched vehicle with the departing vehicle as the same vehicle.

10. An edge device, characterized in that, It includes a processor and a memory, the memory storing a computer program that can be executed by the processor to implement the vehicle identification method according to any one of claims 1-8.

Images