Vehicle Image Analysis

Abstract

A method for determining a vehicle fingerprint includes receiving, from at least one sensor, a vehicle identifier and at least one vehicle appearance, each of the vehicle appearances including image data providing information about a vehicle scan, the appearance associated with a unique appearance time tag. The image data is then segmented into segments, each of the segments providing information about a vehicle component. Marker instances are then determined from the image scan, each marker instance being associated with a marker class and a marker feature. Data indicative of the vehicle fingerprint, including the vehicle identifier and its corresponding (i) vehicle appearance and associated appearance time tag, and (ii) the marker instance so determined, are then stored, thereby facilitating verification of the vehicle fingerprint in future vehicle scans.

Description

[Technical Field]

[0001] The subject matter of this disclosure relates generally to the field of vehicle image analysis, and more particularly to methods and systems for vehicle identity verification. [Background technology]

[0002] Vehicle identification is commonly performed in a variety of use cases, such as parking entry, toll roads, restricted access control, etc. In some cases, the identification process is performed using a vehicle license plate number, which is acquired either by a human or an automatic license plate recognition system. In some cases, the acquired license plate number is compared to a known set of license plates that contain additional information, such as a toll address for toll roads, entry and exit times for parking charges, or vehicles that can access a secure facility.

[0003] The main limitation of such methods is that license plates are easily altered, making them an unreliable method of vehicle identity verification.

[0004] The following invention describes a system for vehicle identity verification by "vehicle fingerprinting" based on independent sensing such as image or audio recording of the vehicle.

[0005] There is a need in the art for a fingerprint verification system for vehicles.

[0006] overview According to one aspect of the subject matter of the present disclosure, there is provided a method for determining a fingerprint of a vehicle, the method comprising: a) receiving a vehicle identifier; b) receiving at least one vehicle appearance from at least one sensor, each appearance including at least one image data indicative of at least a partial vehicle scan, the appearance being associated with a unique appearance time tag; c) segmenting the image data into segment data, each segment providing information about at least one respective subcomponent of the vehicle; d) determining a plurality of marker instances from the image scan or segment data, each marker instance being associated with a marker class and at least one marker feature; e) storing in storage data indicative of a vehicle fingerprint including a vehicle identifier and its corresponding at least (i) vehicle appearance and associated appearance time tag, and (ii) marker instances so determined; and storing the vehicle fingerprint, thereby facilitating verification of the vehicle fingerprint in future vehicle scans.

[0007] According to one embodiment of the subject matter of the present disclosure, there is provided a method further including: determining at least one marker feature instance for each one of the marker instances and for at least one of the marker classes of the marker instances; determining at least one marker feature instance that is position-dependent; and storing, for each marker instance, the so-determined marker feature instance including component-dependent marker features.

[0008] According to one embodiment of the subject matter of the present disclosure, there is further provided a method including repeating (b) through (e) for at least one more vehicle appearance having a corresponding unique time tag with respect to the vehicle, and for at least one other vehicle.

[0009] According to one embodiment of the presently disclosed subject matter, there is further provided a method wherein the marker class is selected from the group including scratches, dents, handwriting, colors, rust marks, cross threads, and printed text.

[0010] According to one embodiment of the presently disclosed subject matter, there is further provided a method wherein at least one of the sensors is an IR sensor and at least one of the vehicle scans is in IR wavelengths.

[0011] According to one embodiment of the subject matter of the present disclosure, there is further provided a method, wherein at least one of the sensors is an audio sensor, further comprising obtaining at least one audio scan of the vehicle and determining from the scan at least one audio marker class providing sound information of at least one module associated with the vehicle.

[0012] According to one embodiment of the subject matter of the present disclosure, there is further provided a method, wherein at least one of the sensors is an electromagnetic sensor, further comprising obtaining at least one electromagnetic scan of the vehicle and determining from the scan at least one electromagnetic marker class providing information of marks hidden beneath a non-metallic surface of the vehicle.

[0013] According to one aspect of the subject matter of the present disclosure, there is also provided a method for verifying a fingerprint of a vehicle, the method comprising, by a processor and associated memory storage: a) receiving a vehicle identifier; b) receiving at least one new vehicle appearance from at least one sensor, each new vehicle appearance including at least one image data indicative of at least a partial vehicle scan, the appearance being associated with a unique appearance time tag; c) segmenting the image data into segment data, each segment providing information about at least one respective subcomponent of the vehicle; d) determining a plurality of new marker instances from the image scan or segment data, each new marker instance being associated with a marker class and at least one marker feature; e) retrieving from the previously stored storage at least one vehicle appearance associated with at least one of the vehicle identifiers and its corresponding marker instances; f) comparing at least one new marker instance of the new appearance with corresponding marker instances of at least one previously stored appearance of the same vehicle identifier, and verifying the vehicle fingerprint if matching criteria are met.

[0014] According to one embodiment of the subject matter of the present disclosure, there is further provided a method in which (d) further includes determining, for each one of the new marker instances and for at least one of the new instance's marker classes, at least one new marker feature instance and determining at least one marker feature instance that is location-dependent; the extracting step includes extracting, for each marker instance, determined marker features including location-dependent marker features; and comparing includes comparing each new marker instance with at least one marker instance of at least one previously-stored appearance of the vehicle to determine a respective similarity score; and if at least one of the scored similarities exceeds a threshold for verifying the new marker instance, determining whether a matching criterion is met based at least on the verified marker instance and the corresponding stored marker instance.

[0015] According to one embodiment of the subject matter of the present disclosure, a method is further provided that further includes determining, for at least one verified marker instance of the newly acquired appearance, at least one candidate reference marker instance from a number of stored reference marker instances of the at least one vehicle appearance using narrowing criteria, and determining whether matching criteria are met based on at least the verified marker instance and the corresponding stored candidate reference marker instance.

[0016] According to one embodiment of the subject matter of this disclosure, a method is further provided, wherein the matching criterion is met if the number of verified marker instances among the corresponding stored marker instances exceeds a given threshold.

[0017] According to one embodiment of the subject matter of this disclosure, there is further provided a method in which at least some of the features are component dependent and the comparison is segment dependent, thereby reducing false alarms and computational complexity of the comparison.

[0018] According to one embodiment of the subject matter of the present disclosure, a method is further provided in which, if the matching criteria are met for the verified vehicle, new marker instances for the verified vehicle that did not meet the similarity score are stored along with their associated feature instances to improve future vehicle verification.

[0019] According to one embodiment of the presently disclosed subject matter, there is further provided a method wherein the marker class is selected from the group including scratches, dents, handwriting, colors, rust marks, cross threads, and printed text.

[0020] According to one embodiment of the presently disclosed subject matter, there is further provided a method wherein at least one of the sensors is an IR sensor and at least one of the vehicle scans is in IR wavelengths.

[0021] According to one embodiment of the subject matter of the present disclosure, there is further provided a method wherein at least one of the sensors is an audio sensor, further comprising obtaining at least one audio scan of the vehicle and determining from the scan at least one audio marker class providing sound information of at least one module associated with the vehicle, wherein the extracting and comparing also applies to the audio marker class.

[0022] According to one embodiment of the subject matter of the present disclosure, there is further provided a method, wherein at least one of the sensors is an electromagnetic sensor, further comprising obtaining at least one electromagnetic scan of the vehicle and determining from the scan at least one electromagnetic marker class providing information of marks hidden beneath a non-metallic surface of the vehicle, wherein the extracting and comparing also applies to the electromagnetic marker class.

[0023] According to one aspect of the presently disclosed subject matter, there is also provided a computerized system for determining a fingerprint of a vehicle, the system comprising: receiving a vehicle identifier; receiving at least one vehicle appearance from at least one sensor, each appearance including at least one image data indicative of at least a partial vehicle scan, the appearance being associated with a unique appearance time tag; Segmenting the image data into segment data, each segment providing information about at least one respective subcomponent of the vehicle; determining a plurality of marker instances from the image scan or segment data, each marker instance being associated with a marker class and at least one marker feature; storing in storage data indicative of a vehicle fingerprint including a vehicle identifier and at least its corresponding (i) vehicle appearance and associated appearance time tag, and (ii) marker instances so determined; and storing, thereby facilitating verification of the vehicle fingerprint upon future vehicle scans.

[0024] According to one embodiment of the presently disclosed subject matter, there is further provided a system for determining a fingerprint of a vehicle, further including any of the embodiments outlined above with reference to the method.

[0025] According to one aspect of the presently disclosed subject matter, there is also provided a computerized system for verifying a fingerprint of a vehicle, the system comprising: receiving a vehicle identifier; receiving at least one new vehicle appearance from at least one sensor, each new vehicle appearance including at least one image data indicative of at least a partial vehicle scan, the appearance being associated with a unique appearance time tag; Segmenting the image data into segment data, each segment providing information about at least one respective subcomponent of the vehicle; determining a plurality of new marker instances from the image scan or segment data, each new marker instance being associated with a marker class and at least one marker feature; Retrieving at least one vehicle appearance associated with at least one of the vehicle identifiers and its corresponding marker instances from a previously stored storage; and comparing at least one new marker instance of the new appearance with a corresponding marker instance of at least one previously stored appearance of the same vehicle identifier, and verifying the vehicle fingerprint if a matching criterion is met.

[0026] According to one embodiment of the presently disclosed subject matter, there is further provided a system for verifying a vehicle fingerprint, further including any of the embodiments outlined above with reference to the method.

[0027] According to one embodiment of the presently disclosed subject matter, there is further provided a non-transitory computer-readable medium containing instructions that, when executed by a computer, cause the computer to perform the method steps outlined above.

[0028] A non-transitory computer-readable medium containing instructions that, when executed by a computer, cause the computer to perform the method steps outlined above. [Brief explanation of the drawings]

[0029] In order to understand the invention and to appreciate how it may be carried out in practice, embodiments will now be described, by way of non-limiting example only, with reference to the accompanying drawings, in which: [Figure 1]1 illustrates generally a block diagram of a computerized system capable of uniquely identifying a vehicle, in accordance with certain embodiments of the presently disclosed subject matter. [Figure 2] 1 shows a generalized flowchart of a vehicle sign-up process. [Figure 3] 1 illustrates an example of an input image and corresponding component segments, according to certain embodiments of the subject matter of this disclosure. [Figure 4] 1 shows a generalized processing flow for a single marker. [Figure 5a] 5a shows an exemplary database structure for a vehicle identification verification process. Specifically, FIG. 5a is a generic database. [Figure 5b] 5b shows an exemplary database structure for the vehicle identification verification process. Specifically, FIG. 5b specifies what information is stored for each marker and its appearance. [Figure 6] 1 illustrates a generalized flowchart of vehicle identification verification in accordance with certain embodiments of the disclosed subject matter. [Figure 7] 1 shows a generalized flowchart for comparing new markers with existing markers, according to certain embodiments of the presently disclosed subject matter. [Figure 8a] 1 illustrates a generalized flowchart for computing vehicle markers of marker class "color," according to certain embodiments of the disclosed subject matter. [Figure 8b] 1 illustrates a generalized flowchart for calculating a vehicle marker of the marker class "cross screw," according to certain embodiments of the disclosed subject matter. [Figure 9] 10A-10C illustrate visual examples of additional marker classes, according to certain embodiments of the subject matter of this disclosure.

[0030] MODE FOR CARRYING OUT THE INVENTION In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the present invention. However, it will be understood by those skilled in the art that the subject matter of the present disclosure may be practiced without these specific details. In other embodiments, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the subject matter of the present disclosure. As will be apparent from the following description, unless otherwise indicated, the use of terms such as "receive," "segment," "determine," "calculate," and "fill" throughout the specification refers to computer operations and / or processes that manipulate and / or transform data into other data, where the data is represented as a physical quantity, e.g., an electronic quantity, and / or the data represents a physical object. The term "computer" should be interpreted expansively to encompass any type of hardware-based electronic device having data processing capabilities, including, by way of non-limiting examples, a computerized system for verifying a vehicle fingerprint, a computerized system for determining and recording a vehicle fingerprint, and the processing and memory circuitry (PMC) of these systems as disclosed herein.

[0031] Operations according to the teachings herein may be performed by a specially constructed computer for the desired purpose, or by a general-purpose computer specially configured for the desired purpose by a computer program stored on a non-transitory computer-readable storage medium.

[0032] As used herein, the terms "non-transitory memory," "non-transitory storage medium," and "non-transitory computer-readable storage medium" should be interpreted broadly to include any volatile or non-volatile computer memory suitable for the subject matter of this disclosure.

[0033] Embodiments of the presently disclosed subject matter are not described with reference to any particular programming language, although it will be appreciated that a variety of programming languages ​​can be used to implement the teachings of the presently disclosed subject matter as described herein.

[0034] As used herein, phrases such as "for example," "such as," "e.g.," and "for example," and variations thereof, describe non-limiting embodiments of the presently disclosed subject matter. References herein to "one instance," "some instances," "other instances," or variations thereof mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the presently disclosed subject matter. Thus, references to "one instance," "some instances," "other instances," or variations thereof do not necessarily refer to the same embodiment.

[0035] It will be understood that certain features of the presently disclosed subject matter, while described in the context of separate embodiments, can also be provided in combination or in a single embodiment. Conversely, various features of the presently disclosed subject matter, while described in the context of a single embodiment, can also be provided separately or in any suitable subcombination. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the methods and apparatus.

[0036] In embodiments of the presently disclosed subject matter, one or more of the steps shown in the figures may be performed in a different order and / or one or more groups of steps may be performed simultaneously, or vice versa.

[0037] The system 100 shown in FIG. 1 is a computer-based vehicle identification and verification system. The system 100 can be configured to acquire, via a hardware I / O interface 126, vehicle scans acquired by one or more sensors 121, 122 that capture information about at least a portion of the vehicle (vehicle images and / or vehicle audio recordings). The sensors may be, for example, image acquisition devices such as cameras, or sound acquisition devices such as microphones. Other non-limiting examples are IR sensors (which can acquire, for example, heat signatures of various parts of the vehicle or images under low-light conditions), or radar / lidar transceiver sensors that can transmit electromagnetic signals that strike the vehicle and are reflected back from it. Such reflected signals may be received by radar and may represent objects ("electromagnetic signatures") such as dents or cracks hidden beneath the surface of the vehicle.

[0038] It should be noted that the term vehicle as used herein should be interpreted broadly to encompass any type of motor vehicle, including, but not limited to, cars, buses, motorcycles, trucks, trains, and airplanes.

[0039] The imaging device 121 may be any type of image acquisition device or general-purpose device with image acquisition capabilities that can be used to capture vehicle images at a particular resolution and frequency, such as, for example, a digital camera with image and / or video recording capabilities.

[0040] As shown, system 100 may include a processing and memory circuit (PMC) 102 operatively connected to an I / O interface 128 and memory storage 123. PMC 102 is configured to provide all processing necessary for operating system 100, which will be described in further detail with reference to FIGS. 2, 4-8. PMC 102 includes a processor (not separately shown) and memory. Processor PMC 102 may be configured to execute several functional modules in accordance with computer-readable instructions embodied on computer-readable, non-transitory computer-readable memory included in the PMC. Such functional modules are hereinafter referred to as being included in the PMC.

[0041] In particular embodiments, the functional modules included in the PMC 102 may include a vehicle segmentation module 104, a marker detection and classification module 106, and a marker comparison module 108. The functional modules within the PMC are operatively connected to one another.

[0042] In accordance with the subject matter of this disclosure, the term marker class should be broadly interpreted to indicate a unique marker type associated with a marker instance. For example, X marker instances are determined from a given vehicle appearance (as described in more detail below), all of which have the same marker class "scratch." Intuitively, in this example, there are X scratch markers (instances). As another example, Z marker instances can be determined from a given appearance, of which X instances are classified as "scratch" (marker class="scratch") and Y instances are classified as "dent" (marker class="dent"), where X + Y = Z. Marker classes may be, for example, visual, audio, IP, electromagnetic, etc. The description herein illustrates numerous examples of marker classes (e.g., visible markers, scratches, dents, cracks, written text, etc., whether on the surface of the vehicle or hidden underneath). Each marker typically, but not necessarily, has a marker class feature that characterizes the marker according to its marker class. Some features (e.g., component coordinates) may be common to different marker classes. The term marker (or marker class feature) instance refers to a particular instance of a marker, e.g., a given scratch determined from a given appearance. The term marker class feature may refer to one or more features of a marker, and the term marker feature instance refers to a particular value of a feature. Different features may exist for different marker classes. Thus, for example, a given marker may be classified as belonging to the marker class "Handwriting" and may have the following features "Component" (information on which component on the vehicle it appears) and "Coordinate" (information on the coordinate on the component where the handwriting marker appears). A feature instance is a particular value of a feature (e.g., "Wheel" and coordinates x, y) of a given marker instance (i.e., a particular "Handwriting" instance), all of which are exemplified in more detail below.

[0043] The marker detection and classification module may be configured to detect and classify vehicle identity marker (VIM) instances (sometimes referred to as "markers" for short) visible on / in the vehicle throughout its lifetime (and extracted from vehicle scans). Other classes of markers may be obtained from sound or electromagnetic scans derived from respective audible and / or electromagnetic sensors (such as radar / lidar), which may be extracted from sound or electromagnetic scans, respectively. The former may be used, for example, to identify sounds originating from a given vehicle's mechanisms (such as the sound of a motor assembly), while the latter may function, for example, from an electromagnetic scan to reveal hidden damage such as hidden (i.e., below the surface) cracks or dents.

[0044] Electromagnetic scans can be obtained by transmitting LIDAR electromagnetic signals and receiving their reflections from the vehicle. At least some of these marker classes are uniquely caused by the use of the vehicle and are unlikely to appear on / in other vehicles. Examples of such VIM classes can be scratches, dents, handwriting, collars, cross-shaped screws, etc. VIM classes and their calculation methods are described in detail in FIGS. 8-9. Calculated VIM classes and their associated feature instances (as described in more detail below) can be stored in memory module 123, which is further described in FIG. 5. Some VIMs are location-dependent, where location refers to the exact location on the vehicle captured by the sensor. An example of a location-dependent VIM is a scratch. A vehicle may have many scratches with similar appearances. According to certain embodiments, other marker classes can be scratches, rust marks, screw orientations, handwriting, paint marks, serial numbers, stickers, dents, or any other mechanical deformations, and the location of electrical wires. These are merely non-limiting examples of marker classes.

[0045] The marker comparison module 108 may be configured to retrieve VIM appearances (marker instances) on different vehicle appearances from the memory storage 123, compare the VIM appearances, and provide a similarity score indicating whether the VIM instance is the same as one or more instances stored in the database, as further described with reference to FIG.

[0046] It should also be noted that the system shown in FIG. 1 can be implemented in a distributed computing environment, and the aforementioned functional modules shown in FIG. 1 can be distributed across several local and / or remote devices and linked through a communications network.

[0047] Those skilled in the art will readily understand that the teachings of the presently disclosed subject matter are not bound by the system shown in FIG. 1. Equivalent and / or modified functionality may be integrated or divided in other ways and may be implemented in any suitable combination of software, firmware, and hardware. The system of FIG. 1, or at least certain components thereof, may be a standalone network entity or may be fully or partially integrated with other network entities. Those skilled in the art will also readily understand that data repositories or storage units therein may be shared with other systems or provided by other systems, including third-party equipment.

[0048] Vehicle identity verification begins with vehicle sign-up, illustrated in Figure 2, which is the process of adding a new vehicle instance (appearance) to the system. According to certain embodiments, the sequence of operations described with reference to Figure 2 (and / or) certain sub-stages described with reference to other figures, illustrate non-limiting examples of determining a vehicle's fingerprint for various purposes, including verifying its identity.

[0049] Thus, first, a vehicle identifier (e.g., a string of characters) is obtained (200). The vehicle identifier can be any unique entry for identifying a vehicle. By way of example, the vehicle identifier can be a vehicle license plate (LP) or a vehicle identification number (VIN) (e.g., a unique code provided by the manufacturer). The identifier can be obtained automatically using a dedicated computer system or in any other manner, including manual entry by a human operator.

[0050] The next step is capturing 202 the vehicle exterior, which involves scanning the vehicle or portions thereof using sensors 120-122. Information captured in 202 can include vehicle images, vehicle sounds, etc., RF signature, WIFI MAC address, etc.

[0051] The information obtained by 202 is stored in 123 together with the vehicle identifier.

[0052] Once the vehicle appearance is obtained, the vehicle image can be provided as input to a vehicle component segmentation module 204 (PMC module 104), which can be within the same segmentation module as described, for example, in U.S. Pat. No. 1,065,0530, column 9, line 35 to column 11, line 10, with reference to FIGS. 1, 2, and 4, the contents of which are incorporated by reference and may be part of the system of the presently disclosed subject matter. The output of the component segmentation module can be a component map corresponding to the input vehicle image, as shown in FIG. 3. How combining component maps contributes to improving the accuracy and reducing the computation time of the system will be described in further detail below.

[0053] It should be noted that each segment may be composed of at least a subcomponent, e.g., a complete separate component, a partial component (e.g., a subcomponent), or multiple complete or partial components, depending on the particular application. A component may also be a separate physical component (e.g., exhaust, fuel tank, drive shaft, oil filter, etc.). As shown, by way of example, FIG. 3 shows a bottom view of a fuel tank 301, a partial bottom view of a chassis 302, etc., all depending on the particular application. For ease of explanation, the description herein assumes that, in a segmentation map, each segment represents a component. The invention is, of course, not bound by this example, and thus the invention applies mutatis mutandis to other options for segments (e.g., partial components or fewer components), etc.

[0054] Additionally, the captured vehicle image can be provided as input to a marker detection and classification module 206 (PMC module 106), which detects a given VIM and outputs the location of the detected VIM class (the terms VIM and marker are used interchangeably). As an example, a scratch-type marker instance (e.g., a defect) can be detected on a particular image pixel. Some indication of the pixel can be provided (bounding box coordinates, pixel list, object center, etc.). Furthermore, the marker detection can be combined with the output of 204 to provide not only the pixel location on the image, but also the mechanical component (if the segment map includes the component). The marker instance is detected, for example, by overlaying the image coordinates of the detected marker with the coordinates of the segment (as obtained in the segmentation map), or invasively if the image falls within the segment. Furthermore, the marker coordinates can be provided as global image coordinates or relative to the corresponding component as obtained using the output of 204. The identified data, such as the appropriate component, coordinates, etc. of the detected marker, are examples of feature instances of the marker. The latter may be stored for the following marker comparison calculation steps, all as explained in more detail below.

[0055] By way of example, the detection and / or classification may form part of a system according to the subject matter of the present disclosure and may be performed by a Deep Neural Network DNN (see also 401 in FIG. 4 ) as such, which is commonly known and is described, for example, in U.S. Pat. No. 1,065,053, column 10, line 9 to U.S. Pat. No. 1,065,053,011, line 10, the contents of which are incorporated herein by reference; the present invention is not bound to the use of ML in general, and not to the use of DNN in particular, and is provided by way of example only.

[0056] In some embodiments, the DNN can be trained using a training dataset of pre-annotated VIMs. The training images and the annotated VIMs are provided as inputs to the detection and classification DNN for training. The training process is designed to optimize the model so that it can correctly predict the VIM location and VIM class on the image. In some cases, different training datasets need to be provided to train the model so that it can detect marker instances on different vehicle types or different vehicle components at runtime. Intuitively, markers for vehicles can be thought of like human fingerprints: every person has a "fingerprint," but when calculating the exact fingerprint features, they are different for each individual. In the case of vehicles, for example, considering the marker class "scratch," most vehicles have scratches, but they are different for each vehicle (e.g., in their location, shape, size, etc.; the latter example may represent a marker feature for a marker of class "scratch"), and therefore each of those marker feature instances can uniquely identify the vehicle (possibly along with other marker instances of the same or different classes). Thus, for example, a "scratch detector," as described above, may be trained to identify scratches that may serve to uniquely identify a vehicle.

[0057] Returning to FIG. 4, the class-specific post-processing module (403) can function for background removal, axis-aligned histogram equalization, etc., all of which are known per se.

[0058] In some embodiments, following the detection and classification step, some of the markers proceed to dedicated processing pipelines predefined for the marker class obtained from the classification step. The class-specific pipelines contain instructions regarding which class-specific feature instances should be computed for a given marker class. For example, as shown in FIG. 4, pipeline 404(1) is configured to compute “angle” and “line reference” feature instances (applicable to a cross-screw marker class). Pipeline 404(2) is configured to compute encoded vector features (which may be used for later comparison between new marker feature instances and previously stored marker feature instances, all as described below), and pipeline 404(3) is configured to compute color histogram features (applicable to a color marker class). Some of the features are common to different pipelines, while others are uniquely usable for a particular pipeline. Note that this is one example implementation; by another example, features computed by the pipelines may be stored in additional database tables. Note also that the present invention is not constrained by utilizing pipeline computation to determine features, and thus other forms of feature computation may be utilized. The present invention is similarly not bound by any particular pipeline.

[0059] Below is an exemplary detailed description of some of the pipelines, each designated to process one or more unique marker classes.

[0060] Color Marker Class (Figure 8a) Given an input vehicle image 800 acquired by 202 and a component segmentation map 801 acquired by 204 (an example of such a map is given by 804), a representation of vehicle color is computed for the different vehicle components. As an example, color can be computed as histogram values ​​per channel (the histogram is characteristic of markers of the "color" marker class in this example) for the entire image or for only pixels of a particular segment such as "paint," as shown in 804. As another example, color can be computed as a representation vector (i.e., the marker class is a vector) obtained by a DNN trained to encode color.

[0061] Data providing information about color marker classes and their associated feature instances may be stored in memory system 123 and is further illustrated in Figure 5B below. Exemplary color class features for marker class colors may be associated components, histograms, etc., as shown with reference to Figure 5. The present invention is not bound by the specific color marker class features provided for illustrative purposes only.

[0062] Cross thread marker class figure 8b Cross threads (also known as "Phillips" threads) are useful markers because they can be used to distinguish (e.g., by the orientation of the threads) the angle of the line in 818 between two completely new vehicles that have just come off the production line (meaning that no other important markers exist on the vehicle yet).

[0063] The screw coordinates (an exemplary feature) may be obtained by the marker detection module 206. The input image and detected screw markers are shown at 816 and 817, respectively. After the screw is detected (206) by 811, in some embodiments, a slot on top of the screw is identified (812), and the direction of the slot is called the thread angle and shown as a dashed line at 818. Intuitively, different angles (possibly along with instances of other marker classes) are likely to characterize different cars. As an example, an edge detection algorithm around the center of the detected screw can be used to identify the slot, and a standard algorithm such as RANSAC can be used to estimate the slot line equation. In the latter example, the "angle" is called a marker feature (for the marker class cross-type screw).

[0064] A reference line (e.g., another feature) on the vehicle image can be selected (814) to provide a reference for the angle calculation. Because different vehicle appearances can occur at different angles, using a semantic component as the reference line allows for normalizing the thread angle relative to the overall vehicle orientation so that angles can be meaningfully compared between different appearances. By way of example, the reference line can be selected as an edge of a semantic component known to be fixed to the vehicle (e.g., chassis). Such an example of a reference line is shown at 818 as a dashed line. Data providing information about the cross-type screw marker class and its associated features (instances) may be stored in memory system 123 and is further illustrated in FIG. 5B below. Exemplary cross-type screw class features for the cross-type screw marker class (discussed above) may be the associated component, thread angle, reference line, screw location within the component, etc. The present invention is not bound by the specified cross-type screw marker class (e.g., "Phillips" screw) nor by the specified marker class features, which are provided for illustrative purposes only.

[0065] Figure 9 shows additional examples of marker classes that may appear on a vehicle as a result of use and are unique to each particular vehicle. This makes all defects, such as printed (or handwritten) text, cracks, dents, scratches, rust, etc. (all of which can be acquired by the marker detection module 206), good identifying markers. Their corresponding features are also calculated, such as the image coordinates where they are detected, the corresponding component where the marker resides (the former can be acquired from the component segmentation map), and the relative coordinates of the component. Other feature data may be extracted and stored in addition to or instead of those identified above, depending on the particular marker class and application.

[0066] FIG. 9 schematically illustrates non-limiting examples of VIM classes, such as a printed text marker class (901) on an oil tank component, a crack marker class (902) on a plastic cover component, and a dent marker class (903) on an exhaust shield component. In some embodiments, encoding vector features may be calculated for detected markers to enable future comparison with other marker instances (obtained from future scans). By way of example, such encoding vectors may be obtained by a deep neural network (DNN), as described in further detail with reference to FIG. 7 below. All obtained feature instances may be stored in a database (FIG. 5) for future use, all as described in more detail below.

[0067] It should be noted that the specified marker classes are provided by way of example only, and that other visual and non-visual (e.g., acoustic / electromagnetic) marker classes may be utilized in addition to, or in place of, one or more of the foregoing examples. Returning now to Figure 2, following completion of the computational pipeline for each marker class (to determine corresponding features), the information (i.e., instances) may be stored (209) in a database, as illustrated in Figure 5.

[0068] Referring to FIG. 5A, upon sign-up of a new vehicle, the database will contain the vehicle identifier (200 and 501), the captured vehicle data (202), and a list of marker instances and their feature instances (502) calculated by 206 and possibly utilizing the marker class pipeline. Also shown in FIG. 5A, the database stores data information for a vehicle appearance list (503), where each vehicle appearance represents a scan of the vehicle under different conditions, such as a time difference (e.g., a scan of license plate number 12345 yesterday and then today), a different direction (e.g., acquiring a license location from a different LOS), different image acquisition conditions (e.g., different quality of license plate acquisition), etc. Each appearance can represent a different scan. Obviously, the same marker instance (e.g., the same scratch) can appear with different appearances (i.e., in different vehicle scans acquired at different time tags).

[0069] FIG. 5b shows an example of a database structure for a marker list (markers, their classes, and their respective features and feature instances). Each marker instance has a unique ID, which can be automatically assigned by the system. At least some of the markers also store, for example, the feature component and coordinates where it was located, as inferred from 204 in combination with 206, and the marker's class (Cross-head screw, Dent, Scratch, Handwriting, Rust, Crack, etc.) obtained from 206. Additional feature instances are stored for each marker class. As shown, by way of example only, for the marker class Handwriting (for the marker instance with ID 0001)—520, the following features are calculated and stored: for each component 504 (e.g., wheel) in which this particular marker instance appears, and the marker's position on the component 505, and appearance (511) (for marker ID 0001, there are three appearances designated 0, 1, and 2, each with a respective timestamp indicating when the corresponding vehicle scan was obtained), an associated feature instance is stored. The coded vector feature 512 may be a vector representation of the marker (i.e., of the handwritten marker instance), and may be useful for future comparisons between the new "handwritten" marker instance and the already stored "handwritten" marker instance, for example, by comparing the "distance" between the newly obtained vector and the already stored vector and comparing the included result with a threshold to determine whether the vectors are close enough, which may provide information of the fact that the newly obtained handwritten marker instance and the already stored handwritten marker instance (in one or more previous appearances) are sufficiently similar, all of which are described herein.

[0070] Additionally, for another maker instance, for example, Cross Thread Marker Class (Marker ID 0002) 521, feature instances are calculated and stored for the component on which the marker appears 506, the position of the marker on the component 507, the angle of the thread 508, and the adjusted reference line 509. The angle, reference line, and position may be stored for each separate scan (appearance).

[0071] Additionally, for another marker instance, for example of the color marker class (marker ID 0003) 530, the following feature instances are calculated and stored: component (eg, paint) 511 and color histogram values ​​510 (stored for each of the appearances).

[0072] Feature instances may be computed by class-specific computational pipelines, possibly shared in common by a small number of classes. For example, the same pipeline may extract the component name and the location of the marker on the component for both the "handwritten" marker class and the "cross-screw" class. Note that there may be additional database tables that specify which features are stored for each marker class.

[0073] It should be noted that the present invention is not bound by the particular data structures shown by way of example only in Figures 5A and 5B, nor is the present invention bound by the specified list of marker classes, nor by their respective characteristics, all of which are provided for illustrative purposes only.

[0074] Upon completion of step 209, the vehicle instance may be registered with the system 100, and for any subsequent scans, the system can use at least the sign-up information for vehicle identification verification.

[0075] It should be noted that the present invention is not bound by the particular sequence of FIG. 2, and thus additional steps may be added, and one or more may be modified and / or removed, all depending on the particular application.

[0076] Referring to FIG. 6, the flow of vehicle identification verification for a return vehicle based on data stored in a database for the same vehicle (e.g., as described with reference to FIG. 5) will be further described. Intuitively, there are N scans of a particular vehicle instance in the system, and the vehicle is scanned N+1 times by the system. The N+1 scan is called a new scan, and the system can verify the vehicle's identity if matching criteria are met, e.g., according to certain embodiments, by "comparing" the marker instance of the N+1 scan with the marker instance of the previously stored N scan to obtain a similarity score. If the new marker instance as a similarity score exceeds a given threshold, it represents a verified new marker instance. The comparison may also refer to the corresponding marker class feature (instance). If the comparison indicates a "sufficient" match between the new marker instance and the previously stored marker instance, the vehicle's identity is verified; otherwise, the vehicle is rejected.

[0077] Referring specifically to Figure 6, according to a particular embodiment, steps 601-604 are identical to steps 200-206 and include obtaining a vehicle identifier (601), capturing a vehicle appearance (602), image segmentation (603), and marker detection and classification (604). The markers obtained by step 604 of the new scan are further referred to as new marker instances, and all marker instances registered for the vehicle instance before the new scan are reference marker class instances. According to this embodiment, the new markers may be processed in an N-marker pipeline to determine marker feature instances (similar to those described with reference to step 404 of Figures 2 and 4) and may be stored in a database (e.g., as described with reference to Figure 5), all in the manner described in detail above with reference to Figure 2.

[0078] In step 605, the reference marker instance is retrieved from the database for comparison with the new marker. In step 606, the new marker instance is compared with the reference marker instance (typically resolved by marker feature instance), and based on the comparison results, a similarity score is obtained and a determination is made (607) whether the matching criteria are met to verify (i.e., identify) the vehicle. In the case of verification (608), new vehicle appearances and their associated new marker feature instances (609) are added to the database (as described in FIG. 5) to obtain more updated data on the vehicle, thus improving the likelihood of identifying this particular vehicle with a higher level of certainty in future vehicle inspections, and an appropriate notification is issued (610). Note that in certain embodiments, manual inspection can be used to verify the results in case of doubt. The manually determined results can be fed into the system to improve future verification processes.

[0079] For example, referring to FIG. 5, consider that a marker with marker ID #0001 520 (of class "handwritten") already has two appearances, #0 and #1, stored, and after step 608 it is determined that the new instance of the marker is "similar" to the previous appearance, then in step 609 the new appearance (#2) with its associated time tag is added to the database, obviously along with their corresponding feature instances, providing information about the fact that it is another appearance of the same handwritten marker #0001.

[0080] Returning to FIG. 6, in case of rejection (611), an appropriate notification is issued (612), for example via an I / O module.

[0081] A more detailed sequence of operations for comparing a new marker with a reference marker according to certain embodiments of the presently disclosed subject matter is described with reference to FIG.

[0082] Thus, in one example, in step 700, for each new marker instance (7001), a candidate marker instance (sometimes also referred to as a candidate reference marker instance or VIM) is selected from a list of reference marker instances (7002). In some cases, a candidate marker instance is selected if it has the same marker class (e.g., the same color or cross-head screw) as the new marker. In other cases, a reference marker is a match candidate if it is located on the same semantic component (e.g., as obtained in 104). According to certain other embodiments, a candidate marker instance is selected if it belongs to the same class and is located within the same component. It should be noted that the specified "class" and / or "location" are only examples, and thus one or more other features may be used instead of or in addition to the above to determine a candidate marker.

[0083] Selecting candidate markers by applying refinement criteria (exemplified in a non-limiting manner above) to reduce the number of candidate markers (among the entire set of possible reference markers) allows for optimizing computational complexity since fewer comparisons are made, and also significantly reduces the probability of false matches. As an example, a simple scratch may appear in multiple locations on a vehicle. If the system attempts to compare a scratch in the upper left corner of the chassis with a scratch located on the exhaust, and the matching problem is that the scratches have a similar visual appearance, this may result in a false match and reduce the effectiveness of the system. The latter scenario can be addressed by requiring that a candidate marker be selected if it has the same class as the new marker and is located in the same segment. Of course, the present invention is not limited to this example.

[0084] For example, if additional features, such as the coordinates of the markers, are considered, a more finely tuned candidate marker is obtained, which increases the likelihood of verifying (or rejecting) the vehicle identification while utilizing fewer computational resources and / or potentially less memory consumption. For example, a reference scratch marker instance is selected as a candidate marker if it has the same class as the new marker instance (i.e., scratch in this example) and is present on the same component (e.g., plastic cover) at coordinates similar (e.g., within a predetermined tolerance) to the coordinates of the new marker on the plastic cover. Of course, these are only non-binding examples.

[0085] 7, if the new marker instance has no candidate matches (701), the new marker instance is temporarily stored as a new marker candidate. If the vehicle identification is positively verified, this new marker is considered a new modification to the vehicle that can later be used for identification verification (i.e., its associated data may be stored in a database as a marker instance for the vehicle so identified). Otherwise, the marker instance is deleted from memory or stored for another purpose.

[0086] If the marker has one or more matching candidate markers, a comparison or marker verification process is invoked at 701. In certain embodiments, for some of the markers, visual alignment of a new reference marker instance may be required due to changes in acquisition conditions (orientation, speed, light, angle, etc.), and for some of these marker instances, visual alignment may be performed with respect to the region bounding the semantic component in which the marker is found (702). The specific algorithms used for visual alignment are generally known per se. A non-limiting example of determining whether matching criteria are met to verify the identity of a vehicle follows below.

[0087] Thus, each visually registered pair of new and reference candidate marker instances is provided as input to a comparison system 703 that determines a pairwise similarity score (704), and if it exceeds a threshold, the new marker instance is verified. In other words, when verified, it may indicate that the new marker instance is a new appearance of the reference marker instance (according to the matching criteria, after the vehicle identity has been verified). Such comparisons may be made according to marker class features. For example, color marker histograms may be compared by any distance metric, such as Euclidean distance. For example, specific histogram data (an example of an instance of a feature) of the marker class color of the currently scanned vehicle (or a segment of the currently scanned vehicle) may be compared with specific histogram data (an example of an instance of a feature) of the marker class color of a stored vehicle, and if the comparison results in a similarity score exceeding a given threshold, this provides information that the new marker instance is verified (similarity histogram), i.e., that the new marker instance is another appearance of the reference marker instance, if indeed the vehicle identification is verified (according to the matching criteria described below). As a non-limiting example, the similarity score may be determined by calculating a distance function (e.g., Euclidean) between the histogram values, and if the distance is sufficiently close, e.g., below a given threshold, this represents a similarity score exceeding the given threshold that provides information of the fact that the marker class "color" of the currently scanned vehicle is the same as the marker instance of a reference (known) candidate vehicle, whose data is already stored in the database, meaning that it is another appearance. Note that because apparently different vehicles may share the same color, the fact that a given marker comparison resulted in a sufficient similarity score does not necessarily mean that the matching criteria were met, i.e., that the candidate vehicle was identified as the reference vehicle.

[0088] Note that to determine whether a current marker instance is similar to a previously stored marker instance (i.e., whether it is another appearance of it), a similarity score may be applied to one feature (e.g., requiring a similarity score above threshold A to verify the new marker instance), or may be applied to two or more features (e.g., requiring a first similarity score above threshold A for a first feature, and a second similarity score above threshold B for a similar feature - A and B being determined according to the features).

[0089] It should be noted that the present invention is not constrained by specified pairwise comparisons, i.e., comparisons between new marker instances and reference marker instances, but rather may involve more than two instances. For example, the feature "vector" of a new marker instance may be compared to two or more reference vector features against a given threshold to determine which (if any) it matches, or, as another non-limiting example that goes beyond pairwise comparisons, a distribution of a small number of reference marker class instances may be calculated, and then the probability of the new marker class instance falling into a specified distribution may be calculated and compared to a threshold to determine whether it falls within the distribution (providing information about "similar") or not (providing information about "dissimilar").

[0090] Returning to another example of pairwise comparison, a cross-shaped screw marker comparison can be performed by subtracting the new (for the currently scanned vehicle) screw angle feature instance from the reference screw angle feature instance (for the already stored vehicle). Thus, for example, if the angle of the new marker instance relative to the reference line is 45 degrees and the reference marker angle instance relative to the same reference line is 43 degrees, the difference is 2 degrees. A threshold (similarity score information) can then be applied to determine whether the marker instance (i.e., the new screw) is verified (if it exceeds the threshold, 1.5 degrees information). This means that if the vehicle's identity is verified, the new marker instance (screw) is a new appearance of the screw of the already stored marker instance (a previous appearance of the same screw); otherwise, it is not.

[0091] While the above example referred to a single feature of the cross-screw marker class, it will be apparent from the description herein that in certain embodiments, two or more marker features are compared before a determination regarding similarity can be made. For example, the "baseline" and "position in component coordinates" features of each of the new cross-screw marker instance and the cross-screw marker instance are compared to corresponding thresholds to determine whether the new marker instance is similar to an already stored marker feature.

[0092] As an example, consider a handwritten marker class and exemplary defect classes such as cracks, dents, etc. The comparison may optionally be performed using visual similarity algorithms such as cross-correlation, structural similarity, or image difference, or machine learning algorithms such as DNN-based algorithms.

[0093] With reference to a particular embodiment of the DNN, the latter is pre-trained such that for a given image patch, i.e. an image patch containing, for example, "hand-drawn" markers that are fed to the ML, the latter outputs a representation vector.

[0094] During the training process, the network is provided with multiple input patches, e.g., if a vehicle is scanned n times (n appearances), each of the n instances of a hand-drawn marker instance is "similar but not the same", and the ML outputs corresponding "similarity" vectors representing the same hand-drawn marker instance, such that some of the patches represent different appearances of the same marker and other patches represent different (i.e., not the same) marker appearances.

[0095] As an example, the loss function of a DNN network is configured to train vector representations of different appearances of the same marker instance to be "closer" to each other than vector representations of other markers.

[0096] The present invention is of course not bound to utilizing ML in general, or DNN in particular, to determine the similarity between new markers and reference candidate markers. According to a particular embodiment, during the inference phase, once the network is trained, the calculations are performed as follows: Two marker appearances A and B are given. Image portions containing markers are provided as input to the DNN, once for each marker. In response, the network outputs vector representations rep_A and rep_B. If the marker appearances represent different appearances of the same marker instance, the resulting rep_A and rep_B (output by the network) are "close" to each other (i.e., similar, i.e., the similarity score exceeds a threshold); if rep_A and rep_B represent appearances of two different marker instances, they are "far" from each other (i.e., dissimilar—the similarity score did not exceed a threshold). Post-processing may then be applied by calculating the distance between the representations using the same distance used to train the network to obtain a similarity score. For example, similarity_score = func(dist(rep_A, rep_B)), where func represents a function. As an example, Func(dist) = dist. By way of example, dist may be the cosine distance, the Euclidean distance, etc.

[0097] The specified similarity score calculations are provided by way of example only. Accordingly, it should be noted that similarity score calculation techniques may vary depending on the application, and different similarity score calculation techniques may be applied to different classes and / or features, all depending on the particular application. It should also be noted that, according to certain embodiments, a similarity score may be applied to a new representation and two or more previously stored representations (e.g., an average of a new representation and two or more previously stored representations), etc.

[0098] The output similarity scores of 704 are provided to 705 (providing information on verified marker instances, i.e., marker instances whose scores exceed a threshold). Here, matching criteria are tested in 705 based on at least the number of verified marker instances and the stored marker instances to determine whether the vehicle identification is verified. For example, if the verified marker instances among the stored marker instances exceed a given threshold (e.g., 60%), the vehicle identification is verified (this means that all verified markers are new appearances of corresponding previously stored marker instances). Otherwise, if the matching criteria are not met, the vehicle identification is rejected. The latter condition involves a reverse test that can be applied in addition to or instead of the previous one, namely, checking the number of marker instances that are not verified (i.e., that do not satisfy the similarity score test).

[0099] Consider the following non-limiting examples. In step 705, the calculated similarity scores are received and combined to provide a single decision as to whether the vehicle identification is accepted or rejected.

[0100] Such a system can be implemented in multiple ways, but one non-limiting example is as follows. a.NV - Number of verified marker instances. A marker is considered verified if it has matching candidate marker instances with a marker similarity score greater than, for example, 0.5. b.NO - the number of original markers before the current verification attempt c.TH - A threshold value set to determine the system sensitivity (an example of a matching criterion)

[0101] If split(NV,NO)>TH: Accept the identification (matching criteria met). Else: Refuse to identify.

[0102] For example, consider 20 different marker instances in a database (as illustrated with reference to FIG. 5) - 20 scratch information revealed in a previous scan and characterizing a vehicle (having license plate XYZ), therefore NO=20 in this example. Furthermore, consider that from the newly acquired markers, only 5 have been verified, i.e., their respective similarity scores when compared with the candidate marker exceed 0.5, i.e., NV=5 (verified markers). By this example, if the threshold for accepting (verifying) a vehicle is, for example, 60%, and considering that in the latter example the decision score is 25% (5 / 20), the decision in this example is rejection.

[0103] The present invention is of course not bound by the number of features processed and / or the respective thresholds used to determine that a new marker instance is a verified appearance of an already stored marker instance (e.g., that it is the same scratch), and is certainly not bound by any particular example of determining vehicle verification; therefore, different and / or more complex conditions (e.g., a particular split (NV, NO) > TH) may be used (based at least on the verified marker instance).

[0104] According to a specific embodiment, if the identification is accepted, all new marker instances that match the reference marker instances are registered in the database (5b) as new appearances. Markers that do not match are saved as new marker instances that were not present in the previous scan (possibly indicating new marker instances, e.g., new scratches). The newly stored marker instances represent a fresher appearance of the vehicle and can therefore be further used for identification verification of future scans of the vehicle, improving the accuracy of the verification. Optionally, the result of the identification acceptance is displayed on the GUI 126.

[0105] Note that in some cases, part of the identification acceptance or rejection mechanism at 705 also takes reference markers into account and enforces that some reference marker conditions be verified as part of the identification verification process. For example, if scan N+1 detected two markers, but the previous scan N detected 20 markers, the identification may be rejected even though two new markers have matches, because the minimum number of previous matches was not met.

[0106] It is understood that the examples and embodiments shown herein for determining and logging (storing) and fingerprinting a vehicle for various purposes, including verifying its identity, by no means include all possible alternatives and are intended to represent non-limiting examples only.

[0107] For simplicity, the description primarily refers to visual aspects of appearance and markers, such as vehicle image scans, visual segments and components, visual markers, and classes (e.g., scratches, dents, etc.), and related features; however, it should be noted that the present invention equally applies, mutatis mutandis, to non-visual markers, such as acoustic markers, e.g., a particular set of frequencies generated by a vehicle's engine, instead of or in addition to visual markers. As noted above, other non-visual markers may be utilized, such as an electromagnetic marker class using radar or lidar sensors (e.g., to detect specific defects hidden beneath a non-metallic surface of a vehicle), and / or an IR marker class using IR sensors (e.g., to represent the thermal signature of a specific component), and / or an audio marker class using, for example, audio (e.g., a microphone). The specified sensors and / or marker classes and / or what they represent in relation to a vehicle are merely non-limiting examples.

[0108] Systems and methods according to various embodiments of the present invention can be used to unambiguously verify vehicles. For example, when a vehicle is authorized to enter a sensitive location, such as an electric utility facility, it is necessary to verify that only authorized vehicles are allowed to enter. To date, vehicle make, color, and license plate have been used as a means of verifying vehicle entry. However, an unauthorized person could easily attach an authorized plate number to an unauthorized vehicle (e.g., a vehicle of a similar model and color) and enter the overly sensitive facility for hostile purposes. In contrast, utilizing techniques according to various embodiments of the present invention can address such dangerous scenarios, as the identification of an unauthorized automobile would be rejected because its marker instance does not have a marker instance that is sufficiently "similar" to the marker instance of an authorized vehicle stored in a database.

[0109] As another example, consider an automated automobile service utility (e.g., a car wash service) where the identity of an automobile must be verified before service is provided. By using the teachings of various embodiments of the present invention, counterfeit use (e.g., by attaching an authorized automobile license plate to an unauthorized automobile) can be prevented, thereby resulting in excessive service of unauthorized automobiles, which can clearly have significant financial consequences. Use of the teachings of various embodiments of the present invention leads to unambiguous verification of the identity of automobiles during service, addressing a particular problem and associated financial loss.

[0110] The present invention is not bound by these examples and is applicable to any use that benefits from unambiguous verification of a vehicle's identity, including those that have financial implications.

[0111] It should be noted that certain stages / steps illustrated in and / or described with reference to the figures may be performed differently, e.g., performed in the opposite order and / or performed simultaneously or sequentially, etc. The present disclosure is not limited by the particular order or sequence illustrated or described herein.

[0112] It is to be understood that the present invention is not limited in its application to the details set forth in the description contained herein or in the drawings. The present invention is capable of other embodiments and of being practiced and carried out in various ways. Accordingly, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. Those skilled in the art will therefore appreciate that the conception upon which this disclosure is based may readily be utilized as a basis for the designing of other structures, methods and systems for carrying out some of the purposes of the subject matter of this disclosure.

[0113] It will also be appreciated that the system according to the present invention may be implemented, at least in part, on a suitably programmed computer. Similarly, the present invention contemplates a computer program readable by a computer for carrying out the method of the present invention. The present invention further contemplates a non-transitory computer-readable memory or storage medium tangibly embodying a program of instructions executable by a computer to carry out the method of the present invention.

[0114] A non-transitory computer-readable storage medium that causes a processor to perform aspects of the present invention may be a tangible device capable of holding and storing instructions for use by an instruction-execution device. The computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.

[0115] Those skilled in the art will readily appreciate that various modifications and changes can be made to the embodiments of the present invention, as described above, without departing from the scope thereof as defined in and by the appended claims.

Claims

1. 1. A method for determining a vehicle fingerprint, comprising: a) receiving a vehicle identifier from at least one sensor, the vehicle identifier including at least one vehicle appearance, each vehicle appearance including image data of at least one vehicle scan depicting at least a portion of a vehicle, the vehicle appearance being associated with a unique appearance time tag; b) segmenting the at least one image data, each segment providing information of at least one subcomponent of a respective one of the vehicles; c) determining a plurality of marker instances from the vehicle scan or subcomponent segment data, each marker instance being associated with a marker class and at least one marker feature; d) storing data indicative of a fingerprint of the vehicle, the data including (i) the vehicle appearance and associated appearance time tags, and (ii) the plurality of marker instances; storing the fingerprint of the vehicle, thereby facilitating verification of the fingerprint of the vehicle in future vehicle scans; e) if at least one other vehicle is present and one or more vehicle appearances have unique appearance time tags, repeating steps (b) through (d) for the other vehicle.

2. The method of claim 1, further comprising: for each one of the marker instances having a marker class, and for at least one of the marker classes of the marker instance, determining at least one marker feature instance; determining at least one marker feature instance that is position dependent; and for each marker instance, storing the so-determined marker feature instances including dependent marker feature instances of the subcomponents.

3. The method of claim 1 or 2, wherein the marker classes are selected from the group comprising scratches, dents, handwriting, colors, rust marks, cross threads, and printed text.

4. The method of any one of claims 1 to 3, wherein at least one of the sensors is an IR sensor and at least one of the vehicle scans is at an IR wavelength.

5. 5. The method of claim 1, wherein at least one of the sensors is an audio sensor, further comprising obtaining at least one audio scan of the vehicle and determining from the audio scan at least one audio marker class providing sound information of at least one module associated with the vehicle.

6. 6. The method of claim 1, wherein at least one of the sensors is an electromagnetic sensor, further comprising obtaining at least one electromagnetic scan of the vehicle with the electromagnetic sensor, and determining from the electromagnetic scan at least one electromagnetic marker class providing information of marks hidden beneath a non-metallic surface of the vehicle.

7. 1. A method for verifying a vehicle fingerprint, comprising: a) receiving a vehicle identifier; b) receiving at least one newly acquired vehicle appearance from at least one sensor, each including at least one image data indicative of at least a partial vehicle scan, the vehicle appearance being associated with a unique appearance time tag; c) segmenting the image data into segment data, each segment providing information about at least one subcomponent of a respective one of the vehicles; d) determining a plurality of new marker instances from the vehicle scan or segment data, each new marker instance being associated with a marker class and at least one marker feature; e) extracting at least one previously stored vehicle appearance associated with at least one of said vehicle identifiers and its corresponding marker instances; f) comparing at least one new marker instance of the newly acquired vehicle appearance with a corresponding marker instance of at least one reference marker, which is a previously stored marker instance of the vehicle appearance associated with the same vehicle identifier, and verifying the vehicle fingerprint if a matching criterion is met.

8. The method of claim 7, wherein determining a plurality of new marker instances from the vehicle scan or segment data further comprises: determining at least one new marker feature instance for each one of the new marker instances and for at least one of the marker classes of the new instances; and determining at least one marker feature instance that is position-dependent; wherein extracting comprises extracting, for each marker instance, determined marker features including position-dependent marker features; and comparing comprises determining a similarity score and comparing each new marker instance with at least one marker instance of at least one previously stored appearance of the vehicle; and if at least one of the similarity scores exceeds a threshold that verifies the new marker instance, determining whether the matching criterion is met based on at least the verified marker instance and corresponding stored reference marker instance.

9. For at least one verified marker instance of the newly acquired vehicle appearance, 9. The method of claim 8, further comprising: determining at least one candidate reference marker instance from the stored reference marker instances of at least one vehicle appearance using refinement criteria; and determining whether the matching criteria are met based on at least the verified marker instance and the corresponding stored candidate reference marker instance.

10. The method according to claim 8 or 9, wherein the matching criterion is met if the verified marker instance of the corresponding stored marker instances exceeds a given threshold.

11. The method of any one of claims 7 to 10, wherein at least some of the marker instances are component-dependent and the comparison is segment-dependent, thereby reducing false alarms and computational complexity of the comparison.

12. 10. The method of claim 9, wherein if a matching criterion is met for the verified vehicle, the new marker instances for the verified vehicle that did not exceed the similarity score threshold are stored along with their associated feature instances to improve future vehicle verification.

13. The method of any one of claims 7 to 12, wherein the marker classes are selected from the group comprising scratches, dents, handwriting, colors, rust marks, cross threads, and printed text.

14. The method of any one of claims 7 to 13, wherein at least one of the sensors is an IR sensor, and wherein at least one of the vehicle scans scans at an IR wavelength.

15. 15. The method of claim 7, wherein at least one of the sensors is an audio sensor, further comprising obtaining at least one audio scan of the vehicle and determining from the audio scan of the vehicle at least one audio marker class providing sound information of at least one module associated with the vehicle, wherein the extracting and comparing also applies to the audio marker class.

16. 16. The method of any one of claims 7 to 15, wherein at least one of the sensors is an electromagnetic sensor, and further comprising obtaining at least one electromagnetic sensor scan of the vehicle and determining from the electromagnetic sensor scan at least one electromagnetic marker class providing information of marks hidden beneath a non-metallic surface of the vehicle, wherein the retrieving of previously stored ones from the memory and comparing the at least one new marker instance of the new appearance with corresponding marker instances also applies to the electromagnetic marker class.

17. 1. A computerized system for determining a fingerprint of a vehicle, said system comprising: receiving a vehicle identifier; receiving at least one vehicle appearance from at least one sensor, the at least one vehicle appearance including at least one image data indicative of at least a partial vehicle scan, the vehicle appearance being associated with a unique appearance time tag; segmenting the image data into segment data, each segment providing information about at least one subcomponent of a respective one of the vehicles; determining a plurality of marker instances from the vehicle scan or segment data, each marker instance being associated with a marker class and at least one marker feature; storing in a storage data indicative of a fingerprint of said vehicle including said vehicle identifier and at least its corresponding (i) vehicle appearance and associated appearance time tag, and (ii) marker instances of said vehicle so determined; and storing, thereby facilitating verification of the fingerprint of the vehicle in future vehicle scans.

18. 1. A computerized system for verifying a vehicle fingerprint, said system comprising: receiving a vehicle identifier; receiving at least one new vehicle appearance from at least one sensor, each new vehicle appearance including at least one image data indicative of at least a partial vehicle scan, the vehicle appearance being associated with a unique appearance time tag; segmenting the image data into segment data, each segment providing information about at least one subcomponent of a respective one of the vehicles; determining a plurality of new marker instances from the at least partial vehicle scan or segment data, each new marker instance being associated with a marker class and at least one marker feature; retrieving at least one previously stored vehicle appearance associated with at least one of the vehicle identifiers and their corresponding vehicle marker instances; comparing at least one new marker instance of the new vehicle appearance with the corresponding marker instance of at least one previously stored vehicle appearance of the same vehicle identifier, and verifying the vehicle fingerprint if a matching criterion is met.

19. A non-transitory computer readable medium comprising instructions that, when executed by a computer, cause the computer to perform the method steps of any one of claims 1 to 6.

20. A non-transitory computer readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method steps of any one of claims 7 to 16.

Images