Vehicle part identification method, device and system

Abstract

An embodiment of the present invention discloses a vehicle part identification method, comprising: obtaining an image to be detected of the vehicle by performing radiation imaging on the vehicle; mapping the image to be detected into a label image by using an image mapping function, wherein the label image includes labels identifying components of the vehicle; mapping the labels in the label image to the image to be detected by using a coordinate mapping function; and determining the location of the component of the vehicle in the image to be detected according to the labels, wherein the image mapping function is obtained by training a convolutional neural network using a cost function based on an input training set of images to be detected and a label image set corresponding to the training set of images to be detected, and wherein the coordinate mapping function is obtained by matching feature points.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a vehicle part recognition method, device and system. Background Art

[0002] Cargo transportation has always been the focus of customs supervision in my country. Criminals often hide undeclared goods inside the car or in different locations on the car body. Since the hiding location is unknown, checking the goods and car one by one seriously reduces the efficiency of passing. In order to improve the inspection efficiency, my country's customs proposed an intelligent customs clearance idea. First, the container or van is imaged by radiation to determine the location of different parts of the vehicle, and then different inspection strategies are adopted for the corresponding locations. In traditional methods, different models need to be trained according to fixed vehicle models. However, this method does not have good generalization performance for vehicles of different models.

[0003] Public Content

[0004] In order to solve the above problems, the present invention provides a method for obtaining the positions of different body parts of a container or a van. The method obtains a unified model for vehicle data, that is, the model has good generalization performance on different vehicle model data.

[0005] According to one aspect of the present invention, a vehicle part recognition method is provided, comprising:

[0006] Acquiring an image of the vehicle to be detected by performing radiation imaging on the vehicle;

[0007] Mapping the image to be detected into a label image by an image mapping function, wherein the label image includes labels identifying components of the vehicle;

[0008] Mapping the label in the label image to the image to be detected by a coordinate mapping function; and

[0009] Determine the location of the component of the vehicle in the image to be detected according to the label,

[0010] The image mapping function is obtained by training the convolutional neural network using a cost function based on an input training set of images to be detected and a label image set corresponding to the training set of images to be detected, and

[0011] Wherein, the coordinate mapping function is obtained by matching feature points.

[0012] In some embodiments, the cost function is a cost function in a Cyclegan model.

[0013] In some embodiments, determining the coordinate mapping function by matching the feature points includes:

[0014] Respectively determining a plurality of feature points in the image to be detected and a plurality of feature points in the label image;

[0015] matching the plurality of feature points in the image to be detected with the plurality of feature points in the label image; and

[0016] The coordinate mapping function is determined according to the matching result.

[0017] In some embodiments, the image mapping function is obtained by training the convolutional neural network based on the input image training set to be detected and the label image set by minimizing the cost function, and the cost function is the sum of the first adversarial loss function, the second adversarial loss function and the cycle consistency loss function, wherein,

[0018] The first adversarial loss function, that is, the cost function for transforming the image to be detected into the label image, is

[0019] The second adversarial loss function, that is, the cost function for transforming the label image into the image to be detected, is and

[0020] The cycle consistency loss function is

[0021] Where X is the training set of images to be detected, x is a sample in X, i.e., an image to be detected, and p data(x) is the probability distribution of x, that is, the distribution of real data; Y is the set of labeled images corresponding to the training set X of the image to be detected, y is a sample in Y, that is, a labeled image, p data(y) is the probability distribution of y, that is, the distribution of label data; G is the mapping function from the image to be detected to the label image, F is the mapping function from the label image to the image to be detected, E is the expected function; and D X It is a discriminant function used to determine whether the image to be detected obtained by F mapping meets the standard. Y It is a discriminant function used to determine whether the label image obtained by G mapping meets the standard.

[0022] In some embodiments, the components of the vehicle include a cab, a container, a body and wheels.

[0023] In some embodiments, the tags are uniquely identified by integer grayscale values.

[0024] According to another aspect of the present invention, a vehicle part identification method and device is provided, comprising:

[0025] An acquisition module is configured to acquire an image to be detected of the vehicle by performing radiation imaging on the vehicle;

[0026] A first mapping module is configured to map the image to be detected into a label image through an image mapping function, wherein the label image includes a label identifying a component of the vehicle;

[0027] A second mapping module is configured to map the label in the label image to the image to be detected through a coordinate mapping function; and

[0028] A first determination module is configured to determine the location of the component of the vehicle according to the label in the image to be detected,

[0029] The image mapping function is obtained by training the convolutional neural network using a cost function based on an input training set of images to be detected and a label image set corresponding to the training set of images to be detected, and

[0030] Wherein, the coordinate mapping function is obtained by matching feature points.

[0031] In some embodiments, the cost function is a cost function in a Cyclegan model.

[0032] In some embodiments, the vehicle part identification device further includes: a second determination module configured to:

[0033] Respectively determining a plurality of feature points in the image to be detected and a plurality of feature points in the label image;

[0034] matching the plurality of feature points in the image to be detected with the plurality of feature points in the label image; and

[0035] The coordinate mapping function is determined according to the matching result.

[0036] In some embodiments, the image mapping function is obtained by training a convolutional neural network based on the input training set of images to be detected and the label image set by minimizing a cost function, wherein the cost function is the sum of a first adversarial loss function, a second adversarial loss function, and a cycle consistency loss function, wherein:

[0037] The first adversarial loss function, that is, the cost function for transforming the image to be detected into the label image, is

[0038] The second adversarial loss function, that is, the cost function for transforming the label image into the image to be detected, is and

[0039] The cycle consistency loss function is

[0040] Where X is the training set of images to be detected, x is a sample in X, i.e., an image to be detected, and p data(x) is the probability distribution of x, that is, the distribution of real data; Y is the set of labeled images corresponding to the training set X of the image to be detected, y is a sample in Y, that is, a labeled image, p data(y) is the probability distribution of y, that is, the distribution of label data; G is the mapping function from the image to be detected to the label image, F is the mapping function from the label image to the image to be detected, E is the expected function; and D X It is a discriminant function used to determine whether the image to be detected obtained by F mapping meets the standard. Y It is a discriminant function used to determine whether the label image obtained by G mapping meets the standard.

[0041] In some embodiments, the components of the vehicle include a cab, a container, a body and wheels.

[0042] In some embodiments, the tags are uniquely identified by integer grayscale values. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0044] Figure 1 A flow chart showing a vehicle part identification method according to an embodiment of the present invention is shown;

[0045] Figure 2 A schematic diagram showing a vehicle part recognition result according to an embodiment of the present invention;

[0046] Figure 3 A schematic diagram showing a training set of images to be detected and a label image set according to an embodiment of the present invention is shown;

[0047] Figure 4 A block diagram showing a vehicle part recognition device according to an embodiment of the present invention; and

[0048] Figure 5 A schematic diagram of a vehicle part identification method system according to an embodiment of the present invention is shown.

[0049] The drawings do not show all circuits or structures of the embodiments. The same reference numerals throughout all drawings represent the same or similar components or features. DETAILED DESCRIPTION

[0050] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concept of the present invention.

[0051] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The words "a", "an (kind)" and "the" used herein should also include the meanings of "multiple" and "multiple", unless the context clearly indicates otherwise. In addition, the terms "including", "comprising" and the like used herein indicate the presence of features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0052] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0053] Figure 1 The flowchart of the vehicle part recognition method 100 according to an embodiment of the present invention is shown. In step S110, the vehicle can be imaged by radiation imaging such as X-rays, gamma rays, etc. to obtain the image to be detected (e.g., radiation image). Here, the vehicle can include ordinary trucks, vans, closed trucks, tank trucks, flatbed trucks, container trucks, dump trucks, etc.

[0054] In step S120, the image to be detected can be mapped into a label image by an image mapping function, and the label image includes labels identifying components of the vehicle. The components of the vehicle may include a cab, a container, a body, and wheels. The label can be uniquely identified by an integer grayscale value. For example, when the label is identified by a grayscale value of 1, the label identifies the cab, when the label is identified by a grayscale value of 2, the label identifies the container, when the label is identified by a grayscale value of 3, the label identifies the body, when the label is identified by a grayscale value of 4, the label identifies the wheel, and so on. The label image can be annotated at the pixel level, and the format of the image can include a PNG format, a BMP format, or a jpg format. Figure 2 A schematic diagram showing a vehicle part recognition result according to an embodiment of the present invention. Figure 2 The left side is the image to be detected. Figure 2The right side of is the label image obtained by mapping the image to be detected through the image mapping function. Through the labels identified by different gray values ​​in the label image, the components of the vehicle identified by each label can be determined.

[0055] Before step S120, the image to be detected may be preprocessed (for example, denoising) to remove noise such as stripes in the image to be detected.

[0056] The image mapping function can be based on the input image training set to be detected (for example, Figure 3 The radiation image training set shown in FIG. 1 ) and the label image set corresponding to the image training set to be detected (eg, registered) (eg, Figure 3 The cost function can be the cost function in the Cyclegan model.

[0057] In one example, the image mapping function can be obtained by training a convolutional neural network based on an input training set of images to be detected and a set of labeled images by minimizing a cost function, where the cost function is the sum of a first adversarial loss function, a second adversarial loss function, and a cycle consistency loss function, wherein,

[0058] The first adversarial loss function, that is, the cost function of transforming the image to be detected into the label image, is

[0059]

[0060] The second adversarial loss function, that is, the cost function of transforming the label image into the image to be detected, is

[0061]

[0062] and

[0063] The cycle consistency loss function is

[0064] Where X is the training set of images to be detected, x is a sample in X, i.e., an image to be detected, and p data(x) is the probability distribution of x, that is, the distribution of real data; Y is the set of labeled images corresponding to the training set X of the image to be detected, y is a sample in Y, that is, a labeled image, p data(y) is the probability distribution of y, that is, the distribution of label data; G is the mapping function from the image to be detected to the label image, F is the mapping function from the label image to the image to be detected, E is the expected function; and D X It is a discriminant function used to determine whether the image to be detected obtained (generated) by F mapping meets the standard, for example, the cross entropy loss function, D YIt is a discriminant function used to determine whether the label image obtained (generated) by G mapping meets the standard, for example, the cross entropy loss function.

[0065] In step S130, the label in the label image can be mapped to the image to be detected by a coordinate mapping function. The coordinate mapping function can be obtained by feature point matching. In one embodiment, determining the coordinate mapping function by feature point matching can include: respectively determining multiple feature points in the image to be detected and multiple feature points in the label image; matching multiple feature points in the image to be detected and multiple feature points in the label image; and determining the coordinate mapping function based on the matching results. Feature points can refer to special points in the image, such as corner points, straight line intersections, T-junctions, high curvature points, and the center and center of gravity of a specific area.

[0066] In one embodiment, matching multiple feature points in the image to be detected and multiple feature points in the label image may include: for a certain feature point in the label image, taking a fixed-size window with the feature point as the center, and according to the position of a certain feature point, determining a circular domain range with a radius of about 20 pixels and a corresponding position with the same spatial coordinates as the center in the image to be checked, taking a window of the same fixed size for each feature point in the circular domain in the image to be checked, and calculating the cross-correlation between the window in the template image and this window, and taking a feature point with the largest cross-correlation as the matching point of a certain feature point. Of course, the feature point matching method in the present invention is not limited to this.

[0067] In step S140, the location of the component of the vehicle may be determined in the image to be detected according to the label.

[0068] Figure 4 The block diagram of a vehicle part identification method and apparatus 400 according to an embodiment of the present invention is shown. The vehicle part identification method and apparatus 400 may include an acquisition module 410 , a first mapping module 420 , a second mapping module 430 and a first determination module 440 .

[0069] The acquisition module 410 can be configured to acquire an image to be detected of the vehicle by performing radiation imaging on the vehicle. The components of the vehicle may include a cab, a container, a body and wheels. The first mapping module 420 can be configured to map the image to be detected into a label image through an image mapping function, and the label image may include a label identifying the components of the vehicle. The label can be uniquely identified by an integer grayscale value. The image mapping function can be obtained by training a convolutional neural network using a cost function based on an input training set of images to be detected and a set of label images corresponding to the training set of images to be detected. The cost function can be a cost function in a Cyclegan model. In one example, the image mapping function can be obtained by training a convolutional neural network by minimizing the cost function based on the input training set of images to be detected and the set of label images, and the cost function is the sum of a first adversarial loss function, a second adversarial loss function and a cycle consistency loss function, wherein,

[0070] The first adversarial loss function, that is, the cost function of transforming the image to be detected into the label image, is

[0071]

[0072] The second adversarial loss function, that is, the cost function of transforming the label image into the image to be detected, is

[0073]

[0074] and

[0075] The cycle consistency loss function is

[0076] Where X is the training set of images to be detected, x is a sample in X, i.e., an image to be detected, and p data(x) is the probability distribution of x, that is, the distribution of real data; Y is the set of labeled images corresponding to the training set X of the image to be detected, y is a sample in Y, that is, a labeled image, p data(y) is the probability distribution of y, that is, the distribution of label data; G is the mapping function from the image to be detected to the label image, F is the mapping function from the label image to the image to be detected, E is the expected function; and D X It is a discriminant function used to determine whether the image to be detected obtained (generated) by F mapping meets the standard, for example, the cross entropy loss function, D Y It is a discriminant function used to determine whether the label image obtained (generated) by G mapping meets the standard, for example, the cross entropy loss function.

[0077] The second mapping module 430 may be configured to map the label in the label image to the image to be detected by a coordinate mapping function. The coordinate mapping function may be obtained by matching feature points. The first determining module 440 may be configured to determine the location of the component of the vehicle in the image to be detected according to the label.

[0078] The vehicle part identification device 400 may also include: a second determination module, configured to: respectively determine multiple feature points in the image to be detected and multiple feature points in the label image; match the multiple feature points in the image to be detected and multiple feature points in the label image; and determine the coordinate mapping function according to the matching results.

[0079] By using the embodiments of the present invention, different types of vehicles can be annotated with a unified label; the image to be detected can be trained with the multi-domain conversion method in deep learning to obtain the label image of the radiation image; the obtained label image can be registered with the image to be detected using a method based on feature point alignment; and the label image can be corrected to correspond to the original image. In this way, the position of the components of the vehicle can be determined according to the coordinate positions of different labels, thereby facilitating subsequent detection. The vehicle part recognition method and device according to the embodiments of the present invention overcome the problems of low accuracy of a single model in detecting different types of vehicles and inaccurate detection of single-vehicle double-boxes. The present invention is also applicable to dual-perspective or multi-perspective images, and the positions of different components of the vehicle can be obtained by processing only the side-view angle image.

[0080] Figure 5 A schematic diagram of a vehicle part identification system 500 according to an embodiment of the present invention is shown. The system 500 may include a processor 510, such as a digital signal processor (DSP). The processor 510 may be a single device or multiple devices for performing different actions of the processes described herein. The system 500 may also include an input / output (I / O) device 530 for receiving signals from other entities or sending signals to other entities.

[0081] In addition, the system 500 may include a memory 520, which may be in the form of a non-volatile or volatile memory, such as an electrically erasable programmable read-only memory (EEPROM), flash memory, etc. The memory 520 may store computer-readable instructions that, when executed by the processor 510, may cause the processor to perform the actions described herein.

[0082] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flow charts may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions, when executed by the processor, may create a device for implementing the functions / operations described in these block diagrams and / or flow charts.

[0083] Therefore, the technology of the present invention can be implemented in the form of hardware and / or software (including firmware, microcode, etc.). In addition, the technology of the present invention can take the form of a computer program product on a computer-readable medium storing instructions, which can be used by an instruction execution system (e.g., one or more processors) or in combination with an instruction execution system. In the context of the present invention, a computer-readable medium can be any medium that can contain, store, transmit, propagate, or transmit instructions. For example, a computer-readable medium can include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, device, or propagation medium. Specific examples of computer-readable media include: magnetic storage devices, such as magnetic tape or hard disk (HDD); optical storage devices, such as compact disks (CD-ROM); memory, such as random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0084] The above detailed description has described many embodiments of the vehicle part identification method, device and system by using schematic diagrams, flow charts and / or examples. In the case where such schematic diagrams, flow charts and / or examples contain one or more functions and / or operations, it should be understood by those skilled in the art that each function and / or operation in such schematic diagrams, flow charts or examples can be implemented individually and / or together by various structures, hardware, software, firmware or substantially any combination thereof. In one embodiment, several parts of the subject matter described in the embodiments of the present invention can be implemented by application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), or other integrated formats. However, it should be recognized by those skilled in the art that some aspects of the embodiments disclosed herein can be equivalently implemented in integrated circuits in whole or in part, implemented as one or more computer programs running on one or more computers (e.g., implemented as one or more programs running on one or more computer systems), implemented as one or more programs running on one or more processors (e.g., implemented as one or more programs running on one or more microprocessors), implemented as firmware, or substantially implemented as any combination of the above, and those skilled in the art will have the ability to design circuits and / or write software and / or firmware code according to the present invention. Furthermore, those skilled in the art will recognize that the mechanisms of the subject matter described herein can be distributed as a program product in a variety of forms, and that the exemplary embodiments of the subject matter described herein are applicable regardless of the specific type of signal-bearing medium actually used to perform the distribution. Examples of signal-bearing media include, but are not limited to: recordable media such as floppy disks, hard drives, compact disks (CDs), digital versatile disks (DVDs), digital tapes, computer memories, etc.; and transmission-type media such as digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

Claims

1. A vehicle part recognition method, comprising: Acquiring an image of the vehicle to be detected by performing radiation imaging on the vehicle; Mapping the image to be detected into a label image by an image mapping function, wherein the label image includes labels identifying components of the vehicle; Mapping the label in the label image to the image to be detected by a coordinate mapping function; as well as Determine the location of the component of the vehicle in the image to be detected according to the label, The image mapping function is obtained by training a convolutional neural network based on an input image training set to be detected and a label image set corresponding to the image training set to be detected using a cost function, wherein the cost function is the sum of a first adversarial loss function, a second adversarial loss function and a cycle consistency loss function, wherein, The first adversarial loss function is , The second adversarial loss function is ,and The cycle consistency loss function is , Where X is the training set of images to be detected, x is a sample in X, i.e., an image to be detected, is the probability distribution of x; Y is the set of labeled images corresponding to the training set X of the image to be detected, y is a sample in Y, that is, a labeled image, is the probability distribution of y; G is the mapping function from the image to be detected to the label image, F is the mapping function from the label image to the image to be detected, E is the expected function; and D X is a discriminant function used to determine whether the image to be detected obtained by F mapping meets the standard, and D Y is a discriminant function used to determine whether the label image obtained by G mapping meets the standard, and Wherein, the coordinate mapping function is obtained by matching feature points.

2. The vehicle part recognition method according to claim 1, wherein: Determining the coordinate mapping function by matching the feature points includes: Respectively determining a plurality of feature points in the image to be detected and a plurality of feature points in the label image; matching the plurality of feature points in the image to be detected with the plurality of feature points in the label image; and The coordinate mapping function is determined according to the matching result.

3. The vehicle part recognition method according to claim 1, wherein: The components of the vehicle include a cab, a container, a vehicle body and wheels.

4. The vehicle part recognition method according to claim 1, wherein: The tags are uniquely identified by integer grayscale values.

5. A vehicle part identification method and device, comprising: An acquisition module is configured to acquire an image to be detected of the vehicle by performing radiation imaging on the vehicle; A first mapping module is configured to map the image to be detected into a label image through an image mapping function, wherein the label image includes a label identifying a component of the vehicle; A second mapping module is configured to map the label in the label image to the image to be detected through a coordinate mapping function; as well as A first determination module is configured to determine the location of the component of the vehicle according to the label in the image to be detected, The image mapping function is obtained by training a convolutional neural network based on an input image training set to be detected and a label image set corresponding to the image training set to be detected using a cost function, wherein the cost function is the sum of a first adversarial loss function, a second adversarial loss function and a cycle consistency loss function, wherein, The first adversarial loss function is , The second adversarial loss function is ,and The cycle consistency loss function is , Where X is the training set of images to be detected, x is a sample in X, i.e., an image to be detected, is the probability distribution of x; Y is the set of labeled images corresponding to the training set X of the image to be detected, y is a sample in Y, that is, a labeled image, is the probability distribution of y; G is the mapping function from the image to be detected to the label image, F is the mapping function from the label image to the image to be detected, E is the expected function; and D X is a discriminant function used to determine whether the image to be detected obtained by F mapping meets the standard, and D Y is a discriminant function used to determine whether the label image obtained by G mapping meets the standard, and Wherein, the coordinate mapping function is obtained by matching feature points.

6. The vehicle part recognition device according to claim 5, further comprising: The second determining module is configured to: Respectively determining a plurality of feature points in the image to be detected and a plurality of feature points in the label image; Matching the multiple feature points in the image to be detected with the multiple feature points in the label image; as well as The coordinate mapping function is determined according to the matching result.

7. The vehicle part recognition device according to claim 5, wherein: The components of the vehicle include a cab, a container, a vehicle body and wheels.

8. The vehicle part recognition device according to claim 5, wherein: The tags are uniquely identified by integer grayscale values.

Images