A general vehicle underbody image foreign object detection method

Abstract

The application discloses a universal vehicle bottom image foreign matter detection method and relates to the technical field of vehicle bottom foreign matter detection. The method compares a target picture with a standard picture, detects the difference area of the two similar pictures, and the detection sensitivity can be adjusted according to the input value. The method has good adaptability to the possible situations during actual vehicle bottom recording, such as distortion, lateral shift and light condition change, and can avoid the situation that misrecognition as foreign matter occurs due to light condition change to a certain extent. Moreover, the method is not a detection method for a specific target, and is universally applicable to various scenes, such as prisons, customs, venue entrances and the like. The operation time of each pair of pictures is less than 1s, compared with the deep learning detection method, the detection time will not be prolonged with the increase of detection targets, different FODs with different difference degrees can be output at different detection levels, the method can be used not only for finding prohibited articles, but also for finding the loss of vehicle bottom components or the modification of chassis and the like scenes.

Description

Technical Field

[0001] This invention relates to the field of foreign object detection technology under vehicles, specifically to a universal method for detecting foreign objects under vehicles using images, applicable not only to finding prohibited items under vehicles but also to finding missing parts or chassis modifications. Background Technology

[0002] Vehicle chassis inspection is primarily used in two scenarios: security and chassis inspection. Security inspection checks for prohibited items carried on the vehicle chassis, while chassis inspection mainly detects modifications, missing parts, or malfunctions. However, the actual acquisition of undercarriage images is affected by various factors, such as vehicle speed, differences in vehicle position, and even turning, causing distortion and changes in relative position in the images. Furthermore, changes in lighting conditions alter the contrast and color tone of the undercarriage in the images. Moreover, the objects to be detected differ in various scenarios, and most scenarios have real-time requirements for detection, all of which pose challenges to existing undercarriage foreign object detection algorithms.

[0003] Most current foreign object detection algorithms for vehicle undercarriage use deep learning methods, but they cannot identify foreign objects other than the training target, such as missing parts of the vehicle undercarriage or disguised objects. They also require a large amount of data and computing power, and the recognition time increases with the number of targets to be detected. Summary of the Invention

[0004] To improve the general applicability of foreign object detection under vehicles, this invention provides a universal method for detecting foreign objects under vehicle images. This method can not only be used to find prohibited items under vehicles, but also to find missing parts under vehicles or chassis modifications.

[0005] The technical solution adopted by the present invention to achieve the above-mentioned technical effects is as follows:

[0006] A general method for foreign object detection under vehicle images includes the following steps:

[0007] S1. Input the target image and the standard image, as well as the FOD_level value;

[0008] S2. Convert the image color space to HSV;

[0009] S3. Use bicubic interpolation with a 4x4 pixel neighborhood to scale both the target image and the standard image to four different size levels.

[0010] S4. Obtain the edges of images at each size level;

[0011] S5. Fuse the brightness components and edge information of the image according to a certain ratio;

[0012] S6. Obtain the tire position based on the change information of the longitudinal brightness value and save it to a list;

[0013] S7. Register target images and standard images at different scales for each size level;

[0014] S8. Calculate the SSD value that needs to be considered as FOD based on the FOD_level value, determine whether it is FOD based on the SSD value of the pixel change value at intervals, eliminate the wrong identification caused by image differences due to changes in lighting conditions, and search based on the image saturation component to avoid missed detection when the brightness components are similar.

[0015] S9. Any object that meets either the condition of foreign object detected by saturation component or foreign object detected by brightness edge fusion image is considered a foreign object.

[0016] S10. Merge adjacent foreign object squares into a rectangle;

[0017] S11. Output a list of rectangles.

[0018] Preferably, in the above-described general method for detecting foreign objects under vehicle images, step S7 includes:

[0019] A mapping is established between regions in the target image and regions in the standard image. Matching is performed at different scales, and after registration, the differences between the corresponding regions are compared.

[0020] Preferably, in the above-described general method for detecting foreign objects under vehicle images, step S7 specifically includes the following steps:

[0021] S71. Register the 256*128 target image and the standard image. Divide the 256*128 target image into 32*16 squares. Use a vector to represent the position of the square in the target image corresponding to the square of the same size in the standard image. The matching vector (x, y) represents the position of the matching square in the standard image after adding (x, y) to the square in the target image. The offset vector (0, 0) represents the position of the square in the target image and the matching square in the standard image.

[0022] S72. Register the 512*256 target image and the standard image, remove the 8-pixel area on each side, and divide the remaining image into 31*63 squares with a square size of 16*16.

[0023] S73. Register the 1024*512 target image and the standard image.

[0024] Preferably, in the above-described general method for detecting foreign objects under vehicle images, step S8 specifically includes the following steps:

[0025] Step S81: Calculate the SSD threshold using the matching vector and SSD value obtained from the 1024*512 image. Images greater than this threshold are considered to be FOD.

[0026] Step S82: On the fused image of size 1024*512 that integrates luminance components and edge information, for each square that is considered a foreign object, the matching area is redefined with the matching vector as the center and {10, 6} as the search range. If the calculated SSD is less than the threshold calculated in step S81, it is considered not to be a FOD.

[0027] Step S83: On the fused image of size 1024*512 that incorporates luminance components and edge information, for each square that is considered a foreign object, calculate the SSD of the pixel change value between the square in the target image and the square pointed to by the matching vector in the standard image. If the SSD is less than the threshold calculated by FOD_level, then it is considered not a foreign object.

[0028] Step S84: On the luminance component image of size 1024*512, for each square that is considered to be a foreign object, with the matching vector as the center and {10, 6} as the search range, the matching area is redefined using SSD. According to the calculated SSD, if it is less than the SSD threshold, it is considered not to be a FOD.

[0029] Step S85: On the luminance component image of size 1024*512, for each square that is considered a foreign object, calculate the SSD of the pixel change value between the square in the target image and the square pointed to by the matching vector in the standard image. If the SSD is less than the threshold calculated by FOD_level, it is considered not a foreign object.

[0030] Step S86: Based on the saturation component image of size 1024*512, find FOD. Using the previously calculated target image grid and the matching grid in the standard image, calculate the SAD of the saturation component images of both. Calculate a threshold for segmenting the SAD distribution to a certain proportion. Based on this threshold and FOD_level, calculate the value to determine whether it is FOD. If it is greater than this value, it is considered to be FOD.

[0031] The beneficial effects of this invention are as follows: This invention compares a target image with a standard image to detect the difference areas between two similar images. Furthermore, the detection sensitivity can be adjusted according to the input value, adapting to interference caused by image distortion and, to a certain extent, avoiding misidentification as foreign objects due to changes in lighting conditions. It can be used not only to find contraband but also to locate missing parts under vehicles or chassis modifications. The calculation time can be controlled within 1 second and is stable. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the segmented 256*128 target image of the present invention;

[0033] Figure 2 This is a schematic diagram of the segmented 512*256 target image of the present invention;

[0034] Figure 3 This is a schematic diagram of the 1024*512 target image segmented according to the present invention;

[0035] Figure 4 Standard images of automobile chassis;

[0036] Figure 5 Input a target image of the car chassis to be detected;

[0037] Figure 6 A car chassis image containing matching vectors;

[0038] Figure 7 Output an image showing the foreign object features on the car chassis in the target image. Detailed Implementation

[0039] To provide a further understanding of the present invention, the invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0040] In the description of this invention, it should be noted that the terms "vertical," "upper," "lower," and "horizontal," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0041] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or a connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0042] Embodiments of the present invention propose a general method for foreign object detection in vehicle undercarriage images, comprising the following steps:

[0043] S1. Input the target image and the standard image, as well as the FOD_level value;

[0044] S2. Convert the image color space to HSV;

[0045] S3. Use bicubic interpolation with a 4x4 pixel neighborhood to scale both the target image and the standard image to four different size levels.

[0046] S4. Obtain the edges of images at each size level;

[0047] S5. Fuse the brightness components and edge information of the image according to a certain ratio;

[0048] S6. Obtain the tire position based on the change information of the longitudinal brightness value and save it to a list;

[0049] S7. Register target images and standard images at different scales for each size level;

[0050] S8. Calculate the SSD value that needs to be considered as FOD based on the FOD_level value, determine whether it is FOD based on the SSD value of the pixel change value at intervals, eliminate the wrong identification caused by image differences due to changes in lighting conditions, and search based on the image saturation component to avoid missed detection when the brightness components are similar.

[0051] S9. Any object that meets either the condition of foreign object detected by saturation component or foreign object detected by brightness edge fusion image is considered a foreign object.

[0052] S10. Merge adjacent foreign object squares into a rectangle;

[0053] S11. Output a list of rectangles.

[0054] Furthermore, in an embodiment of the present invention, in a general method for detecting foreign objects under a vehicle image, step S7 includes:

[0055] A mapping is established between regions in the target image and regions in the standard image, and matching is performed at different scales. Matching at different scales reduces computational cost and ensures registration accuracy. After registration, the differences between corresponding regions are compared, which can adapt to distortions and changes in relative position that occur during the recording of images of the undercarriage.

[0056] Furthermore, in an embodiment of the present invention, step S7 specifically includes the following steps:

[0057] S71. Register the 256*128 target image and the standard image. Divide the 256*128 target image into 32*16 squares. Use a vector to represent the position of the square in the target image corresponding to the square of the same size in the standard image. The matching vector (x, y) represents the position of the matched square in the standard image after adding (x, y) to the square in the target image. The offset vector (0, 0) indicates that the square in the target image is at the same position as the matched square in the standard image.

[0058] like Figure 1 The image shown is a schematic diagram of the 256*128 target image segmented in step S71. The symbols marked on the squares indicate the order of inter-square retrieval. The specific retrieval order is as follows:

[0059] 1. First, search the squares marked 'o'. Using the square at offset vector (0, 0) as the center, within the search range of {16, 8}, use the SSD value to find the most similar region. (The smaller the SSD value, the more similar the two regions are.) Record the matching vector with the smallest SSD value and the SSD value.

[0060] 2. Retrieve the squares marked with '#', and using the offset vectors of the three squares marked with 'o' as the center, within a search range of {10, 5}, use the SSD value to find the most similar region. Record the matching vector with the smallest SSD value and its corresponding SSD value.

[0061] 3. Retrieve the squares marked with '@'. In column 1, the '@' squares are centered on the matching vectors marked with 'o' and '#' in columns 3, 5, and 7 of the corresponding rows. Within a search range of {6, 3}, the SSD value is used to find the most similar region. Record the matching vector with the smallest SSD value and its corresponding SSD value.

[0062] 4. Search for squares marked with '$'. Using the matching vectors of the top-left, bottom-left, top-right, and bottom-right squares as the center, and within a search range of {6, 3}, use the SSD value to find the most similar region. Record the matching vector with the smallest SSD value and its corresponding SSD value.

[0063] 5. Retrieve squares marked with '^'. Centering on the matching vectors of the top-left, bottom-left, top-right, and bottom-right squares respectively, and within a search range of {4, 2}, use the SSD value to find the most similar region. Record the matching vector with the smallest SSD value and its corresponding SSD value. Note: For reference squares whose matching vectors have not yet been calculated, they are not included in the calculation.

[0064] 6. Search the cells marked with '<', and using the matching vectors of the cells in the same row (columns 1, 2, 3, and 4) as the center, within a search range of {2, 1}, use the SSD value to find the most similar region. Record the matching vector with the smallest SSD value and its corresponding SSD value.

[0065] 7. For the squares marked with ">", using the matching vectors of the squares in the same row (columns a, b, c, and d) as the center, and within a search range of {2, 1}, use the SSD value to find the most similar region. Record the matching vector with the smallest SSD value and its corresponding SSD value.

[0066] 8. Use the median method to remove vectors with abnormal matches. For the matching vector of each square in the target image, calculate the offset variable of the 3*3 square centered on it, calculate the total value of the x and y components respectively, and combine them into a new offset variable as the offset vector of the current square.

[0067] S72. Register the 512*256 target image and the standard image, remove the 8-pixel area on each side, and divide the remaining image into 31*63 squares, with each square being 16*16.

[0068] like Figure 2 The image shown is a schematic diagram of the 512*256 target image after segmentation in step S72. The symbols marked on the squares indicate the order of inter-square retrieval. The specific retrieval order is as follows:

[0069] 1. First, search for squares marked with 'o'. Using the offset vector of the 256*128 square as the center, within a search range of {4, 2}, use the SSD value to find the most similar region. Record the matching vector with the smallest SSD value and its corresponding SSD value.

[0070] 2. For the squares marked with '$', using the matching vectors of the top-left, bottom-left, top-right, and bottom-right squares as the center, and within a search range of {6, 3}, use the SSD value to find the most similar region. Record the matching vector with the smallest SSD value and its corresponding SSD value. Note: For reference squares whose matching vectors have not yet been calculated, they are not included in the calculation.

[0071] 3. Retrieve the squares marked with '@', centering on the matching vectors of the top-left, bottom-left, top-right, and bottom-right squares respectively, and using the SSD value to find the most similar region within a search range of {6, 3}. Record the matching vector with the smallest SSD value and its corresponding SSD value. Note: For reference squares whose matching vectors have not yet been calculated, they are not included in the calculation.

[0072] 4. Retrieve the square marked with '=", using the offset vector of the 256*128 square as the center, and within a search range of {2, 1}, use the SSD value to find the most similar region. Record the matching vector with the smallest SSD value and its corresponding SSD value.

[0073] 5. Use the median method to remove vectors with abnormal matches. For the matching vector of each square in the target image, calculate the offset variable of the 3*3 square centered on it, calculate the total value of the x and y components respectively, and combine them into a new offset variable as the offset vector of the current square.

[0074] S73. Register the 1024*512 target image and the standard image.

[0075] like Figure 3 The image shown is a schematic diagram of the 1024*512 target image segmented in step S73. The symbols marked on the squares indicate the order of inter-square retrieval. The specific retrieval order is as follows:

[0076] 1. First, search the squares marked with 'o'. Using the offset vector of the 512*256 square as the center, within a search range of {2, 1}, use the SSD value to find the most similar region. Record the matching vector with the smallest SSD value and its corresponding SSD value.

[0077] 2. For the squares marked with '$', using the matching vectors of the top-left, bottom-left, top-right, and bottom-right squares as the center, and within a search range of {4, 2}, use the SSD value to find the most similar region. Record the matching vector with the smallest SSD value and its corresponding SSD value. Note: For reference squares whose matching vectors have not yet been calculated, they are not included in the calculation.

[0078] 3. Retrieve the squares marked with '@', centering on the matching vectors of the top-left, bottom-left, top-right, and bottom-right squares respectively, and using the SSD value to find the most similar region within a search range of {4, 2}. Record the matching vector with the smallest SSD value and its corresponding SSD value. Note that matching vectors for reference squares that have not yet been calculated are not included in the calculation.

[0079] 4. Retrieve the square marked with '=", using the offset vector of the 256*128 square as the center, and within a search range of {2, 1}, use the SSD value to find the most similar region. Record the matching vector with the smallest SSD value and its corresponding SSD value.

[0080] 5. Remove outlier vectors using the median method. For the matching vector of each square in the target image, calculate the offset variable of the 3*3 square centered on it, and calculate the total value of the x and y components respectively. Combine them to form a new offset variable as the offset vector of the current square.

[0081] Furthermore, in an embodiment of the present invention, step S8 specifically includes the following steps:

[0082] Step S81: Calculate the SSD threshold using the matching vector and SSD value obtained from the 1024*512 image. Images greater than this threshold are considered to be FOD.

[0083] Step S82: On the fused image of size 1024*512 that integrates luminance components and edge information, for each square that is considered a foreign object, the matching area is redefined with the matching vector as the center and {10, 6} as the search range. If the calculated SSD is less than the threshold calculated in step S81, it is considered not to be a FOD.

[0084] Step S83: On the fused image of size 1024*512 that incorporates luminance components and edge information, for each square that is considered a foreign object, calculate the SSD of the pixel change value between the square in the target image and the square pointed to by the matching vector in the standard image. If the SSD is less than the threshold calculated by FOD_level, then it is considered not a foreign object.

[0085] Step S84: On the luminance component image of size 1024*512, for each square that is considered to be a foreign object, with the matching vector as the center and {10, 6} as the search range, the matching area is redefined using SSD. According to the calculated SSD, if it is less than the SSD threshold, it is considered not to be a FOD.

[0086] Step S85: On the luminance component image of size 1024*512, for each square that is considered a foreign object, calculate the SSD of the pixel change value between the square in the target image and the square pointed to by the matching vector in the standard image. If the SSD is less than the threshold calculated by FOD_level, it is considered not a foreign object.

[0087] Step S86: Based on the saturation component image of size 1024*512, find FOD. Using the previously calculated target image grid and the matching grid in the standard image, calculate the SAD of the saturation component images of both. Calculate a threshold for segmenting the SAD distribution to a certain proportion. Based on this threshold and FOD_level, calculate the value to determine whether it is FOD. If it is greater than this value, it is considered to be FOD.

[0088] in, Figure 4 Standard images of car chassis. Figure 5 Input a target image of the car chassis to be detected. Figure 6 For a car chassis image containing matching vectors, Figure 7 Output an image showing the foreign object features on the car chassis in the target image. The location of the foreign object is indicated by the red box in the image. In embodiments of the present invention, the corresponding terms are explained as follows:

[0089] FOD: Foreign Object Detection;

[0090] SSD: Sum of squared differences;

[0091] SAD: Absolute Error and Algorithm;

[0092] HSV: A color space described by hue (H), saturation (S), and value (V).

[0093] The FOD detection algorithm of this invention is highly adaptable to various situations that may occur during actual vehicle under-situ recording, such as distortion, lateral displacement, and changes in lighting conditions. Furthermore, it is not targeted at a specific object and is generally applicable to various scenarios, such as prisons, customs, and stadium entrances / exits. The processing time for each pair of images is less than 1 second, and compared to deep learning detection methods, the detection time does not increase with the number of targets. It can output FOD values ​​of varying degrees at different detection levels.

[0094] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A general method for detecting foreign objects under vehicle images, characterized in that, Including the following steps: S1. Input the target image and the standard image, as well as the FOD_level value; S2. Convert the image color space to HSV; S3. Use bicubic interpolation with a 4x4 pixel neighborhood to scale both the target image and the standard image to four different size levels. S4. Obtain the edges of images at each size level; S5. Fuse the brightness components and edge information of the image according to a certain ratio; S6. Obtain the tire position based on the change information of the longitudinal brightness value and save it to a list; S7. Register target images and standard images at different scales for each size level; S8. Calculate the SSD value that needs to be considered as FOD based on the FOD_level value, determine whether it is FOD based on the SSD value of the pixel change value at intervals, eliminate the wrong identification caused by image differences due to changes in lighting conditions, and search based on the image saturation component to avoid missed detection when the brightness components are similar. S9. Any object that meets either the condition of foreign object detected by saturation component or foreign object detected by brightness edge fusion image is considered a foreign object. S10. Merge adjacent foreign object squares into a rectangle; S11. Output a list of rectangles.

2. The universal method for detecting foreign objects under vehicle images according to claim 1, characterized in that, Step S7 includes: A mapping is established between regions in the target image and regions in the standard image. Matching is performed at different scales, and after registration, the differences between the corresponding regions are compared.

3. The universal method for detecting foreign objects under vehicle images according to claim 2, characterized in that, Step S7 specifically includes the following steps: S71. Register the 256*128 target image and the standard image. Divide the 256*128 target image into 32*16 squares. Use a vector to represent the position of the square in the target image corresponding to the square of the same size in the standard image. The matching vector (x,y) represents the position of the matching square in the standard image after adding (x,y) to the square in the target image. The offset vector (0,0) represents the position of the square in the target image and the matching square in the standard image. S72. Register the 512*256 target image and the standard image, remove the 8-pixel area on each side, and divide the remaining image into 31*63 squares with a square size of 16*16. S73. Register the 1024*512 target image and the standard image.

4. The universal method for detecting foreign objects under a vehicle using images according to claim 1, characterized in that, Step S8 specifically includes the following steps: Step S81: Calculate the SSD threshold using the matching vector and SSD value obtained during registration of the 1024*512 size layer image. Images greater than this threshold are considered to be FOD. Step S82: On the fused image of size 1024*512 that integrates luminance components and edge information, for each square that is considered a foreign object, the matching area is redefined with the matching vector as the center and {10,6} as the search range. According to the calculated SSD, if it is less than the threshold calculated in step S81, it is considered not to be a FOD. Step S83: On the fused image of size 1024*512 that incorporates luminance components and edge information, for each square that is considered a foreign object, calculate the SSD of the pixel change value between the square in the target image and the square pointed to by the matching vector in the standard image. If the SSD is less than the threshold calculated by FOD_level, then it is considered not a foreign object. Step S84: On the luminance component image of size 1024*512, for each square that is considered to be a foreign object, with the matching vector as the center and {10, 6} as the search range, the matching area is redefined using SSD. According to the calculated SSD, if it is less than the SSD threshold, it is considered not to be a FOD. Step S85: On the luminance component image of size 1024*512, for each square that is considered a foreign object, calculate the SSD of the pixel change value between the square in the target image and the square pointed to by the matching vector in the standard image. If the SSD is less than the threshold calculated by FOD_level, it is considered not a foreign object. Step S86: Based on the saturation component image of size 1024*512, find FOD. Use the previously calculated target image grid and the matching grid in the standard image to calculate the SAD of the saturation component images of both. Calculate the threshold for segmenting the SAD distribution to a certain proportion. Calculate the value to determine whether it is FOD based on this threshold and FOD_level. If it is greater than this value, it is considered to be FOD.

Images