Method for locating vehicle logo based on visual saliency and binary map projection segmentation

Abstract

The application discloses a vehicle logo positioning method based on visual saliency and binary map projection segmentation, and comprises the following steps: positioning a license plate, obtaining a vehicle logo rough positioning map according to the positions of four corner points of the license plate; performing vehicle logo accurate positioning on the vehicle logo rough positioning map by using visual saliency and binary map projection segmentation in parallel, and obtaining a candidate vehicle logo vector and a vehicle logo positioning rectangular vector respectively; generating a vehicle logo candidate rectangular vector based on the candidate vehicle logo vector and the vehicle logo positioning rectangular vector; and filtering non-vehicle logos in the vehicle logo candidate rectangular vector by using a vehicle logo-non-vehicle logo sorting model based on deep learning, and generating a vehicle logo rectangular vector. The application performs vehicle logo rough positioning based on visual saliency, then positions the vehicle logo according to an empirical value, simultaneously detects a gray peak value area of a vehicle logo visual saliency map by using a binary map projection segmentation method, supplements a map in which visual saliency positioning fails, overcomes the shortcoming that a single method is difficult to cope with variable environments, and finally filters non-vehicle logos by using a vehicle logo-non-vehicle logo sorting model, so that the success rate of vehicle logo positioning is superior to that of an existing method.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of intelligent transportation and computer vision technology, and particularly relates to a vehicle logo positioning method based on visual saliency and binary graph projection segmentation. BACKGROUND

[0002] At present, the existing technology generally adopts the method of combining edge extraction and morphological processing, based on visual saliency, based on template matching, based on Adaboost and HOG features, or based on CNN and improved HOG features for vehicle logo positioning. It has many shortcomings, such as poor environmental robustness of the existing technology, difficulty in meeting the vehicle logo positioning needs of environmental changes and various types of vehicle logos, single method difficult to have good positioning results for all cases, large time consumption, unable to achieve real-time processing, only capable of processing images of good quality and consuming a lot of time, and the need for a large amount of training data to achieve satisfactory results. SUMMARY

[0003] The technical problem to be solved by the present application is to overcome the shortcomings of the prior art. The vehicle logo positioning method based on visual saliency and binary graph projection segmentation is provided. The vehicle logo is roughly positioned based on visual saliency, and then the vehicle logo is positioned according to the empirical value. At the same time, the binary graph projection segmentation method is used to detect the gray peak value area of the vehicle logo visual saliency graph, supplement the failed graph of visual saliency positioning, overcome the shortcomings of single method difficult to cope with changing environment, and finally use the vehicle logo-non-vehicle logo sorting model for non-vehicle logo filtering. The success rate of vehicle logo positioning is better than that of the existing method, which can provide effective support for vehicle brand identification in intelligent transportation.

[0004] To achieve the above technical purpose, the technical scheme adopted by the present application is as follows:

[0005] The vehicle logo positioning method based on visual saliency and binary graph projection segmentation comprises:

[0006] Step 1: locate the license plate, and obtain the vehicle logo rough positioning graph M1 according to the positions of the four corner points of the license plate;

[0007] Step 2: use visual saliency and binary graph projection segmentation to perform vehicle logo accurate positioning on the vehicle logo rough positioning graph M1, and obtain candidate vehicle logo vector V1' and vehicle logo positioning rectangular vector V2', respectively;

[0008] Step 3: generate vehicle logo candidate rectangular vector V based on candidate vehicle logo vector V1' and vehicle logo positioning rectangular vector V2';

[0009] Step 4: filter the non-vehicle logo in the vehicle logo candidate rectangular vector V using the vehicle logo-non-vehicle logo sorting model based on deep learning, and generate vehicle logo rectangular vector V'.

[0010] To optimize the above technical solutions, the specific measures taken also include:

[0011] The car logo coarse positioning map obtained in step 1 is:

[0012]

[0013]

[0014] carLogoCoarseBRX=PRBRX

[0015] carLogoCoarseBRY=PRTLY

[0016] Wherein, (carLogoCoarseTLX, carLogoCoarseTLY) is the horizontal and vertical coordinates of the top left corner of the car logo coarse positioning rectangle;

[0017] (carLogoCoarseBRX, carLogoCoarseBRY) is the horizontal and vertical coordinates of the bottom right corner of the car logo coarse positioning rectangle;

[0018] (PRTLX, PRTLY) is the horizontal and vertical coordinates of the top left corner of the license plate;

[0019] (PRBRX, PRBRY) is the horizontal and vertical coordinates of the bottom right corner of the license plate;

[0020] PRH is the height of the license plate; SC is the proportionality coefficient.

[0021] The SC above is the ratio of the distance from the center of the car logo to the center of the license plate to the height of the license plate.

[0022] Step 2 above uses visual saliency to perform car logo fine positioning on the car logo coarse positioning map M1 to obtain a candidate car logo vector V1', specifically including:

[0023] Extract the visual saliency map M2 from the car logo coarse positioning map M1, binarize the visual saliency map M2 to obtain a binary map M3, and then perform a closing operation on the binary map M3 to obtain a closed map M4;

[0024] Then extract the contour of the closed map M4 to obtain a car logo rectangle vector V1;

[0025] Filter out the rectangles in the car logo rectangle vector V1 that are not likely to be car logos according to the car logo aspect ratio, position, and car logo area size, and obtain a candidate car logo vector V1' after filtering.

[0026] Step 2 above uses binary map projection segmentation to perform car logo fine positioning on the car logo coarse positioning map M1 to obtain a car logo positioning rectangle vector V2', specifically including:

[0027] (1) First, the vertical texture VE and the horizontal texture HE are extracted from the logo rough positioning image M1 using the SOBEL operator;

[0028] (2) Second, the vertical texture VE and the horizontal texture HE are binarized using a fixed threshold to obtain the horizontal texture binarization image EX and the vertical texture binarization image EY. A 5*5 structure element is used to perform a close operation on the binarization images to obtain the horizontal texture binarization image EX' and the vertical texture binarization image EY' after the close operation;

[0029] (3) Then, the texture direction is determined by comparing EX' and EY'. Different segmentation strategies are used according to the texture direction to segment the logo region. The corresponding logo positioning rectangle vector is denoted as V2;

[0030] (4) The length-width ratio, position, and area of the logo are used to filter the logo positioning rectangle vector V2 to remove rectangles that are unlikely to be logos. After filtering, the final logo positioning rectangle vector V2' is obtained.

[0031] The judgment method for determining the texture direction is as follows:

[0032] If EX' >= EY', it is a horizontal texture, i.e., the texture direction is the X direction;

[0033] If EX' < EY', it is a vertical texture, i.e., the texture direction is the Y direction.

[0034] For the horizontal texture, the segmentation strategy is as follows:

[0035] First, the Y-direction projection image of the horizontal texture is generated. A horizontal bar is used for the first step of segmentation using Y-direction projection. The segmentation method is to find the maximum peak value of the Y-direction texture, then find the two 0 values closest to the maximum peak value, and find the corresponding row number as the upper and lower boundaries of the logo. Based on this, a horizontal bar region containing the logo region is generated;

[0036] Then, the segmented logo horizontal bar is subjected to secondary segmentation. First, the X-direction projection of the first logo segmentation region is calculated, then the maximum peak value of the projection is found, and the column numbers corresponding to the two 0 value points closest to the maximum peak value are found. These two corresponding column numbers correspond to the left and right boundaries of the logo region, based on which the logo region is segmented.

[0037] For the vertical texture, the segmentation strategy is as follows:

[0038] First, the X-direction projection image of the vertical texture is generated. The X-direction projection is used for the first segmentation to generate a vertical bar;

[0039] Then, the first segmentation result is used to perform a second segmentation to find the upper and lower boundaries of the car logo area. The left and right boundaries of the car logo area are obtained based on the segmented vertical bars. Combined with the upper and lower boundaries corresponding to the second segmentation, the rectangular area corresponding to the car logo area is obtained.

[0040] Step 4 above involves batch extracting the vehicle logo and non-vehicle logo regions to establish a vehicle logo-non-vehicle logo database. This database is then used to train a deep learning-based vehicle logo-non-vehicle logo sorting model.

[0041] The trained deep learning-based vehicle logo-non-vehicle logo sorting model is used to sort and filter non-vehicle logos in the candidate rectangular vector V of vehicle logos, generating a rectangular vector V′ of vehicle logos.

[0042] This invention has the following advantages: It adopts a car logo localization method that combines visual saliency and binary image projection segmentation. Visual saliency is used for coarse localization of the car logo, while binary image projection segmentation can effectively detect visually salient regions in the image. However, it is difficult to cope with changing environments and supplements images where visual saliency localization fails. Finally, a car logo-non-car logo sorting model is used to filter non-car logos to obtain the final car logo candidate region. The car logo localization success rate of this invention is better than that of existing methods. Attached Figure Description

[0043] Figure 1 This is a flowchart of the vehicle logo localization method based on visual saliency and binary map projection segmentation according to the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0045] Although the steps in this invention are arranged by reference numerals, this is not intended to limit the order of the steps. Unless the order of the steps is explicitly stated or the execution of a step requires other steps as a basis, the relative order of the steps can be adjusted. It is understood that the term "and / or" as used herein refers to and covers any and all possible combinations of one or more of the associated listed items.

[0046] like Figure 1 As shown, the vehicle logo localization method of the present invention based on visual saliency and binary map projection segmentation includes:

[0047] Step 1: License plate localization: Locate the license plate using a license plate localization algorithm; Coarse logo localization: Obtain the coarse logo localization map M1 based on the positions of the four corner points of the license plate;

[0048] The resulting rough positioning diagram of the car logo is as follows:

[0049]

[0050]

[0051] carLogoCoarseBRX=PRBRX (3)

[0052] carLogoCoarseBRY=PRTLY (4)

[0053] Where (carLogoCoarseTLX, carLogoCoarseTLY) are the coordinates of the upper left corner of the car logo's thick positioning rectangle;

[0054] (carLogoCoarseBRX, carLogoCoarseBRY) are the coordinates of the lower right corner of the car logo's thick positioning rectangle;

[0055] (PRTLX, PRTLY) are the coordinates of the top left corner of the license plate;

[0056] (PRBRX, PRBRY) are the coordinates of the bottom right corner of the license plate;

[0057] PRH is the height of the license plate;

[0058] SC is a proportional coefficient, an empirical value, representing the ratio of the distance from the center of the vehicle logo to the center of the license plate to the height of the license plate. In practice, SC is set to 6 to ensure that the vehicle logo is included in the coarse positioning area.

[0059] Step 2: Precise positioning of the car logo: Visual saliency and binary image projection segmentation are used in parallel to perform precise positioning of the car logo in the coarse positioning image M1, resulting in candidate car logo vector V1′ and car logo positioning rectangle vector V2′.

[0060] Step 2 specifically includes the following steps:

[0061] The precise positioning process for car logos is divided into two parallel parts:

[0062] Car logo positioning based on visual prominence;

[0063] First, visual saliency map M2 is extracted from the coarse positioning map M1 of the car logo. Then, visual saliency map M2 is binarized to obtain M3. Finally, a closing operation is performed on the binarized saliency map to obtain M4, the purpose of which is to connect small areas.

[0064] Then, the outline of M4 is extracted to obtain the car logo rectangular vector V1;

[0065] According to the aspect ratio of the vehicle logo, the area size of the vehicle logo, etc. to filter out rectangles that are impossible to be vehicle logos, such as setting the area filtering principle to be less than 150 pixels as non-vehicle logos, and the aspect ratio > 5 as non-vehicle logos, and the position filtering is to filter out positions that are impossible to be vehicle logos, such as the top area of the coarse positioning area.

[0066] After filtering, a candidate vehicle logo vector V1' based on visual saliency is generated;

[0067] 2, vehicle logo positioning based on binary graph projection segmentation;

[0068] First, use the SOBEL operator to extract the longitudinal texture VE and the horizontal texture HE in the vehicle logo coarse positioning area;

[0069] Secondly, the texture is binarized by a fixed threshold to obtain the horizontal texture binarization graph EX and the longitudinal texture binarization graph EY, and then a 5*5 structure element is used for closing operation on the binarization graph to connect the small areas and prevent them from being disconnected, to obtain EX', EY';

[0070] Then judge the texture direction, the judgment method is as follows:

[0071] If EX' >= EY', it is horizontal texture;

[0072] If EX' < EY', it is vertical texture;

[0073] If EX' ≈ EY', it is mesh texture;

[0074] According to the texture direction, different segmentation strategies are used to segment the vehicle logo area, which is described in detail as follows:

[0075] For horizontal texture:

[0076] First, generate the Y direction projection graph of the horizontal texture, use the Y direction projection to generate horizontal bars for the first step of segmentation, the segmentation method is to find the maximum peak value of the Y direction texture, then find the two 0 values closest to the maximum peak value, find the corresponding row number as the upper and lower boundaries of the vehicle logo, and generate the horizontal bar area containing the vehicle logo area according to this; Then the segmented vehicle logo horizontal bar is subjected to secondary segmentation, first find the X direction projection of the first vehicle logo segmentation area, then find the maximum peak value of the projection, and similarly find the column numbers corresponding to the two 0 value points closest to the peak value, then the two corresponding column numbers correspond to the left and right boundaries of the vehicle logo area, according to which the vehicle logo area can be segmented out.

[0077] It should be noted that if the projection graph does not exist in the column or row direction 0 value, first the projection can be subtracted by the minimum value of the projection column (or projection row), then the left and right or top and bottom 0 values closest to the projection peak are found, and then the left and right or top and bottom boundaries corresponding to the vehicle logo are found.

[0078] For longitudinal texture, the principle is similar:

[0079] First, generate the X direction projection map of the longitudinal texture, use the X direction projection to generate the first segmentation to generate the vertical bar; then the second segmentation is performed on the first segmentation result to find the upper and lower boundaries of the logo area (similarly, first find the Y direction projection of the vertical bar, then find the row number corresponding to the projection peak, and then find the row number corresponding to the two 0 value points closest to the peak, that is, the upper and lower boundaries of the logo area), according to the vertical bar segmented, the left and right boundaries of the logo area are known, and the upper and lower boundaries corresponding to the second segmentation are combined, that is, the rectangular area corresponding to the logo is obtained.

[0080] The logo positioning rectangular vector generated by using the binaryzation projection segmentation is denoted as V2, this method can consider using the n 0 value points closest to the projection peak when performing the second segmentation, so as to increase the detection rate of the logo, and then using the length-width ratio, area and other rules of the logo to filter, and generate the logo positioning rectangular vector V2'.

[0081] Step 3: generating a logo candidate rectangular vector V based on the candidate logo vector V1' and the logo positioning rectangular vector V2'.

[0082] The logo candidate rectangular vector V = V1' + V2'.

[0083] Step 4: using a logo-non logo sorting model based on deep learning to filter the non logo in the logo candidate rectangular vector V, and generating a logo rectangular vector V'.

[0084] The step 4 comprises the following steps:

[0085] A "logo-non logo" library can be established, the logo and non logo area intercepted by a program can be used to build the library, and a logo-non logo sorting model can be constructed using a deep learning algorithm.

[0086] Finally, the "sorting model" is used to filter the non logo in the logo candidate matrix vector V to obtain the final logo rectangular vector V', and the positioning is completed.

[0087] It is apparent to those skilled in the art that the present application is not limited to the details of the foregoing exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended to encompass all changes falling within the meaning and scope of the equivalent elements of the claims. Any reference signs in the claims should not be considered as limiting the claims involved.

[0088] Furthermore, it should be understood that although the specification is described in terms of embodiments, not every embodiment includes every feature or implementation described herein. The specification can include implicit combinations of explicitly mentioned features and / or implicit combinations of implicitly mentioned features. Such combinations are also expressly included within the scope of the specification and an embodiment.

Claims

1. A vehicle logo localization method based on visual saliency and binary map projection segmentation, characterized in that, include: Step 1: Locate the license plate. Based on the positions of the four corner points of the license plate, obtain the rough positioning diagram M1 of the license plate logo. Step 2: Visual saliency and binary image projection segmentation are used in parallel to perform precise positioning of the car logo in the coarse positioning image M1, resulting in candidate car logo vector V1′ and car logo positioning rectangle vector V2′, respectively; Visual saliency is used to perform precise positioning of the car logo in the coarse positioning image M1, resulting in the candidate car logo vector V1′, which specifically includes: Extract the visual saliency map M2 from the coarse positioning map M1 of the car logo, binarize the visual saliency map M2 to obtain the binarized map M3, and then perform a closing operation on the binarized map M3 to obtain the closed map M4. Then, the outline is extracted from the closed image M4 to obtain the car logo rectangular vector V1; Based on the aspect ratio, position, and area size of the car logo, the rectangle vector V1 that cannot be the car logo is filtered out, and the candidate car logo vector V1′ is obtained after filtering. Binary image projection segmentation is used to perform precise positioning of the car logo on the coarse positioning image M1, resulting in the car logo positioning rectangle vector V2′, which specifically includes: (1) First, the SOBEL operator is used to extract the vertical texture VE and the horizontal texture HE from the coarse positioning map M1 of the car logo; (2) Next, the vertical texture VE and the horizontal texture HE are binarized with a fixed threshold to obtain the horizontal texture binarized image EX and the vertical texture binarized image EY. The binarized images are then subjected to a 5*5 structuring element to perform a closing operation to obtain the horizontal texture binarized image EX′ and the vertical texture binarized image EY′ after the closing operation. (3) Then, by comparing EX′ and EY′, the texture direction is determined, and different projection segmentation strategies are used to segment the car logo area according to the texture direction. The corresponding car logo positioning rectangle vector is denoted as V2. (4) Use the aspect ratio, position and area of ​​the car logo to filter the car logo positioning rectangle vector V2. The rectangle in V2 that cannot be the car logo is filtered out. After filtering, the final car logo positioning rectangle vector V2′ is obtained. Step 3: Generate candidate logo rectangle vector V based on candidate logo vector V1′ and logo positioning rectangle vector V2′; Step 4: Use a deep learning-based vehicle logo-non-vehicle logo sorting model to filter out non-vehicle logos in the candidate rectangular vector V of vehicle logos, and generate a rectangular vector V′ of vehicle logos.

2. The vehicle logo localization method based on visual saliency and binary map projection segmentation according to claim 1, characterized in that, The rough positioning image of the car logo obtained in step 1 is as follows: carLogoCoarseBRX=PRBRX carLogoCoarseBRY=PRTLY Where (carLogoCoarseTLX, carLogoCoarseTLY) are the horizontal and vertical coordinates of the upper left corner of the car logo's thick positioning rectangle; (carLogoCoarseBRX, carLogoCoarseBRY) are the horizontal and vertical coordinates of the lower right corner of the car logo's thick positioning rectangle; (PRTLX, PRTLY) are the horizontal and vertical coordinates of the top left corner of the license plate; (PRBRX, PRBRY) are the horizontal and vertical coordinates of the bottom right corner of the license plate; PRH is the height of the license plate; SC is the proportional coefficient.

3. The vehicle logo localization method based on visual saliency and binary map projection segmentation according to claim 2, characterized in that, The SC value is the ratio of the distance from the center of the vehicle logo to the center of the license plate to the height of the license plate.

4. The vehicle logo localization method based on visual saliency and binary map projection segmentation according to claim 1, characterized in that, The method for determining the texture direction is as follows: If EX′ >= EY′, it is horizontal texture, that is, the texture direction is the X direction; If EX′ < EY′, it is vertical texture, that is, the texture direction is the Y direction.

5. The vehicle logo localization method based on visual saliency and binary map projection segmentation according to claim 4, characterized in that, For horizontal texture, the projection segmentation strategy is as follows: First, generate the Y-direction projection map of the horizontal texture. Use the Y-direction projection to generate horizontal bars for the first segmentation. The segmentation method is to find the maximum peak of the Y-direction texture, and then find the two 0 values closest to the maximum peak along the maximum peak. The corresponding row numbers found are the upper and lower boundaries of the logo, and based on this, a horizontal bar area containing the logo area is generated; Then, perform secondary segmentation on the segmented logo horizontal bar. First, find the X-direction projection of the first logo segmentation area, then find the maximum peak of this projection, and find the column numbers corresponding to the two 0 value points closest to the maximum peak. These two corresponding column numbers respectively correspond to the left and right boundaries of the logo area, and based on this, the logo area is segmented.

6. The vehicle logo localization method based on visual saliency and binary map projection segmentation according to claim 4, characterized in that, For vertical texture, the projection segmentation strategy is as follows: First, generate the X-direction projection map of the vertical texture, and use the X-direction projection for the first segmentation to generate vertical bars; Then, perform secondary segmentation on the result of the first segmentation to find the upper and lower boundaries of the logo area, obtain the left and right boundaries of the logo area based on the segmented vertical bars, and combine the upper and lower boundaries corresponding to the secondary segmentation to obtain the rectangular area corresponding to the logo area.

7. The vehicle logo localization method based on visual saliency and binary map projection segmentation according to claim 1, characterized in that, In step 4, the logo and non-logo areas are batch intercepted to establish a logo-non-logo library, and a logo-non-logo sorting model based on deep learning is trained using the logo-non-logo library; The non-logos in the logo candidate rectangular vector V are sorted and filtered using the trained logo-non-logo sorting model based on deep learning to generate a logo rectangular vector V′.

Images