A method for identifying an automobile dashboard

Abstract

A kind of automobile instrument panel identification method, the device includes support frame, area light source, code scanner, camera, photoelectric switch, industrial computer and electrical control cabinet;The area light source, code scanner, camera, photoelectric switch are arranged in support frame;The industrial computer is electrically connected with camera and inventory management system;The present application can realize for each hanging of hanging chain has no instrument panel judgment, further, it can also be respectively the type and color of instrument panel, greatly reduce artificial workload.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and in particular to a method for recognizing automotive dashboards. Background Technology

[0002] Currently, major automotive interior manufacturers often use rigid suspension chains to store and manage finished automotive dashboards that have completed production and assembly but have not yet left the warehouse. Rigid suspension chains, in conjunction with pre-set tracks and motors, can easily move the products. Before leaving the warehouse, the products are categorized and counted. Currently, three main methods are used: manual labor, photoelectric switches, or a combination of photoelectric switches and barcode scanners to count the type, color, and quantity of dashboards on the suspension chain. However, method one has drawbacks: poor real-time data updates and a large manual workload; method two only checks whether the suspension chain has products, but cannot further identify the type and color information; method three requires barcodes or QR codes containing product color and type information to be affixed to specific locations, and the barcode scanner needs to be aligned with the barcode or QR code, increasing production steps and the difficulty of suspending the suspension chain. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for recognizing automotive dashboards that is simple in structure and easy to use.

[0004] A rigid suspension chain automotive dashboard recognition device includes a support frame, a surface light source, a barcode scanner, a camera, a photoelectric switch, an industrial computer, and an electrical control cabinet; the surface light source, barcode scanner, camera, and photoelectric switch are mounted on the support frame; the industrial computer is electrically connected to the camera and an inventory management system.

[0005] Furthermore, the support frame includes a crossbeam, vertical rods, and a horizontal bar; there are two vertical rods, which are arranged parallel to each other on both sides of the detection point; the horizontal beam is arranged between the vertical rods; there are four surface light sources, arranged in pairs, each on one of the two vertical rods; the horizontal bar is arranged between two surface light sources on the same vertical rod; four cameras are arranged at both ends of the horizontal bar; a photoelectric switch is arranged on the horizontal beam, and a straight rod is arranged between the photoelectric switch and the horizontal beam; a barcode scanner is arranged on the vertical rod, and a straight rod is arranged between the barcode scanner and the vertical rod; the photoelectric switch and the barcode scanner are respectively arranged on both sides of the hanger.

[0006] Furthermore, the horizontal bar is perpendicular to the vertical bar and the beam, and the middle part of the horizontal bar is fixedly connected to the vertical bar; the camera includes a camera for capturing product type and a camera for capturing product color; the four cameras form a quadrilateral with vertices, and the types of cameras on the same side are different, with the same cameras set on the diagonal.

[0007] A method for recognizing a car dashboard with a rigid suspension chain, the method being based on the aforementioned car dashboard recognition device with a rigid suspension chain, the method including human-machine interaction operation; the human-machine interaction operation includes the following steps:

[0008] Step 1: Receive the power-on command to open the software; the system will enter the initial interface.

[0009] Step 2: The system receives the initialization command and enters the initialization interface;

[0010] Step 3: The system receives the parameter setting command, enters the parameter setting interface, receives the parameter data, receives the OK button, and returns to the initialization interface;

[0011] Step 4: The system receives the detection command and enters the detection process;

[0012] Step 5: The system receives the exit command and closes the software.

[0013] Furthermore, the detection process includes the following steps:

[0014] Step 101: The photoelectric switch is triggered by the hanger passing through the support frame, and the photoelectric switch wakes up the four cameras to collect one frame of image respectively. The cameras transmit the collected images to the industrial control computer; the code reader reads the serial number information of the hanger passing through the support frame and transmits it to the industrial control computer.

[0015] Step 102: The industrial control computer identifies the image sent by the camera and first performs an empty hanger detection on the image; if it is an empty hanger, the industrial control computer summarizes the hanger serial number and hanger image, uploads the summarized content to the inventory management system, and the recognition process of the hanger ends; if it is not an empty hanger, a single hanger detection is performed.

[0016] Step 103: If the hanger is a single hanger, further determine whether the hanger is a left hanger; if the hanger is a full hanger, save the full hanger determination result and perform product type detection and product color detection, and proceed to step 105.

[0017] Step 104: If it is further determined to be a left-side hanging product, save the determination result and perform product type detection and product color detection to identify the type and color of the product on the left; otherwise, it is a right-side hanging product, save the determination result and perform product type detection and product color detection to identify the type and color of the product on the right.

[0018] Step 105: After the inspection is completed, the industrial control computer summarizes the fixture serial number, fixture image, product type and color, and uploads it to the inventory management system. The identification process of the fixture is then complete.

[0019] If the industrial control computer fails to recognize the image during the testing process, the system will copy the unrecognized product image to the unrecognized folder.

[0020] Furthermore, the detection steps for the empty-load detection include:

[0021] Step 201: Extract the ROI (Region of Interest) for product type detection based on the extracted coordinate location information;

[0022] Step 202: Convert the ROI to a grayscale image;

[0023] Step 203: Loop through all pixel values ​​in the ROI and count the number N of pixels whose values ​​are greater than the set grayscale value;

[0024] Step 204: Determine whether the number N is greater than the set value of the total number of pixels;

[0025] Step 205: End the testing process.

[0026] Furthermore, the detection steps for the single-attachment detection include:

[0027] Step 301: First, perform mean filtering on the ROI of the product type detection area; then perform Sober edge detection; then perform thresholding; finally, use Hough transform to find and save the line segments with slopes greater than the set threshold.

[0028] Step 302: Traverse the saved set of line segments, calculate and find the longest line segment, and determine whether its length is greater than the set threshold; if yes, it is determined that there is no product hanging on this side; if no, it is determined that there is a product hanging on this side.

[0029] Step 303: End the testing process.

[0030] Furthermore, the product type detection process includes extracting feature points from the ROI region converted to grayscale. The steps for extracting feature points include:

[0031] Step 401: Select a point P from the ROI, and draw a circle with P as the center and a radius r of 3 pixels; if there are n consecutive pixels whose absolute difference in grayscale value from point P is greater than a set value t, then P is considered a feature point;

[0032] Step 402: Use the ID3 algorithm to select the pixel with the largest pixel gain as the basis for judging the feature point, generate a decision tree, input the pixels on the circumference of each feature point as features into the decision tree, and select the optimal feature point based on the features.

[0033] Step 403: Construct a scoring function V; compare the V values ​​of any two adjacent feature points and delete the feature points with lower V values;

[0034] Step 404: Calculate the centroid of the feature point within a radius of r using moments, and take the vector formed by the coordinates of the feature point to the centroid as the direction of the feature point; the definition of a moment is as follows:

[0035] m pq =∑ x，y∈r x P y q I(x, y) (1)

[0036] Where, m pq Let I(x, y) represent the geometric moment of order p+q, where I(x, y) is the expression for the image gray level; the centroid C of this moment is:

[0037]

[0038] Where, m 00 This represents the zeroth geometric moment when p = 0 and q = 0. Similarly, m 10 Let m represent the first-order geometric moment when p = 1 and q = 0. 01 This represents the first-order geometric moment when p = 0 and q = 1;

[0039] Taking the coordinates of the feature point as the origin, the angle of the vector is the direction of that feature point; the formula for calculating the angle θ of the vector is as follows:

[0040]

[0041] In step 401, the gray values ​​at positions 1 and 9 on the circle are checked first. If the difference between the gray value of at least one pixel at these two positions and the gray value of point P is greater than t, then the gray values ​​at positions 5 and 13 on the circle are checked. If the difference between the gray values ​​of the pixels at positions 1 and 9 and the gray value of point P is not greater than t, then the point is determined to be a non-feature point.

[0042] In step 403, the scoring function V is defined as the sum of the absolute values ​​of the differences between the grayscale values ​​of pixel P and its 16 surrounding pixels.

[0043] Furthermore, the product type detection process also includes feature point description and feature matching; the feature point description uses the rBRIEF algorithm, and the specific description steps include:

[0044] Step 501: For each feature point, select a 31*31 neighborhood centered on the feature point. Within each neighborhood, select the average gray value of a 5*5 sub-neighborhood and compare it with the gray value of the pixel at the center of the 5*5 sub-neighborhood. This yields n = (31-5+1)*(31-5+1) sub-neighborhoods, resulting in a total of m = C(n, 2) point pairs, where C(n, 2) represents the numerical values ​​of the permutations and combinations. Form a matrix Q from the point pairs of each neighborhood. Each column vector of matrix Q (a i b i ), 1≤i≤m represents a pair of points:

[0045]

[0046] Rotate matrix Q using the direction θ of the feature points to obtain the rotated matrix Q. θ Q θ =R θ Q;R θ Represented as:

[0047]

[0048] According to equation (6), for matrix Q θ Compare the grayscale values ​​of each point in the image.

[0049]

[0050] Where p(a) represents the gray value at point a in a pair of points, p(b) represents the gray value at point b in a pair of points, and τ(p; a, b) represents converting the gray values ​​of the pair of points into binary values.

[0051] For m pairs of points, we can obtain an initial binary string descriptor f of length m. m (p):

[0052] f m (p)=∑ 1≤i≤m 2 i-1 τ(p;a i b i (7)

[0053] By combining the orientation of feature points, a directional descriptor g can be obtained. m (p, θ):

[0054] g m (p, θ) = f m (p)|(a i b i )∈Q θ (8)

[0055] Step 502: Select k point pairs from m point pairs. The selection method includes the following steps:

[0056] Step 5021: Reassemble all feature points and their corresponding initial binary strings to obtain matrix W, where the number of rows corresponds to the number of extracted feature points, and each row contains the initial binary string descriptor f for each feature point. m (p);

[0057] Step 5022: Sort the recombined matrix W in ascending order according to the absolute difference between the mean of each column and 0.5, to obtain matrix T;

[0058] Step 5023: Greedy Choice:

[0059] Place the first column of matrix T into matrix U, and then remove that column from matrix T. Matrix U is initially empty.

[0060] Each column of matrix T is selected sequentially and compared with all columns in matrix U. If the similarity exceeds a set threshold, it is ignored and the next column is moved on. Otherwise, the column in matrix T is added to matrix U and removed from matrix T.

[0061] Repeat the above process until k columns are selected; if the matrix U obtained by traversing matrix T has less than k columns, lower the threshold until matrix U has k columns.

[0062] The feature matching method includes: calculating the Hamming distance between feature points in two images to obtain the similarity distance between feature points; finding the closest and second closest feature points in the image to be matched to the template image in terms of Hamming distance; comparing the ratio of the closest distance to the second closest distance with a set threshold; if the ratio is less than the threshold, then the point is a matching point; finding all matching points in the image to be matched and counting the number of matching points.

[0063] Based on the matching results of feature points, determine whether the product belongs to a certain type or is unknown; if the maximum number of matching points is greater than the set threshold, the matching is considered successful and the product belongs to a certain type; if the maximum number of matching points is not greater than the set threshold, the matching is considered unsuccessful and the product belongs to an unknown type.

[0064] Furthermore, the product color detection step includes:

[0065] Step 601: Extract the ROI of the product color detection area;

[0066] Step 602: Perform color space conversion on the ROI. The conversion formula is as follows:

[0067] max = max(R, G, B);

[0068] min = min(R, G, B);

[0069] V = max / 255;

[0070] If max ≠ 0, then S = (max - min) / max;

[0071] If max = 0, then S = 0;

[0072] If R = max, then H = (GB) / (max - min) * 60

[0073] If G = max, then H = 120 + (BR) / (max - min) * 60

[0074] If B = max, then H = 240 + (RG) / (max - min) * 60

[0075] If H < 0, then H = H + 360;

[0076] After conversion, the ranges of H, S, and V are 0≤H≤360, 0≤S≤1, and 0≤V≤1, respectively.

[0077] Step 603: Divide the ROI into equal regions of the set value;

[0078] Step 604: Identify the color of each equal region based on the different threshold ranges of the HSV corresponding to different colors;

[0079] Step 605: Statistically analyze the color recognition results of each region, and determine the product color based on the region with the most regions that are identified as a certain color;

[0080] Step 606: End product color detection;

[0081] The method for recognizing car dashboards also includes creating templates for images in the unrecognized folder, the template creation comprising the following steps:

[0082] Step 701: First, take a saved but unrecognized product type image, manually determine and save the coordinate position information of the product type detection area ROI;

[0083] Step 702: Next, based on the determined ROI coordinate location information, extract the ROI from each image in a certain sample set of images collected from the saved and unidentified product type images, and convert them into grayscale images.

[0084] Step 703: Then extract and describe the feature points for each ROI, and save the feature point description matrix of each feature point in a container;

[0085] Step 704: Then, traverse the container, perform feature matching on two different feature point description matrices each time, and remove one of the matrices that does not meet the threshold for the number of matching points; after the traversal, the remaining feature point description matrices in the container are the templates.

[0086] Step 705: End template creation.

[0087] The beneficial effects of using the present invention are as follows:

[0088] This invention, through the cooperation of devices, enables the determination of whether each hanging device on the suspension chain has an instrument panel. Furthermore, it can also distinguish the type and color of the instrument panel.

[0089] The industrial control computer of this invention interfaces with the inventory management system, which can update various dashboard data in real time, facilitating inventory statistics and analysis and enabling timely responses.

[0090] The hangers are allowed to have a wide range of limits, and there is no need to strictly require the hangers to be aligned. The hangers are allowed to rotate within a certain angle.

[0091] This invention significantly reduces manual workload and lowers labor costs through image recognition technology. Attached Figure Description

[0092] Figure 1 This is a schematic diagram of the detection device of the present invention;

[0093] Figure 2 This is a flowchart of the human-computer interaction interface operation of the present invention;

[0094] Figure 3 This is the overall testing flowchart of the present invention;

[0095] Figure 4 This is a flowchart of the empty-load detection process of the present invention;

[0096] Figure 5 This is a flowchart illustrating the process of whether or not a product is hung on one side in this invention.

[0097] Figure 6 This is a flowchart illustrating the product type identification process for this invention.

[0098] Figure 7 This invention provides a flowchart for identifying product colors.

[0099] Figure 8 A flowchart for creating a template of this invention type. Detailed Implementation

[0100] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0101] Example 1:

[0102] like Figure 1As shown, a rigid suspension chain automotive dashboard recognition device includes a support frame 12, surface light sources (2, 5, 7, 10), a barcode scanner 1, cameras (3, 4, 8, 9), a photoelectric switch 6, an industrial control computer 14, and an electrical control cabinet 15. The support frame 12 has a U-shaped structure and includes vertical rods and a crossbeam. There are two vertical rods, which are parallel to each other on both sides of the detection point. The crossbeam is located between the two vertical rods. A track is also provided within the quadrilateral enclosed space formed by the two vertical rods, the crossbeam, and the bottom surface. The direction of the track is perpendicular to the crossbeam. A hanger 13 is mounted on the track and can move on it. Both sides of the hanger 13 can be used to mount the automotive dashboard. There are four surface light sources, arranged in groups of two, with each of the four surface light sources mounted on one of the two vertical rods. The support frame 12 also includes a crossbeam, which is perpendicular to the plane formed by the vertical rods and the crossbeam. The crossbeam is located between two surface light sources on the same vertical rod, and the middle part of the crossbeam is fixedly connected to the vertical rod. The cameras include cameras for capturing product type and cameras for capturing product color. Cameras (3, 9) are used to capture product type images, and cameras (4, 8) are used to capture product color images. There are four cameras in total, located at both ends of the horizontal bar, forming a quadrilateral with the four cameras as vertices. Cameras of different types are on the same side, and identical cameras are located on opposite diagonals. The photoelectric switch 6 is located on the horizontal beam, and a straight rod is installed between the photoelectric switch 6 and the horizontal beam. The straight rod is located in the plane formed by the vertical bar and the horizontal beam, perpendicular to the horizontal beam, and located below the horizontal beam. The photoelectric switch 6, located on the straight rod, faces the hanger 13 to be detected, and can identify whether the hanger has passed by. The barcode scanner 1 and the photoelectric switch 6 are respectively located on both sides of the hanger. The barcode scanner 1 is located on the vertical bar, and a second straight rod is installed between the barcode scanner 1 and the vertical bar. The second straight rod is perpendicular to the vertical bar and located in the plane formed by the vertical bar and the horizontal beam. The industrial control computer 14 is electrically connected to the camera and also electrically connected to the inventory management system; the electrical control cabinet 15 is electrically connected to each electrical module in this device, and is used to convert the 220V mains power into the voltage and current required by each electrical module and transmit it to each electrical module.

[0103] During the implementation of the above-mentioned device, a photoelectric switch first identifies whether a hanger has passed by. If a hanger has passed by, a barcode scanner is triggered to read the barcode information on the hanger and a camera is triggered to capture an image, which is then transmitted to the industrial control computer. The industrial control computer receives and processes the captured product type image, product color image, and hanger serial number information, summarizes the identification results, and uploads them to the server.

[0104] like Figure 2 As shown, a method for recognizing a car dashboard based on a rigid suspension chain using the aforementioned device includes operating a human-machine interface. The operation steps of the human-machine interface include:

[0105] Step 1: Open the software; the system enters the initial interface. The initial interface includes an initialization button and an exit button. Clicking the initialization button sends an initialization command and enters the initialization interface; clicking the exit button sends an exit command and closes the software.

[0106] Step Two: The system receives the initialization command and enters the initialization interface. Entering the initialization interface is accompanied by an initialization process, which includes clearing the system cache and opening the camera and initializing camera parameters. The initialization interface includes a settings button, a detection button, a template creation button, and an exit button. Clicking the settings button sends a settings command, and the system enters the parameter settings interface, where parameters can be manually set. The parameter settings interface includes an confirm button and a cancel button. Clicking the confirm button results in successful parameter setting and returns to the initialization interface; clicking the cancel button results in failed parameter setting and returns to the initialization interface. Clicking the detection button enters the detection process; clicking the template creation button enters the template creation interface; clicking the exit button exits and closes the software.

[0107] Step 3: The system receives the parameter setting command, enters the parameter setting interface, receives the parameter data, receives the OK button, and returns to the initialization interface.

[0108] Step 4: The system receives the detection command and enters the detection process.

[0109] Step 5: The system receives the exit command and closes the software.

[0110] like Figure 3As shown, the detection process begins with a photoelectric switch determining whether a hanger has passed the support frame. If the hanger passes the support frame, the photoelectric switch triggers four cameras to each capture an image frame. Each camera uploads the captured image to the industrial control computer via a network port. When the hanger passes the support frame, a barcode reader reads the serial number information on the hanger and sends it to the industrial control computer. The industrial control computer receives the hanger signal sent by the barcode reader and the image sent by the camera and saves it. After saving, the industrial control computer begins to identify the images sent by the cameras. First, an empty hanger detection is performed on the image to determine if it is an empty hanger. An empty hanger is one where neither side of the hanger has a dashboard; correspondingly, a single hanger means that one side of the hanger has a dashboard; a full hanger means that both sides of the hanger have dashboards. If it is an empty hanger, the industrial control computer summarizes the hanger serial number and hanger image, uploads the summarized content to the inventory management system, and the identification process for that hanger ends. If it is not an empty hanger, a single hanger detection is further performed to determine if it is a single hanger. If it is a single hanger, it is further determined whether it is a left hanger. If the product is mounted on the left, the system saves the judgment result and performs product type and color detection to identify the type and color of the product on the left. If not, it's mounted on the right, saving the judgment result and performing product type and color detection to identify the type and color of the product on the right. If it's not a single mount, it's a full mount, saving the judgment result and performing product type and color detection to identify the type and color of the products on both sides. The industrial control computer summarizes the mount serial number, mount image, product type, and color, and uploads it to the inventory management system, ending the recognition process for that mount. For product images not recognized by the industrial control computer, the system copies the unrecognized product images to the unrecognized folder. Clicking the template creation button in the initialization interface allows access to the template creation interface, where templates can be created for new product images or images of unrecognized products in the unrecognized folder, enhancing the device's recognition capabilities.

[0111] like Figure 4 As shown, the detection steps for the empty vehicle detection include:

[0112] Step 101: For the already saved images of the left and right products, extract the product type detection region (ROI) based on the extracted coordinate position information.

[0113] Step 102: Convert the ROI to a grayscale image. Specifically, converting the image to grayscale involves converting the 24-bit image to an 8-bit 3-channel image, and then converting the 8-bit 3-channel image to a single-channel 256-color image.

[0114] Step 103: Iterate through all pixel values ​​in the ROI again and count the number N of pixels whose values ​​are greater than the set grayscale value.

[0115] Step 104: Determine whether the number N is greater than the set value of the total number of pixels. In this embodiment, the set value is 60% of the total number of pixels. If it is greater than the set value of the total number of pixels, it is determined to be an empty frame; otherwise, it is determined to be a non-empty frame.

[0116] Step 105: End the testing process.

[0117] like Figure 5 As shown, the detection steps for the single-mount detection include:

[0118] Step 201: First, perform mean filtering on the product type detection area ROI converted to grayscale image in the empty hanging detection; then perform Sober edge detection; then perform thresholding; finally, use Hough transform to find and save the line segments with slope greater than the set threshold.

[0119] Step 202: Traverse the saved set of line segments, calculate and find the longest line segment, and determine whether its length is greater than a set threshold. If yes, it is determined that there are no products hanging on that side; otherwise, it is determined that there are products hanging on that side.

[0120] Step 203: End the testing process.

[0121] The mean filtering method involves taking the target pixel as the center, combining it with eight pixels surrounding the target pixel to form a filtering template, and then replacing the target pixel value with the average value of all pixels in the template.

[0122] The Sober edge detection process includes:

[0123] (1) The matrix representation of the pixel values ​​of the detection region ROI is matrix A;

[0124] (2) Find the derivatives of matrix A in the horizontal direction (x) and the vertical direction (y) respectively.

[0125] In the horizontal direction, A is convolved with a kernel of size 3 to obtain the horizontal derivative G. x The specific calculation is shown in formula (1):

[0126]

[0127] Convolve A with a kernel of size 3 in the vertical direction to obtain the vertical derivative G. y The specific calculation is shown in formula (2):

[0128]

[0129] The gray values ​​of each pixel in the image in the horizontal and vertical directions are combined using formula (3) to calculate the gray value G of that point.

[0130]

[0131] The thresholding process determines the optimal grayscale threshold k of the grayscale image using the maximum inter-class variance method. * , using k * -1 is used for thresholding grayscale images, and values ​​greater than k are used for thresholding. * Pixels with a grayscale value of -1 are set to 255, and all other pixels have a grayscale value of 0. After thresholding, the grayscale image becomes a black and white image, where a grayscale value of 0 represents black and a grayscale value of 255 represents white.

[0132] The optimal grayscale threshold k is obtained by the maximum inter-class variance method. * As shown in the following formula:

[0133]

[0134] Where k represents the assumed grayscale threshold. This represents the variance between classes.

[0135] The specific implementation ideas of the maximum inter-class variance method include:

[0136] First, the grayscale image to be detected has 256 grayscale value levels, represented as [1, 2, ..., 256]. The number of pixels with grayscale value i is represented as n. i The total number of pixels is denoted as N.

[0137] The probability distribution of gray values ​​in an image is represented by a normalization method, as shown in formula (4):

[0138]

[0139] Where p i This represents the proportion of pixels with a grayscale value of i out of the total number of pixels.

[0140] Suppose that a grayscale threshold k is used to divide the pixels in an image into two classes: C0 and C1; C0 represents pixels with grayscale values ​​[1, 2, ..., k], and C1 represents pixels with grayscale values ​​[k+1, ..., 256]. Then, the probability of each class and the average grayscale level of each class are given by the following formulas:

[0141]

[0142]

[0143]

[0144]

[0145] Where ω0 represents the pixel belonging to C0, ω1 represents the probability of the pixel belonging to C1, ω(k) represents the probability that the gray value of the pixel belongs to [1, 2, ..., k], μ0 represents the average gray value of the pixel set C0, μ1 represents the average gray value of the pixel set C1, and μ(k) represents the sum of the gray values ​​of the pixels whose gray values ​​belong to [1, 2, ..., k]. The expressions for ω(k) and μ(k) are shown in equations (9) and (10).

[0146]

[0147]

[0148] in addition

[0149]

[0150] μ T This represents the average gray value of the image. Also, for any chosen k, we have:

[0151] ω0μ0+ω1μ1=μ T , ω0+ω1=1. (12)

[0152] η is selected as the measurement standard for evaluating the separability of k as the threshold. The value of η can be obtained from equation (13):

[0153]

[0154] in:

[0155]

[0156] in Let C0 and C1 represent the inter-class variance of their grayscale values. Combining equation (12), we can derive:

[0157]

[0158] in The total variance of the image grayscale values.

[0159] By sequentially searching with different values ​​of k, the optimal threshold k is found according to equations (9) and (10). * Making η(k) reach its maximum value, or equivalently making It has reached its maximum value.

[0160]

[0161]

[0162] Where η(k) represents the value of η corresponding to different values ​​of k. This indicates that different k values ​​correspond to Value. Optimal threshold k * satisfy:

[0163]

[0164] The implementation idea of ​​finding line segments using the Hough line transform is as follows:

[0165] First, the straight line is represented by the polar coordinate system. The expression of the straight line is as shown in equation (19), where r represents the polar radius and θ represents the polar angle.

[0166]

[0167] Simplifying, we get:

[0168] r = x cosθ + y sinθ (20)

[0169] Next, for a given point (x0, y0), a family of lines passing through that point can be uniformly defined as:

[0170] r θ =x0 cosθ + y0 sin (21)

[0171] This means that every pair (r) θ ,θ) represents a straight line passing through the point (x0, y0).

[0172] For a given invariant point (x0, y0), r θ As θ changes, a sine curve is obtained by plotting all straight lines passing through the point (x0, y0) in the polar coordinate plane relative to the polar radius and polar angle. The horizontal axis of the polar coordinate plane relative to the polar radius and polar angle is θ, and the vertical axis is r, which is the plane θ-r mentioned below.

[0173] The above operation can be performed on all points in the image. If the curves obtained after performing the above operation on two different points intersect in the plane θ-r, then the two different points pass through the same straight line.

[0174] The Hough line transform aims to trace the intersection points of curves in the plane θ-r corresponding to each point in the image. If the number of curves intersecting at a single point exceeds a certain threshold, then the parameter pair represented by this intersection point can be considered as a straight line in the original image. The straight line can then be obtained by reverse calculation using formula (19).

[0175] like Figure 6 As shown, the product type detection process includes extracting feature points from the ROI region converted to grayscale. The steps for extracting feature points include:

[0176] Step 301: Select a point P from the ROI, and draw a circle with radius r of 3 pixels centered at P. There are 16 pixels on the circumference of this circle. If the absolute value of the difference between the grayscale values ​​of n consecutive pixels and point P is greater than a set value t, then P is considered a feature point. In this embodiment, n is set to 12. To accelerate feature point extraction and quickly eliminate non-feature points, first check the grayscale values ​​of the pixels at positions 1 and 9 on the circle. If at least one pixel at these two positions has a grayscale value greater than t in absolute terms of the difference between its grayscale value and that of point P, then continue checking the grayscale values ​​of the pixels at positions 5 and 13 on the circle. If the grayscale values ​​of the pixels at positions 1 and 9 do not both have a grayscale value greater than t in absolute terms of the difference between their grayscale values ​​and that of point P, then this point is determined to be a non-feature point. This is because if point P is a feature point, then at least three of the pixels at positions 1, 5, 9, and 13 on the circle must have a grayscale value greater than t in absolute terms of the difference between their grayscale values ​​and that of point P.

[0177] Step 302: Use the ID3 algorithm to select the pixel with the largest pixel gain as the basis for judging the feature point, generate a decision tree, and input the 16 pixels on the circumference of each feature point as 16 features into the decision tree, and select the optimal feature point based on the features. There are 16 input features for the ID3 algorithm, and the state of each input feature is greater than t, less than t, or equal to t.

[0178] Step 303: Construct a scoring function V for all feature points obtained in step two. V is defined as the sum of the absolute values ​​of the differences in grayscale values ​​between pixel P and its 16 surrounding pixels. Compare the V values ​​of any two adjacent feature points and delete the feature point with the lower V value.

[0179] Step 304: Calculate the centroid of the feature point within a radius of r using moments, and take the vector formed by the coordinates of the feature point to the centroid as the direction of the feature point. The definition of a moment is as follows:

[0180] m pq =∑ x，y∈r x P y q I(x, y) (21)

[0181] Where I(x, y) is the image grayscale expression, m pq Let m represent the geometric moment of order p+q. 00 This represents the zeroth geometric moment when p = 0 and q = 0. Similarly, m 10 Let m represent the first-order geometric moment when p = 1 and q = 0. 01 Let C represent the first-order geometric moment when p = 0 and q = 1. The centroid C of this moment is:

[0182]

[0183] Taking the coordinates of the feature point as the origin, the angle of the vector is the direction of that feature point. The formula for calculating the angle θ of the vector is as follows:

[0184]

[0185] After extracting the feature points, the next step is to describe them using the rBRIEF algorithm. The specific steps include:

[0186] Step 401: For each feature point, select a 31*31 neighborhood centered on the feature point. Within each neighborhood, select the average gray value of a 5*5 sub-neighborhood to replace the gray value of the pixel at the center of the 5*5 sub-neighborhood for comparison. Excluding pixels at the neighborhood edges, n = (31-5+1)*(31-5+1) comparable sub-neighborhoods are obtained, resulting in a total of m = C(n, 2) point pairs, where C(n, 2) represents the numerical value of the permutation and combination. Form a matrix Q from the point pairs of each neighborhood. Each column vector of matrix Q (a... i b i ), where 1≤i≤m represents a pair of points.

[0187]

[0188] To ensure the rotation invariance of the feature point description, the matrix Q is rotated using the feature point orientation θ, and this is achieved through the rotation matrix R. θ The rotated matrix Q is obtained. θ Q θ =R θ Q;R θ Represented as:

[0189]

[0190] According to equation (26), for matrix Q θ Compare the grayscale values ​​of each point in the image.

[0191]

[0192] Where p(a) represents the gray value at point a in a pair of points, p(b) represents the gray value at point b in a pair of points, and τ(p; a, b) represents converting the gray values ​​of the pair of points into binary values.

[0193] For m pairs of points, we can obtain an initial binary string descriptor f of length m. m (p):

[0194] f m (p)=∑ 1≤i≤m 2 i-1 τ(p;ai b i (27)

[0195] By combining the orientation of feature points, a directional descriptor g can be obtained. m (p, θ):

[0196] g m (p, θ) = f m (p)|(a i b i )∈Q θ (28)

[0197] Step 402: Select k point pairs from m point pairs. The selection method uses the principle of maximizing the mean and the variance between points, and includes the following steps:

[0198] (a) Reassemble all feature points and their corresponding initial binary strings to obtain matrix W, where the number of rows represents the number of extracted feature points, and each row contains the initial binary string descriptor f for each feature point. m (p);

[0199] (b) Sort the recombined matrix W in ascending order of the absolute difference between the mean of each column and 0.5 to obtain matrix T;

[0200] (c) Greedy choice:

[0201] Place the first column of matrix T into matrix U, and then remove that column from matrix T. Matrix U is initially empty.

[0202] Each column of matrix T is selected sequentially and compared with all columns in matrix U. If the similarity exceeds a set threshold, it is ignored and the next column is moved on. Otherwise, the column in matrix T is added to matrix U and removed from matrix T.

[0203] Repeat the above process until k columns are selected, so that each feature point has k descriptors consisting of 0s and 1s. If the matrix U obtained by traversing matrix T has fewer than k columns, lower the threshold until matrix U has k columns. Matrix U is the final feature description matrix.

[0204] Finally, feature matching is performed, using the following method:

[0205] Calculate the Hamming distance between feature points in two images to obtain the similarity distance between feature points. In the image to be matched, find the feature points with the closest and second closest Hamming distance to the template image. Compare the ratio of the closest distance to the second closest distance with a set threshold. If the ratio is less than the threshold, the point is a matching point. Following the above feature matching method, find all matching points in the image to be matched and count the number of matching points.

[0206] Based on the matching results of feature points, the product type is determined to be either unknown or undefined. If the maximum number of matching points is greater than a set threshold, the match is considered successful, and the product type is determined to be a certain type; if the maximum number of matching points is not greater than the set threshold, the match is considered unsuccessful, and the product type is determined to be unknown.

[0207] like Figure 7 As shown, the product color detection steps include:

[0208] Step 501: Extract the ROI of the product color detection area;

[0209] Step 502: Perform color space conversion on the ROI, that is, convert the 24-bit three-channel image from the RGB color space to the HSV color space. In the RGB color space, component R represents red, component G represents green, and component B represents blue. In the HSV color space, component H represents hue, component S represents saturation, and component V represents lightness. Where 0≤R≤255, 0≤G≤255, and 0≤B≤255, the conversion formula is as follows:

[0210] max = max(R, G, B);

[0211] min = min(R, G, B);

[0212] V = max / 255;

[0213] If max ≠ 0, then S = (max - min) / max;

[0214] If max = 0, then S = 0;

[0215] If R = max, then H = (GB) / (max - min) * 60

[0216] If G = max, then H = 120 + (BR) / (max - min) * 60

[0217] If B = max, then H = 240 + (RG) / (max - min) * 60

[0218] If H < 0, then H = H + 360;

[0219] After conversion, the ranges of H, S, and V are 0≤H≤360, 0≤S≤1, and 0≤V≤1, respectively.

[0220] Step 503: Divide the ROI into equal regions of the set value;

[0221] Step 504: Identify the color of each equal region based on the different threshold ranges of the HSV corresponding to different colors;

[0222] Step 505: Statistically analyze the color recognition results of each region, and determine the product color based on the region with the most regions that are identified as a certain color;

[0223] Step 506: End product color detection.

[0224] like Figure 8 As shown, the process for creating product type templates on both the left and right sides is the same. In this embodiment, the left side is used as an example. First, a saved but unrecognized product type image is taken, and the coordinate position information of the product type detection area (ROI) is manually determined and saved. Then, based on the determined ROI coordinate position information, the ROI is extracted from each image in a certain sample set collected from the saved but unrecognized product type image, and converted to grayscale. Next, feature points are extracted and described for each ROI, and each feature point description matrix is ​​saved in a container. Then, the container is traversed, and feature matching is performed on two different feature point description matrices each time, eliminating one of the matrices that does not meet the threshold for the number of matching points. After the traversal, the remaining feature point description matrices in the container are the templates. This process ends.

[0225] The above description is merely a specific embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes or alterations made without departing from the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for recognizing automotive dashboards, characterized in that, This includes human-computer interaction operations; the human-computer interaction operations include the following steps: Step 1: Receive the power-on command to open the software; the system will enter the initial interface. Step 2: The system receives the initialization command and enters the initialization interface; Step 3: The system receives the parameter setting command, enters the parameter setting interface, receives the parameter data, receives the OK button, and returns to the initialization interface; Step 4: The system receives the detection command and enters the detection process; The testing process includes empty vehicle testing and single vehicle testing. The detection steps for unloaded devices include: Step 201: Extract the ROI (Region of Interest) for product type detection based on the extracted coordinate location information; Step 202: Convert the ROI to a grayscale image; Step 203: Loop through all pixel values ​​in the ROI and count the number N of pixels whose values ​​are greater than the set grayscale value; Step 204: Determine whether the number N is greater than the set value of the total number of pixels; Step 205: End the testing process; The testing steps for a single-mounted vehicle include: Step 301: First, perform mean filtering on the ROI of the product type detection area; then perform Sober edge detection; then perform thresholding; finally, use Hough transform to find and save the line segments with slopes greater than the set threshold. Step 302: Traverse the saved set of straight line segments, calculate and find the longest straight line segment, and determine whether its length is greater than the set threshold; if yes, then determine that there is no product hanging on the side of the image sent by the industrial control computer and the camera in Step 102; if no, then determine that there is a product hanging on the side of the image sent by the industrial control computer and the camera in Step 102. Step 303: End the testing process; The steps of the detection process include: Step 101: The photoelectric switch is triggered by the hanger passing through the support frame, and the photoelectric switch wakes up the four cameras to collect one frame of image respectively. The cameras transmit the collected images to the industrial control computer; the code reader reads the serial number information of the hanger passing through the support frame and transmits it to the industrial control computer. Step 102: The industrial control computer identifies the image sent by the camera and first performs an empty hanger detection on the image; if it is an empty hanger, the industrial control computer summarizes the hanger serial number and hanger image, uploads the summarized content to the inventory management system, and the recognition process of the hanger ends; if it is not an empty hanger, a single hanger detection is performed. Step 103: If the hanger is a single hanger, further determine whether the hanger is a left hanger; if the hanger is a full hanger, save the full hanger determination result and perform product type detection and product color detection, and proceed to step 105. Step 104: If it is further determined to be a left-side hanging product, save the determination result and perform product type detection and product color detection to identify the type and color of the product on the left; otherwise, it is a right-side hanging product, save the determination result and perform product type detection and product color detection to identify the type and color of the product on the right. Step 105: After the inspection is completed, the industrial control computer summarizes the fixture serial number, fixture image, product type and color, and uploads it to the inventory management system. The identification process of the fixture is then complete. If the industrial control computer fails to recognize the image during the testing process, the system will copy the unrecognized product image to the unrecognized folder. Mean filtering involves taking the target pixel as the center, combining it with the eight pixels surrounding the target pixel to form a filter template, and then replacing the target pixel value with the average value of all pixels in the template. Step 5: The system receives the exit command and closes the software.

2. The method for recognizing a car dashboard according to claim 1, characterized in that, The Sober edge detection process includes: The matrix representation of the pixel values ​​of the detection region ROI is denoted as matrix A; Find the derivatives of matrix A in the horizontal (x) and vertical (y) directions respectively.

3. The method for recognizing a car dashboard according to claim 1, characterized in that, The product type detection process includes extracting feature points from the ROI region converted to grayscale. The steps for extracting feature points include: Step 401: Select a point P from the ROI, and draw a circle with P as the center and a radius r of 3 pixels; if there are n consecutive pixels whose absolute difference in grayscale value from point P is greater than a set value t, then P is considered a feature point; Step 402: Use the ID3 algorithm to select the pixel with the largest pixel gain as the basis for judging the feature point, generate a decision tree, input the pixels on the circumference of each feature point as features into the decision tree, and select the optimal feature point based on the features. Step 403: Construct a scoring function V; compare the V values ​​of any two adjacent feature points and delete the feature points with lower V values; Step 404: Calculate the centroid of the feature point within a radius of r using moments, and take the vector formed by the coordinates of the feature point to the centroid as the direction of the feature point; the definition of a moment is as follows: （1）； in, Indicates about First-order geometric moments, The grayscale expression for the image; the centroid of the moment. for: （2）； in, This represents the zeroth geometric moment when p=0 and q=0, and similarly... This represents the first-order geometric moment when p=1 and q=0. This represents the first-order geometric moment when p=0 and q=1; Taking the coordinates of the feature point as the origin, the angle of the vector is the direction of that feature point; where the angle of the vector... The calculation formula is as follows: （3）； In step 401, the gray values ​​at positions 1 and 9 on the circle are checked first. If the difference between the gray value of at least one pixel at these two positions and the gray value of point P is greater than t, then the gray values ​​at positions 5 and 13 on the circle are checked. If the difference between the gray values ​​of the pixels at positions 1 and 9 and the gray value of point P is not greater than t, then the pixel is determined to be a non-feature point. In step 403, the scoring function V is defined as the sum of the absolute values ​​of the differences between the grayscale values ​​of pixel P and its 16 surrounding pixels.

4. The method for recognizing a car dashboard according to claim 3, characterized in that, The product type detection process also includes feature point description and feature matching; the feature point description uses the rBRIEF algorithm, and the specific description steps include: Step 501: For each feature point, select a 31*31 neighborhood centered on the feature point. Within each neighborhood, select the average gray value of a 5*5 sub-neighborhood and compare it with the gray value of the pixel at the center of the 5*5 sub-neighborhood. This results in n = (31-5+1)*(31-5+1) sub-neighborhoods, and a total of m = C(n,2) point pairs, where C(n,2) represents the numerical value of the permutation and combination. Form a matrix from the point pairs of each neighborhood. ,matrix Each column vector ( ), Represents a pair of points: （4）； Using the direction of feature points For matrix Perform rotation to obtain the rotated matrix. ,in ; Represented as: （5）； According to equation (6), for the matrix Compare the grayscale values ​​of each point in the image. (6)； in, Represents the points in a pair of points grayscale value at that location Represents the points in a pair of points grayscale value at that location This indicates that the grayscale values ​​of a point pair are converted into binary values; For m pairs of points, we can obtain an initial binary string descriptor of length m. : （7）； By combining the orientation of feature points, a directional descriptor can be obtained. : （8） ； Step 502: Select k point pairs from m point pairs. The selection method includes the following steps: Step 5021: Reassemble all feature points and their corresponding initial binary strings to obtain matrix W, where the number of rows corresponds to the number of extracted feature points, and each row contains the initial binary string descriptor for each feature point. ; Step 5022: Sort the recombined matrix W in ascending order according to the absolute difference between the mean of each column and 0.5, to obtain matrix T; Step 5023: Greedy Choice: Place the first column of matrix T into matrix U, and then remove that column from matrix T. Matrix U is initially empty. Each column of matrix T is selected sequentially and compared with all columns in matrix U. If the similarity exceeds a set threshold, it is ignored and the next column is moved on. Otherwise, the column in matrix T is added to matrix U and removed from matrix T. Repeat the above process until k columns are selected; if the matrix U obtained by traversing matrix T has less than k columns, lower the threshold until matrix U has k columns. The feature matching method includes: calculating the Hamming distance between feature points in two images to obtain the similarity distance between feature points; finding the closest and second closest feature points in the image to be matched to the template image in terms of Hamming distance; comparing the ratio of the closest distance to the second closest distance with a set threshold; if the ratio is less than the threshold, then the point is a matching point; finding all matching points in the image to be matched and counting the number of matching points. Based on the matching results of feature points, determine whether the product belongs to a certain type or is unknown; if the maximum number of matching points is greater than the set threshold, the matching is considered successful and the product belongs to a certain type; if the maximum number of matching points is not greater than the set threshold, the matching is considered unsuccessful and the product belongs to an unknown type.

5. The method for recognizing a car dashboard according to claim 1, characterized in that, The steps for product color detection include: Step 601: Extract the ROI of the product color detection area; Step 602: Perform color space conversion on the ROI. The conversion formula is as follows: The ranges of values ​​for H, S, and V after conversion are as follows: , , ; Step 603: Divide the ROI into equal regions of the set value; Step 604: Identify the color of each equal region based on the different threshold ranges of the HSV corresponding to different colors; Step 605: Statistically analyze the color recognition results of each region, and determine the product color based on the region with the most regions that are identified as a certain color; Step 606: End product color detection; The method for recognizing car dashboards also includes creating templates for images in the unrecognized folder, the template creation comprising the following steps: Step 701: First, take a saved but unrecognized product type image, manually determine and save the coordinate position information of the product type detection area ROI; Step 702: Next, based on the determined ROI coordinate location information, extract the ROI from each image in a certain sample set of images collected from the saved and unidentified product type images, and convert them into grayscale images. Step 703: Then extract and describe the feature points for each ROI, and save the feature point description matrix of each feature point in a container; Step 704: Then, traverse the container, perform feature matching on two different feature point description matrices each time, and remove one of the matrices that does not meet the threshold for the number of matching points; after the traversal, the remaining feature point description matrices in the container are the templates. Step 705: End template creation.

Images