Vision-based computer keyboard quality inspection method and system
By using multi-source visual inspection and a nested edge detection network, the problems of large errors and high false positive/false negative rates caused by uneven lighting and complex structure in keyboard quality inspection are solved, thus achieving high-precision keyboard quality inspection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING UNIV OF ARTS & SCI
- Filing Date
- 2026-02-03
- Publication Date
- 2026-06-02
AI Technical Summary
Existing vision-based keyboard quality inspection methods suffer from unstable imaging under uneven lighting conditions, large character defect recognition errors, and insufficient detection accuracy and robustness due to the complex keyboard structure. They also struggle to achieve multi-dimensional detection and suffer from high rates of missed and false detections.
The keyboard's overall frontal grayscale image, key area close-up image, and key gap image are obtained using multi-source visual detection data. Image preprocessing is performed by combining Zhang Zhengyou calibration method and adaptive histogram equalization. Feature extraction is performed using a nested edge detection network and coherent line drawing algorithm. The keyboard's structure and appearance are determined by fusing edge probability maps.
It enables multi-dimensional detection of keyboards, reduces detection and recognition errors and false positives and misses, and improves the accuracy and consistency of quality inspection. It is suitable for automated batch keyboard quality inspection.
Smart Images
Figure CN122134640A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of quality inspection, and specifically to a vision-based method and system for inspecting the quality of computer keyboards. Background Technology
[0002] With the advancement of information technology and office automation, computer keyboards, as core input devices for human-computer interaction, have their manufacturing quality directly impacting user experience and product reliability. In large-scale, automated keyboard manufacturing processes, rapid and accurate quality inspection of finished keyboards is a crucial step in ensuring product quality before shipment.
[0003] Currently, vision-based automatic inspection technology has been widely used in industrial quality inspection. For example, existing technical solutions acquire an overall image of the keyboard from a single perspective and use image processing algorithms or traditional machine learning models for defect identification. However, these methods still face the following technical bottlenecks in practical applications: First, semi-transparent or highly reflective keycaps are difficult to image under uneven lighting conditions, resulting in large errors in character defect identification; second, the keyboard structure is complex, and a single overall image is insufficient to clearly present the details of each key and gap, leading to missed or false detections; third, detection often relies on template matching or simple feature judgment, resulting in insufficient accuracy and robustness.
[0004] Therefore, given the special structure and materials of keyboard products, in order to solve the problems of large identification errors, missed detections and false detections, and low detection accuracy in the keyboard quality inspection process, there is an urgent need for a new computer keyboard quality inspection method and system that can achieve multi-dimensional inspection of the keyboard, reduce detection and identification errors and missed detections and false detections, and improve the accuracy of quality inspection. Summary of the Invention
[0005] In view of this, the purpose of this invention is to overcome the defects in the prior art and provide a vision-based computer keyboard quality inspection method and system, which can realize multi-dimensional inspection of the keyboard, reduce detection and recognition errors and false positive and false negative rates, and improve the accuracy of quality inspection.
[0006] The present invention provides a vision-based computer keyboard quality detection method, comprising:
[0007] Acquire multi-source visual inspection data of the computer keyboard to be inspected; the multi-source visual inspection data includes a grayscale image of the keyboard as a whole front view, a close-up image of a key area, and an image of the key gaps.
[0008] Image preprocessing is performed on the multi-source visual detection data to obtain the preprocessed keyboard image;
[0009] The preprocessed keyboard image is divided into regions to obtain the detection sub-region corresponding to each key.
[0010] Feature extraction is performed on each detection sub-region to obtain a structure line diagram;
[0011] The surface of the keycaps is inspected to obtain a probability map of the fusion edges;
[0012] Based on the structural line diagram and the fused edge probability diagram, the keyboard structural state and keyboard appearance state are determined.
[0013] Furthermore, image preprocessing is performed on the multi-source visual detection data, specifically including:
[0014] The camera intrinsic parameter matrix K and distortion coefficients s were obtained using the Zhang Zhengyou calibration method, and the image was then processed to remove distortion.
[0015] For keyboard keycaps with reflective or semi-transparent materials, adaptive histogram equalization is used, and bilateral filtering is used to remove high-frequency noise while preserving edge information.
[0016] Furthermore, the preprocessed keyboard image is divided into regions, specifically including:
[0017] A topology matrix M of a standard keyboard is pre-constructed, in which each key is recorded. The theoretical center coordinates (x, y) and physical dimensions (w, h);
[0018] Template matching is used to locate the Enter or Space key in the image as a reference anchor point. The detection sub-regions of all keys are mapped according to the topological structure matrix M. Then, the boundaries of the detection sub-regions are fine-tuned based on the local gradient maximum method so that the detection sub-regions completely enclose the actual keys.
[0019] Furthermore, feature extraction is performed on each detection sub-region, specifically including:
[0020] Calculate the edge tangential flow field for each detection sub-region This vector field indicates the main direction of edge extension at each point in the image; through iterative smoothing, the flow field maintains a consistent direction at the edges.
[0021] Gaussian difference filtering is performed along the tangential flow field direction at the edge to enhance edge uniformity; the response function of the Gaussian difference filter operator is determined according to the following formula. :
[0022] ;
[0023] in, The coordinates of the current pixel; For the tangential flow field at the edge The perpendicular normal vector; and These are the central Gaussian function and the surrounding Gaussian function, respectively. This is a sensitivity parameter used to control the degree of noise suppression; The line integral intensity of the input image along the normal direction; Let be the radius of integration; For integration variables;
[0024] right By performing overlay thresholding, a high-precision structural line diagram is obtained. The structural line diagram Used to calculate the length and width dimensional errors of the buttons and the uniformity of the gaps between adjacent buttons.
[0025] Furthermore, a nested edge detection network is used to detect the keycap surface. This nested edge detection network uses VGG16 as the backbone, pruning the fully connected layers and retaining five side output layers. Each side output layer extracts feature maps at different scales, from small-scale texture details to large-scale character structures. The side output layers include shallow and deep output layers. The shallow output layers capture high-frequency details, detecting minor scratches and dents; the deep output layers capture semantic structures, detecting character integrity and missing characters.
[0026] Furthermore, the fusion edge probability map is determined according to the following formula. :
[0027] ;
[0028] in, For corresponding The weights; For the first The output of the side output layer.
[0029] Furthermore, the keyboard structure state is determined using the following method:
[0030] Structural line diagram Binarization is performed to obtain a binarized edge map; the connected components of each button contour are extracted using the binarized edge map.
[0031] For each key connected component, calculate its minimum bounding rectangle; this rectangle is defined by the coordinates of its center point. ,width ,high and rotation angle composition;
[0032] The calculated and The size is compared with the preset standard keycap size. If the error exceeds the set threshold, it is judged as a size deviation.
[0033] Calculate the distance between the edges of the outer rectangles of adjacent buttons. If the variance of the gap between the buttons in a row or column is greater than a preset threshold, it indicates that the buttons are not aligned and is judged as poor assembly.
[0034] Furthermore, the keyboard's appearance is determined using the following method:
[0035] Using the center point of the bounding rectangle of the keycap and the rotation angle, perform an affine transformation on the keycap area to be inspected, so that it is perfectly aligned with the standard character template in the spatial coordinate system; let the image of the aligned keycap be... The standard template image is Perform pixel-by-pixel subtraction on the two images to obtain the residual image. : ;
[0036] If the residual intensity within the character area is high, it indicates that there is missing printing, reprinting, or character misalignment, and it is judged as a printing defect.
[0037] If highlighted residual lines appear in flat areas that are not characters, then combine the edge probability map. The shallow features are used to determine whether the residual intensity at that location exceeds the residual threshold. If so, it is determined to be a surface scratch.
[0038] A vision-based computer keyboard quality inspection system includes a data acquisition unit, a data processing unit, a region division unit, and a quality inspection unit.
[0039] The data acquisition unit is used to acquire multi-source visual inspection data of the computer keyboard to be inspected; the multi-source visual inspection data includes a grayscale image of the keyboard as a whole, a close-up image of a key area, and an image of the key gaps.
[0040] The data processing unit is used to perform image preprocessing on multi-source visual detection data to obtain a preprocessed keyboard image.
[0041] The region division unit is used to divide the preprocessed keyboard image into regions to obtain the detection sub-region corresponding to each key.
[0042] The quality detection unit is used to extract features from each detection sub-region to obtain a structural line map, detect the keycap surface to obtain a fusion edge probability map, and determine the keyboard structural state and keyboard appearance state based on the structural line map and the fusion edge probability map.
[0043] The beneficial effects of this invention are as follows: This invention discloses a vision-based computer keyboard quality inspection method and system. By acquiring a grayscale image of the entire keyboard from the front view, close-up images of key areas, and key gap images, it achieves simultaneous perception of the overall keyboard structure and local details. Through unified image preprocessing and region segmentation, the inspection task is precisely mapped to the inspection sub-region corresponding to each key, thereby significantly improving the consistency of inspection. Structural line diagrams are extracted from each inspection sub-region and combined with the fusion edge probability map of the keycap surface to achieve a comprehensive judgment of key size, gap consistency, and surface defects. This invention can simultaneously evaluate the structural and appearance conditions of the keyboard, reducing the subjectivity of manual inspection, improving inspection accuracy and efficiency, and is suitable for automated, batch keyboard quality inspection scenarios. Attached Figure Description
[0044] The present invention will be further described below with reference to the accompanying drawings and embodiments:
[0045] Figure 1 This is a schematic diagram of the keyboard quality testing method of the present invention;
[0046] Figure 2 This is a schematic diagram of the keyboard quality detection system of the present invention. Detailed Implementation
[0047] The present invention will be further described below with reference to the accompanying drawings, as shown in the figures:
[0048] This embodiment discloses a vision-based computer keyboard quality detection method, including the following steps:
[0049] S1. Acquire multi-source visual inspection data of the computer keyboard to be inspected; the multi-source visual inspection data includes a grayscale image of the keyboard as a whole front view, a close-up image of a key area, and an image of the key gaps;
[0050] S2. Perform image preprocessing on the multi-source visual detection data to obtain the preprocessed keyboard image;
[0051] S3. Divide the preprocessed keyboard image into regions to obtain the detection sub-regions corresponding to each key;
[0052] S4. Extract features from each detection sub-region to obtain a structure line diagram;
[0053] S5. Inspect the surface of the keycaps to obtain a probability map of the fusion edges;
[0054] S6. Determine the keyboard structure state and keyboard appearance state based on the structural line diagram and the fused edge probability diagram.
[0055] In this embodiment, in step S1, a high-resolution industrial camera combined with multi-angle light sources is used to acquire multi-source visual inspection data of the computer keyboard to be inspected. The multi-source visual inspection data includes a grayscale image of the keyboard as a whole (for overall layout analysis), a close-up image of a key area (for detecting minor defects), and a key gap image obtained by side lighting.
[0056] In this embodiment, step S2 involves image preprocessing of the multi-source visual detection data, specifically including:
[0057] Distortion correction is performed, including: obtaining the camera intrinsic parameter matrix K and distortion coefficients s using the Zhang Zhengyou calibration method, performing distortion removal processing on the image, and ensuring the measurement accuracy of the button geometry.
[0058] Lighting compensation and noise reduction are performed, including: adaptive histogram equalization is used to improve local contrast for reflective or semi-transparent materials on keyboard keycaps; and bilateral filtering is used to remove high-frequency noise while preserving edge information.
[0059] In this embodiment, step S3 involves dividing the preprocessed keyboard image into regions, specifically including:
[0060] First, topology mapping is performed, including: pre-constructing a topology matrix M of a standard keyboard, where each key is recorded. The theoretical center coordinates (x, y) and physical dimensions (w, h);
[0061] Then, coarse localization and fine adjustment are performed, including: using template matching to locate the Enter or Space key in the image as a reference anchor point, mapping out the detection sub-regions of all keys according to the topological structure matrix M, and then fine-tuning the boundaries of the detection sub-regions based on the local gradient maximum method so that the detection sub-regions completely enclose the actual keys.
[0062] In this embodiment, step S4 involves extracting keyboard structural features based on coherent line drawing (CLD); the keyboard structural features include key outlines, dimensions, and gaps; specifically, feature extraction is performed on each detection sub-region, including:
[0063] Calculate the edge tangential flow field for each detection sub-region (In vector form), this vector field indicates the main direction of edge extension at each point in the image; through iterative smoothing, the flow field maintains a consistent direction at the edges.
[0064] Gaussian difference filtering is performed along the tangential flow field direction at the edge to enhance edge uniformity; the response function of the Gaussian difference filter operator is determined according to the following formula. :
[0065] ;
[0066] in, The coordinates of the current pixel; For the tangential flow field at the edge The perpendicular normal vector; and These are the central Gaussian function and the surrounding Gaussian function, respectively. This is a sensitivity parameter used to control the degree of noise suppression; The line integral intensity of the input image along the normal direction; The integration radius defines the coverage area of the filter in the normal direction (i.e., from -s to s), which determines the distance of surrounding pixels the algorithm considers. The variable for integration represents the vector along the normal direction. The amount of offset displacement; It is the surrounding Gaussian function and the central Gaussian function. By combining differentiators, a wider range of weight distributions can be controlled through variance to suppress background noise.
[0067] right By performing overlay thresholding, a high-precision structural line diagram is obtained. The structural line diagram Used to calculate the length and width dimensional errors of the buttons and the uniformity of the gaps between adjacent buttons.
[0068] This invention abandons the traditional Canny operator and innovatively adopts the Coherent-Line-Drawing (CLD) algorithm. The CLD algorithm can combine edge tangential flow (ETF) to generate highly continuous and clean line drawings while suppressing texture noise. It is particularly suitable for detecting the physical contours and assembly gaps of keys, thus effectively solving the problem of breakage at the rounded corners of keycaps caused by traditional operators.
[0069] In this embodiment, in step S5, a nested edge detection network is used to detect the surface of the keycap; wherein, the nested edge detection network uses VGG16 as the backbone network, the fully connected layers are pruned, and 5 side output layers are retained; each side output layer extracts feature maps of different scales, from small-scale texture details to large-scale character structures;
[0070] The side output layer includes a shallow output layer and a deep output layer. The shallow output layer is used to capture high-frequency details and detect minor scratches and dents. The deep output layer is used to capture semantic structure and detect character integrity and missing characters. The shallow output layer has a small receptive field, while the deep output layer has a large receptive field. There are a total of five shallow and deep output layers, and the specific number of shallow and deep output layers can be set according to actual operating conditions.
[0071] It should be noted that, in order to address the sample imbalance problem caused by the extremely low proportion of defective pixels in the keyboard image, this embodiment employs the Class-Balanced Cross-Entropy Loss function during the model training phase. This function increases the contribution of defective edge pixels to the loss calculation by introducing weight factors.
[0072] In the feature fusion stage, the network extracts feature maps of different depths through multiple side output layers. The final fused edge probability map... The fusion edge probability map is obtained by weighting the outputs from each side. Specifically, it is determined according to the following formula. :
[0073] ;
[0074] in, For corresponding The weights; For the first The output of the side output layer.
[0075] The fusion edge probability map It can simultaneously render clear character outlines (from deep features) and subtle surface scratches (from shallow features).
[0076] By employing a Holistically-Nested Edge Detection (HED) network, compared to traditional non-deep learning methods (such as Sobel and Prewitt), the HED network can perform end-to-end image learning, understand multi-scale semantic edges, and thus effectively distinguish the stroke edges of characters themselves from surface scratches, dirt, and defects.
[0077] In this embodiment, in step S6, the keyboard structure state is determined according to the following method:
[0078] Structural line diagram Binarization is performed to obtain a binarized edge map; the connected components of each button contour are extracted using the binarized edge map.
[0079] For each key connected component, calculate its minimum bounding rectangle; this rectangle is defined by the coordinates of its center point. ,width ,high and rotation angle composition;
[0080] The calculated and Compare with the preset standard keycap size; if the error exceeds... If so, it is determined to be a dimensional deviation;
[0081] Calculate the distance between the outer edges of the rectangles of adjacent keys. If the variance of the gap between keys in an entire row or column is greater than a preset threshold, it indicates that the keys are not aligned and is considered a poor assembly. The preset threshold can be determined based on the statistical characteristics of key gaps under normal assembly conditions, for example, a value of 0.02–0.03 mm².
[0082] Determine the keyboard's appearance condition using the following method:
[0083] Using the center point of the bounding rectangle of the keycap and the rotation angle, perform an affine transformation on the keycap area to be inspected, so that it is perfectly aligned with the standard character template in the spatial coordinate system; let the image of the aligned keycap be... The standard template image is Perform pixel-by-pixel subtraction on the two images to obtain the residual image. : ;
[0084] If the residual intensity within the character area is high, it indicates that there is missing printing, reprinting, or character misalignment, and it is judged as a printing defect.
[0085] If highlighted residual lines appear in flat areas that are not characters, then combine the edge probability map. The system identifies shallow features and determines whether the residual intensity at that location exceeds the residual threshold. If so, it is classified as a surface scratch. The residual threshold can be determined based on the grayscale fluctuations or the probability distribution of the fusion edge in the flat area of the keycap surface, for example, by taking a value between 20 and 30 grayscale levels or a probability value range of 0.45 to 0.6.
[0086] The present invention also relates to a vision-based computer keyboard quality inspection system, which corresponds to the quality inspection method of the above embodiments and can be understood as a quality inspection system that implements the above quality inspection method. The quality inspection system includes a data acquisition unit, a data processing unit, a region division unit, and a quality inspection unit.
[0087] The data acquisition unit is used to acquire multi-source visual inspection data of the computer keyboard to be inspected; the multi-source visual inspection data includes a grayscale image of the keyboard as a whole, a close-up image of a key area, and an image of the key gaps.
[0088] The data processing unit is used to perform image preprocessing on multi-source visual detection data to obtain a preprocessed keyboard image.
[0089] The region division unit is used to divide the preprocessed keyboard image into regions to obtain the detection sub-region corresponding to each key.
[0090] The quality detection unit is used to extract features from each detection sub-region to obtain a structural line map, detect the keycap surface to obtain a fusion edge probability map, and determine the keyboard structural state and keyboard appearance state based on the structural line map and the fusion edge probability map.
[0091] This invention provides an objective and scientific method and system for inspecting computer keyboard quality. By utilizing edge-tangential flow-guided Gaussian difference filtering for keyboard geometry detection, it enhances the utilization of directionality and solves the problem of inaccurate detection of keyboard rounded corners and gaps. By employing multi-scale depth supervision and category-balanced loss for defect detection on the keyboard surface, it enhances the robustness for detecting minute, low-contrast defects (such as light-colored scratches). Furthermore, it uses a dual-branch detection scheme, combining CLD for structural dimensions and HED for surface image quality, significantly improving the keyboard quality inspection effect.
[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A vision-based method for inspecting the quality of a computer keyboard, characterized in that: Includes the following steps: Acquire multi-source visual inspection data of the computer keyboard to be inspected; the multi-source visual inspection data includes a grayscale image of the keyboard as a whole front view, a close-up image of a key area, and an image of the key gaps. Image preprocessing is performed on the multi-source visual detection data to obtain the preprocessed keyboard image; The preprocessed keyboard image is divided into regions to obtain the detection sub-region corresponding to each key. Feature extraction is performed on each detection sub-region to obtain a structure line diagram; The surface of the keycaps is inspected to obtain a probability map of the fusion edges; Based on the structural line diagram and the fused edge probability diagram, the keyboard structural state and keyboard appearance state are determined.
2. The vision-based computer keyboard quality inspection method according to claim 1, characterized in that: Image preprocessing of multi-source visual detection data includes: The camera intrinsic parameter matrix K and distortion coefficients s were obtained using the Zhang Zhengyou calibration method, and the image was then processed to remove distortion. For keyboard keycaps with reflective or semi-transparent materials, adaptive histogram equalization is used, and bilateral filtering is used to remove high-frequency noise while preserving edge information.
3. The vision-based computer keyboard quality inspection method according to claim 1, characterized in that: The preprocessed keyboard image is divided into regions, specifically including: A topology matrix M of a standard keyboard is pre-constructed, in which each key is recorded. The theoretical center coordinates (x, y) and physical dimensions (w, h); Template matching is used to locate the Enter or Space key in the image as a reference anchor point. The detection sub-regions of all keys are mapped according to the topological structure matrix M. Then, the boundaries of the detection sub-regions are fine-tuned based on the local gradient maximum method so that the detection sub-regions completely enclose the actual keys.
4. The vision-based computer keyboard quality inspection method according to claim 1, characterized in that: Feature extraction is performed on each detection sub-region, specifically including: Calculate the edge tangential flow field for each detection sub-region This vector field indicates the main direction of edge extension at each point in the image; through iterative smoothing, the flow field maintains a consistent direction at the edges. Gaussian difference filtering is performed along the tangential flow field direction at the edge to enhance edge uniformity; the response function of the Gaussian difference filter operator is determined according to the following formula. : ； in, The coordinates of the current pixel; For the tangential flow field at the edge The perpendicular normal vector; and These are the central Gaussian function and the surrounding Gaussian function, respectively. This is a sensitivity parameter used to control the degree of noise suppression; The line integral intensity of the input image along the normal direction; Let be the radius of integration; For integration variables; right By performing overlay thresholding, a high-precision structural line diagram is obtained. The structural line diagram Used to calculate the length and width dimensional errors of the buttons and the uniformity of the gaps between adjacent buttons.
5. The vision-based computer keyboard quality inspection method according to claim 1, characterized in that: A nested edge detection network is used to detect the surface of the keycaps. This network uses VGG16 as the backbone, with fully connected layers pruned, retaining five side output layers. Each side output layer extracts feature maps at different scales, from small-scale texture details to large-scale character structures. The side output layers include shallow and deep output layers. The shallow output layers capture high-frequency details, detecting minor scratches and dents; the deep output layers capture semantic structures, detecting character integrity and missing characters.
6. The vision-based computer keyboard quality inspection method according to claim 5, characterized in that: The fusion edge probability map is determined according to the following formula. : ； in, For corresponding The weights; For the first The output of the side output layer.
7. The vision-based computer keyboard quality inspection method according to claim 1, characterized in that: The keyboard structure state can be determined using the following method: Structural line diagram Binarization is performed to obtain a binarized edge map; the connected components of each button contour are extracted using the binarized edge map. For each key connected component, calculate its minimum bounding rectangle; this rectangle is defined by the coordinates of its center point. ,width ,high and rotation angle composition; The calculated and The size is compared with the preset standard keycap size. If the error exceeds the set threshold, it is judged as a size deviation. Calculate the distance between the edges of the outer rectangles of adjacent buttons. If the variance of the gap between the buttons in a row or column is greater than a preset threshold, it indicates that the buttons are not aligned and is judged as poor assembly.
8. The vision-based computer keyboard quality inspection method according to claim 7, characterized in that: Determine the keyboard's appearance condition using the following method: Using the center point of the bounding rectangle of the keycap and the rotation angle, perform an affine transformation on the keycap area to be inspected, so that it is perfectly aligned with the standard character template in the spatial coordinate system; let the image of the aligned keycap be... The standard template image is Perform pixel-by-pixel subtraction on the two images to obtain the residual image. : ; If the residual intensity within the character area is high, it indicates that there is missing printing, reprinting, or character misalignment, and it is judged as a printing defect. If highlighted residual lines appear in flat areas that are not characters, then combine the edge probability map. The shallow features are used to determine whether the residual intensity at that location exceeds the residual threshold. If so, it is determined to be a surface scratch.
9. A vision-based computer keyboard quality inspection system, characterized in that: It includes a data acquisition unit, a data processing unit, a region division unit, and a quality inspection unit; The data acquisition unit is used to acquire multi-source visual inspection data of the computer keyboard to be inspected; the multi-source visual inspection data includes a grayscale image of the keyboard as a whole, a close-up image of a key area, and an image of the key gaps. The data processing unit is used to perform image preprocessing on multi-source visual detection data to obtain a preprocessed keyboard image. The region division unit is used to divide the preprocessed keyboard image into regions to obtain the detection sub-region corresponding to each key. The quality detection unit is used to extract features from each detection sub-region to obtain a structural line map, detect the keycap surface to obtain a fusion edge probability map, and determine the keyboard structural state and keyboard appearance state based on the structural line map and the fusion edge probability map.