Keyboard key surface defect detection method, device, equipment and medium

Abstract

The invention discloses a keyboard key surface defect detection method and device, equipment and a medium. The method comprises the following steps: acquiring a sample picture and a detection picture of a keyboard key surface defect; labeling the sample pictures according to defect types, and labeling the pictures with the same defect type with the same defect name; training the labeled sample pictures by using a convolutional neural network to obtain an optimal keyboard key surface detection model; and detecting the surface defects of the keyboard keys through the keyboard key surface detection model. According to the invention, the detection of a plurality of keyboard keys can be processed at the same time, the detection accuracy is high, the positioning precision is more accurate, and the trained model is more convenient to use.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to the technical field of defect detection, and in particular to a keyboard key surface defect detection method. BACKGROUND

[0002] With the rapid development of science and technology, machine vision technology, as an important branch of artificial intelligence, is gradually penetrating into various industries. Among them, the manufacturing and quality control of high-end electronic products such as notebook computers are the stage where machine vision technology can fully demonstrate its capabilities.

[0003] In particular, for the keyboard, a component frequently interacted by users, the quality detection of its surface is particularly important. Traditional keyboard surface defect detection relies on manual visual inspection. By observing the defects of keycaps on an entire keyboard with the naked eye, and combining personal experience and knowledge, the keycaps are viewed from multiple angles and defects are judged. Manual detection relies on subjective judgment of humans, which may be affected by factors such as fatigue, distraction, different standards of each person, and the like, and there is a certain amount of misjudgment and missed detection. On the other hand, manual detection is relatively low in efficiency, and is not sufficient for large-scale and high-speed production lines. In addition, detection personnel need to rest and rotate to avoid detection efficiency decline due to fatigue, which results in that the production capacity of the factory is always at a relatively stable number, and it is difficult to have a greater improvement.

[0004] There are also other methods on the market for detecting defects on the surface of the keyboard. A 2D area array camera is used to detect defects in combination with traditional vision algorithms. Although this method can be faster than manual detection, it still has several problems. First, the photographing time is too long, and the time required for multiple photographs is much longer than the time required for a single photograph by a line scanning camera. Second, traditional vision algorithms are much worse than deep learning in defect processing, and there are more misjudgments and missed detections due to the small area of the defects. Third, the area array camera can only capture front defects, and is powerless for side defects.

[0005] Therefore, the prior art needs to be improved. SUMMARY

[0006] In view of the above defects or deficiencies in the prior art, it is desirable to provide a keyboard key surface defect detection method that can meet the specific needs of keyboard key surface defect detection.

[0007] Based on one aspect of the embodiments of the present application, the embodiments of the present application provide a keyboard key surface defect detection method, comprising:

[0008] Obtaining sample pictures and detection pictures of keyboard key surface defects, the sample pictures including pictures classified according to defect types after photographing keyboard key surface defects, and the detection pictures being pictures classified according to defect types after photographing keyboard key surface defects different from the sample pictures, the sample pictures and the detection pictures being stored in different folders respectively;

[0009] Labeling the sample pictures according to defect types, and labeling the same defect name for pictures of the same defect type;

[0010] Training the sample pictures with labeling completed by using a convolutional neural network to obtain an optimal keyboard key surface detection model;

[0011] Detecting keyboard key surface defects by using the keyboard key surface detection model.

[0012] In another embodiment, the obtaining sample pictures and detection pictures of keyboard key surface defects comprises:

[0013] Collecting sample pictures of keyboard keys manually and obtaining pictures of keyboard key surface defect areas by using a camera;

[0014] Classifying the obtained pictures of defect areas according to defect types, and storing the pictures classified to a first folder as sample pictures of keyboard key surface defects;

[0015] Obtaining pictures of keyboard key surface defect areas, classifying the obtained pictures of keyboard key surface defect areas according to defect types, and storing the pictures classified to a second folder as detection pictures of keyboard key surface defects.

[0016] In another embodiment, the training the sample pictures with labeling completed by using a convolutional neural network to obtain an optimal keyboard key surface detection model comprises:

[0017] Inputting the sample pictures to an input layer of the convolutional neural network to form a two-dimensional matrix representing intensity values of pixels;

[0018] Extracting the intensity values of pixels, performing convolution operation on the intensity values of the input pixels to obtain a first feature map, and the convolution operation is used to capture local features and spatial structures of the intensity values of the pixels of the input sample pictures to realize feature extraction and representation of the input data;

[0019] Performing dimension reduction and sampling on the first feature map to generate a second feature map, and the second feature map is the first feature map after dimension reduction and sampling, which reduces the number of parameters and the computational complexity;

[0020] The feature vectors in the first feature map and the second feature map are combined and classified to obtain high-level feature learning and classification of the input sample picture, and a keyboard key surface detection model is generated.

[0021] According to the keyboard key surface defect, a new sample picture is collected, the keyboard key surface detection model is input with the new sample picture, the keyboard key surface detection model is further trained, and after the training target is reached, the best keyboard key surface detection model is obtained.

[0022] In another embodiment, the keyboard key surface defect is detected by the keyboard key surface detection model, comprising:

[0023] The keys on the two keyboards are accurately positioned to determine the position of the keycap on each key;

[0024] A corresponding first target detection area is established for each keycap, the first target detection area is a detection area corresponding to each keycap, and the detection area corresponding to each keycap is a pooling layer ROI (Region of interest) of the keyboard key surface detection model;

[0025] The threshold difference between each keycap image and its background area image in the first target detection area is calculated to automatically extract the contour image of each keycap area;

[0026] The minimum circumscribed rectangle of the contour image of each keycap area is obtained through the contour image of each keycap area, and the minimum circumscribed rectangle contains the keycap area contour image surrounded by the minimum circumscribed rectangle;

[0027] The minimum circumscribed rectangle of each keycap area contour image is created as a second target detection area, and the second target detection area is a pooling layer ROI of the minimum circumscribed rectangle of each keycap area contour image;

[0028] The minimum circumscribed rectangle of each keycap area contour image in the second target detection area is cropped to obtain a third target detection area;

[0029] Each keycap area contour image in the third target detection area is input to the keyboard key surface detection model for detection;

[0030] The defect coordinates detected by the keyboard key surface detection model are converted into coordinates corresponding to the entire keyboard through affine transformation, and the defects are displayed.

[0031] In another embodiment, the keys on the two keyboards are accurately positioned to determine the position of the keycap on each key, comprising:

[0032] Obtaining two edges of two keyboard keys, the obtaining two edges of two keyboard keys using an add_metrology_object_generic() algorithm to find;

[0033] Obtaining the included angle of two edges, the obtaining the included angle using a line_line_intersection() algorithm;

[0034] Accurately positioning two keyboard keys through the included angle;

[0035] Determining the position of each keycap of two keyboard keys through the accurate positioning of two keyboard keys.

[0036] In another embodiment, the calculating the threshold difference between each keycap image and the background area image of the first target detection area, automatically extracting the contour image of each keycap area, comprises:

[0037] Calculating the threshold difference between each keycap image and the background area image in the first target detection area through a threshold() algorithm;

[0038] According to the size of the threshold difference, distinguishing the keycap image from the background area image, and obtaining the keycap area image;

[0039] Using a Select_shape() algorithm to calculate the area corresponding to the keycap area image, and screening out the keycap area according to the area size;

[0040] Extracting the contour of the screened keycap area image to obtain the contour image of each keycap area.

[0041] In another embodiment, the defect coordinates obtained by the keyboard key surface detection model are converted into the coordinates corresponding to the entire keyboard through affine transformation, and the defects are displayed, comprising:

[0042] Obtaining the defect coordinates of the keycap area image through the keyboard key surface detection model;

[0043] Using a hom_mat2d_identity algorithm to establish a first matrix, the first matrix being a coordinate conversion matrix, used to convert the input coordinate data into the coordinate data corresponding to the keyboard key;

[0044] Inputting the defect coordinates of the keycap area image into the first matrix, and converting the defect coordinates of the keycap area image into corresponding keyboard key coordinates through the first matrix;

[0045] Printing the converted keyboard key coordinates onto the keyboard key picture.

[0046] According to another aspect of the embodiments of the present application, a keyboard key surface defect detection device is disclosed, comprising:

[0047] a obtaining module, configured to obtain sample pictures and detection pictures of keyboard key surface defects, wherein the sample pictures comprise pictures classified according to defect types after photographing keyboard key surface defects, the detection pictures are pictures classified according to defect types after photographing keyboard key surface defects different from the sample pictures, and the sample pictures and the detection pictures are respectively stored in different folders;

[0048] a training module, configured to mark the sample pictures according to defect types, mark pictures of the same defect type with the same defect name, and train the sample pictures marked to obtain an optimal keyboard key surface detection model by using a convolutional neural network;

[0049] a detection module, configured to detect keyboard key surface defects by using the keyboard key surface detection model.

[0050] According to still another aspect of the embodiments of the present application, an electronic device is disclosed, comprising one or more processors and a memory, the memory is configured to store one or more programs, and when the one or more programs are executed by the processor, the processor is caused to implement the keyboard key surface defect detection method provided by the embodiments of the present application.

[0051] According to still another aspect of the embodiments of the present application, a computer readable storage medium storing a computer program is disclosed, and the computer program is executed to implement the keyboard key surface defect detection method provided by the embodiments of the present application.

[0052] In the embodiments of the present application, sample pictures and detection pictures of keyboard key surface defects are obtained, the sample pictures are marked according to defect types, pictures of the same defect type are marked with the same defect name, the sample pictures marked are trained by using a convolutional neural network to obtain an optimal keyboard key surface detection model, and keyboard key surface defects are detected by using the keyboard key surface detection model. The present application can simultaneously detect multiple keyboard keys, has high detection accuracy, has more accurate positioning accuracy, and the trained model is more convenient to use. BRIEF DESCRIPTION OF DRAWINGS

[0053] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments made with reference to the accompanying drawings:

[0054] Figure 1 is a flowchart of the keyboard key surface defect detection method provided by an embodiment of the present application;

[0055] Figure 2Fig. 1 is a structural schematic diagram of a keyboard key surface defect detection device provided by an embodiment of the present application;

[0056] Figure 3 Fig. 1 is a structural schematic diagram of a keyboard key surface defect detection device provided by an embodiment of the present application; DETAILED DESCRIPTION

[0057] The present application will be further described below in conjunction with the drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended for the purpose of interpretation of the present application and are not intended to limit the present application. In addition, it should be noted that only parts related to the present application are shown in the drawings for the purpose of description.

[0058] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the drawings and embodiments.

[0059] In one embodiment, as shown in Fig. 1, a keyboard key surface defect detection method is provided. Figure 1

[0060] Please refer to Fig. 1, which shows a flowchart of the keyboard key surface defect detection method to which the embodiments of the present application can be applied. Figure 1 As shown in Fig. 1, the keyboard key surface defect detection method comprises the following steps.

[0061] Figure 1 Step 101, acquiring sample pictures and detection pictures of keyboard key surface defects, wherein the sample pictures comprise pictures classified according to defect types after photographing keyboard key surface defects, the detection pictures are pictures classified according to defect types after photographing keyboard key surface defects different from the sample pictures, and the sample pictures and the detection pictures are respectively stored in different folders.

[0062] Specifically, the acquiring of the sample pictures and the detection pictures of keyboard key surface defects comprises the following steps.

[0063] Specifically, the acquiring of the sample pictures and the detection pictures of keyboard key surface defects comprises the following steps.

[0064] Collecting sample pictures of keyboard keys by hand and obtaining pictures of keyboard key surface defect areas by using a camera;

[0065] Classifying the obtained pictures of defect areas according to defect types, and storing the classified pictures in a first folder as sample pictures of keyboard key surface defects.

[0066] In this way, pictures of keyboard key surface defect areas are obtained, and the obtained pictures of keyboard key surface defect areas are classified according to defect types, and the classified pictures are stored in a second folder as detection pictures of keyboard key surface defects. ​​

[0067] Specifically, when collecting pictures, first divide the pictures into sample pictures and detection pictures, the sample pictures are used to train the keyboard key surface detection model through the deep learning algorithm, the detection pictures are used to further enrich the training samples of the keyboard key surface detection model and the effect of the detection model, the sample pictures are saved in a first folder, in specific application, the first folder is named as a train folder, the detection pictures are saved in a second folder, in specific application, the second folder is named as a test folder, which is convenient for subsequent file acquisition.

[0068] Step 102, labeling the sample pictures according to defect types, and labeling the same defect name for pictures of the same defect type;

[0069] Specifically, the purpose of labeling according to defect types is to train different keyboard keys of different defect types through a convolutional neural network, so as to accurately identify different types of keyboard key defects.

[0070] Step 103, training the sample pictures with completed labeling by using a convolutional neural network to obtain an optimal keyboard key surface detection model;

[0071] Specifically, convolution operation is one of the core operations in the convolutional neural network CNN, which simulates the receptive field mechanism in the biological visual system, and obtains an output feature map by sliding a convolution kernel (Filter) on the input data for weighted summation. The convolution operation can be expressed as:

[0072] ;

[0073] In the formula, I is the input image, K is the convolution kernel, C is the output feature map, (i, j) is the coordinate of the output feature map, (m, n) is the coordinate of the convolution kernel, and by continuously adjusting the parameters of the convolution kernel, different features in the input data can be extracted, such as the convex point, scratch and indentation of the keycap.

[0074] The convolutional neural network includes the following layers:

[0075] First, the input layer: receiving the original image data, forming a two-dimensional matrix representing the intensity value of the pixel.

[0076] Second, the convolution layer: the convolution layer is one of the most important components in the convolutional neural network CNN, which is responsible for extracting the features of the input data. In the convolution layer, a set of convolution kernels is defined to perform convolution operation on the input data, thereby obtaining a series of feature maps. Convolution operation can effectively capture local features and spatial structure, and realize feature extraction and representation of input data.

[0077] Third, the pooling layer (also known as the down-sampling layer): the pooling layer is used to reduce the dimension and sample the feature map output by the convolution layer, to reduce the number of model parameters and computational complexity. Common pooling operations include max pooling and average pooling, which take the maximum or average value in a certain area to retain important information and reduce the amount of data, and the pooling operation can also enhance the model's robustness to translation and scale changes, and improve the model's generalization ability.

[0078] Fourth, the fully connected layer: the fully connected layer is usually located in the last few layers of the convolutional neural network, responsible for combining and classifying the features extracted by the convolutional and pooling layers. Each neuron in the fully connected layer is connected to all the neurons in the previous layer, and through learning the weight and bias parameters, it realizes high-level feature learning and classification of input data.

[0079] Specifically, the use of a convolutional neural network to train a labeled sample picture to obtain an optimal keyboard key surface detection model includes:

[0080] The sample picture is input into the input layer of the convolutional neural network to form a two-dimensional matrix representing the intensity value of the pixels;

[0081] The intensity value of the pixels is extracted, and the input pixel intensity value is convolved to obtain a first feature map. The convolution operation is used to capture the local features and spatial structure of the input sample picture pixel intensity value, and to realize feature extraction and representation of the input data.

[0082] The first feature map is reduced in dimension and sampled to generate a second feature map. The second feature map is the first feature map after dimension reduction and sampling, which reduces the number of parameters and computational complexity.

[0083] The feature vectors in the first and second feature maps are combined and classified to obtain high-level feature learning and classification of the input sample picture, and a keyboard key surface detection model is generated.

[0084] According to the keyboard key surface defect, a new sample picture is collected, the keyboard key surface detection model is input with the new sample picture, and the keyboard key surface detection model is further trained to achieve the training goal. After the training goal is achieved, the optimal keyboard key surface detection model is obtained.

[0085] Specifically, a model needs to be continuously labeled with new samples. Only by continuously learning and training it can the desired results of the project be achieved.

[0086] Step 104, detecting the keyboard key surface defect through the keyboard key surface detection model.

[0087] Specifically, the keyboard key surface defect is detected through the keyboard key surface detection model, which includes:

[0088] precisely positioning the keys on the two keyboards to determine the position of the keycap on each key;

[0089] establishing a corresponding first target detection area for each keycap, the first target detection area being a detection area corresponding to each keycap, the detection area corresponding to each keycap being a pooling layer ROI (Region of Interest) of the keyboard key surface detection model;

[0090] calculating the threshold difference between the image of each keycap in the first target detection area and the image of the background area thereof, and automatically extracting the contour image of each keycap area;

[0091] obtaining the minimum bounding rectangle of the corresponding keycap area contour image through the contour image of each keycap area, the minimum bounding rectangle containing the keycap area contour image surrounded by the minimum bounding rectangle;

[0092] creating the minimum bounding rectangle of each keycap area contour image as a second target detection area, the second target detection area being a pooling layer ROI of the minimum bounding rectangle of each keycap area contour image;

[0093] cropping the minimum bounding rectangle of each keycap area contour image in the second target detection area to obtain a third target detection area;

[0094] inputting each keycap area contour image in the third target detection area to the keyboard key surface detection model for detection;

[0095] converting the defect coordinates detected by the keyboard key surface detection model into the corresponding coordinates of the entire keyboard through affine transformation, and displaying the defects.

[0096] Specifically, the precise positioning of the keys on the two keyboards to determine the position of the keycap on each key includes:

[0097] obtaining two edges of the keys of the two keyboards using the add_metrology_object_generic() algorithm;

[0098] calculating the included angle of the two edges using the line_line_intersection() algorithm;

[0099] precisely positioning the two keyboard keys through the included angle;

[0100] determining the position of each keycap of the two keyboard keys through the precise positioning of the two keyboard keys.

[0101] Specifically, the step of calculating the threshold difference between each keycap image and its background region image within the first target detection area, and automatically extracting the contour image of each keycap region, includes:

[0102] The threshold difference between each keycap image and its background image within the first target detection area is calculated using the threshold() algorithm.

[0103] Based on the threshold difference, the keycap image is distinguished from the background area image to obtain the keycap area image;

[0104] The Select_shape() algorithm is used to calculate the area corresponding to the keycap region image, and the keycap region is filtered out based on the area size.

[0105] The selected keycap area images are subjected to contour extraction to obtain the contour image of each keycap area.

[0106] Specifically, the step of converting the defect coordinates detected by the keyboard key surface detection model into coordinates corresponding to the entire keyboard through affine transformation and displaying the defects includes:

[0107] The defect coordinates of the keycap area image are obtained by detecting defects using a keyboard key surface detection model.

[0108] The first matrix is ​​established using the hom_mat2d_identity algorithm. This first matrix is ​​a coordinate transformation matrix used to convert the input coordinate data into the coordinate data corresponding to the keyboard keys.

[0109] Input the defect coordinates of the keycap area image into the first matrix, and convert the defect coordinates of the keycap area image into the corresponding keyboard key coordinates through the first matrix;

[0110] Print the converted keyboard key coordinates onto the keyboard key image.

[0111] This application's keyboard key surface defect detection method involves acquiring sample images and detection images of keyboard key surface defects; labeling the sample images according to defect type, with images of the same defect type labeled with the same defect name; training the labeled sample images using a convolutional neural network to obtain an optimal keyboard key surface detection model; and detecting keyboard key surface defects using the keyboard key surface detection model. This application can simultaneously process the detection of multiple keyboard keys, achieving high detection accuracy, more precise positioning, and a more convenient use of the trained model.

[0112] like Figure 2 As shown, Figure 2 This is a schematic diagram of a keyboard key surface defect detection device according to an embodiment of this application. The keyboard key surface defect detection device includes:

[0113] the acquisition module, the training module and the detection module;

[0114] the acquisition module is configured to acquire sample pictures and detection pictures of surface defects of keyboard keys, the sample pictures comprise pictures classified according to defect types after photographing surface defects of keyboard keys, and the detection pictures are pictures classified according to defect types after photographing surface defects of keyboard keys different from the sample pictures, the sample pictures and the detection pictures are stored in different folders respectively;

[0115] the training module is configured to label the sample pictures according to defect types, label the pictures of the same defect type with the same defect name, train the sample pictures with completed labels by using a convolutional neural network, and acquire an optimal keyboard key surface detection model;

[0116] the detection module is configured to detect surface defects of keyboard keys by using the keyboard key surface detection model;

[0117] Specifically, the acquisition module is configured to collect sample pictures of keyboard keys manually, and acquire pictures of defect areas of keyboard key surfaces by using a camera; the acquired pictures of defect areas are classified according to defect types, and the pictures with completed classification are stored in a first folder as sample pictures of surface defects of keyboard keys; pictures of defect areas of keyboard key surfaces are acquired again, and the acquired pictures of defect areas of keyboard key surfaces are classified according to defect types, and the pictures with completed classification are stored in a second folder as detection pictures of surface defects of keyboard keys.

[0118] Specifically, the training module is configured to input the sample pictures into an input layer of the convolutional neural network to form a two-dimensional matrix representing intensity values of pixels; the intensity values of pixels are extracted, and the input intensity values of pixels are subjected to convolution operation to obtain a first feature map, the convolution operation is configured to capture local features and spatial structures of the intensity values of pixels of the input sample pictures, and realize feature extraction and representation of input data; the first feature map is subjected to dimension reduction and sampling to generate a second feature map, the second feature map is the first feature map after dimension reduction and sampling, and the number of parameters and the calculation complexity are reduced; feature vectors in the first feature map and the second feature map are combined and classified to acquire high-level feature learning and classification of the input sample pictures, and generate a keyboard key surface detection model; new sample pictures are collected according to surface defects of keyboard keys, the new sample pictures are input into the keyboard key surface detection model, the keyboard key surface detection model is further trained, and an optimal keyboard key surface detection model is acquired after reaching a training target.

[0119] Specifically, the detection module is configured to accurately position the keys on the two keyboards, determine the position of the keycap on each key, establish a corresponding first target detection area for each keycap, the first target detection area being a detection area corresponding to each keycap, the detection area corresponding to each keycap being a pooling layer ROI (Region of interest) of the keyboard key surface detection model, calculate the threshold difference between the image of each keycap in the first target detection area and the image of the background area thereof, automatically extract the contour image of each keycap area, obtain the minimum bounding rectangle of the corresponding keycap area contour image through the contour image of each keycap area, the minimum bounding rectangle containing the keycap area contour image surrounded by the minimum bounding rectangle, create the minimum bounding rectangle of each keycap area contour image as a second target detection area, the second target detection area being a pooling layer ROI of the minimum bounding rectangle of each keycap area contour image, crop the minimum bounding rectangle of each keycap area contour image in the second target detection area to obtain a third target detection area, input each keycap area contour image in the third target detection area to the keyboard key surface detection model for detection, convert the defect coordinates detected by the keyboard key surface detection model into coordinates corresponding to the entire keyboard through affine transformation, and display the defects.

[0120] Specifically, the detection module is configured to obtain two edges of the keys of the two keyboards using the add_metrology_object_generic() algorithm, calculate the included angle of the two edges using the line_line_intersection() algorithm, accurately position the two keys through the included angle, and determine the position of each keycap of the two keys through the accurate positioning of the two keys.

[0121] Specifically, the detection module is configured to calculate the threshold difference between the image of each keycap in the first target detection area and the image of the background area thereof through the threshold() algorithm, distinguish the keycap image from the background area image according to the size of the threshold difference to obtain a keycap area image, calculate the area corresponding to the keycap area image using the Select_shape() algorithm, and screen out the keycap area according to the size of the area. The keycap area image screened out is subjected to contour extraction to obtain the contour image of each keycap area.

[0122] Specifically, the detection module is configured to detect the keycap region image by using a keyboard key surface defect detection model to obtain defect coordinates of the keycap region image; a first matrix is established by using a hom_mat2d_identity algorithm, the first matrix being a coordinate conversion matrix configured to convert input coordinate data into coordinate data corresponding to the keyboard keys; the defect coordinates of the keycap region image are input into the first matrix, and the defect coordinates of the keycap region image are converted into corresponding keyboard key coordinates by using the first matrix; and the converted keyboard key coordinates are printed on the keyboard key picture.

[0123] The keyboard key surface defect detection device provided in the present application obtains sample pictures and detection pictures of keyboard key surface defects through an acquisition module; a training module labels the sample pictures according to defect types, and pictures of the same defect type are labeled with the same defect name; a convolutional neural network is used to train the labeled sample pictures to obtain an optimal keyboard key surface detection model; and a detection module detects keyboard key surface defects by using the keyboard key surface detection model. The present application can simultaneously detect multiple keyboard keys, has high detection accuracy, has more accurate positioning accuracy, and is more convenient to use.

[0124] Specific limitations of the keyboard key surface defect detection device can be found in the limitations of the keyboard key surface defect detection method described above, and will not be repeated here. Each module in the keyboard key surface defect detection device described above can be realized by software, hardware, and combinations thereof, in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above modules.

[0125] In particular, according to an embodiment of the present disclosure, as shown in Figure 3 The electronic device includes one or more processors and a memory, the memory storing one or more programs; when the one or more programs are executed by the processor, the processor implements the keyboard key surface defect detection method described in the embodiments of the present application.

[0126] In particular, according to an embodiment of the present disclosure, the keyboard key surface defect detection method described in any of the above embodiments can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product including a computer program tangibly embodied on a machine-readable medium, the computer program including program code for executing the keyboard key surface defect detection method. In such an embodiment, the computer program can be downloaded and installed from a network by a communication part, and / or installed from a detachable medium.

[0127] The one or more programs stored in the read-only memory (ROM) or the random access memory (RAM) perform various appropriate actions and processes. In the random access memory (RAM), there are included software programs for the server to complete respective services, and various programs and data required for vehicle driving operations. The server, the hardware devices controlled thereby, the read-only memory (ROM), and the random access memory (RAM) are connected to each other via a bus, and various input / output interfaces are also connected to the bus.

[0128] The following components are connected to the input / output interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; and a communication part including a network interface card such as a LAN card, a modem, etc. The communication part performs communication processing via a network such as the Internet. A drive is also connected to the input / output interface as necessary. A removable medium such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive as necessary, so that a computer program read out therefrom is installed in the memory as necessary.

[0129] In particular, according to embodiments of the present disclosure, the keyboard key surface defect detection method described in any of the above embodiments can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product including a computer program tangibly embodied on a machine-readable medium, the computer program including program code for executing a keyboard key surface defect detection method. In such embodiments, the computer program can be downloaded and installed from a network via the communication part, and / or installed from a removable medium.

[0130] The units or modules described in the embodiments of the present application can be implemented by means of software, or can be implemented by means of hardware. The described units or modules can also be arranged in a processor. The names of the units or modules do not constitute a limitation on the units or modules themselves in some cases.

[0131] The above description is merely the preferred embodiments of the present application and the explanation of the applied technical principles. Those skilled in the art should understand that the scope of the application involved in the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features can be replaced with technical features disclosed in the present application (but not limited to) having similar functions to form technical solutions.

Claims

1. A method of detecting surface defects of a key of a keyboard, characterized by, The method comprises the following steps: obtaining sample pictures and detection pictures of keyboard key surface defects, wherein the sample pictures comprise pictures classified according to defect types after photographing keyboard key surface defects, and the detection pictures are pictures classified according to defect types after photographing keyboard key surface defects different from the sample pictures, and the sample pictures and the detection pictures are stored in different folders respectively; annotating the sample pictures according to defect types, and annotating pictures of the same defect type with the same defect name; training the sample pictures with annotations by using a convolutional neural network to obtain an optimal keyboard key surface detection model; detecting keyboard key surface defects by using the keyboard key surface detection model.

2. The keyboard key surface defect detection method according to claim 1, characterized by, The method for obtaining sample pictures and detection pictures of keyboard key surface defects comprises the following steps: collecting sample pictures of keyboard keys by using a camera to obtain pictures of defect areas on keyboard key surfaces; classifying the obtained pictures of defect areas according to defect types, and storing the classified pictures in a first folder as sample pictures of keyboard key surface defects; obtaining pictures of defect areas on keyboard key surfaces again, classifying the obtained pictures of defect areas on keyboard key surfaces again according to defect types, and storing the classified pictures in a second folder as detection pictures of keyboard key surface defects.

3. The keyboard key surface defect detection method according to claim 1, characterized by, The method for training sample pictures with annotations by using a convolutional neural network to obtain an optimal keyboard key surface detection model comprises the following steps: inputting the sample pictures into an input layer of the convolutional neural network to form a two-dimensional matrix representing intensity values of pixels; extracting the intensity values of pixels, performing convolution operation on the input intensity values of pixels to obtain a first feature map, and using the convolution operation to capture local features and spatial structures of the intensity values of pixels of the input sample pictures to realize feature extraction and representation of the input data; performing dimension reduction and sampling on the first feature map to generate a second feature map, wherein the second feature map is the first feature map after dimension reduction and sampling, and the second feature map has reduced number of parameters and computational complexity; combining and classifying feature vectors in the first feature map and the second feature map to obtain high-level feature learning and classification of the input sample pictures, and generating a keyboard key surface detection model; collecting new sample pictures according to keyboard key surface defects, inputting the new sample pictures into the keyboard key surface detection model, further training the keyboard key surface detection model, and obtaining an optimal keyboard key surface detection model after reaching a training target.

4. The keyboard key surface defect detection method according to claim 1, characterized by, The method for detecting keyboard key surface defects by using the keyboard key surface detection model comprises the following steps: precisely positioning keys on two keyboards to determine positions of keycaps on each key; establishing a corresponding first target detection area for each keycap, wherein the first target detection area is a detection area corresponding to each keycap, and the detection area corresponding to each keycap is a pooling layer ROI (Region of interest) of the keyboard key surface detection model. Calculate the threshold difference between each keycap image and its background area image in the first target detection area, and automatically extract the contour image of each keycap area; Obtain the minimum bounding rectangle of the corresponding keycap area contour image through the contour image of each keycap area, and the minimum bounding rectangle contains the keycap area contour image surrounded by the minimum bounding rectangle; Create the minimum bounding rectangle of each keycap area contour image as a second target detection area, which is the pooling layer ROI of the minimum bounding rectangle of each keycap area contour image; Crop the minimum bounding rectangle of each keycap area contour image in the second target detection area to obtain a third target detection area; Input each keycap area contour image in the third target detection area into the keyboard key surface detection model for detection; Convert the defect coordinates detected by the keyboard key surface detection model into the corresponding coordinates of the entire keyboard through affine transformation, and display the defects.

5. The keyboard key surface defect detection method according to claim 4, characterized by, The accurate positioning of the keys on the two keyboards and the determination of the position of the keycap on each key include: Obtain two edges of the keys of the two keyboards, which uses the add_metrology_object_generic() algorithm to find; Calculate the included angle of the two edges, which uses the line_line_intersection() algorithm; Accurately position the two keyboard keys through the included angle; Determine the position of each keycap of the two keyboard keys through accurate positioning.

6. The keyboard key surface defect detection method according to claim 4, characterized by, The calculation of the threshold difference between each keycap image and its background area image in the first target detection area and the automatic extraction of the contour image of each keycap area include: Calculate the threshold difference between each keycap image and its background area image in the first target detection area through the threshold() algorithm; Distinguish the keycap image from the background area image according to the size of the threshold difference to obtain the keycap area image; Use the Select_shape() algorithm to calculate the area corresponding to the keycap area image, and screen out the keycap area according to the area size; Extract the contour of the screened keycap area image to obtain the contour image of each keycap area.

7. The keyboard key surface defect detection method according to claim 4, characterized by, The conversion of the defect coordinates detected by the keyboard key surface detection model into the corresponding coordinates of the entire keyboard through affine transformation and the display of the defects include: Detect the defect coordinates of the keycap area image through the keyboard key surface detection model; Use the hom_mat2d_identity algorithm to establish a first matrix, which is a coordinate conversion matrix used to convert the input coordinate data into keyboard key corresponding coordinate data; Input the defect coordinates of the keycap area image into the first matrix, and convert the defect coordinates of the keycap area image into corresponding keyboard key coordinates through the first matrix; Print the converted keyboard key coordinates to the keyboard key picture.

8. A device for detecting surface defects of keyboard keys, characterized in that, It includes: An acquisition module is configured to acquire sample pictures and detection pictures of keyboard key surface defects, the sample pictures including pictures classified according to defect types after photographing keyboard key surface defects, and the detection pictures including pictures classified according to defect types after photographing keyboard key surface defects different from the sample pictures, the sample pictures and the detection pictures being stored in different folders respectively; A training module is configured to label the sample pictures according to defect types, pictures of the same defect type being labeled with the same defect name, and to train the sample pictures with completed labels by using a convolutional neural network to obtain an optimal keyboard key surface detection model; A detection module is configured to detect keyboard key surface defects by using the keyboard key surface detection model.

9. An electronic device, comprising: The device includes one or more processors and a memory configured to store one or more programs; The one or more programs, when executed by the processor, cause the processor to implement the method of any one of claims 1-7.

10. A computer readable storage medium storing a computer program, characterized in that, The computer program, when executed, implements the method of any one of claims 1-7.

Images