An online detection method for keyboard character defects based on computer vision

Abstract

The present invention relates to a computer vision-based online detection method for keyboard character defects, particularly in the field of pattern recognition or machine learning for images or videos. The method comprises: performing a fixed number of multiple trainings on a deep neural network to obtain an intelligent character defect detection model; and using the intelligent character defect detection model to intelligently detect whether a preset number of randomly selected adjacent keyboard characters contain one or more character keys with appearance defects based on a preset number of computer vision data and random sub-images. The present invention replaces the online detection mode of each keyboard product, which is performed on each character, by randomly selecting a different number of adjacent characters for each keyboard product as a whole, and performing intelligent online detection on whether the whole character set contains one or more characters with appearance defects, thereby reducing the various operating costs of the detection system and avoiding affecting the production efficiency of the keyboard product production line.

Description

Technical Field

[0001] The present invention relates to the field of pattern recognition or machine learning that is particularly applicable to images or videos, and in particular to an online detection method for keyboard character defects based on computer vision. Background Art

[0002] During the process of keyboard product production and piece-by-piece production on the keyboard product production line, the video acquisition mode and visual analysis mode can be used to perform online detection of defects in each keyboard character on each newly-produced keyboard product. In this way, online detection of defects in each keyboard character on each newly-produced keyboard product can be performed in a non-contact manner. For example, online detection of defects in the appearance of each keyboard character can be performed, thereby ensuring the quality of each keyboard product produced by the keyboard product production line and preventing inferior products from entering the market.

[0003] For example, Chinese invention patent publication CN106570511A proposes a computer vision-based online keyboard character detection system and method. The system includes an industrial camera, a computer, a single-chip microcomputer system, a frequency converter, a motor, and a conveyor belt. The operating steps of the method are: (1) using image grayscale and median filtering methods to improve image quality, (2) using surf feature points for image matching, (3) determining character positions, (4) marking characters with book sequence numbers, and (5) separating defective characters. The detection algorithm of the present invention adopts the SVM classification method, marking normal characters as positive samples and erroneous characters as negative samples, and extracting their size, center, and moment features for training. The present invention is simple to install and has high detection efficiency, and will play a huge role in the keyboard production process.

[0004] For example, Chinese invention patent publication CN110658194A proposes a keyboard detection method. When testing a keyboard to be tested, a first-level detection is first performed to determine whether the printing of each character in the keyboard to be tested meets a first condition. If not, the keyboard to be tested is determined to be defective and no further detection is performed. If the printing of each character in the keyboard to be tested meets the first condition, a second-level detection is performed to determine whether the setting position of each character on the key to be tested meets a second condition. Thus, a hierarchical detection method is adopted to improve the efficiency of screening out defective keyboards. Moreover, each step in the detection method provided in the embodiment of the present application can be executed by a hardware structure, which can realize intelligent online detection, and requires fewer templates, making it easy to promote and use on a large scale.

[0005] Obviously, the above-mentioned technical solutions in the prior art either adopt a visual detection mode or a non-visual detection mode to perform online detection on each character of each keyboard product to determine whether it is a defective character. This online detection mode of each keyboard product and each character requires a large amount of computing resources, time resources and power resources, while reducing the shipment speed of keyboard products and adversely affecting the production efficiency of the keyboard product production line. Summary of the Invention

[0006] In order to solve the technical defects in the prior art, the present invention provides an online detection method for keyboard character defects based on computer vision, which replaces the online detection mode of each keyboard product and each character one by one, and randomly selects different numbers of adjacent multiple characters for each keyboard product as a whole. Based on the overall computer vision data of this overall character set, the number of currently selected characters, and multiple imaging parameters of the visual imaging device, a customized character defect intelligent detection model is used to perform intelligent online detection on whether there is more than one character with appearance defects in this overall character set. Therefore, there is no need to perform online detection based on visual data for all characters of each keyboard product, which reduces the computing resources, time resources and power resources of the online detection system of the keyboard product production line, while improving the shipment speed of the keyboard products and ensuring the production efficiency of the keyboard product production line.

[0007] According to the present invention, a method for online detection of keyboard character defects based on computer vision is provided, the method comprising:

[0008] The latest keyboard product off the production line is used as the target keyboard. The front of the horizontally placed target keyboard is photographed in a top-down mode to obtain and output the corresponding top-down product image.

[0009] Identifying character image blocks respectively occupied by character keys corresponding to respective characters of the target keyboard based on key shape imaging features, taking a preset number of randomly selected adjacent character image blocks as a whole, and obtaining a sub-frame occupied by the whole in the overhead image of the product as a random sub-frame;

[0010] Using the grayscale values, vertical coordinate values, horizontal coordinate values, and depth of field values ​​corresponding to the pixels of the random sub-picture as computer vision data of the random sub-picture;

[0011] Performing a fixed number of multiple trainings on the deep neural network to obtain the deep neural network after the multiple trainings and outputting the deep neural network as a character defect intelligent detection model;

[0012] The character defect intelligent detection model is used to intelligently detect whether multiple character keys corresponding to the adjacent multiple character image blocks within the whole have one or more character keys with external defects based on the preset number, the computer vision data of the random sub-image, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead photography device performing the overhead photography operation;

[0013] Among them, the preset number, the computer vision data of the random sub-images, and the imaging focal length, resolution, exposure and aperture value of the visual overhead shooting device that performs the overhead shooting operation are synchronously input into the character defect intelligent detection model, and the character defect intelligent detection model is executed to obtain the character defect intelligent detection model output by the character defect intelligent detection model to obtain the character key appearance defect identification indicating whether there are more than one appearance defect in the multiple character keys corresponding to the multiple adjacent character image blocks within the whole.

[0014] It can be seen that the present invention has at least the following five outstanding substantive features:

[0015] Substantive Feature A: Multiple adjacent character keys are randomly selected for each newly launched keyboard product as the target keyboard. A customized artificial intelligence model is used to intelligently detect whether there is one or more character keys with appearance defects on the randomly selected adjacent character keys of the target keyboard based on multiple basic information including computer vision data. This eliminates the need to individually inspect each character key of each newly launched keyboard product for appearance defects. Instead, the randomly selected multiple character keys are treated as a whole for appearance defect inspection. This reduces the number of inspection operations while ensuring inspection effectiveness, thereby improving the speed and efficiency of appearance defect detection.

[0016] Substantive Feature B: The AI ​​model that performs intelligent detection of multiple adjacent character keys for overall appearance defects, i.e., the intelligent character defect detection model, is a deep neural network that has undergone multiple training sessions. The number of training sessions performed by the deep neural network is monotonically positively correlated with the total number of characters on the keyboard. This allows AI models with different structures to be constructed for different types of keyboards, ensuring the effectiveness and stability of the intelligent detection results.

[0017] Substantive Feature C: Multiple pieces of basic information, including computer vision data, are introduced to participate in the intelligent detection of whether adjacent character keys as a whole have appearance defects. The multiple pieces of basic information specifically include the number of randomly selected adjacent character keys, computer vision data of a random sub-screen composed of multiple character image blocks respectively occupied by the adjacent character keys, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead photography device performing the overhead photography operation. The computer vision data of the random sub-screen includes the grayscale values, vertical coordinate values, horizontal coordinate values, and imaging depth of field values ​​corresponding to each pixel point of the random sub-screen. The full and comprehensive screening of the above multiple pieces of basic information further ensures the effectiveness and stability of the intelligent detection results.

[0018] Substantive Feature D: During each training session of the deep neural network, a character key shape defect indicator (a symbol indicating whether one or more shape defects exist among multiple adjacent character keys on a previously discontinued keyboard product) is used as a single output of the deep neural network. Computer vision data of a random sub-image consisting of the number of adjacent character keys, the character image blocks occupied by each of the adjacent character keys, and the imaging focal length, resolution, exposure, and aperture value of a visual overhead camera device performing an overhead camera operation on the keyboard product are used as multiple inputs of the deep neural network to complete the training session, thereby ensuring the effectiveness of each training session of the deep neural network.

[0019] Substantive feature E: For a random sub-screen composed of multiple character image blocks respectively occupied by a plurality of randomly selected adjacent character keys, since the number of randomly selected adjacent character keys is different, the size of the random sub-screen under different selections is different. In order to ensure the normalization of the number of input data of the character defect intelligent detection model, the grayscale values, vertical coordinate values, horizontal coordinate values ​​and imaging depth of field values ​​corresponding to each pixel point of the random sub-screen are used as computer vision data of the random sub-screen, including: when the number of each pixel point of the random sub-screen is less than the set number threshold, the multiple grayscale values, vertical coordinate values, and imaging depth of field values ​​corresponding to the multiple pixel points of the difference between the two are used. A plurality of horizontal coordinate values ​​and a plurality of imaging depth of field values ​​are set to zero to obtain the grayscale values, vertical coordinate values, horizontal coordinate values ​​and imaging depth of field values ​​corresponding to each pixel point of the set quantity threshold, and replace the grayscale values, vertical coordinate values, horizontal coordinate values ​​and imaging depth of field values ​​corresponding to each pixel point of the random sub-picture as the computer vision data of the random sub-picture to complete the normalization of the number of pixels of the computer vision data of random sub-pictures of different sizes, thereby completing the digital construction of the character defect intelligent detection model as an artificial intelligence model, and improving the robustness and effectiveness of the character defect intelligent detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The embodiments of the present invention will be described below with reference to the accompanying drawings, in which:

[0021] Figure 1 The technical flow chart of a method for online detection of keyboard character defects based on computer vision according to the present invention.

[0022] Figure 2 The figure is a flowchart showing the steps of a method for online detection of keyboard character defects based on computer vision according to the first embodiment of the present invention.

[0023] Figure 3 The figure is a flowchart showing a method for online detection of keyboard character defects based on computer vision according to a second embodiment of the present invention.

[0024] Figure 4 The figure is a flowchart showing a method for online detection of keyboard character defects based on computer vision according to the third embodiment of the present invention.

[0025] Figure 5 The figure is a flowchart showing a method for online detection of keyboard character defects based on computer vision according to the fourth embodiment of the present invention.

[0026] Figure 6The figure is a flowchart showing a method for online detection of keyboard character defects based on computer vision according to the fifth embodiment of the present invention. DETAILED DESCRIPTION

[0027] like Figure 1 As shown, a technical flow chart of a method for online detection of keyboard character defects based on computer vision according to the present invention is given.

[0028] like Figure 1 As shown, the specific technical process of the present invention is as follows:

[0029] Technical Process 1: The latest keyboard product off the production line is used as the target keyboard to conduct targeted screening and obtain multiple basic information required for online intelligent detection of target keyboard character key defects;

[0030] Specifically, if Figure 1 As shown, the multiple pieces of basic information include computer vision data corresponding to a whole character key set consisting of a plurality of adjacent character keys randomly selected from the target keyboard;

[0031] More specifically, a top-down shooting mode is used to shoot the front of a horizontally placed target keyboard to obtain and output a corresponding top-down shot of the product. Based on the key shape imaging features, the character image blocks corresponding to the character keys of the target keyboard are identified, such as Figure 1 As shown, the randomly selected adjacent multiple character keys are visually represented as adjacent multiple character image blocks corresponding to the randomly selected adjacent multiple character keys, and a preset number of randomly selected adjacent multiple character image blocks are taken as a whole character key set. A sub-screen occupied by the whole character key set in the overhead image of the product is obtained as a random sub-screen, and each grayscale value, each vertical coordinate value, each horizontal coordinate value, and each imaging depth of field value corresponding to each pixel point of the random sub-screen are taken as computer vision data of the random sub-screen, that is, computer vision data corresponding to the whole character key set;

[0032] In addition, if Figure 1 As shown, the multiple basic information also includes a randomly selected preset number and an imaging focal length, resolution, exposure, and aperture value of the visual overhead shooting device that performs the overhead shooting operation. The imaging focal length, resolution, exposure, and aperture value of the visual overhead shooting device are multiple imaging parameters of the visual overhead shooting device;

[0033] In this way, comprehensive and targeted screening of the above-mentioned multiple basic information ensures random selection while improving the stability and reliability of intelligent detection;

[0034] Technical Process 2: Design a customized character defect intelligent detection model for online intelligent detection of target keyboard character key defects;

[0035] For example, the structural customization of the character defect intelligent detection model mainly manifests in the following three aspects:

[0036] First aspect: the character defect intelligent detection model is a deep neural network that has been trained multiple times;

[0037] For example, the deep neural network includes a single output layer, a single input layer, and a plurality of hidden layers, wherein the plurality of hidden layers are located between the single output layer and the single input layer;

[0038] Second aspect: The number of trainings performed by the deep neural network is monotonically positively correlated with the total number of characters on the keyboard, thereby constructing artificial intelligence models with different structures for different types of keyboards;

[0039] Thirdly, during each training session of the deep neural network, a character key shape defect indicator, indicating whether one or more shape defects exist among a plurality of adjacent character keys on a previously discontinued keyboard product, is used as a single output of the deep neural network. Computer vision data of a random sub-image consisting of the number of adjacent character keys, the character image blocks occupied by the adjacent character keys, and the imaging focal length, resolution, exposure, and aperture value of a visual overhead photography device performing an overhead photography operation on the keyboard product are used as multiple inputs of the deep neural network to complete this training, thereby ensuring the effectiveness of each training session of the deep neural network.

[0040] In this way, the multiple customized structural designs of the above-mentioned character defect intelligent detection model further ensure the randomness of selection while improving the stability and reliability of intelligent detection;

[0041] Technical Process 3: Using the multiple basic information targeted by Technical Process 1 and the customized character defect intelligent detection model designed according to Technical Process 2, a preset number of adjacent character images randomly selected from the target keyboard are divided into blocks as a whole to perform online intelligent detection on whether there is more than one character key with appearance defects;

[0042] Obviously, since the character key set as a whole in the target keyboard is randomly selected, it can reflect whether the target keyboard as a whole has a defective state of more than one character key with defective appearance;

[0043] Furthermore, based on the present invention, a method of randomly selecting multiple keyboard products from each keyboard product produced piece by piece can be adopted to further reduce various testing costs while ensuring the testing effect;

[0044] Technical process 4: Dynamically select the destination of the target keyboard based on the online intelligent detection results of technical process 3;

[0045] Specifically, if there is one or more character keys with appearance defects in the entire set of character keys to be intelligently inspected, the target keyboard is pushed into an inspection container located in the manual inspection area as a product with appearance defects;

[0046] Specifically, if there is no more than one character key with an appearance defect in the entire set of character keys being intelligently inspected, the target keyboard is pushed to the next inspection area as a product with a qualified appearance;

[0047] It can be seen from this that the present invention does not need to perform appearance defect detection operations on each character key of each newly produced keyboard product one by one. Instead, a plurality of randomly selected character keys are used as a whole to perform appearance defect detection operations. While ensuring the detection effect, the number of detection operations is reduced, and the speed and efficiency of appearance defect detection are improved.

[0048] The key points of the present invention are: a random selection mechanism for online intelligent detection of appearance defects of multiple adjacent character keys of the latest keyboard products as a whole, a customized structure of a character defect intelligent detection model for performing appearance defect detection operations on the randomly selected multiple adjacent character keys as a whole, and a sufficient and comprehensive screening of multiple basic data for performing appearance defect detection operations on the randomly selected multiple adjacent character keys as a whole.

[0049] Hereinafter, a computer vision-based online keyboard character defect detection method of the present invention will be specifically described in the form of an embodiment.

[0050] First embodiment

[0051] Figure 2 The figure is a flowchart showing the steps of a method for online detection of keyboard character defects based on computer vision according to the first embodiment of the present invention.

[0052] like Figure 2 As shown, the online detection method for keyboard character defects based on computer vision includes the following steps:

[0053] Step S1: taking the latest keyboard product off the production line as the target keyboard, performing a top-down shooting operation on the front of the horizontally placed target keyboard in a top-down shooting mode to obtain and output a corresponding top-down shooting image of the product;

[0054] Specifically, the latest keyboard product off the production line is used as the target keyboard, and a top-down shooting operation is performed on the front of the horizontally placed target keyboard in a top-down shooting mode to obtain and output a corresponding top-down shooting image of the product. The following includes: the keyboard product production line generally adopts a piece-by-piece production mode of keyboard products at a uniform speed. The detection speed of the online detection of keyboard character defects based on computer vision should be faster than or equal to the production speed of the keyboard products to avoid slowing down the production efficiency of the keyboard product production line;

[0055] Therefore, it can be seen that there is a need to improve the detection speed of the online detection system for keyboard character defects at the end of the keyboard product production line;

[0056] Step S2: identifying character image blocks respectively occupied by character keys corresponding to respective characters of the target keyboard based on the key shape imaging features, taking a preset number of randomly selected adjacent character image blocks as a whole, and obtaining a sub-frame occupied by the whole in the overhead image of the product as a random sub-frame;

[0057] For example, identifying character image blocks respectively occupied by character keys corresponding to respective characters of a target keyboard based on the key shape imaging features, taking a preset number of randomly selected adjacent character image blocks as a whole, and obtaining a sub-image occupied by the whole in the overhead image of the product as a random sub-image includes: the key shape imaging features may be a standard shape pattern corresponding to a single key;

[0058] Step S3: using the grayscale values, vertical coordinate values, horizontal coordinate values, and depth of field values ​​corresponding to the pixels of the random sub-image as computer vision data of the random sub-image;

[0059] Specifically, the grayscale values, vertical coordinate values, horizontal coordinate values, and depth of field values ​​corresponding to the pixels of the random sub-image are used as computer vision data of the random sub-image, including: the grayscale value corresponding to each pixel is in the range of 0-255;

[0060] Step S4: performing a fixed number of trainings on the deep neural network to obtain a deep neural network after the multiple trainings and outputting the deep neural network as a character defect intelligent detection model;

[0061] For example, the MATLAB toolbox can be used to perform a fixed number of trainings on the deep neural network to obtain the deep neural network after the trainings and use it as the output of the character defect intelligent detection model. Testing and simulation of the process;

[0062] Step S5: using the character defect intelligent detection model to intelligently detect whether there are at least one character key with an external shape defect in a plurality of character keys corresponding to the plurality of adjacent character image blocks within the whole according to the preset number, the computer vision data of the random sub-image, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead photography device performing the overhead photography operation;

[0063] Specifically, the character defect intelligent detection model is used to intelligently detect whether multiple character keys corresponding to the adjacent multiple character image blocks within the whole have more than one appearance defect based on the preset number, the computer vision data of the random sub-image, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead photography device that performs the overhead photography operation. Among the character keys, the values ​​of the different preset numbers corresponding to different keyboard products are randomly distributed. For example, a random function that changes with time can be designed to execute the selection of the number of adjacent multiple character keys of each newly-released keyboard product, that is, the selection of the value of the preset number;

[0064] The preset number, the computer vision data of the random sub-images, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead photography device performing the overhead photography operation are synchronously input into the character defect intelligent detection model, and the character defect intelligent detection model is executed to obtain the character key appearance defect identifier output by the character defect intelligent detection model, indicating whether a plurality of character keys corresponding to the plurality of adjacent character image blocks within the whole have one or more appearance defects;

[0065] The method further comprises: performing a fixed number of trainings on the deep neural network multiple times to obtain the deep neural network after the multiple trainings and outputting the deep neural network as the character defect intelligent detection model; and the method further comprises: the fixed number of values ​​being monotonically positively correlated with the total number of characters on the target keyboard;

[0066] For example, the fixed number of values ​​that are monotonically positively correlated with the total number of characters on the target keyboard include: when the total number of characters on the target keyboard is 26, the corresponding number of deep neural network training times, i.e., the fixed number of values, is 200; when the total number of characters on the target keyboard is 30, the corresponding number of deep neural network training times, i.e., the fixed number of values, is 300; when the total number of characters on the target keyboard is 36, the corresponding number of deep neural network training times, i.e., the fixed number of values, is 350, and so on;

[0067] The monotonically positive correlation between the fixed number of values ​​and the total number of characters on the target keyboard includes: using a numerical conversion formula to express a numerical conversion relationship in which the fixed number of values ​​and the total number of characters on the target keyboard are monotonically positively correlated;

[0068] And wherein, performing a fixed number of multiple trainings on the deep neural network to obtain the deep neural network after the multiple trainings and outputting it as a character defect intelligent detection model includes: when performing each training on the deep neural network, using the appearance defect identification of the character keys on a keyboard product that has been offline in the past, whether there is one or more appearance defects in the adjacent multiple character keys, as the single output content of the deep neural network, using the number of the adjacent multiple character keys, the computer vision data of the random sub-screen composed of the character image blocks respectively occupied by the adjacent multiple character keys as a whole, and the imaging focal length, resolution, exposure and aperture value of the visual overhead shooting device that performs an overhead shooting operation on the keyboard product as multiple input contents of the deep neural network to complete this training.

[0069] Second embodiment

[0070] Figure 3 The figure is a flowchart showing a method for online detection of keyboard character defects based on computer vision according to a second embodiment of the present invention.

[0071] like Figure 3 As shown, compared with Figure 2 Before taking the latest keyboard product off the production line as the target keyboard and performing a top-down photography operation on the front of the horizontally placed target keyboard in a top-down photography mode to obtain and output a corresponding top-down photography image of the product, that is, before step S1, the method further includes:

[0072] Step S6: Push the latest keyboard product off the production line onto the horizontal bracket to complete the horizontal placement of the latest keyboard product off the production line;

[0073] For example, an intelligent driving mechanism including a driving motor, a robotic arm, a positioning unit, and a micro-control unit can be used to complete the intelligent operation of pushing the latest keyboard product off the production line onto the horizontal bracket.

[0074] Third embodiment

[0075] Figure 4 The figure is a flowchart showing a method for online detection of keyboard character defects based on computer vision according to the third embodiment of the present invention.

[0076] like Figure 4 As shown, compared with Figure 2 After using the character defect intelligent detection model to intelligently detect whether there are at least one character key with an external shape defect in the plurality of character keys corresponding to the plurality of adjacent character image blocks within the whole according to the preset number, the computer vision data of the random sub-image, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead photography device performing the overhead photography operation, that is, after step S5, the method further includes:

[0077] Step S7: if there is one or more shape defects on the character keys corresponding to the adjacent character image blocks in the whole, the target keyboard is pushed into a detection container located in the manual detection area as a product with shape defects;

[0078] For example, when intelligently detecting that there are multiple character keys corresponding to multiple adjacent character image blocks within the whole having more than one character key with external defects, the target keyboard is pushed into the inspection container located in the manual inspection area as an externally defective product, including: the inspection container located in the manual inspection area is large enough to store a large number of keyboard products with various defects.

[0079] Fourth embodiment

[0080] Figure 5 The figure is a flowchart showing a method for online detection of keyboard character defects based on computer vision according to the fourth embodiment of the present invention.

[0081] like Figure 5 As shown, compared with Figure 2 After using the character defect intelligent detection model to intelligently detect whether there are at least one character key with an external shape defect in the plurality of character keys corresponding to the plurality of adjacent character image blocks within the whole according to the preset number, the computer vision data of the random sub-image, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead photography device performing the overhead photography operation, that is, after step S5, the method further includes:

[0082] Step S8: intelligently detecting that there are no character keys with more than one appearance defect corresponding to the plurality of adjacent character image blocks within the entire keyboard, and pushing the target keyboard to the next inspection area as a product with qualified appearance;

[0083] For example, when intelligently detecting that there are no character keys with more than one appearance defect among the multiple character keys corresponding to the multiple adjacent character image blocks within the whole, pushing the target keyboard as a product with qualified appearance to the next inspection area includes: the next inspection area can be an online inspection area for keyboard color defects.

[0084] Fifth embodiment

[0085] Figure 6 The figure is a flowchart showing a method for online detection of keyboard character defects based on computer vision according to the fifth embodiment of the present invention.

[0086] like Figure 6 As shown, compared with Figure 2After performing a fixed number of trainings on the deep neural network to obtain a deep neural network after the trainings and outputting the deep neural network as a character defect intelligent detection model, that is, after step S4, the method further includes:

[0087] Step S9: receiving the character defect intelligent detection model and completing model storage of the character defect intelligent detection model by storing various model parameters of the character defect intelligent detection model;

[0088] Specifically, a TF storage device or an MMC storage device may be selected to receive the character defect intelligent detection model and store various model parameters of the character defect intelligent detection model to complete the model storage of the character defect intelligent detection model.

[0089] Next, various embodiments of the present invention will be further described.

[0090] In each of the above embodiments, optionally, in the computer vision-based online keyboard character defect detection method:

[0091] Taking the respective grayscale values, the respective vertical coordinate values, the respective horizontal coordinate values, and the respective imaging depth of field values ​​corresponding to the respective pixel points of the random sub-picture as the computer vision data of the random sub-picture includes: when the number of the respective pixel points of the random sub-picture is less than a set number threshold, setting the respective grayscale values, the respective vertical coordinate values, the respective horizontal coordinate values, and the respective imaging depth of field values ​​corresponding to the plurality of pixel points of the difference number between the two to zero, so as to obtain the respective grayscale values, the respective vertical coordinate values, the respective horizontal coordinate values, and the respective imaging depth of field values ​​corresponding to the respective pixel points of the set number threshold, and replacing the respective grayscale values, the respective vertical coordinate values, the respective horizontal coordinate values, and the respective imaging depth of field values ​​corresponding to the respective pixel points of the random sub-picture as the computer vision data of the random sub-picture, so as to complete the normalization processing of the number of pixel points of the computer vision data of random sub-pictures of different sizes;

[0092] For example, a numerical simulation mode may be selected to complete the test and simulation of the normalization process of the number of pixels of computer vision data of random sub-pictures of different sizes;

[0093] Wherein, when the number of each pixel point of the random sub-picture is less than a set number threshold, a plurality of grayscale values, a plurality of vertical coordinate values, a plurality of horizontal coordinate values, and a plurality of imaging depth of field values ​​corresponding to a plurality of pixel points of the difference number between the two are set to zero values, so as to obtain each grayscale value, each vertical coordinate value, each horizontal coordinate value, and each imaging depth of field value corresponding to each pixel point of the set number threshold, and replace each grayscale value, each vertical coordinate value, each horizontal coordinate value, and each imaging depth of field value corresponding to each pixel point of the random sub-picture as the computer vision data of the random sub-picture, so as to complete the normalization processing of the number of pixels of the computer vision data of random sub-pictures of different sizes, including: the number of each pixel point of the random sub-picture is a first pixel point number, the set number threshold is a second pixel point number, and the difference between the first pixel point number and the second pixel point number is the difference between the two;

[0094] And wherein, when the number of each pixel point of the random sub-image is less than the set number threshold, the multiple grayscale values, multiple vertical coordinate values, multiple horizontal coordinate values ​​and multiple imaging depth of field values ​​corresponding to the multiple pixel points of the difference number between the two are set to zero values, so as to obtain the grayscale values, vertical coordinate values, horizontal coordinate values ​​and imaging depth of field values ​​corresponding to each pixel point of the set number threshold, and replace the grayscale values, vertical coordinate values, horizontal coordinate values ​​and imaging depth of field values ​​corresponding to each pixel point of the random sub-image as the random sub-image. The computer vision data of the picture, in order to complete the normalization processing of the number of pixel points of the computer vision data of random sub-pictures of different sizes, also includes: setting the number threshold of each pixel point corresponding to each grayscale value, each vertical coordinate value, each horizontal coordinate value and each imaging depth of field value, including each grayscale value, each vertical coordinate value, each horizontal coordinate value and each imaging depth of field value corresponding to each pixel point of the random sub-picture, and multiple grayscale values, multiple vertical coordinate values, multiple horizontal coordinate values ​​and multiple imaging depth of field values ​​corresponding to multiple pixel points of the difference between the two.

[0095] In each of the above embodiments, optionally, in the computer vision-based online keyboard character defect detection method:

[0096] When executing each training of the deep neural network, the appearance defect identification of the character keys of a keyboard product that has been offline in the past, which is known to have one or more appearance defects, is used as a single output content of the deep neural network, and the number of the adjacent character keys, the computer vision data of a random sub-screen composed of the character image blocks respectively occupied by the adjacent character keys as a whole, and the imaging focal length, resolution, exposure and aperture value of the visual overhead photography device that performs an overhead photography operation on the keyboard product are used as multiple input contents of the deep neural network. Completing this training includes: the computer vision data of the random sub-screen composed of the character image blocks respectively occupied by the adjacent character keys as a whole is the grayscale values, vertical coordinate values, horizontal coordinate values ​​and imaging depth of field values ​​corresponding to each pixel point of the random sub-screen composed of the character image blocks respectively occupied by the adjacent character keys as a whole;

[0097] In each training of the deep neural network, the appearance defect identification of a character key of a keyboard product that has been offline in the past, which is known to have one or more appearance defects, is used as a single output of the deep neural network. The computer vision data of a random sub-screen composed of the number of the adjacent character keys and the character image blocks respectively occupied by the adjacent character keys as a whole, and the imaging focal length, resolution, exposure and aperture value of a visual overhead photography device that performs an overhead photography operation on the keyboard product are used as multiple inputs of the deep neural network. Completing this training also includes: a random sub-screen composed of the character image blocks respectively occupied by the adjacent character keys as a whole;

[0098] And wherein, the random sub-screen composed of the character image blocks respectively occupied by the adjacent multiple character keys as a whole includes: performing a bird's-eye view operation on the certain keyboard product to obtain the corresponding historical bird's-eye view screen, identifying the character image blocks respectively occupied by the character keys corresponding to the respective characters of the certain keyboard product based on the appearance imaging features of the keys, taking the multiple character image blocks respectively corresponding to the adjacent multiple character keys as a whole, and obtaining the sub-screen occupied by the whole in the historical bird's-eye view screen as the random sub-screen.

[0099] And in each of the above embodiments, optionally, in the computer vision-based online keyboard character defect detection method:

[0100] Using a numerical conversion formula to express a numerical conversion relationship in which the fixed number of values ​​is monotonically positively correlated with the total number of characters on the target keyboard includes: in the numerical conversion formula, the total number of characters on the target keyboard is an input value of the numerical conversion formula;

[0101] And wherein, the numerical conversion relationship in which the fixed number of values ​​is monotonically positively associated with the total number of characters on the target keyboard is represented by a numerical conversion formula also includes: in the numerical conversion formula, the fixed number of values ​​corresponding to the total number of characters on the target keyboard is the output value of the numerical conversion formula.

[0102] In addition, in a computer vision-based online keyboard character defect detection method according to the present invention:

[0103] Synchronously inputting the preset number, the computer vision data of the random sub-pictures, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead shooting device that performs the overhead shooting operation into the character defect intelligent detection model, and executing the character defect intelligent detection model to obtain the character key appearance defect identification output by the character defect intelligent detection model, indicating whether a plurality of character keys corresponding to the plurality of adjacent character image blocks within the whole have more than one appearance defect, including: performing binary value conversion processing on the preset number, the computer vision data of the random sub-pictures, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead shooting device that performs the overhead shooting operation, and then synchronously inputting them into the character defect intelligent detection model;

[0104] For example, performing binary value conversion on the preset number, the computer vision data of the random sub-pictures, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead shooting device that performs the overhead shooting operation, and then synchronously inputting the binary value conversion on the preset number, the computer vision data of the random sub-pictures, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead shooting device that performs the overhead shooting operation, and then synchronously inputting the binary value conversion on the preset number, the computer vision data of the random sub-pictures, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead shooting device that performs the overhead shooting operation, to the character defect intelligent detection model using a synchronous driving component;

[0105] And wherein, the preset number, the computer vision data of the random sub-images, and the imaging focal length, resolution, exposure and aperture value of the visual overhead shooting device that performs the overhead shooting operation are synchronously input into the character defect intelligent detection model, and the character defect intelligent detection model is executed to obtain the character defect intelligent detection model output, which indicates whether there are more than one external defects in the multiple character keys corresponding to the adjacent multiple character image blocks within the whole. The shape defect identification of the character key also includes: the character defect intelligent detection model output, which indicates whether there are more than one external defects in the multiple character keys corresponding to the adjacent multiple character image blocks within the whole. The shape defect identification of the character key is represented in the form of a binary value.

[0106] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0107] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0108] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are protected by the present invention.

Claims

1. A computer vision-based online detection method for keyboard character defects, characterized in that: The method comprises: The latest keyboard product off the production line is used as the target keyboard. The front of the horizontally placed target keyboard is photographed in a top-down mode to obtain and output the corresponding top-down product image. Identifying character image blocks respectively occupied by character keys corresponding to respective characters of the target keyboard based on key shape imaging features, taking a preset number of randomly selected adjacent character image blocks as a whole, and obtaining a sub-frame occupied by the whole in the overhead image of the product as a random sub-frame; Using the grayscale values, vertical coordinate values, horizontal coordinate values, and depth of field values ​​corresponding to the pixels of the random sub-picture as computer vision data of the random sub-picture; Performing a fixed number of multiple trainings on the deep neural network to obtain the deep neural network after the multiple trainings and outputting the deep neural network as a character defect intelligent detection model; The character defect intelligent detection model is used to intelligently detect whether multiple character keys corresponding to the adjacent multiple character image blocks within the whole have one or more character keys with external defects based on the preset number, the computer vision data of the random sub-image, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead photography device performing the overhead photography operation; Among them, the preset number, the computer vision data of the random sub-images, and the imaging focal length, resolution, exposure and aperture value of the visual overhead shooting device that performs the overhead shooting operation are synchronously input into the character defect intelligent detection model, and the character defect intelligent detection model is executed to obtain the character defect intelligent detection model output by the character defect intelligent detection model to obtain the character key appearance defect identification indicating whether there are more than one appearance defect in the multiple character keys corresponding to the multiple adjacent character image blocks within the whole.

2. The computer vision-based online keyboard character defect detection method according to claim 1, wherein: Performing a fixed number of multiple trainings on the deep neural network to obtain a deep neural network after the multiple trainings and outputting the deep neural network as the character defect intelligent detection model further includes: the fixed number of values ​​is monotonically positively correlated with the total number of characters on the target keyboard; Among them, the monotonically positive correlation between the fixed number of values ​​and the total number of characters on the target keyboard includes: using a numerical conversion formula to express the numerical conversion relationship of the monotonically positive correlation between the fixed number of values ​​and the total number of characters on the target keyboard.

3. The computer vision-based online keyboard character defect detection method according to claim 2, wherein: Performing a fixed number of multiple trainings on the deep neural network to obtain the deep neural network after the multiple trainings and outputting it as a character defect intelligent detection model includes: when performing each training on the deep neural network, using the appearance defect identification of the character keys on a keyboard product that has been offline in the past, which is known to have more than one appearance defect, as the single output content of the deep neural network, using the number of the adjacent character keys, the computer vision data of the random sub-screen composed of the character image blocks respectively occupied by the adjacent character keys as a whole, and the imaging focal length, resolution, exposure and aperture value of the visual overhead shooting device that performs an overhead shooting operation on the keyboard product as multiple input contents of the deep neural network to complete this training.

4. The method for online detection of keyboard character defects based on computer vision according to claim 3, wherein: Before taking the latest keyboard product off the production line as the target keyboard and performing a top-down photography operation on the front of the horizontally placed target keyboard in a top-down photography mode to obtain and output a corresponding top-down photography image of the product, the method further includes: Push the latest keyboard product off the production line onto the horizontal bracket to complete the horizontal placement of the latest keyboard product off the production line.

5. The method for online detection of keyboard character defects based on computer vision according to claim 3, wherein: After intelligently detecting whether there is one or more character keys with appearance defects on a plurality of character keys corresponding to a plurality of adjacent character image blocks within the whole using the character defect intelligent detection model based on the preset number, the computer vision data of the random sub-images, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead photography device performing the overhead photography operation, the method further includes: During intelligent detection, if there is one or more character keys with appearance defects on the multiple character keys corresponding to the multiple adjacent character image blocks within the whole, the target keyboard is pushed into the detection container located in the manual inspection area as a product with appearance defects.

6. The method for online detection of keyboard character defects based on computer vision according to claim 3, wherein: After intelligently detecting whether there is one or more character keys with appearance defects on a plurality of character keys corresponding to a plurality of adjacent character image blocks within the whole using the character defect intelligent detection model based on the preset number, the computer vision data of the random sub-images, and the imaging focal length, resolution, exposure, and aperture value of the visual overhead photography device performing the overhead photography operation, the method further includes: During the intelligent detection, if there are no character keys with more than one appearance defect corresponding to the multiple adjacent character image blocks within the whole, the target keyboard will be pushed to the next detection area as a product with qualified appearance.

7. The method for online detection of keyboard character defects based on computer vision according to claim 3, wherein: After performing a fixed number of multiple trainings on the deep neural network to obtain a deep neural network after the multiple trainings and outputting the deep neural network as a character defect intelligent detection model, the method further includes: The character defect intelligent detection model is received and the model storage of the character defect intelligent detection model is completed by storing various model parameters of the character defect intelligent detection model.

8. The computer vision-based online keyboard character defect detection method according to any one of claims 3 to 7, characterized in that: Taking the respective grayscale values, the respective vertical coordinate values, the respective horizontal coordinate values, and the respective imaging depth of field values ​​corresponding to the respective pixel points of the random sub-picture as the computer vision data of the random sub-picture includes: when the number of the respective pixel points of the random sub-picture is less than a set number threshold, setting the respective grayscale values, the respective vertical coordinate values, the respective horizontal coordinate values, and the respective imaging depth of field values ​​corresponding to the plurality of pixel points of the difference number between the two to zero, so as to obtain the respective grayscale values, the respective vertical coordinate values, the respective horizontal coordinate values, and the respective imaging depth of field values ​​corresponding to the respective pixel points of the set number threshold, and replacing the respective grayscale values, the respective vertical coordinate values, the respective horizontal coordinate values, and the respective imaging depth of field values ​​corresponding to the respective pixel points of the random sub-picture as the computer vision data of the random sub-picture, so as to complete the normalization processing of the number of pixel points of the computer vision data of random sub-pictures of different sizes; Wherein, when the number of each pixel point of the random sub-picture is less than a set number threshold, a plurality of grayscale values, a plurality of vertical coordinate values, a plurality of horizontal coordinate values, and a plurality of imaging depth of field values ​​corresponding to a plurality of pixel points of the difference number between the two are set to zero values, so as to obtain each grayscale value, each vertical coordinate value, each horizontal coordinate value, and each imaging depth of field value corresponding to each pixel point of the set number threshold, and replace each grayscale value, each vertical coordinate value, each horizontal coordinate value, and each imaging depth of field value corresponding to each pixel point of the random sub-picture as the computer vision data of the random sub-picture, so as to complete the normalization processing of the number of pixels of the computer vision data of random sub-pictures of different sizes, including: the number of each pixel point of the random sub-picture is a first pixel point number, the set number threshold is a second pixel point number, and the difference between the first pixel point number and the second pixel point number is the difference between the two; Among them, when the number of each pixel point of the random sub-image is less than the set number threshold, the multiple grayscale values, multiple vertical coordinate values, multiple horizontal coordinate values ​​and multiple imaging depth of field values ​​corresponding to the multiple pixel points of the difference number between the two are set to zero values, so as to obtain the grayscale values, vertical coordinate values, horizontal coordinate values ​​and imaging depth of field values ​​corresponding to each pixel point of the set number threshold, and replace the grayscale values, vertical coordinate values, horizontal coordinate values ​​and imaging depth of field values ​​corresponding to each pixel point of the random sub-image as the random sub-image. The computer vision data of the surface is used to complete the normalization processing of the number of pixel points of the computer vision data of random sub-pictures of different sizes, and the normalization processing also includes: setting the number threshold of each pixel point corresponding to each grayscale value, each vertical coordinate value, each horizontal coordinate value and each imaging depth of field value, including each grayscale value, each vertical coordinate value, each horizontal coordinate value and each imaging depth of field value corresponding to each pixel point of the random sub-picture, and the difference between the two, and multiple grayscale values, multiple vertical coordinate values, multiple horizontal coordinate values ​​and multiple imaging depth of field values ​​corresponding to multiple pixel points.

9. The computer vision-based online keyboard character defect detection method according to any one of claims 3 to 7, wherein: When executing each training of the deep neural network, the appearance defect identification of the character keys of a keyboard product that has been offline in the past, which is known to have one or more appearance defects, is used as a single output content of the deep neural network, and the number of the adjacent character keys, the computer vision data of a random sub-screen composed of the character image blocks respectively occupied by the adjacent character keys as a whole, and the imaging focal length, resolution, exposure and aperture value of the visual overhead photography device that performs an overhead photography operation on the keyboard product are used as multiple input contents of the deep neural network. Completing this training includes: the computer vision data of the random sub-screen composed of the character image blocks respectively occupied by the adjacent character keys as a whole is the grayscale values, vertical coordinate values, horizontal coordinate values ​​and imaging depth of field values ​​corresponding to each pixel point of the random sub-screen composed of the character image blocks respectively occupied by the adjacent character keys as a whole; In each training of the deep neural network, the appearance defect identification of a character key of a keyboard product that has been offline in the past, which is known to have one or more appearance defects, is used as a single output of the deep neural network. The computer vision data of a random sub-screen composed of the number of the adjacent character keys and the character image blocks respectively occupied by the adjacent character keys as a whole, and the imaging focal length, resolution, exposure and aperture value of a visual overhead photography device that performs an overhead photography operation on the keyboard product are used as multiple inputs of the deep neural network. Completing this training also includes: a random sub-screen composed of the character image blocks respectively occupied by the adjacent character keys as a whole; Among them, the random sub-screen composed of the character image blocks respectively occupied by the adjacent multiple character keys as a whole includes: performing a bird's-eye view operation on the certain keyboard product to obtain the corresponding historical bird's-eye view screen, identifying the character image blocks respectively occupied by the character keys corresponding to the various characters of the certain keyboard product based on the appearance imaging features of the keys, taking the multiple character image blocks respectively corresponding to the adjacent multiple character keys as a whole, and obtaining the sub-screen occupied by the whole in the historical bird's-eye view screen as the random sub-screen.

10. The computer vision-based online keyboard character defect detection method according to any one of claims 3 to 7, wherein: Using a numerical conversion formula to express a numerical conversion relationship in which the fixed number of values ​​is monotonically positively correlated with the total number of characters on the target keyboard includes: in the numerical conversion formula, the total number of characters on the target keyboard is an input value of the numerical conversion formula; Among them, the numerical conversion relationship in which the fixed number of values ​​is monotonically positively associated with the total number of characters on the target keyboard is represented by a numerical conversion formula also includes: in the numerical conversion formula, the fixed number of values ​​corresponding to the total number of characters on the target keyboard is the output value of the numerical conversion formula.

Images