Image processing method and device, electronic equipment and storage medium

Abstract

This disclosure provides an image processing method, apparatus, electronic device, and storage medium, relating to the field of image processing technology. The method includes: acquiring a first keyboard image and a second keyboard image, wherein the first keyboard image is a keyboard image obtained by converting the image format of the second keyboard image, and both the first and second keyboard images include multiple key objects; extracting actual key object information of the key objects from the first keyboard image based on pixel features; extracting reference key object information of the key objects from the second keyboard image based on pixel features; and determining whether the first keyboard image has defects based on the actual key object information and the reference key object information. This allows for accurate and effective detection of defects in the keyboard image.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device and storage medium. Background Technology

[0002] During the design process of keyboard images, the format of the keyboard images generated by modeling differs from common image formats, making it difficult to intuitively determine whether they meet layer specification requirements. They typically need to be converted to common image formats for inspection. However, errors frequently occur during format conversion, and existing image detection methods largely rely on manual review, which is not only time-consuming and labor-intensive but also prone to missed detections, resulting in low detection accuracy. Summary of the Invention

[0003] This disclosure provides an image processing method, apparatus, electronic device, and storage medium to at least solve the above-mentioned technical problems existing in the prior art.

[0004] In a first aspect, embodiments of this disclosure provide an image processing method, the method comprising:

[0005] Acquire a first keyboard image and a second keyboard image. The first keyboard image is obtained by converting the image format of the second keyboard image. Both the first keyboard image and the second keyboard image include multiple key objects. Based on the pixel features of the first keyboard image, the actual key object information of the key object is extracted from the first keyboard image; Based on the pixel features of the second keyboard image, reference key object information of the key object is extracted from the second keyboard image; Based on the actual key object information and reference key object information of the key object, it is determined whether the first keyboard image has defects.

[0006] Secondly, embodiments of this disclosure provide an image processing apparatus, the apparatus comprising: The acquisition module is used to acquire a first keyboard image and a second keyboard image. The first keyboard image is a keyboard image obtained by converting the image format of the second keyboard image. Both the first keyboard image and the second keyboard image include multiple key objects. The extraction module is used to extract the actual key object information of the key object from the first keyboard image based on the pixel features of the first keyboard image; The extraction module is used to extract reference key object information of the key object from the second keyboard image based on the pixel features of the second keyboard image; The determination module is used to determine whether the first keyboard image has defects based on the actual key object information and reference key object information of the key object.

[0007] Thirdly, embodiments of this disclosure provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the image processing method of the first aspect.

[0008] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the image processing method according to the first aspect.

[0009] Based on the image processing method provided in this disclosure, it is possible to acquire a first keyboard image to be detected after format conversion and a second keyboard image as a reference, and extract actual key object information and reference key object information based on the pixel features of the first keyboard image and the second keyboard image, respectively. Then, by comparing the actual key object information with the reference key object, it is possible to accurately and efficiently detect whether there are defects in the first detection image.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] Figure 1 This is a scenario example in a related technology provided by an embodiment of this disclosure. Figure 1 ; Figure 2 This is a scenario example in a related technology provided by an embodiment of this disclosure. Figure 2 ; Figure 3 This is a scenario example in a related technology provided by an embodiment of this disclosure. Figure 3 ; Figure 4 This is a schematic flowchart of an image processing method provided in an embodiment of this disclosure; Figure 5 This is a scenario example of an image processing method provided in this disclosure. Figure 1 ; Figure 6 This is a scenario example of an image processing method provided in this disclosure. Figure 2 ; Figure 7(a) is a scenario example of an image processing method provided in an embodiment of this disclosure. Figure 3 ; Figure 7(b) is a scenario example of an image processing method provided in an embodiment of this disclosure. Figure 4 ; Figure 8 This is a scenario example of an image processing method provided in this disclosure. Figure 5 ; Figure 9(a) is a scenario example of an image processing method provided in an embodiment of this disclosure. Figure 6 ; Figure 9(b) is a scenario example of an image processing method provided in an embodiment of this disclosure. Figure 10 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this disclosure; Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0012] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0013] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0014] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0015] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0016] As described in the background section, keyboard images are prone to conversion errors during format conversion, and there are many different keyboard image formats. Existing image detection methods still rely on manual review, which is not only time-consuming and labor-intensive, but also makes it difficult to identify subtle flaws in the images. For example, ... Figure 1 As shown, the actual size of the button object is deformed relative to the standard size. If the deformation is small, the human eye cannot accurately determine whether deformation exists. Furthermore, blind key markings usually contain connected components, and the sizes of these connected components drawn by different designers vary, making it difficult for the human eye to judge whether the blind key markings conform to specifications. Figure 2 As shown. Furthermore, while the human eye can make a preliminary judgment about whether a button has shifted position, it cannot accurately determine the degree of shift, such as... Figure 3 As shown.

[0017] Based on this, the present disclosure provides an image processing method, apparatus, device, and medium to at least solve the technical problem in the prior art that it is difficult to accurately and efficiently detect whether there are defects in keyboard images.

[0018] It should be noted that the image processing method provided in this disclosure can be executed by an image processing device or a control module within that image processing device. This disclosure uses an image processing device executing the image processing method as an example to illustrate the image processing method provided in this disclosure.

[0019] The image processing method provided in this disclosure will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] Figure 4 This is one of the schematic flowcharts of an image processing method provided in this embodiment.

[0021] like Figure 4 As shown, the entity executing this image processing method can be an image processing device. Based on this, the image processing method can specifically include the following steps: S410, acquire the first keyboard image and the second keyboard image.

[0022] The first keyboard image and the second keyboard image can be images obtained based on the same keyboard. Specifically, the first keyboard image can be a keyboard image obtained by converting the image format of the second keyboard image; no specific limitation is made here.

[0023] In addition, both the first keyboard image and the second keyboard image can include multiple key objects, without specific limitations here.

[0024] S420: Based on the pixel features of the first keyboard image, extract the actual key object information of the key object from the first keyboard image.

[0025] The actual key object information of the key object can refer to the actual key object information to be detected extracted from the first keyboard image.

[0026] S430 extracts reference key object information of the key object from the second keyboard image based on the pixel features of the second keyboard image.

[0027] The reference key object information of the key object can refer to the key object information extracted from the second keyboard image for benchmark comparison.

[0028] S440, based on the actual key object information and the reference key object information, determine whether there is a defect in the first keyboard image.

[0029] Specifically, the image processing device can acquire a first keyboard image and a second keyboard image. The first keyboard image can be obtained by converting the image format of the second keyboard image, and both the first and second keyboard images can include multiple key objects. Based on this, the image processing device can extract the actual key object information of the key object from the first keyboard image based on its pixel features, and simultaneously extract the reference key object information of the key object from the second keyboard image based on its pixel features. Furthermore, it can compare the actual key object information and the reference key object information to determine whether the first keyboard image obtained after format conversion has any defects.

[0030] In one example, with Figure 5 Taking the keyboard image shown as an example (i.e., the first keyboard image or the second keyboard image) as an example, after acquiring the first and second keyboard images, the outer contours of the acquired first and second keyboard images can be searched, and the image boundaries of the first and second keyboard images can be shrunk respectively to identify multiple objects in the first and second keyboard images. Then, all extracted outer contours are filled with white areas to obtain the following result: Figure 6 The image shown has each white area corresponding to a pixel area of ​​a button object.

[0031] Based on this, a connected component analysis algorithm (such as connectedComponentsWithStats) can be used to traverse each white independent connected component (i.e., the pixel area of ​​the aforementioned key object), extracting the width and height of the bounding rectangle of each connected component. The width and height of the bounding rectangle of each connected component can be recorded using preset variables avaKeyBoradRegionWidth and avaKeyBoradRegionHeight, thereby obtaining the size information of each key object. Additionally, the position information of each key object can be recorded. For example, the keyboard image can be divided into four regions: top, bottom, left, and right (e.g., the top corresponds to the row containing the F key, the left and right sides are the areas on either side of the keyboard, and the bottom includes the arrow keys, etc.). This region division is only an example and not specifically limited. Based on this, the coordinates of the top-left corner and the width _Topx of each key object in the top region, and the coordinates of the top-left corner and the height _leftEdgeRegion of each key object in the left region can be recorded. The coordinates of the top right corner and the height (_Righty) of each button object in the right area are recorded. Since there may be protruding arrow keys in the lower area, the coordinates of the bottom left corner and the width (_Shadowx) of the bottom edge button object that is level with the bottom left corner of the last button object in the left area are recorded to obtain the position information of each button object.

[0032] Therefore, actual key object information, including the actual position and actual size of the key element, is extracted from the first keyboard image; reference key object information, including the reference position and reference size of the key element, is extracted from the second keyboard image, which facilitates subsequent defect detection of the first detection image.

[0033] Based on the image processing method provided in this disclosure, it is possible to acquire a first keyboard image to be detected after format conversion and a second keyboard image as a reference, and extract actual key object information and reference key object information based on the pixel features of the first keyboard image and the second keyboard image, respectively. Then, by comparing the actual key object information with the reference key object, it is possible to accurately and efficiently detect whether there are defects in the first detection image.

[0034] In order to provide a comprehensive and detailed description of the image processing method provided in the embodiments of this disclosure, in one embodiment, the image processing method provided in the embodiments of this disclosure may further include the following steps: The first keyboard image is divided into multiple keyboard image regions.

[0035] Each of the aforementioned keyboard image areas may include at least one key object, without specific limitations here.

[0036] Based on this, the above-mentioned S440 may specifically include the following steps: Determine the defect detection rules corresponding to the keyboard image area; According to the defect detection rules corresponding to the keyboard image area, the actual key object information and reference key object information of the key objects in the keyboard image area are processed to determine whether there are defects in the keyboard image area.

[0037] The defect detection rules corresponding to the aforementioned keyboard image area can be determined based on the distribution of each key object within that area, and are used to detect whether defects exist in the keyboard image area. For example, the defect detection rules may include the detection of the number, position, and size of key objects. Taking key count detection as an example, it can be determined whether the number of keys within the keyboard image area meets a preset threshold; however, further limitations are not specified here.

[0038] In this way, the image processing device can divide the first keyboard image into multiple keyboard image regions and determine the defect detection rules corresponding to each keyboard image region. Then, for each keyboard image region, according to the defect detection rules corresponding to the keyboard image region, it can process the actual key object information and reference key object of the key object in the keyboard image region to determine whether there is a defect in the keyboard image region.

[0039] In this embodiment, the first keyboard image can be divided into multiple keyboard image regions, and the key objects in the corresponding keyboard image regions can be detected according to defect detection rules based on the key distribution in different keyboard image regions, so as to accurately determine whether there are defects in the keyboard image region. In this way, differentiated defect detection based on the distribution of key objects in each keyboard image region can be achieved, effectively improving the accuracy of defect detection.

[0040] Based on this, in one embodiment, the step of dividing the first keyboard image to obtain multiple keyboard image regions may specifically include: Based on the actual key object information of the key object, the first keyboard image is divided into multiple keyboard image regions according to the preset first region division rules.

[0041] The aforementioned preset first region division rule can be determined according to the actual situation and is used to divide multiple keyboard image regions. No specific limitation is made here.

[0042] Specifically, the image processing device can divide the first keyboard image into multiple keyboard image regions based on the actual key object information of the key object and according to a preset first region division rule.

[0043] In one example, the image processing device can calculate the typical values ​​of the width and height of the key object using statistical methods, and use them as the reference width and reference height of the standard letter key, thereby determining the letter key area from the first keyboard image area. Specifically, the vectors avaKeyBoradRegionWidth and avaKeyBoradRegionHeight are first sorted to obtain the element (i.e., width or height) that appears consecutively most frequently. In practical applications, images are prone to noise after encoding transmission and cross-platform decoding, resulting in pixel-level local differences. Therefore, when searching for the mode, an error of 1-2 pixels is allowed as the basis for judging the same element, as shown in the following formula (1): (1) Among them, if If the standard width of the letter keys is given, then... This is to obtain the actual width of the corresponding letter key. Correspondingly, if If the standard height of the letter keys is given, then... This is to obtain the actual height of the corresponding letter key.

[0044] Thus, the final determined typical values ​​are denoted as avaKeyW and avaKeyH. Based on these typical values, the letter key area can be accurately located, and the remaining keyboard image areas can be further determined according to the relative positional relationship between the letter key area and other keyboard areas, thereby realizing the area division of the first keyboard image.

[0045] In another example, the image processing device can pre-divide the first keyboard image based on its actual size and the relative positional relationship between each keyboard image region in the keyboard image. Then, based on the actual key information of the key objects, it can calculate the coverage between each key object and each preset image region, determine the final actual image region, and thus realize the region division of the first keyboard image.

[0046] In this embodiment, the first keyboard image can be divided into regions based on the actual key object information of the key objects and according to the preset first region division rule. This allows for region division based on the distribution of key objects in each keyboard image region, avoiding detection errors caused by arbitrary division of the keyboard image and effectively improving the accuracy of image detection.

[0047] In another embodiment, the step of dividing the first keyboard image into multiple keyboard image regions may specifically include: Identify the key identification information of the key object, and based on the key identification information, divide the first keyboard image into multiple keyboard image regions according to the preset second region division rule.

[0048] The key identification information of the key object can be related information used to identify different key objects. For example, if the key object is the A key on the keyboard, the corresponding key object identification information is "A". This will not be elaborated on further here.

[0049] In addition, the aforementioned preset second region division rule can be a rule used to divide multiple keyboard image regions, depending on the actual situation. It is different from the aforementioned preset first region division rule, and no specific limitation is made here.

[0050] Specifically, it is possible to identify the key identification information of each key object in the first keyboard image, and based on the key identification information of each key object, divide the first keyboard image into multiple keyboard image regions according to a preset second region division rule.

[0051] In this embodiment, the first keyboard image can be divided into regions by recognizing the key identification information of the key objects and according to the preset second region division rule. In this way, the region division can be performed based on the semantics of each key object, which effectively improves the accuracy of region division and facilitates a significant improvement in the accuracy of subsequent region detection.

[0052] In order to accurately detect whether there are defects in the first detection image, in some embodiments, the above-mentioned actual button object information may include the actual position and / or actual size of the button element, and correspondingly, the reference button object information may include the reference position and / or reference size of the button element.

[0053] Based on this, the above S140 may include the following steps: Based on the actual position and reference position of the button element, determine whether the button element has a positional offset defect; And / or, Based on the actual size and reference size of the button element, determine whether the button object has a deformation defect.

[0054] Thus, if the actual button object information can include the actual position of the button element, the corresponding reference button object information can include the reference position of the button element. In this way, the image processing device can determine whether the button object has a positional offset defect based on the actual position and reference position of the button element.

[0055] Alternatively, if the actual button object information may include the actual dimensions of the button object's features, then the corresponding reference object information of the button object may include the reference dimensions of the button features. In this way, the image processing device can determine whether the button object has a deformation defect based on the actual dimensions and reference dimensions of the button object's features.

[0056] Alternatively, if the actual button object information can include the actual position and actual size of the button object's features, then the corresponding reference object information can include the reference position and reference size of the button object's features. In this way, the image processing device can determine whether the button object has a positional offset defect based on the actual position and reference position of the button object's features. Simultaneously, it can determine whether the button object has a deformation defect based on the actual size and reference size of the button object's features.

[0057] In this embodiment, since the actual button object information may include the actual position and / or actual size of the button element, the corresponding reference button element information may include the reference position and / or reference size of the button element. Based on this, by comparing the actual position and reference position of the button element, it can be determined whether the button object has a positional offset defect; and / or, by comparing the actual size and reference size of the button element, it can be determined whether the button object has a deformation defect. This achieves accurate determination of positional offset defects and / or deformation defects of the button object, effectively improving the efficiency and accuracy of defect detection.

[0058] Since the key elements in this embodiment may include keycaps and blind key markings, and the detection methods for displacement offset defects corresponding to different key elements are different, in order to comprehensively and accurately detect the position offset defects of key elements, in one embodiment, if the key element is a keycap, the step of determining whether the key element has a position offset defect based on the actual position and reference position of the key element may specifically include: A button object is considered to have a positional offset defect if its button elements meet at least one of the following conditions: The positional difference between the actual position of the keycap element and the reference position of the element is greater than the first preset positional difference; The difference between the actual key spacing and the reference key spacing between any two adjacent keycaps is greater than the preset spacing difference.

[0059] In some embodiments, any two adjacent keycaps may include the keycap; the actual key spacing is determined based on the actual position of the elements of any two adjacent keycaps, and the reference key spacing is determined based on the reference position of the elements of any two adjacent keycaps.

[0060] In addition, the first preset position difference can be a position threshold used to determine the position of a single keycap, depending on the actual situation. The preset spacing difference can be a spacing threshold used to determine the position of at least two adjacent keycaps, depending on the actual situation, and is not specifically limited here.

[0061] If the key elements can include keycaps, then for a single keycap, the image processing device can determine whether the position difference between the actual position of the keycap element and the reference position of the element is greater than a first preset position difference. If it is greater, then it is determined that the corresponding key object has a position offset defect.

[0062] And / or, For any two adjacent keycaps, the image processing device can determine the actual key spacing between them based on the actual positions of their features. Simultaneously, it can determine a reference key spacing based on the reference dimensions of the features of the two keycaps. Thus, the image processing device can determine whether the difference between the actual key spacing and the reference key spacing is greater than a preset difference. If it is greater, it can determine that the keycap has a positional misalignment defect.

[0063] In one example, the positional difference between the actual position and the reference position of the keycap element is greater than a first preset positional difference, satisfying the following formula: (2) in, For the first The actual position of each keycap element. For the first The reference position of each keycap. This is the first preset position difference.

[0064] The difference between the actual key spacing and the reference key spacing between any two adjacent keycaps is greater than the preset spacing difference, satisfying the following formula: (3) in, For the first The actual position of the elements of the first keycap is related to the first keycap. The actual key spacing is determined by the actual position of each keycap element. For the first The reference position of the keycap elements is the same as the first keycap. The reference key spacing is determined by the reference position of each keycap element. This is the preset spacing difference. The keycap and the first Each keycap is an adjacent keycap.

[0065] In addition, since the first keycap of the first row in the right region (_Righty) of the keyboard image is horizontally aligned with the first keycap in the top left corner of the left region (_LeftEdgeRegion), and the horizontal offset between the last keycap in the bottom right corner of the right region (_Righty) and the last keycap in the bottom left corner of the left region (_LeftEdgeRegion) does not exceed 0.5 times the standard key height, based on the above positional constraints, positional displacement defects can also be detected using the following constraints, as shown in the following formula: (4) (5) Here, Righty.begin is the starting coordinate of the first keycap in the first row of the right region, _LeftEdgeRegion.begin is the starting coordinate of the first keycap in the left region, _LeftEdgeRegion.end is the ending coordinate of the last keycap in the left region, and _Righty.end is the ending coordinate of the last keycap in the right region. It should be noted that the units of the coordinates mentioned above are pixel values, which will not be elaborated on here.

[0066] In this embodiment, by comparing the actual position of a single keycap with a reference position, and by comparing the actual key spacing of adjacent keycaps with a reference key spacing, accurate detection of keycap position misalignment defects can be achieved. This not only detects the absolute positional deviation of a single keycap but also identifies abnormal relative spacing between adjacent keycaps, effectively improving the accuracy of position misalignment defect detection.

[0067] In another embodiment, if the key element includes a blind key identifier, then the step of determining whether the key element has a positional offset defect based on the actual position and reference position of the key element may specifically include the following steps: A button object is considered to have a positional offset defect if its button elements meet at least one of the following conditions: The positional difference between the actual position and the reference position of the element marked by the blind key is greater than the second preset positional difference; The distance deviation of the blind key marker relative to the target keycap is greater than the preset deviation.

[0068] In some embodiments, a blind key marker may be provided on the target keycap. Furthermore, the aforementioned distance deviation may be the difference between the actual distance of the blind key marker relative to the target keycap and a reference distance. The actual distance may be determined based on the actual positions of the elements of the blind key marker and the target keycap, and the reference distance may be determined based on the reference positions of the elements of the blind key marker and the target keycap. It should be noted that if the key element includes a blind key marker, the actual position of the element of the blind key marker may include the actual position of the marker area corresponding to the blind key marker. Correspondingly, the reference position of the element of the blind key marker may include the reference position of the marker area corresponding to the blind key marker; this will not be elaborated further here.

[0069] In addition, the aforementioned second preset position difference can be determined according to the actual situation and is used to detect the absolute positional deviation of the blind key icon. The aforementioned preset deviation can be determined according to the actual situation and is used to detect the distance deviation of the blind key icon relative to the corresponding target keycap; no specific limitation is made here.

[0070] If the button elements may include blind key icons, the image processing device can determine whether the position difference between the actual position of the blind key icon and the reference position of the icon is greater than a second preset position difference. If it is greater, it can be determined that the corresponding button object has a position offset defect.

[0071] And / or, the image processing device can determine the actual distance between the blind key icon and the target keycap based on the actual positions of their respective elements, and simultaneously determine a reference distance between them based on the reference positions of their respective elements. Furthermore, it can determine the distance deviation of the blind key icon relative to the target keycap based on the actual distance and reference distance between the blind key icon and the corresponding target keycap, and then determine whether the distance deviation of the blind key icon relative to the target keycap is greater than a preset deviation. If it is greater, it is determined that a positional offset defect has occurred in the key object.

[0072] In one example, the positional difference between the actual position and the reference position of the feature identified by the blind key is greater than the second preset positional difference, satisfying the following formula: (6) in, The actual location of the elements marked by the blind key. This is a reference location for the actual position of the blind key markings. This is the second preset position difference.

[0073] The distance deviation of the blind key marker relative to the target keycap mentioned above is greater than a preset deviation, satisfying the following formula: (7) in, To indicate the actual distance of the blind keypad relative to the target keycap. The reference distance for blind key markers relative to the target keycap. The distance deviation of the blind key marker relative to the target keycap. This is the preset deviation.

[0074] It should also be noted that the image processing device can locate the key objects in the fourth row (the row containing the blind keys) of the keyboard image based on _leftEdgeRegio, and extract all the key objects in the fourth row into a set 'keys'. Each key object in the set 'keys' is then inverted, and connected component analysis is performed, retaining only the connected component with the largest area, and then inverted again. This yields the images shown in Figures 7(a) and 7(b).

[0075] Based on this, considering that the center of the blind keyprint marking area should be near the central axis of the keycap in the lower 1 / 3 of the keycap, the relative position of the blind keyprint relative to the target keycap must also satisfy the relationship shown in the following formula: (8) (9) in, The X-axis position of the center point of the marking area for the blind keypad. The width of the target keycap. The Y-axis position of the center point of the label area for blind key markings. The width of the target keycap.

[0076] In addition, since the actual location of a blind key marker can also include the actual location of the connected region corresponding to the blind key marker, and the reference location of a blind key marker can also include the reference location of the connected region corresponding to the blind key marker, it is possible to obtain the complete set `marks` containing blind key markers for the connected region corresponding to the blind key marker. Each image in the set `marks` is as follows: Figure 8 As shown, a connected component analysis is performed to determine the label region corresponding to the blind key label. The width w, height h, and center position center of the label region corresponding to the blind key label are recorded.

[0077] Next, the image processing device is able to... Figure 8 The image is inverted to make... Figure 8 The black areas in the diagram are transformed into white areas, and the white areas are transformed into black areas. For a qualified blind key mark, there should be a solid black line inside, as shown in Figure 9(a), and after inversion, it should be a solid white line, as shown in Figure 9(b).

[0078] Connectivity analysis is performed on the corresponding region of the inverted tactile key marker to obtain the width and height information (state) and center position information (centroid) of the connected components. All white areas within the tactile key marker are located, and their shape and number are evaluated. If the evaluation passes, the connected component should be located at the center of the entire tactile key marker, with a vertical displacement of no more than one pixel. That is, the following conditions must be met: (10) in, This indicates the position of the connected component corresponding to the blind key on the X-axis.

[0079] In this embodiment, by comparing the actual position of the blind key icon itself with its reference position, and by judging the distance deviation of the blind key icon relative to the target keycap, accurate detection of blind key icon position offset defects can be achieved. Thus, not only can the absolute positional deviation of the blind key icon itself be detected, but also its relative positional anomalies relative to the keycap can be identified, effectively improving the accuracy of position offset defect detection.

[0080] In the image processing method provided in this disclosure, the deformation detection method is the same for different button elements. Therefore, in order to accurately detect deformation defects in the button object, in one embodiment, the actual size of the element may include an actual size in a first direction and a size in a second direction, and correspondingly, a first-direction reference size and a second-direction reference size. Wherein, the first-direction actual size and the second-direction actual size are the actual sizes of the button element in the first and second directions, respectively, and the first-direction reference size and the second-direction reference size are the reference sizes of the button element in the first and second directions, respectively, and the first and second directions are perpendicular to each other.

[0081] Based on this, the steps described above for determining whether a button object has a deformation defect based on the actual size and reference size of the button element can specifically include: A button object is considered to have a deformation defect if its button elements satisfy at least one of the following conditions: The first size difference between the actual size of the button element in the first direction and the reference size in the first direction is greater than a first preset size threshold. The difference between the actual size of the button element in the second direction and the reference size in the second direction is greater than the second preset size threshold. The difference between the actual size ratio and the reference size ratio of the button element is greater than the preset size ratio; In some embodiments, the actual size ratio is determined based on the actual size in the first direction and the actual size in the second direction, and the reference size ratio is determined based on the reference size in the first direction and the reference size in the second direction.

[0082] In addition, the aforementioned first preset size threshold can be determined according to the actual situation and is used to detect the size of the button element in the first direction. The aforementioned second preset size threshold can be determined according to the actual situation and is used to detect the size of the button element in the second direction. The preset size ratio difference can be determined according to the actual situation and is used to detect the size ratio of the button element; no specific limitation is made here.

[0083] Specifically, the image processing device can determine whether the first size difference between the actual size of the button element in the first direction and the reference size in the first direction is greater than a first preset size threshold. If it is greater, it determines that the button object has a deformation defect.

[0084] And / or, the image processing device can determine whether the second size difference between the actual size of the button element in the second direction and the reference size in the second direction is greater than a second preset size threshold. If it is greater, then it is determined that the button object has a deformation defect.

[0085] And / or, the image processing device can determine the actual size ratio based on the actual size in the first direction and the actual size in the second direction, and can simultaneously determine the reference size ratio based on the reference size in the first direction and the reference size in the second direction. In this way, it can determine whether the difference between the actual size ratio and the reference size ratio corresponding to the button element is greater than a preset size ratio; if it is greater, it is determined that the button object has a deformation defect.

[0086] In one example, the first size difference between the actual size of the button element in the first direction and the reference size in the first direction is greater than a first preset size threshold, satisfying the following formula: (11) in, The actual dimension of the button element in the first direction. Between the first directional reference dimensions of the button elements, The first preset size threshold.

[0087] The difference between the actual size of the button element in the second direction and the reference size in the second direction is greater than a second preset size threshold, satisfying the following formula: (12) in, The actual dimension of the button element in the first direction. Between the first directional reference dimensions of the button elements, The first preset size threshold.

[0088] If the difference between the actual size ratio and the reference size ratio of the above button elements is greater than the preset size ratio, and the following formula is satisfied: (13) in, This refers to the actual size proportions of the button elements. The reference size scale for button elements. This is a preset size ratio.

[0089] Or it satisfies the following formula: (14) in, This refers to the actual size proportions of the button elements. The reference size scale for button elements. This is a preset size ratio.

[0090] Based on this, taking letter key objects as an example, the reference size ratio of the letter key object is 1. Therefore, to determine whether the difference between the actual size ratio of the key element and the reference size ratio is greater than the preset size ratio, a deformation defect in the key object can be determined, which can satisfy the following formula: (15) in, It can be set to .

[0091] Furthermore, taking the connected component region corresponding to the blind key icon as an example, since the icon region of the blind key icon requires one and only one connected component with a height greater than 1, and secondly, the width of the connected component region corresponding to the blind key icon should be smaller than the width of the icon region of the blind key icon. Therefore, deformation detection can also be performed using the following constraints, as detailed below:

[0092]

[0093] in, The height of the connected components identified by the blind key. The width of the connected component identified by the blind key. The width of the label area for blind key icons.

[0094] In this embodiment, by comparing the actual dimensions of the button elements in the first and second directions with reference dimensions, and by comparing the ratio of the actual dimensions in the two directions with the ratio of the reference dimensions, accurate detection of deformation defects in the button object can be achieved. Thus, not only can linear deformations such as stretching and compression of the button elements in specific directions be identified, but also dimensional ratio errors such as twisting and deformation of the overall shape can be detected, effectively improving the accuracy of detecting deformation defects in the button object.

[0095] Based on the same inventive concept, this disclosure provides an image processing apparatus, which can be specifically described in conjunction with the appendix. Figure 10 A detailed description of an image processing apparatus provided in the embodiments of this disclosure will be given.

[0096] Figure 10 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this disclosure.

[0097] like Figure 10 As shown, the image processing apparatus 1000 may include: The acquisition module 1010 is used to acquire a first keyboard image and a second keyboard image. The first keyboard image is a keyboard image obtained by converting the image format of the second keyboard image. Both the first keyboard image and the second keyboard image include multiple key objects. The extraction module 1020 is used to extract the actual key object information of the key object from the first keyboard image based on the pixel features of the first keyboard image; The extraction module 1020 is used to extract reference key object information of the key object from the second keyboard image based on the pixel features of the second keyboard image; The determination module 1030 is used to determine whether the first keyboard image has defects based on the actual key object information and reference key object information of the key object.

[0098] In one embodiment, the image processing apparatus provided in this disclosure may further include: A segmentation module is used to segment the first keyboard image to obtain multiple keyboard image regions, each keyboard image region including at least one of the key objects; The determination module is specifically used to determine the defect detection rules corresponding to the keyboard image area; The determination module is specifically used to process the actual key object information and reference key object information of the key objects in the keyboard image area according to the defect detection rules corresponding to the keyboard image area, and determine whether there is a defect in the keyboard image area.

[0099] In one embodiment, the image processing apparatus provided in this disclosure may further include: The segmentation module is specifically used to segment the first keyboard image into multiple keyboard image regions based on the actual key object information of the key object and according to a preset first region segmentation rule.

[0100] In one embodiment, the image processing apparatus provided in this disclosure may further include: The segmentation module is specifically used to identify the key identification information of the key object, and based on the key identification information, to segment the first keyboard image to obtain the multiple keyboard image regions according to the preset second region segmentation rule.

[0101] In one embodiment, the actual button object information includes the actual position and / or actual size of the button element, and the reference button object information includes the reference position and / or reference size of the button element; the image processing apparatus provided in this disclosure embodiment may further include: The determination module is specifically used to determine whether the button element has a positional offset defect based on the actual position of the button element and the reference position of the button element; And / or, The determination module is specifically used to determine whether the button object has a deformation defect based on the actual size of the button element and the reference size of the element.

[0102] In one embodiment, when the key element is a keycap, the image processing apparatus provided in this disclosure may further include: The determination module is specifically used to determine that the button object has a positional offset defect by determining that the button object meets at least one of the following conditions: The positional difference between the actual position of the keycap element and the reference position of the element is greater than the first preset positional difference. The difference between the actual key spacing and the reference key spacing between any two adjacent keycaps is greater than the preset spacing difference; Wherein, any two adjacent keycaps include the keycaps; the actual key spacing is determined based on the actual position of the elements of any two adjacent keycaps, and the reference key spacing is determined based on the reference position of the elements of any two adjacent keycaps.

[0103] In one embodiment, when the key elements include blind key markings, the image processing apparatus provided in this disclosure may further include: The determination module is specifically used to determine that the button object has a positional offset defect by determining that the button object meets at least one of the following conditions: The positional difference between the actual position and the reference position of the element marked by the blind key is greater than the second preset positional difference; The distance deviation of the blind key mark relative to the target keycap is greater than a preset deviation; The target keycap is provided with the blind key mark; the distance deviation is the difference between the actual distance of the blind key mark relative to the target keycap and the reference distance; the actual distance is determined based on the actual position of the elements of the blind key mark and the target keycap, and the reference distance is determined based on the reference position of the elements of the blind key mark and the target keycap.

[0104] In one embodiment, the actual size of the feature includes an actual size in a first direction and an actual size in a second direction, and the reference size of the feature includes a reference size in a first direction and a reference size in a second direction; the image processing apparatus provided in this disclosure embodiment may further include: The process of determining whether the button object has a deformation defect based on the actual size of the button element and the reference size of the element includes: The determination module is specifically used to determine whether the button elements of the button object meet at least one of the following conditions, thereby determining that the button object has a deformation defect: The first size difference between the actual size of the button element in the first direction and the reference size in the first direction is greater than a first preset size threshold. The second dimension difference between the actual second dimension and the reference second dimension of the button element is greater than the second preset dimension threshold. The difference between the actual size ratio and the reference size ratio of the button element is greater than the preset size ratio; The actual size ratio is determined based on the actual size in the first direction and the actual size in the second direction, and the reference size ratio is determined based on the reference size in the first direction and the reference size in the second direction.

[0105] It is understood that, when implementing the corresponding image processing method, the image processing apparatus provided in the above embodiments can allocate the above processing to different program modules as needed to complete all or part of the processing described above. Furthermore, the apparatus and the corresponding method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0106] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform an image processing method.

[0107] This application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored and, when executed by a processor, will cause the processor to execute the image processing method provided in this application.

[0108] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0109] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0110] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0111] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0112] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure; as shown below. Figure 11 As shown, the electronic device 110 includes: a processor 1101, and a memory 1102 communicatively connected to the processor 1101; the memory 1102 stores instructions executable by the processor 1101. The instructions are executed by the processor 1101 to enable the processor 1101 to perform: Acquire a first keyboard image and a second keyboard image. The first keyboard image is obtained by converting the image format of the second keyboard image. Both the first keyboard image and the second keyboard image include multiple key objects. Based on the pixel features of the first keyboard image, the actual key object information of the key object is extracted from the first keyboard image; Based on the pixel features of the second keyboard image, reference key object information of the key object is extracted from the second keyboard image; Based on the actual key object information and reference key object information of the key object, it is determined whether the first keyboard image has defects.

[0113] The electronic devices and corresponding image processing methods provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0114] In practical applications, the electronic device 110 may further include at least one network interface 1103. The various components of the electronic device 110 are coupled together via a bus system 1104. It is understood that the bus system 1104 is used to realize communication between these components. In addition to a data bus, the bus system 1104 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 11 All buses are labeled as bus system 1104. The number of processors 1101 can be at least one, and the number of memories 1102 can be at least one. The network interface 1103 is used for wired or wireless communication between the electronic device 110 and other devices.

[0115] The memory 1102 in this embodiment is used to store various types of data to support the operation of the electronic device 110.

[0116] The methods disclosed in the above embodiments of this disclosure can be applied to processor 1101, or implemented by processor 1101. Processor 1101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 1101 or by instructions in the form of software. The processor 1101 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 1101 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this disclosure can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 1102. Processor 1101 reads the information in memory 1102 and, in conjunction with its hardware, completes the steps of the aforementioned image processing method.

[0117] In some embodiments, the electronic device 110 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned methods.

[0118] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0119] In the above description, the term "some embodiments" refers to a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0120] Unless otherwise defined, all technical and scientific terms used in this disclosure have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used in this disclosure is for the purpose of describing embodiments of this disclosure only and is not intended to be limiting of this disclosure.

[0121] It should be understood that in the various embodiments of this disclosure, the sequence number of each implementation process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0122] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0123] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, The method includes: Acquire a first keyboard image and a second keyboard image. The first keyboard image is obtained by converting the image format of the second keyboard image. Both the first keyboard image and the second keyboard image include multiple key objects. Based on the pixel features of the first keyboard image, the actual key object information of the key object is extracted from the first keyboard image; Based on the pixel features of the second keyboard image, reference key object information of the key object is extracted from the second keyboard image; Based on the actual key object information and reference key object information of the key object, it is determined whether the first keyboard image has defects.

2. The method according to claim 1, characterized in that, The method further includes: The first keyboard image is divided to obtain multiple keyboard image regions, and each keyboard image region includes at least one of the key objects; Determining whether the first keyboard image has defects based on the actual key object information and reference key object information of the key object includes: Determine the defect detection rules corresponding to the keyboard image region; According to the defect detection rules corresponding to the keyboard image area, the actual key object information and reference key object information of the key objects in the keyboard image area are processed to determine whether there is a defect in the keyboard image area.

3. The method according to claim 2, characterized in that, The process of dividing the first keyboard image into multiple keyboard image regions includes: Based on the actual key object information of the key object, the first keyboard image is divided into multiple keyboard image regions according to the preset first region division rule; or, Identify the key identification information of the key object, and based on the key identification information, divide the first keyboard image into multiple keyboard image regions according to a preset second region division rule.

4. The method according to any one of claims 1 to 3, characterized in that, The actual button object information includes the actual position and / or actual size of the button element, and the reference button object information includes the reference position and / or reference size of the button element. Determining whether the first keyboard image has defects based on the actual key object information and reference key object information of the key object includes: Based on the actual position of the button element and the reference position of the element, determine whether the button element has a positional offset defect; And / or, Based on the actual size of the button element and the reference size of the element, determine whether the button object has a deformation defect.

5. The method according to claim 4, characterized in that, When the key element is a keycap, determining whether the key element has a positional offset defect based on the actual position of the key element and the reference position of the key element includes: If the button elements of the button object satisfy at least one of the following conditions, it is determined that the button object has a positional offset defect: The positional difference between the actual position of the keycap element and the reference position of the element is greater than the first preset positional difference. The difference between the actual key spacing and the reference key spacing between any two adjacent keycaps is greater than the preset spacing difference; Wherein, any two adjacent keycaps include the keycaps; the actual key spacing is determined based on the actual position of the elements of any two adjacent keycaps, and the reference key spacing is determined based on the reference position of the elements of any two adjacent keycaps.

6. The method according to claim 4, characterized in that, When the key element includes a blind key identifier, determining whether the key element has a positional offset defect based on the actual position of the key element and the reference position of the key element includes: If the button elements of the button object satisfy at least one of the following conditions, it is determined that the button object has a positional offset defect: The positional difference between the actual position and the reference position of the element marked by the blind key is greater than the second preset positional difference; The distance deviation of the blind key mark relative to the target keycap is greater than a preset deviation; The target keycap is provided with the blind key mark; the distance deviation is the difference between the actual distance of the blind key mark relative to the target keycap and the reference distance; the actual distance is determined based on the actual position of the elements of the blind key mark and the target keycap, and the reference distance is determined based on the reference position of the elements of the blind key mark and the target keycap.

7. The method according to claim 4, characterized in that, The actual dimensions of the element include the actual dimensions in the first direction and the actual dimensions in the second direction, and the reference dimensions of the element include the reference dimensions in the first direction and the reference dimensions in the second direction. The process of determining whether the button object has a deformation defect based on the actual size of the button element and the reference size of the element includes: If the button elements of the button object satisfy at least one of the following conditions, it is determined that the button object has a deformation defect: The first size difference between the actual size of the button element in the first direction and the reference size in the first direction is greater than a first preset size threshold. The second dimension difference between the actual second dimension and the reference second dimension of the button element is greater than the second preset dimension threshold. The difference between the actual size ratio and the reference size ratio of the button element is greater than the preset size ratio; The actual size ratio is determined based on the actual size in the first direction and the actual size in the second direction, and the reference size ratio is determined based on the reference size in the first direction and the reference size in the second direction.

8. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire a first keyboard image and a second keyboard image. The first keyboard image is a keyboard image obtained by converting the image format of the second keyboard image. Both the first keyboard image and the second keyboard image include multiple key objects. The extraction module is used to extract the actual key object information of the key object from the first keyboard image based on the pixel features of the first keyboard image; The extraction module is used to extract reference key object information of the key object from the second keyboard image based on the pixel features of the second keyboard image; The determination module is used to determine whether the first keyboard image has defects based on the actual key object information and reference key object information of the key object.

9. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the image processing method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the image processing method according to any one of claims 1 to 7.

Images