Control segmentation method and device, storage medium, and electronic device

Abstract

The present application discloses a control segmentation method and device, storage medium, and electronic device, which belong to the field of image processing. The method includes: converting the original image into a morphological gradient image; using a first edge detection algorithm and a second edge detection algorithm to iteratively detect the morphological gradient image in the first axis and the second axis respectively, and generating a number of sub-images in each iteration cycle; segmenting the several sub-images in the original image to obtain a control icon. The present application solves the technical problem of low control detection accuracy in the related art, improves the accuracy of control segmentation, and improves robustness.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method and device for segmenting a control, a storage medium, and an electronic device. Background Art

[0002] In some application scenarios of related technologies, it is necessary to extract controls from the entire image and then further process the controls, such as converting a design draft image into code, but the prerequisite is that the controls must be accurately segmented.

[0003] In the related art, there are some algorithms that perform character segmentation based on edge detection and then vertical projection. The characters of these algorithms are relatively simple and regular. Since detection is only performed on one axis (usually the X-axis), the detection capability is limited. They can only be used to extract characters in simple scenes such as license plates, and cannot be applied to images with complex backgrounds, complex colors, large sizes, and multiple control types.

[0004] With respect to the above-mentioned problems existing in the related technologies, no effective solutions have been found so far. Summary of the invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides a control segmentation method and device, a storage medium, and an electronic device.

[0006] According to one aspect of an embodiment of the present application, a method for segmenting a control is provided, including: converting an original image into a morphological gradient image; using a first edge detection algorithm and a second edge detection algorithm to iteratively detect the morphological gradient image in a first axial direction and a second axial direction respectively, and generating a number of sub-images in each iteration cycle; segmenting the number of sub-images in the original image to obtain a control icon.

[0007] Furthermore, a first edge detection algorithm and a second edge detection algorithm are used to iteratively detect the morphological gradient image in the first and second axial directions respectively, and a plurality of sub-images are generated in each iteration cycle, including: using the first edge detection algorithm to detect the morphological gradient image in the first axial direction to generate a first edge image; dividing the first edge image into a plurality of first edge sub-images; dividing the first sub-image corresponding to the first edge sub-image in the original image, and using the second edge detection algorithm to detect a plurality of first sub-images in the second axial direction to generate a second edge image; dividing the second edge image into a plurality of second edge sub-images; if the plurality of second edge sub-images do not meet the preset conditions, dividing the second sub-image corresponding to the second edge sub-image in the original image, and continuing to iteratively detect the plurality of second sub-images until the edge sub-image generated in the current iteration cycle meets the preset conditions.

[0008] Furthermore, dividing the first edge map into a plurality of first edge sub-maps includes: performing pixel value projection on the first edge map along the first axis and detecting non-zero segments of the first edge map along the first axis; dividing the first edge map using a starting non-zero line of each non-zero segment as a dividing line to obtain a plurality of first edge sub-maps.

[0009] Furthermore, a first edge detection algorithm and a second edge detection algorithm are used to iteratively detect the morphological gradient image in the first and second axial directions respectively, and a plurality of sub-images are generated in each iteration cycle, including: obtaining a first upper threshold and a first lower threshold of the first edge detection algorithm, and obtaining a second upper threshold and a second lower threshold of the second edge detection algorithm; iteratively performing the following steps until the edge sub-image of the current cycle meets the preset conditions: based on the first upper threshold and the first lower threshold, the first edge detection algorithm is used to detect the morphological gradient image on the Y axis to generate a plurality of Nth sub-images; based on the second upper threshold and the second lower threshold, the second edge detection algorithm is used to detect the plurality of Nth sub-images on the X axis to generate a plurality of N+1th sub-images, and it is determined whether the edge sub-images corresponding to the plurality of N+1th sub-images meet the preset conditions, wherein N is an integer greater than 0.

[0010] Furthermore, a first edge detection algorithm and a second edge detection algorithm are used to iteratively detect the morphological gradient image in the first and second axial directions respectively, and a plurality of sub-images are generated in each iteration cycle, including: obtaining a first upper threshold and a first lower threshold of the first edge detection algorithm, and obtaining a second upper threshold and a second lower threshold of the second edge detection algorithm; iteratively performing the following steps until the edge sub-image of the current cycle meets the preset conditions: based on the first upper threshold and the first lower threshold, the first edge detection algorithm is used to detect the morphological gradient image on the X-axis to generate a plurality of Nth sub-images; based on the second upper threshold and the second lower threshold, the second edge detection algorithm is used to detect the plurality of Nth sub-images on the Y-axis to generate a plurality of N+1th sub-images, and it is determined whether the edge sub-images corresponding to the plurality of N+1th sub-images meet the preset conditions, wherein N is an integer greater than 0.

[0011] Further, the plurality of sub-images are segmented in the original image to obtain a control icon, including: for each sub-image in the plurality of sub-images, detecting the number of non-zero segments in the sub-image; if the number of non-zero segments in the sub-image is 1, segmenting the sub-image in the original image to obtain a control icon.

[0012] Further, converting the original image into a morphological gradient image includes: converting the original image into a grayscale image, wherein the original image is a color image in RGB format; and converting the grayscale image into the morphological gradient image.

[0013] According to another aspect of an embodiment of the present application, a control segmentation device is also provided, including: a conversion module, used to convert an original image into a morphological gradient image; a processing module, used to iteratively detect the morphological gradient image in a first axial direction and a second axial direction using a first edge detection algorithm and a second edge detection algorithm, respectively, to generate a number of sub-images in each iteration cycle; a segmentation module, used to segment the number of sub-images in the original image to obtain a control icon.

[0014] Furthermore, the processing module includes: a first generating unit, used to detect the morphological gradient image in a first axial direction using a first edge detection algorithm to generate a first edge image; a first segmentation unit, used to segment the first edge image into a plurality of first edge sub-images; a second generating unit, used to segment a first sub-image corresponding to the first edge sub-image in the original image, and detect a plurality of first sub-images in a second axial direction using a second edge detection algorithm to generate a second edge image; a second segmentation unit, used to segment the second edge image into a plurality of second edge sub-images; and a processing unit, used to segment a second sub-image corresponding to the second edge sub-image in the original image if the plurality of second edge sub-images do not meet the preset conditions, and continue to iteratively detect the plurality of second sub-images until the edge sub-image generated in the current iteration cycle meets the preset conditions.

[0015] Furthermore, the first segmentation unit includes: a detection subunit, used to project pixel values ​​of the first edge map along the first axis and detect non-zero segments of the first edge map along the first axis; and a segmentation subunit, used to segment the first edge map using the starting non-zero line of each non-zero segment as a segmentation line to obtain a plurality of first edge sub-maps.

[0016] Further, the processing module includes: an acquisition unit, used to acquire a first upper threshold and a first lower threshold of the first edge detection algorithm, and to acquire a second upper threshold and a second lower threshold of the second edge detection algorithm; a first iteration unit, used to iteratively execute the following steps until the edge sub-image of the current cycle meets the preset conditions: based on the first upper threshold and the first lower threshold, using the first edge detection algorithm to detect the morphological gradient image on the Y-axis to generate a number of Nth sub-images; based on the second upper threshold and the second lower threshold, using the second edge detection algorithm to detect the number of Nth sub-images on the X-axis to generate a number of N+1th sub-images, and judging whether the edge sub-images corresponding to the number of N+1th sub-images meet the preset conditions, wherein N is an integer greater than 0.

[0017] Further, the processing module includes: an acquisition unit, used to acquire a first upper threshold and a first lower threshold of the first edge detection algorithm, and to acquire a second upper threshold and a second lower threshold of the second edge detection algorithm; a second iteration unit, used to iteratively execute the following steps until the edge sub-image of the current cycle meets the preset conditions: based on the first upper threshold and the first lower threshold, using the first edge detection algorithm to detect the morphological gradient image on the X-axis to generate a number of Nth sub-images; based on the second upper threshold and the second lower threshold, using the second edge detection algorithm to detect the number of Nth sub-images on the Y-axis to generate a number of N+1th sub-images, and judging whether the edge sub-images corresponding to the number of N+1th sub-images meet the preset conditions, wherein N is an integer greater than 0.

[0018] Furthermore, the segmentation module includes: a detection unit, used to detect the number of non-zero segments in each sub-image of the plurality of sub-images; and a segmentation unit, used to segment the sub-image in the original image to obtain a control icon if the number of non-zero segments in the sub-image is 1.

[0019] Furthermore, the conversion module includes: a first conversion unit, used to convert the original image into a grayscale image, wherein the original image is a color image in RGB format; and a second conversion unit, used to convert the grayscale image into the morphological gradient image.

[0020] According to another aspect of an embodiment of the present application, a storage medium is further provided, which includes a stored program, and the above steps are executed when the program is run.

[0021] According to another aspect of an embodiment of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein: the memory is used to store computer programs; and the processor is used to execute the steps in the above method by running the program stored in the memory.

[0022] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the steps in the above method.

[0023] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

[0024] In an embodiment of the present application, the original image is converted into a morphological gradient image, and the first edge detection algorithm and the second edge detection algorithm are used to iteratively detect the morphological gradient image in the first and second axes respectively, and several sub-images are generated in each iteration cycle. Several sub-images are segmented in the original image to obtain control icons. By preprocessing the original image with morphological gradients, the disadvantage that complex images easily filter low-gradient controls and improve the recognition rate of controls can be overcome, and the recognition rate of controls can be improved. By using two edge detection algorithms to iteratively detect the morphological gradient image in the first and second axes respectively, the detection accuracy of controls can be improved, which solves the technical problem of low control detection accuracy in related technologies, improves the accuracy of control segmentation, and improves robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0027] Figure 1 is a flow chart of a method for segmenting a control according to an embodiment of the present invention;

[0028] Figure 2 is a rendering of an edge map according to an embodiment of the present invention;

[0029] Figure 3 is a schematic diagram of cutting an edge graph by a non-zero segment according to an embodiment of the present invention;

[0030] Figure 4 This is an effect diagram of splitting a control in an original image according to an embodiment of the present invention;

[0031] Figure 5 is a flow chart of obtaining an upper threshold and a lower threshold according to an embodiment of the present invention;

[0032] Figure 6 is a flowchart of an implementation of an embodiment of the present invention in a specific scenario;

[0033] Figure 7 is a structural block diagram of a control segmentation device according to an embodiment of the present invention;

[0034] Figure 8 It is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0036] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another similar entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0037] In this embodiment, a method for segmenting a control is provided. Figure 1 is a flow chart of a method for segmenting a control according to an embodiment of the present invention, such as Figure 1 As shown, the process includes the following steps:

[0038] Step S102, converting the original image into a morphological gradient image;

[0039] The original image in this embodiment can be a picture in application scenarios such as a photo, a screenshot of a screen, a screenshot of a video, an electronic poster, etc. The original image includes a number of controls, and the controls can be areas enclosed by characters, people, scenery, props, etc. in the picture.

[0040] Step S104, using a first edge detection algorithm and a second edge detection algorithm to iteratively detect the morphological gradient image in the first axial direction and the second axial direction respectively, and generating a plurality of sub-images in each iteration cycle;

[0041] Optionally, the first edge detection algorithm and the second edge detection algorithm respectively use different upper and lower threshold combinations to alternately detect the morphological gradient image in two axes of the two-dimensional coordinates and segment out sub-images.

[0042] Step S106, dividing the original image into several sub-images to obtain a control icon;

[0043] Through the above steps, the original image is converted into a morphological gradient image, and the first edge detection algorithm and the second edge detection algorithm are used to iteratively detect the morphological gradient image in the first and second axes respectively. Several sub-images are generated in each iteration cycle, and several sub-images are segmented in the original image to obtain control icons. By preprocessing the original image with morphological gradients, the disadvantage that complex images are easy to filter low-gradient controls can be overcome, and the recognition rate of controls can be improved. By using two edge detection algorithms to iteratively detect the morphological gradient image in the first and second axes respectively, the detection accuracy of controls can be improved, which solves the technical problem of low control detection accuracy in related technologies, improves the accuracy of control segmentation, and improves robustness.

[0044] For the above step S104 of this embodiment, the first edge detection algorithm and the second edge detection algorithm are a canny edge detection algorithm, and the implementation steps include: Gaussian smoothing filtering of the image to be detected (in this embodiment, the morphological gradient map) to reduce the image noise; calculating the gradient information (including intensity and direction) of the filtered image; performing non-maximum suppression on the pixels in the image according to the gradient information to eliminate edge misdetection; and finally detecting and connecting the edges using a double threshold algorithm to generate an edge map. Among them, the double threshold includes an upper threshold T1 and a lower threshold T2. If the pixel point in the image is greater than the upper threshold, it is considered to be an edge (called a strong edge), and if it is less than the lower threshold T2, it is considered not to be an edge. If it is between the two, it is considered to be a candidate (called a weak edge) and needs to be further processed. It is assumed that among the edge points of the two types of edges after non-maximum suppression, the gradient value exceeds T1 and is called a strong edge, the gradient value is less than T1 and greater than T2 and is called a weak edge, and the gradient is less than T2 and is not an edge. What is certain is that a strong edge must be an edge point, so the value of T1 is relatively high, requiring the gradient value of the pixel point to be large enough (the change is drastic enough), while a weak edge may be an edge or noise, which can be determined by edge hysteresis tracking, regional growing and other algorithms. For example, when there is a strong edge point in the 8 neighborhoods around the weak edge (areas in 8 directions: directly above, directly below, directly left, directly right, right front, left front, lower right, and upper right), the weak edge point will be converted into a strong edge point to complement the strong edge.

[0045] In one implementation of this embodiment, the first edge detection algorithm and the second edge detection algorithm are used to iteratively detect the morphological gradient image in the first axis and the second axis respectively, and several sub-images are generated in each iteration cycle, including:

[0046] S11, using a first edge detection algorithm to detect a morphological gradient map in a first axial direction to generate a first edge map;

[0047] The process of generating the first edge map includes: performing Gaussian smoothing filtering on the morphological gradient map to be detected to reduce image noise; calculating the gradient information (including intensity and direction) of the filtered image; performing non-maximum suppression on the pixels in the image according to the gradient information to eliminate false edge detection; and finally using a double threshold algorithm to detect and connect edges of the image along the first axis to generate the first edge map.

[0048] Figure 2 This is a rendering of the edge map of an embodiment of the present invention, with white pixels as boundary points to form an edge map (the text in the figure has no actual meaning), wherein the white pixels are pixels with a pixel value of 255, and the black pixels are pixels with a pixel value of 0.

[0049] S12, dividing the first edge graph into a plurality of first edge subgraphs;

[0050] In one example, dividing the first edge map into a plurality of first edge sub-maps includes: performing pixel value projection on the first edge map along a first axis and detecting non-zero segments of the first edge map along the first axis; dividing the first edge map using a starting non-zero line of each non-zero segment as a dividing line to obtain a plurality of first edge sub-maps.

[0051] Here, the first axis is taken as the Y axis and the second axis is taken as the X axis as an example. Through pixel value projection, the number of pixels in each row or column in the pixel matrix can be counted to determine whether the current pixel row or pixel column is a non-zero segment. If the sum of the pixel values ​​of each row or column is not zero, it is a non-zero segment.

[0052] S13, cutting a first sub-image corresponding to the first edge sub-image in the original image, detecting a plurality of first sub-images in a second axial direction using a second edge detection algorithm, and generating a second edge image;

[0053] In this embodiment, the first sub-image is a part of the original image, and each first edge sub-image corresponds to a first sub-image in the original image. Since the first edge image is obtained by marginalizing the original image, the first edge image and the original image are the same in size. Therefore, several first edge sub-images obtained by cutting the first edge image can be referred to to obtain several corresponding first sub-images in the original image.

[0054] The process of generating the second edge map includes: performing Gaussian smoothing filtering on the first sub-image to be detected to reduce image noise; calculating the gradient information (including intensity and direction) of the filtered image; performing non-maximum suppression on the pixels in the image according to the gradient information to eliminate false edge detection; and finally using a double threshold algorithm to detect and connect edges of the image along the second axis to generate a second edge map.

[0055] S14, dividing the second edge graph into a plurality of second edge subgraphs;

[0056] Similar to the first edge map, dividing the second edge map into a plurality of second edge sub-maps includes: performing pixel value projection on the second edge map along the second axis, and detecting non-zero segments of the second edge map along the second axis; dividing the second edge map using the starting non-zero line of each non-zero segment as a dividing line to obtain a plurality of second edge sub-maps. For example, Figure 3 : is a schematic diagram of cutting an edge graph by non-zero segments according to an embodiment of the present invention, the second axis is the X-axis, the second edge graph is projected on the X-axis, and the sum of the pixel values ​​of each column in the pixel matrix of the second edge graph is counted, and the pixel values ​​of each column in the pixel matrix are obtained to be 00332025 in sequence, the 3rd, 4th, 5th, 7th, and 8th columns are non-zero segments, and the remaining 1st, 2nd, and 6th columns are zero segments, and four edge sub-graphs (corresponding to Figure 3 The marginal Figure 1 , edge sub Figure 2 , edge sub Figure 3 , edge sub Figure 4 ), which are respectively composed of pixels in the 1st to 2nd columns, 3rd to 5th columns, 6th column, and 7th to 8th columns.

[0057] S15, if a plurality of second edge sub-images do not meet the preset condition, a second sub-image corresponding to the second edge sub-image is cut out from the original image, and the plurality of second sub-images are continuously iterated and detected until an edge sub-image generated in the current iteration cycle meets the preset condition.

[0058] The preset condition of this embodiment can be set by the detection accuracy. When the detection accuracy is the highest, the preset condition is that the number of non-zero segments of the edge sub-image obtained by cutting in the current iteration cycle is 1. In one embodiment, a plurality of sub-images are segmented in the original image to obtain a control icon, including: for each of the plurality of sub-images, detecting the number of non-zero segments in the sub-image; if the number of non-zero segments in the sub-image is 1, determining that the preset condition is met, stopping iteration, segmenting the sub-image in the original image, and obtaining the control icon.

[0059] Figure 4 This is an effect diagram of dividing the controls in the original image according to an embodiment of the present invention, where each frame corresponds to a control ( Figure 4 The colors are not shown, and the text in the figure has no actual meaning), Figure 2 The edge graph shown corresponds to .

[0060] In this embodiment, when iteratively detecting the morphological gradient image to generate several sub-images, the detection order of the first edge detection algorithm and the second edge detection algorithm can be selected according to the type of the morphological gradient image, such as selecting the detection order of the two edge detection algorithms according to the layout type of the original image or the morphological gradient image. In each iteration cycle, the image with a vertical layout (such as a waterfall image) is first detected on the horizontal axis (the X-axis of the two-dimensional coordinate) using the second edge detection algorithm, and then continues to be detected on the vertical axis (the Y-axis of the two-dimensional coordinate) using the first edge detection algorithm until the iteration ends. The image with a horizontal layout is first detected on the vertical axis using the first edge detection algorithm, and then continues to be detected on the horizontal axis using the second edge detection algorithm until the iteration ends.

[0061] In this embodiment, regardless of the first edge detection algorithm or the second edge detection algorithm, the upper threshold and the lower threshold can be specified manually, or the algorithm can be designed to be adaptively specified. By setting different threshold combinations, different detection effects can be achieved. In an example in which an algorithm is used to set the upper threshold and the lower threshold, the upper threshold of the edge detection algorithm is set by the global iteration method on the X-axis, and the lower threshold of the edge detection algorithm is set by the OTSU (inter-class variance maximization threshold segmentation algorithm), which can overcome the false edge problem of the edge detection algorithm. At the same time, the OTSU threshold and the 0.1* threshold are used as the upper and lower thresholds of the edge detection algorithm for the Y-axis. When the edge detection algorithm of the upper and lower thresholds of the global iteration method and the OTSU are used on the X-axis, the morphological gradient map is binarized by the global iteration method to obtain the optimal threshold T1, and the gradient map is binarized by OTSU to obtain the optimal threshold T2.

[0062] In one embodiment, a first edge detection algorithm and a second edge detection algorithm are used to iteratively detect a morphological gradient map in a first axial direction and a second axial direction respectively, and a plurality of sub-maps are generated in each iteration cycle, including: obtaining a first upper threshold and a first lower threshold of the first edge detection algorithm, and obtaining a second upper threshold and a second lower threshold of the second edge detection algorithm; iteratively performing the following steps until the edge sub-map of the current cycle meets a preset condition: based on the first upper threshold and the first lower threshold, the first edge detection algorithm is used to detect the morphological gradient map on the Y axis to generate a plurality of Nth sub-maps; based on the second upper threshold and the second lower threshold, the second edge detection algorithm is used to detect a plurality of Nth sub-maps on the X axis to generate a plurality of N+1th sub-maps, and it is determined whether the edge sub-maps corresponding to the plurality of N+1th sub-maps meet the preset condition, wherein N is an integer greater than 0.

[0063] Different canny edge detection algorithms are used for the Y-axis and X-axis respectively, and the upper and lower thresholds used by the two canny edge detection algorithms are different. Then the edges are alternately projected onto the Y-axis and X-axis, divided into sub-graphs, and then recursively detected and projected on the sub-graphs.

[0064] In another embodiment, a first edge detection algorithm and a second edge detection algorithm are used to iteratively detect a morphological gradient map in a first axial direction and a second axial direction respectively, and a plurality of sub-maps are generated in each iteration cycle, including: obtaining a first upper threshold and a first lower threshold of the first edge detection algorithm, and obtaining a second upper threshold and a second lower threshold of the second edge detection algorithm; iteratively performing the following steps until the edge sub-map of the current cycle meets a preset condition: based on the first upper threshold and the first lower threshold, the first edge detection algorithm is used to detect the morphological gradient map on the X-axis to generate a plurality of Nth sub-maps; based on the second upper threshold and the second lower threshold, the second edge detection algorithm is used to detect a plurality of Nth sub-maps on the Y-axis to generate a plurality of N+1th sub-maps, and it is determined whether the edge sub-maps corresponding to the plurality of N+1th sub-maps meet the preset condition, wherein N is an integer greater than 0.

[0065] Different canny edge detection algorithms are used for the Y-axis and X-axis respectively, and the upper and lower thresholds used by the two canny edge detection algorithms are different. Then the edges are alternately projected onto the X-axis and Y-axis, divided into sub-graphs, and then recursively detected and projected on the sub-graphs.

[0066] Figure 5 This is a flowchart of obtaining the upper threshold and the lower threshold in an embodiment of the present invention, and the process includes: determining the original image in RGB format, that is, the RGB image, converting the RGB image into a grayscale image, performing 3*3 morphological gradient detection on the grayscale image to obtain a gradient image, selecting two algorithms (selecting the global iteration method and selecting OTSU), respectively, binarizing the gradient image by the global iteration method to obtain S, and finally starting to execute the edge edge with T and 0.5*S as the upper threshold and the lower threshold respectively. The upper threshold and the lower threshold obtained by this process can be applied to one of the axes (such as applied to the X-axis), or can be applied to both axes at the same time. By adaptively processing the original image to generate the upper threshold and the lower threshold of edge detection, the false edges generated by the detection can be reduced.

[0067] Optionally, converting the original image into a morphological gradient image includes: converting the original image into a grayscale image, wherein the original image is a color image in RGB format; and converting the grayscale image into a morphological gradient image.

[0068] It should be noted that the original image can also be a color image in cmyk (cyan, magenta, yellow, black) format. The image in cmyk format can be converted into RGB format first, and then converted into a grayscale image through YUV mapping. In the process of converting the grayscale image into a morphological gradient image, the grayscale image can be converted into a morphological gradient image by performing 3*3 morphological gradient detection on the grayscale image. By introducing morphological gradient image preprocessing, the disadvantage that low-gradient controls are easily filtered out by complex images can be overcome, thereby improving the recognition rate of controls.

[0069] Figure 6 is an implementation flow chart of an embodiment of the present invention in a specific scenario, such as Figure 6 As shown in FIG. 1 , the process includes the following steps: converting the RGB image into a grayscale image; converting the grayscale image into a morphological gradient image (referred to as the gradient image in the figure); selecting OTSU and the global iteration method to perform binarization on the gradient image to obtain T value and S value, respectively setting T and 0.1*T as the upper and lower thresholds for edge detection on the Y axis, respectively, and setting T and 0.5*S as the upper and lower thresholds for edge detection on the X axis, respectively; performing edge detection from the Y axis of the gradient image, generating an edge sub-image, performing Y-axis projection on the edge sub-image, detecting non-zero segments, judging whether the number of non-zero segments is greater than 1, recording the judgment result, and projecting the original image from the Y axis. The image is divided into several sub-images, and then each sub-image is converted into a grayscale image, and the grayscale image is converted into a gradient image. Edge detection is performed starting from the X-axis of the gradient image to generate an edge sub-image. It is determined whether the number of non-zero segments of the edge sub-image is greater than 1. If so, the next iteration cycle is started; if not, and the judgment result of the non-zero segment of the Y-axis projection is also no, the iteration ends, and the original image is divided according to the edge sub-image generated on the X-axis in the last iteration cycle. Each sub-image of the original image corresponds to a control pattern, and then it is determined whether the edge detection result of the X-axis in the last iteration cycle has a non-zero segment. If not, the process ends. If so, the process ends after the control position is corrected.

[0070] Through the scheme of this embodiment, the introduction of morphological gradient preprocessing can overcome the disadvantage that complex images are easy to filter out low-gradient controls. The Y-axis and X-axis alternating projection method is adopted. The Y-axis is first projected and segmented, and then the sub-image is projected on the X-axis. The sub-image cut from the X-axis is projected on the Y-axis again, and this is iterated until it cannot be segmented. High-precision control segmentation is achieved, and the robustness of the control segmentation process is also improved.

[0071] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0072] In this embodiment, a control segmentation device is also provided to implement the above-mentioned embodiments and preferred implementation modes, which have been described and will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0073] Figure 7 is a structural block diagram of a control segmentation device according to an embodiment of the present invention, such as Figure 7 As shown, the device includes: a conversion module 70, a processing module 72, and a segmentation module 74, wherein:

[0074] A conversion module 70, used to convert the original image into a morphological gradient image;

[0075] A processing module 72, configured to iteratively detect the morphological gradient image in a first axial direction and a second axial direction using a first edge detection algorithm and a second edge detection algorithm, respectively, and generate a plurality of sub-images in each iteration cycle;

[0076] The segmentation module 74 is used to segment the original image into the plurality of sub-images to obtain control icons.

[0077] Optionally, the processing module includes: a first generating unit, used to detect the morphological gradient map in the first axis direction using a first edge detection algorithm to generate a first edge map; a first segmentation unit, used to segment the first edge map into a plurality of first edge sub-maps; a second generating unit, used to segment a first sub-map corresponding to the first edge sub-map in the original image, and detect a plurality of first sub-maps in the second axis direction using a second edge detection algorithm to generate a second edge map; a second segmentation unit, used to segment the second edge map into a plurality of second edge sub-maps; and a processing unit, used to segment a second sub-map corresponding to the second edge sub-map in the original image if the plurality of second edge sub-maps do not meet a preset condition, and continue to iteratively detect the plurality of second sub-maps until the edge sub-map generated in the current iteration cycle meets the preset condition.

[0078] Optionally, the first segmentation unit includes: a detection subunit, used to project pixel values ​​of the first edge map along the first axis and detect non-zero segments of the first edge map along the first axis; and a segmentation subunit, used to segment the first edge map using the starting non-zero line of each non-zero segment as a segmentation line to obtain a plurality of first edge sub-maps.

[0079] Optionally, the processing module includes: an acquisition unit, used to acquire a first upper threshold and a first lower threshold of the first edge detection algorithm, and to acquire a second upper threshold and a second lower threshold of the second edge detection algorithm; a first iteration unit, used to iteratively execute the following steps until the edge sub-image of the current cycle meets a preset condition: based on the first upper threshold and the first lower threshold, using the first edge detection algorithm to detect the morphological gradient image on the Y-axis to generate a number of Nth sub-images; based on the second upper threshold and the second lower threshold, using the second edge detection algorithm to detect the number of Nth sub-images on the X-axis to generate a number of N+1th sub-images, and determining whether the edge sub-images corresponding to the number of N+1th sub-images meet the preset condition, wherein N is an integer greater than 0.

[0080] Optionally, the processing module includes: an acquisition unit, used to acquire a first upper threshold and a first lower threshold of the first edge detection algorithm, and to acquire a second upper threshold and a second lower threshold of the second edge detection algorithm; a second iteration unit, used to iteratively execute the following steps until the edge sub-image of the current cycle meets a preset condition: based on the first upper threshold and the first lower threshold, using the first edge detection algorithm to detect the morphological gradient image on the X-axis to generate a number of Nth sub-images; based on the second upper threshold and the second lower threshold, using the second edge detection algorithm to detect the number of Nth sub-images on the Y-axis to generate a number of N+1th sub-images, and judging whether the edge sub-images corresponding to the number of N+1th sub-images meet the preset condition, wherein N is an integer greater than 0.

[0081] Optionally, the segmentation module includes: a detection unit, used to detect the number of non-zero segments in each sub-image among the several sub-images; and a segmentation unit, used to segment the sub-image in the original image to obtain a control icon if the number of non-zero segments in the sub-image is 1.

[0082] Optionally, the conversion module includes: a first conversion unit, used to convert the original image into a grayscale image, wherein the original image is a color image in RGB format; and a second conversion unit, used to convert the grayscale image into the morphological gradient image.

[0083] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0084] The embodiment of the present application also provides an electronic device, Figure 8 is a structural diagram of an electronic device according to an embodiment of the present invention, such as Figure 8 As shown, it includes a processor 81, a communication interface 82, a memory 83 and a communication bus 84, wherein the processor 81, the communication interface 82 and the memory 83 communicate with each other through the communication bus 84, and the memory 83 is used to store computer programs; when the processor 81 is used to execute the program stored in the memory 83, the following steps are implemented: converting the original image into a morphological gradient image; using a first edge detection algorithm and a second edge detection algorithm to iteratively detect the morphological gradient image in the first axis and the second axis respectively, and generating a plurality of sub-images in each iteration cycle; segmenting the plurality of sub-images in the original image to obtain a control icon.

[0085] Furthermore, a first edge detection algorithm and a second edge detection algorithm are used to iteratively detect the morphological gradient image in the first and second axial directions respectively, and a plurality of sub-images are generated in each iteration cycle, including: using the first edge detection algorithm to detect the morphological gradient image in the first axial direction to generate a first edge image; dividing the first edge image into a plurality of first edge sub-images; dividing the first sub-image corresponding to the first edge sub-image in the original image, and using the second edge detection algorithm to detect a plurality of first sub-images in the second axial direction to generate a second edge image; dividing the second edge image into a plurality of second edge sub-images; if the plurality of second edge sub-images do not meet the preset conditions, dividing the second sub-image corresponding to the second edge sub-image in the original image, and continuing to iteratively detect the plurality of second sub-images until the edge sub-image generated in the current iteration cycle meets the preset conditions.

[0086] Furthermore, dividing the first edge map into a plurality of first edge sub-maps includes: performing pixel value projection on the first edge map along the first axis and detecting non-zero segments of the first edge map along the first axis; dividing the first edge map using a starting non-zero line of each non-zero segment as a dividing line to obtain a plurality of first edge sub-maps.

[0087] Furthermore, a first edge detection algorithm and a second edge detection algorithm are used to iteratively detect the morphological gradient image in the first and second axial directions respectively, and a plurality of sub-images are generated in each iteration cycle, including: obtaining a first upper threshold and a first lower threshold of the first edge detection algorithm, and obtaining a second upper threshold and a second lower threshold of the second edge detection algorithm; iteratively performing the following steps until the edge sub-image of the current cycle meets the preset conditions: based on the first upper threshold and the first lower threshold, the first edge detection algorithm is used to detect the morphological gradient image on the Y axis to generate a plurality of Nth sub-images; based on the second upper threshold and the second lower threshold, the second edge detection algorithm is used to detect the plurality of Nth sub-images on the X axis to generate a plurality of N+1th sub-images, and it is determined whether the edge sub-images corresponding to the plurality of N+1th sub-images meet the preset conditions, wherein N is an integer greater than 0.

[0088] Furthermore, a first edge detection algorithm and a second edge detection algorithm are used to iteratively detect the morphological gradient image in the first and second axial directions respectively, and a plurality of sub-images are generated in each iteration cycle, including: obtaining a first upper threshold and a first lower threshold of the first edge detection algorithm, and obtaining a second upper threshold and a second lower threshold of the second edge detection algorithm; a second iteration unit is used to iteratively execute the following steps until the edge sub-image of the current cycle meets the preset conditions: based on the first upper threshold and the first lower threshold, the first edge detection algorithm is used to detect the morphological gradient image on the X-axis to generate a plurality of Nth sub-images; based on the second upper threshold and the second lower threshold, the second edge detection algorithm is used to detect the plurality of Nth sub-images on the Y-axis to generate a plurality of N+1th sub-images, and it is determined whether the edge sub-images corresponding to the plurality of N+1th sub-images meet the preset conditions, wherein N is an integer greater than 0.

[0089] Further, the plurality of sub-images are segmented in the original image to obtain a control icon, including: for each sub-image in the plurality of sub-images, detecting the number of non-zero segments in the sub-image; if the number of non-zero segments in the sub-image is 1, segmenting the sub-image in the original image to obtain a control icon.

[0090] Further, converting the original image into a morphological gradient image includes: converting the original image into a grayscale image, wherein the original image is a color image in RGB format; and converting the grayscale image into the morphological gradient image.

[0091] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0092] The communication interface is used for communication between the above terminal and other devices.

[0093] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0094] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0095] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the control segmentation method described in any of the above embodiments.

[0096] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the control segmentation method described in any one of the above embodiments.

[0097] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0098] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

[0099] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for segmenting a control, characterized in that: include: Convert the original image into a morphological gradient image; Iteratively detecting the morphological gradient image in the first axial direction and the second axial direction respectively using a first edge detection algorithm and a second edge detection algorithm, generating a plurality of sub-images in each iteration cycle; Segmenting the plurality of sub-images in the original image to obtain a control icon; The morphological gradient image is iteratively detected in the first axial direction and the second axial direction respectively using the first edge detection algorithm and a second edge detection algorithm, and a plurality of sub-images are generated in each iteration cycle, including: Using a first edge detection algorithm to detect the morphological gradient map in a first axial direction to generate a first edge map; Dividing the first edge graph into a plurality of first edge sub-graphs; Cutting a first sub-image corresponding to the first edge sub-image in the original image, detecting a plurality of first sub-images in a second axial direction using a second edge detection algorithm, and generating a second edge image; dividing the second edge graph into a plurality of second edge subgraphs; If the plurality of second edge sub-images do not meet the preset condition, a second sub-image corresponding to the second edge sub-image is cut out from the original image, and the plurality of second sub-images are continuously iterated and detected until an edge sub-image generated in the current iteration cycle meets the preset condition.

2. The method according to claim 1, characterized in that Dividing the first edge graph into a plurality of first edge subgraphs comprises: Performing pixel value projection on the first edge map along the first axial direction, and detecting non-zero segments of the first edge map along the first axial direction; The first edge graph is divided by taking the starting non-zero line of each non-zero segment as a dividing line to obtain a plurality of first edge sub-graphs.

3. The method according to claim 1, characterized in that The morphological gradient image is iteratively detected in the first axial direction and the second axial direction respectively using the first edge detection algorithm and a second edge detection algorithm, and a plurality of sub-images are generated in each iteration cycle, including: Acquire a first upper threshold and a first lower threshold of the first edge detection algorithm, and acquire a second upper threshold and a second lower threshold of the second edge detection algorithm; The following steps are iteratively performed until the edge sub-image of the current cycle meets the preset conditions: based on the first upper threshold and the first lower threshold, the first edge detection algorithm is used to detect the morphological gradient image on the Y axis to generate a plurality of Nth sub-images; based on the second upper threshold and the second lower threshold, the second edge detection algorithm is used to detect the plurality of Nth sub-images on the X axis to generate a plurality of N+1th sub-images, and it is determined whether the edge sub-images corresponding to the plurality of N+1th sub-images meet the preset conditions, wherein N is an integer greater than 0.

4. The method according to claim 1, characterized in that: The morphological gradient image is iteratively detected in the first axial direction and the second axial direction respectively using the first edge detection algorithm and a second edge detection algorithm, and a plurality of sub-images are generated in each iteration cycle, including: Acquire a first upper threshold and a first lower threshold of the first edge detection algorithm, and acquire a second upper threshold and a second lower threshold of the second edge detection algorithm; The following steps are iteratively performed until the edge sub-image of the current cycle meets the preset conditions: based on the first upper threshold and the first lower threshold, the first edge detection algorithm is used to detect the morphological gradient image on the X-axis to generate a plurality of N-th sub-images; based on the second upper threshold and the second lower threshold, the second edge detection algorithm is used to detect the plurality of N-th sub-images on the Y-axis to generate a plurality of N+1-th sub-images, and it is determined whether the edge sub-images corresponding to the plurality of N+1-th sub-images meet the preset conditions, wherein N is an integer greater than 0.

5. The method according to claim 1, characterized in that The plurality of sub-images are segmented in the original image to obtain a control icon, including: For each sub-image in the plurality of sub-images, detecting the number of non-zero segments in the sub-image; If the number of non-zero segments in the sub-image is 1, the sub-image is segmented in the original image to obtain a control icon.

6. The method according to claim 1, characterized in that Converting the original image into a morphological gradient image includes: Convert the original image into a grayscale image, wherein the original image is a color image in RGB format; The grayscale image is converted into the morphological gradient image.

7. A control segmentation device, characterized in that: include: The conversion module is used to convert the original image into a morphological gradient image; A processing module, configured to iteratively detect the morphological gradient image in a first axial direction and a second axial direction respectively using a first edge detection algorithm and a second edge detection algorithm, and generate a plurality of sub-images in each iteration cycle; A segmentation module, used for segmenting the plurality of sub-images in the original image to obtain control icons; The processing module includes: a first generating unit, used to detect the morphological gradient image in a first axial direction using a first edge detection algorithm to generate a first edge image; a first segmentation unit, used to segment the first edge image into a plurality of first edge sub-images; a second generating unit, used to segment a first sub-image corresponding to the first edge sub-image in the original image, and detect a plurality of first sub-images in a second axial direction using a second edge detection algorithm to generate a second edge image; a second segmentation unit, used to segment the second edge image into a plurality of second edge sub-images; and a processing unit, used to segment a second sub-image corresponding to the second edge sub-image in the original image if the plurality of second edge sub-images do not meet a preset condition, and continue to iteratively detect the plurality of second sub-images until the edge sub-image generated in the current iteration cycle meets the preset condition.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein the processor executes the method according to any one of claims 1 to 6 when running the program.

9. An electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein: The processor, the communication interface, and the memory communicate with each other via a communication bus; wherein: Memory, used to store computer programs; A processor, configured to execute the method according to any one of claims 1 to 6 by running a program stored in a memory.

Images