Pattern element grouping method and apparatus based on hypergraph learning

Abstract

The application discloses a pattern element grouping method and device based on hypergraph learning, and the method comprises the following steps: performing a pretreatment operation on a pattern, and obtaining an image segmentation result of elements in the pattern; extracting gestalt rule features of the pattern elements, including similarity, proximity, continuity and mixed features; modeling the correlation between the pattern elements based on different gestalt rules by using a hypergraph; designing an adaptive hypergraph fusion method to obtain a hypergraph fused with multiple gestalt rules; and using a label propagation method based on hypergraph learning and user guidance to obtain a grouping result of the pattern. The application proposes a representation of gestalt grouping rules on pattern elements, models the connection relationship between the pattern elements according to different grouping rules by using a hypergraph, and uses the hypergraph for unified representation, so that the fusion of the gestalt rules is realized, the problem of gestalt rule conflicts is effectively solved, and personalized grouping of the pattern elements according to the grouping intention of a user is realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision, and particularly relates to a pattern element grouping method and device based on hypergraph learning. BACKGROUND

[0002] The research on patterns has been one of the research hotspots in the field of computer vision. Due to the abstractness, sparsity and diversity of patterns, it is very difficult to group patterns. Therefore, different grouping rules may lead to conflicting grouping results, and the perception ability of human beings to the same pattern is also often different.

[0003] At present, the research on patterns mainly focuses on recognition, retrieval and completion, and with the emphasis of researchers on fine-grained operations of patterns, more and more attention has been paid to the grouping of pattern elements. However, in the current pattern element grouping method, the following deficiencies still exist: (1) Most of the existing methods only consider two Gestalt rules, and cannot solve the diversity and complexity of patterns. (2) The existing data-driven grouping method uses a universally applicable mixed Gestalt rule for grouping, ignoring the problem of large distribution difference of pattern elements. (3) The existing grouping method rarely considers the grouping intention of human beings, and the obtained grouping result is difficult to meet the grouping requirements of individuals. SUMMARY

[0004] The main purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide a pattern element grouping method and device based on hypergraph learning. The present application proposes the representation of Gestalt grouping rules on pattern elements, and simultaneously uses hypergraph to model the connection relationship between pattern elements according to different grouping rules, and uses hypergraph for unified representation, realizes the fusion of Gestalt rules, solves the conflict problem between different grouping rules, and finally realizes the pattern element grouping conforming to the perception of users through the mode of human-computer interaction.

[0005] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0006] In the first aspect, the present application provides a pattern element grouping method based on hypergraph learning, comprising the following steps:

[0007] Preprocessing the pattern elements, the preprocessing including image segmentation operation on the pattern elements to obtain a set of segmented pattern elements;

[0008] Extracting the Gestalt features of the pattern elements, using ring projection vectors to describe the similarity of the outlines of the pattern elements; using the minimum distance between the element outline point sets to describe the proximity between the elements; using Mask R-CNN to extract the continuity between the elements; using a data-driven encoder to extract the mixed features between the elements;

[0009] Based on the Gestalt features of the extracted elements, the Euclidean distance of the features between the elements is calculated, and a corresponding Gestalt hypergraph is constructed to describe the Gestalt correlation between the pattern elements;

[0010] An adaptive hypergraph fusion grouping method is constructed, and the adaptive hypergraph is constructed according to the distribution of the pattern elements and the pre-clustering result to guide the construction of the hyperedge in the hypergraph, so that the hyperedge can flexibly represent the correlation between different elements, and more accurate hyperedge connection is obtained; The hypergraph fusion grouping method is to fuse the similarity, proximity, continuity and mixed hypergraph by using the easy expansion ability of the hypergraph, so as to flexibly realize the fusion of the Gestalt rules, and the multiple Gestalt rules are used for the grouping of the pattern elements together;

[0011] The label propagation method is used to obtain the grouping result of the pattern elements by taking the fused hypergraph and the initial input grouping label of the user as conditions.

[0012] As a preferred technical solution, the ring projection vector is used to describe the similarity of the pattern element contour, specifically:

[0013] The ring projection vector is used to describe the rotation invariance feature of the pattern element contour, and then the similarity of the contours between the pattern elements is calculated, and the ring projection vector is defined as follows:

[0014]

[0015] Where f(x,y is the pixel value of the image f at (x,y), 2πr is the circumference of the ring with radius r, (x0,y0) is the center coordinate, r∈[0,R], R is the maximum circle radius,

[0016] For the ring projection vectors P a and P b extracted from element a and element b, the similarity distance of element a and element b can be expressed as:

[0017]

[0018] As a preferred technical solution, the minimum distance between the element contour point sets is used to describe the proximity between the elements, specifically:

[0019] The minimum distance between the limited point sets is used to describe the proximity between the elements, first, the Marr edge detection operator is used to extract the edge contour of the pattern element, then the proximity distance d of the edge point set of each element is calculated, and the distance d of element p and element q is:

[0020]

[0021] Where pi represents the i-th pixel point coordinate of element p, q j represents the j-th pixel point coordinate of element q.

[0022] As a preferred technical solution, the continuity between the elements is extracted using Mask R-CNN, specifically:

[0023] A pattern dataset containing continuity rules is constructed to retrain the detection model Mask R-CNN, thereby obtaining the continuity mask in the pattern, and then constructing a continuity hypergraph according to the mask.

[0024] As a preferred technical solution, the mixed features between the elements are extracted using a data-driven encoder, specifically:

[0025] Using a data-driven approach, the machine learns mixed features that integrate multiple Gestalt rules from a large amount of pattern data. An atomic encoder and a structural encoder are constructed to encode the features as mixed features of the pattern elements, extracting the mixed features F∈R N*40 of the elements in the pattern, and then normalizing F to F′∈R N*40 , and then calculating the Euclidean distance D∈R N*N between each pair of elements.

[0026] As a preferred technical solution, the Gestalt features of the extracted elements are used to calculate the Euclidean distance between the features of the elements, and a corresponding Gestalt hypergraph is constructed to describe the Gestalt correlation between the pattern elements, specifically:

[0027] The Euclidean distance between the features of each pair of elements is calculated to construct a distance matrix D∈R N*N , where N represents the number of pattern elements, and then the distance matrix D is converted to a weight matrix W:

[0028]

[0029] The weight matrix W describes the correlation between the pattern elements, and based on the weight matrix W, a hypergraph H is constructed to represent the correlation between the pattern elements:

[0030]

[0031] where m represents different Gestalt hypergraphs, topk(w T (j represents the set of the top k elements with the largest values in the j-th column of w T If w T (i,j) is in the set, it is connected by a hyperedge.

[0032] As a preferred technical solution, the supergraph label propagation method is defined as follows:

[0033]

[0034] Wherein H is a constructed supergraph, D v is a degree matrix of nodes in the supergraph, D e is a degree matrix of superedges, W is a weight matrix of superedges, Y represents a labeled label matrix, and F is a label matrix to be learned, that is, a predicted grouping label of the pattern element.

[0035] In a second aspect, the application provides a pattern element grouping system based on supergraph learning, applied to the pattern element grouping method based on supergraph learning, and comprising a pattern preprocessing module, a gestalt feature extraction module, a gestalt correlation construction module, an adaptive supergraph fusion module, and a label propagation module.

[0036] The pattern preprocessing module is configured to preprocess the pattern elements, and the preprocessing comprises an image segmentation operation on the pattern elements to obtain a set of segmented pattern elements.

[0037] The gestalt feature extraction module is configured to extract gestalt features of the pattern elements, use a ring projection vector to describe the similarity of the element contours, use the minimum distance between the element contour point sets to describe the proximity between the elements, use a Mask R-CNN to extract the continuity between the elements, and use a data-driven encoder to extract the hybrid features between the elements.

[0038] The gestalt correlation construction module is configured to calculate the Euclidean distance of the features between the elements based on the extracted gestalt features of the elements, and construct a corresponding gestalt supergraph to describe the gestalt correlation between the pattern elements.

[0039] The adaptive supergraph fusion module is configured to construct an adaptive supergraph fusion grouping method, wherein the adaptive supergraph is used to guide the construction of superedges in the supergraph according to the distribution of the pattern elements and the pre-clustering results, so that the superedges can flexibly represent the correlation between different elements and obtain more accurate superedge connections; and the supergraph fusion grouping method is used to fuse the similarity, proximity, continuity, and hybrid supergraph by using the easy expansion capability of the supergraph, so as to flexibly realize the fusion of the gestalt rules and use multiple gestalt rules for the grouping of the pattern elements.

[0040] The label propagation module is configured to use a label propagation method to obtain the grouping result of the pattern elements, with the fused supergraph and the initial grouping label input by the user as conditions.

[0041] In a third aspect, the application provides an electronic device, which comprises:

[0042] at least one processor; and

[0043] a memory communicatively connected with the at least one processor; wherein

[0044] the memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to perform the pattern element grouping method based on hypergraph learning.

[0045] In a fourth aspect, the present application provides a computer readable storage medium storing a program, and the program is executed by a processor to implement the pattern element grouping method based on hypergraph learning.

[0046] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0047] 1. The present application defines four Gestalt rules applied to the grouping of pattern elements, including similarity, proximity, continuity and mixed Gestalt rules. When grouping different patterns, appropriate Gestalt rules can be selected for grouping, thereby improving the generalization of the grouper.

[0048] 2. The present application defines an adaptive hypergraph fusion method. This method utilizes the easy expandability of hypergraph to flexibly realize the fusion of Gestalt rules. The correlation between different Gestalt rules is modeled through hypergraph construction, and then the hypergraph is used for fusion. This method realizes the joint action of multiple Gestalt rules on pattern element grouping, thereby improving the robustness of the grouping.

[0049] 3. The present application proposes a new grouping method named pattern element grouping method based on hypergraph learning through Gestalt rules and hypergraph structure. Experiments on a challenging benchmark set (Pattern17) show that the pattern element grouping method based on hypergraph learning of the present application is superior to the baseline method and achieves excellent performance. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0051] Figure 1 is a flowchart of the pattern element grouping method based on hypergraph learning of the present application;

[0052] Figure 2 is a schematic diagram of a hypergraph construction method based on continuity rules according to an embodiment of the present application;

[0053] Figure 3 is a schematic diagram of a hypergraph fusion grouping method according to an embodiment of the present application;

[0054] Figure 4 is a user interaction interface display diagram according to an embodiment of the present application;

[0055] Figure 5 is a grouping result obtained by using the adaptive hypergraph fusion grouping method according to an embodiment of the present application, the first row is an input pattern, the second row is a predicted grouping result, and the third row is a labeled grouping result;

[0056] Figure 6 is a structural schematic diagram of a pattern element grouping system based on hypergraph learning according to an embodiment of the present application;

[0057] Figure 7 is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to enable persons skilled in the art to better understand the present application scheme, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor are within the scope of protection of the present application.

[0059] In the present application, the phrase "embodiment" means that the specific features, structures or characteristics described in conjunction with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor is it an independent or alternative embodiment to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in the present application can be combined with other embodiments.

[0060] Please refer to Figure 1 The present embodiment is a pattern element grouping method based on hypergraph learning, which comprises the following steps:

[0061] S1, pre-processing the elements in the pattern, performing segmentation operation on the elements in the pattern to obtain a segmented pattern element image set; in an embodiment of the present application, the pattern elements are cropped in the manner of minimum bounding box.

[0062] S2, extract the gestalt features of the pattern elements; use ring projection vectors to describe the similarity of the element contour; use the minimum distance between the element contour point sets to describe the proximity between the elements; use Mask R-CNN to extract the continuity between the elements; use a data-driven encoder to extract the hybrid features between the elements.

[0063] Further, step S2 is specifically:

[0064] S21, use ring projection vectors to describe the rotation invariance features of the pattern element contour, and then calculate the similarity of the contours between the pattern elements, the ring projection vector is defined as follows:

[0065]

[0066] Where f(x,y is the pixel value of the image f at (x,y), 2πr is the circumference of the ring with radius r, (x0,y0) is the center coordinate, r∈[0,R], R is the maximum circle radius. For the ring projection vectors P a and P b extracted from element a and element b, the similarity distance of element a and element b can be expressed as:

[0067]

[0068] S22, use the minimum distance between the limited point sets to describe the proximity between the elements, first use Marr edge detection operator to extract the edge contour of the pattern elements, then calculate the proximity distance d of the edge point sets of each other element, the distance d of element p and element q is:

[0069]

[0070] Where p i represents the i-th pixel point coordinate of element p, q j represents the j-th pixel point coordinate of element q.

[0071] S23, retrain the detection model Mask R-CNN by constructing a pattern data set containing continuity rules, so as to obtain the continuity mask in the pattern, and then construct the continuity hypergraph according to the mask.

[0072] S24, use a data-driven way to let the machine learn the hybrid features that integrate multiple gestalt rules from a large amount of pattern data, construct an atomic encoder and a structure encoder to encode the features as the hybrid features of the pattern elements. Extract the hybrid features F∈R N*40 of the elements in the pattern, then standardize F to F′∈R N*40 , and then use the Euclidean distance to calculate the distance D∈R between each other element.N*N .

[0073] S3, based on the extracted elements of Gestalt characteristics, calculating the Euclidean distance between the features of the elements, and constructing a corresponding Gestalt hypergraph to describe the Gestalt correlation between the pattern elements;

[0074] Further, step S3 is specifically:

[0075] S31, calculating the Euclidean distance between the features of each two elements, and constructing a matrix D e R N *N N represents the number of pattern elements. Then the distance matrix D is converted into a weight matrix W:

[0076]

[0077] S32, the weight matrix W describes the correlation between the pattern elements, based on the weight matrix W, a hypergraph H is constructed to represent the correlation between the pattern elements:

[0078]

[0079] Where m represents different Gestalt hypergraphs, topk(w T (j represents the w T The first k elements with the largest value in the jth column form a set, if w T (i,j) is in the set, then it is connected with a hyperedge.

[0080] Please refer to Figure 2 On the representation of the continuity hypergraph, the input pattern first passes through the hypergraph construction module to construct a continuity hypergraph. In the hypergraph construction module, the trained Mask R-CNN is first used to extract the detection mask of the input pattern, and each mask is provided with a set of continuity grouping information. Then, each mask is ANDed with the input pattern, that is, the elements covered by the mask are extracted, and these elements are divided into a group according to the continuity rule to construct a hyperedge, and the number of masks corresponds to the number of hyperedges.

[0081] S4, constructing an adaptive hypergraph fusion method, the adaptive hypergraph is used to guide the construction of the hyperedge in the hypergraph according to the distribution of the pattern elements and the pre-clustering result, so that the hyperedge can flexibly represent the correlation between different elements, and more accurate hyperedge connection is obtained; the hypergraph fusion grouping method is to use the easy expansion ability of the hypergraph to fuse the similarity, proximity, continuity and mixed hypergraph, so as to flexibly realize the fusion of the Gestalt rules, and multiple Gestalt rules are used together for the grouping of the pattern elements.

[0082] S5, using the fused hypergraph and the initial input of the user's grouping label as conditions, using a label propagation method to obtain the grouping result of the pattern element.

[0083]

[0084] Wherein H is the constructed hypergraph, D v is the degree matrix of the nodes in the hypergraph, D e is the degree matrix of the hyperedge, W is the weight matrix of the hyperedge, Y represents the labeled label matrix, and F is the label matrix to be learned, that is, the predicted grouping label of the pattern element.

[0085] Referring to Figure 3 , the user first gives the set of labeled elements in the pattern P l , the label set is Y l , and the remaining is the set of elements to be grouped P u , wherein the pattern element set P=P l +P u .Then the label set Y l and the pattern element set P are input. Then according to the appearance similarity between elements, the proximity degree on the position, the continuity degree on the distribution and the correlation degree based on the mixed Gestalt, the similarity hypergraph H sim , the proximity hypergraph H prx , the continuity hypergraph H con and the mixed hypergraph H mix are constructed to describe the correlation degree between elements based on different rules. Further, the four hypergraphs are fused into a hypergraph H, and the hypergraph H is passed through a label propagation module to obtain the grouping result of the set of elements to be grouped P u .

[0086] Figure 4 The interface of user interaction is shown, and the user inputs the grouping intention of the user by specifying the grouping label of part of the elements in the interface, selecting the Gestalt rule and the rule weight.

[0087] Referring to Figure 5 , the objective grouping result graph of the method of the present application is shown. Figure 5 The first behavior in the figure is the pattern with the user input label, the second behavior is the predicted grouping result, and the third behavior is the annotated grouping result. Analyzing the second row of the predicted result, it is observed that the subjective grouping intention is very consistent with the user's grouping intention, and almost has no difference with the true annotation.

[0088] The objective evaluation results of the method are shown in Table 1 and Table 2, Table 1 is the grouping result using non-adaptive hypergraph, and Table 2 is the grouping result using adaptive hypergraph fusion. The present application uses purity and Landau coefficient as evaluation indexes to evaluate the Pattern17 dataset, and the higher the purity and Landau coefficient, the better the grouping result. Compared with Table 1, the use of adaptive hypergraph fusion algorithm makes all the gestalt combinations obtain better performance.

[0089] Table 1

[0090]

[0091] Table 2

[0092]

[0093] The present application provides a new idea for pattern element perception grouping by constructing an adaptive hypergraph fusion grouping method. The gestalt grouping rules are represented on the pattern elements, and the connection relationship between the pattern elements is modeled and uniformly represented by using the hypergraph according to different grouping rules, the fusion of the gestalt rules is realized, and finally the pattern element grouping conforming to the user perception is realized through the man-machine interaction, effectively solving the problem of gestalt rule conflict, so that the machine can group the pattern elements according to the user's grouping intention.

[0094] It should be noted that for the foregoing method embodiments, in order to facilitate the description, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously.

[0095] Based on the same idea as the pattern element grouping method based on hypergraph learning in the above embodiment, the present application also provides a pattern element grouping system based on hypergraph learning, which can be used to execute the above pattern element grouping method based on hypergraph learning. For the convenience of description, in the structural schematic diagram of the embodiment of the pattern element grouping system based on hypergraph learning, only the part related to the embodiment of the present application is shown, and those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and can include more or fewer components than the illustrated, or combine certain components, or different component arrangements.

[0096] Please refer to Figure 6 In another embodiment of the present application, a pattern element grouping system 100 based on hypergraph learning is provided, which includes a pattern preprocessing module 101, a gestalt feature extraction module 102, a gestalt correlation construction module 103, an adaptive hypergraph fusion module 104 and a label propagation module 105.

[0097] The pattern preprocessing module 101 is configured to preprocess the pattern elements, and the preprocessing includes an image segmentation operation on the pattern elements to obtain a set of segmented pattern elements.

[0098] The Gestalt feature extraction module 102 is configured to extract the Gestalt features of the pattern elements, use ring projection vectors to describe the similarity of the element contours, use the minimum distance between the element contour point sets to describe the proximity between the elements, use Mask R-CNN to extract the continuity between the elements, and use a data-driven encoder to extract the hybrid features between the elements.

[0099] The Gestalt correlation construction module 103 is configured to calculate the Euclidean distance of the features between the elements based on the extracted Gestalt features of the elements, and construct a corresponding Gestalt hypergraph to describe the Gestalt correlation between the pattern elements.

[0100] The adaptive hypergraph fusion module 104 is configured to construct an adaptive hypergraph fusion grouping method, and the adaptive hypergraph is used to guide the construction of the hyperedges in the hypergraph according to the distribution and pre-clustering results of the pattern elements, so that the hyperedges can flexibly represent the correlation between different elements and obtain more accurate hyperedge connections.

[0101] The label propagation module 105 is configured to use a label propagation method to obtain the grouping results of the pattern elements, with the fused hypergraph and the initial grouping labels input by the user as conditions.

[0102] It should be noted that the pattern element grouping system based on hypergraph learning of the present application corresponds to the pattern element grouping method based on hypergraph learning of the present application, and the technical features and advantages described in the above embodiment of the pattern element grouping method based on hypergraph learning are applicable to the embodiment of the pattern element grouping based on hypergraph learning, and the specific content can be referred to the description in the method embodiment of the present application, which will not be repeated here.

[0103] In addition, in the embodiment of the pattern element grouping system based on hypergraph learning of the above embodiment, the logical division of each program module is only an example, and in actual application, the above functions can be completed by different program modules according to the needs, for example, the configuration requirements of the corresponding hardware or the convenience of software implementation, that is, the internal structure of the pattern element grouping system based on hypergraph learning is divided into different program modules to complete all or part of the functions described above.

[0104] Please refer to Figure 7In one embodiment, an electronic device implementing a pattern element grouping method based on hypergraph learning is provided. The electronic device 200 can include a first processor 201, a first memory 202, and a bus, and can further include a computer program, such as a pattern element grouping program 203 based on hypergraph learning, stored in the first memory 202 and executable on the first processor 201.

[0105] The first memory 202 includes at least one type of readable storage medium, including a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the first memory 202 can be an internal storage unit of the electronic device 200, such as a mobile hard disk of the electronic device 200. In other embodiments, the first memory 202 can also be an external storage device of the electronic device 200, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the first memory 202 can include both an internal storage unit and an external storage device of the electronic device 200. The first memory 202 can be used to store application software installed in the electronic device 200 and various data, such as the code of the pattern element grouping program 203 based on hypergraph learning, and can also be used to temporarily store data that has been output or will be output.

[0106] The first processor 201 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, combinations of various control chips, etc. The first processor 201 is the control unit of the electronic device, which connects various components of the entire electronic device through various interfaces and lines, and executes or runs programs or modules stored in the first memory 202 and calls data stored in the first memory 202 to perform various functions and process data of the electronic device 200.

[0107] Figure 7 Only an electronic device with components is shown, and those skilled in the art can understand that, Figure 7 The structure shown does not constitute a limitation on the electronic device 200, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0108] The first memory 202 in the electronic device 200 stores a pattern element grouping program 203 based on hypergraph learning, which is a combination of multiple instructions and can achieve the following when running in the first processor 201:

[0109] Preprocessing the pattern elements, including image segmentation operation on the pattern elements to obtain a set of segmented pattern elements;

[0110] Extracting the Gestalt features of the pattern elements, using ring projection vectors to describe the similarity of the element contours; using the minimum distance between the element contour point sets to describe the proximity between the elements; using Mask R-CNN to extract the continuity between the elements; using a data-driven encoder to extract hybrid features between the elements;

[0111] Based on the extracted Gestalt features of the elements, calculating the Euclidean distance of the features between the elements and constructing a corresponding Gestalt hypergraph to describe the Gestalt correlation between the pattern elements;

[0112] Constructing an adaptive hypergraph fusion grouping method, the adaptive hypergraph is constructed according to the distribution of the pattern elements and the pre-clustering results to guide the construction of the hyperedges in the hypergraph, so that the hyperedges can flexibly represent the correlation between different elements and obtain more accurate hyperedge connections; the hypergraph fusion grouping method is to use the easy expansion ability of the hypergraph to fuse the similarity, proximity, continuity and hybrid hypergraph, thereby flexibly realizing the fusion of the Gestalt rules and using multiple Gestalt rules for the grouping of the pattern elements;

[0113] Using a label propagation method to obtain the grouping results of the pattern elements based on the fused hypergraph and the user's initial input grouping labels.

[0114] Further, the modules / units of the electronic device 200, if implemented in the form of software function units and sold or used as independent products, can be stored in a non-volatile computer readable storage medium. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).

[0115] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer readable storage medium, and when the program is executed, the processes of the above-mentioned embodiment methods can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0116] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0117] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations and simplifications made without departing from the spirit and principles of the present application should be equivalent replacement methods, and are included in the protection scope of the present application.

Claims

1. A pattern element grouping method based on hypergraph learning, characterized in that, The method comprises the following steps: The pattern elements are preprocessed, and the preprocessing comprises an image segmentation operation on the pattern elements to obtain a set of segmented pattern elements; Gestalt features of the pattern elements are extracted, and a ring projection vector is used to describe the similarity of the element contours; the minimum distance between the element contour point sets is used to describe the proximity between the elements; Mask R-CNN is used to extract the continuity between the elements; a data-driven encoder is used to extract mixed features between the elements; Based on the extracted gestalt features of the elements, the Euclidean distance of the features between the elements is calculated, and a corresponding gestalt hypergraph is constructed to describe the gestalt correlation between the pattern elements; An adaptive hypergraph fusion grouping method is constructed, and the adaptive hypergraph is used to guide the construction of the hyperedges in the hypergraph according to the distribution and pre-clustering results of the pattern elements, so that the hyperedges can flexibly represent the correlation between different elements and obtain more accurate hyperedge connections; the hypergraph fusion grouping method is used to fuse the similarity, proximity, continuity and mixed hypergraph by using the easy expansion capability of the hypergraph, so that the fusion of the gestalt rules is flexibly realized, and the gestalt rules are used for the grouping of the pattern elements; The grouping results of the pattern elements are obtained by using a label propagation method based on the fused hypergraph and the initial grouping labels input by the user.

2. The method of claim 1, wherein, The ring projection vector is used to describe the similarity of the element contours, and the specific process is as follows: The ring projection vector is used to describe the rotation invariance feature of the element contour, and then the similarity of the element contours is calculated, and the ring projection vector is defined as follows: where f(x, y) is the pixel value of the image f at (x, y), 2πr is the circumference of a ring of radius r, (x0, y0) is the center coordinate, r e [0, R], R is the maximum circle's circumference radius; For element a and element b extracted ring projection vectors P a and P b , the similarity distance of element a and element b can be expressed as:

3. The method of claim 1, wherein, The minimum distance between the limited point sets is used to describe the proximity between the elements, and the specific process is as follows: The minimum distance between the limited point sets is used to describe the proximity between the elements, and the specific process is as follows: where p i represents the i-th pixel point coordinate of element p, q j represents the j-th pixel point coordinate of element q.

4. The method of claim 1, wherein, The continuity between the elements is extracted by using Mask R-CNN, and the specific process is as follows: The detection model Mask R-CNN is retrained by constructing a pattern data set containing the continuity rule, so as to obtain the continuity mask in the pattern, and then the continuity hypergraph is constructed according to the mask.

5. The method of claim 1, wherein, The data-driven encoder is used to extract the mixed features between the elements, and the specific process is as follows: Using data-driven way, let the machine learn the mixed features of fusion of multiple Gestalt rules from a large amount of pattern data, construct atomic encoder and structure encoder to encode features as mixed features of pattern elements, extract mixed features F∈R of elements in the pattern N*40 Then standardize F to F'∈R N*40 Then use Euclidean distance to calculate the distance D∈R between each two elements N*N .

6. The method of claim 1, wherein, Based on the extracted gestalt features of the elements, the Euclidean distance of the features between the elements is calculated, and a corresponding gestalt hypergraph is constructed to describe the gestalt correlation between the pattern elements, and the specific process is as follows: The correlation between the pattern elements based on different gestalt rules is represented by a supergraph, the Euclidean distance of the features between each pair of elements is calculated, and a distance matrix D e R is constructed to describe the distance between the pattern elements N*N N represents the number of pattern elements, and then the distance matrix D is converted into a weight matrix W: The weight matrix W describes the correlation between the pattern elements, and based on the weight matrix W, the hypergraph H is constructed to represent the correlation between the pattern elements: where m represents different Gestalt hypergraphs, topk(w T (j)) represents w T The set of the first k elements with the largest values in the jth column, if w T (i, j) is in the set, then it is connected by a hyperedge.

7. The method of claim 1, wherein, The hypergraph label propagation method is defined as follows: where H is the constructed hypergraph, D v is the degree matrix of the nodes in the hypergraph, D e is the degree matrix of the hyperedges, W is the weight matrix of the hyperedges, Y represents the labeled label matrix, and F is the label matrix to be learned, i.e., the predicted grouping label of the pattern elements.

8. A pattern element grouping system based on hypergraph learning, characterized in that, The pattern element grouping method based on hypergraph learning is applied to any one of claims 1-7, comprising a pattern preprocessing module, a gestalt feature extraction module, a gestalt correlation construction module, an adaptive hypergraph fusion module and a label propagation module; The pattern preprocessing module is configured to preprocess the pattern elements, and the preprocessing includes an image segmentation operation on the pattern elements to obtain a set of segmented pattern elements. The Gestalt feature extraction module is configured to extract Gestalt features of the pattern elements, use ring projection vectors to describe the similarity of the element contours, use minimum distances between the element contour point sets to describe the proximity between the elements, use Mask R-CNN to extract the continuity between the elements, and use a data-driven encoder to extract mixed features between the elements. The Gestalt correlation construction module is configured to calculate the Euclidean distances of the features between the elements based on the extracted Gestalt features of the elements, and construct a corresponding Gestalt hypergraph to describe the Gestalt correlations between the pattern elements. The adaptive hypergraph fusion module is configured to construct an adaptive hypergraph fusion grouping method, the adaptive hypergraph is constructed according to the distribution and pre-clustering results of the pattern elements to guide the construction of hyperedges in the hypergraph, so that the hyperedges can flexibly represent the correlations between different elements to obtain more accurate hyperedge connections, and the hypergraph fusion grouping method is used to fuse the similarity, proximity, continuity and mixed hypergraph by using the easy expansion capability of the hypergraph, thereby flexibly realizing the fusion of the Gestalt rules and using multiple Gestalt rules for the grouping of the pattern elements. The label propagation module is configured to use a label propagation method to obtain the grouping results of the pattern elements based on the fused hypergraph and the initial input grouping labels of the user.

9. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to perform the pattern element grouping method based on hypergraph learning according to any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that, The program is executed by the processor to implement the pattern element grouping method based on hypergraph learning according to any one of claims 1-7. The program is executed by the processor to implement the pattern element grouping method based on hypergraph learning according to any one of claims 1-7.

Images