Key value matching method, device, readable medium and electronic device

Abstract

The present disclosure relates to a key-value matching method, device, readable medium and electronic device. The key-value matching method obtains the first position information of at least one attribute data and the second position information of at least one attribute value data in the image to be detected; generates a first relationship graph according to the first position information of the at least one attribute data, and generates a second relationship graph according to the second position information of the at least one attribute value data; inputs the first position information of the at least one attribute data, the second position information of the at least one attribute value data, the first relationship graph, the second relationship graph and the image to be detected into a preset graph matching model to obtain the matching relationship between the attribute data and the attribute value data. In this way, by performing key-value matching through the preset graph matching model, not only can key-value matching be performed for different scenarios to improve the generalization of the model, but also the accuracy of the key-value matching results can be effectively guaranteed.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a key-value matching method, device, readable medium, and electronic device. Background Art

[0002] Key-value matching in document images refers to the process of extracting pairs of text that form key-value relationships within a document image. For example, in a business license image, the name forms a key-value relationship with "Company X," in an ID card image, the name forms a key-value relationship with "Mr. Zhang," and in a diploma image, the school forms a key-value relationship with "University X." Key-value matching involves identifying and extracting key-value pairs within these key-value relationships.

[0003] Current key-value matching methods typically either combine the text recognition results with a preset relational dictionary for search and matching based on the positional relationship between key-value pairs, or perform classification prediction on the key-value pairs to be matched. However, both relational dictionary search and matching and classification prediction matching have poor generalization and can only be applied to problems in specific scenarios. Summary of the Invention

[0004] This summary is provided to briefly introduce concepts that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] The present disclosure provides a key-value matching method, device, readable medium, and electronic device.

[0006] In a first aspect, the present disclosure provides a key-value matching method, the method comprising:

[0007] Acquire first position information of at least one attribute data and second position information of at least one attribute value data in the image to be detected;

[0008] generating a first relationship graph according to the first position information of the at least one attribute data, and generating a second relationship graph according to the second position information of the at least one attribute value data;

[0009] The first position information of the at least one attribute data, the second position information of the at least one attribute value data, the first relationship graph, the second relationship graph and the image to be detected are input into a preset graph matching model to obtain a matching relationship between the attribute data and the attribute value data.

[0010] In a second aspect, the present disclosure provides a key-value matching device, the device comprising:

[0011] A first acquisition module is used to acquire first position information of at least one attribute data and second position information of at least one attribute value data in the image to be detected;

[0012] a relationship graph generating module, configured to generate a first relationship graph according to the first position information of the at least one attribute data, and generate a second relationship graph according to the second position information of the at least one attribute value data;

[0013] A determination module is used to input the first position information of the at least one attribute data, the second position information of the at least one attribute value data, the first relationship graph, the second relationship graph and the image to be detected into a preset graph matching model to obtain a matching relationship between the attribute data and the attribute value data.

[0014] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect above.

[0015] In a fourth aspect, the present disclosure provides an electronic device, comprising:

[0016] a storage device having a computer program stored thereon;

[0017] A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect above.

[0018] The above technical solution inputs the first position information of the at least one attribute data, the second position information of the at least one attribute value data, the first relationship graph corresponding to the first position information of the at least one attribute data, the second relationship graph corresponding to the second position information of the at least one attribute value data, and the image to be detected into a preset graph matching model, so that the preset graph matching model outputs the matching relationship between the attribute data and the attribute value data. In this way, key-value matching is performed through the preset graph matching model, which not only can perform key-value matching for different scenarios and improve the generalization of the model, but also can effectively ensure the accuracy of the key-value matching results.

[0019] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale. In the drawings:

[0021] Figure 1is a flowchart of a key-value matching method shown in an exemplary embodiment of the present disclosure;

[0022] Figure 2 This is a flowchart of a key-value matching method according to an exemplary embodiment of the present disclosure;

[0023] Figure 3 is based on Figure 1 The illustrated embodiment shows a flow chart of a key-value matching method;

[0024] Figure 4 is based on Figure 1 A flow chart of another key-value matching method shown in the illustrated embodiment;

[0025] Figure 5 is a block diagram of a key-value matching device shown in another exemplary embodiment of the present disclosure;

[0026] Figure 6 is based on Figure 5 The illustrated embodiment shows a block diagram of a key-value matching device;

[0027] Figure 7 It is a block diagram of an electronic device shown in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0029] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0030] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0031] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0032] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0033] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0034] Before introducing the specific implementation methods of the present disclosure, the application scenarios of the present disclosure are first described as follows. The present disclosure can be applied to the process of identifying and extracting key-value pairs in document images, wherein the document image can be a business license image, a degree certificate image, a graduation certificate image, an ID card image or other document image. The key-value pair refers to a group of texts with a key-value relationship. For example, in a business license image, the name and a certain company form a key-value relationship and belong to a key-value pair; in an ID card image, the name and Zhang San form a key-value relationship and belong to a key-value pair; in a graduation certificate image, the school and a certain university form a key-value relationship and belong to a key-value pair.

[0035] In the related art, the method for key-value matching is either to search and match the results of text recognition based on the positional relationship between the key values ​​in combination with a preset relational dictionary. This method often fails to find the matching key value in the dictionary because the key-value pair data collected in the relational dictionary is small and the scenarios involved are relatively simple; or the data in the processed key-value pairs are divided into n categories and a multi-label detection model is used to detect the data in the image. This method usually requires training different multi-label detection models for different scenarios, and the multi-label detection model in each scenario requires a large amount of multi-label annotated data. In other words, the annotation cost required to train the multi-label detection model is relatively high, and the obtained multi-label detection model has weak generalization and can only be applied to specific scenarios. It cannot be applied to scenarios other than specific scenarios. Obviously, both relational dictionary search and matching and classification prediction matching have poor generalization and can only be applied to specific scenarios.

[0036] In order to solve the above technical problems, the present disclosure provides a key-value matching method, device, readable medium and electronic device. The key-value matching method obtains the first position information of at least one attribute data and the second position information of at least one attribute value data in the image to be detected; generates a first relationship graph according to the first position information of the at least one attribute data, and generates a second relationship graph according to the second position information of the at least one attribute value data; inputs the first position information of the at least one attribute data, the second position information of the at least one attribute value data, the first relationship graph, the second relationship graph and the image to be detected into a preset graph matching model to obtain the matching relationship between the attribute data and the attribute value data. In this way, by performing key-value matching through the preset graph matching model, not only can key-value matching be performed for different scenarios to ensure the generalization of the model, but also the accuracy of the key-value matching results can be effectively guaranteed.

[0037] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0038] Figure 1 is a flowchart of a key-value matching method shown in an exemplary embodiment of the present disclosure; see Figure 1 , the method may include:

[0039] Step 101: Acquire first position information of at least one attribute data and second position information of at least one attribute value data in an image to be detected.

[0040] The image to be detected can be a business license image, degree certificate image, graduation certificate image, ID card image, etc. The attribute data is the data corresponding to the key in the key-value pair, and the attribute value data is the data corresponding to the value in the key-value pair. The key and value form a key-value pair.

[0041] In this step, the attribute data and attribute value data in the image to be detected can be detected in categories and positions using a binary classification detection model in the prior art to output a first detection frame corresponding to each attribute data and a second detection frame corresponding to each attribute value data in the image to be detected. The first position information is determined based on the first detection frame, and the second position information is determined based on the second detection frame. For example, in one embodiment, the position of the first detection frame can be used as the first position information, and the second detection frame can be used as the second position information; in another embodiment, the center position of the first detection frame can be used as the first position information, and the center position of the second detection frame can be used as the second position information; in yet another embodiment, any point on the first detection frame can be used as the first position information, and any point on the second detection frame can be used as the second position information.

[0042] Step 102: Generate a first relationship graph based on the first position information of the at least one attribute data, and generate a second relationship graph based on the second position information of the at least one attribute value data.

[0043] The first relationship diagram includes an attribute node corresponding to each attribute data location and first lines between different attribute nodes, and the second relationship diagram includes an attribute value node corresponding to each attribute value data location and second lines between different attribute value nodes.

[0044] In this step, the first relationship graph may be generated by a Delaunay triangulation mapping method according to the first position information of the at least one attribute data; and the second relationship graph may be generated by a full-connection mapping method according to the second position information of the at least one attribute value data.

[0045] It should be noted that the Delaunay triangulation mapping method (Delaunay triangulation algorithm) and the fully connected mapping method (i.e., establishing a fully connected network topology graph) are both commonly used mapping methods in the prior art and will not be described in detail in this disclosure.

[0046] Step 103: Input the first position information of the at least one attribute data, the second position information of the at least one attribute value data, the first relationship graph, the second relationship graph and the image to be detected into a preset graph matching model to obtain a matching relationship between the attribute data and the attribute value data.

[0047] The preset graph matching model is trained through the following steps S1 to S3:

[0048] S1, obtain model training sample data.

[0049] Among them, the model training sample data includes multiple image samples to be detected, the first position information of each attribute data in each image sample to be detected, the second position information of each attribute value data in the image sample to be detected, and the first relationship graph and the second relationship graph corresponding to each image sample to be detected.

[0050] It should be noted that the image sample to be detected includes annotations of types of attribute data and attribute value data, as well as annotation data of corresponding relationships between the attribute data and attribute value data.

[0051] S2, according to the first position information of each attribute data, obtain the first feature corresponding to each attribute node in the first relationship graph of the image sample to be detected and the second feature corresponding to each first connection from the image sample to be detected through the preset network initial model, and according to the second position information of each attribute value data, obtain the third feature corresponding to each attribute value node in the second relationship graph of the image sample to be detected and the fourth feature corresponding to each second connection from the image sample to be detected through the preset network initial model.

[0052] The preset network initial model may include a first initial sub-network and a second initial sub-network. The preset initial network model may be a neural network model or other machine learning algorithm model.

[0053] In this step, the first position information of at least one attribute data corresponding to the image sample to be detected, the first relationship graph and the image to be detected can be used as the input of the first initial sub-network to output the first feature corresponding to each attribute node corresponding to the image sample to be detected, and the first position information of at least one attribute data corresponding to the image sample to be detected, the first relationship graph and the image to be detected can be used as the input of the second initial sub-network to output the second feature corresponding to each first line in the image sample to be detected; the second position information of at least one attribute value data corresponding to the image sample to be detected, the second relationship graph and the image to be detected can be used as the input of the first initial sub-network to output the third feature corresponding to each attribute value node in the image sample to be detected, and the second position information of at least one attribute value data corresponding to the image sample to be detected, the second relationship graph and the image to be detected can be used as the input of the second initial sub-network to output the fourth feature corresponding to each second line in the image sample to be detected.

[0054] Among them, the depth of the second initial sub-network is greater than the depth of the first initial sub-network, that is, the number of hidden layers of the first initial sub-network is less than that of the second initial sub-network, the first feature corresponding to the attribute node and the third feature corresponding to the attribute value node are output by the shallow network (first initial sub-network), and the second feature corresponding to the first connection and the fourth feature corresponding to the second connection are output by the deep network (second initial sub-network).

[0055] It should be noted that, first, the first feature and the third feature are obtained by the shallow network, and the second feature and the fourth feature are obtained by the deep network, which can effectively obtain the image features corresponding to the attribute node, the attribute value node, the first link, and the second link. Secondly, since the shallower the network depth, the closer it is to the input, the more feature details it contains, and the more accurate the description of the node (the attribute node or the attribute value node) by the extracted features, the accuracy of the first feature of the attribute node and the third feature of the attribute value node mentioned can be effectively guaranteed, which is conducive to providing a reliable data basis for key-value matching; extracting the features of the edges (i.e., the first link and the second link) through the deep network can effectively reduce the amount of data processing and improve the model processing efficiency.

[0056] In addition, the first initial sub-network and the second initial sub-network may include a pooling layer. When obtaining the first feature, the second feature, the third feature, and the fourth feature, the extracted features may be average pooled through the pooling layer. For example, the shape of the feature map is 256×64×64 (number of channels 256, length 64, width 64). When extracting features, the detection box corresponding to the node may be scaled proportionally, and then features are extracted at the corresponding positions. The extracted features are then average pooled through the pooling layer to obtain a feature vector (256×1×1) for each detection box. In this way, the data dimension can be effectively reduced, thereby simplifying the network complexity, reducing the amount of model calculation, and achieving the technical effect of improving model efficiency.

[0057] S3, based on the first feature corresponding to each attribute node and the second feature corresponding to each first connection in the first relationship graph of the image sample to be detected, as well as the third feature corresponding to each attribute value node and the fourth feature corresponding to each second connection in the second relationship graph of the image sample to be detected, calculate the loss value corresponding to the distance vector between each attribute node and the attribute value node to be matched through a preset loss function, and iteratively train the preset initial network model according to the loss value to obtain the preset graph matching model.

[0058] In this step, a node similarity matrix can be determined based on the first feature corresponding to each attribute node and the third feature corresponding to each attribute value node, and a line similarity matrix can be determined based on the second feature corresponding to each first line and the fourth feature corresponding to each second line; a target relationship matrix is ​​generated based on the node similarity matrix and the line similarity matrix; a double random matrix corresponding to the target relationship matrix is ​​obtained; a distance vector between each attribute node and the attribute value node to be matched is determined based on the double random matrix; and the loss value is determined based on the distance vector using a preset loss function.

[0059] For example, the first relationship graph can be represented by the first adjacency matrix A1, the second relationship graph can be represented by the second adjacency matrix A2, and the formula A=GH T Determine that the correlation matrices corresponding to the first adjacency matrix A1 are G1 and H1, and the correlation matrices corresponding to the second adjacency matrix A2 are G2 and H2. In step S2, when the first feature is determined to be P1, the second feature is E1, the third feature is P2, and the fourth feature is E2, M p =P1P2 to obtain the node similarity matrix M p , and determine the link similarity matrix M by the following formula 1 e :

[0060] M e =[E1G1|E1H1]Λ[E2G2|E2H2] T ...Formula 1

[0061] In the above formula 1, the Λ can be a symmetric parameter matrix;

[0062] Then, the node similarity matrix M can be calculated by the following formula 2 p And the link similarity matrix M e Determine the target relationship matrix M:

[0063]

[0064] In the above formula 2: vec(x) represents the row-wise expansion of x, [x] represents the diagonal matrix of x, Kroneckerproduct, Kronecker product.

[0065] Furthermore, the eigenvector V corresponding to the target relationship matrix M can be obtained, and then the eigenvector is double-randomized to obtain a double-random matrix S corresponding to the eigenvector V. The distance vector between the attribute node and each attribute value node is determined according to the double-random matrix using the following formula 3:

[0066]

[0067] In the above formula 3, α is a preset coefficient, for example, it can be 200, S is a double random matrix, i represents the row number of the double random matrix S, j represents the column number of the double random matrix S, S(i,1…m) represents the i-th row of the double random matrix S, and the double random matrix S has a total of m rows. P is the position set of the attribute value node, P i is the position of the i-th attribute node, The weight that represents the feature of the i-th attribute value node relative to the position set P of the attribute value node.

[0068] Next, the loss value corresponding to the distance vector between each attribute node and the attribute value node to be matched can be calculated by the following preset loss function L(d), where the preset loss function is as follows:

[0069]

[0070] In the above loss function, The distance vector of the target is calculated based on the location of the marked attribute data and the location of the corresponding attribute value data, and ∈ is a random decimal.

[0071] During the training process, the loss value corresponding to the distance vector between each attribute node and the attribute value node to be matched can be obtained. When the loss value is less than or equal to the preset loss value threshold, the model training is determined to be completed and the optimal preset graph matching model is obtained.

[0072] The preset graph matching model obtained through the above training method has strong generalization and can be applied to a variety of different key-value matching scenarios. For example, it can be used for key-value matching of ID card images, as well as key-value matching in multiple scenarios such as business license images and degree certificate images.

[0073] For example, Figure 2 As shown, Figure 2 This is a flowchart of a key-value matching method shown in an exemplary embodiment of the present disclosure. Figure 2 The image 1 to 4 are included. The image 1 is an image to be detected. The image to be detected includes attribute data Key1 and Key2, as well as attribute value data Value1 and Value2. The positions of Key1, Key2, Value1, and Value2 in the image 1 are obtained by the binary classification detection model, thereby obtaining the detection box in the image 2. Each detection box in the image 2 represents the position of the identified attribute data or attribute value data. After the preset graph matching model, the matching of Key1 and Value1, and the matching of Key2 and Value2 are obtained (such as Figure 2 3), according to the matching relationship, through text recognition, the text information Key1: Value1 and Key2: Value2 (such as Figure 2 4). In this way, by performing key-value matching through the preset graph matching model, the accuracy of the key-value matching results can be effectively guaranteed.

[0074] The above technical solution inputs the first position information of at least one attribute data, the second position information of at least one attribute value data, the first relationship graph corresponding to the first position information of at least one attribute data, the second relationship graph corresponding to the second position information of at least one attribute value data, and the image to be detected into a preset graph matching model, so that the preset graph matching model outputs the matching relationship between the attribute data and the attribute value data. In this way, key-value matching is performed through the preset graph matching model, which not only can perform key-value matching for different scenarios and improve the generalization of the model, but also can effectively ensure the accuracy of the key-value matching results.

[0075] Furthermore, Figure 1 In step 103, the preset graph matching model is performed by the following Figure 3 The steps shown determine the matching relationship between the attribute data and the attribute value data based on the first position information of the at least one attribute data, the second position information of the at least one attribute value data, the first relationship graph, the second relationship graph and the image to be detected. Figure 3 is based on Figure 1 The embodiment shown is a flowchart of a key-value matching method, see Figure 3 , this preset graph matching model can be used for:

[0076] Step 1031: Obtain, from the image to be detected, the first feature corresponding to each attribute node in the first relationship graph and the second feature corresponding to each first connection according to the first position information of the at least one attribute data, and obtain, from the image to be detected, the third feature corresponding to each attribute value node in the second relationship graph and the fourth feature corresponding to each second connection according to the second position information of the at least one attribute value data.

[0077] In this step, the first position information of the at least one attribute data, the first relationship graph and the image to be detected can be used as inputs of the first sub-network to output the first feature corresponding to each attribute node, and the first position information of the at least one attribute data, the first relationship graph and the image to be detected can be used as inputs of the second sub-network to output the second feature corresponding to each first connection line, wherein the number of hidden layers of the first sub-network is less than that of the second sub-network; the second position information of the at least one attribute value data, the second relationship graph and the image to be detected can be used as inputs of the first sub-network to output the third feature corresponding to each attribute value node, and the second position information of the at least one attribute value data, the second relationship graph and the image to be detected can be used as inputs of the second sub-network to output the fourth feature corresponding to each second connection line.

[0078] It should be noted that the first subnetwork has fewer hidden layers than the second subnetwork, that is, the depth of the second subnetwork is greater than the depth of the first subnetwork. The first feature corresponding to the attribute node and the third feature corresponding to the attribute value node are output by the first subnetwork (i.e., a shallow network with fewer hidden layers), while the second feature corresponding to the first link and the fourth feature corresponding to the second link are output by the second subnetwork (i.e., a deep network with more hidden layers). Since the shallower the network depth, the closer it is to the input, the more feature details it contains, and the more accurate the description of the node (the attribute node or attribute value node) by the extracted features, outputting the first and third features through the shallow network can effectively ensure the accuracy of the first and third features, which is conducive to providing a reliable data basis for key-value matching. Extracting the features of the edges (i.e., the first and second links) through the deep network can effectively reduce the amount of data processing and improve model processing efficiency.

[0079] Step 1032: determine the matching relationship between the attribute data and the attribute value data based on the first feature corresponding to each attribute node and the second feature corresponding to each first connection, as well as the third feature corresponding to each attribute value node and the fourth feature corresponding to each second connection.

[0080] In this step, a possible implementation may include the following steps S21 to S23:

[0081] S21 , determining a node similarity matrix according to the first feature corresponding to each attribute node and the third feature corresponding to each attribute value node.

[0082] In this step, when the first feature corresponding to the attribute node is P1 and the third feature corresponding to the attribute value node is P2, M p =P1P2 to obtain the node similarity matrix M p .

[0083] S22: Determine a link similarity matrix according to the second feature corresponding to each first link and the fourth feature corresponding to each second link.

[0084] In this step, when the first feature corresponding to the attribute node is P1, the third feature corresponding to the attribute value node is P2, the second feature corresponding to the first connection is E1, and the fourth feature corresponding to the second connection is E2, the connection similarity matrix M can be determined by the following formula e :

[0085] M e =[E1G1|E1H1]Λ[E2G2|E2H2] T

[0086] In the above formula, Λ may be a symmetric parameter matrix, for example, a 2×2 symmetric parameter matrix.

[0087] S23: Determine a matching relationship between each attribute data and each attribute value data according to the node similarity matrix and the link similarity matrix.

[0088] In this step, the node similarity matrix M can be calculated by the following formula p And the link similarity matrix M e Determine the target relationship matrix M:

[0089]

[0090] In the above formula: vec(x) represents the row-wise expansion of x, [x] represents the diagonal matrix of x, Kronecker product, Kronecker product.

[0091] After the target relationship matrix M is obtained, the eigenvector V corresponding to the target relationship matrix M can be obtained, and the matching relationship between each attribute data and each attribute value data can be determined based on the eigenvector V.

[0092] Optionally, in this step, the eigenvector V may be subjected to a double randomization process to obtain a double random matrix S, and the matching relationship between each attribute data and each attribute value data is determined based on the double randomization matrix S. The process of performing the double randomization process is related to the prior art. For details on the process of performing the double randomization process on the eigenvector V, reference may be made to the implementation methods in the prior art, and this disclosure will not elaborate on this further.

[0093] For example, if the image to be detected includes three attribute data (Key1, Key2, Key3) and three attribute value data (Value1, Value2, Value3), and the obtained double random matrix S is: Among them, the rows of the matrix represent Key1, Key2, Key3, and the columns of the matrix represent Value1, Value2, Value3, which indicates that the Key1 matches the Value1, the Key2 matches the Value3, and the Key3 matches the Value2.

[0094] The above technical solution can determine the matching relationship between the attribute data and the attribute value data based on the first position information of the at least one attribute data, the second position information of the at least one attribute value data, the first relationship diagram, the second relationship diagram and the image to be detected, and can effectively ensure the accuracy of the key-value matching results.

[0095] Optionally, in Figure 1Before generating the first relationship graph according to the first position information of the at least one attribute data and generating the second relationship graph according to the second position information of the at least one attribute value data in step 102, the method may further include the following steps: Figure 4 The steps shown are: Figure 4 is based on Figure 1 The embodiment shown is a flowchart of another key-value matching method, as shown in FIG. Figure 4 As shown, the method may further include:

[0096] Step 104: Obtain a first quantity of the attribute data and a second quantity of the attribute value data.

[0097] For example, the first number of the attribute data may be the number of Key detection boxes in the image to be detected, and the second number of the attribute value data may be the number of Value detection boxes in the image to be detected.

[0098] Step 105, when the first quantity is not equal to the second quantity, the quantity of the attribute data is made equal to the quantity of the attribute value data by adding the same attribute data at at least one location where the attribute data is located, or adding the same attribute value data at at least one location where the attribute value data is located.

[0099] In this step, a possible implementation includes: when the first quantity is less than the second quantity, adding an identical attribute data at the location of each attribute data, obtaining a first difference between twice the first quantity and the second quantity, obtaining a number of target attribute value data corresponding to the first difference from at least one of the attribute value data, and adding an identical attribute value data at the location of each target attribute value data, so as to obtain twice the first quantity of attribute data and twice the first quantity of attribute value data;

[0100] Accordingly, according to the above embodiment, Figure 1 The implementation of step 102 may include: generating a first relationship graph based on first position information corresponding to twice the first amount of attribute data; generating a second relationship graph based on second position information corresponding to twice the first amount of attribute value data.

[0101] For example, if the first number of attribute data is a and the second number of attribute value data is b, and if a is less than b, one data point is directly added to the location of each attribute data point in the image to be detected, thus obtaining 2a attribute data points. For the attribute value data points, 2a-b attribute value data points are randomly selected from the image to be detected, and one attribute value data point is sequentially added to the locations of these attribute value data points. Thus, 2a attribute data points and 2a attribute value data points are obtained.

[0102] In this step, another possible implementation includes: when the first quantity is greater than the second quantity, adding an identical attribute value data at the location of each attribute value data, obtaining a second difference between twice the second quantity and the first quantity, and obtaining a target attribute data corresponding to the second difference from at least one of the attribute data, and adding an identical attribute data at the location of the target attribute data to obtain twice the second quantity of attribute data and twice the second quantity of attribute value data.

[0103] Accordingly, according to the above embodiment, Figure 1 The implementation of step 102 may include: generating a first relationship graph based on the first position information corresponding to the twice second amount of attribute data; generating a second relationship graph based on the second position information corresponding to the twice second amount of attribute value data.

[0104] For example, if the first number of attribute data is a and the second number of attribute value data is b, and a is greater than b, one data point is directly added to the location of each attribute value data point in the image to be detected, thus obtaining 2b attribute value data points. For the attribute data points, 2b-a attribute data points are randomly selected from the image to be detected, and one attribute data point is sequentially added to the locations of these attribute data points. This results in 2b attribute data points and 2b attribute value data points.

[0105] The above technical solution can effectively ensure that the quantity of the attribute data is equal to the quantity of the attribute value data, thereby ensuring that each attribute data can be matched to the corresponding attribute value data, which is conducive to expanding the range of processable images to be detected and expanding the scope of application of the key-value matching method.

[0106] Figure 5 is a block diagram of a key-value matching device shown in another exemplary embodiment of the present disclosure; see Figure 5 The key-value matching device may include:

[0107] A first acquisition module 401 is configured to acquire first position information of at least one attribute data and second position information of at least one attribute value data in an image to be detected;

[0108] a relationship graph generating module 402, configured to generate a first relationship graph according to the first position information of the at least one attribute data, and generate a second relationship graph according to the second position information of the at least one attribute value data;

[0109] Determination module 403 is used to input the first position information of the at least one attribute data, the second position information of the at least one attribute value data, the first relationship graph, the second relationship graph and the image to be detected into a preset graph matching model to obtain a matching relationship between the attribute data and the attribute value data.

[0110] The above technical solution, by inputting the first position information of at least one attribute data, the second position information of at least one attribute value data, the first relationship graph corresponding to the first position information of at least one attribute data, the second relationship graph corresponding to the second position information of at least one attribute value data, and the image to be detected into a preset graph matching model, so that the preset graph matching model outputs the matching relationship between the attribute data and the attribute value data. In this way, key-value matching is performed through the preset graph matching model, which not only can perform key-value matching for different scenarios and improve the generalization of the model, but also can effectively ensure the accuracy of the key-value matching results.

[0111] Optionally, the first relationship graph includes an attribute node corresponding to each attribute data location and first links between different attribute nodes, and the second relationship graph includes an attribute value node corresponding to each attribute value data location and second links between different attribute value nodes. The preset graph matching model is used to:

[0112] Obtaining, from the image to be detected, a first feature corresponding to each attribute node in the first relationship graph and a second feature corresponding to each first connection according to the first position information of the at least one attribute data, and obtaining, from the image to be detected, a third feature corresponding to each attribute value node in the second relationship graph and a fourth feature corresponding to each second connection according to the second position information of the at least one attribute value data;

[0113] The matching relationship between the attribute data and the attribute value data is determined according to the first feature corresponding to each attribute node and the second feature corresponding to each first connection, as well as the third feature corresponding to each attribute value node and the fourth feature corresponding to each second connection.

[0114] Optionally, the preset graph matching model includes a first sub-network and a second sub-network, and the preset graph matching model is used to:

[0115] Using the first position information of the at least one attribute data, the first relationship graph, and the image to be detected as inputs of a first subnetwork to output a first feature corresponding to each attribute node, and using the first position information of the at least one attribute data, the first relationship graph, and the image to be detected as inputs of a second subnetwork to output a second feature corresponding to each first link, wherein the first subnetwork has fewer hidden layers than the second subnetwork;

[0116] The second position information of the at least one attribute value data, the second relationship graph and the image to be detected are used as inputs of the first sub-network to output a third feature corresponding to each attribute value node, and the second position information of the at least one attribute value data, the second relationship graph and the image to be detected are used as inputs of the second sub-network to output a fourth feature corresponding to each second line.

[0117] Optionally, the preset graph matching model is used to:

[0118] Determine a node similarity matrix based on the first feature corresponding to each attribute node and the third feature corresponding to each attribute value node;

[0119] Determine a link similarity matrix according to the second feature corresponding to each first link and the fourth feature corresponding to each second link;

[0120] The matching relationship between each attribute data and each attribute value data is determined according to the node similarity matrix and the link similarity matrix.

[0121] Optionally, the relationship diagram generating module is used to:

[0122] generating the first relationship graph by using a Delaunay triangulation mapping method according to the first position information of the at least one attribute data;

[0123] The second relationship graph is generated by a fully connected graph construction method according to the second position information of the at least one attribute value data.

[0124] Figure 6 is based on Figure 5 The embodiment shown is a block diagram of a key-value matching device; see Figure 6 , the device may further include:

[0125] A second acquisition module 404 is configured to acquire a first quantity of the attribute data and a second quantity of the attribute value data;

[0126] The data adding module 405 is used to make the quantity of the attribute data equal to that of the attribute value data by adding the same attribute data at at least one location where the attribute data is located, or by adding the same attribute value data at at least one location where the attribute value data is located, when the first quantity is not equal to the second quantity.

[0127] Optionally, the data adding module is used to:

[0128] If the first quantity is less than the second quantity, add an identical attribute data at the location of each attribute data, obtain a first difference between twice the first quantity and the second quantity, obtain target attribute value data corresponding to the first difference from at least one of the attribute value data, and add an identical attribute value data at the location of each target attribute value data, so as to obtain twice the first quantity of attribute data and twice the first quantity of attribute value data;

[0129] Accordingly, the relationship diagram generation module is used to:

[0130] generating a first relationship graph according to the first position information corresponding to the twice first amount of attribute data;

[0131] A second relationship graph is generated according to the second position information corresponding to the twice first amount of attribute value data.

[0132] Optionally, the data adding module is used to:

[0133] If the first quantity is greater than the second quantity, add the same attribute value data at the location of each attribute value data, obtain a second difference between twice the second quantity and the first quantity, obtain target attribute data corresponding to the second difference from at least one of the attribute data, and add the same attribute data at the location of the target attribute data, so as to obtain twice the second quantity of attribute data and twice the second quantity of attribute value data;

[0134] Accordingly, the relationship diagram generation module is used to:

[0135] generating a first relationship graph according to the first position information corresponding to the twice second amount of attribute data;

[0136] A second relationship graph is generated according to the second position information corresponding to twice the second amount of attribute value data.

[0137] The above technical solution can effectively ensure that the quantity of the attribute data is equal to the quantity of the attribute value data, thereby ensuring that each attribute data can be matched to the corresponding attribute value data, which is conducive to expanding the range of processable images to be detected and expanding the scope of application of the key-value matching method.

[0138] Optionally, the preset graph matching model is trained by the following steps:

[0139] Obtain model training sample data, where the model training sample data includes a plurality of image samples to be detected, first position information of each attribute data in each of the image samples to be detected, second position information of each attribute value data in the image sample to be detected, and a first relationship graph and a second relationship graph corresponding to each of the image samples to be detected;

[0140] According to the first position information of each attribute data, a first feature corresponding to each attribute node and a second feature corresponding to each first connection in the first relationship graph of the image sample to be detected are obtained from the image sample to be detected through a preset network initial model; and according to the second position information of each attribute value data, a third feature corresponding to each attribute value node and a fourth feature corresponding to each second connection in the second relationship graph of the image sample to be detected are obtained from the image sample to be detected through the preset network initial model;

[0141] According to the first feature corresponding to each attribute node and the second feature corresponding to each first connection in the first relationship graph of the image sample to be detected, as well as the third feature corresponding to each attribute value node and the fourth feature corresponding to each second connection in the second relationship graph of the image sample to be detected, the loss value corresponding to the distance vector between each attribute node and the attribute value node to be matched is calculated by a preset loss function, and the preset initial network model is iteratively trained according to the loss value to obtain the preset graph matching model.

[0142] Optionally, the calculating, by using a preset loss function, a loss value corresponding to a distance vector between each attribute node and an attribute value node to be matched based on the first feature corresponding to each attribute node and the second feature corresponding to each first connection in the first relationship graph of the image sample to be detected, and the third feature corresponding to each attribute value node and the fourth feature corresponding to each second connection in the second relationship graph of the image sample to be detected, includes:

[0143] Determining a node similarity matrix based on the first feature corresponding to each attribute node and the third feature corresponding to each attribute value node, and determining a link similarity matrix based on the second feature corresponding to each first link and the fourth feature corresponding to each second link;

[0144] Generate a target relationship matrix according to the node similarity matrix and the link similarity matrix;

[0145] Obtaining a double random matrix corresponding to the target relationship matrix;

[0146] Determine the distance vector between each attribute node and the attribute value node to be matched according to the double random matrix;

[0147] The loss value is determined according to the distance vector by using a preset loss function.

[0148] The preset graph matching model obtained through the above training method has strong generalization and can be applied to a variety of different key-value matching scenarios. For example, it can be used for key-value matching of ID card images, as well as key-value matching in multiple scenarios such as business license images and degree certificate images.

[0149] Reference below Figure 7 , which shows a schematic structural diagram of an electronic device 600 suitable for implementing the embodiments of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0150] like Figure 7 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0151] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Figure 7 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0152] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0153] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0154] In some embodiments, the server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0155] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0156] The computer-readable medium carries one or more programs. When executed by the electronic device, the one or more programs cause the electronic device to: obtain first position information of at least one attribute data and second position information of at least one attribute value data in the image to be detected; generate a first relationship graph based on the first position information of the at least one attribute data, and generate a second relationship graph based on the second position information of the at least one attribute value data; and input the first position information of the at least one attribute data, the second position information of the at least one attribute value data, the first relationship graph, the second relationship graph, and the image to be detected into a preset graph matching model to obtain a matching relationship between the attribute data and the attribute value data.

[0157] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0158] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0159] The modules described in the embodiments of the present disclosure may be implemented in software or hardware. In some cases, the module name does not limit the module itself. For example, the first acquisition module may be described as "acquiring first position information of at least one attribute data and second position information of at least one attribute value data in the image to be detected."

[0160] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0161] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0162] According to one or more embodiments of the present disclosure, Example 1 provides a key-value matching method, the method including:

[0163] Acquire first position information of at least one attribute data and second position information of at least one attribute value data in the image to be detected;

[0164] generating a first relationship graph according to the first position information of the at least one attribute data, and generating a second relationship graph according to the second position information of the at least one attribute value data;

[0165] The first position information of the at least one attribute data, the second position information of the at least one attribute value data, the first relationship graph, the second relationship graph and the image to be detected are input into a preset graph matching model to obtain a matching relationship between the attribute data and the attribute value data.

[0166] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, wherein the first relationship graph includes an attribute node corresponding to the location of each attribute data and first links between different attribute nodes; the second relationship graph includes an attribute value node corresponding to the location of each attribute value data and second links between different attribute value nodes; and the preset graph matching model is used to:

[0167] Acquire, from the image to be detected, a first feature corresponding to each attribute node in the first relationship graph and a second feature corresponding to each first connection according to the first position information of the at least one attribute data, and acquire, from the image to be detected, a third feature corresponding to each attribute value node in the second relationship graph and a fourth feature corresponding to each second connection according to the second position information of the at least one attribute value data;

[0168] The matching relationship between the attribute data and the attribute value data is determined according to the first feature corresponding to each attribute node and the second feature corresponding to each first link, as well as the third feature corresponding to each attribute value node and the fourth feature corresponding to each second link.

[0169] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 2, wherein the preset graph matching model includes a first subnetwork and a second subnetwork, and obtaining, from the image to be detected, the first feature corresponding to each attribute node in the first relationship graph and the second feature corresponding to each first connection based on the first position information of the at least one attribute data, and obtaining, from the image to be detected, the third feature corresponding to each attribute value node in the second relationship graph and the fourth feature corresponding to each second connection based on the second position information of the at least one attribute value data, including:

[0170] Using the first position information of the at least one attribute data, the first relationship graph, and the image to be detected as inputs of a first sub-network to output a first feature corresponding to each attribute node, and using the first position information of the at least one attribute data, the first relationship graph, and the image to be detected as inputs of a second sub-network to output a second feature corresponding to each first link, wherein the first sub-network has fewer hidden layers than the second sub-network;

[0171] The second position information of the at least one attribute value data, the second relationship graph and the image to be detected are used as inputs of the first sub-network to output a third feature corresponding to each attribute value node, and the second position information of the at least one attribute value data, the second relationship graph and the image to be detected are used as inputs of the second sub-network to output a fourth feature corresponding to each second line.

[0172] According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 2, wherein determining the matching relationship between the attribute data and the attribute value data based on the first feature corresponding to each attribute node and the second feature corresponding to each first link, and the third feature corresponding to each attribute value node and the fourth feature corresponding to each second link includes:

[0173] Determine a node similarity matrix according to the first feature corresponding to each attribute node and the third feature corresponding to each attribute value node;

[0174] Determine a link similarity matrix according to the second feature corresponding to each first link and the fourth feature corresponding to each second link;

[0175] The matching relationship between each attribute data and each attribute value data is determined according to the node similarity matrix and the link similarity matrix.

[0176] According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 1, wherein generating a first relationship graph based on the first position information of the at least one attribute data and generating a second relationship graph based on the second position information of the at least one attribute value data includes:

[0177] generating the first relationship graph by using a Delaunay triangulation mapping method according to the first position information of the at least one attribute data;

[0178] The second relationship graph is generated by a fully connected graph construction method according to the second position information of the at least one attribute value data.

[0179] According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 1, wherein before generating the first relationship graph based on the first position information of the at least one attribute data and generating the second relationship graph based on the second position information of the at least one attribute value data, the method further includes:

[0180] Obtaining a first quantity of the attribute data and a second quantity of the attribute value data;

[0181] When the first quantity is not equal to the second quantity, the quantity of the attribute data is made equal to the quantity of the attribute value data by adding the same attribute data at the location of at least one of the attribute data, or by adding the same attribute value data at the location of at least one of the attribute value data.

[0182] According to one or more embodiments of the present disclosure, Example 7 provides the method of Example 6, wherein, when the first quantity and the second quantity are not equal, adding the same attribute data at at least one location where the attribute data is located includes:

[0183] When the first quantity is less than the second quantity, adding an identical attribute data at the location of each attribute data, obtaining a first difference between twice the first quantity and the second quantity, obtaining a number of target attribute value data corresponding to the first difference from at least one of the attribute value data, and adding an identical attribute value data at the location of each of the target attribute value data, so as to obtain twice the first quantity of attribute data and twice the first quantity of attribute value data;

[0184] Accordingly, generating a first relationship graph according to the first position information of the at least one attribute data, and generating a second relationship graph according to the second position information of the at least one attribute value data, includes:

[0185] generating a first relationship graph according to first position information corresponding to twice the first amount of attribute data;

[0186] A second relationship graph is generated according to the second position information corresponding to the twice first amount of attribute value data.

[0187] According to one or more embodiments of the present disclosure, Example 8 provides the method of Example 6, wherein the adding the same attribute value data to at least one location where the attribute value data is located so that the number of the attribute data is equal to that of the attribute value data further includes:

[0188] If the first quantity is greater than the second quantity, add the same attribute value data at the location of each attribute value data, obtain a second difference between twice the second quantity and the first quantity, obtain target attribute data corresponding to the second difference from at least one of the attribute data, and add the same attribute data at the location of the target attribute data, so as to obtain twice the second quantity of attribute data and twice the second quantity of attribute value data;

[0189] Accordingly, the preset graph matching model generates a first relationship graph according to the first position information of the at least one attribute data, and generates a second relationship graph according to the second position information of the at least one attribute value data, including:

[0190] generating a first relationship graph according to the first position information corresponding to the twice second amount of attribute data;

[0191] A second relationship graph is generated according to the second position information corresponding to twice the second amount of attribute value data.

[0192] According to one or more embodiments of the present disclosure, Example 9 provides the method of any one of Examples 1-8, wherein the preset graph matching model is trained by the following steps:

[0193] Acquire model training sample data, the model training sample data including a plurality of image samples to be detected, first position information of each attribute data in each of the image samples to be detected, second position information of each attribute value data in the image sample to be detected, and a first relationship graph and a second relationship graph corresponding to each of the image samples to be detected;

[0194] According to the first position information of each attribute data, a first feature corresponding to each attribute node and a second feature corresponding to each first connection in the first relationship graph of the image sample to be detected are obtained from the image sample to be detected through a preset network initial model; and according to the second position information of each attribute value data, a third feature corresponding to each attribute value node and a fourth feature corresponding to each second connection in the second relationship graph of the image sample to be detected are obtained from the image sample to be detected through the preset network initial model;

[0195] According to the first feature corresponding to each attribute node and the second feature corresponding to each first connection in the first relationship graph of the image sample to be detected, as well as the third feature corresponding to each attribute value node and the fourth feature corresponding to each second connection in the second relationship graph of the image sample to be detected, the loss value corresponding to the distance vector between each attribute node and the attribute value node to be matched is calculated by a preset loss function, and the preset initial network model is iteratively trained according to the loss value to obtain the preset graph matching model.

[0196] According to one or more embodiments of the present disclosure, Example 10 provides the method shown in Example 9, wherein, based on the first feature corresponding to each attribute node and the second feature corresponding to each first connection in the first relationship graph of the image sample to be detected, and the third feature corresponding to each attribute value node and the fourth feature corresponding to each second connection in the second relationship graph of the image sample to be detected, a loss value corresponding to the distance vector between each attribute node and the attribute value node to be matched is calculated using a preset loss function, including:

[0197] Determining a node similarity matrix based on the first feature corresponding to each attribute node and the third feature corresponding to each attribute value node, and determining a link similarity matrix based on the second feature corresponding to each first link and the fourth feature corresponding to each second link;

[0198] Generate a target relationship matrix according to the node similarity matrix and the link similarity matrix;

[0199] Obtaining a double random matrix corresponding to the target relationship matrix;

[0200] Determine the distance vector between each attribute node and the attribute value node to be matched according to the double random matrix;

[0201] The loss value is determined according to the distance vector by using a preset loss function.

[0202] According to one or more embodiments of the present disclosure, Example 11 provides a key-value matching device, the device comprising:

[0203] A first acquisition module is used to acquire first position information of at least one attribute data and second position information of at least one attribute value data in the image to be detected;

[0204] a relationship graph generating module, configured to generate a first relationship graph according to the first position information of the at least one attribute data, and generate a second relationship graph according to the second position information of the at least one attribute value data;

[0205] A determination module is used to input the first position information of the at least one attribute data, the second position information of the at least one attribute value data, the first relationship graph, the second relationship graph and the image to be detected into a preset graph matching model to obtain a matching relationship between the attribute data and the attribute value data.

[0206] According to one or more embodiments of the present disclosure, Example 12 provides a computer-readable medium having a computer program stored thereon, which implements the steps of the method described in any one of Examples 1-10 when executed by a processing device.

[0207] According to one or more embodiments of the present disclosure, Example 13 provides an electronic device, including:

[0208] a storage device having a computer program stored thereon;

[0209] A processing device is used to execute the computer program in the storage device to implement the steps of the method described in any one of Examples 1-10.

[0210] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0211] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0212] Although the subject matter has been described using language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims. Regarding the apparatus in the above-described embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method and will not be elaborated upon here.

Claims

1. A key value matching method, characterized in that: The method comprises: Acquire first position information of at least one attribute data and second position information of at least one attribute value data in the image to be detected; generating a first relationship graph according to first position information of the at least one attribute data, and generating a second relationship graph according to second position information of the at least one attribute value data; Inputting the first position information of the at least one attribute data, the second position information of the at least one attribute value data, the first relationship graph, the second relationship graph and the image to be detected into a preset graph matching model to obtain a matching relationship between the attribute data and the attribute value data; The first relationship graph includes an attribute node corresponding to the location of each attribute data and a first connection line between different attribute nodes, the second relationship graph includes an attribute value node corresponding to the location of each attribute value data and a second connection line between different attribute value nodes, and the preset graph matching model is used to: Acquire, from the image to be detected, a first feature corresponding to each attribute node in the first relationship graph and a second feature corresponding to each first link according to the first position information of the at least one attribute data, and acquire, from the image to be detected, a third feature corresponding to each attribute value node in the second relationship graph and a fourth feature corresponding to each second link according to the second position information of the at least one attribute value data; The matching relationship between the attribute data and the attribute value data is determined according to the first feature corresponding to each attribute node and the second feature corresponding to each first link, as well as the third feature corresponding to each attribute value node and the fourth feature corresponding to each second link.

2. The method according to claim 1, characterized in that The preset graph matching model includes a first subnetwork and a second subnetwork, and the first feature corresponding to each attribute node in the first relationship graph and the second feature corresponding to each first connection line are obtained from the image to be detected according to the first position information of the at least one attribute data, and the third feature corresponding to each attribute value node in the second relationship graph and the fourth feature corresponding to each second connection line are obtained from the image to be detected according to the second position information of the at least one attribute value data, including: Using the first position information of the at least one attribute data, the first relationship graph and the image to be detected as inputs of a first sub-network to output a first feature corresponding to each attribute node, and using the first position information of the at least one attribute data, the first relationship graph and the image to be detected as inputs of a second sub-network to output a second feature corresponding to each first connection line, wherein the number of hidden layers of the first sub-network is less than that of the second sub-network; The second position information of the at least one attribute value data, the second relationship graph and the image to be detected are used as inputs of the first sub-network to output a third feature corresponding to each attribute value node, and the second position information of the at least one attribute value data, the second relationship graph and the image to be detected are used as inputs of the second sub-network to output a fourth feature corresponding to each second line.

3. The method according to claim 1, characterized in that The determining the matching relationship between the attribute data and the attribute value data according to the first feature corresponding to each of the attribute nodes and the second feature corresponding to each of the first links, and the third feature corresponding to each of the attribute value nodes and the fourth feature corresponding to each of the second links, includes: Determine a node similarity matrix according to the first feature corresponding to each of the attribute nodes and the third feature corresponding to each of the attribute value nodes; Determine a link similarity matrix according to the second feature corresponding to each of the first links and the fourth feature corresponding to each of the second links; The matching relationship between each of the attribute data and each of the attribute value data is determined according to the node similarity matrix and the link similarity matrix.

4. The method according to claim 1, characterized in that: The step of generating a first relationship graph according to the first position information of the at least one attribute data, and generating a second relationship graph according to the second position information of the at least one attribute value data, comprises: Generate the first relationship graph by using Delaunay triangulation mapping method according to the first position information of the at least one attribute data; The second relationship graph is generated by a fully connected graph construction method according to the second position information of the at least one attribute value data.

5. The method according to claim 1, characterized in that Before generating the first relationship graph according to the first position information of the at least one attribute data and generating the second relationship graph according to the second position information of the at least one attribute value data, the method further includes: Acquire a first quantity of the attribute data and a second quantity of the attribute value data; When the first quantity is not equal to the second quantity, the quantity of the attribute data is made equal to the quantity of the attribute value data by adding the same attribute data at the location of at least one of the attribute data, or by adding the same attribute value data at the location of at least one of the attribute value data.

6. The method according to claim 5, characterized in that When the first quantity is not equal to the second quantity, adding the same attribute data at the location where at least one of the attribute data is located comprises: In the case where the first quantity is less than the second quantity, adding an identical attribute data to the location of each attribute data, obtaining a first difference between twice the first quantity and the second quantity, obtaining a target attribute value data corresponding to the first difference from at least one of the attribute value data, and adding an identical attribute value data to the location of each of the target attribute value data, so as to obtain twice the first quantity of attribute data and twice the first quantity of attribute value data; Accordingly, generating a first relationship graph according to the first position information of the at least one attribute data, and generating a second relationship graph according to the second position information of the at least one attribute value data, comprises: Generate a first relationship graph according to the first position information corresponding to the twice first amount of attribute data; A second relationship graph is generated according to second position information corresponding to twice the first amount of attribute value data.

7. The method according to claim 5, characterized in that The adding the same attribute value data at the location where at least one of the attribute value data is located so that the quantity of the attribute data is equal to that of the attribute value data further includes: In the case where the first quantity is greater than the second quantity, an identical attribute value data is added at the location of each attribute value data, a second difference between twice the second quantity and the first quantity is obtained, and a target attribute data corresponding to the second difference is obtained from at least one of the attribute data, and an identical attribute data is added at the location of the target attribute data, so as to obtain twice the second quantity of attribute data and twice the second quantity of attribute value data; Accordingly, the preset graph matching model generates a first relationship graph according to the first position information of the at least one attribute data, and generates a second relationship graph according to the second position information of the at least one attribute value data, including: generating a first relationship graph according to the first position information corresponding to the twice second amount of attribute data; A second relationship graph is generated according to the second position information corresponding to the twice second amount of attribute value data.

8. The method according to any one of claims 1 to 7, characterized in that: The preset graph matching model is trained by the following steps: Acquire model training sample data, wherein the model training sample data includes a plurality of image samples to be detected, first position information of each attribute data in each of the image samples to be detected, second position information of each attribute value data in the image samples to be detected, and a first relationship graph and a second relationship graph corresponding to each of the image samples to be detected; According to the first position information of each attribute data, a first feature corresponding to each attribute node in the first relationship graph of the image sample to be detected and a second feature corresponding to each first connection are obtained from the image sample to be detected through a preset network initial model, and according to the second position information of each attribute value data, a third feature corresponding to each attribute value node in the second relationship graph of the image sample to be detected and a fourth feature corresponding to each second connection are obtained from the image sample to be detected through the preset network initial model; According to the first feature corresponding to each attribute node and the second feature corresponding to each first connection in the first relationship graph of the image sample to be detected, as well as the third feature corresponding to each attribute value node and the fourth feature corresponding to each second connection in the second relationship graph of the image sample to be detected, the loss value corresponding to the distance vector between each attribute node and the attribute value node to be matched is calculated by a preset loss function, and the preset network initial model is iteratively trained according to the loss value to obtain the preset graph matching model.

9. The method according to claim 8, characterized in that The method calculates the loss value corresponding to the distance vector between each attribute node and the attribute value node to be matched by a preset loss function according to the first feature corresponding to each attribute node and the second feature corresponding to each first connection in the first relationship graph of the image sample to be detected, and the third feature corresponding to each attribute value node and the fourth feature corresponding to each second connection in the second relationship graph of the image sample to be detected, including: Determine a node similarity matrix according to the first feature corresponding to each of the attribute nodes and the third feature corresponding to each of the attribute value nodes, and determine a link similarity matrix according to the second feature corresponding to each of the first links and the fourth feature corresponding to each of the second links; Generate a target relationship matrix according to the node similarity matrix and the line similarity matrix; Obtaining a double random matrix corresponding to the target relationship matrix; Determine the distance vector between each attribute node and the attribute value node to be matched according to the double random matrix; The loss value is determined according to the distance vector by using a preset loss function.

10. A key value matching device, characterized in that: The device comprises: A first acquisition module, used for acquiring first position information of at least one attribute data and second position information of at least one attribute value data in the image to be detected; A relationship graph generating module, configured to generate a first relationship graph according to the first position information of the at least one attribute data, and to generate a second relationship graph according to the second position information of the at least one attribute value data; A determination module, configured to input the first position information of the at least one attribute data, the second position information of the at least one attribute value data, the first relationship graph, the second relationship graph, and the image to be detected into a preset graph matching model to obtain a matching relationship between the attribute data and the attribute value data; The first relationship graph includes an attribute node corresponding to the location of each attribute data and a first connection line between different attribute nodes, the second relationship graph includes an attribute value node corresponding to the location of each attribute value data and a second connection line between different attribute value nodes, and the preset graph matching model is used to: Acquire, from the image to be detected, a first feature corresponding to each attribute node in the first relationship graph and a second feature corresponding to each first link according to the first position information of the at least one attribute data, and acquire, from the image to be detected, a third feature corresponding to each attribute value node in the second relationship graph and a fourth feature corresponding to each second link according to the second position information of the at least one attribute value data; The matching relationship between the attribute data and the attribute value data is determined according to the first feature corresponding to each attribute node and the second feature corresponding to each first link, as well as the third feature corresponding to each attribute value node and the fourth feature corresponding to each second link.

11. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processing device, the steps of the method described in any one of claims 1 to 9 are implemented.

12. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 9.

Images