An image set construction method for multiple image insertion

Abstract

The application discloses a kind of image set construction methods for multiple image insertion, comprising: obtaining the tree diagram of existing image set and image to be inserted;Establish the first authorized directed graph of image to be inserted;Generate the first minimum tree diagram without any constraint;Determine the level of each image to be inserted and existing image;Establish the authorized directed graph for all images and connect the new directed edge meeting node requirement and calculate its weight to obtain the second authorized directed graph, change the level of each image corresponding node in the second authorized directed graph to not more than the target depth of corresponding tree diagram of image set after insertion;According to target depth, the second authorized directed graph and the level of each image to be inserted and existing image, generate the second minimum tree diagram of depth and node layer constraint;Determine the reference image of each image to be inserted according to the second minimum tree diagram and encode the image to be inserted to obtain the image set after insertion.The application can reduce the storage space of image set after insertion and improve the coding efficiency.

Description

Technical Field

[0001] This invention belongs to the field of cloud computing and big data, and specifically relates to a method for constructing an image set for inserting multiple images. Background Technology

[0002] In the past decade or so, with the rapid development of industries such as the internet, big data, and cloud computing, digital images and other multimedia content have experienced explosive growth. According to a report by Facebook, the world's largest social networking service, it already stores over 200 billion photos, and this number is increasing by more than 300 million per day. This massive amount of image, video, and other multimedia data has placed immense pressure on cloud image storage, the storage management of other internet image databases, and the processing of image big data, including mining, classification, and recognition.

[0003] Therefore, researchers have been developing new digital image compression technologies. Taking the widely used cloud storage as an example, the large number of images stored on cloud servers contain many photos taken in the same scenes, exhibiting significant similarity. However, compressing these images using common JPEG, JPEG-2000, H.264 intra-frame coding, or HEVC intra-frame coding techniques results in large storage space requirements and insufficient coding efficiency. Therefore, it is necessary to utilize the correlation between similar images and analyze them using visual content analysis techniques to establish corresponding visual models or visual connections. This helps eliminate visual redundancy between similar images and improves coding performance. The current optimal approach is to place many similar images in a tree diagram and then apply video coding techniques to each branch, effectively eliminating visual redundancy between related images and generating corresponding image sets to improve coding efficiency. However, when adding multiple newly captured similar images to an existing image set in a cloud server, it is necessary to break the original tree diagram structure, generate a new tree diagram, and construct the image set after image insertion. Therefore, how to create a new tree diagram and construct the inserted image set to reduce storage space and improve coding efficiency is a problem that urgently needs to be solved by researchers. Currently, there are roughly four methods for constructing an image set after inserting multiple images:

[0004] The first method is the image-by-image insertion method, such as the one mentioned in the paper "Personal photo album compression and management" published by Ruobing Zou, Oscar C. Au, Guyue Zhou, Wei Dai, Wei Hu and Pengfei Wan at the 2013 IEEE International Symposium on Circuits Systems. This method decodes all images in the existing image set, finds the decoded image with the lowest prediction cost for encoding the first image to be inserted, and uses this decoded image as the parent node of the image to be inserted. This parent node is then used as a reference image for encoding the image to be inserted. A similar method is then used to find and encode parent nodes for each image to be inserted, until all images to be inserted have been encoded. While this method finds the best reference image for the image to be inserted in the existing image set, it ignores the correlation between multiple images to be inserted and does not find the best reference image among other images to be inserted, thus reducing encoding efficiency.

[0005] The second method is a root-node-based image insertion method, such as the one mentioned in the paper "Lossless Compression of JPEG Coded Photo Collections" published by Hao Wu, Xiaoyan Sun, Jingyu Yang, Wenjun Zeng, and Feng Wu in Transactions on Image Processing in June 2016. This method decodes the root node in the minimal tree diagram of the existing image set and uses this decoded image as the parent node of all images to be inserted. Using this parent node as a reference image, all images to be inserted are encoded to obtain the image set after insertion. Although this method has the lowest complexity among existing methods, the image to be inserted may not be highly similar to the image corresponding to the root node, but is more related to other images in the existing image set. This means that the image corresponding to the root node is not the optimal reference image for the image to be inserted, resulting in a lower encoding efficiency than the first method.

[0006] The third method is an image insertion method based on re-encoding all images. The paper "Lossless Compression of JPEG Coded Photo Collections" published in Transactions on Image Processing in June 2016 by Hao Wu, Xiaoyan Sun, Jingyu Yang, Wenjun Zeng, and Feng Wu describes this method: First, all images in the existing image set are decoded, discarding the original tree graph of the existing image set. All decoded images and multiple images to be inserted are placed into a node set. The distance between any two corresponding images of any two nodes is calculated and used as the weight of the directed edge connecting the two nodes, forming a set of directed edges. Then, a new tree graph is generated from the weighted directed graph formed by the node set and the directed edge set. All images are then encoded based on the obtained tree graph. This method requires re-encoding all images, including all images in the existing image set and multiple images to be inserted. Although it greatly improves the encoding effect, its computational complexity is extremely high, making it unsuitable for managing cloud image sets.

[0007] The fourth method is the image insertion method based on depth-constrained minimum tree graph generation. For example, the method mentioned in Xu Dongmei's master's thesis "Image Insertion and Image Set Merging Algorithm in Group Images" published in June 2018: Decode the first-level nodes of the existing image set, generate a depth-constrained minimum tree graph with the image to be inserted, then increment the level by 1, decode the images corresponding to the child nodes of the parent nodes of the image to be inserted in the previous level, regenerate the depth-constrained minimum tree graph, and repeat this process until no node in a certain level serves as the parent node of the newly inserted image or the level reaches the maximum depth constraint. This method considers the correlation between multiple images to be inserted, improving coding efficiency compared to the first two methods. However, the minimum tree graph generated by this method using a cyclic approach has a high sum of edge weights, meaning the reference image found for the image to be inserted may not be the optimal reference image, resulting in the coding efficiency of the image set after insertion not reaching the ideal level. Summary of the Invention

[0008] The purpose of this invention is to provide a method for constructing an image set for inserting multiple images, thereby improving coding efficiency. The specific technical solution is as follows:

[0009] A method for constructing an image set for multiple image insertions includes:

[0010] Obtain the tree diagram of the existing image set and the image to be inserted;

[0011] Establish a first weighted directed graph for the image to be inserted;

[0012] Based on the first weighted and directed graph, generate a first minimal tree graph that is not subject to any constraints for the image to be inserted;

[0013] Based on the first minimum tree diagram, determine the level corresponding to each image to be inserted;

[0014] Based on the tree diagram of the existing image set, determine the level corresponding to each existing image in the existing image set;

[0015] Based on the tree diagram of the existing image set and the first weighted directed graph, a weighted directed graph for all images is established; and based on the weighted directed graph for all images, new directed edges that meet the node requirements are connected and their weights are calculated to obtain a second weighted directed graph;

[0016] The hierarchy of each node corresponding to each image in the second weighted and directed graph is modified to not exceed the target depth; wherein, the target depth is the pre-set depth of the tree graph corresponding to the inserted image set;

[0017] Based on the target depth, the second weighted and directed graph, and the levels corresponding to each image to be inserted and the existing images, a second minimum tree graph with depth and node layer constraints is generated.

[0018] Based on the second minimum tree diagram, a reference image is determined for each image to be inserted, and the corresponding image to be inserted is encoded using each reference image. The encoded results of all images to be inserted constitute the inserted image set.

[0019] In one embodiment of the present invention, establishing a first weighted directed graph for the image to be inserted includes:

[0020] Each image to be inserted is abstracted into a node;

[0021] Starting from k, assign a node number to the node corresponding to each image to be inserted, and form the first node set by all the nodes corresponding to the images to be inserted that carry the node numbers; where k is a preset natural number;

[0022] In the first set of nodes, a directed edge is connected from any node to any other node, and the weight of the directed edge between the two connected nodes is calculated based on the relevance of the image.

[0023] All directed edges and their corresponding weights obtained from the first set of nodes constitute the first set of directed edges, and the first set of nodes and the first set of directed edges constitute the first weighted directed graph for the image to be inserted.

[0024] In one embodiment of the present invention, generating a first minimal tree graph unconstrained by any rules for the image to be inserted based on the first weighted directed graph includes:

[0025] Using the first weighted directed graph as input to the Zhu Liu method, a first minimal tree graph, unconstrained for the image to be inserted, is obtained.

[0026] In one embodiment of the present invention, determining the level corresponding to each existing image in the existing image set based on the tree diagram of the existing image set, or determining the level corresponding to each image to be inserted based on the first minimum tree diagram, includes:

[0027] For the tree diagram to be processed, based on the level of the current parent node, add h to the level of all child nodes of the current parent node, and update each child node of the current parent node to the new current parent node. Repeat the process of adding h to the level of all child nodes of the current parent node based on the level of the current parent node until the node with the current determined level has no child nodes.

[0028] The tree diagram to be processed includes the tree diagram of the existing image set or the first minimum tree diagram; for the tree diagram to be processed, the current parent node is the root node; when the tree diagram to be processed is the tree diagram of the existing image set, the level of the root node is set to a first value; when the tree diagram to be processed is the first minimum tree diagram, the level of the root node is set to a second value; wherein the second value is greater than the first value, and the absolute value of the difference between the two is h; h is a preset natural number greater than 0.

[0029] In one embodiment of the present invention, the step of establishing a weighted directed graph for all images based on the tree graph of the existing image set and the first weighted directed graph includes:

[0030] Continuing from the node labels in the first node set, assign non-repeating and consecutive node labels to each node in the tree diagram of the existing image set; and add all nodes carrying node labels in the tree diagram of the existing image set to the first node set to obtain the second node set;

[0031] Decode all existing images in the existing image set to obtain the corresponding decoded images;

[0032] Connect the directed edges from any node corresponding to the decoded image to any node corresponding to the image to be inserted, and calculate the weight of the directed edges between the two connected nodes based on the relevance of the images.

[0033] All the obtained directed edges and their corresponding weights are added to the first set of directed edges to obtain an updated set of directed edges. The weighted directed graph for all images is obtained by the second set of nodes and the updated set of directed edges.

[0034] In one embodiment of the present invention, the step of connecting new directed edges that satisfy the node requirements and calculating weights on the weighted directed graph for all images to obtain a second weighted directed graph includes:

[0035] Based on the weighted and directed graph for all images, new directed edges that meet the node requirements are connected, and the weight of the directed edge between the two connected nodes is calculated according to the relevance of the images; wherein, the node requirements are that the two nodes of each new directed edge are nodes corresponding to the images to be inserted, and the level of the starting node of the new directed edge is less than the level of the ending node.

[0036] All the new directed edges and their corresponding weights are added to the updated set of directed edges to obtain a second set of directed edges. The second weighted directed graph is obtained from the second set of nodes and the second set of directed edges.

[0037] In one embodiment of the present invention, generating a second minimum tree graph with depth and node layer constraints based on the target depth, the second weighted directed graph, and the levels corresponding to each image to be inserted and existing images respectively includes:

[0038] a1, for the current iteration, obtain the corresponding input parameters, including the node set V. B and V U and the set of directed edges E B and E G Among the input parameters of the first iteration, V B It contains only the root node v of the existing image set. R V U E B All are empty, E G The second weighted directed graph contains v R All directed edges originating from the first element are labeled as belonging to the first class.

[0039] a2, except V B The highest-level node, excluding existing nodes, joins V. U In the middle, from the current V U Among all directed edges of type I with the middle node as the endpoint, select the directed edge with the smallest weight. <v p ,v q >;

[0040] a3, determine the current V BAre all nodes in the hierarchy less than or equal to node v? q The level; if so, then v q Add to current V B and directed edges <v p ,v q Marked as Category 2 and added to E B In, and from the current V U Delete v q If not, then there will be a directed edge. <v p ,v q Mark it as category 3 and perform step a2;

[0041] a4, select a line starting with v q Unlabeled directed edge starting from <v q ,v o > If node v o All three of the following conditions must be met: v o Not belonging to the current V B And v o The level is equal to v q The hierarchy plus h, and v q to v R If the number of directed edges on the path is less than L, then... <v q ,v o Mark it as the first category and add it to the current E G In the middle, execute step a5; if node v o If any of the above conditions are not met, there will be a directed edge. <v q ,v o > Mark it as the third category and execute step a6; where L is the quotient of the difference between the target depth and h;

[0042] a5, in v o Choose the directed edge with the smallest weight from all first-class directed edges that terminate at the destination. <v s ,v o Mark all remaining first-class directed edges as third-class and remove them from the current E G Delete;

[0043] a6, determine with respect to v q Check if all directed edges originating from this point have been marked; if so, proceed to step a7; otherwise, proceed to step a4.

[0044] a7, determine if the iteration number corresponding to the current iteration is equal to the total number of nodes in the second node set minus two; if not, change the final V obtained in the current iteration. B V U E B and E GAs the input parameter for the next iteration, the iteration number corresponding to the current iteration is incremented by 1 to become the iteration number for the next iteration, and step a1 is executed; if it equals, the V obtained from the current iteration is... B and E B The second minimum tree diagram T2 = (V) is obtained. B E B ).

[0045] In one embodiment of the present invention, the step of correcting the level of each image-corresponding node in the second weighted directed graph to not exceed the target depth includes:

[0046] For each image in the second weighted directed graph, determine whether the level of the node corresponding to the image is greater than the target depth;

[0047] If so, correct the layer of the corresponding node in the image to the target depth;

[0048] If not, keep the hierarchy of the corresponding node in the image unchanged.

[0049] In one embodiment of the present invention, the calculation formula used to calculate the weight of the directed edge between two connected nodes based on the correlation of the image includes:

[0050]

[0051] Where, d n represents the weight between any pair of SIFT feature matching points between the images corresponding to the two nodes, and N is the number of SIFT feature matching point pairs between the images corresponding to any two nodes.

[0052] In one embodiment of the present invention, the step of determining a reference image for each image to be inserted based on the second minimum tree diagram, and encoding the corresponding image to be inserted using each reference image, wherein the encoded results of all images to be inserted constitute the inserted image set, includes:

[0053] For each image to be inserted, the image corresponding to the parent node of the image to be inserted in the second minimum tree diagram is used as the reference image of the image to be inserted.

[0054] Calculate the perspective transformation matrix between the image to be inserted and the corresponding reference image;

[0055] Based on the perspective transformation matrix, a geometric transformation is performed on the reference image corresponding to the image to be inserted to obtain the geometrically transformed reference image corresponding to the image to be inserted.

[0056] The geometrically transformed reference image is optically transformed to obtain the geometrically and optically transformed reference image corresponding to the image to be inserted.

[0057] The reference image after the geometric optical transformation is used to perform inter-frame coding on the image to be inserted.

[0058] The beneficial effects of this invention are:

[0059] In the solution provided by this embodiment of the invention, firstly, a first weighted directed graph of the image to be inserted is established, and a first minimal tree graph without any constraints is generated for the image to be inserted. Secondly, based on the first minimal tree graph, the level corresponding to each image to be inserted is determined, and also based on the tree graph of the existing image set, the level corresponding to each existing image in the existing image set is determined. Next, based on the tree graph of the existing image set and the first weighted directed graph, a weighted directed graph for all images is established, and new directed edges that meet the node requirements are connected on it and weights are calculated to obtain a second weighted directed graph. Then, the level of the node corresponding to each image in the second weighted directed graph is corrected to not exceed the target depth of the tree graph corresponding to the inserted image set. Then, based on the target depth, the second weighted directed graph, and the levels corresponding to each image to be inserted and the existing images respectively, a second minimal tree graph with depth and node layer constraints is generated. Finally, based on the second minimal tree graph, a reference image for each image to be inserted is determined, and the corresponding image to be inserted is encoded using each reference image to obtain the inserted image set. For a weighted directed graph, the unconstrained minimal tree graph generated by the Zhu-Liu method has the lowest total edge weight compared to constrained minimal tree graphs generated by other methods. Therefore, the image set constructed based on the unconstrained minimal tree graph also has the highest encoding efficiency. The method of this invention uses the unconstrained first minimal tree graph as guidance and generates a second minimal tree graph using a depth and node-level constraint-based minimal tree graph generation method. Therefore, it can determine the optimal reference image for each image to be inserted under target depth constraints without changing the existing image set architecture. Compared to existing schemes that do not utilize the unconstrained first minimal tree graph as guidance, the method of this invention can reduce the storage space required for the inserted image set, further improving encoding efficiency. The method of this invention can be used to manage image sets stored in the cloud and to add multiple images to an existing image set in image database management scenarios. Attached Figure Description

[0060] Figure 1 This is a flowchart illustrating a method for constructing an image set for inserting multiple images, provided in an embodiment of the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] To improve encoding efficiency, this invention provides a method for constructing an image set for inserting multiple images.

[0063] It should be noted that the execution entity of the image set construction method for multiple image insertion provided in this embodiment of the invention can be an image set construction device for multiple image insertion, and the device can run in an electronic device. The electronic device can be a server or a terminal device, but is not limited to these.

[0064] like Figure 1 As shown in the embodiment of the present invention, an image set construction method for inserting multiple images is provided. This method can be used to manage image sets stored in the cloud and to add multiple images to an existing image set in an image database management scenario. The method may include the following steps:

[0065] S1, obtain the tree diagram of the existing image set and the image to be inserted.

[0066] In this embodiment of the invention, the existing image set contains multiple existing images, and the existing image set is stored in the form of a tree diagram, specifically in a storage device such as a cloud server. Each existing image can be abstracted as a node in this tree diagram.

[0067] In this embodiment of the invention, at least one image to be inserted can be obtained through image acquisition and data transmission, and the specific process is not limited here.

[0068] S2, establish a first weighted directed graph for the image to be inserted.

[0069] In one optional implementation, this step may include the following steps:

[0070] S21, abstract each image to be inserted as a node.

[0071] S22, starting from k, assign node labels to the nodes corresponding to each image to be inserted, and form the first node set by all the nodes corresponding to the images to be inserted that carry node labels; where k is a preset natural number, such as 0, 1, etc., and the specific value can be set as needed.

[0072] S23, in the first set of nodes, connect directed edges from any node to any other node, and calculate the weight of the directed edges between the two connected nodes based on the correlation of the image.

[0073] S24, all directed edges and their corresponding weights obtained from the first node set constitute a first directed edge set, and a first weighted directed graph for the image to be inserted is obtained from the first node set and the first directed edge set.

[0074] For ease of description, the first set of nodes is denoted by V1, the weight of the directed edge between two connected nodes is denoted by d, the first set of directed edges is denoted by E1, and the first weighted directed graph is denoted by G1(V1, E1).

[0075] In practice, the algorithm execution process for this part can be as follows: First, set the first node set V1 and the first directed edge set E1 as empty sets; second, abstract each image to be inserted as a node, and assign a node label to each node starting from, for example, 0, requiring that the node labels of all nodes corresponding to the images to be inserted are connected sequentially and without repetition; next, add all the nodes corresponding to the images to be inserted after adding node labels to the original empty first node set V1, so that the first node set V1 now contains multiple nodes; then, connect the directed edges from any node in V1 to any other node, and according to the relevant information of the image... The weights of directed edges between each pair of connected nodes are calculated. Understandably, the higher the relevance of the images corresponding to two nodes, the smaller the weight of the directed edge between them. For nodes 1 and 2, the weights of directed edges starting from node 1 and ending at node 2 are not the same as those starting from node 2 and ending at node 1. Finally, all the resulting directed edges and their corresponding weights are added to the initially empty first set of directed edges, E1. From the final V1 and E1, a weighted directed graph G1(V1, E1) of the image to be inserted is obtained. For ease of description later, this is named the first weighted directed graph. For the process of generating a weighted directed graph G(V, E) from a set of nodes V and a set of directed edges E, please refer to existing graph theory techniques for understanding.

[0076] The calculation formula used to calculate the weight of the directed edge between two connected nodes based on image relevance includes:

[0077]

[0078] Where, d n represents the weight between any pair of SIFT feature matching points between the images corresponding to the two nodes, and N is the number of SIFT feature matching point pairs between the images corresponding to any two nodes.

[0079] For details on determining SIFT feature matching points and obtaining corresponding weights, please refer to relevant existing technologies, which will not be described here.

[0080] S3, based on the first weighted and directed graph, generate a first minimal tree graph that is not subject to any constraints for the image to be inserted.

[0081] In graph theory, to generate a minimum tree graph for a set of nodes, we first need to construct a weighted directed graph for these nodes, including a set of nodes and a set of directed edges. The set of directed edges includes bidirectional directed edges between every two nodes and the weights of each directed edge. Then, we use the constructed weighted directed graph as input to the minimum tree graph generation method to construct the corresponding minimum tree graph.

[0082] In one alternative implementation, this step may include:

[0083] Using the first weighted directed graph as input to the Zhu Liu method, a first minimal tree graph, unconstrained for the image to be inserted, is obtained. For ease of description, the first minimal tree graph is denoted as T1.

[0084] The purpose of generating the first minimum tree graph T1 for the images to be inserted is to facilitate the subsequent acquisition of reference relationships between the images to be inserted, the determination of their hierarchy, and for each image to be inserted, to determine the best reference image for that image among the other images to be inserted without any constraints. Furthermore, among all the tree graphs composed of the images to be inserted, the first minimum tree graph T1 generated by the Zhu Liu method without any constraints has the lowest total edge weight compared to the minimum tree graphs with constraints generated by other methods. Therefore, the first minimum tree graph T1 can be used to provide better guidance for the minimum tree graph of the inserted image set.

[0085] Among them, the Zhu-Liu method is a minimum tree graph algorithm proposed by Zhu Yongjin and Liu Zhenhong in 1965. It uses the operations of graph contraction and expansion to draw a polynomial algorithm for finding the minimum tree graph in a directed graph, and is also known as the "Zhu-Liu algorithm".

[0086] Specifically, the implementation steps using the Zhu-Liu method are as follows:

[0087] Step 3a), define an initially empty set of directed edges P;

[0088] Step 3b), set up a virtual node v1, Connect the directed edges from v1 to any node in G1(V1, E1), and let the weight of the directed edge be the sum of the weights of all nodes in G1(V1, E1).

[0089] Step 3c): Traverse all directed edges in G1(V1, E1). For any two nodes v i and v j If from v i to v j There are multiple edges; only the edge with the smallest weight is retained.

[0090] Step 3d): Among all directed edges terminating at each node except v1, find the directed edge with the smallest weight, and let the weight of the found node v1 be the smallest. i The starting point of the directed edge with the minimum weight is π(v). i Add the directed edge with the smallest weight corresponding to all found nodes to the directed edge set P;

[0091] Step 3e): Determine if P can form a cycle. If it cannot form any cycle, then the connected graph formed by P and all its vertices is the first minimal tree graph T1 without any constraints. If P can form at least one cycle, proceed to step 3f).

[0092] Step 3f) performs loop shrinking on G1(V1, E1) and recursively calls steps 3d) and 3e). Here, the loop is denoted by C, and the loop shrinking rules are as follows:

[0093] if <v i ,v j > is a directed edge e in the first set of directed edges E1, where the starting point and ending point satisfy the following conditions: v j If ∈C, then shrink the ring C into a new node v. l , obtain new edge e new = <v i ,v l >, its weight is ω(e new )=ω(e)-ω(π(v j ),v j ).

[0094] if <v i ,v j > is a directed edge e in the first set of directed edges E1, where the starting and ending points satisfy the following conditions: v i ∈C, Then we shrink ring C into a new node v. l , obtain new edge e new = <v l ,v j >, its weight is ω(e new )=ω(v i ,v j ).

[0095] For specific details regarding the Zhu-Liu method, please refer to the relevant existing technology for understanding; further explanation will not be provided here.

[0096] S4. Based on the first minimum tree diagram, determine the level corresponding to each image to be inserted.

[0097] In this embodiment of the invention, for any tree diagram to be processed, the corresponding image level can be determined according to a similar processing method, including:

[0098] For the tree diagram to be processed, based on the level of the current parent node, add h to the level of all child nodes of the current parent node, and update each child node of the current parent node to the new current parent node. Repeat the process of adding h to the level of all child nodes of the current parent node based on the level of the current parent node until the node with the current determined level has no child nodes.

[0099] In this process, adding 'h' to the hierarchy of all child nodes of the current parent node indicates that the parent node serves as a reference image for the child nodes during encoding. The tree diagram to be processed includes the tree diagram of the existing image set or the first minimum tree diagram. For the tree diagram to be processed, the current parent node is initially the root node; that is, during the initial processing in the above manner, the current parent node is the root node. 'h' is a preset natural number greater than 0, such as 1, 2, etc.

[0100] The level of the root node can be set to different values ​​depending on the different tree diagrams to be processed. When the tree diagram to be processed is a tree diagram of the existing image set, the level of the root node is set to a first value; when the tree diagram to be processed is the first minimum tree diagram, the level of the root node is set to a second value; wherein the second value is greater than the first value, and the absolute value of the difference between the two is h. Both the first value and the second value are natural numbers greater than 0, and the specific values ​​can be selected as needed based on h.

[0101] In this embodiment of the invention, since the root node of the existing image set's tree diagram is to be used as the root node of the inserted image set, while the root node of the first minimum tree diagram cannot be used as the root node of the inserted image set, in order to ensure the uniqueness of the level of the root node of the inserted image set, the level of the root node of the existing image set's tree diagram is not the same as the level of the root node of the first minimum tree diagram, and the former is smaller.

[0102] For ease of understanding, the following explanation uses h as 1, the first value as 1, and the second value as 2 as examples. Regarding S4, the tree diagram to be processed is the first minimum tree diagram, and the specific implementation steps may include:

[0103] Step 4a), set the level of the root node in the first minimum tree diagram T1 to 2;

[0104] In other words, when the tree diagram to be processed is the first minimum tree diagram, the level of the root node is set to 2.

[0105] Step 4b): Read all child nodes of the root node in T1, and set the level of all child nodes of the root node in T1 to be 1 greater than the level of their parent node. That is, the level of all child nodes of the root node in T1 is assigned to 3.

[0106] Step 4c): Read the child nodes of each child node of the root node in T1 in turn, and set the level of the child nodes of the root node T1 to be 1 greater than the level of their parent node. That is, the level of the child nodes of the root node in T1 is assigned to 4.

[0107] Step 4d) Repeat the search for the next child node until all nodes in T1 have been read and the hierarchy has been set.

[0108] For the first minimum tree diagram, the level corresponding to each image to be inserted can be reflected in the reference relationship between each image to be inserted and its parent node without any constraints.

[0109] S5. Based on the tree diagram of the existing image set, determine the level corresponding to each existing image in the existing image set.

[0110] S5 still uses the same processing method as S4, which determines the level of the corresponding image for any tree diagram to be processed. We will still use h=1 and the first value=1 as an example for illustration.

[0111] For S5, the tree diagram to be processed is the tree diagram of the existing image set, and the specific implementation steps may include:

[0112] Step 5a), set the level of the root node in the tree diagram of the existing image set to 1;

[0113] In other words, when the tree diagram to be processed is the tree diagram of the existing image set, the level of the root node is set to 1. For ease of description, the tree diagram of the existing image set is denoted as T0.

[0114] Step 5b): Read all child nodes of the root node in T0, and set the level of all child nodes of the root node in T0 to be 1 greater than the level of their parent node. That is, the level of all child nodes of the root node in T0 is assigned to 2.

[0115] Step 5c): Read the child nodes of each child node of the root node in T0 in turn, and set the level of the child nodes of the root node T0 to be 1 greater than the level of their parent node. That is, the level of the child nodes of the root node in T0 is assigned to 3.

[0116] Step 5d): Repeat the search for the next child node until all nodes in T0 have been read and the hierarchy has been set.

[0117] The tree diagram of the existing image set, which determines the level corresponding to each existing image in the existing image set, can reflect the position information of each existing image in the existing image set.

[0118] It should be noted that S5 does not necessarily have to be executed after S4, but can be completed after S1.

[0119] S6. Based on the tree diagram of the existing image set and the first weighted directed graph, establish a weighted directed graph for all images; and connect new directed edges that meet the node requirements on the basis of the weighted directed graph for all images and calculate the weights to obtain a second weighted directed graph.

[0120] In one optional implementation, the step of constructing a weighted directed graph for all images based on the tree diagram of the existing image set and the first weighted directed graph may include:

[0121] S6A-1, Continuing from the node labels in the first node set, assign non-repeating and consecutive node labels to each node in the tree diagram of the existing image set; and add all nodes carrying node labels in the tree diagram of the existing image set to the first node set to obtain the second node set;

[0122] Specifically, the largest node label in the first node set V1 is read, and starting from the next value after the largest node label, node labels are sequentially assigned to all nodes in the tree diagram T0 of the existing image set. Each newly assigned node label is not repeated with the existing node labels in the node set V1 and the newly assigned node labels are consecutive. Then, all nodes with assigned node labels in the tree diagram of the existing image set are added to the node set V1, and the resulting new node set is named the second node set, denoted as V2.

[0123] S6A-2, Decode all existing images in the existing image set to obtain the corresponding decoded images;

[0124] Specifically, for each existing image in the existing image set, the existing decoding method is used to decode it to obtain the corresponding decoded image. Here, no specific restrictions are placed on the existing decoding method.

[0125] S6A-3 connects directed edges from any node corresponding to a decoded image to any node corresponding to an image to be inserted, and calculates the weight of the directed edge between the two connected nodes based on the correlation of the images.

[0126] In this step, the weight of the directed edge between the two connected nodes is calculated based on the correlation of the image. The calculation formula used is shown in formula (1) in S2, and will not be repeated here.

[0127] S6A-4, add all the obtained directed edges and their corresponding weights to the first directed edge set to obtain an updated directed edge set, and obtain a weighted directed graph for all images from the second node set and the updated directed edge set.

[0128] In one optional implementation, the step of connecting new directed edges that satisfy the node requirements and calculating weights on the weighted directed graph for all images to obtain a second weighted directed graph includes:

[0129] S6B-1, based on the weighted and directed graph for all images, connect new directed edges that satisfy the node requirements, and calculate the weight of the directed edge between the two connected nodes according to the correlation of the images.

[0130] The nodes are required to be such that both nodes of each new directed edge are nodes corresponding to the image to be inserted, and the level of the starting node of the new directed edge is lower than the level of the ending node.

[0131] Specifically, based on the weighted directed graph for all images, each new directed edge is connected. <v a ,v b >, where node v a and v b All of these are nodes corresponding to the image to be inserted, and v a The level is less than v b The hierarchy; calculate the v of each group of nodes based on the correlation of the images. a v b The weight of the directed edge obtained by the connection; wherein, in this step, the weight of the directed edge between the two connected nodes is calculated according to the correlation of the image. The calculation formula used is shown in formula (1) in S2, and will not be repeated here.

[0132] S6B-2, add all the new directed edges and their corresponding weights to the updated directed edge set to obtain the second directed edge set, and obtain the second weighted directed graph from the second node set and the second directed edge set.

[0133] All the newly obtained directed edges and their corresponding weights are added to the updated set of directed edges. The resulting set of directed edges is named the second set of directed edges, denoted by E2. From the second set of nodes V2 and the second set of directed edges E2, the second weighted directed graph G2(V2, E2) is obtained.

[0134] It should be noted that S6 does not necessarily have to be executed after S5, but can be completed after S2.

[0135] As mentioned earlier, constructing a first weighted and directed graph for all images to be inserted is to use it as input to the Zhu Liu method to generate a first minimal tree graph of the images to be inserted, so as to obtain the reference relationship between the images to be inserted, determine the corresponding level, and provide good guidance for the image set after insertion.

[0136] This step establishes a second weighted and directed graph for all images, which is used as input to a minimum tree graph generation method with preset depth and node layer constraints, so as to generate a second minimum tree graph with depth and node layer constraints for all images in the subsequent S8.

[0137] S7, modify the level of each image corresponding node in the second weighted and directed graph to not exceed the target depth.

[0138] The target depth is the pre-set depth of the tree diagram corresponding to the inserted image set.

[0139] In this embodiment of the invention, before step S7, a specific value can be set for the maximum depth of the tree diagram corresponding to the inserted image set obtained after inserting the image to be inserted into the existing image set. This maximum depth is named after the target depth and can be represented by D.

[0140] For example, in one optional implementation, the target depth can be set in the form of parameter initialization before S1.

[0141] Since the target depth limits the depth of the tree graph corresponding to the inserted image set, the level of each node corresponding to each image in the second weighted directed graph cannot be greater than the target depth.

[0142] Specifically, this step may include:

[0143] For each image in the second weighted directed graph, determine whether the level of the node corresponding to the image is greater than the target depth;

[0144] If so, correct the layer of the corresponding node in the image to the target depth;

[0145] If not, keep the hierarchy of the corresponding node in the image unchanged.

[0146] Through the above-mentioned hierarchical correction, the hierarchy of each image node in the second weighted directed graph satisfies the constraint of the target depth.

[0147] S8. Based on the target depth, the second weighted and directed graph, and the levels corresponding to each image to be inserted and the existing images, generate a second minimum tree graph with depth and node layer constraints.

[0148] The process can be as follows: The second weighted directed graph, the target depth, and the levels corresponding to each image to be inserted and each existing image are used as inputs to a minimum tree graph generation method with preset depth and node layer constraints, resulting in a second minimum tree graph for the inserted image set. The levels corresponding to each image to be inserted and each existing image are levels corrected using the target depth. The purpose of this step is to ensure that the second minimum tree graph, while satisfying the target depth constraint and without altering the tree graph of the existing image set, determines the optimal reference image for each image to be inserted under the depth and node layer constraints, thereby maximizing coding efficiency and minimizing storage space while achieving the lowest possible computational complexity.

[0149] The process of generating S8 using a minimum tree diagram with preset depth and node layer constraints includes the following steps:

[0150] a1, for the current iteration, obtain the corresponding input parameters, including the node set V. B and V U and the set of directed edges E B and E G ;

[0151] Among the input parameters of the first iteration, V B It contains only the root node v of the existing image set. R V U E B All are empty, E G The second weighted directed graph contains v R All directed edges originating from the first element are labeled as belonging to the first class.

[0152] In this embodiment of the invention, according to the iteration order, the current iteration is successively the 0th, 1st, 2nd, etc., and the corresponding number represents the iteration number of the current iteration.

[0153] In this embodiment of the invention, VB and E B V represents the set of nodes and the set of directed edges of the second minimal tree graph of the inserted image set. U and E G This is to obtain the node set and directed edge set of the intermediate process for obtaining the node set and directed edge set of the second minimum tree graph of the inserted image set.

[0154] Before the iteration begins, the root node of the existing image set's tree diagram T0 can be set to v. R The second node set V2 contains Q nodes, and the node set V... B V U An empty set of directed edges E B E G Empty.

[0155] To avoid altering the existing tree diagram of the image set, since its root node has no parent node, this root node should also serve as the root node of the tree diagram of the inserted image set. Therefore, before starting the iteration, the root node of the existing image set should first be added to the node set of the second minimal tree diagram, indicating that v will be... R It serves as the root node of the inserted image set.

[0156] Then, in G2(V2, E2), v R All directed edges originating from E are labeled as Class I, and these Class I directed edges are added to the previously empty E. G In the given set of directed edges, V is used as the candidate for the second minimum tree graph. Therefore, the input parameters for the first iteration are: V B It contains only the root node v of the existing image set. R V U E B All are empty, E G The second weighted directed graph contains v R All directed edges originating from the first point are labeled as Class I.

[0157] The embodiments of the present invention can employ any method of marking directed edges to distinguish different directed edges, such as using different symbols or numbers, or different line types, etc.

[0158] For example, in one optional implementation, different colors can be used to distinguish them. For instance, directed edges marked as type 1 can be colored green, directed edges marked as type 2 can be colored blue, directed edges marked as type 3 can be colored red, and so on. For ease of understanding, the specific description of the subsequent steps will use this type of coloring as an example.

[0159] It should be noted that, in this embodiment of the invention, for the same iteration number, the initially obtained input parameter V during the processing... B V U E B and E G During the current iteration, the content of the dataset may change continuously. Therefore, in the following text, although each step will still refer to V... B V U E B and E G This indicates that, but both represent the current V at the corresponding step. B V U E B and E G .

[0160] a2, except V B The highest-level node, excluding existing nodes, joins V. U In the middle, from the current V U Among all directed edges of type I with the middle node as the endpoint, select the directed edge with the smallest weight. <v p ,v q >;

[0161] In this step, the directed edge with the smallest weight <v p ,v q > indicates that v p Among all images that serve as a reference image, v is used p For v q Encoding is the least expensive method. Finding the directed edge with the minimum weight is to determine the next edge to be added to V. B The node and its addition to E B The directed edge.

[0162] For example, this step specifically involves: removing V B The highest-level node, excluding existing nodes, joins V. U In the middle, from the current V U Among all green directed edges with the middle node as the endpoint, select the directed edge with the smallest weight. <v p ,v q >;Among them, directed edges <v p ,v q In the diagram, the starting point and the ending point are nodes v. p and v q Understandably, for the first iteration, V, as the input parameter... B Only v in the middle R In this step, v q =v R .

[0163] a3, determine the current V B Are all nodes in the hierarchy less than or equal to node v? q The level; if so, then v q Add to current V B and directed edges <v p ,v q Marked as Category 2 and added to E B In, and from the current V U Delete v q If not, then there will be a directed edge. <v p ,v q Mark it as category 3 and perform step a2;

[0164] In this step, v q Add to current V B and directed edges <v p ,v q Marked as Category 2 and added to E B The text indicates that v q and <v p ,v q The next one that can be added to V is... B The node and can be added to E B The directed edge.

[0165] For example, this step specifically involves: if the current V B All nodes in the hierarchy are less than or equal to v. q The level will then v q Add to current V B In the middle, there will be a directed edge <v p ,v q >Dye blue and add to E B In the diagram, represents the directed edge selected as a deterministic choice, and starting from the current V. U Delete node v q If the current V B All nodes in the hierarchy are at a level greater than v. q The hierarchy will then have directed edges. <v p ,v q > Color it red, then perform step a2 to reselect directed edges.

[0166] a4, select a line starting with v q Unlabeled directed edge starting from <v q ,v o > If node v o All three of the following conditions must be met: v o Not belonging to the current V B And v o The level is equal to v qThe hierarchy plus h, and v q to v R If the number of directed edges on the path is less than L, then... <v q ,v o Mark it as the first category and add it to the current E G In the middle, execute step a5; if node v o If any of the above conditions are not met, there will be a directed edge. <v q ,v o > Marked as the third category, execute step a6; where L is the quotient of the difference between the target depth and h, i.e., L = (D-1) / h;

[0167] Satisfying the above three conditions indicates that v o Not yet joined V B In the middle, v q It is v o The parent node, and v q The target depth constraint must be satisfied. If these three conditions are met, then a directed edge will be formed. <v q ,v o >As someone who can join E B The candidate directed edges are to be further verified.

[0168] For example, this step specifically involves: selecting a [v] q Uncolored directed edge starting from <v q ,v o >. If node v o If all three conditions are met, then the directed edge is... <v q ,v o >Make it green and add it to set E G In the middle, and execute step a5; if node v o If any of the above conditions are not met, there will be a directed edge. <v q ,v o >Painted in red, proceed to step a6.

[0169] a5, in v o Choose the directed edge with the smallest weight from all first-class directed edges that terminate at the destination. <v s ,v o Mark all remaining first-class directed edges as third-class and remove them from the current E G Delete;

[0170] This step selects the directed edge with the smallest weight. <v s ,v o > indicates that with v s Encode v for reference image o The required encoding cost is higher than encoding v using other images as reference images. oThe required encoding cost should be low. Therefore, it can be... <v s ,v o >As candidate directed edges, to further verify whether E can be added B For other directed edges, they are not considered as edges that can be added to E. B Candidate directed edges.

[0171] For example, this step specifically involves: in node v o Choose the directed edge with the smallest weight from all the green directed edges at the destination. <v s ,v o > Color all remaining green directed edges red, and remove these directed edges from the current E. G Delete it.

[0172] a6, determine with respect to v q Check if all directed edges originating from this point have been marked; if so, proceed to step a7; otherwise, proceed to step a4.

[0173] This step applies to v q Check all child nodes; if there are still v... q If the directed edges to this child node have not yet been marked, proceed to step a4 to mark them, until v. q All directed edges to its child nodes have been marked.

[0174] A6 is specifically judged based on v. q Check if all directed edges originating from v have been colored. If all have been colored, proceed to step a7; if not, proceed to step a4, and select a new directed edge originating from v. q Unlabeled directed edge starting from <v q ,v o Process until v q All directed edges originating from this point have been colored.

[0175] a7, determine if the iteration number corresponding to the current iteration is equal to the total number of nodes in the second node set minus two; if not, change the final V obtained in the current iteration. B V U E B and E G As the input parameter for the next iteration, the iteration number corresponding to the current iteration is incremented by 1 to become the iteration number for the next iteration, and step a1 is executed; if it equals, the V obtained from the current iteration is... B and E B The second minimum tree diagram T2 = (V) is obtained. B E B ).

[0176] If the iteration number corresponding to the current iteration reaches Q-2, it indicates that all images to be inserted and all images in the existing image set have been added to V. B If there are no more nodes to add, then the iteration stopping condition has been met, and the iteration stops. The latest V in the current iteration is then used. B and E B This forms the second minimum tree diagram, denoted by T2. If the iteration number corresponding to the current iteration has not reached Q-2, it indicates that the iteration stopping condition has not been met. In this case, the iteration number needs to be incremented by 1 for the next iteration, and the process returns to step a1. After returning to step a1, "for the current iteration, obtain the corresponding input parameters" refers to the V obtained in the current iteration before returning. B V U E B and E G .

[0177] S9. Based on the second minimum tree diagram, determine the reference image for each image to be inserted, and use each reference image to encode the corresponding image to be inserted. The encoding results of all images to be inserted constitute the inserted image set.

[0178] In one alternative implementation, this step may include:

[0179] S91, for each image to be inserted, the image corresponding to the parent node of the image to be inserted in the second minimum tree diagram is used as the reference image of the image to be inserted;

[0180] S92, calculate the perspective transformation matrix between the image to be inserted and the corresponding reference image;

[0181] For an explanation of the concept of the perspective transformation matrix and the calculation method of the perspective transformation matrix between the image to be inserted and the corresponding reference image, please refer to the relevant prior art; it will not be explained in detail here.

[0182] S93, perform a geometric transformation on the reference image corresponding to the image to be inserted according to the perspective transformation matrix to obtain the geometrically transformed reference image corresponding to the image to be inserted.

[0183] Geometric transformation of an image, also known as image space transformation, maps coordinate positions in one image to new coordinate positions in another image. Common geometric transformation methods include rotation, translation, scaling, mirroring, transpose, shearing, and several combined transformations, such as rigid body transformation, affine transformation, and homography transformation. In this step, a specific geometric transformation method can be appropriately selected according to needs.

[0184] S94, perform optical transformation on the geometrically transformed reference image to obtain the geometrically and optically transformed reference image corresponding to the image to be inserted;

[0185] Since similar images may not have been taken by the same person or at the same time, there may be differences in brightness between them. The purpose of optical transformation is to process the brightness of the reference image after geometric transformation, so that the brightness of the reference image after geometric optical transformation is as close as possible to the brightness of the image to be inserted. Common optical transformation methods include optical transformation based on linear fitting and optical transformation based on quadratic fitting. In this step, the specific geometric transformation method can be selected appropriately according to needs.

[0186] S95, the reference image after the geometric optical transformation is used to perform inter-frame coding on the image to be inserted.

[0187] In addition to the HEVC inter-frame coding method, other inter-frame coding methods such as H.264 and H.266 can also be used.

[0188] Since the unconstrained minimum tree graph generated by the Zhu-Liu method has the lowest total edge weights for a weighted directed graph compared to constrained minimum tree graphs generated by other methods, the image set constructed based on the unconstrained minimum tree graph also has the highest coding efficiency. It can be seen that although the minimum tree graph of the image set after image insertion has a depth constraint, using the unconstrained minimum tree graph as a guide during the generation of this minimum tree graph allows for the determination of the optimal reference image for each image to be inserted under the target depth constraint and without changing the coding architecture of the existing image set. This further improves the coding efficiency of the image set after image insertion and further reduces its required storage space. Based on the above concept, this embodiment of the invention proposes an image set construction method for multiple image insertions.

[0189] In the solution provided by this embodiment of the invention, firstly, a first weighted directed graph of the image to be inserted is established, and a first minimal tree graph without any constraints is generated for the image to be inserted. Secondly, based on the first minimal tree graph, the level corresponding to each image to be inserted is determined, and also based on the tree graph of the existing image set, the level corresponding to each existing image in the existing image set is determined. Next, based on the tree graph of the existing image set and the first weighted directed graph, a weighted directed graph for all images is established, and new directed edges that meet the node requirements are connected on it and weights are calculated to obtain a second weighted directed graph. Then, the level of the node corresponding to each image in the second weighted directed graph is corrected to not exceed the target depth of the tree graph corresponding to the inserted image set. Then, based on the target depth, the second weighted directed graph, and the levels corresponding to each image to be inserted and the existing images respectively, a second minimal tree graph with depth and node layer constraints is generated. Finally, based on the second minimal tree graph, a reference image for each image to be inserted is determined, and the corresponding image to be inserted is encoded using each reference image to obtain the inserted image set. The method of this invention utilizes a first minimal tree diagram without any constraints as guidance, and generates a second minimal tree diagram using a minimal tree diagram generation method with depth and node layer constraints. This enables the determination of the optimal reference image for each image to be inserted, under target depth constraints and without altering the existing image set architecture. Compared to existing schemes that do not utilize a minimal tree diagram without any constraints as guidance, the method of this invention can reduce the storage space required for the inserted image set, further improving coding efficiency. The method of this invention can be used to manage image sets stored in the cloud, and to add multiple images to an existing image set in image database management scenarios.

[0190] To verify the effectiveness of the embodiments of the present invention, the following simulation experiments further illustrate the technical effects of the present invention:

[0191] 1) Experimental conditions:

[0192] The system uses Windows Server 2008, with an Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz processor and 64GB of RAM. The programming language is C++, and the programming software is VS2010.

[0193] Detailed information about the experimental test images is shown in Table 1:

[0194] Table 1

[0195] Xidian 12 9 1024x1024 sculpture 11 7 768x1024 corner 10 7 1024x1024 defense 8 4 1024x1024 pantheon 14 4 1024x1024

[0196] The above five test image sets, xidian, sculpture, corner, defense, and pantheon, are used as existing image sets. After calculating the weights of the directed edges between corresponding nodes of two images using the scale-invariant feature transformation method SIFT corresponding to formula (1) in S2 of this embodiment, the number of nodes in the tree diagram corresponding to the abstracted existing image set xidian is 12, and the number of nodes corresponding to the image to be inserted is 9. The same applies to the other test image sets.

[0197] Experimental content and results analysis:

[0198] Using the method of this invention (hereinafter referred to as the method of this invention) and the existing image insertion method that does not utilize any unconstrained minimum tree graph (hereinafter referred to as the Xu Dongmei method), image sets after image insertion were constructed for the above five test sets, i.e., the inserted image sets. The total edge weights of the corresponding minimum tree graphs were determined, and the results are shown in Table 2. As can be seen from Table 2, the total edge weights of the minimum tree graphs corresponding to the inserted image sets constructed by this invention are all less than the total edge weights of the minimum tree graphs generated based on the Xu Dongmei method for each existing image set. Therefore, this invention method reduces the total edge weights of the minimum tree graphs corresponding to the inserted image sets, indicating that compared with the Xu Dongmei method, this invention method requires less storage space for the image sets, indicating higher encoding efficiency.

[0199] Table 2

[0200]

[0201]

[0202] Furthermore, by constructing the inserted image set according to the two methods described above, the improvement in coding efficiency of the present invention compared to Xu Dongmei's method was obtained, as shown in Table 3. In Table 3, each value represents the percentage reduction in the required coding bit rate of the present invention compared to Xu Dongmei's method on an existing image set. In Table 3, under the same peak signal-to-noise ratio (PSNR), the lower the required bit rate, the higher the coding efficiency.

[0203] As can be seen from Table 3, compared with Xu Dongmei's method, the method of the present invention requires less bit rate for each test image set. For example, for the Xidian test set, the bit rate required by the present invention is 5.27% lower, indicating that the image set constructed by the method of the present invention requires less storage space and has higher coding efficiency.

[0204] Table 3

[0205]

[0206] In summary, compared with existing technologies, the inserted image set constructed by the method of this invention utilizes a minimal tree diagram without any constraints to guide the determination of reference images for the images to be inserted, thereby obtaining the optimal reference image under depth constraints. This reduces the storage space required for the inserted image set and further improves coding efficiency. The method of this invention can be applied to managing image sets stored in the cloud and adding multiple images to an existing image set in image database management scenarios, significantly improving coding efficiency.

[0207] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. An image set construction method for multiple image insertion, characterized by, include: Obtain the tree diagram of the existing image set and the image to be inserted; Establish a first weighted directed graph for the image to be inserted; Based on the first weighted and directed graph, generate a first minimal tree graph that is not subject to any constraints for the image to be inserted; Based on the first minimum tree diagram, determine the level corresponding to each image to be inserted; Based on the tree diagram of the existing image set, determine the level corresponding to each existing image in the existing image set; Based on the tree diagram of the existing image set and the first weighted directed graph, a weighted directed graph for all images is established; and based on the weighted directed graph for all images, new directed edges that meet the node requirements are connected and their weights are calculated to obtain a second weighted directed graph; The hierarchy of each node corresponding to each image in the second weighted and directed graph is modified to not exceed the target depth; wherein, the target depth is the pre-set depth of the tree graph corresponding to the inserted image set; Based on the target depth, the second weighted and directed graph, and the levels corresponding to each image to be inserted and the existing images, a second minimum tree graph with depth and node layer constraints is generated. Based on the second minimum tree diagram, a reference image is determined for each image to be inserted, and the corresponding image to be inserted is encoded using each reference image. The encoded results of all images to be inserted constitute the inserted image set. The step of generating a second minimum tree graph with depth and node layer constraints based on the target depth, the second weighted directed graph, and the levels corresponding to each image to be inserted and existing images includes: a1, for the current iteration, retrieves the corresponding input parameters, including the node set. and and the set of directed edges and Among the input parameters of the first iteration, It contains only the root node of the existing image set. , , All are empty. The second weighted directed graph contains All directed edges originating from the first element are labeled as belonging to the first class. a2, will be removed The highest-level node, excluding existing nodes, has joined. In the middle, from the current Among all directed edges of type I with the middle node as the endpoint, select the directed edge with the smallest weight. ; a3, determine the current state Are all nodes in the hierarchy less than or equal to the node? The level; if so, then Join current and directed edges Marked as Category 2 and added to In the middle, and from the present Delete If not, then there will be a directed edge. Mark it as category 3 and perform step a2; a4, select one Unlabeled directed edge starting from If node All three of the following conditions must be met: Not belonging to the current ,and The level equals The hierarchy ,as well as arrive The number of directed edges on the path is less than Then Mark as the first category and add to the current category. In the middle, execute step a5; if node If any of the above conditions are not met, there will be a directed edge. Marked as category three, execute step a6; where, This is a pre-defined natural number greater than 0, representing the hierarchical interval between parent nodes and child nodes in the tree diagram; The difference between the target depth and the value after subtracting one is then multiplied by... The quotient; a5, in the context of Choose the directed edge with the smallest weight from all first-class directed edges that terminate at the destination. All remaining first-class directed edges are marked as third-class and removed from the current... Delete; a6, judged by Check if all directed edges originating from this point have been marked; if so, proceed to step a7; otherwise, proceed to step a4. a7, determine if the iteration number corresponding to the current iteration is equal to the total number of nodes in the second node set minus two; if not, change the final result obtained in the current iteration. , , and As the input parameter for the next iteration, the iteration number corresponding to the current iteration is incremented by 1 to become the iteration number for the next iteration, and step a1 is executed; if equal, the result obtained from the current iteration is... and The second minimum tree diagram is obtained. .

2. The image set construction method for multiple image insertion according to claim 1, characterized in that, The establishment of the first weighted directed graph for the image to be inserted includes: Each image to be inserted is abstracted into a node; from First, assign node labels to the nodes corresponding to each image to be inserted, and then form the first node set by all the nodes corresponding to the images to be inserted that carry the node labels; where, The default natural number; In the first set of nodes, a directed edge is connected from any node to any other node, and the weight of the directed edge between the two connected nodes is calculated based on the relevance of the image. All directed edges and their corresponding weights obtained from the first set of nodes constitute the first set of directed edges, and the first set of nodes and the first set of directed edges constitute the first weighted directed graph for the image to be inserted.

3. The image set construction method for multiple image insertion according to claim 1, characterized in that, The step of generating a first minimal tree graph without any constraints for the image to be inserted based on the first weighted directed graph includes: Using the first weighted directed graph as input to the Zhu Liu method, a first minimal tree graph, unconstrained for the image to be inserted, is obtained.

4. The image set construction method for multiple image insertion according to claim 1, characterized in that, The step of determining the level corresponding to each existing image in the existing image set based on the tree diagram of the existing image set, or the step of determining the level corresponding to each image to be inserted based on the first minimum tree diagram, includes: For the tree diagram to be processed, based on the current parent node's level, add the levels of all child nodes of the current parent node. Then, update each child node of the current parent node to the new current parent node, and repeat the process of adding the hierarchy of all child nodes of the current parent node to the hierarchy of the current parent node. The process continues until the currently determined level of nodes has no child nodes; The tree diagram to be processed includes either the tree diagram of the existing image set or the first minimum tree diagram. For the tree diagram to be processed, the current parent node is initially the root node. When the tree diagram to be processed is the tree diagram of the existing image set, the level of the root node is set to a first value. When the tree diagram to be processed is the first minimum tree diagram, the level of the root node is set to a second value. The second value is greater than the first value, and the absolute value of the difference between the two is... .

5. The method for constructing an image set for inserting multiple images according to claim 2, characterized in that, The step of constructing a weighted directed graph for all images based on the tree diagram of the existing image set and the first weighted directed graph includes: Continuing from the node labels in the first node set, assign non-repeating and consecutive node labels to each node in the tree diagram of the existing image set; and add all nodes carrying node labels in the tree diagram of the existing image set to the first node set to obtain the second node set; Decode all existing images in the existing image set to obtain the corresponding decoded images; Connect the directed edges from any node corresponding to the decoded image to any node corresponding to the image to be inserted, and calculate the weight of the directed edges between the two connected nodes based on the relevance of the images. All the obtained directed edges and their corresponding weights are added to the first set of directed edges to obtain an updated set of directed edges. The weighted directed graph for all images is obtained by the second set of nodes and the updated set of directed edges.

6. The method for constructing an image set for inserting multiple images according to claim 5, characterized in that, The step of connecting new directed edges that satisfy the node requirements and calculating weights on the weighted directed graph for all images to obtain a second weighted directed graph includes: Based on the weighted and directed graph for all images, new directed edges that meet the node requirements are connected, and the weight of the directed edge between the two connected nodes is calculated according to the relevance of the images; wherein, the node requirements are that the two nodes of each new directed edge are nodes corresponding to the images to be inserted, and the level of the starting node of the new directed edge is less than the level of the ending node. All the new directed edges and their corresponding weights are added to the updated set of directed edges to obtain a second set of directed edges. The second weighted directed graph is obtained from the second set of nodes and the second set of directed edges.

7. The method for constructing an image set for inserting multiple images according to claim 1, characterized in that, The step of correcting the level of each image corresponding node in the second weighted directed graph to not exceed the target depth includes: For each image in the second weighted directed graph, determine whether the level of the node corresponding to the image is greater than the target depth; If so, correct the layer of the corresponding node in the image to the target depth; If not, keep the hierarchy of the corresponding node in the image unchanged.

8. The method for constructing an image set for inserting multiple images according to any one of claims 2, 5, and 6, characterized in that, The calculation formula used to calculate the weight of the directed edge between two connected nodes based on the correlation of the image includes: in, The weights between any pair of SIFT feature matching points in the images corresponding to the two nodes. This represents the number of SIFT feature matching point pairs between any two nodes.

9. The method for constructing an image set for inserting multiple images according to claim 1, characterized in that, The step of determining a reference image for each image to be inserted based on the second minimum tree diagram, and encoding the corresponding image to be inserted using each reference image, with the encoded results of all images to be inserted constituting the inserted image set, includes: For each image to be inserted, the image corresponding to the parent node of the image to be inserted in the second minimum tree diagram is used as the reference image of the image to be inserted. Calculate the perspective transformation matrix between the image to be inserted and the corresponding reference image; Based on the perspective transformation matrix, a geometric transformation is performed on the reference image corresponding to the image to be inserted to obtain the geometrically transformed reference image corresponding to the image to be inserted. The geometrically transformed reference image is optically transformed to obtain the geometrically and optically transformed reference image corresponding to the image to be inserted. The reference image after the geometric optical transformation is used to perform inter-frame coding on the image to be inserted.

Images