Multi-modal alignment method and system based on multi-similarity fusion and large model decision

Abstract

The application provides a multi-modal alignment method and system based on multi-similarity fusion and large model decision, relates to the field of multi-modal alignment, and comprises the following steps: obtaining a graph-text pair to be matched and a graph-text pair in a knowledge base; performing feature coding on the graph-text pair to obtain shallow features and deep features; constructing a multi-level feature matching model and training the same; inputting the shallow features and the deep features into the trained multi-level feature matching model to obtain a semantic similarity matrix and a structural similarity matrix; calculating a total similarity score of the semantic similarity matrix and the structural similarity matrix through a designed similarity fusion and noise balancing mechanism; and inputting the graph-text pair with a total similarity score greater than a preset value into a large model to determine a most similar graph-text pair from the graph-text pairs in the knowledge base. The multi-modal matching method has high matching accuracy.

Description

Technical Field

[0001] This application relates to the field of multimodal alignment, and in particular to a multimodal alignment method and system based on multi-similarity fusion and large model decision-making. Background Technology

[0002] With the rapid development of the Internet, the generation and dissemination of massive amounts of data have become unprecedentedly rapid and complex. These data not only contain text, but also multimodal information such as images, videos, and audio. How to effectively associate data from different modalities but pointing to the same entity has become an urgent problem to be solved. However, existing multimodal alignment methods have the following shortcomings: (1) Insufficient cross-modal semantic alignment: Traditional methods rely on simple cosine similarity to calculate the distance between images and text, which makes it difficult to align the semantic information contained in data from different modalities. (2) Insufficient feature capture: When converting images and text into feature vectors, some key information is lost. (3) Difficulty in dealing with noise interference: Pairs of images and text in big data are not necessarily related, and existing methods cannot cope with noise interference in the data. The purpose of this invention is to provide a multimodal alignment method based on a feature multi-level matching network to address the problems of insufficient cross-modal semantic alignment and difficulty in dealing with noise interference in existing multimodal methods proposed in the above background technology. Summary of the Invention

[0003] The purpose of this invention is to address the problem of poor alignment accuracy in existing multimodal alignment methods by providing a multimodal alignment method and system based on multi-similarity fusion and large model decision-making.

[0004] The above-mentioned objective of this application is achieved through the following technical solution:

[0005] S1: Get the image-text pairs to be matched and the image-text pairs in the knowledge base;

[0006] S2: Perform feature encoding on the image-text pair to obtain shallow features and deep features;

[0007] S3: Construct and train a multi-level feature matching model; input shallow and deep features into the trained multi-level feature matching model to obtain the semantic similarity matrix and the structural similarity matrix;

[0008] S4: Calculate the total similarity score of the semantic similarity matrix and the structural similarity matrix through the designed similarity fusion and noise balance mechanism;

[0009] S5: Input the image-text pairs with a total similarity score greater than the preset value into the large model, and determine the image-text pairs that are most similar to the image-text pair to be matched from the image-text pairs in the knowledge base.

[0010] Optionally, step S1 includes:

[0011] The image-text pair to be matched can be represented as ,in and They represent the first For matching images and text, This represents the total number of image-text pairs to be matched; image-text pairs in the knowledge base are represented as... ,in and They represent the first For matching images and text, This represents the total number of text-image pairs in the knowledge base.

[0012] Optionally, step S2 includes:

[0013] S21: Add a CLS tag at the beginning of each sentence in the text, and concatenate sentences representing different information in the text and separate them with a SEP tag; resize the images to a uniform size of 512×512;

[0014] S22: Input the image-text pair into the CLIP model, obtain the output of the last layer of the Transformer module in CLIP, and get the shallow features, as follows:

[0015]

[0016]

[0017] in These represent the shallow features of the image to be matched and the images in the database, respectively. These represent the shallow features of the text to be matched and the text in the database, respectively.

[0018] S23: Map the shallow features to obtain the mapped features. and ;

[0019] S24: Normalize the mapped features to obtain deep features;

[0020]

[0021] The deep features of the image to be matched are: The deep features of the images in the database are The deep features of the text to be matched are: The deep features of the text in the database are .

[0022] Optionally, step S3 includes:

[0023] S31: Construct the feature representation of the image-text pair to be matched using shallow and deep features. Feature representation of text-image pairs in the knowledge base ;

[0024] S32: Through a multi-level feature matching model, features... and characteristics Similarity calculations were performed, resulting in 6 similarity matrices.

[0025] Optionally, step S32 includes:

[0026] The multi-level feature matching model includes: Block1 unit, Block2 unit, Block3 unit, Block4 unit, Block5 unit, and Block6 unit;

[0027] Block1 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the similarity matrix. ;

[0028] Block2 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the semantic similarity matrix. ;

[0029] Block3 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the semantic similarity matrix. ;

[0030] Block4 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the structural similarity matrix. ;

[0031] Block5 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the structural similarity matrix. ;

[0032] Block6 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the structural similarity matrix. .

[0033] This application employs the above-described technical solution, where Block 1-3 units use an attention mechanism to calculate the semantic similarity between features, and Block 4-6 units calculate the structural differences between features. Existing technologies typically use complex neural networks to calculate the semantic similarity between features, neglecting symbolic differences between texts and structural differences between images. This invention can simultaneously acquire similarities across different dimensions between different modalities.

[0034] Optionally, step S4 includes:

[0035] matrix sum matrix Adding them together gives ;

[0036] matrix sum matrix Adding them together gives ;

[0037] matrix sum matrix Adding them together gives ;

[0038] matrix , and These represent the image similarity, text similarity, and image-text cross-modal similarity between the image-text pair to be matched and the image-text pairs in the database, respectively.

[0039] The image similarity, text similarity, and image-text cross-modal similarity are weighted and fused to obtain the total similarity score, as shown below:

[0040]

[0041] in , as well as Indicates the weighting coefficient; This represents the total similarity score.

[0042] A multimodal alignment system based on multi-similarity fusion and large model decision-making, the system includes: an acquisition module, a processing module and a display module;

[0043] The acquisition module, processing module, and display module are connected sequentially.

[0044] The acquisition module is used to acquire the image-text pairs to be matched and the image-text pairs in the knowledge base;

[0045] The processing module is used to perform feature encoding on the image-text pairs to obtain shallow features and deep features;

[0046] The processing module is also used to construct and train a multi-level feature matching model; shallow features and deep features are input into the trained multi-level feature matching model to obtain semantic similarity matrix and structural similarity matrix;

[0047] The processing module is also used to calculate the total similarity score of the semantic similarity matrix and the structural similarity matrix through the designed similarity fusion and noise balance mechanism;

[0048] The processing module is also used to input image-text pairs with a total similarity score greater than a preset value into the large model, and to determine the image-text pair most similar to the image-text pair to be matched from the image-text pairs in the knowledge base;

[0049] The display module is used to visualize the most similar pairs of images and text.

[0050] This application employs the aforementioned technical solution, inputting an image-text pair and retrieving the most similar image-text pair from the database. Six similarity metrics are obtained, three of which are semantic similarity and the other three are structural similarity. Different similarities within the same modality are fused, such as the fusion of semantic and structural similarity between texts, and between images and image-text pairs. If the image is noise interference, the text similarity still plays a role; conversely, if the text is noise, the image similarity still plays a role. Therefore, the technical solution of this application has the ability to resist noise interference. During the matching process, the cross-modal similarity of images, text, or image-text pairs contributes differently to the overall matching task; therefore, three trainable weights are set to balance the contributions of different similarities.

[0051] An electronic device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to enable the electronic device to perform a multimodal alignment method based on multi-similarity fusion and large model decision-making.

[0052] A computer-readable storage medium storing instructions that, when executed, perform a multimodal alignment method based on multi-similarity fusion and large model decision-making.

[0053] The beneficial effects of the technical solution provided in this application are:

[0054] By extracting shallow and deep features, the system effectively integrates images and text in different feature spaces, significantly improving the accuracy of cross-modal semantic matching. The designed noise balancing mechanism can intelligently identify and eliminate interference factors in the data, ensuring stable performance of the system in complex data environments. Combined with the intelligent decision-making capabilities of large models, the system can accurately locate the optimal match from massive candidate results, ultimately achieving efficient and reliable cross-modal entity association, providing strong technical support for applications such as intelligent search and content understanding. Attached Figure Description

[0055] The present application will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0056] Figure 1 This is a flowchart of an embodiment of this application;

[0057] Figure 2 This is a model structure diagram in the embodiments of this application;

[0058] Figure 3 This is a comparison chart of experimental results in the embodiments of this application;

[0059] Figure 4 This is a schematic diagram of the electronic device structure in the embodiments of this application. Detailed Implementation

[0060] To provide a clearer understanding of the technical features, objectives, and effects of this application, the specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0061] The embodiments of this application provide a multimodal alignment method based on multi-similarity fusion and large model decision-making.

[0062] Please refer to Figure 1 , Figure 1 This is a flowchart of a multimodal alignment method based on multi-similarity fusion and large model decision-making in an embodiment of this application, including:

[0063] S1: Get the image-text pairs to be matched and the image-text pairs in the knowledge base;

[0064] S2: Perform feature encoding on the image-text pair to obtain shallow features and deep features;

[0065] S3: Construct and train a multi-level feature matching model; input shallow and deep features into the trained multi-level feature matching model to obtain the semantic similarity matrix and the structural similarity matrix;

[0066] As one example, training the model and updating the relevant network parameters using a contrastive loss function can increase the distance between negative samples and shorten the distance between positive samples, thus effectively promoting model training.

[0067] S4: Calculate the total similarity score of the semantic similarity matrix and the structural similarity matrix through the designed similarity fusion and noise balance mechanism;

[0068] S5: Input the image-text pairs with a total similarity score greater than the preset value into the large model, and determine the image-text pairs that are most similar to the image-text pair to be matched from the image-text pairs in the knowledge base.

[0069] As one example, the image-text pairs to be matched and the data in the database are fed into a trained model. The model can obtain the similarity ranking of all image-text pairs in the database with the image-text pairs to be matched, and select the top 20 with the highest probability to feed into a large model (such as DeepSeek). The large model then selects the image-text pair most similar to the one to be matched from the 20 with the highest probability. Through the powerful understanding ability of the large model, a judgment can be made effectively.

[0070] As one implementation, DeepSeek was adapted to similarity fusion. A similarity matrix was generated through similarity fusion and noise balance, representing the similarity between all data in the database and the user-input image-text pairs. However, the highest-predicted similarity pair is not guaranteed to be correct; therefore, the top-k image-text pairs are selected and fed into a larger model for evaluation. Here, k is set to 10, indicating that the 10 most similar image-text pairs are selected from the model's predicted similarity and fed into the larger model, which then determines which of these 10 pairs is the best match based on the user's input.

[0071] Step S1 includes:

[0072] The image-text pair to be matched can be represented as ,in and They represent the first For matching images and text, This represents the total number of image-text pairs to be matched; image-text pairs in the knowledge base are represented as... ,in and They represent the first For matching images and text, This represents the total number of text-image pairs in the knowledge base.

[0073] Step S2 includes:

[0074] S21: Add a CLS tag at the beginning of each sentence in the text, and concatenate sentences representing different information in the text and separate them with a SEP tag; resize the images to a uniform size of 512×512;

[0075] S22: Input the image-text pair into the CLIP model, obtain the output of the last layer of the Transformer module in CLIP, and get the shallow features, as follows:

[0076]

[0077]

[0078] in These represent the shallow features of the image to be matched and the images in the database, respectively. These represent the shallow features of the text to be matched and the text in the database, respectively.

[0079] S23: Map the shallow features to obtain the mapped features. and ;

[0080] As one example, features of the mapping layer in CLIP are further extracted. The mapping layer is formed by adding pooling and linear layers to the Transformer module. The linear layer can map images and text to the same vector space, and the features of images and text representing the same semantic information will be more similar.

[0081] S24: Normalize the mapped features to obtain deep features;

[0082]

[0083] Among them, the deep features of the image to be matched are The deep features of the images in the database are The deep features of the text to be matched are: The deep features of the text in the database are .

[0084] Step S3 includes:

[0085] S31: Construct the feature representation of the image-text pair to be matched using shallow and deep features. Feature representation of text-image pairs in the knowledge base ;

[0086] S32: Through a multi-level feature matching model, features... and characteristics Similarity calculations were performed, resulting in 6 similarity matrices.

[0087] Step S32 includes:

[0088] As one example, such as Figure 2 As shown, the matching model consists of 6 modules (Blcok1-Block6), which can be divided into two groups. The first group (Blcok1-Block3) calculates the similarity between the image-text pair to be matched and the image-text pair in the knowledge base based on shallow features. The second group (Blcok4-Block6) calculates the similarity based on deep features.

[0089] The multi-level feature matching model includes: Block1 unit, Block2 unit, Block3 unit, Block4 unit, Block5 unit, and Block6 unit;

[0090] Block1 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the similarity matrix. ;

[0091] As one example, the input to Block1 is the shallow features of the image to be matched. and coarse features of images in the knowledge base Block1 primarily uses an attention mechanism to calculate the similarity between images, which can be represented as: In the image-to-image attention mechanism, shallow features of the images to be matched are used. Image features in the knowledge base, serving as the query vector Q. As the key vector K and value vector V, where , , The attention score is used as the prediction result to obtain a similarity matrix between the image to be matched and the images in the knowledge base. , representing the set of semantic similarity matrices between M candidate images and N images in the knowledge base.

[0092] Block2 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the semantic similarity matrix. ;

[0093] As one example, the input to Block2 is and ,Will As a query Q, Using the key vector K and value vector V, we can obtain the semantic similarity matrix between the M candidate texts and the N knowledge base texts. .

[0094] Block3 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the semantic similarity matrix. ;

[0095] Block4 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the structural similarity matrix. ;

[0096] Block5 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the structural similarity matrix. ;

[0097] Block6 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the structural similarity matrix. .

[0098] Step S4 includes:

[0099] matrix sum matrix Adding them together gives ;

[0100] matrix sum matrix Adding them together gives ;

[0101] matrix sum matrix Adding them together gives ;

[0102] matrix , and These represent the image similarity, text similarity, and image-text cross-modal similarity between the image-text pair to be matched and the image-text pairs in the database, respectively.

[0103] The image similarity, text similarity, and image-text cross-modal similarity are weighted and fused to obtain the total similarity score, as shown below:

[0104]

[0105] in , as well as Indicates the weighting coefficient; This represents the total similarity score.

[0106] As one embodiment, the total score is obtained by weighting the individual scores of different modalities. When a certain modality generates noise interference, the other similarity scores can still function normally. The technical solution of this invention is compared with experimental results of different methods on three public datasets. Figure 3 As shown.

[0107] A multimodal alignment system based on multi-similarity fusion and large model decision-making, the system includes: an acquisition module, a processing module and a display module;

[0108] The acquisition module, processing module, and display module are connected sequentially.

[0109] The acquisition module is used to acquire the image-text pairs to be matched and the image-text pairs in the knowledge base;

[0110] The processing module is used to perform feature encoding on the image-text pairs to obtain shallow features and deep features;

[0111] The processing module is also used to construct and train a multi-level feature matching model; shallow features and deep features are input into the trained multi-level feature matching model to obtain semantic similarity matrix and structural similarity matrix;

[0112] The processing module is also used to calculate the total similarity score of the semantic similarity matrix and the structural similarity matrix through the designed similarity fusion and noise balance mechanism;

[0113] The processing module is also used to input image-text pairs with a total similarity score greater than a preset value into the large model, and to determine the image-text pair most similar to the image-text pair to be matched from the image-text pairs in the knowledge base;

[0114] The display module is used to visualize the most similar pairs of images and text.

[0115] This application also discloses an electronic device. (See reference...) Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. The electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0116] The communication bus 502 is used to enable communication between these components.

[0117] The user interface 503 may include a display screen, and optionally, the user interface 503 may also include a standard wired interface or a wireless interface.

[0118] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0119] This application also discloses a computer-readable storage medium storing multiple instructions adapted for loading by a processor to execute the aforementioned multimodal alignment method based on multi-similarity fusion and large model decision-making.

[0120] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure.

[0121] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A multimodal alignment method based on multi-similarity fusion and large model decision-making, characterized in that, The method includes the following steps: S1: Get the image-text pairs to be matched and the image-text pairs in the knowledge base; S2: Perform feature encoding on the image-text pair to obtain shallow features and deep features; S3: Construct and train a multi-level feature matching model; input shallow and deep features into the trained multi-level feature matching model to obtain the semantic similarity matrix and the structural similarity matrix; Step S3 includes: S31: Construct the feature representation of the image-text pair to be matched using shallow and deep features. Feature representation of text-image pairs in the knowledge base ; These represent the shallow features of the image to be matched and the images in the database, respectively. These represent the shallow features of the text to be matched and the text in the database, respectively; the deep features of the image to be matched are... The deep features of the images in the database are The deep features of the text to be matched are: The deep features of the text in the database are ; S32: Through a multi-level feature matching model, features... and characteristics Similarity calculations were performed, resulting in 6 similarity matrices; Step S32 includes: The multi-level feature matching model includes: Block 1 unit, Block 2 unit, Block 3 unit, Block 4 unit, Block 5 unit, and Block 6 unit; Block1 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the similarity matrix. ; Block2 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the semantic similarity matrix. ; Block3 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the semantic similarity matrix. ; Block4 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the structural similarity matrix. ; Block5 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the structural similarity matrix. ; Block6 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the structural similarity matrix. ; S4: Calculate the total similarity score of the semantic similarity matrix and the structural similarity matrix through the designed similarity fusion and noise balance mechanism; Step S4 includes: matrix sum matrix Adding them together gives ; matrix sum matrix Adding them together gives ; matrix sum matrix Adding them together gives ; matrix , and These represent the image similarity, text similarity, and image-text cross-modal similarity between the image-text pair to be matched and the image-text pairs in the database, respectively. The image similarity, text similarity, and image-text cross-modal similarity are weighted and fused to obtain the total similarity score, as shown below: in , as well as Indicates the weighting coefficient; This represents the total similarity score; S5: Input the image-text pairs with a total similarity score greater than the preset value into the large model, and determine the image-text pairs that are most similar to the image-text pair to be matched from the image-text pairs in the knowledge base.

2. The multimodal alignment method based on multi-similarity fusion and large model decision-making as described in claim 1, characterized in that, Step S1 includes: The image-text pairs to be matched are represented as follows ,in and They represent the first For matching images and text, This represents the total number of image-text pairs to be matched; image-text pairs in the knowledge base are represented as... ,in and They represent the first For matching images and text, This represents the total number of text-image pairs in the knowledge base.

3. The multimodal alignment method based on multi-similarity fusion and large model decision-making as described in claim 2, characterized in that, Step S2 includes: S21: Add a CLS tag at the beginning of each sentence in the text, and concatenate sentences representing different information in the text and separate them with a SEP tag; resize the images to a uniform size of 512×512; S22: Input the image-text pair into the CLIP model, obtain the output of the last layer of the Transformer module in CLIP, and get the shallow features, as follows: in These represent the shallow features of the image to be matched and the images in the database, respectively. These represent the shallow features of the text to be matched and the text in the database, respectively. S23: Map the shallow features to obtain the mapped features. and ; S24: Normalize the mapped features to obtain deep features; The deep features of the image to be matched are: The deep features of the images in the database are The deep features of the text to be matched are: The deep features of the text in the database are .

4. A multimodal alignment system based on multi-similarity fusion and large model decision-making, used to implement the multimodal alignment method based on multi-similarity fusion and large model decision-making as described in any one of claims 1-3, characterized in that, The system includes: an acquisition module, a processing module, and a display module; The acquisition module, processing module, and display module are connected sequentially. The acquisition module is used to acquire the image-text pairs to be matched and the image-text pairs in the knowledge base; The processing module is used to perform feature encoding on the image-text pairs to obtain shallow features and deep features; The processing module is also used to construct and train a multi-level feature matching model; shallow features and deep features are input into the trained multi-level feature matching model to obtain semantic similarity matrix and structural similarity matrix; Step S3 includes: S31: Construct the feature representation of the image-text pair to be matched using shallow and deep features. Feature representation of text-image pairs in the knowledge base ; These represent the shallow features of the image to be matched and the images in the database, respectively. These represent the shallow features of the text to be matched and the text in the database, respectively; the deep features of the image to be matched are... The deep features of the images in the database are The deep features of the text to be matched are: The deep features of the text in the database are ; S32: Through a multi-level feature matching model, features... and characteristics Similarity calculations were performed, resulting in 6 similarity matrices; Step S32 includes: The multi-level feature matching model includes: Block 1 unit, Block 2 unit, Block 3 unit, Block 4 unit, Block 5 unit, and Block 6 unit; Block1 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the similarity matrix. ; Block2 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the semantic similarity matrix. ; Block3 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the semantic similarity matrix. ; Block4 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the structural similarity matrix. ; Block5 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the structural similarity matrix. ; Block6 units are used to compute features through an attention mechanism. and characteristics The similarity between them is used to obtain the structural similarity matrix. ; The processing module is also used to calculate the total similarity score of the semantic similarity matrix and the structural similarity matrix through the designed similarity fusion and noise balance mechanism; Step S4 includes: matrix sum matrix Adding them together gives ; matrix sum matrix Adding them together gives ; matrix sum matrix Adding them together gives ; matrix , and These represent the image similarity, text similarity, and image-text cross-modal similarity between the image-text pair to be matched and the image-text pairs in the database, respectively. The image similarity, text similarity, and image-text cross-modal similarity are weighted and fused to obtain the total similarity score, as shown below: in , as well as Indicates the weighting coefficient; This represents the total similarity score; The processing module is also used to input image-text pairs with a total similarity score greater than a preset value into the large model, and to determine the image-text pair most similar to the image-text pair to be matched from the image-text pairs in the knowledge base; The display module is used to visualize the most similar pairs of images and text.

5. An electronic device, characterized in that, The device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed by a computer, perform the method as described in any one of claims 1-3.

Images