Method and apparatus for characterizing cultural symbols

Abstract

To provide a method for characterizing cultural symbols by using a machine learning model.SOLUTION: A method includes: receiving multiple materials about a symbol unit, the multiple materials at least including a picture drawing the symbol unit, pronunciation of the symbol unit, and an image or a video showing cultural meaning of the symbol unit, and the symbol unit being a single cultural symbol or a combination of multiple cultural symbols; for each material of the multiple materials, analyzing and learning the material to extract features of the material to form a set of feature vectors; fusing all of the formed feature vectors into a tensor; and analyzing and learning the tensor to generate a concept vector characterizing the symbol unit, the concept vector being directly and consistently associated with the symbol unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] Non-limiting, exemplary embodiments of the present disclosure relate generally to the technical field of machine learning, and more particularly, to a method and apparatus for characterizing cultural symbols by using a machine learning model. [Background technology]

[0002] This section introduces aspects that may facilitate a better understanding of the present disclosure. Accordingly, the statements in this section are to be read in this light and are not to be understood as admissions about material in the prior art or material that is not in the prior art.

[0003] Machine learning is the technique of creating model hypotheses about a research problem, using computers to learn model parameters from training data, and ultimately to predict and analyze data. Machine learning models are the core component of machine learning.

[0004] In essence, a machine learning model can be understood as a function that receives data as input and generates an output. One type of machine learning model includes internal parameters and calculates the input based on the internal parameters to generate an output. Another type of machine learning model is a deep learning model, which has multiple layers of machine learning models stacked together to process the input and generate an output, for example, a deep neural network used by a deep learning model includes an input layer, an output layer, and one or more hidden layers between the input layer and the output layer, and each layer performs a nonlinear transformation on its input and generates its output.

[0005] Recently, various technologies have emerged that use machine learning to process human cultural symbols such as texts to make people's lives more convenient. For example, FastText is a fast text classifier developed by Facebook that provides a simple and efficient method of text classification and representation learning. FastText uses machine learning models to analyze the structure and correlation between characters to digitally encode English words, but cannot be applied to hieroglyphics. As another example, Google's vision technology uses machine learning models to analyze the correlation between images and text on the same website page, and can search for images that appear on the same website page based on the text, but cannot use the description of the image itself as a search keyword to find the image.

[0006] Additionally, there also currently exist machine learning solutions for processing human cultural symbols, as disclosed in the following patent applications:

[0007] The solution of the document bearing the patent number U.S. Pat. No. 10,621,420 allows converting an image into a digital code, which is directly and consistently associated with the image and is different from the corresponding digital codes of other images. This solution can be applied to extract the visual features of cultural symbols, but it cannot represent the cultural meanings of the symbols and the interrelationships between the symbols.

[0008] The solution of the document bearing the patent number U.S. Pat. No. 9,779,085 allows the conversion of text into a digital code that is directly and consistently associated with the text and is distinct from the corresponding digital codes of other texts. Although this solution is applicable to the structure and correlation analysis of symbol systems, the digital code cannot represent the cultural meaning of the symbols themselves.

[0009] The solution of the document bearing the patent number U.S. Pat. No. 1,063,5949 allows converting the text of an image into a digital code that is directly and consistently associated with the text and that differs from the digital codes of other texts but is limited only to the visual characteristics of the text.

[0010] The solution in the publication US 2020 / 0342183 A1 uses machine learning models to process text strings and generate meaningful text strings as a response, which is based on text structure and correlation analysis of symbols, but does not include visual or auditory processing of the symbols.

[0011] The solution of the document bearing publication number CN106909625 allows converting the text of an image into a digital code, which has a direct and consistent association with the text, different from the digital code of other texts, and can be used for image-to-image retrieval. This solution does not involve auditory processing of any symbols. [Prior art documents] [Patent documents]

[0012] [Patent Document 1] U.S. Pat. No. 1,062,1420 [Patent Document 2] U.S. Patent No. 9,779,085 [Patent Document 3] U.S. Pat. No. 1,063,5949 [Patent Document 4] US Patent Application Publication No. 2020 / 0342183 [Patent Document 5] China Patent Application Publication No. 106909625 Summary of the Invention [Problem to be solved by the invention]

[0013] To solve or ameliorate at least one of the above problems, the inventors of the present disclosure contemplate a solution for processing human cultural symbols by using machine learning models to generate concept vectors containing the cultural meaning of the symbols for use in artificial intelligence applications such as, for example, sequence conversion and symbol understanding. [Means for solving the problem]

[0014] According to a first aspect of the present disclosure, the object is achieved by a method for characterizing cultural symbols by using a machine learning model, the method comprising: receiving a plurality of materials relating to a symbol unit, the plurality of materials including at least a picture depicting the symbol unit, a pronunciation of the symbol unit, and an image or video showing the cultural meaning of the symbol unit, the symbol unit being a single cultural symbol or a combination of multiple cultural symbols; for each material of the plurality of materials, analyzing and learning the material to extract features of the material and form a set of feature vectors, combining all of the formed feature vectors into a tensor; and analyzing and learning the tensor to generate concept vectors characterizing the symbol unit, the concept vectors being directly and consistently associated with the symbol unit.

[0015] According to a second aspect of the present disclosure, the object is achieved by a machine including a processor and a memory storing instructions which, when executed by the processor, cause the machine to perform a method according to the first aspect.

[0016] According to a third aspect of the present disclosure, the object is realized by a machine-readable medium storing instructions which, when executed on a machine, cause the machine to perform a method according to the first aspect. Effect of the Invention

[0017] The concept vector obtained by digital encoding of cultural symbols in the solution of the present disclosure includes the meaning of multiple features, including, for example, visual features, auditory features, grammatical features, parts of the speech of the symbol, and can characterize the cultural meaning of the symbol itself and the interrelationships between the symbols. Thus, the solution of the present disclosure can characterize a wide range of cultural symbols, such as hieroglyphs, written symbols, mathematical symbols, logical symbols, trademarks, flags, political symbols, written terms, abbreviated characters, mathematical formulas, logical expressions, etc., and can be used for multimodal information retrieval, such as image-to-image, text-to-image, image-to-text, text-to-sound, text-to-video, etc.

[0018] The above and other aspects, features, and advantages of the present disclosure will become more fully apparent from the following detailed description, taken in conjunction with the accompanying drawings, in which like numerals or characters are used to designate similar or equivalent elements. The drawings are illustrated to facilitate a better understanding of the embodiments of the present disclosure and are not necessarily drawn to scale. [Brief description of the drawings]

[0019] [Figure 1] 1 shows a flow chart of a method according to the present disclosure. [Diagram 2] 4 illustrates the formation of feature vectors and tensors in an exemplary process of a method according to the present disclosure. [Diagram 3] 1 illustrates the generation of concept vectors in an exemplary process of a method according to the present disclosure. [Figure 4] 13 illustrates inference of feature vectors and generation of alternative concept vectors when one or two materials are missing in an exemplary process of a method according to the present disclosure. [Diagram 5] 1 illustrates significant differences between concept vectors in a vector space in an exemplary process of a method according to the present disclosure. [Figure 6] 13 illustrates the generation of further concept vectors in an exemplary process of a method according to the present disclosure. [Figure 7] FIG. 1 is a schematic block diagram of a machine according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] The embodiments herein are more fully described below with reference to the accompanying drawings. However, the embodiments herein may be embodied in many different forms and should not be construed as limiting the scope of the appended claims.

[0021] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms "a," "an," and "the" include the plural unless the context clearly indicates otherwise. It is further understood that the terms "comprises," "comprising," "includes," and / or "including," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0022] Additionally, the use of ordinal terms such as "first," "second," "third," and the like, herein to modify elements does not by themselves imply any priority, precedence, or order of one element relative to another element, or a chronological order for performing actions of a method, but is merely used as a marker to distinguish one element having a certain name from another element having the same name (but for the purposes of using ordinal terms) to distinguish between the elements.

[0023] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood. It is further understood that the terms used herein should be interpreted as having a meaning consistent with their meaning in the context of this specification and related art, and should not be interpreted in an idealized or overly formal sense unless so expressly defined herein.

[0024] A flow chart of a method 100 for characterizing a cultural symbol by using a machine learning model according to the present disclosure is shown in Fig. 1. The method 100 includes receiving 101 a plurality of materials related to a symbol unit, the plurality of materials including at least a picture depicting the symbol unit, a pronunciation of the symbol unit, and an image or video showing the cultural meaning of the symbol unit, the symbol unit being a single cultural symbol or a combination of multiple cultural symbols, and for each material of the plurality of materials, analyzing and learning the material to extract features of the material and form a set of feature vectors 102, combining all of the formed feature vectors into a tensor 103, and analyzing and learning the tensor to generate concept vectors characterizing the symbol unit, the concept vectors being directly and consistently associated with the symbol unit 104.

[0025] In one embodiment, when one or more additional materials regarding the symbol unit are later received by the machine learning model, and if the one or two materials are not found in the one or more additional materials, the machine learning model can infer one or two sets of feature vectors each corresponding to one or two materials in a picture depicting the symbol unit, a pronunciation of the symbol unit, and an image or video indicating the cultural meaning of the symbol unit.

[0026] In one embodiment, method 100 further includes combining the concept vector and one or more concept vectors arranged in a particular order to form an additional tensor, where the one or more concept vectors each characterize one or more other symbol units and are generated by the machine learning model, where the arrangement of the symbol units and the one or more other symbol units in the particular order forms a context having cultural meaning; and analyzing and learning the additional tensor to generate additional concept vectors, where the additional concept vectors characterize features of the symbol units themselves and associations of the symbol units with one or more other symbol units.

[0027] In one embodiment, the single cultural symbol is a text symbol, a mathematical symbol, a logical symbol, a trademark, a flag, or a political symbol, and the combination of multiple cultural symbols is a text term, an abbreviation, a mathematical formula, or a logical expression.

[0028] In one embodiment, the plurality of materials further includes explanations of the cultural meanings of the symbol units in the dictionary.

[0029] In one embodiment, the machine learning model is a deep learning model, a perfect autoencoder, an incomplete autoencoder, and / or a mathematical statistical model.

[0030] In one embodiment, the machine learning model is a single model for machine learning or a combination of multiple models for machine learning.

[0031] Herein, certain embodiments are described in relation to Chinese words. Although the embodiments herein are described in relation to Chinese words, it is understood that the embodiments can also be applied to other cultural symbols, such as hieroglyphics, other written symbols (e.g., English words), mathematical symbols, logical symbols, trademarks, flags, political symbols, etc. It is also understood that although certain terms are used in the embodiments, the embodiments are not limited to those specific terms, but can include all similar terms.

[0032] An exemplary process of the method according to the present disclosure for characterizing symbol units is shown in FIGS. 2 to 6. In the exemplary process, the symbol unit is a single cultural symbol, specifically a Chinese word.

[0033] As shown in FIG. 2, first, the machine learning model receives, as input, a plurality of materials related to the Chinese word "狗" which means dog. According to the present disclosure, the plurality of materials should include at least a picture in which the Chinese word is depicted, the pronunciation of the Chinese word, and an image or video showing the cultural meaning of the Chinese word. Specifically, the plurality of materials received by the machine learning model include, in this example, a picture in which "狗" is depicted, the pronunciation of the Chinese word, and an image showing a dog. In one embodiment, the plurality of materials further includes an explanation of the definition of the Chinese word "狗" in a dictionary.

[0034] Next, for each material of the plurality of materials, the machine learning model analyzes and learns the material to extract the features of the material and form a set of feature vectors. Specifically, in this example, after the analysis and learning, the machine learning model extracts the features of the picture that form the first set of feature vectors, extracts the features of the image and forms the second set of feature vectors, and extracts the features of the pronunciation and forms the third set of feature vectors, as shown in FIG. 2. In one embodiment, the lengths of the three sets of feature vectors are the same. Next, the machine learning model combines all of the formed feature vectors into a tensor.

[0035] As shown in FIG. 3, after forming the tensor, the machine learning model analyzes and learns the tensor to generate a concept vector V that characterizes the Chinese word. The concept vector is directly and consistently associated with the Chinese word, and as a result, the concept vector is quite different from other concept vectors that characterize other Chinese words. For example, in the vector space of FIG. 5, the concept vector V is quite different from the concept vector V'' corresponding to the Chinese word "猫" which means cat.

[0036] In one embodiment, the machine learning model may later receive one or more additional materials related to the Chinese word "狗", and one or more of the additional materials lack one or two of the picture drawings depicting "狗", the pronunciation of the Chinese word, and the images or videos showing dogs. For example, in FIG. 4, the machine learning model later receives only the picture depicting "狗" and does not receive the pronunciation of the Chinese word and the images showing dogs. In this embodiment, tensors may enable the machine learning model to infer one or two sets of feature vectors corresponding to the one or two missing materials respectively. For example, in FIG. 4, the tensor (formed in FIG. 2) enables the machine learning model to infer two sets of feature vectors corresponding to the pronunciation of the Chinese word and the images showing dogs respectively. In a further embodiment, as shown in FIG. 4, the machine learning model may combine all of the feature vectors (including the two inferred sets of feature vectors and the set of feature vectors formed for the picture in this exemplary process) into another tensor, and the another tensor is further used to generate another concept vector V' that characterizes the symbol unit (i.e., the Chinese word "狗" in this exemplary process). The another concept vector is also directly and consistently associated with the symbol unit, and as a result, the another concept vector will also be quite different from other concept vectors that characterize other symbol units. For example, in the vector space of FIG. 5, the another concept vector V' is quite different from the concept vector V'' corresponding to the Chinese word "猫".

[0037] In a cultural symbol system such as text, symbol units may have a context and semantic correlation with each other. That is, the arrangement of two or more symbol units in a specific order may form a cultural meaning with a context. For example, the Chinese word "狗" is combined with 8 Chinese words in a specific order to form a meaningful sentence. JPEG0007607708000001.jpg9113 can be arranged. Similarly, for eight Chinese words, the machine learning model can generate eight concept vectors that characterize each of those eight Chinese words. In this example, as shown in FIG. 6, the machine learning model combines the concept vector corresponding to "dog" and the eight concept vectors arranged in a specific order to form a further tensor, and then analyzes and learns the further tensor to generate a further concept vector, and the further concept vector characterizes the features of the Chinese word "dog" itself and its vector relationship with those eight Chinese words.

[0038] In addition, in some cultural symbol systems similar to the alphabet symbol system, the cultural symbols themselves (e.g., English letters) have no deep meaning, and the meaning of the cultural symbols is determined by the combination of cultural symbols (e.g., English words). Thus, in one embodiment, the symbol unit characterized by the machine learning model according to the present disclosure is a combination of a plurality of cultural symbols instead of a single cultural symbol, and a plurality of materials regarding the combination of a plurality of cultural symbols are received as input, and the machine learning model can also perform the above-described method according to the present disclosure to generate a concept vector that characterizes the combination of a plurality of cultural symbols.

[0039] Furthermore, a single cultural symbol can be not only a character symbol, but also a mathematical symbol, a logical symbol, a trademark, a flag, or a political symbol, and the combination of a plurality of cultural symbols can be not only a character term, but also an abbreviation, a mathematical formula, or a logical expression.

[0040] Furthermore, as described above, the machine learning model can be a deep learning model, a complete autoencoder, an incomplete autoencoder, and / or a mathematical statistics model, and can refer to a single model for machine learning or a combination of a plurality of models for machine learning.

[0041] It will be understood that a block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by computer program instructions that can be provided to a processor of a general purpose computer, special purpose computer, and / or other programmable data processing apparatus to create a machine, such that the instructions, when executed by the processor of the computer and / or other programmable data processing apparatus, create means for performing the function / actions specified in the block or blocks of the block diagrams and / or flowchart illustrations.

[0042] It will also be understood that the functions / actions of the blocks in the flowcharts may occur out of the order noted in the operational diagrams. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order depending on the functions / actions involved. Although some of the figures include communication path arrows indicating a primary direction of communication, it will be understood that communication may occur in the opposite direction to that of the arrows shown.

[0043] Furthermore, the solutions of the present disclosure may take the form of a computer program in a memory having computer usable or computer readable program code embodied in a medium for use by or in association with an instruction execution system. In the context of this document, a memory may be any medium adapted to contain, store, or convey a program for use by or in association with an instruction execution system, instruction execution apparatus, or instruction execution device.

[0044] Thus, as shown in Figure 7, the present disclosure also provides a machine 700 including a processor 701 and a memory 702. In the machine 700, the memory 702 stores instructions that, when executed by the processor 701, cause the machine 700 to perform a method according to the present disclosure as described above.

[0045] The present disclosure also provides a machine-readable medium (not shown) storing instructions that, when executed on a machine, cause the machine to perform the methods according to the present disclosure as described above.

[0046] Although this specification contains many details of specific embodiments, these should not be construed as limiting the scope of any embodiment or the scope of what may be claimed, but rather as descriptions of features that may be specific to a particular embodiment of a particular embodiment. Certain features described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may operate in a particular combination as described above and may even be initially claimed as such, one or more features from a claimed combination may in some cases be deleted from that combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0047] It is obvious to those skilled in the art. As the technology advances, the concept of the present invention can be implemented in various ways. It should be understood that the above-described embodiments are given to illustrate rather than limit the present disclosure, and modifications and variations can be used without departing from the spirit and scope of the present disclosure as easily understood by those skilled in the art. Such modifications and variations are considered to be within the scope of the present disclosure and the appended claims. The scope of protection of the present disclosure is defined by the appended claims.

Claims

1. A method for a computer to characterize cultural symbols by using a machine learning model, comprising: The computer includes: receiving a plurality of materials relating to a symbol unit, the plurality of materials including at least a picture depicting the symbol unit, a pronunciation of the symbol unit, and an image or video showing a cultural meaning of the symbol unit, the symbol unit being a single cultural symbol or a combination of multiple cultural symbols; For each material of the plurality of materials, analyzing and learning the material to extract features of the material and form a set of feature vectors; combining all of the formed feature vectors into a tensor; analyzing and learning the tensors to generate concept vectors that characterize the symbol units, the concept vectors being directly and consistently associated with the symbol units; A method for performing.

2. 2. The method of claim 1 , wherein, when one or more further materials relating to the symbol unit are subsequently received by the machine learning model and the one or two materials are not found in the one or more further materials, the machine learning model can infer one or two sets of feature vectors corresponding respectively to one or two materials in a picture depicting the symbol unit, a pronunciation of the symbol unit, and an image or video showing a cultural meaning of the symbol unit.

3. The computer, combining the concept vector and one or more concept vectors arranged in a particular order to form a further tensor, the one or more concept vectors each characterizing one or more other symbol units and generated by the machine learning model, the arrangement of the symbol units and the one or more other symbol units in the particular order forming a context having cultural meaning; analyzing and learning the additional tensors to generate additional concept vectors, the additional concept vectors characterizing the features of the symbol units themselves and associations of the symbol units with the one or more other symbol units; The method of claim 1 , further comprising:

4. The method of claim 1 , wherein the single cultural symbol is a textual symbol, a mathematical symbol, a logical symbol, a trademark, a flag, or a political symbol, and the combination of multiple cultural symbols is a textual term, an abbreviation, a mathematical formula, or a logical expression.

5. The method of claim 1 , wherein the plurality of materials further comprises an explanation of the cultural meaning of the symbol unit in a dictionary.

6. The method of claim 1 , wherein the machine learning model is a deep learning model, a perfect autoencoder, an incomplete autoencoder, and / or a mathematical statistical model.

7. The method of claim 1 , wherein the machine learning model is a single model for machine learning or a combination of multiple models for machine learning.

8. A machine-readable medium storing instructions that cause a computer to perform the method of any one of claims 1 to 7.

Images