Language image natural model architecture and device
By constructing a natural model architecture for language images, using human limb morphology and action sequences to establish the mapping between visual images and text, the low-level error problem of Transformer model in visual understanding is solved, and efficient and concise information expression and processing is achieved.
Patent Information
- Application Number
- CN202410218716.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-28
- Publication Date
- 2025-08-29
AI Technical Summary
Existing Transformer-type language models are difficult to effectively understand the internal logic of the content when processing visual image information, and are prone to low-level mistakes and are expensive. How to extract and express the information of interest from visual image video information?
A natural model architecture for language images is constructed, through analyzing the morphology, position and motion trajectory of human limbs, establishing a direct corresponding mapping relationship between visual images and text, and using the morphology and action sequences of limb parts such as arms, palms, and fingers to express specific meanings, forming a simple and efficient mapping and conversion system.
It realizes efficient, concise and flexible information mapping between visual images and text, with a wide range of applications and is suitable for a variety of scenarios, including morphological recorders and AI digital people, improving the efficiency and accuracy of information processing.
Smart Images
Figure CN120563552A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of computer information processing, artificial intelligence language information processing, and visual image and video information processing. Background Art
[0002] In the 2020s, ChatGPT language model technology evolved rapidly, demonstrating great technological potential in the field of artificial intelligence and leading to a round of explosive development of AI technology.
[0003] Multimodal AI technology aims to integrate and unify information forms such as language text, graphics, images, audio and video.
[0004] Among them, the language model plays the core role of underlying processing, on top of which visual graphics, image, video and audio information processing modules are built and connected, thus constructing a complex multimodal information processing system.
[0005] Through big data and large-scale deep learning, a wide range of parameters are obtained, enabling the system to have the intelligent ability to cope with various complex scenarios.
[0006] The entire learning and processing process has very high hardware requirements. Big data, large models, large scale, processing of massive amounts of information, scanning operations, and high-intensity computing power are required, which is costly.
[0007] The basic principle of large-scale Transformer-type language models is to scan and read a large corpus of existing human knowledge texts, establish a library of word association probability parameters, and then use calculations to obtain probability values and predictively generate word sequences. The advantage is that after training, the model can cope with various intellectual challenges. However, the potential problem is that it may not understand the underlying logic of the content, making it prone to making basic mistakes and intellectual vulnerabilities. To address these vulnerabilities, various patches are needed to improve the AI's intelligence. As the saying goes, every failure is a lesson learned. However, if these vulnerabilities are scattered, searching for them one by one is very costly.
[0008] AIGC based on Transformer can assist humans in completing certain work tasks.
[0009] Visual information contains a vast amount of information, a mix of all kinds. However, the information we are interested in and focus on only accounts for a fraction of it. How can we extract the information we are interested in and focus on from visual graphics, images, and video footage? How can we express this information? What level of abstraction can we achieve? Summary of the Invention
[0010] The purpose of this invention is to build a concise, effective and lightweight language image model architecture.
[0011] The idea of the present invention is to establish a direct corresponding mapping relationship between visual images and text elements.
[0012] Before proposing a specific plan, take a look at sign language and flag signals.
[0013] Sign language: Use finger shapes to represent letters and form words.
[0014] Flag signals involve waving arms in various positions and angles around the body, using a combination of arm positions and angles to represent letters. Just like the hour and minute hands on a clock face, their positions and angles indicate different times. In flag signals, the arms act like the hands on a clock, mapping their positions and angles to letters.
[0015] Principle of the present invention:
[0016] Make full, comprehensive and integrated use of the main limbs and the terminal small limbs, including the form, shape and positional relationship of the arms, palms, fingers, legs, feet, mouth and tongue.
[0017] Analyze and summarize the placement of the arms, simplify and abstract them into symbols with similar shapes to depict the posture of the arms.
[0018] Analyze and summarize the hand placement, including the direction of the palm, the upper and lower relationship between the palm and the wrist, and simplify and abstract it into symbols with similar shapes to depict the hand shape and posture.
[0019] The movement trajectories of the limb parts during the process of shape change are analyzed, summarized and classified, and symbolic descriptions are made based on the shape characteristics of the trajectories.
[0020] A specific sequence of posture movements has a specific function and completes a specific function. It can be fixed, programmed, and streamlined, and can be continuously optimized until it is finally finalized.
[0021] The target object of the expression of the meaning of a specific modeling action posture can be the function of the modeling posture itself, or it can refer to the interactive objects involved in the modeling.
[0022] Based on the static graphic images, the front and back modeling postures are inferred to fill in and form a complete modeling action sequence.
[0023] Limbs and organs are fundamental components of the human and animal bodies, serving as essential tools for interacting with nature. External limbs differ from internal organs in one key way: while internal organs are normally invisible and lack visual representation, external limbs are visually visible. Since they are visually visible, they possess the ability to generate visual information. The primary and fundamental functions of limbs are support, locomotion, and predation. Visually presenting information is an additional function, serving as a means of demonstration, communication, and exchange. Whether a species can fully exploit and utilize this additional information is a crucial aspect of its intellectual development and evolution. Humans have surpassed other species and evolved a highly intelligent form because they have fully grasped and mastered the inherent talents conveyed by visual information.
[0024] body, part, limb
[0025] The arms and fingers are crucial limbs in humans and related species like monkeys and apes. The relationships between the arms are complex and diverse, with varying positions and shapes. They can perform a wide variety of tasks and have evolved to the point of being able to do whatever one desires.
[0026] It is necessary to analyze and describe various placement positions and shapes in detail.
[0027] For humans to understand things in the world, they must establish an interactive relationship with them. Only when they have established a relationship with the part of themselves that they can control, and can express, reproduce, and understand it, can they truly master it.
[0028] The arms, fingers and other parts of the human body are limbs and organs that can be controlled at will. They are the most familiar and controllable resource tools.
[0029] The body shapes, postures, and movements of humans and animals are visually visible and can be transformed into expressions through copying, tracing, and depicting. This allows for symbolic expression, with symbols and movements mapped and transformed into each other. From the copied movements, symbolic texts are generated, and from the symbolic texts, the movements can be restored.
[0030] To accomplish specific tasks, humans or animals need to construct a series of subtle movement patterns and forms, and then transform these forms to achieve a specific task. Over time, a specific task and a specific sequence of movement patterns become bound together. The movement pattern sequences gradually become fixed, programmed, and process-based.
[0031] Different individuals will observe and summarize movements and gestures from different perspectives, resulting in different results. In the evolutionary process, the most simplified, abstract, and expressive solutions will gain greater recognition. For existing symbolic schemes, compatibility should be maintained as much as possible.
[0032] Analyze and examine several common life and labor scenarios: carrying a stone, carrying a child, holding a stick, and standing. Look at the details of the body postures in these basic scenarios.
[0033] First, let's look at the basic movements of the human arms. When the body is standing, the arms hang naturally and are in an idle state. When labor is required, generally the arms will be placed in front of the abdomen to interact with a certain object. The state where the arms hang naturally and are slightly extended obliquely can be depicted by T, similar to the Chinese character "个". The arms are naturally bent in front of the body, and the shape of the arms is a semi-enclosed circular posture, depicted by C, also known as the C shape. Standing, holding a stone, lifting it, and carrying it away, carefully decompose this series of actions and see that the basic postures of T shape and C shape will inevitably appear.
[0034] The action modeling decomposition of "carrying a stone" involves the following set of posture action sequences:
[0035] First, the action subject stands in front of the stone.
[0036] Depict a standing posture:
[0037] S. Depict the head;
[0038] T. Depict the arms on both sides of the thighs, similar to the Chinese character "个";
[0039] Then the arms start to change their actions:
[0040] O. Depict the arms enclosing into a circular shape (holding the stone);
[0041] Then describe the actions of the legs:
[0042] n. Depict the legs;
[0043] e. Depict the legs spreading apart to the left and right;
[0044] In this way, the action modeling sequence of carrying a stone is obtained: STone. The action postures can also be restored from this symbol sequence.
[0045] Action decomposition of carrying an object:
[0046] C. Depict the arms bending to form a semi-circular shape; <^
[0047] a. Depict the thumb extending to stabilize the object being carried;
[0048] r. Indicate the upward movement trajectory of the arms;
[0049] r. Indicate the further upward movement trajectory of the arms;
[0050] y. Depict the arms above the head;
[0051] In this way, we get the action sequence of carrying: carry. The action posture can also be restored from this symbol sequence.
[0052] For example, when it comes to the concept of a child, humans initially didn't know how to articulate it. If it were just squeaking and screaming, it wouldn't be easy to identify or understand. However, the interactive relationship adults can have with children is expressed through the fingers and arms, which inevitably involves holding the child. Humans all use the same gestures for children, innately so. The standard posture for holding a child with two arms: one arm around the back, the other supporting the hips, one hand high, the other low, so the child remains in a sitting position.
[0053] How to describe the action of "holding a child"?
[0054] The basic shape is:
[0055] The arms are lowered to form a C-shape;
[0056] The high arms form an H-shape;
[0057] Ch is combined into the posture of holding a child in both arms;
[0058] Then continue to add related action elements:
[0059] i. Describe and refer to the small body of the child being held, which can be represented by the little finger;
[0060] l. Describe the arm swinging and sliding downward;
[0061] d. Describe the shape of the hand, with the back of the hand facing forward;
[0062] Together, they form the word "Child." The four "Child" elements represent basic body movement gestures, forming a relatively complex action sequence. "i" represents the object being manipulated. Once the action sequence of "holding a child" is fixed and formalized, it can be used to refer to the child itself. The action of holding a child is highly repetitive and recognizable, so using this action sequence to represent a child is a mental model.
[0063] The above-mentioned baby-holding action can be further expanded, with the arms shaking up and down, to obtain a sequence like this:
[0064] In the word "children", "l" means the arm is sinking down, and "r" means the arm is lifting up. This way, you can hold the child and rock him / her downwards, interacting with the child through this action.
[0065] Holding a stick is the most primitive and basic movement posture of human beings. In the process of mastering the use and swinging of a stick, human wisdom is accumulated, evolved and upgraded.
[0066] A series of actions of holding a stick:
[0067] First, the acting subject maintains a standing posture.
[0068] ST is a standing posture model, where:
[0069] S depicts the head;
[0070] T depicts the arms on both sides of the thighs;
[0071] Then, it is stated that there is an object in the hand:
[0072] i depicts an object, a wildcard, represented by the little finger;
[0073] Then, the hands pose a specific model:
[0074] CK depicts the position and morphological model of the arms. One arm is bent and in front of the abdomen, forming the shape of a single arm in the C model, and the other arm extends forward and upward to form the shape of the upper single arm in the K model;
[0075] CK can also be that the two arms first form a C shape and then transform into a K shape, with the same result and effect.
[0076] Thus, the action sequence of holding a stick is obtained: STiCK.
[0077] This is the standard standing posture of holding a stick with both hands. Using the action posture model of grasping the stick to represent the item of the stick also conforms to the aforementioned thinking model.
[0078] In the early evolution of humans, it took millions of years to evolve from using hands and feet simultaneously to standing upright. How to express the action of standing upright?
[0079] Let's look at the action model of the human body standing again:
[0080] S depicts the head;
[0081] T depicts the arms on both sides of the thighs, similar to the Chinese character "个" in shape;
[0082] Further details are given about the terminal small limbs:
[0083] a. The thumb extends; [[ID=
[0088] The arms, fingers, legs, and other limbs can be switched between various poses. The process of change involves the movement of spatial positions, including up, down, left, right, inside, and outside.
[0089] The present invention uses direction vector arrows to express and guide the movement direction and movement trajectory of body parts.
[0090] Use r(Г) to represent upward movement (a simplified form of an upward arrow);
[0091] Use L to indicate downward movement (a simplified form of a down arrow);
[0092] Use O to indicate that the movement towards each other forms a circle;
[0093] For example, by combining the morphological symbols C, r(Г), L, and O, we can describe the following related action forms (paying attention to the temporal order of the actions):
[0094] The arms are bent in front of the body to form a semicircle, and then move upward, which is depicted using the Cr(Г) combination sequence;
[0095] The arms are bent in front of the body to form a semicircle, then moved downward, depicted using the CL combination sequence;
[0096] Bend your arms in front of your body to form a semicircle, then move your hands toward each other to form a closed circle, depicted using the CO combination sequence;
[0097] The arms are bent in front of the body to form a semicircle, then move upward, and then the hands move toward each other to form a closed circle, which is depicted using the Cro (CГO) combination sequence;
[0098] The arms are bent in front of the body to form a semicircle, then moved downward, and then the hands move toward each other to form a closed circle, depicted using the CLO combination sequence;
[0099] The arms are bent in front of the body to form a semicircle, and then the hands move toward each other to form a closed circle, and then move upward, depicted using the Cor(Г) combination sequence;
[0100] Bend your arms in front of your body to form a semicircle, then move your hands toward each other to form a closed circle, and then move downward, depicted using the COL combination sequence;
[0101] Advantages and effects
[0102] A language image natural model architecture and equipment device is a natural corresponding mapping conversion system mechanism architecture that constructs a natural corresponding mapping conversion system mechanism architecture between language text and visual graphic images and video images. It analyzes, summarizes and classifies the main body large limb modeling, the terminal small limb modeling, and the motion trajectory involved in the modeling transformation, and uses or expresses or understands specific meanings and intentions based on the modeling and motion trajectory sequence.
[0103] The present invention complies with the laws of nature.
[0104] Simple, efficient and flexible.
[0105] Easy to use,
[0106] Strong scalability,
[0107] Wide application,
[0108] Strong versatility.
[0109] Application scenarios of the present invention:
[0110] Morphological recorder: identifies the shape and movement of arms, hands and other limbs, and converts them into text information.
[0111] AI digital human: Based on the text, it controls its arms, hands, and body parts to express the corresponding body movements. This is then displayed on a display device.
[0112] Humanoid robot: Based on the text, it controls its arms, hands, and body parts to express the corresponding body movements through artificial limbs. BRIEF DESCRIPTION OF THE DRAWINGS
[0113] In the drawings of the specification, Figure 1 It is a system composition architecture diagram. DETAILED DESCRIPTION
[0114] The key to implementing the present invention, a language-image natural model architecture and device, is to construct complete mapping rules.
[0115] 1. The purpose of the language image natural model architecture:
[0116] Construct corresponding mapping relationships between language text symbols and various forms and gestures in visual images and videos.
[0117] Transform the shapes and movements into language text symbols, and transform the language symbols into shapes and movements, and express them in a corresponding manner;
[0118] Describe parts and locations based on basic shape features.
[0119] Describe the posture and position of the components according to their placement characteristics.
[0120] Describe the motion type based on the movement of component parts and the trajectory of the movement direction.
[0121] Comprehensively utilize the above types of descriptions or models to form a sequence of text symbols or modeling actions, constituting an expression with a specific meaning, which can be used for recording, reproduction, and communication.
[0122] 2. Use S to depict the head, especially the side head shape, and indicate above the head;
[0123] Use U to depict the torso.
[0124] 3. Make corresponding descriptions according to the shape and position of the limbs and arms:
[0125] Use T to depict the shape of the two arms. The two arms are located on the sides of the body, extend obliquely downward, and the hands are suspended outside the two sides of the thighs. There is an angle between the arms and the torso, which is basically similar to the shape of the Chinese character "个".
[0126] Use C to depict the shape of the two arms. The hands are in front of the abdomen, and the two arms are bent and curled into a semi-circular shape (not closed into a circle).
[0127] Use K to depict the shape of the two arms. The two arms are in front of the body, extending forward and upward and forward and downward respectively, one up and one down, and there is an angle between the two arms.
[0128] Use h to depict the shape of the arm. The elbow is lifted upward, and it can be flush with or higher than the abdomen.
[0129] Use W to depict the shape of the two arms. The two arms are on both sides of the body, the elbows are bent, and a single arm presents a V shape, and the two arms together present a W shape. The hands are flush with or above the shoulders.
[0130] Use O to depict the shape of the two arms. The hands are enclosed to form a closed circular shape.
[0131] Use J to depict the shape of a single arm. The arm is lifted upward and the hand is above the head.
[0132] Use y to depict the shape of the two arms. The two arms are lifted upward and extended to both sides, and one hand can be slightly higher than the other hand.
[0133] Use X to depict the shape of the two arms. The two arms are in front of the body, and the two forearms present a crossed shape.
[0134] Use Z to depict the shape of the two arms. The two arms are in front of the body, the two forearms are parallel up and down, the right arm extends to the left, and the left arm extends to the right.
[0135] The asymmetrical combination of double arms. The aforementioned CTWK y and other shapes are symmetrical shapes of double arms. When necessary, they can be simplified to single-arm shapes. One arm maintains the shape in the above shape, and the other arm is placed in another shape, including: Ch, Th, Wh, CT, TW, etc.
[0136] Use CK to draw a double-arm combination shape, with one arm placed in the shape of C shape and the other arm placed in the shape of the upper arm in K shape;
[0137] Use n to describe legs, especially thighs;
[0138] Use g to depict legs and feet, calves and feet, as well as to indicate the area under the feet (Note: the symbol g here can also be written, displayed or printed in a shape similar to 9, the same below).
[0139] Use ng to describe a combination of parts, such as legs and feet.
[0140] 4. Draw the shape of the palm and fingers accordingly:
[0141] (The front in the following text refers to the direction in which the body stands upright and looks forward, and the opposite direction to the front is the back)
[0142] Use p to depict the shape of your hand, with your palm resting on top of your wrist and facing forward.
[0143] Use q to depict the hand shape, with the palm above the wrist and the palm facing backward;
[0144] Use d to outline the hand, with the palm under the wrist and facing backwards;
[0145] Use b to draw the hand shape, with the palm under the wrist and the palm facing forward;
[0146] In pqdb, finger morphology is not limited;
[0147] Use F to depict the hand shape, with the palm in a loose grip, fingertips facing forward, thumb and other fingers separated, thumb placed under the other fingers;
[0148] Use m to depict fingers, especially the middle three fingers, with the fingertips tilted downward;
[0149] Use n to depict the two fingers other than the thumb, with the fingertips pointing downward;
[0150] Use a to depict the thumb;
[0151] Use i to depict the pinky finger;
[0152] Use V to depict the tongue, bending the shape;
[0153] Use W to depict the tongue, bending the shape;
[0154] Use C to depict the mouth, with the mouth open on the side;
[0155] Use O to depict the mouth, with the mouth open from the front;
[0156] Use O to depict circular objects;
[0157] Use l to describe line-shaped objects and things, including straight legs and arms;
[0158] Use V and W to depict waves, oscillations, and vibrations.
[0159] 5. Describe the direction of movement, position change, and movement trajectory of the part based on the trajectory shape and position height:
[0160] Use r(Г), to depict upward movement;
[0161] Use L to depict downward movement;
[0162] Use o to describe inward enclosure, backward, and left (note: it can have multiple meanings depending on the context, the same below);
[0163] Use e to draw outward separation, parabola, forward, and to the right;
[0164] Use p, symbols and corresponding hand shapes to indicate being in the upper position;
[0165] Use b, symbols and corresponding hand shapes to indicate the lower and posterior positions;
[0166] Use d, symbols and corresponding hand shapes to indicate being in the lower position;
[0167] Use m, symbol and corresponding hand shape to indicate being in the middle position;
[0168] Use F, symbol and corresponding hand shape to indicate the front position.
[0169] 6. Designate symbols or action shapes to depict additional characteristics, including strength, size, reality, etc., as well as wildcard symbols;
[0170] Use a to indicate strengthening, affirmation, confirmation, increase in intensity, amplitude and enhancement;
[0171] Use i to indicate weakening, negation, illusion, reduced intensity, reduced amplitude, and weakening;
[0172] Use i to refer to an item or thing, as a wildcard to refer to the items involved in the action;
[0173] Use n to indicate a connection form, where two or more objects are connected together or placed side by side.
[0174] 7. Comprehensively utilize various morphological modeling actions, action trajectories, modeling posture switching, additional descriptions and other elements to combine and connect to construct an action symbol sequence and form an expression with specific meaning; optimize the action symbol sequence based on the frequency of use, streamline high-frequency expressions to reduce sequence length, and balance ambiguity and efficiency factors.
[0175] Optimize and simplify processing where necessary:
[0176] The double-arm modeling is simplified to a single-arm expression;
[0177] The main body's large limbs can be expressed with the small limbs at the end when necessary: use fingers to express y, w, v, n, etc.
[0178] The limbs involved may be various animal bodies or artificially generated objects with similar limb features;
[0179] The target object of the meaning of a specific modeling gesture can be the function of the modeling gesture itself, or it can refer to the interactive objects involved in the modeling;
[0180] Based on the static graphic images, the front and back modeling postures are inferred to fill in and form a complete modeling action sequence.
[0181] In the drawings of the specification, Figure 1 It is a system composition architecture diagram.
[0182] Figure 1 The system architecture includes the following components:
[0183] ① Operation processor module, which completes the mapping operation function between symbols and action forms;
[0184] ② Storage module, which carries the rule base, knowledge base, database, and sample library;
[0185] ③Text reading module;
[0186] ④Text generation output module;
[0187] ⑤ Modeling and morphology recognition module, which collects visual information, including pictures, images, video animations, and real-world image information, identifies body parts, modeling postures, posture switching, and motion trajectories, and converts them into corresponding symbol sequences;
[0188] ⑥Graphic image reading;
[0189] ⑦ Video image reading;
[0190] ⑧Real-time image acquisition;
[0191] ⑨ Modeling posture generation module: Determine the modeling, form, and motion trajectory of the component parts based on the action instruction sequence;
[0192] ⑩ Visual display module, which controls the body parts according to the pose sequence, arranges them into corresponding poses and postures, switches the poses, and displays them on the display device;
[0193] ⑾ Posture display module, i.e., the execution module of the artificial mechanical device, controls the mechanical limb parts according to the posture sequence, arranges them into corresponding shapes and postures, and switches postures;
[0194] This language image natural model architecture device includes functional components such as a computing processor module, a memory module, a visual acquisition input module, and a modeling posture display module.
[0195] Embed computer program codes, rules, parameters, and data implemented according to the aforementioned architectural method into system devices to achieve mapping and conversion between text, images, videos, and modeling actions;
[0196] The operation processor module completes the mapping operation function between symbols and action forms;
[0197] Storage module, which carries relevant rule base, knowledge base, database, and sample library;
[0198] Text reading module, obtains text sequence;
[0199] Text generation output module, outputs text sequence;
[0200] The shape recognition module collects visual information, including pictures, images, video animations, and real-time image information from the real world, identifies body parts, shape postures, posture switching, and motion trajectories, and converts them into corresponding symbol sequences;
[0201] Modeling posture generation module: determines the modeling, form, and motion trajectory of the component parts based on the action instruction sequence;
[0202] The visual display module controls the body parts according to the pose sequence, arranges them into corresponding poses and postures, switches the poses, and displays them on the display device;
[0203] The posture display module, i.e., the execution module of the artificial mechanical device, controls the mechanical limb parts according to the posture sequence, arranges them into corresponding shapes and postures, and switches postures;
[0204] The learning and training process includes rehearsal training before actual operation, reading various preset data from the visual information acquisition equipment, and training, extracting, and optimizing processing parameters and processes.
Claims
1. A natural model architecture for language and images, constructing a corresponding mapping relationship between language text symbols and various morphological postures in visual images and videos, symbolizing the shapes and action forms with language text symbols, and morphing the language symbols into shapes and action forms for corresponding mapping expression; Its characteristics are: Depicting components and parts according to basic shape features, Depicting the postures and positions of components according to the placement shape features, Depicting the movement types according to the movement trajectories of component parts, Comprehensively using the above types of depictions or shapes to form a string of text symbols or a sequence of shape actions, constituting an expression with a specific meaning, which can be used for recording, reproduction, and communication.
2. The architecture method according to claim 1, characterized in that: Using S to depict the head, especially the side head shape, and indicating above the head; Using U to depict the torso.
3. The architecture method according to claim 1, characterized in that: Making corresponding depictions according to the shape and position of the limbs and arms: Using T to depict the shape of the two arms, with the two arms on the sides of the body, extending obliquely downward, and the hands hanging in the outer regions of both thighs, with an angle maintained between the arms and the torso, basically similar to the Chinese character "个" in shape; Using C to depict the shape of the two arms, with the hands in front of the abdomen, and the two arms bent and curled into a semi-circular shape (not closed into a circle); Using K to depict the shape of the two arms, with the two arms in front of the body, extending forward and upward and forward and downward respectively, one up and one down, and an angle maintained between the two arms; Using h to depict the shape of the arm, with the elbow lifted upward, which can be flush with or higher than the abdomen; Using W to depict the shape of the two arms, with the two arms on both sides of the body, the elbows bent, and each single arm presenting a V shape, and the two arms combined presenting a W shape, with the hands flush with or above the shoulders; Using O to depict the shape of the two arms, with the hands enclosed into a closed circular shape; Using J to depict the shape of a single arm, with the arm lifted upward and the hand passing over the head; Using y to depict the shape of the two arms, with the two arms lifted upward and extended to both sides, and one hand can be slightly higher than the other; Using X to depict the shape of the two arms, with the two arms in front of the body and the two forearms presenting a crossed shape; Using Z to depict the shape of the two arms, with the two arms in front of the body, the two forearms parallel up and down, the right arm extending to the left, and the left arm extending to the right; For the asymmetric combined shape of the two arms, the aforementioned CTWKy and other shapes are symmetric shapes of the two arms, which can be simplified to be represented by a single-arm shape when needed, with one arm maintaining the shape in the above shapes and the other arm placed in another shape, including: Ch, Th, Wh, CT, TW, etc.; Using CK to depict the combined shape of the two arms, with one arm placed in the shape in C and the other arm placed in the upper-arm shape in K; Using n to depict the two legs, especially the two thighs; Using g to depict the legs, calves and feet, and indicating under the feet (Note: Here, the symbol g can also be written, displayed or printed in a shape similar to 9, the same below). Using ng to depict the combined parts, indicating the two legs plus the feet.
4. The architecture method according to claim 1, characterized in that: Making corresponding depictions according to the shapes of the palms and fingers of the hands: (The front in the following text refers to the direction in which the body stands upright and looks forward. The opposite direction to the front is the back.) Use p to describe the shape of the hand, with the palm above the wrist and the palm facing forward; Use q to depict the hand shape, with the palm above the wrist and the palm facing backward; Use d to outline the hand, with the palm under the wrist and facing backwards; Use b to draw the hand shape, with the palm under the wrist and the palm facing forward; In pqdb, finger morphology is not limited; Use F to depict the hand shape, with the palm in a loose grip, fingertips facing forward, thumb and other fingers separated, thumb placed under the other fingers; Use m to depict fingers, especially the middle three fingers, with the fingertips tilted downward; Use n to depict the two fingers other than the thumb, with the fingertips pointing downward; Use a to depict the thumb; Use i to depict the pinky finger; Use V to depict the tongue, bending the shape; Use W to depict the tongue, bending the shape; Use C to depict the mouth, with the mouth open on the side; Use O to depict the mouth, with the mouth open from the front; Use O to depict circular objects; Use l to describe line-shaped objects and things, including straight legs and arms; Use V and W to depict waves, oscillations, and vibrations.
5. The architecture method according to claim 1, characterized in that: Describe the movement direction, position change, and movement trajectory of the part based on the trajectory shape and position height: Use r(Г), to depict upward movement; Use L to depict downward movement; Use o to describe inward enclosure, backward, and left (note: it can have multiple meanings depending on the context, the same below); Use e to draw outward separation, parabola, forward, and to the right; Use p, symbols and corresponding hand shapes to indicate being in the upper position; Use b, symbols and corresponding hand shapes to indicate the lower and posterior positions; Use d, symbols and corresponding hand shapes to indicate being in the lower position; Use m, symbol and corresponding hand shape to indicate being in the middle position; Use F, symbol and corresponding hand shape to indicate the front position.
6. The architecture method according to claim 1, characterized in that: Designated symbols or action shapes to depict additional characteristics, including strength, size, reality, etc., as well as wildcard symbols; Use a to indicate strengthening, affirmation, confirmation, increase in intensity, amplitude and enhancement; Use i to indicate weakening, negation, illusion, reduced intensity, reduced amplitude, and weakening; Use i to refer to an item or thing, as a wildcard to refer to the items involved in the action; Use n to indicate a connection form, where two or more objects are connected together or placed side by side.
7. The architecture method according to claims 1-6, characterized in that: Comprehensively utilize various morphological modeling actions, action trajectories, modeling posture switching, additional descriptions and other elements to form a combination of connection to construct an action symbol sequence and form an expression with specific meaning; according to the frequency of use, the action symbol sequence is optimized, and the high-frequency expression is streamlined to reduce the sequence length and balance the ambiguity and efficiency factors. Optimize and simplify processing where necessary: The double-arm modeling is simplified to a single-arm expression; The main body's large limbs can be expressed with the small limbs at the end when necessary: use fingers to express y, w, v, n, etc. The limbs involved may be various animal bodies or artificially generated objects with similar limb features; The target object of the meaning of a specific modeling gesture can be the function of the modeling gesture itself, or it can refer to the interactive objects involved in the modeling; Based on the static graphic images, the front and back modeling postures are inferred to fill in and form a complete modeling action sequence.
8. A language image natural model architecture device, comprising a processing module, a memory module, a visual acquisition input module, a modeling posture display module and other functional components, Its characteristics are: Embed computer program codes, rules, parameters, and data implemented according to the architecture method described in claims 1 to 7 into system devices to achieve mapping and conversion between text, images, videos, and modeling actions; The operation processor module completes the mapping operation function between symbols and action forms; Storage module, which carries relevant rule base, knowledge base, database, and sample library; Text reading module, obtains text sequence; Text generation output module, outputs text sequence; The shape recognition module collects visual information, including pictures, images, video animations, and real-time image information from the real world, identifies body parts, shape postures, posture switching, and motion trajectories, and converts them into corresponding symbol sequences; Modeling posture generation module: determines the modeling, form, and motion trajectory of the component parts based on the action instruction sequence; The visual display module controls the body parts according to the pose sequence, arranges them into corresponding poses and postures, switches the poses, and displays them on the display device; The posture display module, i.e., the execution module of the artificial mechanical device, controls the mechanical limb parts according to the posture sequence, arranges them into corresponding shapes and postures, and switches postures; The learning and training process includes rehearsal training before actual operation, reading various preset data from the visual information acquisition equipment, and training, extracting, and optimizing processing parameters and processes.