Graph representation of events of interactive environment

Abstract

The present principles relate to a method for evaluating cascade events in an augmented reality scene. The method includes obtaining a scene description, the scene description including a scene graph associated with a list of behaviors. The behavior can be a root behavior or can be a sub-behavior of at least one behavior. The method includes activating a root behavior. For each activated behavior, the method includes activating a root trigger of a trigger graph of the activated behavior. When the condition of the activated trigger is reached, if there is a sub-trigger of the activated trigger, the sub-trigger of the activated trigger is activated, and the currently activated trigger is deactivated. Alternatively, if there is no sub-trigger, the action of the action graph that has activated the behavior is performed. When the action is performed, if there is a sub-behavior of the activated behavior, the sub-behavior of the activated behavior is activated at its time and the activated behavior is deactivated.

Description

[0001] Cross-references to related applications This application claims the benefit of European Application No. 23305581.3, filed on 17 April 2023, the entire contents of which are incorporated herein by reference. 1. Technical Field This principle generally relates to the realm of interactivity within 3D virtual environments. It is also understood within the context of cascading events (such as triggers, actions, and behaviors) with the aim of creating a hierarchical connection graph of events between objects in a virtual scene. Specifically, this document addresses the generation of cascading events within an interactivity framework and the linking of any combination of triggers, actions, and behaviors. 2. Background Technology This section aims to introduce the reader to various aspects of this art, which may relate to the various aspects of this principle described below and / or claimed. This discussion is believed to help provide the reader with background information to facilitate a better understanding of the various aspects of this principle. Therefore, it should be understood that these statements should be read in this context and not as an admission of prior art.

[0004] Extended Reality (XR) is a technology that enables interactive experiences where real-world environments and / or video content are augmented by virtual content. This virtual content can be defined across multiple sensory modalities, including visual, auditory, and tactile. During application runtime, the virtual content (e.g., 3D content or audio / video files) is rendered in real-time in a manner consistent with the user context (environment, perspective, device, etc.). Scene graphs (such as, for example, those proposed by Khronos / glTF and their extensions defined according to the MPEG scene description format or Apple / USDZ) are possible ways to represent the content to be rendered. They combine, on the one hand, a declarative description of the scene structure (linking nodes describing virtual objects and actions on these virtual objects), and on the other hand, a binary representation of the virtual content. Binary representations of real-world objects can also be included in the nodes of the scene description. This representation of real-world objects may have been obtained, for example, by scanning the real environment. The scene description framework ensures that timing media and corresponding associated virtual content are available at any time during application rendering. Scene descriptions can also carry scene-level data describing how scene objects behave and interact at runtime for use in immersive XR experiences. Certain actions on virtual objects are triggered via proximity triggers.

[0005] Existing interactivity support at the scene or node level supports a limited set of triggers and actions. Triggers and actions are parts of behavior that define the relationships between them, control parameters for the pair, priority clauses, and options to interrupt actions. The control parameter "triggerCombinationControl" allows a set of logical operators to act on triggers (e.g., triggers / actions can be activated using logical operators "AND," "OR," and "NOT"). However, behavior cannot be tuned at a hierarchical level, such as using cascading events on behaviors, triggers, or actions. Furthermore, the ability to activate events based on the confirmation and completion of previous event actions is currently not present in current interactivity standards and is a powerful and useful property for interactivity with 3D environments. Therefore, a mechanism is lacking that enables the implementation of a graph structure at the behavior, trigger, and action levels to allow cascading events in interactive 3D scenes. 3. Summary of the Invention The following is a simplified summary of this principle to provide a basic understanding of some aspects of it. This summary is not a comprehensive overview of the principle. It is not intended to identify key or important elements of the principle. The summary below presents only some aspects of the principle in a simplified form as a prelude to the more detailed description provided below.

[0007] This principle relates to a method for evaluating cascading events in an extended reality scene. The method includes: obtaining a scene description, the scene description including a scene graph associated with a list of behavioral elements. The scene graph links nodes of virtual objects describing the extended reality scene, and behavioral elements associate a trigger element combination graph with an action element graph. A behavioral element can be a root behavioral element or a child behavioral element of at least one behavioral element. The method further includes: activating the root behavioral element. For each activated behavioral element, the method includes: activating the root trigger combination of the trigger combination graph of the activated behavioral element. When a condition of an activated trigger combination is met, if a child trigger element of the activated trigger element combination exists, the child trigger element of the activated trigger element combination is activated, and the currently activated trigger element combination is deactivated. Alternatively, if no child trigger element exists, an action of the action element graph of the activated behavioral element is executed. When an action is executed, if a child behavioral element of the activated behavioral element exists, the child behavioral element of the activated behavioral element is activated at that time, and the activated behavioral element is deactivated.

[0008] In one embodiment, a sequential mode is established, wherein a behavior element, trigger element, or action element is activated only when the preceding sibling element is deactivated. In another embodiment, a parallel mode is established, wherein each child element of the element is activated when the behavior element, trigger element, or action element is deactivated.

[0009] This principle also relates to an apparatus that includes a memory associated with a processor configured to implement the above methods. 4. Description of the attached drawings This disclosure will be better understood upon reading the following description, which refers to the accompanying drawings, wherein: - Figure 1 Figure 10 shows an exemplary description of an extended reality scene; - Figure 2 Four scenarios are shown in the dynamic scene; - Figure 3 An exemplary architecture of a device according to this principle is shown, which can be configured to implement a method for encoding or decoding point clouds; - Figure 4 An example illustrating an embodiment of the syntax of a stream when data is transmitted via a packet-based transport protocol; - Figure 5 The processing according to this principle is illustrated in the diagram. Figure 1 The behavior, triggers, and actions of cascading events in the scene shown; - Figure 6 The diagram illustrates a method for processing trigger diagrams based on this principle; - Figure 7 It is a representation of the cascading event processing model using a graph structure based on this principle; - Figure 8 The diagram illustrates the verification model to be used before the execution of the scenario description file, based on this principle. - Figure 9 The diagram illustrates a state verification model used to evaluate the state of a tiered trigger condition. 5. Detailed Implementation The principle will now be described more fully with reference to the accompanying drawings, which illustrate examples of the principle. However, the principle can be embodied in many alternative forms and should not be construed as limited to the examples set forth herein. Therefore, while the principle is readily available in various modifications and alternative forms, specific examples are shown by way of example in the drawings and will be described in detail herein. However, it should be understood that the principle is not intended to be limited to the specific forms disclosed, but rather, this disclosure should cover all modifications, equivalents, and alternatives falling within the spirit and scope of the principle as defined in the claims.

[0012] The terminology used herein is for the purpose of describing particular examples only and is not intended to limit the principles. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that, when used in this specification, the terms “comprising” and / or “including” specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof. Furthermore, when an element is referred to as “in response to” or “connected to” another element, it is capable of directly responding to or connecting to said other element, or an intermediary element may be present. In contrast, when an element is referred to as “directly responding to” or “directly connected to” another element, no intermediary element is present. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items and may be abbreviated to “ / ”.

[0013] It will be understood that while the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the teachings of this principle, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element.

[0014] Although some illustrations include arrows along the communication path to indicate the direction of the primary communication, it should be understood that communication can occur in the opposite direction to the depicted arrows.

[0015] Regarding block diagrams and operation flowcharts, some examples are described where each block represents a circuit element, module, or portion of code, including one or more executable instructions for implementing one or more specified logical functions. It should also be noted that in other implementations, the functions (one or more) marked in the blocks may not occur in the order indicated. For example, depending on the functions involved, two blocks shown successively may actually be executed substantially in parallel, or these blocks may sometimes be executed in reverse order.

[0016] In this document, references to "according to an example" or "in an example" mean that a particular feature, structure, or characteristic described in connection with that example can be included in at least one implementation of this principle. The appearance of the phrase "according to an example" or "in an example" in various places in the specification does not necessarily refer to the same example in all instances, nor is it necessarily a separate or alternative example that is mutually exclusive with other examples.

[0017] Reference numerals appearing in the claims are for illustrative purposes only and shall not have a limiting effect on the scope of the claims. Although not explicitly described, these examples and variations may be adopted in any combination or sub-combination.

[0018] Figure 1 An exemplary figure 10 illustrates an extended reality scene description. In this example, the scene graph may include: descriptions of real-world objects, such as a “flat horizontal surface” (which could be a table or a road); and descriptions of virtual objects 12, such as an animation of a car. The scene description is organized as an array 10 of nodes. Nodes can be linked to child nodes to form a scene structure 11. Nodes can carry descriptions of real-world objects (e.g., semantic descriptions) or descriptions of virtual objects. Figure 1 In the example, node 101 describes a virtual camera located within the 3D volume of an XR application. Node 102 describes a virtual car and includes an index of the car's representation, such as an index in an array of 3D meshes. This representation may have been obtained, for example, through scanning of a real environment, or may be a model generated by a content creator. This representation corresponds to the object described in this node and is therefore located within the 3D scene.

[0019] Scene descriptions may include: numerous arrays describing various aspects of the scene, such as arrays containing instructions for several animations, arrays containing meshes of several virtual or real objects, or arrays containing material descriptions. It may also include descriptions of actions applied to virtual objects when running an XR application, such as when an object controlled by a human user touches a virtual object to push or grab it, or more generally, when a virtual object collides with or approaches the boundaries of other virtual objects or the scene. Node 103 is a child node of node 102 and includes a description of one wheel of a car. In the same manner, it includes an index of the 3D mesh of that wheel. Because the scale, location, and orientation of objects are described in scene nodes, the same 3D mesh can be used for several objects in the 3D scene. Scene diagram 10 also includes nodes describing the spatial relationships between virtual objects.

[0020] Figure 2Four scenarios, A, B, C, and D, are shown for a dynamic scene. The scene includes three objects: camera 0, ball 1, and wall 2. In this exemplary scenario, ball 1 is transformed (e.g., rotated, translated, or rescaled) when it is visible from the camera, and the material of ball 1 changes when the camera is close enough (e.g., closer than 1 meter), but only when it is visible from camera 0. In scenario A, ball 1 is not visible from camera 0 because it is obscured by wall 2. In scenario B, ball 1 is visible from camera 2, but it is far from camera 2, so it is transformed but retains its material. In scenario C, the ball is visible and close to the camera, so it is transformed and its material changes. In scenario D, ball 1 is close enough to camera 0, but it is invisible, so nothing happens. This scenario can be represented using cascading events.

[0021] According to this principle, the interactivity of a 3D scene and virtual objects within it is represented by a scene graph. In this paper, the format proposed according to this principle follows the glTF format and is compatible with current MPEG efforts to extend glTF using MPEG extensions. However, the meaning and usage are general and can be encoded using any other format (e.g., XML, USD, ...). The scene graph includes: nodes, representing virtual objects and their relationships; triggers, representing conditions that activate behaviors; and actions, applied to virtual objects. Objects, triggers, actions, and behaviors are stored in arrays and specified by their indices within the arrays.

[0022] Based on this principle, the "parent" and "child" attributes are set in triggers, actions, and behaviors. These two attributes are arrays of indices of elements of the same type. Therefore, the parent attribute of a trigger is an array of indices of other triggers. Special values ​​(e.g., -1) are used to indicate that the element is the root element (i.e., it has no parent). An element cannot be its own parent or its own child. The parent and child elements of an element form a graph. Loops in this graph are not allowed to avoid infinite loops. Triggers, actions, and behaviors also include the string attribute "timestamp," which represents the application time when the element was triggered. This will protect the graph at runtime to prevent child elements from being triggered or referenced twice by their parent elements. This is an optional attribute but is important for ensuring the proper behavior of the application.

[0023] For these three types of elements (triggers, actions, behaviors), the graph continues as soon as an event is detected. Events are always initiated at the root behavior element and propagated to its child elements. An event is propagated when the parent element has initiated its action; otherwise, it waits for the action to be initiated. Child elements can be initiated in two modes: "parallel" or "sequential." In parallel mode, all child elements are run in parallel, and events are propagated to their own child elements only if their sibling elements (initiated in parallel) have initiated their actions. In sequential mode, if the array of child elements has more than one element, the "i+1" element is initiated when the previous "i" element has completed cascading events to the terminal leaf nodes. Graph processing ends when all terminal leaf elements have completed their action events.

[0024] The following table summarizes the semantics of parent and child attributes, where "O" indicates "optional".

[0025] Figure 5 The processing according to this principle is illustrated in the diagram. Figure 1 The diagram illustrates the behavior, triggers, and actions of a cascading event in the scenario. Multiple instantiations are possible; this example is provided for illustrative purposes. The array of behaviors includes three root behaviors B0, B1, and B3, and one child behavior B2, which references two parent nodes, B0 and B1. The array of triggers includes two root triggers T1 and T2, and one child trigger T0, which references T2 as its parent trigger. The array of actions includes two actions A0 and A2, and one child action A1, which references A0 as its parent action. When T1 is triggered, behavior B0 is activated and action A1 is initiated, and behavior B2 begins evaluating the trigger condition (it waits for T0 to be triggered). When T0 is triggered, B2 is activated and action A2 is initiated.

[0026] Behavior B3 contains a trigger graph. In behavior B3, action A1 is initiated only if the condition of trigger T2 is met, and then the condition of trigger T0 is met. In the case of behavior B1, actions A0 and A1 are initiated when T1 is triggered, because action A0 has a sub-action A1 and the actions are not time-dependent. Although a delay between their initiation is possible, in this example, they are initiated simultaneously. For understanding, the explanation of this example is simplified. The table below presents a representation of the scene in glTF format. Figure 6 The diagram illustrates method 60 for processing a trigger graph according to this principle. This diagram is an exemplary illustration of how a graph can iterate over child nodes when child nodes exist. In this example, if a trigger has "n" child triggers, the graph will wait for all trigger conditions to be met before initiating associated actions. Actions are defined at the behavior level, and actions are initiated once all triggers have been activated. In step 61, the i-th trigger is evaluated. If, in step 62, the trigger has not validated its condition, the process continues to the next scene update and evaluates the same trigger until it is activated or the application ends. Once the trigger is activated, in step 63, it continues to its next child trigger (if a next child trigger exists) and executes the same logic recursively until all child triggers have been activated / conditions validated, thus initiating all actions associated with this hierarchical system of triggers in step 64.

[0028] Figure 7 This is a representation of the cascading event processing model using a graph structure based on this principle. This process represents the execution in the sequential mode, where each child element completes all its dependencies before moving to the next child element. In the parallel mode, each trigger runs in parallel, and therefore their dependencies wait until all events have been triggered and actions have been initiated before moving to the next behavior's child node. If the root behavior has more than one child behavior, it is also initiated in parallel, and therefore its triggers are evaluated. Figure 7 The graph processing model describes behavior as follows: Behaviors define triggers and actions, and each trigger and action can have its own graph hierarchy. If this is the case, the actions associated with a behavior will be initiated only if the graph conditions of the primary trigger and its individual triggers are valid. Before initiating the primary action and dependent actions, the processing model forces a state wait until all triggers are active.

[0029] Figure 8 The diagram illustrates the verification model used prior to the execution of the scene description file according to this principle. Given the complexity of the graph model and graph representation according to this principle, invalid graphs may exist; for example, leaf nodes will not be activated because previous trigger conditions have already been met. This can occur due to the reuse of triggers between behaviors, resulting in the behavior graph never being executed and becoming redundant and complex, making it difficult to read according to the description file format. For this reason, a verification processing model for verifying the correctness of the interactive graph structure is proposed. This model tests whether the behavior graph is valid and whether its processing has no dead leaves (e.g., unreachable areas in the graph). This model is a simple binary model that returns "false" for invalid graphs or "true" for valid graphs.

[0030] Figure 9 The diagram illustrates a state verification model for evaluating the state of hierarchical trigger conditions. Given the complex nature of graphs and interactivity, this model is proposed to verify whether the parent behavior condition remains valid when the current scene is updated. This enforces conditional cascading events, which may be necessary for richer interactive scenes, but are not mandatory for the normal behavior of the graph. For each triggered activation, the behavior's state and trigger ID are cached in memory, and evaluation is performed to check the current state of the original condition. If the condition is no longer satisfied, the model returns the behavior and trigger IDs to force the application to return to the previously evaluated state.

[0031] Figure 3 An exemplary architecture of an XR processing engine 30 is shown, which can be configured to implement this principle. According to Figures 6 to 9 The device in the architecture is linked to other devices via its bus 31 and / or via I / O interface 36.

[0032] Device 30 includes the following components linked together by data and address buses 31:D. - Processor 32 (or CPU), which is, for example, a DSP (or digital signal processor); - ROM (or Read-Only Memory) 33; - RAM (or random access memory) 34; - Storage interface 35; - I / O interface 36, used to receive data to be transmitted from the application; and - Power supply (not indicated) Figure 2 (In the middle), for example, batteries.

[0033] According to the example, the power supply is located externally to the device. In each of the mentioned memories, the term "register" as used in this specification may correspond to a small area (a few bits) or a very large area (e.g., the entire program or a large amount of received or decoded data). ROM 33 includes at least a program and parameters. ROM 33 may store algorithms and instructions to execute techniques according to these principles. Upon power-on, CPU 32 uploads the program to RAM and executes the corresponding instructions.

[0034] RAM 34 includes a program executed by CPU 32 and uploaded after device 30 is powered on, input data, intermediate data at different states of the method, and other variables for the execution of the method.

[0035] The implementations described herein can be implemented in, for example, methods or processes, devices, computer program products, data streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., only as a method or device), the features discussed can be implemented in other forms (e.g., programs). Devices can be implemented in, for example, suitable hardware, software, and firmware. Methods can be implemented in, for example, devices (such as, for example, processors), where processor generally refers to processing means, including, for example, computers, microprocessors, integrated circuits, or programmable logic devices. Processors also include communication means, such as, for example, computers, cellular phones, portable / personal digital assistants (“PDAs”), and other means that facilitate communication of information between end users.

[0036] Device 30 is linked, for example, via bus 31 to a set of sensors 37 and a set of rendering devices 38. Sensors 37 may be, for example, a camera, microphone, temperature sensor, inertial measurement unit, GPS, humidity sensor, IR or UV light sensor, or wind sensor. Rendering devices 38 may be, for example, a display, speaker, vibrator, heater, fan, etc.

[0037] According to the example, device 30 is configured to implement the method according to this principle and belongs to the set including the following: - Mobile devices; - communication devices; - Gaming devices; - Tablet (or tablet computer); - Laptop computers; - Still image camera; - Video camera.

[0038] Figure 4 An example of an embodiment of the syntax for encoding a data stream that encodes an extended reality scene description according to this principle is shown. Figure 3 An exemplary structure 4 for an XR scene description is shown. This structure exists within a container that organizes the stream into individual syntax elements. The structure may include a header portion 41, which is a common set of data for each syntax element in the stream. For example, the header portion includes some metadata about the syntax elements, describing the properties and function of each syntax element. The structure also includes a payload, which includes syntax elements 42 and 43. Syntax element 42 includes data representing media content items described in nodes of the scene graph associated with the virtual elements. Images, meshes, and other raw data may have been compressed according to a compression method. Syntax element 43 is part of the payload of the data stream and includes data encoded as described in accordance with this principle.

[0039] The implementations described herein can be implemented in, for example, methods or processes, devices, computer program products, data streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., only as a method or device), the features discussed can be implemented in other forms (e.g., programs). Devices can be implemented in, for example, suitable hardware, software, and firmware. Methods can be implemented in, for example, devices (such as, for example, processors), where processor generally refers to processing devices, including, for example, computers, microprocessors, integrated circuits, or programmable logic devices. Processors also include communication devices, such as, for example, smartphones, tablet computers, computers, mobile phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate communication of information between end users.

[0040] The implementation of the various processes and features described herein can be embodied in a wide variety of equipment or applications (particularly, equipment or applications associated with data encoding, data decoding, view generation, texture processing, and other processing of image and associated texture and / or depth information). Examples of such equipment include encoders, decoders, post-processors that process the output from the decoder, pre-processors that provide input to the encoder, video encoders, video decoders, video codecs, web servers, set-top boxes, laptop computers, personal computers, cellular phones, PDAs, and other communication devices. It should be understood that such equipment can be mobile and may even be installed in mobile vehicles.

[0041] Additionally, the method can be implemented by executing instructions from a processor, and these instructions (and / or data values ​​generated by the implementation) can be stored on a processor-readable medium, such as, for example, an integrated circuit, a software carrier, or other storage device, such as, for example, a hard disk, a compact disc (“CD”), an optical disc (such as, for example, a DVD, often referred to as a digital universal disc or digital video disc), random access memory (“RAM”), or read-only memory (“ROM”). Instructions can form an application tangibly embodied on the processor-readable medium. Instructions can, for example, reside in hardware, firmware, software, or a combination thereof. Instructions can exist, for example, in an operating system, a separate application, or a combination of both. A processor can therefore be characterized as, for example, a means configured to execute a process and a means comprising a processor-readable medium (such as a storage device) having instructions for executing the process. Furthermore, in addition to or instead of instructions, the processor-readable medium can store data values ​​generated by the implementation.

[0042] It will be apparent to those skilled in the art that implementations can generate various signals, which are formatted to carry information, such as information that can be stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal may be formatted to carry as data rules the syntax for writing or reading the described embodiments or as data actual syntax values ​​written by the described embodiments. For example, such a signal may be formatted as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave using the encoded data stream. The information carried by the signal may be, for example, analog or digital information. It is well known that signals can be transmitted via various wired or wireless links. Signals may be stored on a processor-readable medium.

[0043] Many implementations have been described. However, it will be understood that various modifications can be made. For example, elements of different implementations can be combined, supplemented, modified, or removed to produce other implementations. Furthermore, those skilled in the art will understand that other structures and processes can replace the disclosed structures and processes, and the resulting implementations will perform at least substantially the same functions in at least substantially the same manner (one or more) to achieve at least substantially the same results as the disclosed implementations (one or more). Therefore, this application contemplates these and other implementations.

Claims

1. A method for managing behavioral elements in an extended reality scenario, the method comprising: - Obtain a scene description, which includes a scene graph associated with a list of behavior elements, wherein the scene graph links nodes of virtual objects describing the extended reality scene, and wherein the behavior elements associate one or more triggers with an action graph, wherein the behavior element is a root behavior element or a child behavior element of at least one different behavior element. - Activate the root behavior element; - For each activated behavior element, activate the one or more triggers of the activated behavior element; - When the condition of the one or more activated triggers is met, if the one or more triggers of the sub-behavioral elements of the activated behavioral element exist, then the one or more triggers of the sub-behavioral elements of the activated behavioral element are activated and the activated triggers are deactivated; or if the one or more triggers of the sub-behavioral elements of the activated behavioral element do not exist, then the action of the action element graph of the activated behavioral element is executed. and - When the action is executed, if there are sub-behavior elements of the activated behavior element, then the sub-behavior elements of the activated behavior element are activated, and the activated behavior is deactivated.

2. The method of claim 1, wherein a sequence mode is established, and wherein a behavior element, trigger element, or action element is activated only when the preceding sibling element is deactivated.

3. The method of claim 1, wherein a parallel mode is established, and wherein each sub-element of the element is activated when the behavior element, trigger element, or action element is deactivated.

4. The method of any one of claims 1 to 3, wherein the one or more triggers are defined as a trigger combination graph.

5. An apparatus for managing behavioral elements in an extended reality scene, the apparatus comprising a processor associated with a memory, the processor being configured to: - Obtain a scene description, which includes a scene graph associated with a list of behavior elements, wherein the scene graph links nodes of virtual objects describing the extended reality scene, and wherein the behavior elements associate one or more triggers with an action graph, wherein the behavior element is a root behavior element or a child behavior element of at least one different behavior element. - Activate the root behavior element; - For each activated behavior element, activate the one or more triggers of the activated behavior element; - When the condition of the one or more activated triggers is met, if the one or more triggers of the sub-behavioral elements of the activated behavioral element exist, then the one or more triggers of the sub-behavioral elements of the activated behavioral element are activated and the activated triggers are deactivated; or if the one or more triggers of the sub-behavioral elements of the activated behavioral element do not exist, then the action of the action element graph of the activated behavioral element is executed. and - When the action is executed, if there are sub-behavior elements of the activated behavior element, then the sub-behavior elements of the activated behavior element are activated, and the activated behavior is deactivated.

6. The apparatus of claim 5, wherein a sequence mode is established, and wherein a behavior element, trigger element, or action element is activated only when the preceding sibling element is deactivated.

7. The apparatus of claim 5, wherein a parallel mode is established, and wherein each sub-element of the element is activated when the behavior element, trigger element, or action element is deactivated.

8. The apparatus of any one of claims 5 to 7, wherein the one or more triggers are defined as a trigger combination diagram.

Images