Behavior tree dynamic construction method, system, equipment and medium
By constructing an event semantic relationship graph and combining it with a large model, the behavior tree is dynamically adjusted, which solves the problem of rigid behavior in emotional robots and improves personalized interaction and empathy capabilities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- FUJIAN STAR NET WISDOM TECH CO LTD
- Filing Date
- 2025-12-10
- Publication Date
- 2026-05-05
AI Technical Summary
Existing emotional robots have rigid behavioral decision-making, lack situational understanding, and cannot make dynamic adjustments, resulting in rigid behavior, lack of personalized expression, and difficulty in forming a long-term sense of companionship and dependence with users.
By constructing an event semantic relationship graph, dynamically building a behavior tree, combining predefined rules and a large model, adjusting the behavior tree structure in real time, and optimizing event confidence based on execution results, dynamic updates and reconstruction of the behavior tree are achieved.
It improves the robot's personalized interaction capabilities, enhances its empathy with users, realizes interpretable reasoning and dynamic adaptability of behavior trees, and improves the user experience.
Smart Images
Figure CN121980144A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of emotion robot technology, and in particular to a method, system, device and medium for dynamic construction of behavior trees. Background Technology
[0002] Desktop emotional companion robots have seen rapid development in recent years in fields such as children's education and personal assistants. Their core value lies in their ability to perceive the user's state and provide companionship and assistance through natural interaction and emotional expression. Currently, the behavior of these robots is mostly controlled using predefined scripts, finite state machines, or fixed behavior trees. These methods can only modify execution conditions and cannot dynamically reconstruct the behavior tree structure, resulting in poor scalability. Event-driven behavior triggers are mostly simple mappings of events to behaviors, leading to a lack of contextual understanding, rigid behavioral decisions, and a lack of personalized expression in actual interactions, making it difficult to establish a long-term sense of companionship and dependence with users.
[0003] Therefore, there is an urgent need to construct a behavior tree that is easy to expand and enriches the personalized expressions of companion robots, in order to achieve a robot companion experience that is "warm, memorable, and personalized". Summary of the Invention
[0004] The technical problem to be solved by this invention is to provide a method, system, device and medium for dynamic construction of behavior trees, which solves the problems of rigid behavioral decision-making and lack of personalized expression in existing emotional robots.
[0005] In a first aspect, the present invention provides a method for dynamically constructing a behavior tree, comprising the following steps: Step S1: Acquire raw data from voice, image and touch sensors, normalize the raw data and perform event abstraction and semantic processing, and create a semantic event object for each event. The event object includes semantic type, participating objects, time sequence attributes, semantic attributes and event confidence. Step S2: Based on the relationship type of the event object, construct an event semantic relationship graph. The event semantic relationship graph is constructed using a hybrid approach of predefined rules and large model. Step S3: Define the mapping rules from the event semantic relationship graph to the behavior tree nodes, and convert the relationship types in the semantic relationship graph into behavior tree node types, including setting the sequence nodes corresponding to causal relationships and the condition nodes corresponding to conditional relationships; Step S4: Parse the event semantic relationship graph into a behavior tree structure according to the mapping rules, and dynamically construct the behavior tree; Step S5: Record the execution results during the behavior tree operation, and adjust the event confidence based on the execution results to optimize the dynamic construction of the behavior tree in the next round.
[0006] Furthermore, the semantic type is used to define the event type, the participating objects include the event subject, object, location and tool, the temporal attributes include time stage, duration of occurrence and scene scope, and the semantic attributes include the determination of event behavior, nature and state.
[0007] Furthermore, the predefined rules are used to handle the relationships corresponding to abnormal situation warning events, the large model is used to handle the relationships of daily interaction events, and the event semantic relationship graph contains causal relationships and conditional relationships.
[0008] Furthermore, step S4 specifically includes: The semantic relationship graph is parsed into a behavior tree structure according to the mapping rules, and the behavior tree structure is automatically combined. Based on the real-time event stream, the behavior tree nodes are added, inserted, replaced, and / or deleted to complete the dynamic construction of the behavior tree.
[0009] Secondly, the present invention provides a behavior tree dynamic construction system, the system comprising: The event acquisition and semantic representation module is used to acquire raw data from voice, image and touch sensors, normalize the raw data and perform event abstraction and semanticization processing, and create a semantic event object for each event. The event object includes semantic type, participating objects, time sequence attributes, semantic attributes and event confidence. The event semantic relationship graph construction module is used to construct an event semantic relationship graph based on the corresponding relationship type parsed from the event object. The event semantic relationship graph is constructed using a hybrid approach of predefined rules and large model. The behavior node mapping module defines the mapping rules from the event semantic relationship graph to behavior tree nodes, and converts the relationship types in the semantic relationship graph into behavior tree node types, including setting the sequence nodes corresponding to causal relationships and the condition nodes corresponding to conditional relationships. The behavior tree dynamic construction module is used to parse the event semantic relationship graph into a behavior tree structure according to the mapping rules and dynamically construct the behavior tree. The execution and feedback optimization module is used to record the execution results during the operation of the behavior tree, adjust the event confidence based on the execution results, and optimize the dynamic construction of the behavior tree in the next round.
[0010] Furthermore, the semantic type is used to define the event type, the participating objects include the event subject, object, location and tool, the temporal attributes include time stage, duration of occurrence and scene scope, and the semantic attributes include the determination of event behavior, nature and state.
[0011] Furthermore, the predefined rules are used to handle the relationships corresponding to abnormal situation warning events, the large model is used to handle the relationships of daily interaction events, and the event semantic relationship graph contains causal relationships and conditional relationships.
[0012] Furthermore, the behavior tree dynamic construction module specifically includes: The semantic relationship graph is parsed into a behavior tree structure according to the mapping rules, and the behavior tree structure is automatically combined. Based on the real-time event stream, the behavior tree nodes are added, inserted, replaced, and / or deleted to complete the dynamic construction of the behavior tree.
[0013] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect.
[0014] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.
[0015] The present invention provides one or more technical solutions, which have at least the following technical effects or advantages: realize the automatic generation of behavior tree structure from multimodal event input, support complex logic such as causality and condition through event semantic relationship modeling, solve the problem of missing semantic layer, and support dynamic update and reconstruction. When new events or relationships change, the behavior tree can be adjusted in real time to improve scalability. Attached Figure Description
[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0017] Figure 1 This is an execution flowchart of a behavior tree dynamic construction method according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of a behavior tree dynamic construction method system according to Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the electronic device in Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the structure of the medium in Embodiment 4 of the present invention. Detailed Implementation
[0018] This application provides a method, system, device, and medium for dynamically constructing behavior trees, thereby enabling the dynamic creation and optimization of behavior trees based on event semantic relationships and improving the personalized interactive experience of robots.
[0019] The overall idea of the technical solution in this application is as follows: by establishing a semantic relationship graph for abstract events, and then constructing a behavior tree in real time based on the semantic relationship graph, this method of modeling through semantic relationship graphs can enable the robot to have interpretable reasoning and decision-making capabilities, and use the feedback of execution results to adjust the semantic relationship to re-optimize the behavior tree, so that the robot gradually conforms to the user's personalized needs during the interaction with the user.
[0020] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods. Example 1
[0021] like Figure 1 As shown, this embodiment provides a method for dynamically constructing a behavior tree, including the following steps: Step S1: Acquire raw data from voice, image and touch sensors, normalize the raw data and perform event abstraction and semantic processing, and create a semantic event object for each event. The event object includes semantic type, participating objects, time sequence attributes, semantic attributes and event confidence. Step S2: Based on the relationship type of the event object, construct an event semantic relationship graph. The event semantic relationship graph is constructed using a hybrid approach of predefined rules and large model. Step S3: Define the mapping rules from the event semantic relationship graph to the behavior tree nodes, and convert the relationship types in the semantic relationship graph into behavior tree node types, including setting the sequence nodes corresponding to causal relationships and the condition nodes corresponding to conditional relationships; Step S4: Parse the event semantic relationship graph into a behavior tree structure according to the mapping rules, and dynamically construct the behavior tree; Step S5: Record the execution results during the behavior tree operation and adjust the event confidence based on the execution results to optimize the dynamic construction of the behavior tree in the next round. For example, after recording the execution results when executing the behavior tree, traverse each node. If the node has related events (e.g., in a causal relationship, when the owner's emotional state is "disappointed" (cause), under this precondition, the robot's behavior node is "play music"), then adjust the confidence of these events based on the execution results. For example, in the behavior tree, the action is to play light music under the condition that the owner's emotional state is "disappointed". When executing this behavior tree branch, if the user stops the music playback, the confidence of the robot playing music under this condition will decrease. Conversely, if the user hums along, the confidence of the robot playing music under this condition will increase. The feedback results directly reconstruct the behavior tree, so the robot will tend to choose the branch with higher confidence when this condition is triggered next time. In a preferred embodiment, when the confidence level continuously increases and reaches 0.90 or above, a rule-based conditioned reflex is formed, that is, the feedback under this condition is marked as a fast feedback chain, and the corresponding behavior is executed the next time the user is in a low mood, thereby further improving the robot's empathy ability.
[0022] Preferably, the semantic type is used to define the event type, the participating objects include the event subject, object, location and tool, the temporal attributes include time stage, duration of occurrence and scene scope, and the semantic attributes include the determination of event behavior, nature and state.
[0023] For example, in a specific scenario: the owner returns home at 8 PM, sits on the sofa, sighs, and in quiet surroundings, the phone suddenly rings. Based on this, the robot receives sound and visual information combined with the current time and can obtain the following list of instance events (in order): Event 1 (Visual): Homeowner returns home (HomeArrival) Event 2 (Visual): The owner is sitting on the sofa (SittingOnSofa) Event 3 (Audio): The owner sighs. Event 4 (Hearing): Quiet Environment Event 5 (Auditory): Phone Ring Event 6 (Time): 8 PM (Evening Time) Innovate the event object by taking the event of the owner returning home: semantic_type="HomeArrival" # Semantic type semantic_event.Confidence = 1.0 # Event confidence level # === Participant Analysis === semantic_event.agent = "home_owner" semantic_event.patient = "sofa" # The object that receives the action semantic_event.location = "living_room" semantic_event.instrument = "sofa" # Also used as a comfort tool # === Timing Attribute Analysis === semantic_event.temporal_phase =TemporalPhase.STATIONARY.value semantic_event.duration = 1800.0 # Estimated duration: 30 minutes semantic_event.spatial_scope = "furniture_interaction" # === Semantic attribute calculation === semantic_event.intensity = 0.4 # Low-intensity but important comfort behaviors semantic_event.valence = 0.3 # Slightly positive (comfort-seeking) semantic_event.arousal = 0.25 # Low arousal (relaxed state) Preferably, the abstracted events in step S1 are used to construct an event semantic relationship graph. This invention employs a hybrid approach of rules and a large model. The predefined rules are used to handle relationships corresponding to abnormal situation warning events, i.e., handling explicit relationships such as chronological order, common-sense causality, and domain knowledge. Events requiring rapid response, including system anomalies and robot conditioned reflexes, can be directly defined through the rule engine. The large model (in this embodiment, the qwen2.5vl large model) is used to handle relationships in daily interaction events. The event semantic relationship graph contains causal and conditional relationships, i.e., it handles complex and implicit relationships. The large model needs to perform a deeper understanding and reasoning to clarify the relationships between events, ensuring that abstract events contain logical semantic relationships. Understanding the internal relationships of events through a dual-engine approach helps improve the accuracy of the robot's event recognition and response behavior. For example: 1. Causal relationship: The owner sighed and sat down on the sofa. (The owner was probably in a bad mood.) It's 8 PM now and the room is very quiet -> the owner is probably relaxing. 2. Conditional Relationship: The robot can only initiate a conversation when its owner looks at it. Take causal relationships as an example: # Predefined causal rule patterns can be set as causal_patterns = { # (Cause event type, Result event type, Time window, Base confidence level, Rule description) ('HomeArrival', 'ComfortSeeking', 60, 0.8, 'Home Arrival leads to a need for rest') ("Home Arrival", "Emotional Release", 120, 0.6, "Home Arrival triggers emotional expression") ("ComfortSeeking", "EmotionalRelease", 30, 0.7, "Easily sighs during rest") ("circadianContext", "AcousticAmbiance", 300, 0.9, "The environment is quieter at night") Preferably, step S4 specifically includes: The semantic relationship graph is parsed into a behavior tree structure according to the mapping rules, and the behavior tree structure is automatically combined. Based on the real-time event stream, the behavior tree nodes are added, inserted, replaced, and / or deleted to complete the dynamic construction of the behavior tree.
[0024] The core of this invention's dynamic behavior tree construction is: updating the semantic relationship graph based on the real-time event stream, and then re-parseing the semantic relationship graph to generate the behavior tree. However, for efficiency, it is not built from scratch every time, but rather incrementally updated. Incremental updates can be implemented in the following ways: When a new event arrives, the semantic relationship graph is updated (new events and relationships are added, and old events may be deleted). Then, the behavior tree is locally modified based on the new event and the existing context.
[0025] Use a behavior tree management class to maintain the current behavior tree and provide an update interface that receives new events, updates the semantic relationship graph, and dynamically adjusts the behavior tree.
[0026] Based on the same inventive concept, this application also provides a system corresponding to the method in Embodiment 1, as detailed in Embodiment 2. Example 2
[0027] like Figure 2 As shown, this embodiment provides a dynamic behavior tree construction system, the system comprising: The event acquisition and semantic representation module is used to acquire raw data from voice, image and touch sensors, normalize the raw data and perform event abstraction and semanticization processing, and create a semantic event object for each event. The event object includes semantic type, participating objects, time sequence attributes, semantic attributes and event confidence. The event semantic relationship graph construction module is used to construct an event semantic relationship graph based on the corresponding relationship type parsed from the event object. The event semantic relationship graph is constructed using a hybrid approach of predefined rules and large model. The behavior node mapping module defines the mapping rules from the event semantic relationship graph to behavior tree nodes, and converts the relationship types in the semantic relationship graph into behavior tree node types, including setting the sequence nodes corresponding to causal relationships and the condition nodes corresponding to conditional relationships. The behavior tree dynamic construction module is used to parse the event semantic relationship graph into a behavior tree structure according to the mapping rules and dynamically construct the behavior tree. The execution and feedback optimization module records execution results during the behavior tree's operation and adjusts event confidence based on these results to optimize the dynamic construction of the behavior tree in the next round. For example, after recording the execution results, each node is traversed. If a node has related events (e.g., in a causal relationship, when the user's emotional state is "disappointed" (cause), the robot's behavior node is "play music"), the confidence of these events is adjusted based on the execution results. For instance, in the behavior tree, the action is to play light music under the condition that the user's emotional state is "disappointed." When executing this behavior tree branch, if the user stops the music, the confidence of the robot playing music under this condition will decrease; conversely, if the user hums along, the confidence of the robot playing music under this condition will increase. The feedback results directly reconstruct the behavior tree, so the robot will tend to choose the branch with higher confidence the next time this condition is triggered. In a preferred embodiment, when the confidence level continuously increases and reaches 0.90 or above, a rule-based conditioned reflex is formed, that is, the feedback under this condition is marked as a fast feedback chain, and the corresponding behavior is executed the next time the user is in a low mood, thereby further improving the robot's empathy ability.
[0028] Preferably, the semantic type is used to define the event type, the participating objects include the event subject, object, location and tool, the temporal attributes include time stage, duration of occurrence and scene scope, and the semantic attributes include the determination of event behavior, nature and state.
[0029] Preferably, the predefined rules are used to handle the relationships corresponding to abnormal situation warning events, and the large model is used to handle the relationships of daily interaction events. The event semantic relationship graph includes causal relationships and conditional relationships. This invention, by using a dual-engine approach to understand the internal relationships of events, helps improve the accuracy of the robot's event recognition and reaction behavior.
[0030] Preferably, the behavior tree dynamic construction module specifically includes: The semantic relationship graph is parsed into a behavior tree structure according to the mapping rules, and the behavior tree structure is automatically combined. Based on the real-time event stream, the behavior tree nodes are added, inserted, replaced, and / or deleted to complete the dynamic construction of the behavior tree.
[0031] Since the system described in Embodiment 2 of this invention is a system used to implement the method of Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this system based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All systems used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.
[0032] Based on the same inventive concept, this application provides an electronic device embodiment corresponding to Embodiment 1, as detailed in Embodiment 3. Example 3
[0033] This embodiment provides an electronic device, such as... Figure 3 As shown, it includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can implement any of the embodiments in Example 1.
[0034] Since the electronic device described in this embodiment is the device used to implement the method in Embodiment 1 of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in Embodiment 1 of this application. Therefore, how the electronic device implements the method in the embodiment of this application will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiment of this application falls within the scope of protection of this application.
[0035] Based on the same inventive concept, this application provides a storage medium corresponding to Embodiment 1, as detailed in Embodiment 4. Example 4
[0036] This embodiment provides a computer-readable storage medium, such as... Figure 4 As shown, a computer program is stored thereon, which, when executed by a processor, can implement any of the embodiments in Example 1.
[0037] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, devices, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0038] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, systems, apparatuses, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0039] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0040] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0041] The technical solutions provided in this application embodiment have at least the following technical effects or advantages: 1. Enhanced dynamic adaptability: Behavior trees no longer rely on fixed designs and can be built and adjusted in real time based on the semantic relationships of events; 2. Combining semantic drive: Through semantic relationship graph modeling, the robot has interpretable reasoning and decision-making capabilities.
[0042] 3. Achieve self-optimization through feedback: By adjusting the confidence level through feedback, the robot can gradually adapt to the preferences of different users, making the system execution more personalized and stable, thus improving the user experience.
[0043] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. A method for dynamically constructing behavior trees, characterized in that: Includes the following steps: Step S1: Acquire raw data from voice, image and touch sensors, normalize the raw data and perform event abstraction and semantic processing, and create a semantic event object for each event. The event object includes semantic type, participating objects, time sequence attributes, semantic attributes and event confidence. Step S2: Based on the relationship type of the event object, construct an event semantic relationship graph. The event semantic relationship graph is constructed using a hybrid approach of predefined rules and large model. Step S3: Define the mapping rules from the event semantic relationship graph to the behavior tree nodes, and convert the relationship types in the semantic relationship graph into behavior tree node types, including setting the sequence nodes corresponding to causal relationships and the condition nodes corresponding to conditional relationships; Step S4: Parse the event semantic relationship graph into a behavior tree structure according to the mapping rules, and dynamically construct the behavior tree; Step S5: Record the execution results during the behavior tree operation, and adjust the event confidence based on the execution results to optimize the dynamic construction of the behavior tree in the next round.
2. The method for dynamically constructing a behavior tree according to claim 1, characterized in that: The semantic type is used to define the event type. The participating objects include the event subject, object, location, and tool. The temporal attributes include time stage, duration of occurrence, and scene scope. The semantic attributes include the determination of the event behavior, nature, and state.
3. The method for dynamically constructing a behavior tree according to claim 1, characterized in that: The predefined rules are used to handle the relationships corresponding to abnormal situation warning events, the large model is used to handle the relationships of daily interaction events, and the event semantic relationship graph contains causal relationships and conditional relationships.
4. The method for dynamically constructing a behavior tree according to claim 1, characterized in that: Step S4 specifically includes: The semantic relationship graph is parsed into a behavior tree structure according to the mapping rules, and the behavior tree structure is automatically combined. Based on the real-time event stream, the behavior tree nodes are added, inserted, replaced, and / or deleted to complete the dynamic construction of the behavior tree.
5. A dynamic behavior tree construction system, characterized in that: The system includes: The event acquisition and semantic representation module is used to acquire raw data from voice, image and touch sensors, normalize the raw data and perform event abstraction and semanticization processing, and create a semantic event object for each event. The event object includes semantic type, participating objects, time sequence attributes, semantic attributes and event confidence. The event semantic relationship graph construction module is used to construct an event semantic relationship graph based on the corresponding relationship type parsed from the event object. The event semantic relationship graph is constructed using a hybrid approach of predefined rules and large model. The behavior node mapping module defines the mapping rules from the event semantic relationship graph to behavior tree nodes, and converts the relationship types in the semantic relationship graph into behavior tree node types, including setting the sequence nodes corresponding to causal relationships and the condition nodes corresponding to conditional relationships. The behavior tree dynamic construction module is used to parse the event semantic relationship graph into a behavior tree structure according to the mapping rules and dynamically construct the behavior tree. The execution and feedback optimization module is used to record the execution results during the operation of the behavior tree, adjust the event confidence based on the execution results, and optimize the dynamic construction of the behavior tree in the next round.
6. The behavior tree dynamic construction system according to claim 5, characterized in that: The semantic type is used to define the event type. The participating objects include the event subject, object, location, and tool. The temporal attributes include time stage, duration of occurrence, and scene scope. The semantic attributes include the determination of the event behavior, nature, and state.
7. A behavior tree dynamic construction system according to claim 5, characterized in that: The predefined rules are used to handle the relationships corresponding to abnormal situation warning events, the large model is used to handle the relationships of daily interaction events, and the event semantic relationship graph contains causal relationships and conditional relationships.
8. A behavior tree dynamic construction system according to claim 5, characterized in that: The behavior tree dynamic construction module specifically includes: The semantic relationship graph is parsed into a behavior tree structure according to the mapping rules, and the behavior tree structure is automatically combined. Based on the real-time event stream, the behavior tree nodes are added, inserted, replaced, and / or deleted to complete the dynamic construction of the behavior tree.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 4.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 4.