Contact point dynamic prediction and optimization method, system, equipment and medium
Dynamically updating the 3D value map through multimodal information fusion and feedback mechanisms solves the problem of difficulty in selecting contact points in complex environments in traditional robot operating systems, significantly improving operational efficiency and success rate.
Patent Information
- Application Number
- CN202411838699.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-13
- Publication Date
- 2025-04-25
AI Technical Summary
Traditional robot operating systems are difficult to effectively select contact points in complex and changeable environments, and lack multimodal information fusion and feedback mechanisms, resulting in low operating efficiency and success rate.
By obtaining task text instructions and visual information, extracting text and visual features, performing multimodal alignment, giving 3D point cloud priority prompts, building a 3D value map, selecting the best contact points, and dynamically updating the 3D value map through feedback mechanisms.
It significantly improves the accuracy of contact point selection and the success rate of robot operation, enhances robustness and adaptability in complex environments, and solves the problem of lack of effective feedback and multimodal information fusion in traditional methods.
Smart Images
Figure CN119888245A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of robot operation, and in particular to a contact point dynamic prediction and optimization method, system, equipment and medium. Background Art
[0002] With the continuous development and deepening application of technologies such as big data, machine learning, and artificial intelligence, the ability of robots to operate autonomously in complex environments has received widespread attention. Traditional robot operating systems usually rely on pre-programmed rules or make decisions based on sensor data from a single modality (such as vision, touch, or force). This approach performs well in structured and predictable environments, but often has difficulty coping with complex and changing real-world environments.
[0003] In practical applications, robots need to perform various tasks in dynamic and unknown environments, such as grasping, assembling, and manipulating objects of different shapes and materials. However, due to the lack of a comprehensive understanding of the environment and tasks, robots often face the following challenges in the process of object manipulation: 1) Difficulty in contact point selection: Robots need to determine suitable contact points to successfully grasp or manipulate objects. Due to the different shapes, sizes, and materials of objects, and the possible existence of occlusions or complex geometric structures, traditional methods are difficult to accurately predict the best contact points; 2) Lack of multimodal information fusion: Single visual or force information cannot fully describe the characteristics of the environment and objects. The lack of effective fusion of multimodal information makes robots limited in understanding task requirements and environmental details; 3) Imperfect feedback mechanism: When the operation fails, traditional robot systems often lack effective feedback mechanisms to analyze the cause of failure and adjust subsequent operation strategies, which leads to low success rate and low efficiency of the operation and cannot meet the needs of practical applications; 4) Poor environmental adaptability: In a dynamically changing environment, robots need to have adaptive capabilities to cope with unknown situations and changes. Traditional methods often cannot update and optimize operation strategies in real time, which limits the environmental adaptability of robots.
[0004] In recent years, with the rise of deep learning and multimodal fusion technology, researchers have begun to try to combine visual, text and other sensor data to enhance the robot's environmental perception and decision-making capabilities. For example, using deep neural networks for image recognition and target detection, or using language models to understand and execute natural language instructions. However, these methods still have some shortcomings, such as a. Model training relies on a large amount of data: the training of deep learning models usually requires a large amount of labeled data, which is expensive to obtain and label, and it is difficult to cover all possible scenarios and objects in practical applications; b. Lack of real-time performance: complex deep learning models are difficult to achieve real-time decision-making and feedback on robot platforms with limited computing resources, affecting operational efficiency; c. Lack of effective feedback utilization: Even if some systems have feedback mechanisms, they often cannot effectively incorporate failure information into the model to guide subsequent operational optimization.
[0005] Therefore, there is an urgent need for a new approach that can effectively integrate multimodal information and predict and optimize contact point selection in real time, as well as perform continuous optimization through feedback mechanisms after operational failures. Summary of the invention
[0006] The purpose of the present invention is to provide a contact point dynamic prediction and optimization method, system, device and medium in order to overcome the defects of the above-mentioned prior art.
[0007] The purpose of the present invention can be achieved by the following technical solutions:
[0008] According to a first aspect of the present invention, a contact point dynamic prediction and optimization method is provided, the method comprising:
[0009] Acquire task text instructions and visual information and depth information of the task area image, extract text features based on the task text instructions, extract visual features based on the visual information, acquire a depth map based on the depth information and convert the depth map into a 3D point cloud;
[0010] Performing multimodal alignment on the text features and the visual features, and assigning a priority prompt to each 3D point in the 3D point cloud based on the multimodal alignment result;
[0011] Constructing a 3D value map based on the 3D point cloud and initializing the 3D value map based on the priority prompts;
[0012] Generate a visual mask using the task area image and the multimodal alignment result, and reprocess the initialized 3D value map based on the visual mask;
[0013] The current best contact point is selected based on the reprocessed 3D value map, and the execution status of the current best contact point is obtained. If the execution status is successful, the current best contact point is stored in the experience data set and the next best contact point is selected; if the execution status of the current best contact point is failed, feedback information is generated to optimize the 3D value map.
[0014] As a preferred technical solution, the method for obtaining the 3D point cloud is: converting each pixel point in the depth map into a three-dimensional coordinate point in a three-dimensional coordinate system according to a preset rule, thereby generating a 3D point cloud.
[0015] As a preferred technical solution, the method used for the multimodal alignment is a cross-modal attention mechanism or a feature fusion layer.
[0016] As a preferred technical solution, the method for constructing a 3D value map and initializing the 3D value map based on the priority prompt is:
[0017] Defining a three-dimensional grid or voxel grid based on the 3D point cloud;
[0018] Dividing the three-dimensional grid or voxel grid into a plurality of units, each unit corresponding to a 3D point;
[0019] The 3D points with high priority are assigned high value scores, and the 3D points with low priority are assigned low value scores.
[0020] As a preferred technical solution, the method for generating a visual mask is: using the task area image in combination with the multimodal alignment result, setting the pixel value of the inoperable area to a specific value, and setting the pixel value of the operable area to another value, to form a visual mask image with the same specifications as the task area image.
[0021] As a preferred technical solution, the reprocessing steps are:
[0022] Converting the task area image into a three-dimensional image and mapping it to the visual mask image;
[0023] Extracting 3D points in the three-dimensional image that are mapped to a specific pixel value area in the visual mask image as mask points;
[0024] Reduces the value score of the cell corresponding to the masked point in the 3D value map.
[0025] As a preferred technical solution, the method for optimizing the 3D value map is:
[0026] Obtain a scene image of the current best contact point, and determine whether the current best contact point is located in an inoperable area;
[0027] Generate feedback text according to the judgment result, and extract feedback text features of the feedback text, wherein when the judgment result is that the current best contact point is located in the inoperable area, the generated feedback text is "the contact point is located in the inoperable area"; if the judgment result is that the current best contact point is not located in the inoperable area, the generated feedback text is "the contact point is in the operable area but the operation fails";
[0028] Performing multimodal alignment on the feedback text feature and the visual feature to obtain a multimodal alignment result;
[0029] New priority cues and new visual masks are regenerated based on the multimodal alignment result, and the 3D value map is optimized using the new priority cues and new visual masks.
[0030] According to a second aspect of the present invention, a contact point dynamic prediction and optimization system is provided, and the system is used in the above method, comprising:
[0031] Data acquisition module: used to collect task text instructions from users or upper-level tasks, as well as images of task areas and their corresponding visual information and depth information;
[0032] Feature extraction and fusion module: including a visual feature extraction unit, a text feature extraction unit and a feature fusion unit, wherein the visual feature extraction unit is used to extract visual features of visual information collected by the data collection module, the text feature extraction unit is used to extract text features of task text instructions of users or upper-level tasks collected by the data collection module, and the feature fusion unit is used to perform multimodal alignment of text features and visual features;
[0033] 3D value map module: used to perform all operations related to the 3D value map based on the processing results of the feature extraction and fusion module, including: 3D value map construction, initialization, reprocessing and optimization;
[0034] Controller: used to select the best contact point based on the 3D value map, control the robot execution, and output the execution status;
[0035] Feedback module: used to receive executions with failed status, analyze the reasons for failure, generate feedback text based on the reasons for failure, and transmit the feedback text to the feature extraction and fusion module.
[0036] According to a third aspect of the present invention, there is provided an electronic device for dynamic prediction and optimization of contact points, comprising a memory and a processor, wherein a computer program is stored in the memory, and the method described is implemented when the processor executes the program.
[0037] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the program implements the method described when executed by a processor.
[0038] Compared with the prior art, the present invention has the following advantages:
[0039] 1) The present invention uses depth information to construct a 3D value map, performs multimodal alignment of text features and visual features, assigns an operational value to each 3D point based on the multimodal alignment result to visualize the priority of each contactable area of the object, and evaluates and dynamically updates the 3D value map in real time based on the execution status of the contact point. Compared with the traditional method that only uses 2D information or fixed rules to select contact points, which leads to insufficient operability and adaptability, the present invention uses the 3D value map to evaluate and dynamically update the operable area in real time, significantly improves the selection accuracy of the best contact point, and also ensures the flexibility of the robot in performing tasks;
[0040] 2) The present invention dynamically updates the 3D value map through an adaptive feedback mechanism to achieve multiple rounds of closed-loop optimization, which enables the system to continuously optimize and improve in continuous attempts, reduces the probability of contact point selection failure, and enhances robustness and adaptability in a changeable and complex environment. This solves the problem in the prior art that the operation strategy is fixed and difficult to adapt to dynamically changing scenarios due to the lack of adaptive feedback updates;
[0041] 3) This application has positive experience accumulation, records and strengthens the best contact points and related feature data, enables the system to select effective contact points more quickly in similar situations, solves the problem of difficulty in quickly repeating successful patterns in similar tasks, and effectively improves operational efficiency and success rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 The present invention is a flow chart of the method. DETAILED DESCRIPTION
[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0044] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.
[0045] Example 1
[0046] This embodiment provides a contact point dynamic prediction and optimization method, which uses pre-trained visual encoders and language models to achieve efficient fusion of visual and text information; through the extraction of feedback information and multiple rounds of optimization, the 3D value map is dynamically updated to improve the accuracy of contact point prediction and the success rate of operation. This method has important research significance and broad application prospects, and can be applied to fields that require high precision and high adaptability, such as robot operation and automated assembly.
[0047] Specifically, the process of this method is as follows Figure 1 As shown, the following steps are included:
[0048] S1. Feature extraction:
[0049] S11. Use a camera to obtain an image of the task area and obtain visual information and depth information, and obtain text instructions from the user or the upper-level task.
[0050] S12. Use the pre-trained CLIP model as a visual encoder, input visual information to extract visual features, and use them to characterize the appearance and structure of the object.
[0051] S13, the language model LLaMA receives the text instruction and performs reverse encoding to generate text features.
[0052] S14. Obtain a depth map based on the depth information, and convert each pixel in the depth map into a coordinate point in a three-dimensional coordinate system in combination with the camera internal parameters to generate a 3D point cloud.
[0053] S2. Multimodal alignment and priority prompt generation:
[0054] S21. Perform multimodal alignment between text features and visual features using a multimodal alignment mechanism, such as a cross-modal attention mechanism or a feature fusion layer.
[0055] S22. Assign a priority hint to each 3D point in the 3D point cloud based on the multimodal alignment result.
[0056] S3, 3D value map construction and initialization:
[0057] S31. defining a three-dimensional grid or voxel grid based on the 3D point cloud;
[0058] S32, dividing the three-dimensional grid or voxel grid into a plurality of units, each unit corresponding to a 3D point;
[0059] S33. The language model LLaMA autonomously decides to assign high value scores to 3D points with high priority (i.e., points more suitable for operation) and assign low value scores to 3D points with low priority.
[0060] S4. Visual mask generation and 3D value map reprocessing:
[0061] S41. Visual mask generation:
[0062] Using the pre-trained visual cue model LLaVA, the task area image combined with the multimodal alignment result is taken as input, and a visual mask image with the same specifications as the task area image is output. Specifically, the pixel value of the inoperable area is set to a specific value, such as 1, or the color of the inoperable area is set to white; the pixel value of the operable area is set to another value, such as 0, or the color of the operable area is set to black.
[0063] S42, 3D value map reprocessing:
[0064] S421, using the visual information and depth information to obtain the pixel (x, y) value and depth value z of the task area image, calculating the corresponding three-dimensional coordinates through the projection matrix, converting it into a three-dimensional image, and mapping it to the visual mask image.
[0065] S422, extracting 3D points in the three-dimensional image that are mapped to a specific pixel value area in the visual mask image as mask points.
[0066] S423. Reduce the value scores of the cells corresponding to the mask points in the 3D value map. By reducing the value scores of these areas, it is ensured that the robot automatically avoids these high-risk areas when selecting contact points, thereby reducing the possibility of operation failure or damage.
[0067] S5, optimal contact point selection and 3D value map optimization:
[0068] S51. Using the language model LLaMA, based on the reprocessed 3D value map, select the 3D point with the highest current value score as the best contact point, avoid low-value inoperable areas, and obtain the execution status of the current best contact point.
[0069] S52: Determine whether the execution status is successful. If successful, execute step S521; otherwise, execute step S522.
[0070] S521. Store the current best contact point in an experience data set and continue to select the next best contact point. The experience data set can be used for subsequent training or fine-tuning of the model to enable it to find the best contact point more quickly in similar scenarios.
[0071] S522, generating feedback information to optimize the 3D value map, the detailed steps include:
[0072] a1. Obtain a scene image of the current best contact point and determine whether the current best contact point is located in an inoperable area.
[0073] a2. Generate feedback text based on the judgment result, and extract feedback text features of the feedback text, wherein when the judgment result is that the current best contact point is located in the inoperable area, the generated feedback text is "the contact point is located in the inoperable area"; if the judgment result is that the current best contact point is not located in the inoperable area, the generated feedback text is "the contact point is in the operable area but the operation fails".
[0074] a3. Perform multimodal alignment on the feedback text features and visual features to obtain the multimodal alignment results.
[0075] a4. Regenerate new priority prompts and new visual masks based on the multimodal alignment results, and use the new priority prompts and new visual masks to adaptively optimize the 3D value map to improve the efficiency of selecting suitable contact points in similar situations.
[0076] By continuously updating the 3D value map in real time, deviations and incorrect priority allocations can be corrected in a timely manner based on failure feedback, and the accuracy of touchpoint selection can be continuously improved through multiple rounds of iterations.
[0077] Example 2
[0078] The above is an introduction to the method embodiment. The following is a further explanation of the solution of the present invention through a system embodiment.
[0079] This embodiment provides a contact point dynamic prediction and optimization system, which is used to implement the method provided in the above embodiment, including:
[0080] Data acquisition module: used to collect task text instructions from users or upper-level tasks, as well as images of task areas and their corresponding visual information and depth information;
[0081] Feature extraction and fusion module: including a visual feature extraction unit, a text feature extraction unit and a feature fusion unit, wherein the visual feature extraction unit is used to extract visual features of visual information collected by the data collection module, the text feature extraction unit is used to extract text features of task text instructions of users or upper-level tasks collected by the data collection module, and the feature fusion unit is used to perform multimodal alignment of text features and visual features;
[0082] 3D value map module: used to perform all operations related to the 3D value map based on the processing results of the feature extraction and fusion module, including: 3D value map construction, initialization, reprocessing and optimization;
[0083] Controller: used to select the best contact point based on the 3D value map, control the robot execution, and output the execution status;
[0084] Feedback module: used to receive executions with failed status, analyze the reasons for failure, generate feedback text based on the reasons for failure, and transmit the feedback text to the feature extraction and fusion module.
[0085] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0086] The present embodiment also provides an electronic device, which includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.
[0087] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.
[0088] The processing unit performs the various methods and processes described above, such as methods S1 to S5 and methods a1 to a4. For example, in some embodiments, methods S1 to S5 and methods a1 to a4 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S1 to S5 and methods a1 to a4 described above may be executed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S5 and methods a1 to a4 by any other appropriate means (e.g., by means of firmware).
[0089] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
[0090] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.
[0091] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0092] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A contact point dynamic prediction and optimization method, characterized in that: The method includes: Acquire task text instructions and visual information and depth information of the task area image, extract text features based on the task text instructions, extract visual features based on the visual information, acquire a depth map based on the depth information and convert the depth map into a 3D point cloud; Performing multimodal alignment on the text features and the visual features, and assigning a priority prompt to each 3D point in the 3D point cloud based on the multimodal alignment result; Constructing a 3D value map based on the 3D point cloud and initializing the 3D value map based on the priority prompts; Generate a visual mask using the task area image and the multimodal alignment result, and reprocess the initialized 3D value map based on the visual mask; The current best contact point is selected based on the reprocessed 3D value map, and the execution status of the current best contact point is obtained. If the execution status is successful, the current best contact point is stored in the experience data set and the next best contact point is selected; if the execution status of the current best contact point is failed, feedback information is generated to optimize the 3D value map.
2. A contact point dynamic prediction and optimization method according to claim 1, characterized in that: The method for obtaining the 3D point cloud is: converting each pixel point in the depth map into a three-dimensional coordinate point in a three-dimensional coordinate system according to a preset rule, thereby generating a 3D point cloud.
3. A contact point dynamic prediction and optimization method according to claim 1, characterized in that: The method used for the multimodal alignment is a cross-modal attention mechanism or a feature fusion layer.
4. A contact point dynamic prediction and optimization method according to claim 1, characterized in that: The method for constructing a 3D value map and initializing the 3D value map based on the priority prompt is: Defining a three-dimensional grid or voxel grid based on the 3D point cloud; Dividing the three-dimensional grid or voxel grid into a plurality of units, each unit corresponding to a 3D point; The 3D points with high priority are assigned high value scores, and the 3D points with low priority are assigned low value scores.
5. A contact point dynamic prediction and optimization method according to claim 1, characterized in that: The method for generating a visual mask is: using the task area image in combination with the multimodal alignment result, setting the pixel value of the inoperable area to a specific value, and setting the pixel value of the operable area to another value, to form a visual mask image with the same specifications as the task area image.
6. A contact point dynamic prediction and optimization method according to claim 5, characterized in that: The steps of reprocessing are: Converting the task area image into a three-dimensional image and mapping it to the visual mask image; Extracting 3D points in the three-dimensional image that are mapped to a specific pixel value area in the visual mask image as mask points; Reduces the value score of the cell corresponding to the masked point in the 3D value map.
7. A contact point dynamic prediction and optimization method according to claim 1, characterized in that: The method for optimizing the 3D value map is: Obtain a scene image of the current best contact point, and determine whether the current best contact point is located in an inoperable area; Generate feedback text according to the judgment result, and extract feedback text features of the feedback text, wherein when the judgment result is that the current best contact point is located in the inoperable area, the generated feedback text is "the contact point is located in the inoperable area"; if the judgment result is that the current best contact point is not located in the inoperable area, the generated feedback text is "the contact point is in the operable area but the operation fails"; Performing multimodal alignment on the feedback text feature and the visual feature to obtain a multimodal alignment result; New priority cues and new visual masks are regenerated based on the multimodal alignment result, and the 3D value map is optimized using the new priority cues and new visual masks.
8. A contact point dynamic prediction and optimization system, characterized in that: The system is used to implement the method according to any one of claims 1 to 7, comprising: Data acquisition module: used to collect task text instructions from users or upper-level tasks, as well as images of task areas and their corresponding visual information and depth information; Feature extraction and fusion module: including a visual feature extraction unit, a text feature extraction unit and a feature fusion unit, wherein the visual feature extraction unit is used to extract visual features of visual information collected by the data collection module, the text feature extraction unit is used to extract text features of task text instructions of users or upper-level tasks collected by the data collection module, and the feature fusion unit is used to perform multimodal alignment of text features and visual features; 3D value map module: used to perform all operations related to the 3D value map based on the processing results of the feature extraction and fusion module, including: 3D value map construction, initialization, reprocessing and optimization; Controller: used to select the best contact point based on the 3D value map, control the robot execution, and output the execution status; Feedback module: used to receive executions with failed status, analyze the reasons for failure, generate feedback text based on the reasons for failure, and transmit the feedback text to the feature extraction and fusion module.
9. An electronic device for dynamic prediction and optimization of contact points, comprising a memory and a processor, wherein a computer program is stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.