Interactive program generation method based on three-dimensional scene segmentation, complementation and editable attribute recognition

Abstract

The invention relates to the technical field of interactive program design, in particular to an interactive program generation method based on three-dimensional scene segmentation, complementation and editable attribute recognition. Through deep analysis and intelligent processing of a three-dimensional scene, accurate identification of objects in the scene, interaction attribute analysis and user interaction task design and evaluation are realized by utilizing the capability of a large language model, object interaction attributes are automatically analyzed by utilizing the large language model, attribute tags are generated, and the user interaction task design and evaluation can be realized through natural interaction modes such as voice and gestures. According to the method, interaction content is edited in real time based on the object attribute analysis, and interaction logic is dynamically optimized based on deep learning until a complete interaction program is formed, so that full-process automation of three-dimensional scene processing, object attribute analysis and interaction content generation is realized, and a content creation threshold is reduced; the technical problems that in the prior art, interaction attributes are not considered in a traditional object recognition and numbering method, and the degree of supporting interaction content generation is poor are solved.

Description

Technical Field

[0001] This invention relates to the field of interactive program design technology, and in particular to an interactive program generation method based on three-dimensional scene segmentation, completion, and editable attribute recognition. Background Technology

[0002] With the rapid development of fields such as metaverse, digital twins, and industrial simulation, efficient processing and intelligent analysis of 3D scene data have become key technical requirements. Existing 3D scene processing solutions typically include three core steps: 1. Data segmentation: using point cloud processing networks or 3D convolutional neural networks to delineate the geometric boundaries of objects in the scene; 2. Data completion: using generative adversarial networks or diffusion models to reconstruct the structure of missing data caused by sensor occlusion or insufficient resolution; 3. Object recognition and numbering: classifying the segmented objects by type and assigning them unique identifiers to support downstream tasks such as scene retrieval and interactive control.

[0003] In the field of intelligent processing of 3D scenes, existing technologies usually focus on using geometric segmentation algorithms, such as PointNet series models based on Euclidean clustering or deep learning, to achieve preliminary identification and numbering of objects in the scene. However, the numbering process often relies on manually preset rules or fixed attribute templates, making it difficult to dynamically parse semantic interactive attributes such as functional associations and operation permissions of objects in complex interactive scenes, resulting in a disconnect between the numbering results and the actual scene requirements.

[0004] Traditional interactive content generation solutions often use pre-set scripts or template-based editing tools. Users need to adjust the content through menu-based operations. The interaction is mechanical and lacks naturalness. Especially in the creation of interactive programs that require multiple rounds of iteration and optimization, there are core pain points such as low creation efficiency and poor scene adaptability. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an interactive program generation method based on 3D scene segmentation, completion, and editable attribute recognition. This method solves the technical problems of existing traditional object recognition and numbering methods that do not consider interactive attributes and have poor support for interactive content generation.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: an interactive program generation method based on 3D scene segmentation, completion, and editable attribute recognition, comprising the following steps: The system segments, completes, and identifies objects in a 3D scene, parses the interactive attributes of the objects based on a large language model to generate attribute tags, and associates and integrates them with a preset interactive logic rule system to generate a set of rules that have clear constraints on the objects. Based on whether the object states in the current 3D scene satisfy any set of rules in a preset 3D scene, a scene stability analysis is performed to determine whether the current 3D scene is in a stable state. In response to the stable state analysis of the scenario, an editable 3D interactive program is generated, and the interactive logic is optimized by constructing natural interaction methods and iterative feedback until a publishable interactive program is formed. The completion rate is evaluated through several evaluation indicators.

[0007] Preferably, the specific steps for segmenting, completing, and recognizing objects in a 3D scene are as follows: The 3D scene is processed to identify different scenes and independent objects within different scenes, as well as their corresponding categories, and then numbered. Identify objects with incomplete models and complete them using a 3D reconstruction algorithm to obtain the object's outline features and key structures.

[0008] Preferably, the specific steps for parsing the interactive attributes of the object based on the large language model to generate attribute tags are as follows: The semantic information contained in the objects in the 3D scene is converted into structured hypertext information; The structured hypertext information is input into a large language model, and the model is guided by preset prompts to parse the interactive attributes of the object and generate attribute tags. In response to a user's deletion command for the attribute tag, the attribute tag's serial number is reassigned.

[0009] Preferably, the rule set includes linear constraints, nonlinear constraints, and logical constraints.

[0010] Preferably, the specific steps of the stable state analysis are as follows: determine whether the state of the object in the current three-dimensional scene satisfies any set of rules in the preset three-dimensional scene; if yes, then determine that the current three-dimensional scene is in a stable state; if no, then determine that the current three-dimensional scene is in an unstable state.

[0011] Preferably, the specific steps of the iterative feedback optimization interaction logic are as follows: Design user interaction tasks based on the requirements of 3D scene transformation; Based on the aforementioned user interaction task, an interactive interface is designed for users to change the state of objects in the current 3D scene through several control commands. Based on natural interaction, the system captures users' interaction intentions in real time and generates editable interactive programs through natural interaction between users and the system, combined with deep learning methods. The system transforms the intentions expressed by the user's natural interaction behavior into specific interactive elements or operation instructions, and generates a preliminary editable interactive program content framework. Through several rounds of editing by the user, an editable interactive program is created, forming a publishable interactive program.

[0012] Preferably, the evaluation indicators include target achievement rate c1, key step pass rate c2, average time spent reflecting operational efficiency c3, voice communication frequency c4, resource allocation fairness c5, and conflict resolution speed c6, which reflect collaborative ability.

[0013] Preferably, the completion assessment includes calculating task completion score, collaboration ability score, individual task comprehensive score, and multi-person task comprehensive score.

[0014] By employing the above technical solution, the present invention provides an interactive program generation method based on 3D scene segmentation, completion, and editable attribute recognition, which has at least the following beneficial effects: 1. This invention utilizes the capabilities of a large language model to achieve accurate object recognition, interactive attribute analysis, adaptive layout planning, and user interaction task design and evaluation within a 3D scene through in-depth analysis and intelligent processing of the scene. It automatically parses object interactive attributes using the large language model, generating structured and scalable attribute tags. Interactive content can be edited in real time through natural interaction methods such as voice and gestures, and the interactive logic is dynamically optimized based on reinforcement learning until a complete program is formed. This achieves full automation of the 3D scene processing, object attribute analysis, and interactive content generation process, lowering the barrier to content creation.

[0015] 2. This invention introduces a natural interaction method and combines deep learning technology to capture the user's interactive intent in real time. Users do not need to master professional 3D modeling or programming knowledge; they can directly manipulate objects in a 3D scene through simple natural behaviors such as speaking and gestures. The system can intelligently recognize these behaviors and convert them into specific interactive elements or operation commands, generating a preliminary editable program content framework. At the same time, through a multi-round iterative feedback mechanism, the system continuously optimizes the interactive logic based on the user's editing operations, dynamically adjusting the program content to ensure that it meets the needs of the scene and user expectations. This approach greatly reduces the creation threshold of interactive programs, improves creation efficiency, and ensures the quality of the final program and user experience, making this method widely applicable in fields such as virtual reality, industrial simulation, and education and training.

[0016] 3. This invention uses a comprehensive evaluation index system to quantitatively assess the completion rate of interactive programs, thereby comprehensively and objectively measuring the quality of interactive programs and user performance. Based on these indicators, the system calculates task completion scores, collaboration ability scores, single-person task comprehensive scores, and multi-person task comprehensive scores, and provides detailed evaluation reports and improvement suggestions to help users identify shortcomings and optimize operations. It also incentivizes users to actively participate and improve their skills through real-time feedback and reward mechanisms. In addition, the evaluation results can be used to further optimize interactive logic and scene design, forming a virtuous cycle and significantly improving the adaptability, educational value, and practical effect of interactive programs. It is suitable for various complex scenarios such as multi-functional halls for meetings, banquets, exhibitions, and other mode conversions. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of the interactive program generation method based on 3D scene segmentation, completion, and editable attribute recognition according to the present invention; Figure 2 This is a schematic diagram of the interactive program generation method of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects, and to facilitate its implementation.

[0019] To address the technical problems of inefficient interactive attribute processing and poor flexibility in interactive content generation in existing object recognition and numbering schemes, this invention provides an interactive program generation method based on 3D scene segmentation, completion, and editable attribute recognition. This invention can be applied to virtual production, industrial simulation, smart education, and other fields, significantly lowering the barrier to entry for creating 3D interactive programs and improving the naturalness of human-computer interaction and scene adaptability. The execution entity of this invention can be a server or any device with data processing capabilities. The specific steps of this interactive program generation method are described in detail below, using a multi-functional hall as an example: First, the objects in the 3D scene need to be segmented, completed, and identified. The specific steps are as follows: First, use 3D point cloud segmentation algorithms or deep learning-based 3D object recognition technology to process the 3D scene, identifying different scenes and independent objects within different scenes, along with their corresponding categories, and assigning them numbers. Different scenes can be labeled as Scene A, Scene B, etc. After accurately segmenting each independent object within the same scene, identify the object category, such as chairs, tables, projectors, etc., which can be labeled as O. A-a O A-b ...identify objects with incomplete models, complete the segmented incomplete models using 3D reconstruction algorithms to obtain the object's contour features and key structures, and then use deep learning-based 3D object recognition technology to process the scene.

[0020] The following section uses a multi-functional hall as an example to illustrate the steps of segmenting, completing, and recognizing objects in a 3D scene: The multi-functional hall is labeled as a 3D space multi-function room (hereinafter referred to as scene M). Using 3D point cloud processing algorithms or deep learning models, scene M is comprehensively analyzed to accurately identify the types of each object in scene M. The multi-functional hall model is segmented into independent object units, such as chairs, tables, projectors, and audio equipment. Based on the recognition results, the objects are categorized into preset categories, such as chairs, tables, and equipment, and each object is assigned a unique number. Considering the functional requirements of the virtual multi-functional hall, the objects can be divided into the following four primary types, further subdivided into secondary types, as shown in Table 1 below: The numbering format can be: first-level type_second-level type_serial number, for example: Furniture_Seat_0001 means furniture class - chair - first chair.

[0021] Next, it is necessary to parse the interactive attributes of objects based on the large language model to generate attribute labels. The specific steps are as follows: In order to leverage the powerful capabilities of the large language model for further analysis, reasoning, and generation tasks, the semantic information contained in the objects in the 3D scene needs to be transformed into structured hypertext information. Structured hypertext information can express rich semantic information such as the category, attributes, and positional relationships of objects in a natural language manner, which is easier to understand and process than pure numerical or structured data. Then, the structured hypertext information can be input into the large language model, and the model can be guided by preset prompts to parse the interactive attributes of objects and generate attribute labels, which can be marked as Properties. A-a-1 Property A-a-2 Property A-b-1...and after receiving a user's instruction to delete attribute tags, the system can respond to it, then reassign the attribute tag numbers. After obtaining the attribute tags, it associates and integrates the attribute tags with the preset interaction logic rule system to generate a rule set R that has clear constraints on the object, regulating the dynamic response and behavior patterns of the object in the interaction scenario. The constraint rules include linear constraints, non-linear constraints, and logical constraints. Linear constraints include spacing limits; non-linear constraints include collision avoidance; and logical constraints include mode switching conditions.

[0022] The following section uses any chair in a multi-functional hall as an example to explain the attribute label generation process in detail: In the processing flow for the multi-functional hall scene, the information of the chair objects after segmentation and recognition is first systematically organized. Specifically, basic information such as the position coordinates, dimensions (length, width, height), material type, and color attributes of each chair in three-dimensional space are collected. Simultaneously, its relative positional relationship with other surrounding objects is recorded, such as its distance from the edge of a table and the angle with the projector light. This rich and comprehensive information is arranged according to the specific input format and specifications of the large language model to generate semantic description text that meets the input requirements of the large language model. After the semantic description text is generated, the large language model's interactive attribute semantic parsing stage begins. The carefully prepared semantic description text is accurately input into the large language model, and carefully conceived and targeted prompts guide the model to deeply analyze the interactive attributes of each chair. These interactive attributes encompass multiple dimensions, such as the chair's mobility (whether it can move freely on the multi-functional hall floor), adjustability (whether the backrest angle and seat height can be adjusted), load-bearing capacity (maximum weight it can withstand), and interoperability with other devices (whether it is connected to the multi-functional hall's intelligent control system and whether its movement or adjustment can be remotely controlled via the system). After model parsing, the interactive attributes of each chair are presented in a clear and unambiguous labeled form, thus obtaining the object's attribute label, such as Furniture_Seat_0001 - property1.mobility (indicating that the first interactive attribute of the first chair in the furniture class is mobility). Simultaneously, these attribute labels obtained from the large language model parsing are precisely associated and deeply integrated with the pre-defined multi-functional hall interaction logic rule system. This rule system is formulated based on the actual usage needs and safety regulations of the multi-functional hall. For example, it stipulates that during meetings, the distance between chairs and the conference table must be maintained within a certain range to ensure the comfort of attendees; during cultural performances, chairs must be quickly moved to designated positions according to a specific arrangement. Through this association and integration, a set of rules is generated that has clear binding force on every chair in the multi-functional hall. These rules will strictly regulate the dynamic response and behavior patterns of the chairs in various interactive scenarios, ensuring that all activities in the multi-functional hall can be carried out smoothly, efficiently, and safely.

[0023] After obtaining the rule sets for each 3D scene, a scene stability analysis can be performed based on whether the object states in the current 3D scene satisfy any preset rule set in the 3D scene. If the object states in the current 3D scene satisfy any preset rule set in the 3D scene, then the current 3D scene is determined to be in a stable state; otherwise, the current 3D scene is determined to be in an unstable state. That is, when the placement, interaction state, and spatial relationship of all objects in the current 3D scene completely satisfy the complete constraint rule set R of a preset 3D scene, the system determines that the scene is in a stable state. When the scene is in an adjustment transition period and some objects do not satisfy any complete constraint rule set of the scene, the system determines that the scene is in an unstable state. Through this explicit constraint rule, phased correction, and real-time feedback method, the efficiency and safety of scene transitions can be achieved, ensuring that the system quickly converges to the nearest neighbor stable state in an unstable state.

[0024] After completing the scene stability analysis, in response to the analysis, an editable 3D interactive program is generated. The interactive logic is optimized through the construction of natural interaction methods and iterative feedback until a publishable interactive program is formed. The specific steps for iterative feedback optimization of the interactive logic are as follows: Based on the 3D scene transformation requirements, user interaction tasks are designed. Tasks are issued to users by adding non-player characters. Tasks can take various forms, such as single-player, multi-player, and level-based. Subsequently, based on the user interaction tasks, an interactive interface is designed for users to change the state of objects in the current 3D scene through several control commands. Users can change the placement of objects by dragging, rotating, scaling, etc., to complete the task objectives. Finally, based on natural interaction methods, the interactive program is optimized through user interaction with the system. Natural interactive operations, such as voice commands, gestures, and touch feedback, combined with deep learning methods, enable the system to capture the user's interactive intent in real time and generate editable interactive programs. The system can understand the intent expressed by the user's natural interactive behavior and, based on this intent, translate it into specific interactive elements or operation instructions, generating a preliminary editable interactive program content framework. Users can then further modify and improve the content according to their needs. Through several rounds of editing by the user, i.e., repeated editing, adjustment, and optimization of the preliminary editable interactive program, the system continuously responds to the user's operations until the user confirms that the interactive content has achieved the expected effect, ultimately forming a publishable interactive program.

[0025] The following section uses a multi-functional hall as an example to detail the process from constructing the initial 3D interactive program to creating a publishable 3D interactive program: First, different usage scenarios for the multi-functional hall are defined, such as Meeting Mode, Banquet Mode, Exhibition Mode, and Training Mode. Each scenario is described in detail, including functional requirements, number of participants, and activity flow. Next, a large language model is trained using extensive text data related to the multi-functional hall scenarios, enabling it to understand the characteristics and requirements of different scenarios. In the interactive program design, based on the user-selected scenario or a scenario automatically identified by the system, the large language model is invoked to determine the ideal placement and interaction state of objects in that scenario. Finally, based on the judgment results of the large language model and the interactive attributes of objects, the object placement forms and interaction schemes for the multi-functional hall in different scenarios are generated. The generated form schemes are optimized, considering factors such as space utilization, ease of personnel flow, and visual effects to ensure that the schemes meet practical application needs.

[0026] After obtaining publishable interactive programs, the system needs to quantitatively evaluate user completion based on a behavioral data analysis model. Non-player roles within the system evaluate completion based on several assessment indicators following user actions. These indicators include goal achievement rate (c1), key step completion rate (c2), average time spent (c3) reflecting operational efficiency, voice communication frequency (c4) reflecting collaboration ability, resource allocation fairness (c5), and conflict resolution speed (c6). Throughout the process, the system provides an intuitive and convenient editing interface and interactive feedback, enabling users to easily create and modify interactive content, ensuring that the final interactive programs meet user needs and creative ideas. The quantification of assessment indicators can be calculated according to Table 2 below. The completion assessment includes calculating the task completion score, collaboration ability score, individual task comprehensive score, and multi-person task comprehensive score. Specifically, the task completion score = target achievement rate score × η1 + key step pass rate × η2, where η1 and η2 are two corresponding coefficients, and η1 + η2 = 100%; the collaboration ability score = voice communication frequency × σ1 + resource allocation fairness × σ2 + conflict resolution speed × σ3, where σ1, σ2, and σ3 are corresponding coefficients, and σ1 + σ2 + σ3 = 100%; the individual task comprehensive score = total task completion score × α1 + total operational efficiency score × α2, where α1 and α2 are corresponding coefficients, and α1 + α2 = 100%; and the multi-person task comprehensive score = total task completion score × β1 + total operational efficiency score × β2 + total collaboration ability score × β3, where β1, β2, and β3 are corresponding coefficients, and β1 + β2 + β3 = 100%.

[0027] Finally, the final comprehensive scores can be graded as follows: 90-100 points are excellent, meaning they fully meet the standards and are highly efficient; 70-89 points are good, meaning they basically meet the standards but need optimization; and <70 points are in need of improvement, meaning they do not meet the standards or have serious problems.

[0028] The following section, using the multi-functional hall as a backdrop, details the completion assessment process: Interactive program design needs to clearly define program objectives, such as having users transform a multi-functional hall from a meeting setting to a banquet setting within a specified time, or optimizing the placement of objects based on given scene requirements. Specific tasks should be presented to users within the interactive program, clearly defining task requirements, time limits, and scoring criteria. Operational guidance and prompts should be provided to help users understand how to achieve scene transformation by changing the placement of objects. Various methods such as animation demonstrations, text descriptions, and voice prompts can be used for guidance. An intuitive and user-friendly interface should be designed, allowing users to move, rotate, and scale objects in the multi-functional hall through mouse dragging, touch operations, etc. The interface should display the object's number, interactive attributes, and ideal state in the current scene in real time for user reference and operation.

[0029] After the interactive program design is completed, reasonable evaluation indicators are set according to the program objectives and task requirements, such as the accuracy of object placement, the time efficiency of scene transitions, and the rationality of space utilization. Each evaluation indicator is assigned a corresponding weight, and the user's completion score is calculated comprehensively. Simultaneously, the user's operational behavior and object status changes in the interactive program are monitored in real time. Based on the preset evaluation indicators and weights, the user's operational results can be automatically evaluated, generating a detailed evaluation report. The evaluation report may include the user's score, detailed scores for each indicator, and improvement suggestions. The evaluation results are then promptly fed back to the user, allowing them to understand their performance and shortcomings. Based on the user's completion score, corresponding reward mechanisms can be further set up, such as points, badges, and virtual prizes, to incentivize users to actively participate in the interactive program and improve their operational skills.

[0030] For example, in a single-player task, the user needs to complete the objective of repairing a space station vulnerability and restoring oxygen supply within 15 minutes during a "Virtual Space Station Crisis Rescue" program held in a multi-functional hall. The user triggers AI to diagnose the space station vulnerability and obtain repair solutions via voice commands, combining this with natural interactions such as gesture-based material grabbing and collaborative space transportation equipment to complete the rescue task. The system collects and evaluates data in real time, including repair progress percentage, oxygen supply restoration time, number of material grabbing errors, tool misuse frequency, time consumption, differences in workload, and number of conflict resolutions. The final evaluation results are: Task completion: 85 points (100% repair progress, but oxygen restoration only 30 seconds ahead of schedule); Operational efficiency: 80 points (3 incorrect tool selections, 1 instance of tool damage); Overall score: 82.5 points, Good.

[0031] For example, in a multi-person task, User A and User B need to transform a multi-functional hall from a meeting mode to a teaching mode within 10 minutes by setting up display boards and moving tables and chairs. Users interact by moving objects with voice commands, zooming display boards with gestures, and dragging tables and chairs to adjust their positions. The system collects the users' operation trajectories, records the task start time, the time of each interaction, and the completion time, and analyzes the accuracy of user commands as data for the evaluation system. The final evaluation results are: Task completion: 80 points (podium position error 0.2 meters); Operation efficiency: 70 points (time taken 12 minutes, exceeding the standard by 20%; 5 invalid operations); Collaboration ability: 85 points (communication was clear, but User B moved one extra object); Overall score: 78.5 points, good.

[0032] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0033] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Since the above embodiments are substantially similar to the method embodiments, their descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0034] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An interactive program generation method based on three-dimensional scene segmentation, completion and editable attribute recognition, characterized in that, The interactive program generation method comprises the following steps: Segmenting, completing and recognizing objects in a three-dimensional scene, and analyzing the interaction attributes of the objects based on a large language model to generate attribute labels, and associating and integrating with a preset interaction logic rule system to generate a rule set with explicit constraints on the objects; According to whether the object state in the current three-dimensional scene satisfies any rule set in the preset three-dimensional scene, analyzing the scene stable state of whether the current three-dimensional scene is in a stable state; In response to the scene stable state analysis, an editable three-dimensional interactive program is generated, and the interactive logic is iteratively optimized through natural interaction mode and feedback until a publishable interactive program is formed, and the completion degree is evaluated through a number of evaluation indexes.

2. The interactive program generating method of claim 1, wherein, The specific steps of segmenting, completing and recognizing objects in a three-dimensional scene are: Processing the three-dimensional scene to identify different scenes, independent objects in different scenes and their corresponding categories, and numbering; Identifying incomplete objects and completing them through three-dimensional reconstruction algorithm to obtain the contour features and key structures of the objects.

3. The method of claim 1, wherein, The specific steps of analyzing the interaction attributes of the objects based on a large language model to generate attribute labels are: Converting the semantic information contained in the objects in the three-dimensional scene into structured hypertext information; Inputting the structured hypertext information into a large language model, and guiding the model to analyze the interaction attributes of the objects through a preset prompt word, and generating attribute labels; In response to the user's deletion instruction of the attribute label, reassigning the serial number of the attribute label.

4. The method of claim 1, wherein, The rule set includes linear constraints, nonlinear constraints and logical constraints.

5. The method of claim 1, wherein, The specific steps of the stable state analysis are: judging whether the object state in the current three-dimensional scene satisfies any rule set in the preset three-dimensional scene; if yes, determining that the current three-dimensional scene is in a stable state; if not, determining that the current three-dimensional scene is in an unstable state.

6. The method of claim 1, wherein, The specific steps of iteratively optimizing the interactive logic are: Designing user interaction tasks based on three-dimensional scene conversion needs; Designing an interaction interface for users to change the object state in the current three-dimensional scene through a number of control instructions based on the user interaction tasks; Based on natural interaction mode, through natural interaction operation of the user and the system, combining deep learning method to make the system capture the user's interaction intention in real time, and generate an editable interactive program; The system converts the intention expressed by the user's natural interaction behavior into specific interactive elements or operation instructions, and generates a preliminary editable interactive program content framework; Through several rounds of editing of the generated editable interactive program by the user, a publishable interactive program is formed.

7. The method of claim 1, wherein, The evaluation indexes include target achievement rate c1, key step pass rate c2, average time consumption c3 reflecting operation efficiency; Voice communication frequency c4 reflecting collaboration ability, resource allocation fairness c5 and conflict resolution speed c6.

8. The method of claim 1, wherein, The completion degree evaluation includes task completion degree score, collaboration ability score, single person task comprehensive score and multi-person task comprehensive score.

Images