Scene interaction point generation method, electronic device, and program product

Abstract

The present disclosure provides a scene interaction point generation method, an electronic device and a program product. The scene interaction point generation method comprises: determining at least one interaction point generation strategy; performing strategy configuration based on the determined interaction point generation strategy to obtain strategy configuration information; in response to an interaction point generation request, obtaining a scene three-dimensional model of a target scene; and based on the strategy configuration information, executing the determined interaction point generation strategy on the scene three-dimensional model to generate an interaction point in the scene three-dimensional model.

Description

Technical Field

[0001] This disclosure relates to a method for generating scene interaction points, an electronic device, and a program product. Background Technology

[0002] In Mixed Reality (MR) applications, virtual content needs to be deeply integrated with the real environment. This requires precisely setting a large number of interaction points (pins) in the 3D virtual scene to achieve functions such as spatial information display, triggering virtual-real interactions, and object replacement previews. Traditional interaction point generation solutions, which rely on manual configuration by designers, are time-consuming, labor-intensive, and suffer from inconsistencies in interaction point generation styles and spatial location choices among different designers. Existing automatic interaction point generation solutions lack a flexible combination mechanism for various interaction point generation strategies, making it difficult to adapt to the complex needs of multi-layered interactions in mixed reality applications. Summary of the Invention

[0003] This disclosure provides a method for generating scene interaction points, an electronic device, and a program product.

[0004] According to one aspect of this disclosure, a method for generating scene interaction points is provided, comprising: determining at least one interaction point generation strategy; configuring the strategy based on the determined interaction point generation strategy to obtain strategy configuration information; in response to an interaction point generation request, acquiring a scene 3D model of a target scene; and executing the determined interaction point generation strategy on the scene 3D model based on the strategy configuration information to generate interaction points in the scene 3D model.

[0005] According to at least one embodiment of the scene interaction point generation method of this disclosure, the determined interaction point generation strategy includes multiple of the following: room interaction point generation strategy, door interaction point generation strategy, window interaction point generation strategy, and furniture interaction point generation strategy. The step of executing the determined interaction point generation strategy on the scene 3D model based on the strategy configuration information to generate interaction points in the scene 3D model includes: Each interaction point generation strategy is run independently to obtain the initial interaction point data corresponding to each interaction point generation strategy; The initial interaction point data are integrated to generate structured interaction point data; Interaction points are generated in the 3D model of the scene based on the structured interaction point data.

[0006] According to at least one embodiment of the scene interaction point generation method of this disclosure, the interaction point generation strategy includes a room interaction point generation strategy, which includes: determining the scene boundary of the target scene based on the scene 3D model; identifying the walls contained in the target scene; meshing the target scene based on the scene boundary to obtain multiple mesh points; performing ray detection based on the mesh points to remove the mesh points corresponding to the walls from the mesh points to obtain multiple effective face points; determining an initial room based on the effective face points using a flood fill algorithm; determining whether the initial room belongs to a nested room; designating the initial room that does not belong to a nested room as a valid room; calculating the room center of the valid room; and generating interaction points based on the room center.

[0007] According to at least one embodiment of the scene interaction point generation method of this disclosure, determining the scene boundary of the target scene based on the scene 3D model includes: traversing objects in the scene 3D model; determining the object boundary of each object based on the renderer component of each object; and merging the object boundaries of each object to obtain the scene boundary.

[0008] According to at least one embodiment of the scene interaction point generation method of this disclosure, identifying walls contained in the target scene includes: identifying based on the attributes of each object, identifying based on the name of each object, or identifying based on the height and thickness of each object.

[0009] According to at least one embodiment of the scene interaction point generation method of this disclosure, ray detection is performed based on the grid points to remove the grid points corresponding to the wall from the grid points to obtain a plurality of effective surface points, including: emitting a virtual ray downward from a rated distance above the grid points and detecting whether the virtual ray hits the ground; and taking the grid points corresponding to the virtual ray that hits the ground as effective surface points.

[0010] According to at least one embodiment of the scene interaction point generation method of this disclosure, determining whether the initial room belongs to a nested room includes: determining whether the initial room simultaneously satisfies a first judgment condition and a second judgment condition, wherein the first judgment condition is: the room boundary of the initial room is inside the room boundary of any other initial room; the second judgment condition is any one of the following conditions: the area ratio of the initial room to any other initial room is less than a first ratio threshold, the area of ​​the initial room is less than an area threshold, the ratio of the area of ​​the initial room to the total area of ​​the scene is greater than a second ratio threshold, and the room boundary of the initial room partially overlaps with the scene boundary.

[0011] According to at least one embodiment of the scene interaction point generation method of this disclosure, the interaction point generation strategy includes a door interaction point generation strategy or a window interaction point generation strategy, wherein the door interaction point generation strategy or window interaction point generation strategy includes: identifying doors or windows contained in the target scene based on a scene 3D model; determining the door boundary of the door or the window boundary of the window based on the renderer component corresponding to the door or window; determining the door center of the door based on the door boundary, or determining the window center of the window based on the window boundary; and generating interaction points based on the door center or the window center.

[0012] A scene interaction point generation method according to at least one embodiment of the present disclosure identifies doors or windows contained in the target scene based on a scene 3D model, including: determining objects contained in the target scene based on the scene 3D model; and finding doors or windows from the objects based on the labels or names of the objects.

[0013] According to at least one embodiment of the present disclosure, the scene interaction point generation method includes a furniture interaction point generation strategy, which includes: identifying furniture contained in the target scene based on a scene 3D model; determining the furniture boundary based on the renderer component corresponding to the furniture; determining the furniture center of each piece of furniture based on the furniture boundary; and generating interaction points based on the furniture center.

[0014] According to at least one embodiment of the scene interaction point generation method of this disclosure, the furniture interaction point generation strategy further includes: determining the diagonal length of the furniture based on the furniture boundary; searching a database for multiple candidate furniture pieces that are of the same type as the furniture and similar in size to the furniture based on the diagonal length; sorting the candidate furniture pieces in descending order of similarity based on the similarity between each candidate furniture piece and the furniture; obtaining a replacement recommendation list for the furniture based on the top N candidate furniture pieces, where N > 0; and outputting the replacement recommendation list.

[0015] The scene interaction point generation method according to at least one embodiment of the present disclosure further includes: obtaining an updated scene 3D model in response to a scene 3D model update prompt; and re-executing each interaction point generation strategy in the strategy configuration information based on the updated scene 3D model.

[0016] The scene interaction point generation method according to at least one embodiment of the present disclosure further includes: generating structured interaction point data based on the interaction points; and persistently storing or exchanging data on the interaction points based on the structured interaction point data.

[0017] According to another aspect of this disclosure, an electronic device is provided, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, causing the processor to execute a scene interaction point generation method according to any embodiment of this disclosure.

[0018] According to another aspect of this disclosure, a readable storage medium is provided, wherein executable instructions are stored therein, which, when executed by a processor, are used to implement the scene interaction point generation method of any embodiment of this disclosure.

[0019] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a scene interaction point generation method according to any embodiment of this disclosure. Attached Figure Description

[0020] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0021] Figure 1 This is a flowchart illustrating a method for generating scene interaction points according to one embodiment of the present disclosure.

[0022] Figure 2 This is a flowchart illustrating a room interaction point generation strategy according to one embodiment of the present disclosure.

[0023] Figure 3 This is a flowchart illustrating the method corresponding to step S210 of one embodiment of the present disclosure.

[0024] Figure 4 This is a flowchart illustrating a door interaction point generation strategy or a window interaction point generation strategy according to one embodiment of this disclosure.

[0025] Figure 5 This is a flowchart illustrating a furniture interaction point generation strategy according to one embodiment of the present disclosure.

[0026] Figure 6 This is a flowchart illustrating a furniture interaction point generation strategy according to another embodiment of the present disclosure.

[0027] Figure 7 This is a flowchart illustrating a method for generating scene interaction points according to another embodiment of this disclosure.

[0028] Figure 8 This is a schematic block diagram of a scene interaction point generation device according to one embodiment of the present disclosure.

[0029] Figure 9 This is a schematic structural block diagram of an electronic device employing a processor-based hardware implementation according to one embodiment of the present disclosure. Detailed Implementation

[0030] The present disclosure will now be described in further detail with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0031] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0032] In mixed reality applications, a large number of interaction points (Pins) need to be precisely set in the 3D virtual scene. Taking MR virtual house viewing as an example, interaction points (viewpoint switching points) need to be set in the center of the room for viewpoint switching and spatial information display; interaction points (local modification points) need to be set at the furniture positions to support furniture style replacement preview; interaction points (jump points) need to be set at the door and window positions to realize fast movement between rooms; and interaction points (information annotation points) need to be set in special areas such as balconies and bay windows to display the highlights of the house. A standard three-bedroom house usually needs to generate dozens to hundreds of interaction points, covering multiple levels of interactive objects such as rooms, furniture, doors and windows.

[0033] Traditional interaction point generation methods rely on designers manually configuring each interaction point in a 3D editor (such as placing GameObjects in the Unity Editor and configuring spatial anchor points, interaction types, associated data, and other attributes). Complete annotation of a single scene can take 2 to 3 hours. When dealing with the need to deploy batch scenes (such as 100 properties), the total time can reach 200 to 300 hours, resulting in high labor costs. More seriously, different designers have subjective differences in their interaction point generation styles and spatial location choices, leading to inconsistent MR experiences. When the scene model changes (such as moving walls or replacing furniture), all interaction points need to be manually readjusted in terms of spatial mapping, resulting in extremely high maintenance costs.

[0034] The existing automatic interaction point generation schemes have significant defects: (1) The automatic interaction point generation method based on fixed rules distributes interaction points evenly on the room grid and lacks understanding of the spatial semantics of the scene. The generated interaction points may be located in the interior of the wall or under the furniture, where there is a conflict between virtual and real occlusion, resulting in a low actual usability. (2) The single-strategy method (such as relying on object name or label matching) is highly dependent on standardized modeling. When the model naming is not standardized or different spatial coordinate systems are used, the single-strategy method will completely fail. (3) Although the spatial understanding method based on deep learning has a high accuracy, it has defects such as large training data requirements and high inference latency, making it difficult to meet the real-time requirements of MR devices and difficult to adjust flexibly according to business logic. (4) The interaction point generation tools provided by the existing MR development platform (such as the spatial anchor point editor of MRTK (MixedReality Toolkit)) are essentially still an enhanced version of manual operation and have failed to fundamentally solve the automation problem.

[0035] To address this, this disclosure proposes the following technical solution, which includes an intelligent, automated, and scalable method for generating scene interaction annotation points. Through a multi-strategy fusion architecture design, the interaction point generation strategies for different objects such as rooms, doors, windows, and furniture are encapsulated as independent, pluggable algorithm modules. Users can select the required algorithm modules (interaction point generation strategies) to adapt to different business scenarios based on their individual needs, without modifying the core code. This disclosure not only enables flexible combination and dynamic expansion of interaction point generation strategies but also significantly improves the efficiency of interaction point generation (e.g., reducing single-scene annotation time from 2-3 hours to minutes), ensuring the consistency and accuracy of spatial annotation (e.g., reducing the spatial positioning error of annotation points to within 0.5 meters, meeting the spatial accuracy requirements of MR devices). Furthermore, this disclosure ensures the consistency of interaction point generation results across multiple executions of the same scene, solving the problem of inconsistent MR experiences caused by differences in designer styles in traditional manual annotation.

[0036] To facilitate description and make the technical solutions of this disclosure easier to understand, the terminology of this disclosure will be explained before describing the technical solutions of this disclosure.

[0037] An interaction point (Pin Point) is a spatial marker in a 3D virtual scene used to display spatial information, enable virtual-real interaction, or preview object replacement.

[0038] A scene 3D model is a digital model composed of multiple 3D scene objects that represents the physical space's geometric structure and semantic information.

[0039] This disclosure applies to application scenarios that require automated spatial annotation and interactive point deployment in a 3D virtual scene, such as virtual house viewing, digital twin factories, and industrial collaborative design.

[0040] Figure 1 A schematic diagram illustrating the overall flow of a scene interaction point generation method according to one embodiment of this disclosure is shown. Figure 1 The method shown includes steps S110 to S140. This method can be executed by electronic devices such as mobile phones and tablets.

[0041] In step S110, at least one interaction point generation strategy is determined.

[0042] Interaction point generation strategies are used to generate interaction points and can be pre-built, independently encapsulated, and standardized algorithm modules. Depending on the function type corresponding to the interaction point, interaction point generation strategies can include various different strategies. For example, interaction point generation strategies can include room interaction point generation strategies, door and window interaction point generation strategies, furniture interaction point generation strategies, etc.

[0043] As one possible implementation, a large number of interaction point generation strategies are stored in the system's strategy database. Users can select at least one interaction point generation strategy from the strategy database as the determined interaction point generation strategy according to their own personalized interaction point generation needs.

[0044] As another possible implementation, the interaction point generation strategy can be one or more interaction point generation strategies uploaded by the user in real time that meet their personalized interaction point generation needs. In this case, the received one or more interaction point generation strategies can be used as the determined interaction point generation strategy.

[0045] In step S120, a strategy is generated based on the determined interaction points to configure the strategy and obtain the strategy configuration information.

[0046] Policy configuration allows you to configure the policy for generating specific interaction points into the system's policy management module (such as the policy manager), so that you can subsequently execute the personalized interaction point generation requirements of the user by executing the policy configuration information contained in the policy management module.

[0047] In step S130, in response to the interaction point generation request, a 3D model of the target scene is obtained.

[0048] As one possible implementation, the target scene can be the scene corresponding to the interaction point generation request. As an example, multiple scene 3D models can be pre-stored in a scene database. Upon receiving an interaction point generation request, the request is first parsed to obtain the scene ID (unique code) corresponding to the request. The scene with the scene ID can be the target scene. Then, based on the scene ID, the corresponding scene 3D model is retrieved from the scene database and loaded into the system.

[0049] As another possible implementation, the target scene can be a scene already recorded in the system. After receiving the interaction point generation request, the 3D model of the already recorded scene can be directly called and loaded into the system.

[0050] In step S140, an interaction point generation strategy is executed on the scene 3D model based on the strategy configuration information to generate interaction points in the scene 3D model.

[0051] During the process of executing a determined interaction point generation strategy on the scene's 3D model based on strategy configuration information, each interaction point generation strategy can be executed sequentially or in parallel.

[0052] This implementation uses a multi-strategy fusion architecture design to encapsulate the interaction point generation strategies for different objects such as rooms, doors, windows, and furniture into independent, pluggable algorithm modules. Users can select the required algorithm modules (interaction point generation strategies) to adapt to different business scenarios according to their individual needs, without modifying the core code. This not only enables flexible combination and dynamic expansion of interaction point generation strategies but also significantly improves the efficiency of interaction point generation, ensuring the consistency and accuracy of spatial annotation. Furthermore, this implementation ensures the consistency of interaction point generation results when executed multiple times in the same scenario, resolving the inconsistency in MR experience caused by differences in designer styles in traditional manual annotation.

[0053] In some embodiments of this disclosure, the interaction point generation strategy determined above includes one or more of the following: room interaction point generation strategy, door interaction point generation strategy, window interaction point generation strategy, and furniture interaction point generation strategy.

[0054] In some embodiments, the interaction point generation strategies determined above in this disclosure include multiple strategies among room interaction point generation strategies, door interaction point generation strategies, window interaction point generation strategies, and furniture interaction point generation strategies.

[0055] For step S140 described above, which involves executing the determined interaction point generation strategy on the scene 3D model based on the strategy configuration information to generate interaction points in the scene 3D model, the following steps are included: Each interaction point generation strategy is run independently to obtain the initial interaction point data corresponding to each interaction point generation strategy; The initial interaction point data are integrated to generate structured interaction point data; Interaction points are generated in the 3D model of the scene based on the structured interaction point data.

[0056] In this embodiment, an interaction point generation strategy is executed on the scene 3D model based on the strategy configuration information to generate interaction points in the scene 3D model. This can be achieved in the following way.

[0057] First, each interaction point generation strategy is run independently to obtain the initial interaction point data corresponding to each strategy. Each interaction point generation strategy is encapsulated as an independent, pluggable algorithm module, which executes independently and can be executed sequentially or in parallel. For example, the room interaction point generation strategy, door interaction point generation strategy, window interaction point generation strategy, and furniture interaction point generation strategy can be executed sequentially or in parallel. Each interaction point generation strategy analyzes and processes the 3D model of the scene according to its own algorithmic logic, generating initial interaction point data for the corresponding type of object. For example, the room interaction point generation strategy generates initial interaction point data corresponding to the center of the room, and the furniture interaction point generation strategy generates initial interaction point data corresponding to the centers of each piece of furniture.

[0058] Next, the initial interaction point data generated by each interaction point generation strategy is integrated to generate structured interaction point data. Since the initial interaction point data generated by different strategies may have different internal representation formats, for easier unified management and subsequent output, all initial interaction point data can be standardized and encapsulated according to a unified data structure to generate structured interaction point data. The data format of the structured interaction point data can be JSON or other structured data formats.

[0059] Next, based on the integrated structured interaction point data, the final interaction points are generated in the scene's 3D model. By standardizing and encapsulating interaction points generated by different interaction point generation strategies using a unified data structure, the interaction point data can support batch export and automated version management (e.g., associating it with the scene version number via timestamps). This provides a consistent data interface for cross-platform data integration between MR devices and backend spatial computing services, effectively reducing data format conversion costs and version compatibility risks during long-term operation and maintenance.

[0060] As one possible implementation, the interaction point generation strategy includes a room interaction point generation strategy. Figure 2 A flowchart illustrating a room interaction point generation strategy according to one embodiment of this disclosure is shown. Figure 2The method shown includes steps S210 to S290.

[0061] In step S210, the scene boundary of the target scene is determined based on the scene 3D model. The scene boundary can be the smallest axis-aligned bounding box (AABB) of all objects contained in the scene 3D model. Objects contained in the scene 3D model can include walls, furniture, doors, windows, etc.

[0062] In step S220, walls contained in the target scene are identified.

[0063] As one possible implementation, identifying walls within a target scene includes: identification based on object attributes, identification based on object names, or identification based on object height and thickness. This implementation can improve the accuracy of wall identification. In one example, during identification based on object attributes, all objects in the scene can be traversed, and objects with the layer attribute "Wall" can be selected and identified as walls. In another example, during identification based on object names, the names of each object can be checked, and objects containing keywords related to walls, such as "wall," "partition," or "separation," can be identified as walls. In yet another example, during identification based on object height and thickness, the height and thickness of each object can be measured, and objects with a height greater than a height threshold and a thickness less than a thickness threshold can be identified as walls. The height and thickness thresholds can be set according to actual needs and are not limited here. For example, the height threshold can be set to 1.0 meter and the thickness threshold can be set to 0.3 meters.

[0064] Those skilled in the art will understand that, through step S220, all walls contained in the target scene can be identified, for example, 18 walls can be identified.

[0065] In step S230, the target scene is meshed based on the scene boundary to obtain multiple mesh points.

[0066] As one possible implementation, during the meshing of the target scene, the mesh can be created based on the mesh resolution contained in the configuration information corresponding to the room interaction point generation strategy. The mesh resolution determines the mesh size. For example, the mesh resolution can be set to 0.5 meters (each mesh is a 0.5-meter × 0.5-meter square), and the target scene size is 10 meters × 8 meters. The number of meshes obtained after meshing is (10 ÷ 0.5) × (8 ÷ 0.5) = 20 × 16 = 320 meshes. A mesh point can be the center point of the mesh.

[0067] In step S240, ray detection is performed based on grid points to remove grid points corresponding to the wall from the grid points, resulting in multiple effective surface points.

[0068] As one possible implementation, ray detection is performed based on grid points to remove grid points corresponding to walls, resulting in multiple effective face points. This includes emitting a virtual ray downwards from a predetermined distance above the grid points and detecting whether the virtual ray touches the ground. The grid points corresponding to the virtual rays that touch the ground are then considered as effective face points. This implementation can automatically filter out wall-occupied areas through ray detection, avoiding the problem of virtual / real occlusion caused by interaction points located inside the wall.

[0069] In one example, during raycasting, for each grid point, a virtual ray is emitted from a distance A (e.g., A=2) meters directly above it along the negative Y-axis. The ray is then used to detect whether it intersects with a ground collision object within a distance of A meters. If the ray detects a collision, and the collision object is not a wall, the corresponding grid point can be considered a valid face point. If the ray does not detect a collision, or the collision object is a wall, the corresponding grid point should be considered a non-valid face point. For example, in the 320 grid points corresponding to the 320 grids mentioned above, raycasting can yield 245 valid face points.

[0070] In step S250, the initial room is determined based on the effective surface points using a flood filling algorithm.

[0071] The floodfill algorithm is a spatial region identification algorithm based on connectivity analysis. The core idea of ​​the floodfill algorithm is to recursively expand from a seed point in adjacent directions, grouping all connected valid points into the same region until an obstacle boundary is encountered. In this implementation, the floodfill algorithm is used to identify initial room regions based on valid face points, solving the technical challenge of "how to automatically divide spatial functional areas in unmarked 3D scenes." As an example, the input of the floodfill algorithm is a set of valid ground points filtered by ray detection (e.g., 245 valid face points), and the output is multiple closed connected regions (i.e., the initial rooms).

[0072] In step S260, it is determined whether the initial room belongs to a nested room.

[0073] Nested rooms can be rooms that have a spatial topological relationship with other initial rooms. As one possible implementation, determining whether an initial room is a nested room includes: determining whether the initial room simultaneously satisfies a first judgment condition and a second judgment condition. The first judgment condition is that the boundary of the initial room is inside the boundary of any other initial room. The second judgment condition is any one of the following: the area ratio of the initial room to any other initial room is less than a first ratio threshold (condition 1); the area of ​​the initial room is less than an area threshold (condition 2); the ratio of the area of ​​the initial room to the total area of ​​the scene is greater than a second ratio threshold (condition 3); and the boundary of the initial room partially overlaps with the scene boundary (condition 4). This implementation, through a nested room filtering mechanism constrained by both "spatial inclusion relationship" and "geometric features," can improve the accuracy of nested room recognition and effectively avoid MR experience interference caused by generating redundant interaction points in nested rooms.

[0074] As an example, the boundary of the open-plan kitchen is inside the boundary of the living room, satisfying the first criterion. The ratio of the area of ​​the open-plan kitchen to the area of ​​the living room is 0.2, which is less than the ratio threshold of 0.8, satisfying condition 1 of the second criterion. Therefore, the open-plan kitchen satisfies both the first and second criterion conditions and is considered a nested room.

[0075] As another example, the recessed niche's room boundary is inside the master bedroom's room boundary, satisfying the first condition. The recessed niche's area is 1.5 square meters, less than the area threshold of 2.0 square meters, satisfying condition 2 of the second condition. Therefore, the recessed niche satisfies both the first and second conditions, and thus belongs to a nested room.

[0076] As another example, the room boundary of the misidentified area of ​​the oversized single-level apartment is inside the room boundary of the fully open space, satisfying the first judgment condition. The area of ​​the misidentified area of ​​the oversized single-level apartment is 130 square meters, and the total area of ​​the scene is 200 square meters. The ratio of the two is 0.65, which is greater than the second ratio threshold of 0.6, satisfying condition 3 of the second judgment condition. Therefore, the misidentified area of ​​the oversized single-level apartment satisfies both the first and second judgment conditions and belongs to the nested room category.

[0077] As another example, the boundary of the semi-open balcony room is inside the boundary of the living room room (overlapping area accounts for 85%), satisfying the first judgment condition. The boundary of the semi-open balcony room partially overlaps with the scene boundary (overlapping length 1.2 meters), satisfying condition 4 of the second judgment condition. Therefore, the semi-open balcony renovation area satisfies both the first and second judgment conditions and belongs to nested rooms.

[0078] In step S270, the initial room that does not belong to the nested rooms is taken as a valid room.

[0079] In step S280, the center of the effective room is calculated.

[0080] As one possible implementation, in calculating the center of an effective room, the grid points comprising the effective room can be determined first, and then the center of each grid point can be calculated based on its coordinates. Finally, the calculated center of each grid point is taken as the room center. For example, an effective room contains 6 grid points, with corresponding 3D coordinates P1(2,0,3), P2(2,0,4), P3(2,0,5), P4(3,0,3), P5(3,0,4), and P6(3,0,5) (in meters). In calculating the center of each grid point, the average value of the coordinates of each grid point along the X, Y, and Z axes can be calculated to obtain the 3D coordinates of the center of each grid point. For example, the coordinates of the center of each grid point along the X-axis are (2+2+2+3+3+3)÷6=2.5 meters; the coordinates of the center of each grid point along the Y-axis are 0 meters; and the coordinates of the center of each grid point along the Z-axis are (3+4+5+3+4+5)÷6=4.0 meters. Therefore, the three-dimensional coordinates of the room center are (2.5, 0, 4.0).

[0081] In step S290, an interaction point is generated based on the center of the room.

[0082] This implementation innovatively combines mesh sampling, ray detection, and valid room identification to achieve automatic and accurate room recognition in markerless 3D scenes. By using a nested room filtering mechanism, the accuracy of generating room interaction points can be improved.

[0083] Regarding step S210, in some embodiments of this disclosure, it may include, for example... Figure 3 Steps S2101 to S2103 are shown.

[0084] In step S2101, the objects in the scene's 3D model are traversed.

[0085] In step S2102, the object boundaries of each object are determined based on the renderer components of each object.

[0086] In step S2103, the object boundaries of each object are merged to obtain the scene boundary.

[0087] This implementation method can accurately obtain the scene boundaries of any complex 3D scene without human intervention.

[0088] One possible implementation is to use the Unity Editor to determine the scene boundaries of the target scene based on the scene's 3D model. In determining the scene boundaries, this is first done by calling methods in the Unity engine used to retrieve all game objects in the scene (e.g., GameObject.FindObjectsOfType). <gameobject>You can retrieve all objects contained in the scene's 3D model using either the `()` method or by traversing the scene hierarchy. For example, the retrieved objects might include 18 wall objects, 27 furniture objects, 8 door and window objects, etc. Then, for each object, you can use methods for retrieving components (such as `GetComponent`). <renderer>The `()` method retrieves the renderer component associated with each object. The `bounds` property of the renderer component defines the visible geometric extent of the object in 3D space. By calling the `bounds` property of the renderer component, the object's boundaries can be obtained. Object boundaries can be represented using six floating-point numbers: `min.x`, `min.y`, `min.z`, and `max.x`, `max.y`, `max.z`, where `min.x`, `min.y`, and `min.z` correspond to the minimum vertex coordinates of the object boundary, and `max.x`, `max.y`, and `max.z` correspond to the maximum vertex coordinates of the object boundary. Finally, the object boundaries of all obtained objects are merged, and the merged result is used as the scene boundary.

[0089] As one possible implementation, the interaction point generation strategy includes a door interaction point generation strategy or a window interaction point generation strategy. Figure 4 A flowchart illustrating a door interaction point generation strategy or window interaction point generation strategy according to one embodiment of this disclosure is shown. Figure 4 The method shown includes steps S410 to S440.

[0090] In step S410, based on the 3D model of the scene, doors or windows contained in the target scene are identified.

[0091] As one possible implementation, identifying doors or windows in a target scene based on a 3D scene model includes: determining objects in the target scene based on the 3D scene model; and finding doors or windows from the objects based on their labels or names. For example, objects labeled "Door" can be searched and identified as doors. Alternatively, objects with names containing keywords related to doors, such as "door" or "door," can also be searched and identified as doors. Window identification is similar to door identification and will not be elaborated further. This implementation can quickly identify door or window objects through a semantic matching mechanism based on labels or names, without requiring complex geometric calculations, thus improving recognition speed and accuracy.

[0092] In step S420, the door boundary of the door or the window boundary of the window is determined according to the renderer component corresponding to the door or window. The door boundary and window boundary can be determined by the object boundary determination method in step S2102, which will not be described in detail here.

[0093] In step S430, the center of the door is determined based on the door boundary, or the center of the window is determined based on the window boundary. The door center can be the center of the rectangle enclosed by the door boundaries. The window center can be the center of the rectangle enclosed by the window boundaries.

[0094] In step S440, an interaction point is generated based on the center of the door or the center of the window.

[0095] This implementation method can achieve rapid and accurate generation of door interaction points or window interaction points.

[0096] As one possible implementation, the interaction point generation strategy includes a furniture interaction point generation strategy. Figure 5 A flowchart illustrating a furniture interaction point generation strategy according to one embodiment of this disclosure is shown. Figure 5 The method shown includes steps S510 to S540.

[0097] In step S510, furniture in the target scene is identified based on the 3D model of the scene. Similar to the identification of doors or windows in step S410, furniture identification can also be determined based on the object's label or name, which will not be elaborated further here.

[0098] In step S520, the furniture boundary is determined based on the renderer component corresponding to the furniture. The furniture boundary can also be determined using the object boundary determination method in step S2102, which will not be elaborated here.

[0099] In step S530, the furniture center of each piece of furniture is determined based on the furniture boundaries. The furniture center can be the center of the rectangle enclosed by the furniture boundaries.

[0100] In step S540, interaction points are generated based on the furniture center.

[0101] This implementation method enables the rapid and accurate generation of furniture interaction points.

[0102] Figure 6 A flowchart illustrating a furniture interaction point generation strategy according to another embodiment of this disclosure is shown. Figure 6 The method shown includes steps S610 to S650.

[0103] In step S610, the diagonal length of the furniture is determined based on its boundaries. The diagonal can be the diagonal line within the rectangle enclosed by the furniture boundaries. The length of the diagonal can be calculated based on the coordinates of the smallest and largest vertices contained within the furniture boundaries.

[0104] In step S620, based on the diagonal length, multiple candidate furniture items with the same type and similar size as the furniture are searched from the database.

[0105] Furniture types can include sofas, beds, wardrobes, etc. The database can be a repository of 3D furniture models. These models can contain multiple 3D models of various furniture types in different sizes and styles. For example, when storing sofas, the styles that can be stored include modern sofas, European sofas, mahogany sofas, etc. Each style of sofa can be stored in sizes including single sofas, double sofas, etc.

[0106] In the process of searching for multiple candidate furniture items of the same type and similar size from the database, it is necessary to first search for all furniture of the same type (such as sofas) in the database. Then, the diagonal length of each of the found furniture items of the same type is determined. Next, a similarity algorithm is used to calculate the similarity between the diagonal length obtained in step S610 and the diagonal length of each of the found furniture items of the same type. Finally, furniture items of the same type with a similarity greater than a preset similarity threshold are selected as candidate furniture. The similarity threshold can be set according to the actual application and is not limited here; for example, it can be set to 0.1.

[0107] In step S630, based on the similarity between each candidate furniture and the furniture, the candidate furniture is sorted in descending order of similarity.

[0108] In step S640, a replacement recommendation list for furniture is obtained based on the top N candidate furniture items, where N > 0. The top N candidate furniture items are the N candidate furniture items with the highest similarity. The value of N corresponds to the number of candidate furniture items included in the replacement recommendation list, and can be set according to the actual application. There is no limit here, for example, it can be set to 10.

[0109] In step S650, a replacement recommendation list is output.

[0110] This implementation method uses diagonal length similarity calculation to achieve precise quantitative ranking for furniture replacement recommendations, thereby improving recommendation accuracy and significantly outperforming traditional keyword matching methods.

[0111] As one possible implementation, the formula for calculating the similarity between the lengths of two diagonals can be expressed as follows: in, Indicates similarity; and These represent the lengths of the two diagonals respectively; This represents the absolute value operation; This indicates the operation of finding the maximum value.

[0112] Furthermore, after the scene's 3D model changes, the interaction points generated based on the original scene's 3D model may no longer be applicable, so it is necessary to regenerate the interaction points based on the updated scene's 3D model.

[0113] As one possible implementation, the scene interaction point generation method further includes: obtaining an updated scene 3D model in response to a scene 3D model update prompt; and re-executing the interaction point generation strategies in the strategy configuration information based on the updated scene 3D model. The scene 3D model update prompt can be automatically generated after the scene 3D model is updated. This implementation, through an automatic interaction point reconstruction mechanism triggered by scene 3D model changes, can achieve dynamic updates of interaction points while retaining the original interaction point generation strategy configuration, significantly reducing the maintenance cost of interaction points after scene adjustments.

[0114] Figure 7 A schematic diagram illustrating the overall flow of a scene interaction point generation method according to another embodiment of this disclosure is shown. Figure 7 The method shown includes steps S710 to S760. Steps S710-S740 respectively correspond to... Figure 1 For details of steps S110-S140 of the implementation method, please refer to [link / reference]. Figure 1 The relevant descriptions of the implementation methods will not be repeated here.

[0115] In step S750, structured interaction point data is generated based on the interaction points. The data format of the structured interaction point data can be JSON or other structured data formats, and there are no restrictions here.

[0116] In step S760, the interaction points are persistently stored or exchanged based on the structured interaction point data.

[0117] This implementation standardizes and encapsulates interaction points generated by different interaction point generation strategies using a unified data structure. This enables batch export and automated version management of interaction point data (associated with timestamps and scene version numbers). It provides a consistent data interface for cross-platform data integration between MR devices and backend spatial computing services. This effectively reduces data format conversion costs and version compatibility risks during long-term operation and maintenance.

[0118] According to any of the above embodiments, this disclosure also provides a scene interaction point generation device 800. Figure 8 This is a schematic block diagram of a scene interaction point generation device 800 according to one embodiment of this disclosure. Figure 8 As shown, the scene interaction point generation device 800 includes a strategy determination module 810, a strategy configuration module 820, a scene 3D model acquisition module 830, and an interaction point generation module 840.

[0119] The strategy determination module 810 is used to determine at least one interaction point generation strategy.

[0120] The strategy configuration module 820 is used to generate a strategy based on the determined interaction points and perform strategy configuration to obtain strategy configuration information.

[0121] The scene 3D model acquisition module 830 is used to acquire the scene 3D model of the target scene in response to the interaction point generation request.

[0122] The interaction point generation module 840 is used to execute a determined interaction point generation strategy on the scene 3D model based on the strategy configuration information, and generate interaction points in the scene 3D model.

[0123] The scene interaction point generation device 800 disclosed herein can be implemented through a computer software architecture.

[0124] According to further embodiments of this disclosure, an electronic device is also provided.

[0125] Figure 9 A schematic block diagram of an electronic device employing a processor-based hardware implementation according to an embodiment of the present disclosure is shown.

[0126] The hardware architecture of the electronic device disclosed herein can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application and overall design constraints of the hardware. Bus 1100 connects various circuits including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400 such as peripherals, voltage regulators, power management circuits, external antennas, etc. Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one connection line is used in this figure, but this does not indicate that there is only one bus or one type of bus.

[0127] This disclosure also provides a readable storage medium storing a computer program that, when executed by a processor, is used to implement the methods described above. "Readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of readable storage media include: an electrical connection with one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0128] This disclosure also provides a computer program product, the methods of which can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, all or part of the processes or functions of this disclosure are performed.

[0129] Computer programs or instructions can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The readable storage medium can be any available medium capable of access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or it can include both volatile and non-volatile types of storage media.

[0130] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, or characteristic described in connection with that embodiment / mode or example is included in at least one embodiment / mode or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0135] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.< / renderer> < / gameobject>

Claims

1. A method for generating scene interaction points, characterized in that, include: Determine at least one interaction point generation strategy; Based on the determined interaction point generation strategy, strategy configuration is performed to obtain strategy configuration information; In response to the interaction point generation request, obtain the 3D model of the target scene; as well as Based on the strategy configuration information, the determined interaction point generation strategy is executed on the scene 3D model to generate interaction points in the scene 3D model.

2. The method as described in claim 1, characterized in that, The determined interaction point generation strategy includes multiple strategies among room interaction point generation strategy, door interaction point generation strategy, window interaction point generation strategy, and furniture interaction point generation strategy. The step of executing the determined interaction point generation strategy on the scene 3D model based on the strategy configuration information to generate interaction points in the scene 3D model includes: Each interaction point generation strategy is run independently to obtain the initial interaction point data corresponding to each interaction point generation strategy; The initial interaction point data are integrated to generate structured interaction point data; Interaction points are generated in the 3D model of the scene based on the structured interaction point data.

3. The method as described in claim 1, characterized in that, The interaction point generation strategy includes a room interaction point generation strategy, which includes: Based on the 3D model of the scene, determine the scene boundary of the target scene; Identify the walls contained in the target scene; Based on the scene boundary, the target scene is meshed to obtain multiple grid points; Ray detection is performed based on the grid points to remove the grid points corresponding to the wall from the grid points, thereby obtaining multiple effective surface points; Based on the effective surface points, the initial room is determined using a flood filling algorithm; Determine whether the initial room is a nested room; The initial room that is not a nested room is considered a valid room; Calculate the center of the effective room; and An interaction point is generated based on the center of the room.

4. The method as described in claim 3, characterized in that, Ray detection is performed based on the grid points to remove the grid points corresponding to the wall, resulting in multiple effective surface points, including: A virtual ray is emitted downwards from a predetermined distance above the grid points, and it is detected whether the virtual ray touches the ground; and The grid points corresponding to the virtual rays that hit the ground are taken as effective surface points.

5. The method as described in claim 3, characterized in that, Determining whether the initial room belongs to a nested room includes: Determine whether the initial room simultaneously satisfies a first judgment condition and a second judgment condition. The first judgment condition is that the boundary of the initial room is inside the boundary of any other initial room. The second judgment condition is any one of the following conditions: the area ratio of the initial room to any other initial room is less than a first ratio threshold; the area of ​​the initial room is less than an area threshold; the ratio of the area of ​​the initial room to the total area of ​​the scene is greater than a second ratio threshold; and the boundary of the initial room partially overlaps with the boundary of the scene.

6. The method as described in claim 1, characterized in that, The interaction point generation strategy includes a door interaction point generation strategy or a window interaction point generation strategy, wherein the door interaction point generation strategy or the window interaction point generation strategy includes: Based on the 3D model of the scene, identify the doors or windows contained in the target scene; Determine the door boundary of the door or the window boundary of the window based on the renderer component corresponding to the door or window; The center of the door is determined based on the door boundary, or the center of the window is determined based on the window boundary; and An interaction point is generated based on the center of the door or the center of the window.

7. The method as described in claim 1, characterized in that, The interaction point generation strategy includes a furniture interaction point generation strategy, which includes: Based on the 3D model of the scene, identify the furniture contained in the target scene; Determine the furniture boundary based on the renderer component corresponding to the furniture; Based on the furniture boundaries, determine the furniture center of each piece of furniture; and Interaction points are generated based on the furniture center.

8. The method as described in claim 7, characterized in that, The furniture interaction point generation strategy also includes: Based on the furniture boundaries, determine the diagonal length of the furniture; Based on the diagonal length, multiple candidate furniture items that are the same type as the furniture and similar in size to the furniture are searched from the database; Based on the similarity between each candidate furniture and the furniture in question, the candidate furniture is sorted in descending order of similarity. A replacement recommendation list for the furniture is obtained based on the top N candidate furniture items, where N > 0; and Output the replacement recommendation list.

9. An electronic device, characterized in that, include: The memory stores execution instructions; as well as A processor that executes the execution instructions stored in the memory, causing the processor to perform the scene interaction point generation method according to any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the scene interaction point generation method according to any one of claims 1 to 8.

Images