3D editing method and system based on mixed reality

Through the 3D editing method and system based on mixed reality, users can edit 3D models in the virtual world and view the effects on augmented reality device, solving the shortcomings in design solutions comparison and editing in the existing technology and achieving better design Solution optimization.

CN119068156APending Publication Date: 2024-12-03HANGZHOU ARCVIDEO TECHNOLOGY CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310634799.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In the prior art, designers cannot timely compare multiple design solutions after the design is completed, and virtualized scenarios cannot perform model editing or view editing effects in augmented reality device.

Method used

Provide a 3D editing method and system based on mixed reality. By acquiring image data, object detection, 3D modeling and editing, users can edit 3D models in the virtual world and display the edited effects on augmented reality devices. .

Benefits of technology

This allows users to freely edit models in virtualized scenarios, including addition, deletion, and movement, and view edited effects in augmented reality devices, simplifying the modification and optimization of design solutions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068156A_ABST
    Figure CN119068156A_ABST
Patent Text Reader

Abstract

The invention discloses a 3D editing method and system based on mixed reality. The method comprises the following steps: collecting image data; performing target detection on the acquired image data, identifying an object in the image, and giving a coordinate position corresponding to the object in the image; performing 3D modeling on the identified object according to the coordinate position of the object to form a 3D model of the object; editing the established 3D model in the virtual world, including operations of adding, deleting and modifying, and recording an edited result; and the edited 3D model is displayed on the augmented reality device, and the actual effect after editing is checked. According to the invention, a user can freely edit the 3D model of each real object in a virtualized scene.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of virtual reality, and in particular relates to a 3D editing method and system based on mixed reality. Background Art

[0002] In the prior art, in the design and planning of some things, after designers have designed the actual goals, they generally have to wait until the construction or installation is completed before they can see the actual design effect, which is not conducive to prioritizing comparison among multiple design schemes.

[0003] Moreover, the current scene virtualization technology generally constructs the virtualized scene on the PC side, and the model cannot be edited, or the edited effect cannot be viewed in the augmented reality device after editing. Summary of the invention

[0004] In view of the above problems, the present invention provides a 3D editing method and system based on mixed reality, so that users can freely edit the 3D models of various real objects in a virtualized scene.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] In one aspect, the present invention provides a 3D editing method based on mixed reality, comprising the following steps:

[0007] Collecting image data;

[0008] Perform target detection on the collected image data, identify the objects in the image, and give the coordinate positions corresponding to the objects in the image;

[0009] Perform 3D modeling on the identified object according to the coordinate position of the object to form a 3D model of the object;

[0010] Edit the established 3D model in the virtual world, including adding, deleting and modifying operations, and record the editing results;

[0011] The edited 3D model can be displayed on the augmented reality device to check the actual effect of the editing.

[0012] In a possible implementation, the object includes any one of a vehicle, a non-motor vehicle, a pedestrian, a pet, a roadblock, a building, and a store.

[0013] In one possible implementation, the augmented reality device is AR glasses.

[0014] In one possible implementation, the edited 3D model is displayed on an augmented reality device, and the actual effect after editing is viewed, which specifically includes: establishing a 3D space, displaying the edited 3D model content in the 3D space in sequence, the size of the object displayed in the 3D space is 1:1 with the size in the real world, running the 3D space on the augmented reality device for display, and viewing the actual effect after editing.

[0015] Another aspect of the present invention provides a 3D editing system based on mixed reality, comprising:

[0016] An image data acquisition module, used for acquiring image data;

[0017] The target detection module is used to perform target detection on the collected image data, identify the object in the image, and give the coordinate position corresponding to the object in the image;

[0018] A 3D modeling module is used to perform 3D modeling on the identified object according to the coordinate position of the object to form a 3D model of the object;

[0019] The 3D model editing module is used to edit the established 3D model in the virtual world, including operations of adding, deleting and modifying, and recording the editing results;

[0020] The augmented reality display module is used to display the edited 3D model on the augmented reality device to view the actual effect after editing.

[0021] In a possible implementation, the object includes any one of a vehicle, a non-motor vehicle, a pedestrian, a pet, a roadblock, a building, and a store.

[0022] In one possible implementation, the augmented reality device is AR glasses.

[0023] In one possible implementation, the edited 3D model is displayed on an augmented reality device, and the actual effect after editing is viewed, which specifically includes: establishing a 3D space, displaying the edited 3D model content in the 3D space in sequence, the size of the object displayed in the 3D space is 1:1 with the size in the real world, running the 3D space on the augmented reality device for display, and viewing the actual effect after editing.

[0024] The use of the present invention has the following beneficial effects: it allows users to freely edit various models in a virtualized scene, including adding, deleting, moving, etc., and displays the editing results in an augmented reality device, making it convenient for users to modify the design plan and obtain a better design plan at a lower cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1is a flowchart of the steps of a 3D editing method based on mixed reality according to an embodiment of the present invention;

[0026] Figure 2 It is a principle block diagram of a 3D editing system based on mixed reality according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] See also Figure 1 , which is a flowchart of a 3D editing method based on mixed reality according to an embodiment of the present invention, comprising the following steps:

[0029] S10, collecting image data;

[0030] S20, performing target detection on the collected image data, identifying the object in the image, and giving the coordinate position corresponding to the object in the image; the object includes any one of a vehicle, a non-motor vehicle, a pedestrian, a pet, a roadblock, a building, and a store.

[0031] S30, performing 3D modeling on the identified object according to the coordinate position of the object to form a 3D model of the object;

[0032] S40, editing the established 3D model in the virtual world, including operations of adding, deleting and modifying, and recording the editing result;

[0033] S50: Display the edited 3D model on an augmented reality device to view the actual effect of the editing. The augmented reality device may be an AR glasses.

[0034] In one embodiment of the present invention, S50, displaying the edited 3D model on an augmented reality device, and viewing the actual effect after editing specifically includes: establishing a 3D space, displaying the edited 3D model content in the 3D space in sequence, the size of the object displayed in the 3D space is 1:1 with the size in the real world, running the 3D space on the augmented reality device for display, and viewing the actual effect after editing.

[0035] In a specific application example, taking the simulation of urban construction as an example, the target objects to be detected include buildings, roads, parks, etc., and objects such as vehicles and pedestrians are not identified. Assuming that there is an empty space in the design area, you want to preview the effect of building a house or a park on the empty space. The augmented reality device takes AR glasses as an example:

[0036] Collecting image data;

[0037] Perform target detection on the collected image data to identify the objects that need to be detected in the image, mainly including buildings, roads, parks, open spaces, etc.

[0038] Calculate the relative position of the object (relative to the user's position coordinates);

[0039] Building a scene in the virtual world and modeling the detected objects and displaying them at the calculated positions;

[0040] Build a model of a high-rise building, place the model at the location of the open space, and record the model location A and model a1;

[0041] Establish a 3D space, and display the edited content in sequence in the 3D space according to the recorded edited content. Run the 3D space on AR glasses to view the effect of building a high-rise building on an empty lot;

[0042] Display the constructed virtual building on the AR glasses: enlarge it to the real world according to its actual size and place it at the location of the detected open space;

[0043] Build a park model again, place the model at the location of the open space, and record the model location A and model a2;

[0044] Build a 3D space, and display the edited content in the 3D space in sequence according to the recorded edited content. Run the 3D space on AR glasses to view the effect of building a park on the vacant land;

[0045] Display the constructed virtual building on the AR glasses: enlarge it to the real world according to its actual size and place it at the location of the detected open space;

[0046] Compare the effects by placing buildings or setting up gardens and choose the better option.

[0047] In another specific application example, suppose there is a street, and the shops along the street have chosen some signs of different shapes and styles. Some shops may not hang signs, which makes it look messy. Now, the technical solution of the present invention is implemented to select the appropriate sign style and color, so that the whole street can be more tidy. The target objects to be detected include buildings, shops, signs, etc. I want to see how to plan the placement of signs, how to select the style and color of signs, etc., to achieve the best effect:

[0048] Collecting corresponding image data;

[0049] Perform target detection on the collected image data. Identify objects in the image, mainly including buildings, stores, signs, etc.;

[0050] Calculate the relative position of the object (relative to the user's position coordinates);

[0051] Build the scene in the virtual world and model the detected objects and display them at the calculated positions;

[0052] Design and build different signboard models;

[0053] Edit the signboard in the original scene: delete, move, place the newly constructed signboard model to the position of the original signboard, etc., and record the position B1 of the original signboard, the position B2 of the new signboard, the model b1 of the original signboard, and the model b2 of the new signboard;

[0054] Establish a 3D space, display the edited content in sequence in the 3D space according to the recorded edited content, and run the 3D space on the AR glasses;

[0055] The size of the new sign in the virtual scene is designed according to the size of the old sign in the virtual scene. Therefore, in the AR glasses, the size of the new sign must also be scaled proportionally according to the size of the old sign, and the new sign is displayed in the appropriate position, so that users can directly see the effect of the new sign.

[0056] Check out the effects of different signs and find the most suitable sign style and color.

[0057] Through the above-mentioned mixed reality-based 3D editing method, users can freely edit various models in the virtual scene, including adding, deleting, moving, etc., and display the editing results in the augmented reality device, which is convenient for users to modify the design plan and obtain a better design plan at a lower cost.

[0058] Corresponding to the method embodiment of the present invention, a 3D editing system based on mixed reality according to an embodiment of the present invention includes: an image data acquisition module for acquiring image data; a target detection module for performing target detection on the acquired image data, identifying objects in the image, and giving the coordinate position corresponding to the object in the image; wherein the object includes any one of a vehicle, a non-motor vehicle, a pedestrian, a pet, a roadblock, a building, and a store. A 3D modeling module for performing 3D modeling on the identified object according to the coordinate position of the object to form a 3D model of the object; a 3D model editing module for editing the established 3D model in the virtual world, including operations of adding, deleting, and modifying, and recording the edited results; an augmented reality display module for displaying the edited 3D model on an augmented reality device to view the actual effect after editing. The augmented reality device may be AR glasses.

[0059] In one embodiment of the present invention, the edited 3D model is displayed on an augmented reality device, and viewing the actual effect after editing specifically includes: establishing a 3D space, displaying the edited 3D model content in the 3D space in sequence, the size of the object displayed in the 3D space is 1:1 with the size in the real world, running the 3D space on the augmented reality device for display, and viewing the actual effect after editing.

[0060] Through the above-mentioned mixed reality-based 3D editing system, users can freely edit various models in the virtual scene, including adding, deleting, moving, etc., and display the editing results in the augmented reality device, which is convenient for users to modify the design plan and obtain a better design plan at a lower cost.

[0061] It should be understood that the exemplary embodiments described herein are illustrative rather than restrictive. Although one or more embodiments of the present invention have been described in conjunction with the accompanying drawings, it should be understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A 3D editing method based on mixed reality, characterized in that: The following steps are involved: Collecting image data; Perform target detection on the collected image data, identify the objects in the image, and give the coordinate positions corresponding to the objects in the image; Perform 3D modeling on the identified object according to the coordinate position of the object to form a 3D model of the object; Edit the established 3D model in the virtual world, including adding, deleting and modifying operations, and record the editing results; The edited 3D model can be displayed on the augmented reality device to check the actual effect of the editing.

2. The 3D editing method based on mixed reality according to claim 1, characterized in that: The object includes any one of a vehicle, a non-motor vehicle, a pedestrian, a pet, a roadblock, a building, and a store.

3. The 3D editing method based on mixed reality according to claim 1, characterized in that: The augmented reality device is AR glasses.

4. The 3D editing method based on mixed reality according to claim 1, characterized in that: The edited 3D model is displayed on an augmented reality device to check the actual effect after editing, which specifically includes: establishing a 3D space, displaying the edited 3D model content in the 3D space in sequence, the size of the object displayed in the 3D space is 1:1 with the size in the real world, running the 3D space on an augmented reality device to display it, and checking the actual effect after editing.

5. A 3D editing system based on mixed reality, characterized in that: include: An image data acquisition module, used for acquiring image data; The target detection module is used to perform target detection on the collected image data, identify the object in the image, and give the coordinate position corresponding to the object in the image; A 3D modeling module is used to perform 3D modeling on the identified object according to the coordinate position of the object to form a 3D model of the object; The 3D model editing module is used to edit the established 3D model in the virtual world, including operations of adding, deleting and modifying, and recording the editing results; The augmented reality display module is used to display the edited 3D model on the augmented reality device to view the actual effect after editing.

6. The 3D editing system based on mixed reality as claimed in claim 1, characterized in that: The object includes any one of a vehicle, a non-motor vehicle, a pedestrian, a pet, a roadblock, a building, and a store.

7. The 3D editing system based on mixed reality as claimed in claim 1, characterized in that: The augmented reality device is AR glasses.

8. The 3D editing system based on mixed reality as claimed in claim 1, characterized in that: The edited 3D model is displayed on an augmented reality device to check the actual effect after editing, which specifically includes: establishing a 3D space, displaying the edited 3D model content in the 3D space in sequence, the size of the object displayed in the 3D space is 1:1 with the size in the real world, running the 3D space on an augmented reality device to display it, and checking the actual effect after editing.

Citation Information

Cited By

  • Three-dimensional model real-time editing method and device based on Unity platform and AR technology

    CN120236020A