Space planning design auxiliary method based on MR technology and digital display system

Abstract

The invention relates to an MR technology-based space planning design auxiliary method and a digital display system, and the method comprises the steps: carrying out the modeling of the content in image information in response to the obtained image information, and obtaining an incomplete model; constructing a complete model by using the incomplete model, wherein the complete model comprises a space model and a sub-model located in the space model; the complete model is modified according to a received instruction, the modification comprises deleting sub-models located in the space model and adding new sub-models in the space model, and when the sub-models located in the space model are deleted, the vacant part of the complete model is repaired according to the texture features of the space model or the related sub-models. According to the spatial planning design auxiliary method based on the MR technology and the digital display system, display can be carried out based on an existing scene, meanwhile, natural display of the modification process can be achieved, and better visualization and interactivity are provided.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a spatial planning and design auxiliary method and a digital display system based on MR technology. Background Art

[0002] MR technology is a technology that integrates the real world with the virtual world, and realizes the coexistence and real-time interaction of physical and digital objects by providing new viewing and input methods. MR technology can provide virtual-real fusion (virtual objects and the real world can be displayed in the same line of sight, realizing a visual experience of virtual-real integration), real-time interaction (users can interact naturally with the real world and virtual objects in real time, improving the realism and interactivity of the user experience) and three-dimensional registration (virtual objects can be accurately aligned with the real world to ensure seamless integration of virtual information and the real environment).

[0003] In the field of interior architectural design, MR technology can provide immersive design references. For example, scenes can be modified directly in the real world to assist in design. Compared with two-dimensional drawing design and three-dimensional model design, this method has more intuitive content output, especially in limited range changes, local changes and future scene displays.

[0004] However, in current actual operations, there are problems such as poor visualization and interactivity. One influencing factor is how to modify and display the process content. Summary of the invention

[0005] The present application provides a space planning and design assistance method and a digital display system based on MR technology, which can be displayed based on existing scenes, and can also achieve a natural display of the modification process, providing better visualization and interactivity.

[0006] The above-mentioned purpose of the present application is achieved through the following technical solutions:

[0007] In a first aspect, the present application provides a spatial planning and design assistance method based on MR technology, comprising:

[0008] In response to the acquired image information, modeling the content included in the image information to obtain an incomplete model;

[0009] Using the incomplete model to build a complete model, the complete model includes the spatial model and the sub-models located inside the spatial model;

[0010] Modify the complete model according to the received instructions, the modification including deleting the sub-models located inside the space model and adding new sub-models inside the space model;

[0011] When deleting a sub-model located inside the space model, the missing part of the complete model is repaired according to the texture features of the space model or the relevant sub-model.

[0012] In a possible implementation manner of the first aspect, using the incomplete model to construct a complete model includes:

[0013] Draw the sight line movement path based on the movement path and direction;

[0014] Selecting at least one sub-model or multiple texture areas on the sight line moving path as a transition between two incomplete models;

[0015] Extracting parts of the two incomplete models belonging to the at least one sub-model or the plurality of texture regions;

[0016] The texture regions belonging to the at least one sub-model or the plurality of texture regions in the two incomplete models are fused and the two incomplete models are fused at the same time.

[0017] In a possible implementation manner of the first aspect, selecting multiple texture areas on the sight line movement path includes:

[0018] Establishing a plurality of rectangular selection regions on the previous incomplete model of the sequential sequence and establishing analysis vector groups inside the rectangular regions;

[0019] Match any two analysis vector groups. If the match fails, move the rectangle selection area corresponding to one of the analysis vector groups until the two analysis vector groups are matched successfully.

[0020] The successfully matched rectangular selection area is used as a texture area;

[0021] A texture grid is constructed using a plurality of texture regions, wherein a size of the texture grid parallel to a line of sight moving path is not less than half of a size of the texture grid perpendicular to the line of sight moving path;

[0022] Wherein, the position of the texture area is adjusted or the texture area is regenerated according to the texture grid, and there is at least one texture area inside the maximum outer edge of the texture grid.

[0023] In a possible implementation manner of the first aspect, establishing the analysis vector group inside the rectangular area includes:

[0024] Perform grayscale processing on the rectangular area to obtain a grayscale rectangular area;

[0025] Select the characteristic lines in the grayscale rectangular area;

[0026] A vector group is established based on the characteristic lines, the vector group includes a plurality of vectors, and a starting point and an end point of a vector are respectively located on two characteristic lines;

[0027] When there is no characteristic line in the grayscale rectangular area, characteristic points on the grayscale rectangular area are selected and a vector group is established based on the characteristic points. The vector group includes multiple vectors, and the starting point and the end point of a vector are respectively located on two characteristic points.

[0028] In a possible implementation manner of the first aspect, repairing the missing part of the complete model according to the texture features of the spatial model or the related sub-model includes:

[0029] determining the missing region of the spatial model and the number of texture features around the missing region;

[0030] The missing area is divided and filled according to the spatial model characteristics of the location of the texture features.

[0031] In a possible implementation manner of the first aspect, when the missing area cannot be completely filled, the missing area is divided into a fillable area and an unfillable area, and the unfillable area is secondary filled, and the secondary filling includes:

[0032] Create fill guides based on the shape of the unfillable area;

[0033] Select multiple reference areas with content on the guide line, and the shape of the reference area is the same as the shape of the non-fillable area;

[0034] The reference areas on the same guide line are merged or selected to obtain a filling content;

[0035] Merge or select multiple primary filling contents to obtain secondary filling contents;

[0036] Use secondary filling content to fill the non-fillable area;

[0037] Among them, the guide lines belonging to the same unfillable area are located on the same plane;

[0038] When the unfillable area is located on multiple planes at the same time, the unfillable area is divided according to the boundary lines of the multiple planes.

[0039] In a possible implementation manner of the first aspect, fusing or selecting reference areas on the same guide line includes:

[0040] Evaluate the similarity of any two reference areas. When the reference areas on the same guide line are similar, they are fused. Otherwise, the reference areas are grouped according to their similarity. Then, a reference area is selected from the group with the largest number of reference areas as the primary filling content.

[0041] When performing fusion processing, two reference areas are selected and adjusted. The adjustment amounts of the two reference areas are the same and the two reference areas completely overlap after adjustment. The above process is repeated until only one reference area remains.

[0042] When adjusting the reference area, it is allowed to adjust by region;

[0043] Allows reference regions to move along guide lines to adjust similarity with other reference regions.

[0044] In a second aspect, the present application provides a spatial planning design auxiliary analysis device based on MR technology, comprising:

[0045] A first modeling unit, configured to model the content included in the image information in response to the acquired image information to obtain an incomplete model;

[0046] A second modeling unit is used to construct a complete model using the incomplete model, wherein the complete model includes a spatial model and a sub-model located inside the spatial model;

[0047] A model modification unit modifies the complete model according to the received instructions, wherein the modification includes deleting the sub-models located inside the space model and adding new sub-models inside the space model;

[0048] When deleting a sub-model located inside the space model, the missing part of the complete model is repaired according to the texture features of the space model or the relevant sub-model.

[0049] In a third aspect, the present application provides a space planning and design assistance method and a digital display system based on MR technology, the system comprising:

[0050] one or more memories for storing instructions; and

[0051] One or more processors, used to call and run the instructions from the memory to execute the method as described in the first aspect and any possible implementation of the first aspect.

[0052] In a fourth aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium comprising:

[0053] Program, when the program is executed by a processor, the method described in the first aspect and any possible implementation of the first aspect is executed.

[0054] In a fifth aspect, the present application provides a computer program product, comprising program instructions. When the program instructions are executed by a computing device, the method described in the first aspect and any possible implementation of the first aspect is executed.

[0055] In a sixth aspect, the present application provides a chip system, which includes a processor for implementing the functions involved in the above aspects, for example, generating, receiving, sending, or processing the data and / or information involved in the above methods.

[0056] The chip system may be composed of chips, or may include chips and other discrete devices.

[0057] In a possible design, the chip system also includes a memory, which is used to store necessary program instructions and data. The processor and the memory can be decoupled and respectively set on different devices, connected by wired or wireless means, or the processor and the memory can also be coupled on the same device. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic flowchart of the steps of a space planning and design auxiliary method provided in this application.

[0059] Figure 2 It is a schematic diagram of an incomplete model provided by this application.

[0060] Figure 3 This is a schematic diagram of the fusion of two incomplete models provided by this application.

[0061] Figure 4 It is a schematic diagram of establishing a rectangular selection area provided by this application.

[0062] Figure 5 This is a schematic diagram for establishing a filling guide line provided in this application.

[0063] Figure 6 It is a schematic diagram of a process for processing multiple reference areas provided by the present application.

[0064] Figure 7 This is another schematic diagram of a process for processing multiple reference areas provided by the present application. DETAILED DESCRIPTION

[0065] The technical solution in this application is further described in detail below in conjunction with the accompanying drawings.

[0066] This application discloses a spatial planning and design assistance method based on MR technology. Figure 1 In some examples, the space planning and design assistance method based on MR technology disclosed in this application includes the following steps:

[0067] S101, in response to the acquired image information, modeling the content included in the image information to obtain an incomplete model;

[0068] S102, using the incomplete model to construct a complete model, where the complete model includes a spatial model and a sub-model located inside the spatial model;

[0069] S103, modifying the complete model according to the received instruction, wherein the modification includes deleting the sub-model located inside the space model and adding a new sub-model inside the space model;

[0070] When deleting a sub-model located inside the space model, the missing part of the complete model is repaired according to the texture features of the space model or the relevant sub-model.

[0071] First of all, it needs to be explained that the space planning and design auxiliary method based on MR technology disclosed in this application needs to be implemented with the help of MR equipment. The MR equipment generates an image after acquiring the surrounding environment, and then modifies the image according to the received instructions to achieve the overlap of virtual and reality.

[0072] In step S101, in response to the acquired image information, the content included in the image information is firstly modeled to obtain an incomplete model ( Figure 2 The incomplete model here is the complete model ( Figure 2 A part of a cube in .

[0073] Because the MR device provides the user's field of view, the incomplete model represents a part of the complete model obtained based on the user's field of view. As the user's field of view moves, the number of incomplete models will increase, and step S102 is executed at this time.

[0074] In step S102, a complete model is constructed using the incomplete model. The complete model consists of two parts: a space model and a sub-model located inside the space model. The space model refers to a building, and the sub-model inside the space model refers to objects placed in the building.

[0075] Here we briefly introduce the modeling of the content included in the image information. In some possible implementations, the processing method used is multi-view stereo vision. The principle of multi-view stereo vision is to take multiple pictures of the same object from different angles, and obtain a three-dimensional point cloud through feature matching, parallax calculation, depth optimization and other processes, and then further generate a mesh model.

[0076] The binocular vision algorithm can be used to determine the distance. This method can be explained by referring to the working method of the human eye. The image information received by two human eyes is different. The distance can be calculated based on the difference in the two image information.

[0077] Finally, in step S103, the complete model is modified according to the received instructions. There are two ways of modification, namely, deleting the sub-model located inside the space model and adding a new sub-model inside the space model. Deleting the sub-model located inside the space model means removing an object inside the building, and adding a new sub-model inside the space model means adding a new object inside the building.

[0078] The process of adding new objects to a building is relatively simple because it can be calculated through direct occlusion relationships. However, removing an object from a building is difficult because it involves repairing the spatial model or related sub-models.

[0079] For example, when removing decorations on the wall, the part blocked by the decorations needs to be repaired at the same time. Because removing decorations on the wall is a virtual process, repairing the part blocked by the decorations is also a virtual process, that is, modifying the real content of the MR device through technical means.

[0080] This is a technical difficulty. In addition, there are also difficulties in using an incomplete model to build a complete model. The difficulty lies in the matching of the incomplete model. When the incomplete model has obvious features, it is easy to match, but when the features of the incomplete model are not obvious, additional algorithms are required for matching.

[0081] In addition, it is also necessary to consider that the movement of the MR device is unstable. For example, for some areas that are quickly skipped, obtaining an incomplete model but displaying the real scene is a better solution. In this case, the incomplete model obtained needs to be transferred to the background for matching. In addition, it is also necessary to consider the possible inconsistency in obtaining the incomplete model, and the matching of the incomplete model cannot be simply based on the positional relationship.

[0082] In some examples, the specific ways to use the incomplete model to build the complete model are:

[0083] S201, drawing a sight line movement path based on the movement path and direction;

[0084] S202, selecting at least one sub-model or multiple texture areas on the sight line moving path as a transition between two incomplete models;

[0085] S203, extracting parts of the two incomplete models belonging to the at least one sub-model or the multiple texture regions;

[0086] S204, fusing the at least one sub-model or multiple texture regions in the two incomplete models and fusing the two incomplete models at the same time.

[0087] In step S201 to step S204, the incomplete models will be fused based on the line of sight movement path. The specific method is to select at least one sub-model or multiple texture areas on the line of sight movement path as a transition between two incomplete models, and then extract the parts of the two incomplete models belonging to the at least one sub-model or multiple texture areas, and finally fuse the parts of the two incomplete models belonging to the at least one sub-model or multiple texture areas and fuse the two incomplete models at the same time.

[0088] This method uses sub-models or texture areas. The advantage of sub-models is that they are easy to match, such as Figure 3 As shown, for example, a sofa appears in two incomplete models at the same time. At this time, the two parts of the sofa (respectively located in the two incomplete models) are spliced ​​together to merge the two incomplete models.

[0089] Texture regions are mainly used in situations where fusion cannot be achieved through sub-models. Using texture regions involves two processes: selection and matching.

[0090] In some examples, multiple texture regions are selected along the eye movement path as follows:

[0091] S301, establishing a plurality of rectangular selection regions on a previous incomplete model of the sequential sequence and establishing analysis vector groups inside the rectangular regions;

[0092] S302, matching any two analysis vector groups, and if the matching fails, selecting a moving rectangle corresponding to one of the analysis vector groups until the two analysis vector groups are matched successfully;

[0093] S303, taking the successfully matched rectangular selection area as a texture area;

[0094] S304, constructing a texture grid using the plurality of texture regions, wherein the size of the texture grid parallel to the line of sight moving path is not less than half of the size of the texture grid perpendicular to the line of sight moving path;

[0095] Wherein, the position of the texture area is adjusted or the texture area is regenerated according to the texture grid, and there is at least one texture area inside the maximum outer edge of the texture grid.

[0096] In step S301 to step S304, firstly, a plurality of rectangular selection areas ( Figure 4 As shown) and establish an analysis vector group inside the rectangular area, then match any two analysis vector groups, if the match is unsuccessful, move the rectangular selection area corresponding to one of the analysis vector groups until the two analysis vector groups are matched successfully, at this time, use the successfully matched rectangular selection area as a texture area and use multiple texture areas to construct a texture grid.

[0097] In the above process, the purpose of matching the analysis vector group is to find the same or similar area on the previous incomplete model of the sequential sequence. The advantage of this method is that in the subsequent comparison process, any texture area can be used to select on the next incomplete model of the sequential sequence.

[0098] At the same time, in order to improve the accuracy of matching, it is also necessary to use multiple texture areas to construct a texture mesh. The purpose is to ensure that the distribution of texture areas is regular. Because when the same texture area appears on two incomplete models at the same time, the two texture areas will be overlapped and the corresponding two incomplete models will be fused together.

[0099] For the texture grid, the size of the texture grid parallel to the line of sight movement path needs to be no less than half of the size of the texture grid perpendicular to the line of sight movement path. In addition, the position of the texture area needs to be adjusted or the texture area needs to be regenerated according to the texture grid, and there is at least one texture area inside the maximum outer edge of the texture grid.

[0100] The above methods are all for ensuring that the texture grid has better matching performance, because compared with using only one texture grid or multiple texture grids on the same straight line, the distribution method of texture grids to form texture areas is more difficult to match. If the match is successful, it also means that the matching of the two incomplete models has higher accuracy.

[0101] The way to establish the analysis vector group inside the rectangular area is as follows:

[0102] Perform grayscale processing on the rectangular area to obtain a grayscale rectangular area;

[0103] Select the characteristic lines in the grayscale rectangular area;

[0104] A vector group is established based on the characteristic lines, the vector group includes a plurality of vectors, and a starting point and an end point of a vector are respectively located on two characteristic lines;

[0105] When there is no characteristic line in the grayscale rectangular area, characteristic points on the grayscale rectangular area are selected and a vector group is established based on the characteristic points. The vector group includes multiple vectors, and the starting point and the end point of a vector are respectively located on two characteristic points.

[0106] The above method is to establish a vector with the help of feature lines or feature points. The feature lines and feature points in the grayscale rectangular area can be distinguished by length. The feature lines with a length greater than the set length are feature lines, and vice versa.

[0107] The color of feature lines and feature points should be significantly darker or lighter than the surrounding area.

[0108] When a vector is established using feature lines, the starting point and end point of a vector are located on two feature lines respectively. At this time, the vector has a clear starting point, end point and direction, and the values ​​(grayscale values) at the starting point and end point are fixed.

[0109] When the color of the feature lines is uniform, the vector is established by fixing the starting point distance and direction. The starting point distance and direction of the two vectors are fixed values, and the end point of the vector is an uncertain value.

[0110] When a vector is established using feature points, the starting point and end point of a vector are located on two feature lines respectively. At this time, the vector has a clear starting point, end point and direction, and the values ​​(grayscale values) at the starting point and end point are fixed.

[0111] When the color of the feature points is uniform, the vector is established by fixing the starting point distance and direction. The starting point distance and direction of the two vectors are fixed values, and the end point of the vector is an uncertain value.

[0112] When matching two incomplete models, first use the vector group to find the texture area on the next incomplete model in the sequential sequence, and then use the found texture area to construct the texture area. If the same texture area can be obtained, the two incomplete models are fused. The fusion method is to move the texture area while moving the incomplete model until the two incomplete models completely overlap.

[0113] The way to match any two analysis vector groups is to put the vectors in the two analysis vector groups into a coordinate system to see if they overlap. The overlap judgment criteria include angle difference and length difference. The difference calculation method at this time is as follows:

[0114] Difference = angle difference*P+length difference*Q, P+Q+1;

[0115] When the difference is less than the set value, the two vectors are considered to overlap. When the ratio of the number of vectors (overlapping) in any analysis group to the total number of vectors is greater than or equal to the set value, the two analysis vector groups are considered to match. During the matching process, all vectors belonging to an analysis vector group are allowed to be enlarged, reduced, moved and rotated to achieve the purpose of reducing the difference.

[0116] When searching for a texture region on a subsequent incomplete model in a sequential sequence, any texture region on a previous incomplete model in the sequential sequence may be used.

[0117] In some examples, the missing parts of the complete model are repaired according to the texture features of the spatial model or the related sub-model as follows:

[0118] S401, determining the missing region of the spatial model and the number of texture features around the missing region;

[0119] S402, dividing and filling the missing area according to the spatial model features of the locations where the texture features are located.

[0120] Specifically, the missing area is first determined, and then the number of texture features around the missing area is determined. Then, the spatial model features of the locations of the texture features are used to divide and fill the missing area. At this time, there will be two situations: the missing area can be completely filled or cannot be completely filled.

[0121] At this time, there may be a sub-model in the missing area of ​​the spatial model. This incomplete sub-model needs to be separated and completed, and then placed in the original position.

[0122] The spatial model features at the location of the texture features are used to divide the missing area. The specific process is as follows: first, the texture features are used to determine an area, and then the shape of the area is determined, such as a rectangle and a circle, and then the graphics are completed, and the texture features are assigned to the completed graphics. When the texture features are assigned to the completed graphics, the existing part of the graphics is copied, and then a smooth transition is performed at the junction.

[0123] However, the missing area may not be completely filled. In this case, the missing area is divided into fillable areas and non-fillable areas, and the non-fillable areas are filled twice. The method of secondary filling is as follows:

[0124] S501, establishing a filling guide line according to the shape of the non-fillable area;

[0125] S502, selecting a plurality of reference areas with contents on the guide line, wherein the shapes of the reference areas are the same as the shapes of the non-fillable areas;

[0126] S503, merging or selecting the reference areas on the same guide line to obtain a filling content;

[0127] S504, merging or selecting a plurality of primary filling contents to obtain secondary filling contents;

[0128] S505, filling the non-fillable area with secondary filling content;

[0129] Among them, the guide lines belonging to the same unfillable area are located on the same plane;

[0130] When the unfillable area is located on multiple planes at the same time, the unfillable area is divided according to the boundary lines of the multiple planes.

[0131] In step S501 to step S505, please refer to Figure 5, fill guide lines will be established according to the shape of the unfillable area. Specifically, the fill guide lines are determined according to the number of edges of the unfillable area, and a fill guide line is set on each edge of the unfillable area.

[0132] Then select multiple reference areas with content on the guide line. The shape of the reference area is the same as the shape of the non-fillable area. There are two ways to process the obtained reference area. Figure 6 The first method is to fuse the reference areas on the same guide line. Figure 7 The second method is to select one of the reference areas on the same guide line. In this case, one filling content is obtained, and the number of filling contents is multiple.

[0133] Then multiple primary filling contents are merged or selected to obtain secondary filling contents, and finally the non-fillable areas are filled with the secondary filling contents.

[0134] At this time, the unfillable area may be located on multiple planes, such as a corner. Therefore, when the unfillable area is located on multiple planes at the same time, the unfillable area is divided according to the boundary lines of the multiple planes, and the guide lines belonging to the same unfillable area are required to be on the same plane. In other words, only the same plane is required to be filled, and cross-plane filling is not allowed.

[0135] The specific method of fusing or selecting the reference areas on the same guide line is:

[0136] Evaluate the similarity of any two reference areas. When the reference areas on the same guide line are similar, they are fused. Otherwise, the reference areas are grouped according to their similarity. Then, a reference area is selected from the group with the largest number of reference areas as the primary filling content. When grouping cannot be achieved, move the position of the reference area until the grouping requirements are met.

[0137] When performing fusion processing, two reference areas are selected and adjusted. The adjustment amounts of the two reference areas are the same and the two reference areas completely overlap after adjustment. The above process is repeated until only one reference area remains.

[0138] When adjusting the reference area, it is allowed to adjust by region;

[0139] Allows reference regions to move along guide lines to adjust similarity with other reference regions.

[0140] In step S601 to step S604, it is necessary to first evaluate the similarity between any two reference regions. There are two situations:

[0141] When the reference areas on the same guide line are similar, fusion processing is performed;

[0142] When not all reference areas on the same guide line are similar, the reference areas are grouped according to their similarity, and then one reference area is selected from a group with the largest number of reference areas as a filling content.

[0143] The way to fuse the reference areas is to select two reference areas and adjust them. The adjustment amounts of the two reference areas are the same and the two reference areas completely overlap after the adjustment. The above process is repeated until only one reference area is left. During the process, the reference area can be adjusted in different areas, and the reference area can be moved along the guide line to adjust the similarity with other reference areas.

[0144] In the above manner, a similar evaluation criterion is that after the two reference areas are overlapped, the ratio of the overlapping area to the area of ​​the reference area is greater than or equal to a set value.

[0145] The present application also provides a space planning design auxiliary analysis device based on MR technology, comprising:

[0146] A first modeling unit, configured to model the content included in the image information in response to the acquired image information to obtain an incomplete model;

[0147] A second modeling unit is used to construct a complete model using the incomplete model, wherein the complete model includes a spatial model and a sub-model located inside the spatial model;

[0148] A model modification unit modifies the complete model according to the received instructions, wherein the modification includes deleting the sub-models located inside the space model and adding new sub-models inside the space model;

[0149] When deleting a sub-model located inside the space model, the missing part of the complete model is repaired according to the texture features of the space model or the relevant sub-model.

[0150] Furthermore, it also includes:

[0151] A sight line movement path establishing unit, used for drawing a sight line movement path based on the movement path and the direction;

[0152] A first selection unit, configured to select at least one sub-model or a plurality of texture regions on a sight line moving path as a transition between two incomplete models;

[0153] An extraction unit, used for extracting parts belonging to the at least one sub-model or the plurality of texture regions in the two incomplete models;

[0154] The fusion unit is used to fuse the texture regions belonging to the at least one sub-model or the multiple texture regions in the two incomplete models and to fuse the two incomplete models at the same time.

[0155] Furthermore, it also includes:

[0156] An analysis vector group establishment unit is used to establish a plurality of rectangular selection areas on a previous incomplete model of the sequential sequence and to establish an analysis vector group inside the rectangular areas;

[0157] The analysis vector group matching unit is used to match any two analysis vector groups. If the matching fails, the moving rectangle selection area corresponding to one of the analysis vector groups is selected until the two analysis vector groups are matched successfully.

[0158] A selection unit, used to select a successfully matched rectangular selection area as a texture area;

[0159] A texture grid construction unit, used to construct a texture grid using a plurality of texture regions, wherein a size of the texture grid parallel to a line of sight moving path is not less than half of a size of the texture grid perpendicular to the line of sight moving path;

[0160] Wherein, the position of the texture area is adjusted or the texture area is regenerated according to the texture grid, and there is at least one texture area inside the maximum outer edge of the texture grid.

[0161] Furthermore, it also includes:

[0162] The grayscale processing unit is used to process the grayscale of the rectangular area to obtain a grayscale rectangular area;

[0163] A first selection unit, used for selecting characteristic lines in a grayscale rectangular area;

[0164] A first vector group building unit is used to build a vector group based on the characteristic lines, the vector group includes a plurality of vectors, and the starting point and the end point of a vector are respectively located on two characteristic lines;

[0165] The second vector group construction unit is used to select feature points on the grayscale rectangular area and establish a vector group based on the feature points when there are no feature lines in the grayscale rectangular area. The vector group includes multiple vectors, and the starting point and end point of a vector are respectively located on two feature points.

[0166] Furthermore, it also includes:

[0167] A quantity determination unit, used for determining the quantity of texture features in the missing region of the spatial model and around the missing region;

[0168] The first processing unit is used to divide and fill the missing area according to the spatial model features of the positions where the texture features are located.

[0169] Furthermore, it also includes:

[0170] Establishing a unit for establishing a filling guide line according to the shape of the non-fillable area;

[0171] A second selection unit is used to select a plurality of reference areas with contents on the guide line, wherein the shapes of the reference areas are the same as the shapes of the non-fillable areas;

[0172] The second processing unit is used to fuse or select the reference areas on the same guide line to obtain a filling content;

[0173] A third processing unit is used to merge or select a plurality of primary filling contents to obtain secondary filling contents;

[0174] A first filling unit is used to fill the non-fillable area with secondary filling content;

[0175] Among them, the guide lines belonging to the same unfillable area are located on the same plane;

[0176] When the unfillable area is located on multiple planes at the same time, the unfillable area is divided according to the boundary lines of the multiple planes.

[0177] Furthermore, it also includes:

[0178] The similarity evaluation unit is used to evaluate the similarity of any two reference areas. When the reference areas on the same guide line are similar, they are fused. Otherwise, the reference areas are grouped according to the similarity, and then a reference area is selected from the group with the largest number of reference areas as the primary filling content.

[0179] A fourth processing unit is used to select two reference areas and adjust them when performing fusion processing, the adjustment amounts of the two reference areas are the same and the two reference areas completely overlap after adjustment, and the above process is repeated until only one reference area remains;

[0180] When adjusting the reference area, it is allowed to adjust by region;

[0181] Allows reference regions to move along guide lines to adjust similarity with other reference regions.

[0182] In one example, the unit in any of the above devices can be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0183] For another example, when the units in the device can be implemented in the form of a processing element scheduling program, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0184] Various objects such as various messages / information / equipment / network elements / systems / devices / actions / operations / processes / concepts that may appear in this application are named. It can be understood that these specific names do not constitute a limitation on the relevant objects. The names assigned may change with factors such as scenarios, contexts or usage habits. The understanding of the technical meaning of the technical terms in this application should be mainly determined from the functions and technical effects embodied / executed in the technical scheme.

[0185] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0186] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0187] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0188] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0189] It should also be understood that in various embodiments of the present application, the first, second, etc. are only used to indicate that multiple objects are different. For example, the first time window and the second time window are only used to indicate different time windows. They should not have any impact on the time window itself, and the first, second, etc. mentioned above should not impose any limitations on the embodiments of the present application.

[0190] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0191] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a computer-readable storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0192] The present application also provides a space planning and design auxiliary method and a digital display system based on MR technology, the system comprising:

[0193] one or more memories for storing instructions; and

[0194] One or more processors are used to call and run the instructions from the memory to execute the method as described above.

[0195] The present application also provides a computer program product, which includes instructions. When the instructions are executed, the terminal device and the network device perform operations of the terminal device and the network device corresponding to the above method.

[0196] The present application also provides a chip system, which includes a processor for implementing the functions involved in the above content, such as generating, receiving, sending, or processing the data and / or information involved in the above method.

[0197] The chip system may be composed of chips, or may include chips and other discrete devices.

[0198] The processor mentioned in any of the above places can be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for executing programs for controlling the above-mentioned feedback information transmission method.

[0199] In a possible design, the chip system also includes a memory, which is used to store necessary program instructions and data. The processor and the memory can be decoupled and respectively set on different devices, connected by wire or wireless means to support the chip system to implement various functions in the above embodiments. Alternatively, the processor and the memory can also be coupled on the same device.

[0200] Optionally, the computer instructions are stored in a memory.

[0201] Optionally, the memory is a storage unit within the chip, such as a register, a cache, etc. The memory can also be a storage unit within the terminal located outside the chip, such as a ROM or other types of static storage devices that can store static information and instructions, RAM, etc.

[0202] It can be understood that the memory in the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0203] The non-volatile memory may be a ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory.

[0204] The volatile memory may be a RAM, which is used as an external cache. There are many different types of RAM, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory.

[0205] The embodiments of this specific implementation method are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Therefore, all equivalent changes made based on the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A spatial planning and design auxiliary method based on MR technology, characterized in that: include: In response to the acquired image information, modeling the content included in the image information to obtain an incomplete model; Using the incomplete model to build a complete model, the complete model includes the spatial model and the sub-models located inside the spatial model; Modify the complete model according to the received instructions, the modification including deleting the sub-models located inside the space model and adding new sub-models inside the space model; When deleting a sub-model located inside the space model, the missing part of the complete model is repaired according to the texture features of the space model or the relevant sub-model.

2. The space planning and design auxiliary method based on MR technology according to claim 1 is characterized in that: Using an incomplete model to build a complete model includes: Draw the sight line movement path based on the movement path and direction; Selecting at least one sub-model or multiple texture areas on the sight line moving path as a transition between two incomplete models; Extracting parts of the two incomplete models belonging to the at least one sub-model or the plurality of texture regions; The texture regions belonging to the at least one sub-model or the plurality of texture regions in the two incomplete models are fused and the two incomplete models are fused at the same time.

3. The space planning and design auxiliary method based on MR technology according to claim 2 is characterized in that: Selecting multiple texture areas along the eye movement path includes: Establishing a plurality of rectangular selection regions on the previous incomplete model of the sequential sequence and establishing analysis vector groups inside the rectangular regions; Match any two analysis vector groups. If the match fails, move the rectangle selection area corresponding to one of the analysis vector groups until the two analysis vector groups are matched successfully. The successfully matched rectangular selection area is used as a texture area; A texture grid is constructed using a plurality of texture regions, wherein a size of the texture grid parallel to a line of sight moving path is not less than half of a size of the texture grid perpendicular to the line of sight moving path; Wherein, the position of the texture area is adjusted or the texture area is regenerated according to the texture grid, and there is at least one texture area inside the maximum outer edge of the texture grid.

4. The space planning and design auxiliary method based on MR technology according to claim 2 is characterized in that: Establishing the analysis vector group inside the rectangular area includes: Perform grayscale processing on the rectangular area to obtain a grayscale rectangular area; Select the characteristic lines in the grayscale rectangular area; A vector group is established based on the characteristic lines, the vector group includes a plurality of vectors, and a starting point and an end point of a vector are respectively located on two characteristic lines; When there is no characteristic line in the grayscale rectangular area, characteristic points on the grayscale rectangular area are selected and a vector group is established based on the characteristic points. The vector group includes multiple vectors, and the starting point and the end point of a vector are respectively located on two characteristic points.

5. The space planning and design auxiliary method based on MR technology according to claim 1, characterized in that: Repairing the missing parts of the complete model based on the texture features of the spatial model or the related sub-model includes: determining the missing region of the spatial model and the number of texture features around the missing region; The missing area is divided and filled according to the spatial model characteristics of the location of the texture features.

6. The space planning and design auxiliary method based on MR technology according to claim 5 is characterized in that: When the missing area cannot be completely filled, the missing area is divided into a fillable area and an unfillable area, and the unfillable area is filled twice. The secondary filling includes: Create fill guides based on the shape of the unfillable area; Select multiple reference areas with content on the guide line, and the shape of the reference area is the same as the shape of the non-fillable area; The reference areas on the same guide line are merged or selected to obtain a filling content; Merge or select multiple primary filling contents to obtain secondary filling contents; Use secondary filling content to fill the non-fillable area; Among them, the guide lines belonging to the same unfillable area are located on the same plane; When the unfillable area is located on multiple planes at the same time, the unfillable area is divided according to the boundary lines of the multiple planes.

7. The space planning and design auxiliary method based on MR technology according to claim 6 is characterized in that: The reference areas on the same guide line are merged or selected, including: Evaluate the similarity of any two reference areas. When the reference areas on the same guide line are similar, they are fused. Otherwise, the reference areas are grouped according to their similarity. Then, a reference area is selected from the group with the largest number of reference areas as the primary filling content. When performing fusion processing, two reference areas are selected and adjusted. The adjustment amounts of the two reference areas are the same and the two reference areas completely overlap after adjustment. The above process is repeated until only one reference area remains. When adjusting the reference area, it is allowed to adjust by region; Allows reference regions to move along guide lines to adjust similarity with other reference regions.

8. A spatial planning design auxiliary analysis device based on MR technology, characterized in that: include: A first modeling unit, configured to model the content included in the image information in response to the acquired image information to obtain an incomplete model; A second modeling unit is used to construct a complete model using the incomplete model, wherein the complete model includes a spatial model and a sub-model located inside the spatial model; A model modification unit modifies the complete model according to the received instructions, wherein the modification includes deleting the sub-models located inside the space model and adding new sub-models inside the space model; When deleting a sub-model located inside the space model, the missing part of the complete model is repaired according to the texture features of the space model or the relevant sub-model.

9. A space planning design auxiliary method and digital display system based on MR technology, characterized in that: The system comprises: one or more memories for storing instructions; and One or more processors, configured to call and execute the instructions from the memory to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer readable storage medium comprises: The program, when the program is executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images