A spatial planning design auxiliary method and digital display system based on MR technology

Abstract

The application relates to a space planning design auxiliary method and a digital display system based on MR technology, the method comprising the following steps: modeling the content included in image information in response to the obtained image information, to obtain a residual model; constructing a complete model by using the residual model, the complete model comprising a space model and a submodel located in the space model; modifying the complete model according to received instructions, the modification comprising deleting the submodel located in the space model and adding a new submodel in the space model, and when the submodel located in the space model is deleted, the missing part of the complete model is repaired according to the texture characteristics of the space model or the related submodel. The space planning design auxiliary method and the digital display system based on MR technology disclosed by the application can display based on an existing scene, and meanwhile, the naturalized display of the modification process can be realized, and better visualization and interactivity are provided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a space planning design auxiliary method based on MR technology and a digital display system. BACKGROUND

[0002] MR technology is a technology that fuses the real world with the virtual world, and realizes the coexistence and real-time interaction of physical and digital objects by providing new viewing and input methods. MR technology can provide virtual-real fusion (virtual objects and the real world can be displayed in the same line of sight to achieve a visual experience of virtual-real combination), real-time interaction (users can interact with the real world and virtual objects in real time to improve the realism and interactivity of user experience) and three-dimensional registration (virtual objects can be accurately aligned with the real world to ensure seamless fusion of virtual information and the real environment).

[0003] In the field of indoor architectural design, MR technology can provide immersive design references, such as directly modifying scenes in the real world for auxiliary design. Compared with two-dimensional paper design and three-dimensional model design, this approach has more intuitive content output, especially in the fields of limited range changes, local changes and future scene display.

[0004] However, in current actual operations, there are problems such as poor visualization and interactivity, and one of the influencing factors is how to modify and process the content of the process. SUMMARY

[0005] The present application provides a space planning design auxiliary method based on MR technology and a digital display system, which can display based on existing scenes, and can also realize natural display of the modification process, providing better visualization and interactivity.

[0006] The above object of the present application is achieved by the following technical solution:

[0007] In a first aspect, the present application provides a space planning design auxiliary method based on MR technology, comprising:

[0008] In response to the acquired image information, modeling the content included in the image information to obtain a defective model;

[0009] Using the defective model to construct a complete model, the complete model including a space model and a sub-model located inside the space model;

[0010] Modifying the complete model according to the received instructions, the modification including deleting the sub-model located inside the space model and adding a new sub-model inside the space model;

[0011] When a sub-model inside the space model is deleted, the missing part of the complete model is repaired according to the texture features of the space model or the related sub-model.

[0012] In a possible implementation manner of the first aspect, the constructing the complete model using the incomplete models comprises:

[0013] drawing a line-of-sight movement path based on the movement path and the orientation;

[0014] selecting at least one sub-model or a plurality of texture regions on the line-of-sight movement path as a transition between two incomplete models;

[0015] extracting parts belonging to the at least one sub-model or the plurality of texture regions in the two incomplete models;

[0016] fusing the parts belonging to the at least one sub-model or the plurality of texture regions in the two incomplete models and fusing the two incomplete models at the same time.

[0017] In a possible implementation manner of the first aspect, the selecting a plurality of texture regions on the line-of-sight movement path comprises:

[0018] establishing a plurality of rectangular selection regions on a previous incomplete model in the sequence and establishing a plurality of analysis vector groups in the rectangular selection regions;

[0019] matching any two analysis vector groups, and when the matching is unsuccessful, a movement rectangular selection region corresponding to one of the analysis vector groups, until the two analysis vector groups are successfully matched;

[0020] taking the successfully matched rectangular selection region as a texture region;

[0021] constructing a texture grid using the plurality of texture regions, and a size of the texture grid in a direction parallel to the line-of-sight movement path is not less than one half of a size of the texture grid in a direction perpendicular to the line-of-sight movement path;

[0022] wherein the position of the texture region is adjusted or the texture region is regenerated according to the texture grid, and there is at least one texture region inside a maximum outer edge of the texture grid.

[0023] In a possible implementation manner of the first aspect, the establishing a plurality of analysis vector groups in the rectangular selection region comprises:

[0024] gray processing the rectangular region to obtain a gray rectangular region;

[0025] selecting a feature line in the gray rectangular region;

[0026] establishing a vector group based on the feature line, the vector group comprising a plurality of vectors, and a start point and an end point of one vector being located on two feature lines respectively;

[0027] When no feature line exists in the gray-scale rectangular region, a feature point on the gray-scale rectangular region is selected, and a vector group is established based on the feature point, the vector group including a plurality of vectors, and a start point and an end point of one vector being located on two feature points respectively.

[0028] In a possible implementation of the first aspect, repairing the missing part of the complete model according to the texture features of the space model or the related sub-model includes:

[0029] Determining the number of texture features around the missing region of the space model and the missing region;

[0030] Dividing and filling the missing region according to the space model features of the positions of the texture features.

[0031] In a possible implementation of the first aspect, when the missing region cannot be completely filled, the missing region is divided into a fillable region and a non-fillable region, the non-fillable region is secondarily filled, and the secondary filling includes:

[0032] Establishing a filling guide line according to the shape of the non-fillable region;

[0033] Selecting a plurality of reference regions with content on the guide line, the reference regions having the same shape as the non-fillable region;

[0034] Fusing or alternatively selecting the reference regions on the same guide line to obtain a primary filling content;

[0035] Fusing or alternatively selecting a plurality of primary filling contents to obtain a secondary filling content;

[0036] Filling the non-fillable region with the secondary filling content;

[0037] The guide lines belonging to the same non-fillable region are located on the same plane.

[0038] When the non-fillable region is located on a plurality of planes at the same time, the non-fillable region is divided according to the intersection lines of the plurality of planes.

[0039] In a possible implementation of the first aspect, fusing or alternatively selecting the reference regions on the same guide line includes:

[0040] Evaluating the similarity of any two reference regions, fusing when the reference regions on the same guide line are similar, and otherwise grouping the reference regions according to the similarity, and then selecting a reference region in a group with the largest number of reference regions as the primary filling content;

[0041] Two reference regions are selected and adjusted when fusion processing is performed, the two reference regions have the same adjustment amount and completely coincide after adjustment, and the above process is repeated until only one reference region is left;

[0042] The reference regions are allowed to be adjusted in a sub-region manner when adjustment is performed on the reference regions;

[0043] The reference regions are allowed to move along the guide line to adjust the similarity with other reference regions.

[0044] In a second aspect, the present application provides a spatial planning design auxiliary analysis device based on MR technology, comprising:

[0045] A first modeling unit is configured to model the content included in the image information to obtain a residual model in response to the acquired image information;

[0046] A second modeling unit is configured to construct a complete model using the residual model, wherein the complete model includes a spatial model and a sub-model located inside the spatial model;

[0047] A model modification unit is configured to modify the complete model according to the received instructions, wherein the modification includes deleting the sub-model located inside the spatial model and adding a new sub-model inside the spatial model;

[0048] When the sub-model located inside the spatial model is deleted, the missing part of the complete model is repaired according to the texture characteristics of the spatial model or the related sub-model.

[0049] In a third aspect, the present application provides a spatial planning design auxiliary method and a digital display system based on MR technology, and the system comprises:

[0050] One or more memories are configured to store instructions; and

[0051] One or more processors are configured to call and run the instructions from the memory, and execute the method as described in the first aspect and any possible implementation manner of the first aspect.

[0052] In a fourth aspect, the present application provides a computer readable storage medium, comprising:

[0053] A program, when the program is run by a processor, the method as described in the first aspect and any possible implementation manner of the first aspect is executed.

[0054] In a fifth aspect, the present application provides a computer program product, comprising program instructions, when the program instructions are run by a computing device, the method as described in the first aspect and any possible implementation manner of the first aspect is executed.

[0055] In a sixth aspect, the present application provides a chip system, which comprises a processor for implementing the functions involved in the above aspects, such as generating, receiving, sending, or processing the data and / or information involved in the above methods.

[0056] The chip system can be composed of a chip, or can comprise a chip and other discrete devices.

[0057] In a possible design, the chip system further comprises a memory for storing necessary program instructions and data. The processor and the memory can be decoupled and arranged on different devices, and connected through wired or wireless means, or the processor and the memory can be coupled on the same device. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 is a schematic block diagram of a step flow of a space planning design auxiliary method provided by the present application.

[0059] Figure 2 is a schematic diagram of a missing model provided by the present application.

[0060] Figure 3 is a schematic diagram of fusion of two missing models provided by the present application.

[0061] Figure 4 is a schematic diagram of establishing a rectangular selection region provided by the present application.

[0062] Figure 5 is a schematic diagram of establishing a filling guide line provided by the present application.

[0063] Figure 6 is a process schematic diagram of processing a plurality of reference regions provided by the present application.

[0064] Figure 7 is another process schematic diagram of processing a plurality of reference regions provided by the present application. DETAILED DESCRIPTION

[0065] The technical solutions in the present application will be further described in detail below with reference to the drawings.

[0066] The present application discloses a space planning design auxiliary method based on MR technology, please refer to Figure 1 In some examples, the space planning design auxiliary method based on MR technology disclosed by the present application comprises the following steps:

[0067] S101, in response to the acquired image information, modeling the content included in the image information to obtain a missing model;

[0068] S102, constructing a complete model using the incomplete model, where the complete model includes a spatial model and a sub-model located within the spatial model;

[0069] S103, modifying the complete model according to the received instruction, wherein the modification includes deleting the sub-model located inside the spatial model and adding a new sub-model inside the spatial model;

[0070] When deleting a sub-model located inside the spatial model, the missing part of the complete model is repaired according to the texture features of the spatial model or the relevant sub-model.

[0071] First of all, it should be explained that the space planning and design auxiliary method based on MR technology disclosed in this application needs to be implemented with the help of MR equipment. The MR equipment generates an image after acquiring the surrounding environment, and then modifies the image according to the received instructions to achieve the overlap of virtual and reality.

[0072] In step S101, in response to the acquired image information, the content included in the image information is first modeled to obtain an incomplete model ( Figure 2 The incomplete model here is the complete model ( Figure 2 A portion of a cube in .

[0073] Because the MR device provides the user's field of view, the incomplete model represents a portion of the complete model obtained based on the user's field of view. As the user's field of view moves, the number of incomplete models will increase, and step S102 is executed at this time.

[0074] In step S102, a complete model is constructed using the incomplete model. The complete model consists of two parts: a space model and a sub-model located inside the space model. The space model refers to a building, and the sub-model inside the space model refers to objects placed in the building.

[0075] Here we briefly introduce the modeling of the content included in the image information. In some possible implementations, the processing method used is multi-view stereo vision. The principle of multi-view stereo vision is to take multiple pictures of the same object from different angles, obtain a three-dimensional point cloud through feature matching, parallax calculation, depth optimization and other processes, and then further generate a mesh model.

[0076] The binocular vision algorithm can be used to determine the distance. This method can be explained by referring to the working method of the human eye. The image information received by two people's eyes is different. The distance can be calculated based on the difference in the two image information.

[0077] Finally in step S103, the complete model is modified according to the received instructions, and the modification is in two ways, one is to delete a sub-model inside the spatial model, which means to remove an item inside the building, and the other is to add a new sub-model inside the spatial model, which means to add a new item inside the building.

[0078] Adding a new item inside the building is relatively simple to process, because it can be calculated by direct occlusion relationship, but removing an item inside the building is difficult, because it involves repairing the spatial model or related sub-models.

[0079] For example, removing the ornament on the wall, while removing the ornament, the part occluded by the ornament needs to be repaired, because removing the ornament on the wall is a virtual process, so repairing the part occluded by the ornament is also a virtual process, that is, modifying the real content of the MR device through technical means.

[0080] This is a technical difficulty, in addition, there is also a difficulty in using the incomplete model to construct the complete model, which lies in the matching of the incomplete model, when the incomplete model has obvious characteristics, it is easy to match, but when the characteristics of the incomplete model are not obvious, it needs to be matched through additional algorithms.

[0081] In addition, it also needs to be considered that the movement of the MR device is unstable, for example, for some areas that are quickly skipped, it is a better processing scheme to obtain an incomplete model but show the real scene, at this time the obtained incomplete model needs to be transferred to the background for matching; in addition, it also needs to be considered that the incoherence of obtaining the incomplete model may occur, which cannot be simply relied on the position relationship to match the incomplete model.

[0082] In some examples, the specific way of using the incomplete model to construct the complete model is as follows:

[0083] S201, drawing a line-of-sight movement path based on the movement path and the orientation;

[0084] S202, selecting at least one sub-model or a plurality of texture regions on the line-of-sight movement path as a transition between two incomplete models;

[0085] S203, extracting the part belonging to the at least one sub-model or the plurality of texture regions in the two incomplete models;

[0086] S204, fusing the part belonging to the at least one sub-model or the plurality of texture regions in the two incomplete models and simultaneously fusing the two incomplete models.

[0087] In steps S201 to S204, the incomplete models will be fused based on the line of sight movement path. The specific method is to select at least one sub-model or multiple texture areas on the line of sight movement path as a transition between the two incomplete models, and then extract the parts of the two incomplete models belonging to the at least one sub-model or multiple texture areas. Finally, the parts of the two incomplete models belonging to the at least one sub-model or multiple texture areas are fused and the two incomplete models are fused at the same time.

[0088] This method uses sub-models or texture areas. The advantage of sub-models is that they are easy to match, such as Figure 3 As shown, for example, a sofa appears in two incomplete models at the same time. At this time, by splicing the two parts of the sofa (located in the two incomplete models respectively), the two incomplete models can be fused.

[0089] Texture regions are mainly used in situations where fusion cannot be achieved through sub-models. Using texture regions involves two processes: selection and matching.

[0090] In some examples, multiple texture regions along the eye movement path are selected as follows:

[0091] S301, establishing a plurality of rectangular selection regions on the previous incomplete model of the sequential sequence and establishing analysis vector groups within the rectangular regions;

[0092] S302, matching any two analysis vector groups. If the matching fails, the moving rectangle corresponding to one of the analysis vector groups is selected until the two analysis vector groups are matched successfully.

[0093] S303, taking the successfully matched rectangular selection area as a texture area;

[0094] S304, constructing a texture grid using the multiple texture regions, wherein the size of the texture grid parallel to the line of sight movement path is not less than half the size of the texture grid perpendicular to the line of sight movement path;

[0095] The position of the texture region is adjusted or the texture region is regenerated according to the texture grid, and at least one texture region exists inside the maximum outer edge of the texture grid.

[0096] In step S301 to step S304, first, multiple rectangular selection areas ( Figure 4 As shown) and establish an analysis vector group inside the rectangular area, then match any two analysis vector groups. If the match is unsuccessful, the moving rectangular selection area corresponding to one of the analysis vector groups is selected until the two analysis vector groups are matched successfully. At this time, the successfully matched rectangular selection area is used as a texture area and multiple texture areas are used to construct a texture mesh.

[0097] In the above process, the purpose of matching the analysis vector group is to find the same or similar area on the previous incomplete model of the sequential sequence, and the advantage of this way is that in the subsequent comparison process, any one texture area can be selected on the next incomplete model of the sequential sequence.

[0098] At the same time, in order to improve the accuracy of matching, a plurality of texture areas are also needed to construct a texture grid, so as to ensure that the distribution of the texture areas has regularity, because when the same texture area appears on the two incomplete models, the two texture areas are overlapped, and the corresponding two incomplete models are also fused together.

[0099] For the texture grid, the size of the texture grid in the direction parallel to the line of sight movement path is not less than half of the size of the texture grid in the direction perpendicular to the line of sight movement path, in addition, the position of the texture area is adjusted or the texture area is regenerated according to the texture grid, and there is at least one texture area inside the maximum outer edge of the texture grid.

[0100] The above methods are all to ensure that the texture grid is better in matching performance, because compared with using only one texture grid or multiple texture grids located on the same straight line, the distribution of the texture areas formed by the texture grid is more difficult to match, and if the matching is successful, it means that the matching of the two incomplete models has higher accuracy.

[0101] The way of establishing an analysis vector group inside the rectangular area is as follows:

[0102] The rectangular area is subjected to gray scale processing to obtain a gray scale rectangular area;

[0103] The characteristic lines in the gray scale rectangular area are selected;

[0104] The vector group is established based on the characteristic lines, and the vector group includes a plurality of vectors, and the starting point and the ending point of a vector are located on two characteristic lines respectively;

[0105] When there is no characteristic line in the gray scale rectangular area, the feature points on the gray scale rectangular area are selected and the vector group is established based on the feature points, and the vector group includes a plurality of vectors, and the starting point and the ending point of a vector are located on two feature points respectively.

[0106] The above method is to establish a vector by means of a characteristic line or a feature point, and the characteristic lines and the feature points in the gray scale rectangular area can be distinguished by length, and the length greater than the set length is a characteristic line, and vice versa.

[0107] The color of the characteristic line and the feature point is obviously darker or lighter than the surrounding area.

[0108] When the vector is established using the feature line, the start point and the end point of a vector are located on two feature lines respectively, at this time the vector has a clear start point, end point and direction, and the values (gray values) at the start point and the end point are fixed.

[0109] When the color of the feature line is uniform, the vector is established in a fixed start point distance and direction, the start point distance of the two vectors is a fixed value, and the direction is a fixed value, at this time the end point of the vector is an uncertain value.

[0110] When the vector is established using the feature point, the start point and the end point of a vector are located on two feature lines respectively, at this time the vector has a clear start point, end point and direction, and the values (gray values) at the start point and the end point are fixed.

[0111] When the color of the feature point is uniform, the vector is established in a fixed start point distance and direction, the start point distance of the two vectors is a fixed value, and the direction is a fixed value, at this time the end point of the vector is an uncertain value.

[0112] When matching two incomplete models, first use the vector group to find the texture area on the next incomplete model in the order sequence, then use the found texture area to construct the texture area, if the same texture area is obtained, fuse the two incomplete models, the fusion method is to move the texture area while moving the incomplete model, until the two incomplete models completely coincide.

[0113] The matching method of any two analysis vector groups is to put the vectors in the two analysis vector groups into a coordinate system to see if they overlap, the overlap judgment standard includes angle difference and length difference, at this time the difference calculation method is as follows:

[0114] Difference = Angle difference * P + Length difference * Q, P + Q + 1;

[0115] When the difference is less than the set value, it is considered that the two vectors overlap, when the ratio of the number of vectors (overlapping) in any one analysis group to the total number of vectors is greater than or equal to the set value, it is considered that the two analysis vector groups match. In the matching process, all vectors belonging to one analysis vector group are allowed to be enlarged, reduced, moved and rotated to achieve the purpose of reducing the difference.

[0116] When finding the texture area on the next incomplete model in the order sequence, any texture area on the previous incomplete model in the order sequence can be used.

[0117] In some examples, the missing part of the complete model is repaired according to the texture characteristics of the space model or the related sub-model as follows:

[0118] S401, determine the missing area of the space model and the number of texture characteristics around the missing area;

[0119] S402, according to the texture feature in the position of the space model feature of the missing area is divided and filled.

[0120] Specifically, it is first to determine the missing area, then determine the number of texture features around the missing area, and then use the texture feature in the position of the space model feature to divide and fill the missing area, at this time there will be two cases that the missing area can be completely filled and cannot be completely filled.

[0121] At this time, there may be a sub-model in the missing area of the space model. For this incomplete sub-model, it needs to be separated out for completion, and then placed in the original position.

[0122] The texture feature in the position of the space model feature divides the missing area, and the specific process is as follows: first, use the texture feature to determine a region, then determine the shape of the region, such as rectangle and circle, etc., then complete the figure, and assign the texture feature to the completed figure. When assigning the texture feature to the completed figure, use the existing part of the figure to copy, and then perform smooth transition at the junction.

[0123] However, at this time, the missing area may not be completely filled, at this time, the missing area is divided into fillable area and non-fillable area, and the non-fillable area is filled twice, and the secondary filling method is as follows:

[0124] S501, according to the shape of the non-fillable area to establish a filling guide line;

[0125] S502, select a plurality of reference areas with content on the guide line, the shape of the reference area is the same as the shape of the non-fillable area;

[0126] S503, fuse or select one of the reference areas on the same guide line to obtain the first filling content;

[0127] S504, fuse or select one of the plurality of first filling contents to obtain the second filling content;

[0128] S505, use the second filling content to fill the non-fillable area;

[0129] Among them, the guide lines belonging to the same non-fillable area are located on the same plane;

[0130] When the non-fillable area is located on multiple planes at the same time, the non-fillable area is divided according to the intersection line of the multiple planes.

[0131] In steps S501 to S505, please refer to Figure 5The filling guide lines are established according to the shape of the unfillable region, and specifically, the filling guide lines are determined according to the number of edges of the unfillable region, and one filling guide line is arranged on each edge of the unfillable region.

[0132] Then, a plurality of reference regions with content are selected on the guide lines, the reference regions have the same shape as the unfillable region, and for the obtained reference regions, there are two processing modes, please refer to Figure 6 The first mode is to fuse the reference regions on the same guide line, please refer to Figure 7 The second mode is to select one of the reference regions on the same guide line, and at this time, one filling content is obtained, and the number of the one filling contents is multiple.

[0133] Then, the plurality of one filling contents are fused or selected one by one to obtain secondary filling contents, and finally, the unfillable region is filled by using the secondary filling contents.

[0134] At this time, the unfillable region may be located on multiple planes, for example, a corner, so when the unfillable region is located on multiple planes, the unfillable region is divided according to the intersection lines of the multiple planes, and it is required that the guide lines belonging to the same unfillable region are located on the same plane, that is, only the same plane filling is allowed, and cross-plane filling is not allowed.

[0135] The specific mode of fusing or selecting one of the reference regions on the same guide line is as follows:

[0136] The similarity of any two reference regions is evaluated, when the reference regions on the same guide line are similar, the fusion processing is performed, otherwise, the reference regions are grouped according to the similarity, and then one reference region is selected as one filling content in the group with the most reference regions, when the grouping cannot be achieved, the position of the reference region is moved until the grouping requirement is met.

[0137] When the fusion processing is performed, two reference regions are selected and adjusted, the adjustment amount of the two reference regions is the same, and the two reference regions completely coincide after adjustment, and the above process is repeated until only one reference region is left;

[0138] When the reference regions are adjusted, the sub-regional adjustment is allowed;

[0139] The reference regions are allowed to move along the guide line to adjust the similarity with other reference regions.

[0140] In steps S601 to S604, the similarity of any two reference regions needs to be evaluated first, and there are the following two cases:

[0141] When the reference regions on the same guide line are similar, the fusion processing is performed;

[0142] When the reference regions on the same guide line cannot all be similar, the reference regions are grouped according to similarity, and then a reference region in a group with the most reference regions is selected as the filling content.

[0143] The reference regions are fused by selecting two reference regions and adjusting them, the two reference regions have the same adjustment amount and completely overlap after adjustment, and the above process is repeated until only one reference region is left, and during the adjustment of the reference regions, the reference regions are allowed to be adjusted in sub-regions, and the reference regions are allowed to move along the guide line to adjust the similarity with other reference regions.

[0144] In the above manner, the evaluation criterion of similarity is that after the two reference regions are overlapped, the ratio of the area that can be overlapped to the area of the reference region is greater than or equal to a set value.

[0145] The application also provides a spatial planning design auxiliary analysis device based on MR technology, comprising:

[0146] A first modeling unit is configured to model the content included in the image information to obtain a defective model in response to the obtained image information.

[0147] A second modeling unit is configured to construct a complete model using the defective model, wherein the complete model includes a space model and a sub-model located inside the space model.

[0148] A model modification unit is configured to modify the complete model according to received instructions, wherein the modification includes deleting the sub-model located inside the space model and adding a new sub-model inside the space model.

[0149] When the sub-model located inside the space model is deleted, the missing part of the complete model is repaired according to the texture characteristics of the space model or the related sub-model.

[0150] Further, it further comprises:

[0151] A line-of-sight movement path establishing unit is configured to draw a line-of-sight movement path based on the movement path and the orientation.

[0152] A first selection unit is configured to select at least one sub-model or a plurality of texture regions on the line-of-sight movement path as a transition between two defective models.

[0153] An extraction unit is configured to extract the part belonging to the at least one sub-model or the plurality of texture regions in the two defective models.

[0154] A fusion unit is configured to fuse the part belonging to the at least one sub-model or the plurality of texture regions in the two defective models and simultaneously fuse the two defective models.

[0155] Further comprising:

[0156] An analysis vector group establishing unit is configured to establish a plurality of rectangular selection regions on a previous incomplete model in the sequential sequence and establish an analysis vector group inside the rectangular regions;

[0157] An analysis vector group matching unit is configured to match any two analysis vector groups, and when the matching is unsuccessful, the moving rectangular selection region corresponding to one of the analysis vector groups is matched until the two analysis vector groups are successfully matched;

[0158] A selection unit is configured to select the rectangular selection region with successful matching as a texture region;

[0159] A texture grid constructing unit is configured to construct a texture grid using a plurality of texture regions, and the size of the texture grid in a direction parallel to the line-of-sight movement path is not less than one-half of the size of the texture grid in a direction perpendicular to the line-of-sight movement path;

[0160] In which, the position of the texture region is adjusted or the texture region is regenerated according to the texture grid, and there is at least one texture region inside the maximum outer edge of the texture grid.

[0161] Further comprising:

[0162] A grayscale processing unit is configured to perform grayscale processing on the rectangular region to obtain a grayscale rectangular region;

[0163] A first selection unit is configured to select a feature line in the grayscale rectangular region;

[0164] A first vector group constructing unit is configured to establish a vector group based on the feature line, and the vector group includes a plurality of vectors, and the start point and the end point of a vector are located on two feature lines, respectively;

[0165] A second vector group constructing unit is configured to select a feature point on the grayscale rectangular region and establish a vector group based on the feature point when the feature line does not exist in the grayscale rectangular region, and the vector group includes a plurality of vectors, and the start point and the end point of a vector are located on two feature points, respectively.

[0166] Further comprising:

[0167] A quantity determining unit is configured to determine the number of texture features of the missing region and the surrounding texture features of the missing region of the spatial model;

[0168] A first processing unit is configured to divide and fill the missing region according to the spatial model features of the position of the texture feature.

[0169] Further comprising:

[0170] The establishing unit is configured to establish the filling guide line according to the shape of the non-fillable region;

[0171] The second selecting unit is configured to select a plurality of reference regions with content on the guide line, the shape of the reference region being the same as that of the non-fillable region;

[0172] The second processing unit is configured to perform fusion processing or alternative selection on the reference regions on the same guide line to obtain primary filling content;

[0173] The third processing unit is configured to perform fusion processing or alternative selection on a plurality of primary filling contents to obtain secondary filling content;

[0174] The first filling unit is configured to fill the non-fillable region with the secondary filling content;

[0175] The guide lines belonging to the same non-fillable region are located on the same plane;

[0176] When the non-fillable region is located on a plurality of planes, the non-fillable region is divided according to the intersection lines of the plurality of planes.

[0177] Further, the method further comprises:

[0178] The similarity evaluating unit is configured to evaluate the similarity of any two reference regions, and when the reference regions on the same guide line are similar, fusion processing is performed, otherwise, the reference regions are grouped according to the similarity, and then a reference region with the largest number of reference regions in a group is selected as the primary filling content;

[0179] The fourth processing unit is configured to select two reference regions and adjust them when fusion processing is performed, the two reference regions having the same adjustment amount and completely overlapping after adjustment, and the above process being repeated until only one reference region is left;

[0180] The reference regions are allowed to be adjusted in a sub-regional manner when adjusted;

[0181] The reference regions are allowed to move along the guide line to adjust the similarity with other reference regions.

[0182] In an example, the units in any of the above apparatuses can be one or more integrated circuits, configured to implement one or more of the above methods, e.g., one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0183] For example, when the units in the apparatuses can be implemented in the form of a processing element scheduler, the processing element can be a general purpose processor, such as a central processing unit (CPU) or other processor that can invoke a program. For another example, the units can be integrated together in the form of a system-on-a-chip (SOC).

[0184] In the present application, various objects such as messages / information / devices / network elements / systems / apparatuses / actions / operations / processes / concepts, etc. that can occur in the present application are named. It can be understood that these specific names do not constitute a limitation on the related objects, and the names can be changed according to the scene, context or usage habits, etc. The technical meaning of the technical terms in the present application should be mainly determined from the function and technical effect embodied / implemented in the technical scheme.

[0185] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, apparatus and unit can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0186] In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method can be implemented in other ways. For example, the above-described apparatus embodiments are merely schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0187] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0188] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical scheme. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0189] It should also be understood that in various embodiments of the present application, first, second, etc. are only to represent that a plurality of objects are different. For example, the first time window and the second time window are only to represent different time windows. The above first, second, etc. should not have any effect on the time window itself, and should not limit the embodiments of the present application.

[0190] It should also be understood that in various embodiments of the present application, the terms and / or descriptions of different embodiments are consistent and can be referred to each other if there is no special description and logical conflict, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0191] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical scheme of the present application or the part of the technical scheme that essentially contributes to the prior art or the part of the technical scheme can be embodied in the form of a software product. The computer software product stored in a computer readable storage medium includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned computer readable storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and various program code storage media.

[0192] The present application also provides a spatial planning design auxiliary method and digital display system based on MR technology, the system comprises:

[0193] One or more memories for storing instructions; and

[0194] One or more processors are used to call and execute the instructions from the memory to perform the method as described above.

[0195] The present application also provides a computer program product, which includes instructions. When the instructions are executed, the terminal device and the network device perform operations of the terminal device and the network device corresponding to the above method.

[0196] The present application also provides a chip system, which includes a processor for implementing the functions involved in the above content, such as generating, receiving, sending, or processing the data and / or information involved in the above method.

[0197] The chip system may be composed of chips, or may include chips and other discrete devices.

[0198] The processor mentioned in any of the above may be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for executing a program for controlling the above-mentioned feedback information transmission method.

[0199] In one possible design, the chip system also includes a memory for storing necessary program instructions and data. The processor and the memory can be decoupled and provided on different devices, respectively, and connected via wired or wireless means to support the chip system in implementing the various functions of the above embodiments. Alternatively, the processor and the memory can be coupled on the same device.

[0200] Optionally, the computer instructions are stored in a memory.

[0201] Optionally, the memory is a storage unit within the chip, such as a register, cache, etc. The memory can also be a storage unit within the terminal located outside the chip, such as ROM or other types of static storage devices that can store static information and instructions, RAM, etc.

[0202] It can be understood that the memory in the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0203] The non-volatile memory may be ROM, programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory.

[0204] The volatile memory can be a RAM used as an external cache. RAM has many different types, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synch link DRAM (SLDRAM), and direct Rambus RAM.

[0205] The embodiments of the present disclosure are all the preferred embodiments of the present application, and are not intended to limit the protection scope of the present application, so that: any equivalent changes made according to the structure, shape, principle of the present application should be covered within the protection scope of the present application.

Claims

1. A spatial planning design assistance method based on MR technology, characterized by, The method comprises: modeling the content included in the image information to obtain a residual model in response to the acquired image information; constructing a complete model using the residual model, the complete model comprising a space model and a sub-model located inside the space model; modifying the complete model according to the received instruction, the modification comprising deleting the sub-model located inside the space model and adding a new sub-model inside the space model; wherein when the sub-model located inside the space model is deleted, the missing part of the complete model is repaired according to the texture characteristics of the space model or the related sub-model; constructing the complete model using the residual model comprises: drawing a line-of-sight movement path based on the movement path and the orientation; selecting at least one sub-model or a plurality of texture regions on the line-of-sight movement path as a transition between two residual models; extracting the parts of the two residual models that belong to the at least one sub-model or the plurality of texture regions; fusing the parts of the two residual models that belong to the at least one sub-model or the plurality of texture regions while fusing the two residual models; repairing the missing part of the complete model according to the texture characteristics of the space model or the related sub-model comprises: determining the missing area of the space model and the number of texture characteristics around the missing area; dividing and filling the missing area according to the space model characteristics at the location of the texture characteristics; when the missing area cannot be completely filled, dividing the missing area into a fillable area and a non-fillable area, and performing secondary filling on the non-fillable area, the secondary filling comprising: establishing a filling guide line according to the shape of the non-fillable area; selecting a plurality of reference regions with content on the guide line, the shape of the reference region being the same as that of the non-fillable area; fusing or selecting one of the reference regions on the same guide line to obtain primary filling content; fusing or selecting one of the primary filling content to obtain secondary filling content; filling the non-fillable area with the secondary filling content; wherein the guide lines belonging to the same non-fillable area are located on the same plane; when the non-fillable area is located on multiple planes at the same time, dividing the non-fillable area according to the junction line of the multiple planes.

2. The MR technique-based spatial planning design assistance method according to claim 1, characterized by, selecting a plurality of texture regions on the line-of-sight movement path comprises: establishing a plurality of rectangular selection regions on the previous residual model in the sequence and establishing an analysis vector group inside the rectangular region; matching any two analysis vector groups, and when the matching is unsuccessful, one of the analysis vector groups corresponding to the movement rectangular selection region, until the two analysis vector groups are successfully matched; the rectangular selection region that is successfully matched is taken as a texture region; constructing a texture grid using a plurality of texture regions, the size of the texture grid in the direction parallel to the line-of-sight movement path being not less than one-half of the size of the texture grid in the direction perpendicular to the line-of-sight movement path; wherein the position of the texture region is adjusted or the texture region is regenerated according to the texture grid, and there is at least one texture region inside the maximum outer edge of the texture grid.

3. The MR technology-based spatial planning design assistance method according to claim 1, characterized by, establishing an analysis vector group inside the rectangular region comprises: gray processing the rectangular region to obtain a gray rectangular region; selecting a feature line in the gray rectangular region; The vector group is established based on the feature lines, and the vector group includes a plurality of vectors, and a start point and an end point of one vector are located on two feature lines respectively; When there is no feature line in the gray rectangular region, a feature point on the gray rectangular region is selected, and a vector group is established based on the feature point, and the vector group includes a plurality of vectors, and a start point and an end point of one vector are located on two feature points respectively.

4. The MR technique-based spatial planning design assistance method according to claim 1, characterized by, The reference regions on the same guide line are fused or selected one by one, including: The similarity of any two reference regions is evaluated, when the reference regions on the same guide line are similar, the fusion processing is performed, otherwise, the reference regions are grouped according to the similarity, and then one reference region in a group with the largest number of reference regions is selected as the filling content; When the fusion processing is performed, two reference regions are selected and adjusted, the adjustment amount of the two reference regions is the same, and the two reference regions completely overlap after adjustment, and the above process is repeated until only one reference region is left; The reference regions are allowed to be adjusted in a divided manner; The reference regions are allowed to move along the guide line to adjust the similarity with other reference regions.

5. A spatial planning and design auxiliary analysis device based on MR technology, characterized in that: Including: The first modeling unit is configured to model the content included in the image information to obtain a defective model in response to the obtained image information; The second modeling unit is configured to construct a complete model using the defective model, and the complete model includes a space model and a sub-model located inside the space model; The model modification unit is configured to modify the complete model according to the received instructions, and the modification includes deleting the sub-model located inside the space model and adding a new sub-model inside the space model; When the sub-model located inside the space model is deleted, the missing part of the complete model is repaired according to the texture characteristics of the space model or the related sub-model; The construction of the complete model using the defective model includes: Drawing a line-of-sight movement path based on the movement path and the orientation; Selecting at least one sub-model or a plurality of texture regions on the line-of-sight movement path as a transition between the two defective models; Extracting the parts of the two defective models belonging to the at least one sub-model or the plurality of texture regions; Fusing the parts of the two defective models belonging to the at least one sub-model or the plurality of texture regions, and simultaneously fusing the two defective models; The repairing of the missing part of the complete model according to the texture characteristics of the space model or the related sub-model includes: Determining the missing area of the space model and the number of texture characteristics around the missing area; Dividing and filling the missing area according to the space model characteristics at the location of the texture characteristics; When the missing area cannot be completely filled, the missing area is divided into a fillable area and a non-fillable area, and the non-fillable area is secondarily filled, and the secondary filling includes: Establishing a filling guide line according to the shape of the non-fillable area; Selecting a plurality of reference regions with content on the guide line, and the shape of the reference region is the same as that of the non-fillable area; Fusing or selecting one by one the reference regions on the same guide line to obtain a primary filling content; Fusing or selecting one by one the plurality of primary filling contents to obtain a secondary filling content; Filling the non-fillable area with the secondary filling content; The guide lines belonging to the same unfillable region are located on the same plane. When the unfillable region is located on multiple planes, the unfillable region is divided according to the intersection line of the multiple planes.

6. A spatial planning design assistance method and digitalized presentation system based on MR technology, characterized in that, The system comprises: one or more memories for storing instructions; and one or more processors for calling and running the instructions from the memories to execute the method as claimed in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises: a program, when the program is run by a processor, the method as claimed in any one of claims 1 to 4 is executed.

Images