Visualization method and system based on digital twin

Abstract

The present invention provides a visualization method and system based on digital twins, which respond to automatic modeling requests from each planning end for a planned park, obtain video data of a physical model shot by a recording device based on a collection perspective, and the physical model includes a dynamic planning area and a fixed planning area. The customized elements configured by each planning end for the dynamic planning area are determined according to the video data, and the initial twin space corresponding to the physical model is customized and updated based on the customized elements. The dynamic planning area that does not meet the cost planning conditions is determined as the area to be adjusted, and the planning adjustment strategy is called to adjust and update the virtual planning area corresponding to the area to be adjusted in the initial twin space, so as to obtain the customized display space corresponding to the corresponding planning end, obtain the selection coefficient corresponding to each of the customized display spaces, and determine that the customized display space with the largest selection coefficient is the planning selected space corresponding to the planned park.

Description

Technical Field

[0001] The present invention relates to data processing technology, and in particular to a visualization method and system based on digital twins. Background Art

[0002] With the continuous expansion of urban population and the accelerated urbanization process, the layout planning of urban parks has become an indispensable and key part of urban development strategy.

[0003] Traditional urban park planning often relies on planners to manually draw two-dimensional drawings to obtain planning data. This method has many limitations, such as low efficiency in generating planning data and the inability to make real-time adjustments to planning data based on the needs of planners.

[0004] Therefore, how to generate and display planning data in accordance with the needs of planners and improve the efficiency of planning data generation has become an urgent problem to be solved. Summary of the invention

[0005] The present invention provides a visualization method and system based on digital twins, which can generate and display planning data in combination with the needs of planners, thereby improving the efficiency of generating planning data.

[0006] A first aspect of the present invention provides a visualization method based on digital twins, comprising:

[0007] In response to automatic modeling requests from each planning terminal for a planned park, video data of a physical model shot by a recording device based on a collection perspective is obtained, wherein the physical model includes a dynamic planning area and a fixed planning area;

[0008] Determining, according to the video data, customized elements configured by each planning end for the dynamic planning area, and customizing and updating the initial twin space corresponding to the entity model based on the customized elements;

[0009] Determine the dynamic planning area that does not meet the cost planning conditions as the area to be adjusted, call the planning adjustment strategy to adjust and update the virtual planning area corresponding to the area to be adjusted in the initial twin space, and obtain the customized display space corresponding to the corresponding planning end;

[0010] The selection coefficient corresponding to each of the customized display spaces is obtained, and the customized display space with the largest selection coefficient is determined as the planned selected space corresponding to the planned park.

[0011] Optionally, in a possible implementation manner of the first aspect, determining, according to the video data, customized elements configured by each planning end for the dynamic planning area, and customizing and updating the initial twin space corresponding to the entity model based on the customized elements, includes:

[0012] Arranging multiple image frames corresponding to the video data in time order to obtain a processing sequence, and sequentially acquiring module contours in each of the image frames in the processing sequence;

[0013] Determine that a module outline with an element tag is a marked outline, and a module outline without the element tag is a pending outline;

[0014] Acquire the pending contours in the adjacent image frames whose contour similarity is greater than the contour similarity threshold as a pending group, perform feature recognition on the pending group, and obtain the customized elements corresponding to the pending group;

[0015] Determine the center point of each of the to-be-determined contours in the to-be-determined group as a positioning point, and obtain the moving trajectory of the customized element according to the positioning point;

[0016] Determine the layout position of the customized element according to the movement trajectory, and update the pending outline corresponding to the pending group to a marked outline based on the layout position, wherein the layout position includes a placement position and a hierarchical position;

[0017] Retrieving a virtual module corresponding to the customized element, acquiring a three-dimensional coordinate system corresponding to the initial twin space, and determining an X-axis coordinate value and a Y-axis coordinate value corresponding to the virtual module according to the top-view coordinates corresponding to the placement position in the image frame;

[0018] Determine a preset height coordinate value corresponding to the hierarchical position of the custom element, and determine a Z-axis coordinate value corresponding to the virtual module according to the height coordinate value;

[0019] A virtual position is determined according to the X-axis coordinate value, the Y-axis coordinate value, and the Z-axis coordinate value, and the virtual module is positioned at the virtual position to customize and update the initial twin space.

[0020] Optionally, in a possible implementation manner of the first aspect, a layout position of the custom element is determined according to the movement trajectory, and based on the layout position, a pending outline corresponding to the pending group is updated to a marked outline, wherein the layout position includes a placement position and a hierarchical position, including:

[0021] When the moving track is interrupted, determining that the hierarchical attribute of the custom element is an obscured attribute, and determining the hierarchical position of the custom element as a lower layer position according to the obscured attribute;

[0022] Determining a blocked direction of the customized element according to the movement trajectory, and determining a placement position of the customized element based on the blocked direction;

[0023] When there are a preset number of consecutive adjacent image frames with the same moving tracks, obtaining an area change trend of the pending contour in the image frames in the pending group, and determining a hierarchical attribute of the custom element according to the area change trend, wherein the hierarchical attribute includes an unobstructed attribute and an obstructed attribute;

[0024] Determine that the hierarchical position of the custom element with no obstruction attribute is an upper position, and the hierarchical position of the custom element with obstruction attribute is a lower position;

[0025] The center point of the undetermined contour is obtained as the placement position of the customized element when the moving trajectory remains unchanged;

[0026] The layout position of the custom element is obtained based on the placement position and the hierarchical position, the element tag corresponding to the custom element is retrieved and bound to the pending group, and the pending outline of the determined layout position is updated to a marked outline.

[0027] Optionally, in a possible implementation manner of the first aspect, determining the blocked direction of the customized element according to the movement trajectory, and determining the placement position of the customized element based on the blocked direction includes:

[0028] The previous image frame when the moving trajectory is interrupted is acquired as a target frame, and the contour distance between the to-be-determined contour and each of the marked contours is determined according to the center point of each of the marked contours in the target frame and the positioning point of the to-be-determined contour corresponding to the customized element;

[0029] Determine that the marked contour whose contour distance is less than the stacking distance threshold and whose contour distance is the smallest is the target contour;

[0030] Acquire a tangent direction corresponding to the positioning point in the moving trajectory as the blocked direction, and determine the undetermined contour with the largest contour area in the undetermined group as the positioning contour;

[0031] Taking the positioning point as a reference, generating a positioning line parallel to the blocked direction, positioning the center point of the positioning contour by using the positioning line, and performing positioning movement on the positioning contour until the positioning contour is completely covered by the target contour, and then stopping the positioning movement of the positioning contour;

[0032] The center point of the positioning contour when it stops moving is obtained as the placement position of the customized element.

[0033] Optionally, in a possible implementation manner of the first aspect, obtaining an area change trend of the to-be-determined contour in the image frame in the to-be-determined group, and determining a hierarchical attribute of the custom element according to the area change trend includes:

[0034] Acquire contour areas of the to-be-determined contours in adjacent image frames in the to-be-determined group, and obtain area difference values ​​corresponding to adjacent image frames according to the difference values ​​of the contour areas;

[0035] When the area difference values ​​of all adjacent image frames in the pending group are less than the area difference threshold, determining that the area change trend of the pending group is a fixed trend, and determining that the hierarchical attribute of the custom element is a non-occlusion attribute according to the fixed trend;

[0036] When there are area differences between adjacent image frames in the pending group that are greater than or equal to an area difference threshold, the area change trend of the pending group is determined to be a dynamic trend, and the hierarchical attribute of the custom element is determined to be an obscured attribute according to the dynamic trend.

[0037] Optionally, in a possible implementation manner of the first aspect, determining a dynamic planning area that does not meet the cost planning condition as a to-be-adjusted area, calling a planning adjustment strategy to adjust and update a virtual planning area corresponding to the to-be-adjusted area in the initial twin space, and obtaining a customized display space corresponding to the corresponding planning end, includes:

[0038] Counting the estimated costs corresponding to the customized elements to obtain the estimated total cost, determining that the dynamic planning area where the estimated total cost is greater than the planned cost does not meet the cost planning condition, and obtaining the corresponding dynamic planning area as the area to be adjusted;

[0039] Receiving an adjustment area demarcated by the planning end in a virtual planning area corresponding to the area to be adjusted, and acquiring an adjustment element configured by the planning end for the adjustment area;

[0040] Retrieving a corresponding planning adjustment strategy according to the adjustment attribute of the adjustment element, wherein the planning adjustment strategy includes a spacing adjustment strategy and a density adjustment strategy;

[0041] Based on the planning adjustment strategy, the virtual module corresponding to the adjustment element in the virtual planning area is adjusted and updated, and the adjustment area is updated according to the virtual module to obtain the customized display space corresponding to the corresponding planning end.

[0042] Optionally, in a possible implementation manner of the first aspect, adjusting and updating a virtual module corresponding to the adjustment element in the virtual planning area based on the planning adjustment strategy, and updating the adjustment area according to the virtual module to obtain a customized display space corresponding to the corresponding planning end, including:

[0043] According to the spacing adjustment strategy, a virtual module corresponding to an adjustment element of the spacing attribute is obtained as a first adjustment module, and a first area where the first adjustment module is located is obtained;

[0044] Acquire a first placement direction of the first adjustment module, and determine a second placement direction configured by the planning end for the adjustment area;

[0045] Determine a first length of the first area in the first placement direction and a second length of the adjustment area in the second placement direction, and obtain a total area length according to the sum of the first length and the second length;

[0046] Obtaining an adjusted placement spacing according to a ratio of the total area length to the total number of modules of the first adjustment module, and adjusting the spacing of the first adjustment modules based on the adjusted placement spacing;

[0047] The first adjustment module whose position is not in the first area after adjustment is obtained as the first moving module, and the first moving module is updated to the adjustment area according to the second placement direction and the adjusted placement spacing to obtain the customized display space corresponding to the corresponding planning end.

[0048] Optionally, in a possible implementation manner of the first aspect, adjusting and updating a virtual module corresponding to the adjustment element in the virtual planning area based on the planning adjustment strategy, and updating the adjustment area according to the virtual module to obtain a customized display space corresponding to the corresponding planning end, including:

[0049] According to the density adjustment strategy, a virtual module corresponding to an adjustment element of a density attribute is obtained as a second adjustment module, and a second area where the second adjustment module is located is obtained;

[0050] Counting the total area of ​​the second region and the adjustment zone, and obtaining an adjustment density ratio according to a ratio of the total number of the second adjustment modules in the second region to the total area;

[0051] Density adjustment is performed on the second adjustment modules in the second area according to the adjustment density ratio, and redundant second adjustment modules are obtained as second mobile modules;

[0052] The adjustment area is filled and updated based on the second moving module to obtain a customized display space corresponding to the corresponding planning end.

[0053] Optionally, in a possible implementation manner of the first aspect, obtaining a selection coefficient corresponding to each of the customized display spaces, and determining that the customized display space with the largest selection coefficient is the planned selected space corresponding to the planned park includes:

[0054] Obtaining a total planning cost of each of the customized display spaces, and obtaining a first offset coefficient according to a ratio of a benchmark planning cost to the total planning cost;

[0055] Determine the heat value of each customized display space, and obtain a second offset coefficient according to a ratio of the heat value to a reference heat value;

[0056] According to the sum of the first offset coefficient and the second offset coefficient, a selection coefficient corresponding to the corresponding customized display space is obtained, and the customized display space with the largest selection coefficient is determined as the planned selected space corresponding to the planned park.

[0057] A second aspect of the present invention provides a visualization system based on digital twins, comprising:

[0058] A response module is used to respond to the automatic modeling request of each planning end for the planned park, and obtain the video data of the physical model shot by the recording device based on the acquisition perspective, wherein the physical model includes a dynamic planning area and a fixed planning area;

[0059] An updating module, configured to determine, according to the video data, the customized elements configured by each planning end for the dynamic planning area, and to customize and update the initial twin space corresponding to the entity model based on the customized elements;

[0060] An adjustment module is used to determine the dynamic planning area that does not meet the cost planning conditions as the area to be adjusted, and to call the planning adjustment strategy to adjust and update the virtual planning area corresponding to the area to be adjusted in the initial twin space to obtain a customized display space corresponding to the corresponding planning end;

[0061] The acquisition module is used to acquire the selection coefficient corresponding to each of the customized display spaces, and determine that the customized display space with the largest selection coefficient is the planned selected space corresponding to the planned park.

[0062] The beneficial effects of the present invention are as follows:

[0063] 1. The present invention can combine the automatic modeling request of the planner to obtain the planning data in the entity model corresponding to the planner, including the data of each module in the entity model and the position data corresponding to each module. By automatically collecting the planning data in the entity model, the corresponding virtual twin space can be automatically generated, so that the display planning data can be generated in combination with the needs of the planner, and the generation efficiency of the planning data can be improved. The present invention can identify the module data placed in the dynamic planning area in combination with the video data corresponding to the planner, and the customized elements corresponding to each module in the entity model can be determined, and the layout position of the customized elements in the entity model can be determined through the video data, so that the customized elements can be located at the corresponding virtual position in the twin space according to the layout position of the customized elements in the entity model, so that the initial twin space can be customized and updated, and the efficiency of the customized update can be improved.

[0064] 2. When acquiring planning data in a physical model, the present invention can record the construction process of the physical model through a recording device, and can synchronously generate a twin space corresponding to the physical model in real time based on the video data captured by the recording device, which can improve the efficiency of generating virtual twin spaces, thereby improving planning efficiency. When determining the layout position of a customized element in a physical model, the present invention can determine the layout position corresponding to the customized element in the physical model through the movement trajectory of the customized element in the video data and the area change trend of the module outline corresponding to the customized element, and can more accurately obtain the layout position corresponding to the customized element, so that the initial twin space can be more closely aligned with the physical model.

[0065] 3. When the planning data in the dynamic planning area does not meet the cost planning conditions, the present invention can determine the dynamic planning area that does not meet the cost planning conditions as the area to be adjusted, and then determine the virtual planning area corresponding to the area to be adjusted in the initial twin space, and update the initial twin space according to the virtual planning area, so as to obtain a customized display space that meets the cost planning conditions and meets the needs of the planner. The present invention obtains the selection coefficient corresponding to each customized display space by combining the planning cost and the heat value. Through the selection coefficient, the customized display space with the best planning cost and high popularity can be selected from multiple customized display spaces as the final planning selected space, so that the optimal customized display space can be quickly obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a flowchart of a visualization method based on digital twins provided in an embodiment of the present invention;

[0067] Figure 2 This is a schematic diagram of a case where the area change trend of a pending group is a dynamic trend, provided by an embodiment of the present invention;

[0068] Figure 3 is a schematic diagram of adjusting a virtual planning area based on a spacing adjustment strategy provided by an embodiment of the present invention;

[0069] Figure 4 is a schematic diagram of adjusting a virtual planning area based on a density adjustment strategy provided by an embodiment of the present invention;

[0070] Figure 5 It is a structural schematic diagram of a digital twin-based visualization system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] See also Figure 1 , is a flowchart of a visualization method based on digital twins provided in an embodiment of the present invention, Figure 1 The execution subject of the method shown may be a software and / or hardware device. The execution subject of the present application may include but is not limited to at least one of the following: user equipment, network equipment, etc. Among them, user equipment may include but is not limited to computers, smart phones, personal digital assistants (PDA) and the electronic devices mentioned above. Network equipment may include but is not limited to a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of computers or network servers, wherein cloud computing is a type of distributed computing, a super virtual computer composed of a group of loosely coupled computers. This embodiment does not limit this. It includes steps S1 to S4, as follows:

[0073] S1, responding to automatic modeling requests from each planning terminal for a planned park, obtaining video data of a physical model shot by a recording device based on a collection perspective, wherein the physical model includes a dynamic planning area and a fixed planning area.

[0074] When this solution constructs a twin space in combination with the needs of planners, planners can first build a corresponding physical model for the planned park, and the physical models built by different planners are different. This solution can combine the planners' automatic modeling requests to obtain the planning data in the physical model corresponding to the planners, including the data of each module in the physical model and the location data corresponding to each module. By automatically collecting the planning data in the physical model, the corresponding virtual twin space can be automatically generated, so that the corresponding display planning data can be generated in combination with the needs of planners, thereby improving the efficiency of generating planning data.

[0075] Specifically, when planners want to build a virtual space corresponding to the park they have planned, they can first apply for automatic modeling. After receiving the planner's automatic modeling application, a physical model can be assigned to the planner for construction. The planner can plan the corresponding planning area through mini modules, such as mini modules of trees, mini modules of flowers, etc. In order to obtain the module data placed by the planners in the physical model, a recording device can be configured for the planner to record the construction process of the physical model in real time from a bird's-eye view. The recording device can record the corresponding position of each mini module in the physical model through a fixed shooting posture, so that the corresponding module data can be obtained from the video data captured by the recording device, and the twin space corresponding to the physical model can be synchronously generated based on the video data.

[0076] Among them, the planning end refers to the terminal held by the planner, the planned park refers to the park for automatic modeling planning, the recording device refers to the device for real-time shooting of the construction process of the physical model, such as a camera, the acquisition perspective refers to the overhead perspective of the recording device when shooting, the physical model refers to the model corresponding to the layout of the planned park, such as a physical sand table model, the video data refers to the data recorded during the construction process of the physical model, the dynamic planning area refers to the area that planners can plan, such as a green area, etc., and the fixed planning area refers to an already planned area, such as a lake water area, etc.

[0077] S2, determining the customized elements configured by each planning end for the dynamic planning area according to the video data, and customizing and updating the initial twin space corresponding to the entity model based on the customized elements.

[0078] It can be understood that the fixed planning area in this scheme is configured and does not require dynamic planning. However, since there can be multiple dynamic planning areas and different planners may configure different module data in the dynamic planning area, the module data placed in the dynamic planning area can be identified in combination with the corresponding video data of the planner, and the planned object corresponding to the module data, that is, the customized element, can be determined. For example, when the module placed by the planner is a model corresponding to a willow tree, the identified customized element can be a willow tree, and the position data of the customized element in the physical model can be determined through the video data. Since the twin space is a virtual space corresponding to the physical model, the virtual module corresponding to the customized element can be positioned to the corresponding position in the twin space according to the position data of the customized element in the physical model, and the initial twin space can be customized and updated.

[0079] Among them, customized elements refer to the specific elements corresponding to the modules configured by planners for each dynamic planning area in the physical model. The initial twin space is a virtual digital space that matches the real physical model and has not been customized and updated. The initial twin space can be customized and updated subsequently based on the planners' planning data.

[0080] Based on the above embodiment, the specific implementation of step S2 may be:

[0081] S21, arranging a plurality of image frames corresponding to the video data in time sequence to obtain a processing sequence, and sequentially acquiring a module outline in each of the image frames in the processing sequence.

[0082] It can be understood that the update of the initial twin space is synchronized in real time and one-to-one with the construction process of the physical model. That is, when the planner places a module in the physical model, the corresponding virtual module will be updated synchronously at the corresponding position in the initial twin space. Therefore, the initial twin space can be updated according to the order in which the planner places the modules.

[0083] Specifically, the recording device can collect multiple continuous image frames in the process of building the physical model, and the multiple image frames can be arranged in the time sequence of shooting to obtain a processing sequence. The modules in the image frames in the processing sequence are extracted in turn by contour extraction technology, and the module contours in each image frame can be obtained. According to the time sequence in which the contours of multiple modules appear, the placement order of multiple modules can be determined, so that the initial twin space can be updated according to the placement order of each module.

[0084] The processing sequence refers to the sequence obtained by arranging the image frames in chronological order, and the module outline refers to the outline corresponding to the module placed by the planner in the physical model.

[0085] S22, determining that the module outline with the element tag is a marked outline, and the module outline without the element tag is a pending outline.

[0086] Since there may be multiple module outlines in an image frame, in order to avoid repeated positioning analysis of the outline corresponding to the module whose placement position has been determined, the module outline corresponding to the module whose placement position has been determined can be marked with a corresponding element label and determined as a marked outline. It can be considered that the placement position of the module corresponding to the marked outline has been determined, and the subsequent re-positioning of the marked outline can be avoided. When there is no element label for the module outline, it means that the placement position of the corresponding module has not been determined. The module outline can be determined as a pending outline, and feature recognition can be performed on the pending outline to determine the placement position of the module corresponding to the pending outline.

[0087] When the placement position of the module outline is determined, an element tag can be added to it, and the positioning point and outline shape corresponding to the module outline can be bound to the element tag, so that in subsequent image frames, the module outline corresponding to the placement position and outline shape can be determined as the marked outline.

[0088] Among them, the element label refers to the label that marks the outline of the module whose position has been determined, the marked outline refers to the outline corresponding to the module whose position has been determined, and the pending outline refers to the outline corresponding to the module whose position has not yet been determined.

[0089] S23, obtaining the pending contours in the adjacent image frames whose contour similarity is greater than the contour similarity threshold as a pending group, performing feature recognition on the pending group, and obtaining the customized elements corresponding to the pending group.

[0090] Specifically, when the contour similarity between the pending contours in adjacent image frames is greater than the contour similarity threshold, it can be considered that the pending contours corresponding to the adjacent image frames are contours corresponding to the same module. Therefore, the pending contours corresponding to the same module can be determined as the same pending group, and the pending contours in the same pending group are feature recognized by contour feature recognition technology. For example, edge detection technology can be used to perform feature recognition on the pending contours, and the custom elements corresponding to the pending contours in the pending group can be determined.

[0091] In some embodiments, in the process of constructing a physical model, when the modules are placed, one module may be occluded by another module. In this case, since the time interval between adjacent image frames is very small, during the occlusion process, the contour changes corresponding to the same module in adjacent image frames are small, so the contour similarity between the module contours corresponding to adjacent image frames will still be greater than the contour similarity threshold. During the occlusion process, the module contour corresponding to the occluded custom element may gradually become smaller, making the pending contour determined later in the pending group may not be complete. Therefore, when performing feature recognition, the relatively complete pending contour in the pending group can be identified, such as the complete contour obtained in the first few frames, thereby improving the accuracy of feature recognition.

[0092] Among them, contour similarity refers to the degree of similarity of pending contours in adjacent image frames, contour similarity threshold refers to the threshold used to judge the similarity of contours. If it is greater than the threshold, it can be considered that the corresponding two pending contours are the same module contours, and pending group refers to multiple pending contours corresponding to the same module.

[0093] Through the above implementation, the accuracy of feature recognition can be improved, so that the customized elements corresponding to the undetermined contour can be obtained more accurately.

[0094] S24, determining the center point of each of the to-be-determined contours in the to-be-determined group as a positioning point, and obtaining a moving trajectory of the customized element according to the positioning point.

[0095] When determining the corresponding position of the customized element in the physical model, since the placement process of the customized element is a dynamic process, the placement position of the customized element can be determined by the movement trajectory of the customized element in the video data. When obtaining the movement trajectory, the center point of the undetermined contour can be determined as the positioning point, and the movement trajectory can be determined based on the positioning point. Specifically, multiple contour points corresponding to the undetermined contour can be obtained, and the multiple contour points can be coordinateized. Through the coordinate values ​​corresponding to the multiple contour points, the coordinate value corresponding to the center point of the undetermined contour can be determined.

[0096] In some embodiments, the coordinate origin corresponding to each image frame of each frame may be the same, and an x-axis is established in the horizontal direction and a y-axis is established in the vertical direction. Then, the maximum coordinate value and the minimum coordinate value of multiple contour points in the x-axis direction, as well as the maximum coordinate value and the minimum coordinate value in the y-axis direction can be determined. By averaging the maximum coordinate value and the minimum coordinate value in the x-axis direction, the coordinate value of the center point in the x-axis direction can be obtained. By averaging the maximum coordinate value and the minimum coordinate value in the y-axis direction, the coordinate value of the center point in the y-axis direction can be obtained. Then, the center point can be determined as the positioning point of the contour to be determined. By obtaining the position change of the positioning point in different frames, the moving trajectory of the customized element can be obtained.

[0097] Among them, the positioning point refers to the point for locating the contour to be determined, which can be the center point of the contour to be determined. The moving trajectory refers to the trajectory of the position change of the model corresponding to the customized element during the planning process, which can be used to determine the position point of the customized element in the physical model.

[0098] S25, determining the layout position of the customized element according to the movement trajectory, and updating the pending outline corresponding to the pending group to a marked outline based on the layout position, wherein the layout position includes a placement position and a hierarchical position.

[0099] Among them, the layout position refers to the position of the module corresponding to the custom element in the physical model, the placement position can be the position point of the module corresponding to the custom element from a bird's-eye view, and the hierarchical position refers to the position point that can be used to determine the occlusion of the module corresponding to the custom element. The layout position can be determined based on the placement position and the hierarchical position. After obtaining the movement trajectory of the custom element, the layout position of the module corresponding to the custom element in the physical model can be determined based on the movement trajectory. After determining the layout position of the module corresponding to the custom element, in order to avoid the subsequent acquisition of the module outline corresponding to the custom element again, the module outline corresponding to the custom element can be updated to a marked outline, that is, the pending outline corresponding to the pending group is updated to a marked outline, and the unmarked pending outline in the image frame is subsequently acquired.

[0100] In some embodiments, step S25 includes S251 to S256, which are specifically as follows:

[0101] S251: When the moving track is interrupted, determining that the hierarchical attribute of the custom element is an obscured attribute, and determining the hierarchical position of the custom element as a lower layer position according to the obscured attribute.

[0102] Specifically, when the movement track of a custom element is interrupted, it can be explained that the module corresponding to the custom element may be completely moved under another module. At this time, the outline of the module corresponding to the custom element cannot be found in the video data. Therefore, it can be considered that the module corresponding to the custom element may be completely blocked by the module corresponding to another custom element. In this case, the hierarchical attribute of the corresponding custom element can be determined as the blocked attribute, and the corresponding hierarchical position can be determined as the lower position. Among them, the blocked attribute refers to the attribute corresponding to the blocked custom element, and the lower position refers to the hierarchical position corresponding to the blocked custom element.

[0103] S252: Determine the blocked direction of the customized element according to the moving trajectory, and determine the placement position of the customized element based on the blocked direction.

[0104] In actual applications, when a module corresponding to a customized element is completely blocked by a module corresponding to another customized element, it can be considered that the placement position of the module corresponding to the blocked customized element is below the module corresponding to the other customized element. At this time, the approximate position of the module corresponding to the customized element can be determined based on the moving trajectory of the pending contour. Specifically, the moving trajectory can be used to determine the direction in which the module corresponding to the customized element may continue to move after being completely blocked, that is, the blocked direction. It can be considered that the placement position of the module corresponding to the customized element is very likely to be in the blocked direction.

[0105] The blocked direction refers to the direction in which the module corresponding to the custom element may continue to move after being completely blocked.

[0106] In some embodiments, the placement of the custom element may be determined by:

[0107] S2521, obtaining the previous image frame when the moving trajectory is interrupted as the target frame, and determining the contour distance between the to-be-determined contour and each of the marked contours according to the center point of each of the marked contours in the target frame and the positioning point of the to-be-determined contour corresponding to the customized element.

[0108] It can be understood that when the moving trajectory of a customized element is interrupted, the outline of the module corresponding to the customized element can be seen before the moving trajectory is interrupted. Therefore, the last frame of image in which the module is blocked can be determined through the previous frame of image, and the module corresponding to the customized element may move according to the trajectory in the frame of image. The previous frame of image when the moving trajectory is interrupted can be determined as the target frame, and the last moving trajectory of the module corresponding to the customized element before being blocked can be obtained through the target frame. The placement position of the customized element can be roughly determined according to the moving trajectory, and the distance between another module that blocks the module may also be close to it. Therefore, the blocked module outline can be determined through the distance between the module and each outline, and the specific position of the module corresponding to the currently planned customized element is determined by the blocked module outline.

[0109] Specifically, since there are pending contours corresponding to the custom elements that are to be occluded and marked contours corresponding to multiple custom elements whose positions have been determined in the target frame, and the nearest marked contour is very likely to occlude the pending contour, the distance between the center point of the marked contour and the positioning point of the pending contour, that is, the contour distance, can be obtained, and the marked contour closest to the pending contour can be determined through the contour distance.

[0110] S2522: Determine that the marked contour whose contour distance is less than the stacking distance threshold and whose contour distance is the smallest is the target contour.

[0111] Among them, the superposition distance threshold refers to the maximum value for judging whether the distance between the pending contour and the marked contour is too close. When the contour distance is less than the superposition distance threshold, it can be considered that the distance between the pending contour and the marked contour is too close. Since there may be multiple marked contours with contour distances less than the superposition distance threshold in the same image frame, the marked contour with a contour distance less than the superposition contour distance and the smallest contour distance can be determined as the target contour. It can be considered that the module corresponding to the target contour is the occlusion encountered by the module corresponding to the obscured custom element during movement. Among them, the target contour refers to the contour of the occlusion encountered by the module corresponding to the obscured custom element during movement. It can be the contour of another custom element. For example, the currently planned module is a flower module. When the planner wants to place the flower module under the tree module, the tree module may obstruct the flower module. Then the target contour can be the contour corresponding to the tree module.

[0112] S2523, obtaining a tangent direction corresponding to the positioning point in the moving trajectory as the blocked direction, and determining the undetermined contour with the largest contour area in the undetermined group as the positioning contour.

[0113] In actual applications, when the moving trajectory is interrupted, it means that the module corresponding to the customized element may be placed below the corresponding module, and the position of the module corresponding to the customized element may be in the area where the outline of the corresponding module is located. The positioning of the module corresponding to the customized element can be roughly determined through the moving trajectory. Specifically, the direction in which the customized element continues to move can be predicted through the moving trajectory. The tangent direction corresponding to the positioning point on the moving trajectory can be determined as the direction in which the customized element continues to move after the moving trajectory is interrupted, that is, the blocked direction. After determining the blocked direction, the placement position of the customized element can be determined according to the blocked direction.

[0114] When determining the placement position, since the image data contained in the pending contour with the largest contour area is relatively complete, the pending contour with the largest contour area in the pending group can be determined as the positioning contour. When the positioning contour is completely blocked, it means that the module corresponding to the customized element is completely blocked.

[0115] S2524, using the positioning point as a reference, generating a positioning line parallel to the blocked direction, using the positioning line to locate the center point of the positioning contour, and performing positioning movement on the positioning contour until the target contour completely covers the positioning contour, and then stopping the positioning movement of the positioning contour.

[0116] Specifically, the positioning point corresponding to the undetermined contour in the target frame is the end point when the moving trajectory is interrupted. After the moving trajectory is interrupted, the custom element is likely to continue to move in the obstructed direction. Therefore, the positioning point can be used as a reference to generate a positioning line parallel to the obstructed direction, and the center point of the positioning contour is positioned on the positioning line. The positioning contour is controlled to move along the positioning line until the target contour completely covers the positioning contour. It can be considered that the positioning contour has reached the placement position, and the positioning movement of the positioning contour can be stopped at this time. Among them, the positioning line refers to the line segment for positioning the positioning contour, and the line segment corresponds to the obstruction direction.

[0117] S2525, obtaining the center point of the positioning contour when it stops moving as the placement position of the customized element.

[0118] Specifically, when the positioning outline stops moving, it can be considered that the positioning outline is completely blocked, so the approximate position of the positioning outline can be considered to be below the target outline, and the center point corresponding to the positioning outline when it stops moving can be determined as the placement position corresponding to the customized element.

[0119] S253, when there are a preset number of consecutive adjacent image frames with the same moving trajectory, obtain the area change trend of the pending contour in the image frames in the pending group, and determine the hierarchical attributes of the custom element according to the area change trend, wherein the hierarchical attributes include an unobstructed attribute and an obstructed attribute.

[0120] Among them, the preset number refers to the number of image frames corresponding to a preset continuous time period. For example, the number of image frames within 1 consecutive minute can be set as the preset number. When the moving trajectory is the same in the image frames within 1 consecutive minute, it can be considered that the customized element may have been placed. The hierarchical attribute refers to the attribute corresponding to whether the customized element is blocked. The hierarchical attribute corresponding to being blocked by another customized element is the blocked attribute, and the hierarchical attribute corresponding to not being blocked by another customized element is the unblocked attribute.

[0121] The placement position corresponding to the customized element can be determined by the end point of the moving trajectory, but the module corresponding to the customized element may be unobstructed at the placement position, or may be partially obstructed by a module corresponding to another customized element. In the above two situations, the corresponding area change trends are different. In order to determine the specific obstruction situation of the customized element, the area change trend of the pending contour can be obtained. The area change trend can be used to determine whether the module corresponding to the customized element is unobstructed or partially obstructed by a module corresponding to another customized element, and then the hierarchical attributes of the customized element can be determined.

[0122] In some embodiments, the step S253 of "obtaining an area change trend of the pending contour in the image frame in the pending group, and determining the hierarchical attribute of the custom element according to the area change trend" includes the following steps:

[0123] S2531, obtaining contour areas of the to-be-determined contours in adjacent image frames in the to-be-determined group, and obtaining area difference values ​​corresponding to adjacent image frames according to the difference values ​​of the contour areas.

[0124] In actual applications, when a customized element is not blocked, the contour area of ​​the customized element will hardly change during movement. When a customized element is partially blocked by another customized element, the contour area of ​​the customized element will gradually decrease and then remain unchanged during movement. Therefore, the hierarchical attribute corresponding to the customized element can be determined according to the change of the contour area corresponding to the customized element. Specifically, the contour area of ​​the undetermined contour in adjacent image frames can be obtained first, and by judging the difference between the contour areas corresponding to adjacent image frames, it can be determined whether the contour area of ​​the customized element changes during movement.

[0125] Among them, the contour area refers to the area of ​​the pending contour in the image frame, and the area difference refers to the difference between the contour areas of the pending contours in adjacent image frames, which can be the difference obtained by subtracting the contour area corresponding to the previous image frame from the contour area corresponding to the next image frame.

[0126] S2532: When the area difference values ​​of all adjacent image frames in the pending group are less than the area difference threshold, determine that the area change trend of the pending group is a fixed trend, and determine that the hierarchical attribute of the custom element is a non-occlusion attribute according to the fixed trend.

[0127] Among them, the area difference threshold refers to the maximum value for judging whether the area difference is too large, and the fixed trend refers to the trend that the contour area of ​​the customized element hardly changes during the movement. When the area difference values ​​corresponding to all adjacent image frames in the pending group are less than the area difference threshold, it can be considered that the contour area of ​​the pending contour corresponding to the pending group has hardly changed. Therefore, the area change trend corresponding to the pending group can be determined as a fixed trend. Since the contour area has hardly changed, it can be considered that the customized element is not obscured. Therefore, it can be determined that the hierarchical attribute of the customized element is the no-occlusion attribute, and when the hierarchical attribute of the customized element is the no-occlusion attribute, the customized element may or may not obstruct another customized element.

[0128] S2533, when there are area differences between adjacent image frames in the pending group that are greater than or equal to an area difference threshold, determine that the area change trend of the pending group is a dynamic trend, and determine that the hierarchical attribute of the custom element is an obscured attribute based on the dynamic trend.

[0129] Specifically, when the area difference values ​​corresponding to all adjacent image frames in the pending group are greater than or equal to the area difference threshold, it can be considered that the contour area of ​​the pending contour corresponding to the pending group has changed during the movement, so the area change trend corresponding to the pending group can be determined as a dynamic trend. Under the dynamic trend, it can be considered that the module corresponding to the customized element is partially blocked by the module corresponding to another customized element during the movement, so the hierarchical attribute of the customized element can be determined as the blocked attribute. The dynamic trend refers to the trend that the contour area of ​​the customized element changes during the movement.

[0130] See also Figure 2 , which is a schematic diagram of a case where the area change trend of a pending group is a dynamic trend provided by an embodiment of the present invention, such as Figure 2 As shown in the figure, from left to right, the pending contour 1 corresponding to the customized element 1 in the image frame is gradually blocked by the pending contour 2 corresponding to the customized element 2. The pending contour 1 corresponding to the customized element 1 can be the contour of a flower, and the pending contour 2 corresponding to the customized element 2 can be the contour of a tree. Therefore, the contour area of ​​the pending contour 1 corresponding to the customized element 1 is gradually decreasing and is gradually blocked by the customized element 2.

[0131] S254: Determine that the hierarchical position of the custom element with the non-blocking attribute is an upper position, and the hierarchical position of the custom element with the blocked attribute is a lower position.

[0132] Specifically, when the hierarchical attribute of a custom element is the unobstructed attribute, it can be considered that there is no obstruction above the module corresponding to the custom element, indicating that at the placement position corresponding to the custom element, the module corresponding to the custom element is in the upper layer, and the hierarchical position of the custom element can be determined to be the upper layer position. When the hierarchical attribute of a custom element is the obstructed attribute, it can be considered that there is an obstruction above the module corresponding to the custom element, indicating that at the placement position corresponding to the custom element, the module corresponding to the custom element is below another module, and the hierarchical position of the custom element can be determined to be the lower layer position. Among them, the upper layer position refers to the hierarchical position corresponding to the custom element when it is not obstructed, and the lower layer position refers to the hierarchical position corresponding to the custom element when it is obstructed.

[0133] S255, obtaining the center point of the undetermined contour as the placement position of the customized element when the moving trajectory remains unchanged.

[0134] Specifically, when the moving trajectory remains unchanged, it can be considered that the customized element is placed at the position where the moving trajectory remains unchanged, the pending contour corresponding to the position can be obtained, the position area where the pending contour is located is determined as the position area where the customized element may be placed, and the center point of the pending contour can be determined as the specific placement position of the customized element.

[0135] S256, obtaining the layout position of the custom element based on the placement position and the hierarchical position, retrieving the element tag corresponding to the custom element and binding it with the pending group, and updating the pending outline of the determined layout position to a marked outline.

[0136] Specifically, the layout position of the customized element in the physical model can be determined by the placement position and the hierarchical position. Since the element tag in this solution can correspond to the customized element, after obtaining the layout position of the customized element, the element tag corresponding to the customized element can be called to bind the pending group corresponding to the customized element, so that the pending group corresponding to the customized element can be marked. Through the element tag, it can be determined that the layout position of the corresponding customized element has been determined, and then the pending outline corresponding to the customized element can be updated to the marked outline.

[0137] Through the above implementation, the layout position corresponding to the customized element can be obtained more accurately, so that the generated virtual twin space can be more closely aligned with the physical model.

[0138] S26, calling the virtual module corresponding to the customized element, obtaining the three-dimensional coordinate system corresponding to the initial twin space, and determining the X-axis coordinate value and the Y-axis coordinate value corresponding to the virtual module according to the top-view coordinate corresponding to the placement position in the image frame.

[0139] In actual applications, each customized element has a corresponding virtual module in the database. After determining the layout position of the customized element, the virtual module corresponding to the customized element can be called out, and the virtual position corresponding to the layout position can be determined in the initial twin space. The virtual module can be moved to the corresponding virtual position, and the initial twin space can be customized and updated.

[0140] Among them, the virtual module refers to the module in the database corresponding to the customized element, and the virtual position refers to the corresponding position of the customized element in the twin space.

[0141] After obtaining the corresponding layout position of the customized element in the physical model, this solution can pre-configure a three-dimensional coordinate system corresponding to the initial twin space, and the plane in the three-dimensional coordinate system and the image data under the top-down perspective correspond to each other. The layout position of the customized element can be mapped to the virtual position in the initial twin space through the three-dimensional coordinate system corresponding to the initial twin space.

[0142] Since a bird's-eye view is used when recording the construction process of the physical module, the image frames obtained are from a bird's-eye view. Therefore, the image frames can be coordinate-converted and the bird's-eye view coordinates can be converted to corresponding positions in the three-dimensional coordinate system, so that the X-axis coordinate value and Y-axis coordinate value corresponding to the virtual module in the three-dimensional coordinate system can be determined.

[0143] S27, determining that the hierarchical position of the custom element corresponds to a preset height coordinate value, and determining the Z-axis coordinate value corresponding to the virtual module according to the height coordinate value.

[0144] After determining the X-axis coordinate value and Y-axis coordinate value corresponding to the virtual module, this solution can pre-configure the corresponding height coordinate value for the custom element based on the hierarchical position of the custom element, and the Z-axis coordinate value corresponding to the virtual module can be determined based on the preset height coordinate value.

[0145] In actual applications, if the module corresponding to the customized element needs to contact the bottom surface of the physical model, then when the virtual module corresponding to the customized element is placed, the bottom surface of the virtual module corresponding to the customized element needs to contact the bottom surface of the virtual twin space. For example, when the customized element is a willow tree, since the willow tree is in contact with the bottom surface of the physical model in the physical model, the bottom surface of the virtual module corresponding to the willow tree also needs to contact the bottom surface in the virtual twin space.

[0146] The height coordinate value refers to the coordinate value corresponding to the preset position height of the customized element, and the preset height coordinate values ​​corresponding to different customized elements may be different.

[0147] S28, determining a virtual position according to the X-axis coordinate value, the Y-axis coordinate value and the Z-axis coordinate value, positioning the virtual module at the virtual position, and performing customized update on the initial twin space.

[0148] Specifically, after obtaining the X-axis, Y-axis and Z-axis coordinate values ​​of the customized element in the virtual twin space, the X-axis, Y-axis and Z-axis coordinate values ​​can be combined to form a three-dimensional coordinate point, which corresponds to the virtual position of the customized element in the initial twin space. By moving the virtual module to the corresponding virtual position, the initial twin space can be customized and updated.

[0149] Through the above implementation, the virtual twin space can be quickly constructed in combination with the needs of planners, the planning efficiency can be improved, and the virtual twin space can be made to fit the physical model more closely.

[0150] S3, determining the dynamic planning area that does not meet the cost planning conditions as the area to be adjusted, calling the planning adjustment strategy to adjust and update the virtual planning area corresponding to the area to be adjusted in the initial twin space, and obtaining the customized display space corresponding to the corresponding planning end.

[0151] It is understandable that when planning a park, the planning cost is limited. In order to ensure that the cost corresponding to the planner's planning data does not exceed the planning cost, the planner can exercise corresponding control over the planning cost during the planning process.

[0152] In practical applications, there are multiple dynamic planning areas corresponding to the planned park. When the planning data in the dynamic planning area does not meet the cost planning conditions, this scheme can determine the dynamic planning area that does not meet the cost planning conditions as the area to be adjusted, and then determine the virtual planning area corresponding to the area to be adjusted in the initial twin space, and call the corresponding adjustment planning strategy to adjust the planning data in the virtual planning area, so that the cost corresponding to the adjusted planning data meets the cost planning conditions.

[0153] Among them, the area to be adjusted refers to the dynamic planning area that does not meet the cost planning conditions, the planning adjustment strategy refers to the strategy for planning adjustment of customized elements in the area to be adjusted, and the virtual planning area refers to the area in the initial twin space corresponding to the area to be adjusted.

[0154] Based on the above embodiment, the specific implementation of step S3 may be:

[0155] S31, calculating the estimated cost corresponding to each of the customized elements to obtain an estimated total cost, determining that the dynamic planning area where the estimated total cost is greater than the planned cost does not meet the cost planning condition, and obtaining the corresponding dynamic planning area as the area to be adjusted.

[0156] Among them, the estimated cost refers to the cost corresponding to each customized element, the estimated total cost refers to the total cost corresponding to all customized elements in the dynamic planning area, the planned cost refers to the limit cost corresponding to the dynamic planning area, and the planned cost corresponding to each dynamic planning area can be different. The cost planning condition means that the estimated total cost is less than or equal to the planned cost.

[0157] This solution can pre-configure the corresponding cost for each customized element. When the planner places the customized elements in the dynamic planning area, the server can count the estimated total costs corresponding to all the customized elements in the dynamic planning area. Each time the planner places a customized element in the dynamic planning area, the estimated total cost corresponding to the dynamic planning area will increase accordingly. The estimated total cost is compared with the planned cost corresponding to the dynamic planning area. If the estimated total cost exceeds the planned cost, it can be considered that the current dynamic planning area does not meet the cost planning conditions and is determined as a waiting area. Subsequently, the customized elements configured in the waiting area can be adjusted accordingly so that the estimated total cost of the final dynamic planning area meets the cost planning conditions.

[0158] S32, receiving an adjustment area demarcated by the planning end in a virtual planning area corresponding to the area to be adjusted, and obtaining an adjustment element configured by the planning end for the adjustment area.

[0159] Specifically, when the estimated total cost in the dynamic planning area is greater than the corresponding planning cost, the server can remind the planner to stop adding customized elements in the dynamic planning area. At this time, the planner can choose to adjust the customized elements in the dynamic planning area or not. When the planner chooses to adjust the customized elements in the dynamic planning area, the twin space can be adjusted to a bird's-eye view, and the virtual planning area corresponding to the area to be adjusted in the initial twin space can be determined from the bird's-eye view. The initial twin space from the bird's-eye view can then be sent to the planner, and the planner can demarcate the virtual planning area so as to obtain the adjustment area where the planner wants to make planning adjustments.

[0160] Since the customized elements that each planner wants to adjust are different, the adjustment elements corresponding to the adjustment area can be obtained in combination with the needs of the planners. For example, if the customized element that planner A wants to adjust is willow, then the adjustment element configured by planner A can be willow. If the customized element that planner B wants to adjust is rose, then the adjustment element configured by planner B can be rose. When obtaining the adjustment elements, multiple corresponding customized elements in the virtual planning area can be obtained, and the multiple customized elements can be generated into a corresponding element list, which is sent to the planning end. The planner can select the customized element that he wants to adjust from the list and determine it as the adjustment element. Among them, the adjustment area refers to the area where the planner wants to adjust the planning, and the adjustment element refers to the customized element that the planner wants to adjust the planning. Among them, customized elements that cannot be adjusted cannot be used as adjustment elements. For example, if the lawn cannot be adjusted, then the lawn cannot be used as an adjustment element.

[0161] S33, calling a corresponding planning adjustment strategy according to the adjustment attribute of the adjustment element, wherein the planning adjustment strategy includes a spacing adjustment strategy and a density adjustment strategy.

[0162] Among them, the adjustment attributes include spacing attributes and density attributes. The spacing attribute corresponds to elements whose spacing can be adjusted, such as trees. The density attribute corresponds to attributes whose density can be adjusted, such as flowers. The corresponding spacing adjustment strategy can be configured for the spacing attribute, and the corresponding density adjustment strategy can be configured for the density attribute. The spacing adjustment strategy refers to the strategy for adjusting the module spacing, and the density adjustment strategy refers to the strategy for adjusting the module density. Specifically, after obtaining the adjustment element that the planner wants to adjust, the planning adjustment strategy corresponding to the adjustment attribute of the adjustment element can be called to adjust the adjustment element in different ways.

[0163] S34, adjusting and updating the virtual modules corresponding to the adjustment elements in the virtual planning area based on the planning adjustment strategy, and updating the adjustment area according to the virtual modules to obtain the customized display space corresponding to the corresponding planning end.

[0164] According to the selected planning adjustment strategy, the virtual modules corresponding to the adjustment elements in the virtual planning area can be adjusted, which may change the position of the virtual modules. As the virtual modules are adjusted, the adjustment area can be updated in real time to reflect the latest layout of the virtual modules. Ultimately, a customized display space that meets cost planning conditions and meets the needs of planners can be generated.

[0165] In some embodiments, the specific implementation of step S34 may be:

[0166] S34A1, obtaining, according to the spacing adjustment strategy, a virtual module corresponding to an adjustment element of a spacing attribute as a first adjustment module, and obtaining a first region where the first adjustment module is located.

[0167] The first adjustment module refers to a virtual module to be adjusted in spacing, and the first area refers to an area where the first module is located. When the adjustment attribute of the adjustment element is a spacing attribute, the corresponding spacing adjustment strategy can be called to adjust the adjustment element, and the virtual module corresponding to the adjustment element of the spacing attribute can be determined as the first adjustment module.

[0168] When determining the first area, the twin space can be adjusted to a bird's-eye view to obtain an image under the bird's-eye view, so that a rectangular frame corresponding to the area where the first adjustment module is located can be generated based on the image under the bird's-eye view. Specifically, when generating the rectangular frame, multiple pixel points corresponding to the first adjustment module in the image under the bird's-eye view can be obtained, and the multiple pixel points are coordinate-converted. According to the coordinates of each pixel point, the maximum x-axis coordinate value, the minimum x-axis coordinate value, the maximum y-axis coordinate value, and the minimum y-axis coordinate value can be obtained. Then, four mutually perpendicular line segments are generated based on these four extreme coordinate values, and a rectangular frame is obtained through the four line segments. The area corresponding to the rectangular frame is the first area.

[0169] S34A2, obtaining a first placement direction of the first adjustment module, and determining a second placement direction configured by the planning end for the adjustment area.

[0170] Among them, the first placement direction refers to the placement direction of the first adjustment module, and the second placement direction refers to the placement direction of the virtual module configured by the planner for the adjustment area.

[0171] In some embodiments, when obtaining the first placement direction, the center point of the first first adjustment module and the center point of the last first adjustment module in the first area can be obtained first, and the two center points can be connected. The first placement direction can be from the center point of the first adjustment module to the center point of the last first adjustment module.

[0172] See also Figure 3, which is a schematic diagram of adjusting a virtual planning area based on a spacing adjustment strategy provided by an embodiment of the present invention, such as Figure 3 As shown in , the first adjustment module can be placed from left to right, so the first placement direction can be from left to right, and the planner can configure a placement direction from top to bottom for the adjustment area, so the second placement direction can be from top to bottom.

[0173] S34A3, determine a first length of the first area in the first placement direction and a second length of the adjustment area in the second placement direction, and obtain a total area length based on the sum of the first length and the second length.

[0174] The first length refers to the length of the first area in the first placement direction, the second length refers to the length of the adjustment area in the second placement direction, and the total area length refers to the sum of the first length and the second length.

[0175] In actual applications, multiple coordinate values ​​corresponding to the first area can be obtained. When the first placement direction is consistent with the x-axis direction, the value obtained by subtracting the minimum x-axis coordinate value from the maximum x-axis coordinate value can be the first length. By coordinate-converting the adjustment area, multiple coordinate values ​​corresponding to the adjustment area can be obtained. When the second placement direction is consistent with the y-axis direction, the value obtained by subtracting the minimum y-axis coordinate value from the maximum y-axis coordinate value can be the second length.

[0176] S34A4, obtaining an adjusted placement spacing according to a ratio of the total area length to the total number of modules of the first adjustment module, and adjusting the spacing of the first adjustment module based on the adjusted placement spacing.

[0177] The total number of modules refers to the total number of first adjustment modules in the first area, and the adjusted placement spacing refers to the spacing between the adjusted first adjustment modules, for example Figure 3 As shown in , the total number of modules of the first adjustment module is 7. Then, by calculating the ratio of the total area length to the total number of modules, the adjusted placement spacing can be obtained, and therefore the spacing distance between the first adjustment modules can be adjusted according to the adjusted placement spacing.

[0178] S34A5, obtaining the first adjustment module whose position is not in the first area after adjustment as the first moving module, updating the first moving module to the adjustment area according to the second placement direction and the adjusted placement spacing, and obtaining the customized display space corresponding to the corresponding planning end.

[0179] It is understandable that after adjusting the first adjustment modules in the first area according to the adjustment placement spacing, the spacing between the first adjustment modules will increase, and the number of first adjustment modules that can be placed in the first area will decrease accordingly. Figure 3As shown in , there are originally 7 first adjustment modules in the first area. When the interval distance between the first adjustment modules is increased, the number of modules that can be placed in the first area will become 4. Then, the placement positions of 3 first adjustment modules will exceed the range of the first area. These 3 first adjustment modules that exceed the range of the first area can be determined as first mobile modules, and these 3 first mobile modules can be moved to the adjustment area for placement.

[0180] In practical applications, when placing the first mobile modules, since the second placement direction can be consistent with the y-axis direction, the first first mobile module can be placed at the minimum y-axis coordinate value, and then the second and third first mobile modules can be placed in the second placement direction according to the same adjusted placement spacing. Figure 3 As shown in the diagram on the right, the spacing between the three first adjustment modules is the same. By adjusting and updating the first adjustment modules, a customized display space that meets the cost planning conditions and meets the needs of planners can be obtained. Among them, the first movable module refers to the first adjustment module that exceeds the scope of the first area after the spacing adjustment. The first movable module can be moved to the adjustment area for placement.

[0181] In some other embodiments, the specific implementation of step S34 may also be:

[0182] S34B1, obtaining a virtual module corresponding to an adjustment element of a density attribute according to the density adjustment strategy as a second adjustment module, and obtaining a second region where the second adjustment module is located.

[0183] The second adjustment module refers to the virtual module to be density adjusted, the second area refers to the area where the second module is located, and when acquiring the second area, the same method as the first area can be used to acquire it. When the adjustment attribute of the adjustment element is a density attribute, the corresponding density adjustment strategy can be called to adjust the adjustment element, and the virtual module corresponding to the adjustment element of the density attribute can be determined as the second adjustment module.

[0184] S34B2: Count the total area of ​​the second region and the adjustment zone, and obtain an adjustment density ratio according to a ratio of the total number of the second adjustment modules in the second region to the total area.

[0185] See also Figure 4 , which is a schematic diagram of adjusting a virtual planning area based on a density adjustment strategy provided by an embodiment of the present invention, such as Figure 4 As shown in , the total area refers to the sum of the area of ​​the second area and the area of ​​the adjustment zone. The adjustment density ratio can be used to represent the number of modules per unit area. According to the total number of modules and the total area, the corresponding adjustment density ratio can be obtained.

[0186] S34B3, performing density adjustment on the second adjustment modules in the second area according to the adjustment density ratio, and obtaining redundant second adjustment modules as second mobile modules.

[0187] It can be understood that after the second adjustment modules in the second area are adjusted according to the adjustment density ratio, the density of the second adjustment modules in the second area will be reduced, and the number of second adjustment modules that can be placed in the second area will be reduced accordingly, so that redundant second adjustment modules will be generated. The redundant second adjustment modules can be determined as second mobile modules, and the second mobile modules can be moved to the adjustment area for placement. Figure 4 As shown in , there are originally 24 second adjustment modules in the second area. After the corresponding adjustment density ratio is obtained, the second adjustment modules that can be placed in the second area are adjusted to 16, so there will be 8 extra second adjustment modules. These 8 second adjustment modules can be determined as second mobile modules, and the second mobile modules can be moved to the adjustment area for placement.

[0188] The second movable module is a module that can be moved to the adjustment area for placement after density adjustment.

[0189] S34B4: Fill and update the adjustment area based on the second mobile module to obtain a customized display space corresponding to the corresponding planning end.

[0190] like Figure 4 As shown in , after the density of the second adjustment modules in the second area is adjusted, the second adjustment modules in the second area become 16, and the second mobile modules in the adjustment area are 8. By updating the placement of the second mobile modules in the adjustment area, a customized display space corresponding to the needs of planners can be obtained.

[0191] Through the above implementation, a customized display space that meets cost planning conditions and meets the needs of planners can be obtained.

[0192] S4, obtaining the selection coefficient corresponding to each of the customized display spaces, and determining the customized display space with the largest selection coefficient as the planned selected space corresponding to the planned park.

[0193] It is understandable that when planning and designing a park, there are multiple planners who are responsible for the planning, and each planner's planning scheme is different. Therefore, the customized display space corresponding to each planner is different. In order to determine the best customized display space from multiple customized display spaces, the selection coefficient corresponding to each customized display space can be obtained, and the final planned selected space can be determined according to the selection coefficient.

[0194] Specifically, after obtaining the customized display spaces corresponding to each planner, the selection coefficient corresponding to each customized display space may be counted, and the customized display space with the largest priority coefficient may be selected as the final planned selected space for display.

[0195] Among them, the selection coefficient refers to the numerical value that evaluates the customized display space from the two aspects of cost and popularity.

[0196] Based on the above embodiment, the specific implementation of step S4 may be:

[0197] S41, obtaining a total planning cost of each of the customized display spaces, and obtaining a first offset coefficient according to a ratio of a benchmark planning cost to the total planning cost.

[0198] Among them, the total planning cost refers to the sum of the planning costs of multiple dynamic planning areas in the customized display space, and the benchmark planning cost refers to the standard planning cost corresponding to the preset customized display space. By calculating the ratio of the benchmark planning cost to the total planning cost, the corresponding first offset coefficient can be obtained. The smaller the total planning cost, the larger the first offset coefficient.

[0199] S42, determining the heat value of each of the customized display spaces, and obtaining a second offset coefficient according to a ratio of the heat value to a reference heat value.

[0200] Among them, the heat value refers to a value that can be used to measure the popularity of a customized display space. The larger the heat value, the more popular the corresponding customized display space is. The benchmark heat value refers to a preset value used to represent normal popularity.

[0201] By calculating the ratio of the heat value to the benchmark heat value, the corresponding second offset coefficient can be obtained. The larger the heat value, the larger the second offset coefficient, which indicates that the corresponding customized display space is more popular. When obtaining the heat value, the customized display spaces corresponding to multiple planners can be uploaded to the public platform for users to vote. The more votes obtained, the larger the corresponding heat value.

[0202] S43, obtaining a selection coefficient corresponding to the corresponding customized display space according to the sum of the first offset coefficient and the second offset coefficient, and determining the customized display space with the largest selection coefficient as the planned selected space corresponding to the planned park.

[0203] Specifically, the priority coefficient corresponding to the customized display space is obtained by adding the first offset coefficient and the second offset coefficient. The selected coefficient comprehensively considers the two key factors of cost and popularity, and can provide a more comprehensive evaluation basis for selecting the best customized display space. The customized display space with the largest selected coefficient can be used as the final planned selected space for the planned park.

[0204] Through the above implementation, a customized display space with optimal planning cost and high popularity can be obtained.

[0205] See also Figure 5 , is a structural diagram of a visualization system based on digital twins provided in an embodiment of the present invention, and the visualization system based on digital twins includes:

[0206] A response module is used to respond to the automatic modeling request of each planning end for the planned park, and obtain the video data of the physical model shot by the recording device based on the acquisition perspective, wherein the physical model includes a dynamic planning area and a fixed planning area;

[0207] An updating module, configured to determine, according to the video data, the customized elements configured by each planning end for the dynamic planning area, and to customize and update the initial twin space corresponding to the entity model based on the customized elements;

[0208] An adjustment module is used to determine the dynamic planning area that does not meet the cost planning conditions as the area to be adjusted, and to call the planning adjustment strategy to adjust and update the virtual planning area corresponding to the area to be adjusted in the initial twin space to obtain a customized display space corresponding to the corresponding planning end;

[0209] The acquisition module is used to acquire the selection coefficient corresponding to each of the customized display spaces, and determine that the customized display space with the largest selection coefficient is the planned selected space corresponding to the planned park.

[0210] Figure 5 The apparatus of the embodiment shown can be used to perform Figure 1 The implementation principles and technical effects of the steps in the method embodiment shown are similar and will not be repeated here.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A visualization method based on digital twins, characterized in that: include: In response to automatic modeling requests from each planning terminal for a planned park, video data of a physical model shot by a recording device based on a collection perspective is obtained, wherein the physical model includes a dynamic planning area and a fixed planning area; Determining, according to the video data, customized elements configured by each planning end for the dynamic planning area, and customizing and updating the initial twin space corresponding to the entity model based on the customized elements, including: Determining a pending contour and a pending group according to the video data; Determine the center point of each undetermined contour in the undetermined group as the positioning point, and obtain the moving trajectory of the customized element according to the positioning point; Determine the layout position of the custom element according to the moving trajectory, and update the pending outline corresponding to the pending group to the marked outline based on the layout position, wherein the layout position includes the placement position and the hierarchical position; Retrieve the virtual module corresponding to the customized element, obtain the three-dimensional coordinate system corresponding to the initial twin space, and determine the X-axis coordinate value and Y-axis coordinate value corresponding to the virtual module according to the top-view coordinate corresponding to the placement position in the image frame; Determine the preset height coordinate value corresponding to the hierarchical position of the custom element, and determine the Z-axis coordinate value corresponding to the virtual module according to the height coordinate value; Determine the virtual position according to the X-axis coordinate value, the Y-axis coordinate value, and the Z-axis coordinate value, and position the virtual module at the virtual position to customize and update the initial twin space; Determine the dynamic planning area that does not meet the cost planning conditions as the area to be adjusted, call the planning adjustment strategy to adjust and update the virtual planning area corresponding to the area to be adjusted in the initial twin space, and obtain the customized display space corresponding to the corresponding planning end; The selection coefficient corresponding to each of the customized display spaces is obtained, and the customized display space with the largest selection coefficient is determined as the planned selected space corresponding to the planned park.

2. The method according to claim 1, characterized in that Determining the customized elements configured by each planning end for the dynamic planning area according to the video data, and customizing and updating the initial twin space corresponding to the entity model based on the customized elements, further comprising: Arranging multiple image frames corresponding to the video data in time order to obtain a processing sequence, and sequentially acquiring module contours in each of the image frames in the processing sequence; Determine that a module outline with an element tag is a marked outline, and a module outline without the element tag is a pending outline; The pending contours in the adjacent image frames whose contour similarity is greater than the contour similarity threshold are acquired as a pending group, and feature recognition is performed on the pending group to obtain the customized elements corresponding to the pending group.

3. The method according to claim 2, characterized in that Determining the layout position of the custom element according to the movement trajectory, and updating the pending outline corresponding to the pending group to a marked outline based on the layout position, wherein the layout position includes a placement position and a hierarchical position, including: When the moving track is interrupted, determining that the hierarchical attribute of the custom element is an obscured attribute, and determining the hierarchical position of the custom element as a lower layer position according to the obscured attribute; Determining a blocked direction of the customized element according to the movement trajectory, and determining a placement position of the customized element based on the blocked direction; When there are a preset number of consecutive adjacent image frames with the same moving tracks, obtaining an area change trend of the pending contour in the image frames in the pending group, and determining a hierarchical attribute of the custom element according to the area change trend, wherein the hierarchical attribute includes an unobstructed attribute and an obstructed attribute; Determine that the hierarchical position of the custom element with no obstruction attribute is an upper position, and the hierarchical position of the custom element with obstruction attribute is a lower position; The center point of the undetermined contour is obtained as the placement position of the customized element when the moving trajectory remains unchanged; The layout position of the custom element is obtained based on the placement position and the hierarchical position, the element tag corresponding to the custom element is retrieved and bound to the pending group, and the pending outline of the determined layout position is updated to a marked outline.

4. The method according to claim 3, characterized in that Determining the blocked direction of the customized element according to the movement trajectory, and determining the placement position of the customized element based on the blocked direction, including: The previous image frame when the moving trajectory is interrupted is acquired as a target frame, and the contour distance between the to-be-determined contour and each of the marked contours is determined according to the center point of each of the marked contours in the target frame and the positioning point of the to-be-determined contour corresponding to the customized element; Determine that the marked contour whose contour distance is less than the stacking distance threshold and whose contour distance is the smallest is the target contour; Acquire a tangent direction corresponding to the positioning point in the moving trajectory as the blocked direction, and determine the undetermined contour with the largest contour area in the undetermined group as the positioning contour; Taking the positioning point as a reference, generating a positioning line parallel to the blocked direction, positioning the center point of the positioning contour by using the positioning line, and performing positioning movement on the positioning contour until the positioning contour is completely covered by the target contour, and then stopping the positioning movement of the positioning contour; The center point of the positioning contour when it stops moving is obtained as the placement position of the customized element.

5. The method according to claim 3, characterized in that: Acquiring an area change trend of the pending contour in the image frame in the pending group, and determining a hierarchical attribute of the custom element according to the area change trend, including: Acquire contour areas of the to-be-determined contours in adjacent image frames in the to-be-determined group, and obtain area difference values ​​corresponding to adjacent image frames according to the difference values ​​of the contour areas; When the area difference values ​​of all adjacent image frames in the pending group are less than the area difference threshold, determining that the area change trend of the pending group is a fixed trend, and determining that the hierarchical attribute of the custom element is a non-occlusion attribute according to the fixed trend; When there are area differences between adjacent image frames in the pending group that are greater than or equal to an area difference threshold, the area change trend of the pending group is determined to be a dynamic trend, and the hierarchical attribute of the custom element is determined to be an obscured attribute according to the dynamic trend.

6. The method according to claim 1, characterized in that Determine the dynamic planning area that does not meet the cost planning conditions as the area to be adjusted, call the planning adjustment strategy to adjust and update the virtual planning area corresponding to the area to be adjusted in the initial twin space, and obtain the customized display space corresponding to the corresponding planning end, including: Counting the estimated costs corresponding to the customized elements to obtain the estimated total cost, determining that the dynamic planning area where the estimated total cost is greater than the planned cost does not meet the cost planning condition, and obtaining the corresponding dynamic planning area as the area to be adjusted; Receiving an adjustment area demarcated by the planning end in a virtual planning area corresponding to the area to be adjusted, and acquiring an adjustment element configured by the planning end for the adjustment area; Retrieving a corresponding planning adjustment strategy according to the adjustment attribute of the adjustment element, wherein the planning adjustment strategy includes a spacing adjustment strategy and a density adjustment strategy; Based on the planning adjustment strategy, the virtual module corresponding to the adjustment element in the virtual planning area is adjusted and updated, and the adjustment area is updated according to the virtual module to obtain the customized display space corresponding to the corresponding planning end.

7. The method according to claim 6, characterized in that Adjusting and updating the virtual module corresponding to the adjustment element in the virtual planning area based on the planning adjustment strategy, and updating the adjustment area according to the virtual module to obtain a customized display space corresponding to the corresponding planning end, including: According to the spacing adjustment strategy, a virtual module corresponding to an adjustment element of the spacing attribute is obtained as a first adjustment module, and a first area where the first adjustment module is located is obtained; Acquire a first placement direction of the first adjustment module, and determine a second placement direction configured by the planning end for the adjustment area; Determine a first length of the first area in the first placement direction and a second length of the adjustment area in the second placement direction, and obtain a total area length according to the sum of the first length and the second length; Obtaining an adjusted placement spacing according to a ratio of the total area length to the total number of modules of the first adjustment module, and adjusting the spacing of the first adjustment modules based on the adjusted placement spacing; The first adjustment module whose position is not in the first area after adjustment is obtained as the first moving module, and the first moving module is updated to the adjustment area according to the second placement direction and the adjusted placement spacing to obtain the customized display space corresponding to the corresponding planning end.

8. The method according to claim 6, characterized in that Adjusting and updating the virtual module corresponding to the adjustment element in the virtual planning area based on the planning adjustment strategy, and updating the adjustment area according to the virtual module to obtain a customized display space corresponding to the corresponding planning end, including: According to the density adjustment strategy, a virtual module corresponding to an adjustment element of a density attribute is obtained as a second adjustment module, and a second area where the second adjustment module is located is obtained; Counting the total area of ​​the second region and the adjustment zone, and obtaining an adjustment density ratio according to a ratio of the total number of the second adjustment modules in the second region to the total area; Density adjustment is performed on the second adjustment modules in the second area according to the adjustment density ratio, and redundant second adjustment modules are obtained as second mobile modules; The adjustment area is filled and updated based on the second moving module to obtain a customized display space corresponding to the corresponding planning end.

9. The method according to claim 1, characterized in that: Obtaining the selection coefficient corresponding to each of the customized display spaces, and determining that the customized display space with the largest selection coefficient is the planned selected space corresponding to the planned park, including: Obtaining a total planning cost of each of the customized display spaces, and obtaining a first offset coefficient according to a ratio of a benchmark planning cost to the total planning cost; Determining a heat value of each of the customized display spaces, and obtaining a second offset coefficient according to a ratio of the heat value to a reference heat value; According to the sum of the first offset coefficient and the second offset coefficient, a selection coefficient corresponding to the corresponding customized display space is obtained, and the customized display space with the largest selection coefficient is determined as the planned selected space corresponding to the planned park.

10. A digital twin-based visualization system according to any one of claims 1 to 9, characterized in that: include: A response module is used to respond to the automatic modeling request of each planning end for the planned park, and obtain the video data of the physical model shot by the recording device based on the acquisition perspective, wherein the physical model includes a dynamic planning area and a fixed planning area; An updating module, configured to determine, according to the video data, the customized elements configured by each planning end for the dynamic planning area, and to customize and update the initial twin space corresponding to the entity model based on the customized elements; An adjustment module is used to determine the dynamic planning area that does not meet the cost planning conditions as the area to be adjusted, and to call the planning adjustment strategy to adjust and update the virtual planning area corresponding to the area to be adjusted in the initial twin space to obtain a customized display space corresponding to the corresponding planning end; The acquisition module is used to acquire the selection coefficient corresponding to each of the customized display spaces, and determine that the customized display space with the largest selection coefficient is the planned selected space corresponding to the planned park.

Images