Three-dimensional dynamic label layout method and device under binocular visual angle based on force and electronic equipment

Abstract

The invention provides a three-dimensional dynamic label layout method and device under a binocular visual angle based on force and electronic equipment. The method comprises the following steps: acquiring a three-dimensional scene model; the three-dimensional scene model comprises at least one object; determining anchor point coordinates and label contents; inputting the three-dimensional scene model, each label content and the corresponding anchor point coordinates, optimizing the label layout by adopting a label layout algorithm, and outputting the optimized label layout; the label layout algorithm comprises a label layout initialization algorithm and a label layout dynamic updating algorithm; and the label layout dynamic updating algorithm is used for updating the label layout under the binocular view angle of the user and adaptively generating the updated label layout. According to the method, the three-dimensional label layout can be dynamically optimized according to the real-time visual angle of the user, the consistency of the label layout and the clear visibility of the label are ensured, and the user experience in a virtual reality or augmented reality environment is improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of virtual reality and augmented reality, and in particular, to a three-dimensional dynamic label layout method, device, and electronic device based on force under a binocular perspective. Background Art

[0002] In a three-dimensional virtual environment, label layout is an important part of information visualization. Especially in virtual reality (VR) and augmented reality (AR) applications, a good label layout can help users quickly identify and understand the content in a complex scene. With the rapid development of VR / AR technology, the user's demand for a clear and beautiful label layout is also increasing day by day.

[0003] However, the existing label layout methods mainly focus on static scenes and are single-object label layout methods, which can only be optimized based on a single perspective and ignore the importance of the binocular perspective in a three-dimensional scene. During actual use, the single-object label layout method may cause the labels to be misaligned or overlapped in the left and right eye views, reducing the readability and visual comfort of the labels. In addition, in label layout, the guiding lines between the labels and the anchor points often cross when the perspective changes, resulting in visual chaos and further increasing the user's cognitive burden. Some existing solutions, such as the label swapping method, can avoid the problem of guiding line crossing, but will cause sudden changes in the label positions, affecting the smoothness of the labels and the beauty of the layout.

[0004] Therefore, the existing single-object label layout methods cannot handle the problems brought by the dynamic change of the user's perspective. The existing methods for optimizing label layout based on a monocular perspective result in problems such as label overlap, visual chaos, and guiding line crossing when the user's perspective changes, affecting the user experience. How to dynamically adjust the label layout under a binocular perspective, avoid problems such as label overlap and guiding line crossing, and at the same time maintain the clarity and beauty of the label layout is still a technical problem that needs to be urgently solved by those skilled in the art. Summary of the Invention

[0005] The present invention provides a three-dimensional dynamic label layout method, device, and electronic device based on force under a binocular perspective, so as to solve the problem of visual chaos and the decline in user experience caused by improper handling of the monocular perspective during the dynamic perspective change of the label layout in the existing 3D virtual environment. By introducing a binocular perspective and combining with a physical force field model, the label layout is jointly optimized in real time from the left-eye and right-eye perspectives in a dynamic scene, improving the readability and visual comfort of the labels. The present invention can maintain the beauty and smooth movement of the labels in a dynamic three-dimensional scene, and avoid phenomena such as guiding line crossing and label overlap.

[0006] The present invention provides a three-dimensional dynamic label layout method based on force under a binocular perspective, including:

[0007] Obtain a three-dimensional scene model; the three-dimensional scene model includes at least one object; in the three-dimensional scene model, determine the anchor coordinates of each anchor point, and determine the label content led out by each anchor point through a guiding line;

[0008] Input the three-dimensional scene model, each label content and its corresponding anchor coordinates, and use a label layout algorithm to optimize the label layout and output the optimized label layout;

[0009] Among them, the label layout algorithm includes a label layout initialization algorithm and a label layout dynamic update algorithm; the label layout initialization algorithm is used to optimize the initial positions of each label; the label layout dynamic update algorithm is used to update the label layout under the binocular view of the user and adaptively generate the updated label layout.

[0010] Preferably, the label layout initialization algorithm is specifically:

[0011] Use the simulated annealing algorithm to perform a global optimization search on the initial positions of the labels, select label candidate positions for each label through multiple iterations, thereby generate a new label candidate layout based on the current label layout, and evaluate the quality of the label candidate layout through a predefined objective function, and then judge whether to accept or reject the label candidate positions of each label, so as to optimize the overall label layout and obtain the initial label layout that meets specific quality criteria.

[0012] Preferably, the objective function is:

[0013] E static = E overlap + E distance + E intersection (1)

[0014] Wherein:

[0015] E static represents the objective function;

[0016] E overlap represents the sum of the overlapping areas between labels and the overlapping areas between labels and objects in the label candidate layout;

[0017] E distance represents the sum of the distances between each pair of labels and their corresponding anchor points in the label candidate layout, that is, the sum of the lengths of all guiding lines;

[0018] E intersection represents the number of intersecting guiding lines in the label candidate layout;

[0019] In the objective function, E overlap 、E distance and Eintersection , are determined respectively through formula (2), formula (3) and formula (4):

[0020]

[0021] Wherein:

[0022] represents the sum of the overlapping areas between labels in the label candidate layout;

[0023] represents the sum of the overlapping areas between a label and an object in the label candidate layout;

[0024] N represents the total number of labels; M represents the total number of objects;

[0025] i and j respectively represent label i and label j , i = 1, 2,..., N, j = 1, 2,..., N, j ≠ i; k represents object k , k = 1, 2,..., M;

[0026] S i，j represents the overlapping area between label i and label j ; S i，k represents the overlapping area between label i and object k ;

[0027] anchor i represents the uniquely corresponding anchor point of label i ;

[0028] D(i, anchor i ) represents the distance from label i to anchor i ;

[0029] e i and e j respectively represent the guiding lines corresponding to label i and label j ;

[0030] I(e i , e j ) represents a function of whether guiding line e i and guiding line e j intersect; if guiding line e i and guiding line e jIf they intersect, the function value is 1; otherwise, the function value is 0.

[0031] ω 1 , ω 2 , ω 3 , ω 4 , respectively represent the weight coefficients of the overlapping area between labels, the weight coefficients of the overlapping area between labels and objects, the weight coefficients of the distance between labels and corresponding anchor points, and the weight coefficients of the intersection of guiding lines.

[0032] Preferably, the label layout dynamic update algorithm is as follows:

[0033] When the user's perspective changes, according to the user's latest perspective, based on the binocular vision principle, the label layout is processed separately for the left-eye perspective and the right-eye perspective, and the binocular vision label layout is jointly optimized. When jointly optimizing the binocular vision label layout, the resultant force received by each label is recalculated using a physical force field model; according to the resultant force received by each label, the label is moved a corresponding distance along the direction of the resultant force, thereby updating the position of the label, and thus obtaining the updated label layout.

[0034] Preferably, based on the binocular vision principle, processing the label layout separately for the left-eye perspective and the right-eye perspective specifically includes: setting two virtual cameras to simulate the perspectives of the left eye and the right eye respectively, projecting the three-dimensional scene model onto the screen spaces of the left-eye virtual camera and the right-eye virtual camera respectively, and then performing joint optimization to solve the problem of label overlap in the current perspective.

[0035] Preferably, recalculating the resultant force received by each label using a physical force field model specifically includes:

[0036] The resultant force received by each label is the resultant force of the repulsive force between labels, the attractive force between the label and the anchor point, the repulsive force of label-label overlap, the repulsive force of label-object overlap, and the force of guiding line intersection;

[0037] Among them:

[0038] The repulsive force between labels refers to the repulsive force received by each label from all other labels. The magnitude of this repulsive force is negatively correlated with the distance between labels, and is used to avoid overlap and being too close between labels, ensuring a certain distance between labels;

[0039] The attractive force between the label and the anchor point refers to the attractive force received by each label from its corresponding anchor point. The magnitude of the attractive force is positively correlated with the distance between the label and the anchor point, ensuring that the label does not move too far away from its corresponding anchor point;

[0040] The label-label overlapping repulsive force refers to the repulsive force that a label receives from the corresponding overlapping label when labels overlap with each other, so as to avoid visual overlap between labels and improve the clarity and readability of label layout;

[0041] The label-object overlapping repulsive force refers to the repulsive force that a label receives from the corresponding overlapping object when the label overlaps with the object, so as to avoid visual overlap between the label and the object and improve the clarity and readability of label layout;

[0042] The force of guide line intersection refers to the force exerted on the label associated with the intersecting guide lines when two guide lines intersect, so as to adjust the position of the label and eliminate the intersection of the guide lines.

[0043] Preferably, for the label label i , the resultant force F i is calculated by formula (5):

[0044]

[0045] Where:

[0046] ① represents the inter-label repulsive force that the label label i receives from the other N - 1 labels; it is calculated by formula (6):

[0047]

[0048] Where: N represents the total number of labels; i and j respectively represent the label label i and the label label j , i = 1, 2,..., N, j = 1, 2,..., N, j ≠ i; l i and l j respectively represent the position coordinates of the label label i and the label label j ; ||l i -l j || represents the distance between the label label i and the label label j ; represents the unit vector pointing from the label label j to the label label i ; k 1 represents the coefficient of inter-label repulsive force;

[0049] ② represents the label label i receiving the force from the corresponding anchor anchor iThe attraction force; calculated by formula (7):

[0050]

[0051] Where: a i Represents the position coordinates of the anchor i ; ||l i -a i || represents the distance between the label i and the anchor i ; Represents the unit vector pointing from the label i to the anchor i ; k 2 Represents the coefficient of the attraction force of the label by the anchor

[0052] ③ Represents the label-label overlap repulsive force of the label i from the overlapping label j ; calculated by formula (8):

[0053]

[0054] Where: Represents the unit vector pointing from the label j to the label i ; k 3 Represents the coefficient of the label-label overlap repulsive force; S i，j Represents the overlapping area of the label i and the label j ; calculated by formula (9):

[0055]

[0056] And Represent the overlapping areas of the label i and the label j from the left-eye view and the right-eye view respectively;

[0057] ④ Represents the label-object overlap repulsive force of the label i from the overlapping object k ; calculated by formula (10):

[0058]

[0059] Where: Represents a unit vector pointing from the object k to the label i ; k 4 Represents the coefficient of the label-object overlap repulsive force; S i,k Represents the label i and the object k The overlapping area, calculated by formula (11):

[0060]

[0061] and Respectively represent the overlapping areas of the label i and the object k in the left-eye view and the right-eye view;

[0062] ⑤ Represents the guiding line e of the label i and the guiding line e of the label i When they cross, the force on the label j due to the crossing of the guiding lines, calculated by formula (12): j Where: k i Represents the coefficient of the force of the guiding line crossing;

[0063]

[0064] Where: k 4 Represents the coefficient of the force of the guiding line crossing;

[0065] D i，j Represents the degree of intersection of the guiding line e i and the guiding line e j , calculated as:

[0066] Project the guiding line e i and the guiding line e j onto the screen plane of the main camera; In the screen plane, the label end of the guiding line e i is projected as the lp i point, and the anchor end of the guiding line e i is projected as the ap i point; The label end of the guiding line e j is projected as the lp j point, and the anchor end of the guiding line e j is projected as the ap j point; The guiding line e i and the guiding line e j intersect at the intersection point u in the screen plane; The intersection point u, lp i point, lp jThe area of the triangle formed by the points is area 1 ; the intersection points u, ap i points, ap j The area of the triangle formed by the points is area 2 ; Therefore, through formula (13), the guiding line e is obtained i and the guiding line e j The degree D of intersection i，j :

[0067]

[0068] Where: min(area 1 , area 2 ) represents the smaller value between the area area 1 and the area area 2 ;

[0069] represents the unit vector of the force of the guiding line intersection received by the label label i , determined by formula (14):

[0070]

[0071] Where: represents the normal vector of the plane determined by the anchor points anchor i , the anchor points anchor j and the label label i ; the normal vector of the plane determined by the anchor points anchor i , the anchor points anchor j and the label label j is θ 1 represents the angle between the normal vector and the normal vector , θ 2 represents the angle between the normal vector and the normal vector .

[0072] Preferably, the label layout dynamic update algorithm further includes:

[0073] When updating the label layout from the binocular perspective of the user, according to the movement and perspective change of the user, the orientation of the label is adjusted in real time so that the label always faces the user to improve the reading experience of the user.

[0074] The present invention also provides a system for implementing the three-dimensional dynamic label layout method based on force under binocular perspective described above, including:

[0075] An acquisition module, configured to acquire a three-dimensional scene model; the three-dimensional scene model includes at least one object; in the three-dimensional scene model, determine the anchor coordinates of each anchor point, and determine the label content led out by each of the anchor points through a guiding line;

[0076] A label layout algorithm module, configured to input the three-dimensional scene model, each label content and its corresponding anchor coordinates, optimize the label layout, and output an optimized label layout;

[0077] The label layout algorithm module includes:

[0078] A label layout initialization sub-module, configured to optimize the initial positions of each label and generate an initial label layout;

[0079] A label layout dynamic update sub-module, configured to update the label layout under the binocular view of the user and adaptively generate an updated label layout.

[0080] The present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the three-dimensional dynamic label layout method based on force under binocular view as described above.

[0081] The three-dimensional dynamic label layout method, device, and electronic device provided by the present invention obtain the real-time position and view angle of the user, and use a physical force field model to dynamically adjust the label layout on the basis of the binocular view to ensure the consistency and readability of the label layout under the binocular view; wherein, the physical force field model includes the repulsive force between labels, the attractive force between a label and an anchor point, the repulsive force of label-label overlap, the repulsive force of label-object overlap, and the force of guiding line intersection, so as to optimize the position and layout of the labels and avoid the problems of label overlap and guiding line intersection. Since a binocular view optimization mechanism is combined in the dynamic label layout process, the generated label layout can adapt to the changes in the user's position and view angle, avoid common problems such as visual ghosting in traditional label layout methods, and significantly improve the user's visual experience and interaction comfort. Therefore, the method, device, and electronic device provided by the embodiments of the present invention can dynamically optimize the three-dimensional label layout according to the real-time view angle of the user, ensure the consistency of the label layout and the clear visibility of the labels, and improve the user experience in a virtual reality or augmented reality environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly show the present invention and related technical solutions, the following will briefly introduce the drawings required in the embodiments or the prior art. Obviously, these drawings are only some embodiments of the present invention, and those skilled in the art can also derive other possible drawings from these drawings without creative efforts.

[0083] Figure 1 Schematic flowchart of the three-dimensional dynamic label layout method based on force under binocular vision provided by the present invention;

[0084] Figure 2 Example diagram of the three-dimensional model provided by the present invention;

[0085] Figure 3 Schematic diagram of the related concepts of the anchor point, guiding line and label provided by the present invention;

[0086] Figure 4 A usage scenario diagram showing multiple objects involved in the three-dimensional scene model and label annotation for each object;

[0087] Figure 5 Usage process diagram of the three-dimensional dynamic label layout method based on force under binocular vision provided by the present invention;

[0088] Figure 6 Schematic diagram of the force calculation for the intersection of guiding lines provided by the present invention;

[0089] Figure 7 Schematic diagram of the structure of the three-dimensional dynamic label layout generation device based on force under binocular vision provided by the present invention;

[0090] Figure 8 Schematic diagram of the physical structure of an electronic device provided by the present invention. Detailed implementation manners

[0091] To more clearly elaborate the purpose, technical solution and advantages of the present invention, the technical solution will be described in detail and completely below in conjunction with the drawings of the present invention. Obviously, the following described embodiments are only a part of the present invention and do not cover all possible embodiments. Any other embodiments that can be derived by those of ordinary skill in the art on the basis of the embodiments of the present invention without creative labor shall fall within the protection scope of the present invention.

[0092] Since most of the existing three-dimensional label layout methods are designed based on static or fixed perspectives, there is a problem that the layout cannot be adaptively optimized as the user's position and perspective change. The following will be combined with Figures 1 - 8 Describe a three-dimensional dynamic label layout method based on force under binocular vision of the present invention. Figure 1 Schematic flowchart of the three-dimensional dynamic label layout method based on force under binocular vision provided by the present invention, as Figure 1 shown, the method includes:

[0093] Step 110, obtain a three-dimensional scene model; the three-dimensional scene model includes at least one object; in the three-dimensional scene model, determine the anchor coordinates of each anchor point, and determine the label content led out by each of the anchor points through a guiding line;

[0094] Specifically, the three-dimensional dynamic label layout method based on binocular vision by force provided by the present invention is implemented based on a constructed algorithm. When using the algorithm, it is necessary to input a three-dimensional scene model, including objects, label content, and their corresponding anchor coordinates. In the three-dimensional scene model, there can be one or more objects, including objects that need to be labeled with labels and various other objects; as a usage mode, the object can be a single three-dimensional model. For example, the object can be a "rubber duck" three-dimensional model. In the usage scenario, different parts of the "rubber duck" three-dimensional model are labeled with labels. As another usage mode, the object is multiple three-dimensional models, used to label the names of each three-dimensional model. In the present invention, the anchor coordinates of each anchor point in the three-dimensional model are preset in advance, and each anchor point leads out a guiding line, and the other end of the guiding line is the labeled label. The guiding line is used to connect the paired anchor points and labels. Therefore, when the label position is optimized and adjusted, the guiding line changes adaptively. Figure 2 This is an example diagram of the three-dimensional model provided by the present invention. As Figure 2 described, the single "rubber duck" three-dimensional model, the single "car" three-dimensional model, and the single "ornament" three-dimensional model are shown in sequence. Figure 3 This shows the schematic diagram of the related concepts of the anchor points, guiding lines, and labels involved in the present invention. Figure 4 This shows a usage scenario diagram of a three-dimensional scene model involving multiple objects and labeling each object with a label.

[0095] Step 120, input the three-dimensional scene model, each label content, and their corresponding anchor coordinates, and use a label layout algorithm to optimize the label layout and output the optimized label layout;

[0096] Among them, the label layout algorithm includes a label layout initialization algorithm and a label layout dynamic update algorithm; the label layout initialization algorithm is used to optimize the initial positions of each label; the label layout dynamic update algorithm is used to update the label layout under the binocular vision of the user and adaptively generate the updated label layout.

[0097] Specifically, input the obtained three-dimensional scene model, as well as the objects, label content, and their corresponding anchor coordinates in it, into the constructed label layout algorithm, and output the label layout result in real time; among them, the label layout algorithm includes a label layout initialization process and a label layout dynamic update process, used to adaptively generate a clear and beautiful layout according to the position and perspective of the user.

[0098] The specific process of the label layout initialization is as follows: The simulated annealing method is used for global optimization search. Through multiple iterations, candidate positions for each label are selected, and the overall label layout is optimized by accepting or rejecting new configurations. During the label layout initialization process, a predefined objective function is utilized to evaluate the quality of the label layout. The construction of the objective function is based on multiple criteria, including avoiding the overlap of labels and labels, avoiding the overlap of labels and objects in the scene, avoiding the crossing of guiding lines in the scene, minimizing the distance between the label and the corresponding anchor point, etc. The label layout initialization process uses the simulated annealing method to find a balance among these criteria, ensuring an initial layout that not only meets specific quality criteria but also lays a good foundation for subsequent label layout updates. The label layout initialization process enables the labels to be more reasonably distributed in the three-dimensional space.

[0099] The process of dynamically updating the label layout is as follows: In each frame, forces are applied to all the labels in the interface, namely: the repulsive force between labels, the attractive force between the label and the anchor point, the repulsive force of label-label overlap, the repulsive force of label-object overlap, and the force to solve the crossing of guiding lines. The labels smoothly move and adjust their positions under the action of the resultant force. The process of dynamically updating the label layout ensures the readability of the labels while maintaining the aesthetic consistency of the overall layout.

[0100] Therefore, the method provided by the embodiments of the present invention can generate a three-dimensional dynamic label layout in real time, improving the clarity, aesthetics, and readability of the label layout.

[0101] Specifically, as Figure 5 shown, the process of using the three-dimensional dynamic label layout method under the binocular vision based on force includes the following four steps a, b, c, and d:

[0102] Step a, the user inputs a three-dimensional scene model, the content of each label, and its corresponding anchor point coordinates;

[0103] Step b, after inputting the three-dimensional scene model, the content of each label, and its corresponding anchor point coordinates, a simulated annealing algorithm is used to perform a global optimization search task to obtain the initialized label layout result;

[0104] Step c, based on the initialized label layout result in step b, a physical force field model is used to dynamically update the label layout in each frame;

[0105] Step d, the optimized label layout result in the three-dimensional scene model is output in real time.

[0106] In this method, the label layout initialization process uses the simulated annealing algorithm to perform a global optimization search task. This process involves multiple iterations and aims to select a suitable candidate position for each label. By judging and accepting or rejecting new configuration schemes, the overall layout effect is continuously optimized. When initializing the label layout, a pre-set objective function is used to evaluate the quality of the current layout. The construction of the objective function is based on multiple criteria, including avoiding label overlap, avoiding overlap between labels and objects in the scene, avoiding the intersection of guiding lines in the scene, and minimizing the distance between the label and the corresponding anchor point, etc.

[0107] Specifically, the label layout initialization process uses the simulated annealing algorithm to perform a global optimization search for the initial positions of the labels and execute the global optimization search task. This process involves multiple iterations and aims to select a suitable candidate position for each label to achieve the best user experience. In each iteration, the algorithm first generates a new label candidate layout based on the current label layout. This process is achieved by randomly selecting new positions within a pre-set search area around the label. The new candidate position is a slight adjustment of the current label position. To evaluate the effectiveness of the new label layout, the algorithm uses a pre-set objective function by comparing the quality of the current label layout and the candidate label layout. By judging and accepting or rejecting new configuration schemes, the simulated annealing algorithm realizes the continuous optimization of the overall label layout effect. Specifically, if the new candidate label layout performs better in the objective function evaluation, the new label layout is accepted. Otherwise, the algorithm will accept the worse label layout scheme with a certain probability, which is determined by the current temperature and the energy difference, allowing the algorithm to explore more in the search space and thus avoid falling into a local optimal solution. As the number of iterations increases, the system gradually reduces the temperature, making the probability of accepting a bad label layout decrease, and finally making the label layout stable near a better solution. This process can not only effectively avoid label overlap and interference between labels and objects, but also improve the visual coherence between the label and its anchor point. In this way, the simulated annealing algorithm can gradually optimize the overall layout effect of the labels, laying a foundation for the subsequent dynamic update of the labels.

[0108] Therefore, the simulated annealing algorithm is used to perform a global optimization search for the initial positions of the labels. Through multiple iterations, label candidate positions are selected for each label, thereby generating a new label candidate layout based on the current label layout, and evaluating the quality of the label candidate layout through a predefined objective function, and then judging whether to accept or reject the label candidate positions of each label, so as to optimize the overall label layout and obtain an initial label layout that meets specific quality criteria.

[0109] In the present invention, when initializing the label layout, a pre-set objective function is used to evaluate the quality of the current label layout. The objective function can be expressed as:

[0110] E static = E overlap + E distance + E intersection (1)

[0111] Wherein:

[0112] E static represents the objective function;

[0113] E overlap represents the sum of the overlapping areas between labels and the overlapping areas between labels and objects in the label candidate layout;

[0114] E distance represents the sum of the distances between each pair of labels and their corresponding anchor points in the label candidate layout, that is, the sum of the lengths of all guiding lines;

[0115] E intersection represents the number of intersecting guiding lines in the label candidate layout;

[0116] In the said objective function, E overlap , E distance and E intersection are determined by formula (2), formula (3) and formula (4) respectively:

[0117]

[0118] Wherein:

[0119] represents the sum of the overlapping areas between labels in the label candidate layout;

[0120] represents the sum of the overlapping areas between labels and objects in the label candidate layout;

[0121] N represents the total number of labels; M represents the total number of objects;

[0122] i and j respectively represent label i and label j , i = 1, 2,..., N, j = 1, 2,..., N, j ≠ i; k represents object k , k = 1, 2,..., M;

[0123] S i，j represents the overlapping area between label i and label j ; S i，k represents the overlapping area between label i and object kThe overlapping area;

[0124] anchor i represents the label i The uniquely corresponding anchor point;

[0125] D(i, anchor i ) represents the label i to the anchor point anchor i distance;

[0126] e i and e j respectively represent the label i and the label j corresponding guiding lines;

[0127] I(e i , e j ) represents the guiding line e i and the guiding line e j whether they intersect function; if the guiding line e i and the guiding line e j intersect, the function value is 1; otherwise, the function value is 0.

[0128] ω 1 , ω 2 , ω 3 , ω 4 , respectively represent the weight coefficient of the overlapping area between labels, the weight coefficient of the overlapping area between the label and the object, the coefficient of the distance between the label and the corresponding anchor point, and the guiding line intersection coefficient.

[0129] In this method, the core of the dynamic update process of the label layout lies in adapting to the user's perspective change in real time. The system first captures the user's head movement data through sensors or head-mounted devices to ensure that it can accurately reflect the user's current perspective. This process provides the basis for the dynamic label layout, enabling the labels to be flexibly adjusted as the user's perspective changes.

[0130] The dynamic update algorithm of the label layout is as follows:

[0131] When the user's perspective changes, based on the user's latest perspective and the binocular vision principle, the label layout is processed separately for the left-eye perspective and the right-eye perspective, and the binocular vision label layout is jointly optimized. When jointly optimizing the binocular vision label layout, the resultant force received by each label is recalculated using a physical force field model; according to the resultant force received by each label, the label is moved a corresponding distance along the direction of the resultant force, thereby updating the position of the label, and thus obtaining the updated label layout. Specifically, when the user's perspective changes, the system will recalculate the resultant force received by each label using a physical force field model. This model comprehensively considers the influence of multiple forces, including the repulsive force between labels, the attractive force between a label and an anchor point, the repulsive force caused by label-label overlap, the repulsive force caused by label-object overlap, and the force of guide line intersection.

[0132] Based on the binocular vision principle, when processing the label layout separately for the left-eye perspective and the right-eye perspective, specifically: two virtual cameras are set up to simulate the perspectives of the left eye and the right eye respectively, and the three-dimensional scene model is projected onto the screen spaces of the left-eye virtual camera and the right-eye virtual camera respectively, and then joint optimization is performed to solve the problem of label overlap in the current perspective.

[0133] When using a physical force field model to recalculate the resultant force received by each label, specifically: during the label layout process, a method based on the physical force field model is introduced to optimize the label layout frame by frame, applying the repulsive force between labels, the attractive force between a label and an anchor point, the repulsive force caused by label-label overlap, the repulsive force caused by label-object overlap, and the force of guide line intersection, to improve the readability, aesthetics, and smoothness of the labels.

[0134] The resultant force received by each label is the resultant force of the repulsive force between labels, the attractive force between a label and an anchor point, the repulsive force caused by label-label overlap, the repulsive force caused by label-object overlap, and the force of guide line intersection;

[0135] a) The repulsive force between labels is used to prevent labels from being too concentrated;

[0136] b) The attractive force between a label and an anchor point is used to ensure the relevance between the label and its annotated object;

[0137] c) The repulsive force caused by label-label overlap is used to reduce the visual overlap of labels;

[0138] d) The repulsive force caused by label-object overlap is used to avoid the overlap of labels with objects in the scene;

[0139] e) The force of guide line intersection is the force to solve the intersection of guide lines and is used to eliminate the intersection phenomenon of label guide lines.

[0140] Therefore, the repulsive force between labels ensures an appropriate distance between each label, avoiding visual congestion; the attractive force between a label and an anchor point ensures that the label is always closely connected to its corresponding object, avoiding deviation from the target. The introduction of the overlapping force and the crossing force helps reduce the visual chaos generated by labels in a dynamic environment, ensuring that users can clearly identify each label.

[0141] Therefore, when dynamically updating the label layout for the three-dimensional dynamic label layout method based on force under binocular disparity provided by the present invention, the following steps are adopted: a) Calculate the resultant force received by the label in each frame, and smoothly adjust the label position according to the calculation result; b) Dynamically update the label layout according to the user's position and perspective in each frame to ensure the clarity, readability, and aesthetics of the label layout.

[0142] Among them:

[0143] The repulsive force between labels refers to the repulsive force received by each label from all other labels. The magnitude of this repulsive force is negatively correlated with the distance between labels, and is used to avoid overlap and excessive proximity between labels, ensuring a certain distance between labels;

[0144] The attractive force between a label and an anchor point refers to the attractive force received by each label from its corresponding anchor point. The magnitude of the attractive force is positively correlated with the distance between the label and the anchor point, ensuring that the label does not move too far away from its corresponding anchor point;

[0145] The label-label overlap repulsive force refers to the repulsive force received by a label from the corresponding overlapping label when the labels overlap visually, so as to avoid visual overlap between labels and improve the clarity and readability of the label layout;

[0146] The label-object overlap repulsive force refers to the repulsive force received by a label from the corresponding overlapping object when the label and the object overlap, so as to avoid visual overlap between the label and the object and improve the clarity and readability of the label layout;

[0147] The force of guide line crossing refers to applying a force to the label associated with the intersecting guide lines when two guide lines intersect, so as to adjust the position of the label and eliminate the crossing of the guide lines.

[0148] For the label label i , the resultant force F i is calculated by formula (5):

[0149]

[0150] Among them:

[0151] ① represents the label label iSubject to the inter-label repulsive force from the other N - 1 labels; calculated by formula (6):

[0152]

[0153] Where: N represents the total number of labels; i and j respectively represent label i and label j , i = 1, 2,..., N, j = 1, 2,..., N, j ≠ i; l i and l j respectively represent the position coordinates of label i and label j ; ||l i - l j || represents the distance between label i and label j ; represents the unit vector pointing from label j to label i ; k 1 represents the coefficient of the inter-label repulsive force;

[0154] ② represents the attractive force on label i from the corresponding anchor i ; calculated by formula (7):

[0155]

[0156] Where: a i represents the position coordinate of anchor i ; ||l i - a i || represents the distance between label i and anchor i ; represents the unit vector pointing from label i to anchor i ; k 2 represents the coefficient of the attractive force of the label on the anchor;

[0157] ③ represents the label-label overlap repulsive force on label i from the overlapping label j ; calculated by formula (8):

[0158]

[0159] Wherein: represents the unit vector pointing from label j to label i ; k 3 represents the coefficient of the label-label overlapping repulsive force; S i，j represents the overlapping area between label i and label l j, which is calculated by formula (9):

[0160]

[0161] and respectively represent the overlapping areas of label i and label j under the left-eye view and the right-eye view;

[0162] ④ represents the label-object overlapping repulsive force that label i receives from the object k with which it overlaps; that is: when label i visually overlaps with any object k in the scene, a pair of repulsive forces is generated between label i and object k ; It is calculated by formula (10):

[0163]

[0164] Wherein: represents the unit vector pointing from object k to label i ; k 4 represents the coefficient of the label-object overlapping repulsive force; S i,k represents the overlapping area between label i and object k which is calculated by formula (11):

[0165]

[0166] and respectively represent the overlapping areas of label i and object k under the left-eye view and the right-eye view;

[0167] ⑤ The guiding line e of the representative label i and the label i When the guiding lines e j cross, the label j is subjected to the force of the crossing of the guiding lines. i Specifically, when the user moves or changes the viewing point, the guiding lines will intersect. Frequent intersections will cause the layout to be chaotic and increase the user's cognitive load. To solve this problem, a new force is introduced, namely: the force to solve the crossing of the guiding lines

[0168] It is calculated by formula (12): where: k

[0169]

[0170] represents the coefficient of the force of the crossing of the guiding lines; 5 D

[0171] represents the degree of intersection of the guiding line e i,j and the guiding line e i The calculation method is: j As shown in

[0172] For example Figure 6 is the schematic diagram of the calculation principle of the force of the crossing of the guiding lines; project the guiding line e i and the guiding line e j onto the screen plane of the main camera; in the screen plane, the label end of the guiding line e i is projected as the lp i point, and the anchor end of the guiding line e i is projected as the ap i point; the label end of the guiding line e j is projected as the lp j point, and the anchor end of the guiding line e j is projected as the ap j point; the guiding line e i and the guiding line e j intersect at the intersection point u in the screen plane; the triangle area formed by the intersection point u, the lp i point, and the lp j point is area 1 ; the triangle area formed by the intersection point u, the ap i point, and the ap j point is area 2 ; therefore, through formula (13), the degree of intersection D i of the guiding line e j and the guiding line e i，j is obtained:

[0173]

[0174] where: mmin(area 1 , area 2 ) represents the smaller value of area 1 and area 2 ;

[0175] represents the unit vector of the force of the guide line crossing received by label i , determined by formula (14):

[0176]

[0177] where: represents the normal vector of the plane determined by anchor i , anchor j and label i ; the normal vector of the plane determined by anchor i , anchor j and label j is θ 1 indicating the normal vector and the normal vector ; θ 2 indicating the normal vector and the normal vector .

[0178] Therefore, when the user's perspective changes, the system will recalculate the resultant force received by each label based on the physical force field model. This model comprehensively considers the influence of various forces, including the repulsive force between labels, the attractive force between a label and an anchor, the repulsive force of label-label overlap, the repulsive force of label-object overlap, and the force of guide line crossing. The repulsive force between labels ensures an appropriate distance between each label, avoiding visual congestion; the attractive force between a label and an anchor ensures that the label is always closely connected to its corresponding labeled object, avoiding deviation from the labeled object. The introduction of the overlap force and the crossing force helps to reduce the visual chaos generated by labels in a dynamic environment, ensuring that users can clearly identify each label.

[0179] By calculating the resultant force acting on the label, the new position of each label is determined. The labels maintain a natural and smooth movement based on the action of the force. To ensure that the user can quickly read the labels while moving, the direction of the labels is adjusted in each frame so that the labels always face the user. The dynamic update process of the labels is an iterative loop, and the system continuously adjusts according to the user's position and perspective to optimize the label layout effect. After the user changes the perspective, the system will quickly respond and update the label positions to ensure that the labels are always clearly visible in the new perspective. This flexibility and adaptability are the keys to improving the user interaction experience in the virtual reality environment.

[0180] Figure 7 The structural schematic diagram of the three-dimensional dynamic label layout system based on binocular vision provided by the present invention is shown in Figure 7 As shown, it includes an acquisition module 710 and a label layout algorithm module 720. Among them, the acquisition module 710 is used to acquire a three-dimensional scene model; at least one object is included in the three-dimensional scene model; in the three-dimensional scene model, the anchor point coordinates of each anchor point are determined, and the label content led out by each anchor point through a guiding line is determined; the label layout algorithm module 720 is used to input the three-dimensional scene model, each label content and its corresponding anchor point coordinates, optimize the label layout, and output the optimized label layout; the label layout algorithm module 720 includes: a label layout initialization sub-module 7201, which is used to optimize the initial positions of each label and generate an initial label layout; a label layout dynamic update sub-module 7202, which is used to update the label layout under the binocular vision of the user and adaptively generate the updated label layout.

[0181] Figure 8 The entity structure schematic diagram of an electronic device provided by the present invention is shown in Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communication interface 820, and the memory 830 complete mutual communication through the communication bus 840. The processor 810 can call the logical instructions in the memory 830 to execute the label layout algorithm based on the input of the three-dimensional scene model, each label content and its corresponding anchor point coordinates. The method includes: inputting the three-dimensional scene model, each label content and its corresponding anchor point coordinates into the label layout algorithm, and outputting the optimized label layout; among them, the label layout algorithm includes a label layout initialization algorithm and a label layout dynamic update algorithm, which are used to adaptively generate a clear and beautiful layout according to the user's position and perspective.

[0182] In addition, when the logic instructions in the above-mentioned memory 830 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0183] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the label layout algorithm provided by the above-mentioned various methods based on the three-dimensional scene model, label content, and their corresponding anchor coordinates. The method includes: inputting the three-dimensional scene model, each label content, and their corresponding anchor coordinates into the label layout algorithm, and outputting an optimized label layout; wherein, the label layout algorithm includes a label layout initialization algorithm and a label layout dynamic update algorithm for adaptively generating a clear and beautiful layout according to the position and perspective of the user.

[0184] On another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the label layout algorithm provided by the above-mentioned various methods based on the three-dimensional scene model, each label content, and their corresponding anchor coordinates. The algorithm includes: obtaining the three-dimensional scene model, each label content, and their corresponding anchor coordinates; inputting the three-dimensional scene model, each label content, and their corresponding anchor coordinates into the label layout algorithm, and outputting an optimized label layout; wherein, the label layout algorithm includes a label layout initialization algorithm and a label layout dynamic update algorithm for adaptively generating a clear and beautiful layout according to the position and perspective of the user.

[0185] The server embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0186] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0187] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A three-dimensional dynamic label layout method based on force in binocular perspective, characterized in that: include: Acquire a three-dimensional scene model; the three-dimensional scene model includes at least one object; In the three-dimensional scene model, determining the anchor point coordinates of each anchor point, and determining the label content led out by each anchor point through the guide line; Input the three-dimensional scene model, each label content and its corresponding anchor point coordinates, use a label layout algorithm to optimize the label layout, and output the optimized label layout; Among them, the label layout algorithm includes a label layout initialization algorithm and a label layout dynamic update algorithm; the label layout initialization algorithm is used to optimize the initial position of each label; the label layout dynamic update algorithm is used to update the label layout under the user's binocular perspective and adaptively generate an updated label layout.

2. According to the force-based binocular perspective three-dimensional dynamic label layout method of claim 1, characterized in that: The label layout initialization algorithm is specifically as follows: A simulated annealing algorithm is used to perform a global optimization search for the initial position of the label. A label candidate position is selected for each label through multiple iterations, so that a new label candidate layout is generated based on the current label layout. The quality of the label candidate layout is evaluated through a predefined objective function, and then the label candidate position of each label is accepted or rejected, thereby optimizing the overall label layout and obtaining an initial label layout that meets specific quality standards.

3. The three-dimensional dynamic label layout method under binocular perspective based on force according to claim 2 is characterized in that: The objective function is: AND static =And overlap +E distance +E intersection (1) in: E static represents the objective function; E overlap Represents the sum of the overlapping areas between labels and between labels and objects in the label candidate layout; E distence Indicates the sum of the distances between each pair of labels and the corresponding anchor points in the label candidate layout, which is the sum of the lengths of all guide lines; E intarsection Indicates the number of intersecting guide lines in the label candidate layout; In the objective function, E overlap 、E distance and E intersection , respectively determined by formula (2), formula (3) and formula (4): in: Represents the sum of the overlapping areas between labels in the label candidate layout; Represents the sum of the overlapping areas between labels and objects in the label candidate layout; N represents the total number of labels; M represents the total number of objects; i and j represent labels respectively i and label j , i=1,2,...,N,j=1,2,...,N,j≠i; k represents object k , k=1,2,...,M; S i,j Represents label label i and label j The overlapping area of ​​​​S i,k Represents label label i and object k The overlapping area of anchor i Represents label label i The only corresponding anchor point; D(i,anchor i ) represents the label label i To the anchor point i distance; e i and e j Represents label label i and label j Corresponding guide lines; I(e i ,e j ) represents the guide line e i and guide line e j Whether the function intersects; if the guide line e i and guide line e j If they intersect, the function value is 1; otherwise, the function value is 0. ω1, ω2, ω3, and ω4 represent the weight coefficients of the overlapping area between labels, the overlapping area between labels and objects, the distance between labels and corresponding anchor points, and the intersection weight coefficient of guide lines, respectively.

4. The three-dimensional dynamic label layout method under binocular perspective based on force according to claim 1 is characterized in that: The tag layout dynamic update algorithm is: When the user's perspective changes, the label layout is processed for the left eye and right eye perspectives respectively according to the user's latest perspective and based on the binocular perspective principle, and the binocular perspective label layout is jointly optimized. When the binocular perspective label layout is jointly optimized, the physical force field model is used to recalculate the resultant force on each label; according to the resultant force on each label, the label is moved a corresponding distance along the direction of the resultant force, and then the label position is updated to obtain an updated label layout.

5. The three-dimensional dynamic label layout method under binocular perspective based on force according to claim 4 is characterized in that: Based on the binocular perspective principle, the label layout is processed separately for the left eye perspective and the right eye perspective. Specifically, two virtual cameras are set to simulate the perspectives of the left eye and the right eye respectively, and the three-dimensional scene model is projected onto the screen space of the left eye virtual camera and the right eye virtual camera respectively, and then joint optimization is performed to solve the overlapping problem of labels under the current perspective.

6. The three-dimensional dynamic label layout method under binocular perspective based on force according to claim 4 is characterized in that: The resultant force on each label is recalculated based on the physical force field model, specifically: The net force on each tag is the net force of the repulsive force between tags, the attractive force between tags and anchor points, the tag-tag overlap repulsive force, the tag-object overlap repulsive force, and the force of the intersection of guide lines; in: The repulsive force between tags refers to the repulsive force that each tag receives from all other tags. The magnitude of the repulsive force is negatively correlated with the distance between tags, and is used to avoid overlapping and excessive proximity between tags, ensuring that a certain distance is maintained between tags. The attraction between the label and the anchor point refers to the attraction that each label receives from its corresponding anchor point. The magnitude of the attraction is positively correlated with the distance between the label and the anchor point, ensuring that the label is not too far away from its corresponding anchor point. The label-label overlap repulsion force refers to the repulsion force exerted on a label by the corresponding overlapping label when the labels overlap, so as to avoid visual overlap between the labels and improve the clarity and readability of the label layout; The label-object overlap repulsion force refers to the repulsion force that a label receives from a corresponding overlapping object when the label overlaps with the object, so as to avoid visual overlap between the label and the object and improve the clarity and readability of the label layout; The guide line crossing force refers to a force applied to tags associated with the intersecting guide lines when two guide lines intersect, so as to adjust the position of the tags and eliminate the crossing of the guide lines.

7. The three-dimensional dynamic label layout method under binocular perspective based on force according to claim 6 is characterized in that: For label label i , the resultant force F i Calculated by formula (5): in: ① Represents label label i The repulsive force between tags from other N-1 tags is calculated by formula (6): Where: N represents the total number of labels; i and j represent the labels i and label j ,i=1,2,...,N,j=1,2,...,N,j≠i;l i and l j Represents label label i and label j The position coordinates of ||l i -l j || represents label label i and label j distance; Represents from label label j Point to label label i The unit vector of ; k1 represents the coefficient of repulsive force between tags; ② Represents label label i Received from the corresponding anchor i The attractiveness of is calculated by formula (7): Among them: a i Represents the anchor point i The position coordinates of ||l i -a i || represents label label i and anchor i distance; Represents from label label i Point to the anchor i The unit vector of ; k2 represents the coefficient of attraction of the label to the anchor point; ③ Represents label label i Influenced by the label that overlaps with it j The label-label overlap repulsion is calculated by formula (8): in: Represents from label label j Point to label label i The unit vector of the label-label overlap repulsion force; k3 represents the coefficient of the label-label overlap repulsion force; S i,j Represents label label i and label j The overlapping area is calculated by formula (9): and Respectively represent the labels under the left eye perspective and the right eye perspective i and label j The overlapping area of ④ Represents label label i Affected by the object that overlaps with it k The label-object overlap repulsion is calculated by formula (10): in: Represents the object k Point to label label i The unit vector of the object; k4 represents the coefficient of the label-object overlap repulsion; S i,k Represents label label i and object k The overlapping area is calculated by formula (11): and Respectively represent the labels under the left eye perspective and the right eye perspective i and object k The overlapping area of ⑤ Represents label label i Leading line i and label j Leading line j When crossing, label i The force of the guide wire crossing is calculated by formula (12): Where: k5 represents the coefficient of the force of the guide wire crossing; D i,j Represents the leading line e i and guide line e j The degree of intersection is calculated as: The guide line i and guide line e j Projected onto the screen plane of the main camera; in the screen plane, the guide line e i The label end projection is lp i Point, guide line i The anchor point projection is ap i Point; guide line j The label end projection is lp j Point, guide line j The anchor point projection is ap j Point; guide line i and guide line e j intersect at the intersection point u in the screen plane; intersection point u, lp i Point, lp j The area of ​​the triangle formed by the points is area1; the intersection points u and ap i Point, ap j The area of ​​the triangle formed by the points is area2; therefore, by formula (13), we get the guide line e i and guide line e j The degree of intersection D i,j : Among them: min(area1,area2) represents the smaller value of area1 and area2; Represents label label i The unit vector of the force acting on the guide wire intersection is determined by formula (14): in: Represents the anchor point i , anchor j and label i The normal vector of the determined plane; anchor point i , anchor j and label j The normal vector of the determined plane is θ1 represents the normal vector and the normal vector The angle between them, θ2 represents the normal vector and the normal vector - The angle between.

8. The three-dimensional dynamic label layout method under binocular perspective based on force according to claim 1 is characterized in that: The label layout dynamic update algorithm also includes: When updating the label layout under the user's binocular perspective, the orientation of the label is adjusted in real time according to the user's movement and perspective changes, so that the label always faces the user, thereby improving the user's reading experience.

9. A system for implementing the force-based three-dimensional dynamic label layout method under binocular perspective as described in any one of claims 1 to 8, characterized in that: include: An acquisition module, used to acquire a three-dimensional scene model; the three-dimensional scene model includes at least one object; In the three-dimensional scene model, determining the anchor point coordinates of each anchor point, and determining the label content led out by each anchor point through the guide line; A label layout algorithm module is used to input the three-dimensional scene model, each label content and its corresponding anchor point coordinates, optimize the label layout, and output the optimized label layout; The label layout algorithm module includes: The label layout initialization submodule is used to optimize the initial position of each label and generate the initial label layout; The label layout dynamic update submodule is used to update the label layout under the user's binocular perspective and adaptively generate the updated label layout.

10. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, a three-dimensional dynamic label layout method under a binocular perspective based on force as described in any one of claims 1 to 8 is implemented.

Images