Object labeling information presentation method and device, electronic equipment and storage medium

Abstract

The application relates to an object annotation information presentation method and device, electronic equipment and a storage medium. The method relates to the field of computer vision. The method comprises the following steps: acquiring a target object in a specified scene, the specified scene being a scene presented at a target position; and presenting annotation information of the target object on an annotation facade of the target object presented on a display interface, the annotation facade being determined from at least two visible facades of the target object according to projection areas of the at least two visible facades presented on the display interface, the visible facade being a facade visible to the target position in an outer facade of the target object. The above method can select an annotation facade according to the projection of a visible facade on a display interface of a specified scene when displaying annotation of a target object in an AR or VR scene, so that a facade with a larger visible area can be dynamically selected to display annotation information in a virtual reality / augmented reality scene, and the display effect of the annotation information is improved.

Description

Technical Field

[0001] This application relates to the field of computer vision, and in particular to a method, apparatus, electronic device and storage medium for presenting object annotation information. Background Technology

[0002] With the continuous development of smart terminals, users can use AR maps to obtain information about surrounding buildings based on their current location, allowing them to choose a direction.

[0003] In related technologies, developers typically pre-set information labels corresponding to buildings on the models of buildings in AR maps. When the building or POI information in the user's current scene is identified, the static information labels on the building model are displayed on the AR map on the user's terminal based on the location of the building or POI information. The labels can be three-dimensional, and the user can clearly see the label content corresponding to the building from a certain perspective.

[0004] However, in related technologies, statically annotated 3D building labels are static and cannot dynamically adjust the position and orientation of the labels from different perspectives. As the user's orientation changes, the display effect of the labels is poor. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for presenting object annotation information. It allows for the selection of annotation facades based on the projection of visible facades onto a display interface of a specified scene, thereby improving the display effect of annotation information. The technical solution is as follows:

[0006] On the one hand, a method for presenting object annotation information is provided, the method comprising:

[0007] Obtain the target object in a specified scene, wherein the specified scene is a scene presented at the target location;

[0008] The labeling information of the target object is presented on the labeled elevation of the target object in the display interface; the labeled elevation is determined from the at least two visible elevations based on the projection areas of each of the at least two visible elevations of the target object presented on the display interface, and the visible elevation is the elevation of the target object's exterior that is visible to the target position.

[0009] In the solution provided in this application embodiment, the visible facade of the target object is the facade visible from the exterior of the target object when it is presented at the target location; that is, the facade that is not completely obscured when the target object is presented in the specified scene corresponding to the target location. The projection area refers to the area on the display interface where the visible facade of the target object is displayed. The target object can be presented in the specified scene corresponding to the target location, and the projection area of ​​the visible facade of the target object onto the display interface can be obtained. Then, based on the projection area, one of the visible facades is determined as the annotation facade, and the annotation information of the target object is presented on the annotation facade. In other words, based on the target location, one of the visible facades corresponding to the target object can be selected as the annotation facade to present the annotation information of the target object. When presenting the annotation information, the orientational relationship between the target location and the facade of the target object is considered, improving the display effect of the annotation information.

[0010] In one possible implementation, the designated scene is an augmented reality scene or a virtual reality scene presented at the target location.

[0011] In the solution provided in this application embodiment, in an augmented reality scene, the target location can be the location of the augmented reality device, and the scene presented in the augmented reality scene is the scene obtained by the augmented reality device at its current location through the image acquisition component corresponding to the augmented reality device; while in a virtual reality scene, the target location can be the location of the virtual character corresponding to the virtual reality device in a three-dimensional virtual scene calculated and modeled by the background of the virtual reality device, and the scene presented in the virtual reality scene is the three-dimensional virtual scene presented by the virtual reality device from the perspective and position corresponding to the virtual character.

[0012] In one possible implementation, the marked facade is the one with the largest projected area on the display interface among the at least two visible facades.

[0013] By using the visible facade of the target object with the largest projected area on the display interface as the label facade to display the labeling information, the labeling information corresponding to the label facade can be displayed at the largest size on the display interface, thus improving the display effect of the labeling information.

[0014] In one possible implementation, the method further includes:

[0015] Based on the visible areas of the at least two visible facades, obtain the projection areas of the at least two visible facades; the visible areas are the areas of the corresponding visible facades that are visible to the target location in the specified scene.

[0016] The projection areas of the at least two visible facades are projected onto the display interface to obtain the projection areas of each of the at least two visible facades on the display interface.

[0017] In the solution provided in this application embodiment, the visible area refers to the visible portion of the visible elevation of the target object, corresponding to the projection area on the display interface. In other words, the visible area on the visible elevation is the area on the visible elevation of the target object in the 3D scene presented in the virtual reality scene, or the area on the visible elevation of the target object in the 3D scene calculated by the computer in the augmented reality scene. Both the visible area and the area to be projected are areas on the visible elevation. The area to be projected can be the entire or a portion of the visible area of ​​the visible elevation. That is, the visible area corresponding to the visible elevation can be entirely projected onto the display interface, or only partially. In this case, the projection area on the display interface can be presented on the projection screen according to an arbitrary shape of the area to be projected. By selecting a label elevation based on the projection area, a suitable label elevation can be selected based on the label information of any shape, enhancing the display effect of the label elevation.

[0018] In one possible implementation, obtaining the projection area of ​​the at least two visible facades based on their visible areas includes:

[0019] The entire visible area of ​​each of the at least two visible facades is obtained as the projection area of ​​the at least two visible facades.

[0020] In one possible implementation, presenting the annotation information of the target object on the annotation elevation of the target object displayed on the display interface includes:

[0021] The marking area is determined from the visible area of ​​the marking facade; the marking area is the largest area among the regions of a first shape included in the visible area of ​​the marking facade.

[0022] The annotation information of the target object is displayed in the annotation area of ​​the annotation facade presented on the display interface.

[0023] In the solution provided in the embodiments of this application, the marking area is the area with the largest area of ​​the first shape included in the visible area of ​​the marking facade, that is, the marking area is all or part of the visible area corresponding to the marking facade on the marking facade.

[0024] In one possible implementation, determining the annotation area from the visible area of ​​the annotation facade includes:

[0025] Obtain the occlusion information of the marked facade, wherein the occlusion information is used to indicate the occluded vertices and occluded edges of the marked facade;

[0026] Based on the occlusion information, the marking area is determined within the visible area of ​​the marking facade.

[0027] In the solution provided in this application embodiment, the occlusion information is the information corresponding to the unpresented part of the marked facade projected onto the display interface, that is, the information corresponding to the area outside the visible area of ​​the marked facade; wherein, the occluded vertex is the vertex of the marked facade located outside the visible area; the occluded edge is the edge of the marked facade whose entire position is located outside the visible area.

[0028] In one possible implementation, the first shape is rectangular, and determining the annotation area within the visible area of ​​the annotation facade based on the occlusion information includes:

[0029] When the occlusion information indicates that there is an occluded vertex on the marked elevation, the diagonal vertex of the occluded vertex is taken as the first target point;

[0030] A first endpoint is determined on the non-adjacent edge corresponding to the first target point, such that the rectangle with the line segment between the first endpoint and the first target point as its diagonal is the rectangle with the largest area within the visible area of ​​the marked elevation.

[0031] The area containing the rectangle whose diagonal is the line segment between the first endpoint and the first target point is defined as the labeled area.

[0032] In the solution provided in this application embodiment, when the annotation area presenting the annotation information is a rectangle, and the annotation facade determined according to the projection area is presented on the display interface, there is a vertex that is occluded. At this time, the first endpoint of the rectangle corresponding to the annotation area can be determined according to the non-adjacent edge of the diagonal vertex of the occluded vertex, and the annotation area can be determined according to the first endpoint and the diagonal vertex of the occluded vertex; wherein, the non-adjacent edge is the edge in the shape corresponding to the visible area of ​​the annotation facade corresponding to the projection area that is not directly connected to the first endpoint.

[0033] In one possible implementation, the first shape is rectangular, and determining the annotation area within the visible area of ​​the annotation facade based on the occlusion information includes:

[0034] When the occlusion information indicates that there are two occluded vertices on the labeled elevation, and the edge between the two occluded vertices is completely occluded, the vertex with the largest sum of the lengths of the unoccluded parts of its adjacent edges among the unoccluded vertices on the labeled elevation is obtained as the second target point.

[0035] A second endpoint is determined on the non-adjacent edge corresponding to the second target point; the second endpoint is located in the visible area of ​​the labeled facade, and the rectangle with the line segment between the second endpoint and the second target point as its diagonal is the rectangle with the largest area in the visible area of ​​the labeled facade.

[0036] The area containing the rectangle whose diagonal is the line segment between the second endpoint and the second target point is defined as the labeled area.

[0037] In one possible implementation, the first shape is rectangular, and determining the annotation area within the visible area of ​​the annotation facade based on the occlusion information includes:

[0038] When the occlusion information indicates that there are two occluded vertices on the labeled facade and no completely occluded edges, a target point set is obtained; the target point set includes the unoccluded vertices of the labeled facade and the boundary points on the adjacent edges of the two occluded vertices; the boundary points are used to distinguish between the occluded and unoccluded areas of the visible facade.

[0039] A third endpoint is determined within the visible area of ​​the marked facade; the rectangle with the line segment between the third endpoint and the third target point as its diagonal is the rectangle with the largest area within the visible area of ​​the marked facade; the third target point is one of the target points in the set.

[0040] The area containing the rectangle whose diagonal is the line segment between the third endpoint and the third target point is defined as the labeled area.

[0041] In one possible implementation, the first shape is rectangular, and determining the annotation area within the visible area of ​​the annotation facade based on the occlusion information includes:

[0042] When the occlusion information indicates that there are three occluded vertices on the marked elevation, the unoccluded vertices on the marked elevation are obtained as the fourth target point;

[0043] The area containing the rectangle formed by the fourth target point and the boundary points on its two adjacent sides is defined as the marked area; the boundary points are used to distinguish between the obscured and unobscured areas of the visible facade.

[0044] In one possible implementation, presenting the annotation information of the target object on the annotation elevation of the target object displayed on the display interface includes:

[0045] Based on the dimensions of the labeled area, a three-dimensional model of the labeled information is generated;

[0046] A three-dimensional model of the annotation information is presented on a plane parallel to the annotation area displayed on the display interface; the parallel plane is a plane located in front of the annotation facade and parallel to the annotation facade.

[0047] In the solution provided in this application embodiment, the annotation information can be presented on the display interface as a three-dimensional model structure, and the size of the three-dimensional model of the annotation information can be determined by the size of the annotation area. Since the three-dimensional model has depth information, it is first necessary to obtain a plane parallel to the annotation facade in front of it. The annotation information is located on this parallel plane. Based on the position of the parallel plane and the size of the three-dimensional model of the annotation information, the three-dimensional model of the annotation information is presented on the display interface. At this time, the annotation information presented on the display interface also has three-dimensional characteristics, improving the display effect of the annotation information.

[0048] In one possible implementation, obtaining the projection area of ​​the at least two visible facades based on their visible areas includes:

[0049] Candidate labeling regions are determined from the visible areas of the at least two visible facades; the candidate labeling region is the region with the largest area among the regions of the second shape contained in the visible area of ​​the corresponding facade.

[0050] The candidate annotation areas corresponding to each of the at least two visible facades are obtained as the projection areas of the at least two visible facades.

[0051] In one possible implementation, presenting the annotation information of the target object on the annotation elevation of the target object displayed on the display interface includes:

[0052] The annotation information of the target object is presented on the annotation surface of the display interface, corresponding to the candidate annotation area of ​​the annotation facade.

[0053] In the solution provided in this application embodiment, firstly, in the visible areas of at least two visible facades of the target object, the visible area containing the largest area of ​​the second shape is obtained as the projection area, that is, a part of the visible area is used as the projection area and then projected onto the display interface; the candidate annotation area corresponding to the largest projection area on the display interface is used as the area for presenting annotation information, that is, firstly, the largest area of ​​the specified shape of each visible facade is compared with the area projected onto the display interface, and the candidate annotation area of ​​the visible facade corresponding to the largest projection area is used to present annotation information. Considering the largest area of ​​the specified shape projected onto the display interface of each visible facade, the information annotation of the largest specified shape can be presented on the display interface, which improves the display effect of annotation information.

[0054] In one possible implementation, the annotation information of the target object is presented on the annotation elevation of the target object displayed on the display interface, including:

[0055] When the area of ​​the visible region of the marked facade projected onto the display interface is greater than a specified area threshold, the marking information of the target object is displayed on the marked facade presented on the display interface.

[0056] In another aspect, an object annotation information presentation device is provided, the device comprising:

[0057] The target object acquisition unit is used to acquire a target object in a specified scene, wherein the specified scene is a scene presented at the target location;

[0058] The annotation information presentation unit is used to present the annotation information of the target object on the annotation facade of the target object presented on the display interface. The annotation facade is determined from the at least two visible facades based on the projection areas of the at least two visible facades of the target object presented on the display interface. The visible facade is the facade of the target object that is visible to the target position.

[0059] In another aspect, an electronic device is provided, comprising a processor and a memory, wherein the memory stores computer instructions, which are loaded and executed by the processor to implement the above-described method for presenting object annotation information.

[0060] In another aspect, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the above-described object annotation information presentation method.

[0061] Furthermore, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A terminal's processor reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the terminal to perform the aforementioned object annotation information presentation method.

[0062] The technical solution provided in this application may include the following beneficial effects:

[0063] By acquiring the visible facade of a target object within a virtual reality or augmented reality scene, and based on the projection of this visible facade onto the display interface, a labeled facade is determined within the visible facade of the target object. The labeled information of the target object is then presented in the area corresponding to the visible region of this labeled facade. Through this method, when displaying the label of a target object, the labeled facade can be selected based on the target location and the projection of the visible facade onto the display interface of the specified scene. This allows for the dynamic selection of facades with larger visible areas to display the labeled information within a given scene, thereby improving the display effect of the labeled information.

[0064] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0065] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0066] Figure 1 This is a schematic diagram illustrating the structure of an object annotation information presentation system according to an exemplary embodiment;

[0067] Figure 2 This is a flowchart illustrating a method for presenting object annotation information according to an exemplary embodiment;

[0068] Figure 3 This is a flowchart of a method for presenting object annotation information according to an exemplary embodiment;

[0069] Figure 4 It shows Figure 3 The illustrated embodiment relates to a facade vertex occlusion classification diagram;

[0070] Figure 5 It shows Figure 3 The illustrated embodiment is a schematic diagram of obtaining the visible area of ​​a facade corresponding to a single vertex occlusion;

[0071] Figure 6 It shows Figure 3 The illustrated embodiment is a schematic diagram of a method for calculating the annotation range;

[0072] Figure 7 It shows Figure 3 The illustrated embodiment is a schematic diagram of a method for calculating the annotation range;

[0073] Figure 8 It shows Figure 3 The illustrated embodiment is a schematic diagram of a method for calculating the annotation range;

[0074] Figure 9 It shows Figure 3 The illustrated embodiment relates to a data resource flowchart;

[0075] Figure 10 It shows Figure 3 The illustrated embodiment relates to a structural diagram of an annotation rendering method;

[0076] Figure 11 It shows Figure 3 The illustrated embodiment relates to a flowchart of a building's visible facade calculation.

[0077] Figure 12 It shows Figure 3 The illustrated embodiment includes a flowchart of a method for calculating the visible area.

[0078] Figure 13 It shows Figure 3 The illustrated embodiment includes a flowchart of a calculation method for a text annotation range.

[0079] Figure 14 It shows Figure 3 The illustrated embodiment relates to a flowchart of a real-time camera pose simulation.

[0080] Figure 15 It shows Figure 3 The illustrated embodiment is a comparative diagram with an AR map technology;

[0081] Figure 16 This is a flowchart of a method for presenting object annotation information according to an exemplary embodiment;

[0082] Figure 17 It shows a flowchart illustrating a method for presenting object annotation information;

[0083] Figure 18 This is a structural block diagram illustrating an object annotation information presentation device according to an exemplary embodiment;

[0084] Figure 19 This is a schematic block diagram illustrating an electronic device provided in an exemplary embodiment;

[0085] Figure 20 This is a schematic diagram illustrating the structure of an electronic device provided in an exemplary embodiment. Detailed Implementation

[0086] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0087] Before describing the various embodiments shown in this application, several concepts involved in this application will be introduced first:

[0088] 1) Computer Vision (CV)

[0089] Computer vision is the science of enabling machines to "see." It refers to using cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further processing the images to create visually more suitable for human observation or for transmission to instruments. Vision is an integral part of various intelligent / autonomous systems in fields such as manufacturing, inspection, document analysis, medical diagnosis, and the military. The challenge of computer vision is to develop visual capabilities for computers and robots that are comparable to human levels. Machine vision requires image signal processing, texture and color modeling, geometric processing and reasoning, and object modeling.

[0090] 2) Augmented Reality (AR)

[0091] Augmented reality (AR) is a technology that cleverly integrates virtual information with the real world. It widely utilizes multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, and sensing technologies to simulate and apply computer-generated text, images, 3D models, music, and videos to the real world. The two types of information complement each other, thus "enhancing" the real world. Also known as augmented reality, AR is a relatively new technology that integrates real-world and virtual information. It uses computer technology to simulate and process physical information that is difficult to experience in the real world, effectively overlaying virtual information onto the real world. This process allows for perception by human senses, creating a sensory experience that transcends reality.

[0092] 3) Virtual Reality (VR)

[0093] Virtual reality (VR) technology, also known as immersive technology, is a novel and practical technology that emerged in the 20th century. It encompasses computer science, electronic information, and simulation technology, and its basic implementation involves using computers to simulate a virtual environment, thus creating a sense of immersion. Virtual reality (VR) is a computer simulation system that can create and allow users to experience virtual worlds. It uses computers to generate a simulated environment, immersing the user within it. VR technology utilizes real-world data, generating electronic signals through computer technology, and combining this with various output devices to transform it into phenomena that people can perceive. These phenomena can be real objects or substances invisible to the naked eye, represented through three-dimensional models.

[0094] Figure 1 This is a schematic diagram illustrating the structure of an object annotation information presentation system according to an exemplary embodiment. The system includes a server 120 and a user terminal 140.

[0095] Server 120 may be a single server, or may include several servers, or may be a virtualization platform, or a cloud computing service center, etc. This application does not impose any restrictions.

[0096] User terminal 140 can be a terminal device with display function, or a terminal device that can realize VR or AR functions. For example, user terminal can be wearable device (such as VR glasses, AR glasses, smart glasses), mobile phone, tablet computer, or e-book reader, etc. The number of user terminals 140 is not limited.

[0097] The user terminal 140 may have a client installed, which may be a 3D map client, an instant messaging client, a browser client, etc. This application embodiment does not limit the type of client software.

[0098] User terminal 140 and server 120 are connected via a communication network. Optionally, the communication network can be a wired network or a wireless network.

[0099] In this embodiment of the application, the server 120 can send the three-dimensional modeling data of the target object to the user terminal 140, and the user terminal 140 can perform three-dimensional modeling of the target object in the VR scene or in the computer background corresponding to the AR scene based on the three-dimensional modeling data.

[0100] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats, including Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0101] Please refer to Figure 2 This is a flowchart illustrating a method for presenting object annotation information according to an exemplary embodiment. The method can be executed by an electronic device, which can be a terminal or a server, or the electronic device can include both a terminal and a server, wherein the terminal can be one of the aforementioned... Figure 1 The user terminal 140 in the illustrated embodiment can be the server described above. Figure 1 Server 120 in the illustrated embodiment. (As shown) Figure 2 As shown, the process of presenting object annotation information may include the following steps:

[0102] Step 21: Obtain the target object in the specified scene, which is the scene presented at the target location.

[0103] In one possible implementation, the specified scene is an augmented reality scene or a virtual reality scene presented at the target location.

[0104] In one possible implementation, the augmented reality scene or virtual reality scene can be a scene presented on a display device corresponding to the user terminal.

[0105] Step 22: Display the annotation information of the target object on the annotation elevation of the target object presented on the display interface; the annotation elevation is determined from the at least two visible elevations of the target object based on the projection areas of each of the at least two visible elevations of the target object presented on the display interface, and the visible elevation is the elevation of the target object's exterior that is visible to the target position.

[0106] The labeled facade is the visible facade corresponding to the target object, used to display the labeled information of the target object. The labeled information can be text labeled information, image labeled information, video labeled information, or any form of labeled information that can be used to display information. The labeled information can be two-dimensional labeled information or three-dimensional labeled information, and this application does not limit it in this regard.

[0107] In summary, the solution described in this application involves acquiring the visible facade of a target object within a virtual reality or augmented reality scene, determining the labeled facade based on the projection of the visible facade onto the display interface, and presenting the labeled information of the target object in the area corresponding to the visible area of ​​the labeled facade. Through this solution, when displaying the label of a target object, the labeled facade can be selected based on the target location and the projection of the visible facade onto the display interface of the specified scene. This allows for the dynamic selection of facades with larger visible areas to display the labeled information within the virtual reality / augmented reality scene, thereby improving the display effect of the labeled information.

[0108] Please refer to Figure 3 This is a flowchart illustrating a method for presenting object annotation information according to an exemplary embodiment. The method can be executed by an electronic device, which can be a terminal or a server, or the electronic device can include both a terminal and a server, wherein the terminal can be one of the aforementioned... Figure 1 The user terminal 140 in the illustrated embodiment can be the server described above. Figure 1 Server 120 in the illustrated embodiment. (As shown) Figure 3 As shown, taking a VR or AR map application scenario where the target object is a building as an example, the process of presenting the object's annotation information can include the following steps:

[0109] Step 301: Obtain the target object in the specified scene.

[0110] The specified scene is the scene presented at the target location.

[0111] In one possible implementation, the specified scene is an augmented reality scene or a virtual reality scene presented at the target location.

[0112] In one possible implementation, when the specified scene is an AR scene, the target location can be the location of the AR device; wherein, the location can be an absolute location, for example, when the AR device is used in an AR map, the AR device can obtain the device's positioning information at this time to obtain the absolute location corresponding to the AR device; the location can also be a relative location, for example, when the AR device is used indoors, the relative position information between the AR device and a certain point in the indoor environment can be obtained to obtain the relative position of the AR device and that point in the indoor environment.

[0113] In one possible implementation, when the specified scene is a VR scene, the target location can also be the location of the virtual character or virtual camera corresponding to the VR device in the virtual 3D scene constructed by the VR device. In this case, each point in the virtual 3D scene can have its own coordinates, and the information corresponding to the target location can be obtained based on the coordinate data.

[0114] In one possible implementation, the visible facade of the target object is the facade that is visible in the exterior facade of the target object when the target object is presented at the target position.

[0115] The visible area refers to the visible portion of the visible facade of the target object, corresponding to the projection area on the display interface.

[0116] In one possible implementation, when the specified scene is a VR or AR scene, a 3D model of the object is constructed based on the 3D model data corresponding to the target object.

[0117] In one possible implementation, when the specified scene is a VR scene, a 3D model of the target object is constructed in the VR scene based on the 3D model data of the target object; when the specified scene is an AR scene, a 3D model of the target object is constructed in the computer background of the AR scene based on the 3D model data of the target object.

[0118] The three-dimensional data corresponding to the target object may include the object's size information, coordinate information, etc.

[0119] In one possible implementation, the visible areas of at least two visible facades of a target object in a specified scene are obtained.

[0120] In one possible implementation, based on the 3D model of the target object and the 3D data corresponding to the 3D model of the target object, the minimum bounding box corresponding to the target object is obtained. The minimum bounding box is the smallest circumscribed cuboid of the target object. Then, the minimum bounding box corresponding to the target object is used as an approximate model of the target object. At this time, when the external shape of the target object is relatively complex, the corresponding calculation of the target object can be realized through the approximate model, which can effectively reduce the amount of calculation.

[0121] In one possible implementation, the facade corresponding to the minimum bounding box of the target object is obtained as the facade of the target object.

[0122] In one possible implementation, the normal vectors of each facade of the target object in the specified scene are obtained, and the facade in which the inner product of the normal vector and the vector corresponding to the display interface of the specified scene is positive is obtained as the visible facade.

[0123] In one possible implementation, the vector corresponding to the display interface of the specified scene can be the normal vector corresponding to the display interface of the specified scene.

[0124] In one possible implementation, the display interface of the specified scene can be a user's observation interface simulated by a virtual camera within the specified scene. In other words, the display interface of the specified scene can be the scene corresponding to the user's observation using the terminal. Taking an AR map as an example, the user terminal obtains real-time location information and, based on the AR map client, obtains the 3D building model information of the surrounding environment corresponding to the target location. The display interface of the specified scene is then the interface on the user terminal displaying the buildings in the corresponding direction; the vector corresponding to the display interface of the specified scene is the vector of the user terminal's orientation.

[0125] In the display interface of a specified scene, the terminal can display the building model corresponding to the current orientation of the terminal, and determine the relationship between the facade and the user's orientation based on the normal vector of each facade of the building model and the inner product of the direction vector of the direction corresponding to the display interface of the specified scene. This determines whether the facade is visible from the user's perspective.

[0126] In one possible implementation, the visible area of ​​the visible facade can be determined by connecting the target location with points on the visible facade.

[0127] For example, in a 3D model constructed based on the 3D model data surrounding the target location, several points on the visible facade of the 3D model corresponding to the target object can be obtained. These points are then connected to the target location. If there are no other buildings corresponding to the 3D models in the line connecting a point to the target location, the location corresponding to that point is determined as the visible area. If there are other buildings corresponding to the 3D models in the line connecting a point to the target location, it means that the line connecting that point to the target location is obscured by other buildings. Therefore, that point can be determined as the invisible area corresponding to the target location. Based on these visible and invisible area points, the visible area of ​​the visible facade can be obtained.

[0128] In one possible implementation, the visible area of ​​the visible facade can be determined based on the boundary point between the target location and the visible facade.

[0129] The dividing point is located on the edge of the visible facade and is used to separate the visible area from the non-visible area.

[0130] Step 302: Obtain the projection area of ​​the at least two visible facades based on the visible areas of the at least two visible facades.

[0131] The visible area is the area of ​​the corresponding visible facade that is visible to the target location in the specified scene.

[0132] In one possible implementation, the entire visible area of ​​each of the at least two visible facades is obtained as the projection area of ​​the at least two visible facades.

[0133] Step 303: Project the projection areas of the at least two visible facades onto the display interface of the specified scene to obtain the projection areas of the visible areas of the at least two visible facades on the display interface of the specified scene.

[0134] In one possible implementation, the normal vector of the display interface of the specified scene is obtained, and the projection area of ​​the visible areas of the at least two visible facades on the display interface of the specified scene is obtained based on the angle between the normal vector of the visible facade and the normal vector of the display interface.

[0135] The visible areas corresponding to at least two visible facades are projected onto the direction corresponding to the display interface of the designated area. In other words, when the visible facade and the display interface of the designated area are not directly opposite each other, that is, when the visible facade and the display interface of the designated area have a certain angle, the visible areas corresponding to the visible facade can be projected onto the direction corresponding to the display interface of the designated area directly observed by the user, based on the angle value between the normal vector of the visible facade and the normal vector of the display interface.

[0136] Step 304: Determine the marked facade from the at least two visible facades based on the projection area of ​​each visible facade on the display interface.

[0137] In one possible implementation, the marked facade is the one with the largest projected area on the display interface among the at least two visible facades.

[0138] When the visible area corresponding to the visible facade is projected onto the display interface of the specified scene to be the largest, that is, when the visible area of ​​the facade is the largest from the user's perspective, the facade can be set as the marked facade.

[0139] In one possible implementation, the visible area of ​​the at least two visible facades is obtained; based on the visible area of ​​the at least two visible facades and the orientational relationship between the at least two visible facades and the display interface of the specified scene, the area of ​​the projection area of ​​each of the visible areas of the at least two visible facades on the display interface of the specified scene is obtained.

[0140] In one possible implementation, occlusion information of the at least two visible facades is obtained, which indicates the occluded vertices and occluded edges of the at least two visible facades; based on the occlusion information, the visible area corresponding to each of the at least two visible facades is obtained.

[0141] When obtaining the visible area of ​​a visible facade, an approximate visible area of ​​the facade can be obtained based on the number of occluded vertices and the number of occluded edges. The visible area of ​​the facade can then be obtained based on this approximate visible area.

[0142] In one possible implementation, when all vertices of the visible facade are not occluded, the area of ​​the visible facade is obtained as the visible area of ​​the visible facade.

[0143] When all vertices of the visible facade are not obscured, it can be assumed that when the visible facade is presented on the display interface at the target location, no other object obscures the facade. Therefore, all areas of the visible facade are visible areas, and the area of ​​the visible facade is obtained as the visible area area of ​​the visible facade.

[0144] In one possible implementation, when there is an occluded vertex on the visible facade, the boundary point on the adjacent edge of the occluded vertex is obtained; the boundary point is used to distinguish between the occluded area and the unoccluded area of ​​the visible facade; based on the boundary point and the unoccluded vertex of the visible facade, the visible area of ​​the visible facade is obtained.

[0145] In one possible implementation, when there are two occluded vertices and one occluded edge on the visible facade, the boundary point on the adjacent edge of the occluded edge is obtained; based on the boundary point and the unoccluded vertices of the visible facade, the visible area of ​​the visible facade is obtained.

[0146] In one possible implementation, when there are two occluded vertices on the visible facade and no occluded edges, the corresponding boundary points on the adjacent edges of the two occluded vertices are obtained respectively; based on the boundary points and the unoccluded vertices of the visible facade, the visible area of ​​the visible facade is obtained.

[0147] In one possible implementation, when there are three occluded vertices on the visible facade, the boundary point on the adjacent edge of the unoccluded vertex of the visible facade is obtained; based on the boundary point and the unoccluded vertex, the visible area of ​​the visible facade is obtained.

[0148] Please refer to Figure 4 This illustrates a facade vertex occlusion classification diagram according to an embodiment of this application. For example... Figure 4 As shown, there are at least seven visible facade situations where the facade vertices are obscured.

[0149] For the visible facade case 401 where no vertices are occluded, since no vertices are occluded by other objects, it can be approximately assumed that the facade corresponding to facade 401 is not occluded. Therefore, the area of ​​the facade is obtained as the visible area of ​​the facade.

[0150] For a visible facade with one vertex occluded, case 402, such as... Figure 5 As shown, it illustrates a schematic diagram of obtaining the visible area of ​​a facade corresponding to a single vertex occlusion, according to an embodiment of this application. Figure 5 As shown, the electronic device obtains the visible area 502 of a certain facade of the object based on the occlusion situation 501 of the facade. From the visible area 502, it can be seen that after point c in the facade is occluded, points a, b, and d are visible. Therefore, based on the occlusion situation on the adjacent edge corresponding to the occluded point c, the boundary points c1 and c2 on the adjacent edge can be recursively obtained using a binary search method. These boundary points c1 and c2 are used as visible endpoints, and together with the unoccluded visible vertices a, b, and d, they form a new shape. This shape is then used as the visible area corresponding to the facade, and the area corresponding to this shape is taken as the visible area area of ​​the facade.

[0151] For a facade with two occluded vertices, where the two occluded vertices share an occluded adjacent edge, as shown in visible facade case 403, the adjacent edge of the two occluded vertices has a boundary point corresponding to the two occluded vertices. Figure 5The method shown recursively obtains the boundary points corresponding to the two occluded vertices through a binary search, which serve as the visible endpoints. These visible endpoints and the two unoccluded visible vertices form a new shape, which is then used as the visible area of ​​the facade. The area of ​​this shape is taken as the visible area of ​​the facade. It should be noted that in visible facade case 403, when the distances between the two boundary points and the corresponding occluded vertices are equidistant, the visible area can be rectangular.

[0152] As shown in visible facade case 404, for a facade with two occluded vertices, where the two occluded vertices are not diagonal (i.e., the two occluded vertices share an adjacent edge, and this adjacent edge is not completely occluded), each occluded vertex has a boundary point corresponding to that occluded vertex on its adjacent edge. Similarly, the boundary points corresponding to these two vertices are obtained as visible endpoints. Based on the two unoccluded vertices and the four visible endpoints corresponding to these two unoccluded vertices, a new shape is constructed, and this shape is taken as the visible area of ​​the facade. The area corresponding to this shape is taken as the visible area of ​​the facade.

[0153] As shown in visible facade case 405, for a facade with two occluded vertices, and the two occluded vertices are diagonal vertices, similar to visible facade case 404, each occluded vertex has a boundary point corresponding to that occluded vertex on its adjacent edge. Similarly, the boundary points corresponding to the two occluded vertices are obtained as visible endpoints. Based on the two unoccluded vertices and the four visible endpoints corresponding to the two unoccluded vertices, a new shape is constructed, and this shape is taken as the visible area of ​​the facade, with the area corresponding to the shape being taken as the visible area of ​​the facade.

[0154] As shown in visible facade case 406, for a facade with three vertices obscured, that is, only one vertex is visible, the boundary point corresponding to the visible vertex is obtained on the adjacent edge corresponding to the visible vertex by the bisection method. Based on the boundary point and the visible vertex, a new shape is formed, and the shape is taken as the visible area corresponding to the facade. The area corresponding to the shape is taken as the visible area of ​​the facade.

[0155] As shown in visible facade case 407, for a facade with four vertices obscured, it can be considered that the facade is completely obscured. Therefore, it is considered that the facade has no corresponding visible area, that is, the visible area of ​​the facade is 0.

[0156] Step 305: Determine the annotation area from the visible area of ​​the annotation facade; the annotation area is the area with the largest area among the regions of the first shape included in the visible area of ​​the annotation facade.

[0157] In one possible implementation, occlusion information of the labeled facade is obtained, which indicates the occluded vertices and edges of the labeled facade; based on the occlusion information, the labeled area is determined within the visible area of ​​the labeled facade.

[0158] In one possible implementation, the first shape is a rectangle. When the occlusion information indicates that there is an occluded vertex on the labeled elevation, the diagonal vertex of the occluded vertex is taken as the first target point. A first endpoint is determined on the non-adjacent edge corresponding to the first target point, such that the rectangle with the line segment between the first endpoint and the first target point as its diagonal is the rectangle with the largest area in the visible area of ​​the labeled elevation. The area where the rectangle with the line segment between the first endpoint and the first target point as its diagonal is located is determined as the labeled area.

[0159] Please refer to Figure 6 This illustration shows a schematic diagram of a method for calculating a labeled range according to an embodiment of this application. For example... Figure 6 As shown, when there is only one occluded vertex, obtain the diagonal vertex a (first target point) of the occluded vertex, and find another point (first endpoint) on the non-adjacent line segment (non-adjacent edge) of a to maximize the area of ​​the rectangle formed. Here, the non-adjacent line segment is the line segment in the visible area that constitutes the facade and is not directly connected to a.

[0160] In one possible implementation, the first shape is a rectangle. When the occlusion information indicates that there are two occluded vertices on the labeled elevation, and the edge between the two occluded vertices is completely occluded, the vertex with the largest sum of the lengths of the unoccluded portions of its adjacent edges among the unoccluded vertices of the labeled elevation is obtained as the second target point. A second endpoint is determined on the non-adjacent edge corresponding to the second target point. The rectangle with the second endpoint located in the visible area of ​​the labeled elevation and the line segment between the second endpoint and the second target point as its diagonal is the rectangle with the largest area in the visible area of ​​the labeled elevation. The area where the rectangle with the line segment between the second endpoint and the second target point as its diagonal is located is determined as the labeled area.

[0161] In one possible implementation, the first shape is a rectangle. When the occlusion information indicates that there are two occluded vertices on the labeled elevation and no completely occluded edges, a target point set is obtained. The target point set includes the unoccluded vertices of the labeled elevation and the boundary points on the adjacent edges of the two occluded vertices. The boundary points are used to distinguish between the occluded and unoccluded areas of the visible elevation. A third endpoint is determined within the visible area of ​​the labeled elevation. The rectangle with the line segment between the third endpoint and the third target point as its diagonal is the rectangle with the largest area within the visible area of ​​the labeled elevation. The third target point is one of the target point sets. The area containing the rectangle with the line segment between the third endpoint and the third target point as its diagonal is determined as the labeled area.

[0162] Please refer to Figure 7 This illustration shows a schematic diagram of a method for calculating a labeled range according to an embodiment of this application. For example... Figure 7 As shown, when there are two occluded vertices in the visible facade, we can analyze three cases. In the first case 701, when the occluded area exceeds the edge line, that is, when the edge between the two occluded vertices is completely occluded, the visible area is a quadrilateral. Points a and b are the two unoccluded vertices corresponding to the marked facade. As can be seen from 701, the sum of the lengths of the unoccluded parts of the two adjacent sides corresponding to a is greater than the sum of the lengths of the unoccluded parts of the two adjacent sides corresponding to b. Therefore, a can be used as the second target vertex (base point). On the non-adjacent line segment of a, find the second endpoint that forms the largest rectangle with a, that is, point c1 in 701. At this time, the rectangle formed by a and c1 as diagonal points is the rectangle with the largest area in the visible area corresponding to 701.

[0163] In the second case 702, when the occluded area does not exceed the edge line and the two unoccluded vertices of the visible facade are diagonal points, the occluded area may form an octagon as shown in 702. In the third case 703, when the occluded area does not exceed the edge line and the two occluded vertices of the visible facade share an adjacent edge, the occluded area may form an octagon as shown in 703.

[0164] For the labeled elevations shown in 702 and 703, obtain the target vertex set of the labeled elevation, that is, obtain the unobstructed vertices of the labeled elevation and the corresponding boundary points on the adjacent sides of the two obstructed vertices. Using one point in the target vertex set as the base point, obtain the rectangle with the largest area in the visible area of ​​the labeled elevation corresponding to that point.

[0165] In one possible implementation, the first shape is a rectangle. When the occlusion information indicates that there are three occluded vertices on the labeled facade, the unoccluded vertex of the labeled facade is obtained as the fourth target point. The area where the rectangle formed by the fourth target point and the boundary point on the two adjacent sides of the fourth target point is located is determined as the labeled area. The boundary point is used to distinguish between the occluded area and the unoccluded area of ​​the visible facade.

[0166] Please refer to Figure 8 This illustration shows a schematic diagram of a method for calculating a labeled range according to an embodiment of this application. For example... Figure 8 As shown, when three vertices are occluded, the unoccluded vertex (i.e., point a) is used as the base point, and the rectangle with the largest area is formed according to the boundary points c1 and c2 corresponding to point a on the adjacent side. The area corresponding to this rectangle is then obtained as the labeled area.

[0167] In one possible implementation, the first shape is a rectangle. When the occlusion information indicates that there are three occluded vertices on the labeled elevation, the unoccluded vertex of the labeled elevation is obtained as the fourth target point; the fourth endpoint is determined within the visible area of ​​the labeled elevation; the rectangle with the line segment between the fourth endpoint and the fourth target point as its diagonal is the rectangle with the largest area within the visible area of ​​the labeled elevation; the area where the rectangle with the line segment between the fourth endpoint and the fourth target point as its diagonal is located is determined as the labeled area.

[0168] When three vertices are occluded, the fourth endpoint within the visible area of ​​the labeled region can be obtained directly from the unoccluded vertices of the labeled facade. The region corresponding to the largest rectangle formed by this fourth endpoint and the unoccluded vertex is then defined as the labeled region. When the occluded region is a rectangle, the rectangle formed by the fourth endpoint and the unoccluded vertex is... Figure 8 The corresponding largest rectangles are the same.

[0169] In one possible implementation, the marked area is a region of a specified shape within the visible facade.

[0170] exist Figures 6 to 8 In the example, the marked area is a rectangular area in the visible facade. In the embodiments of this application, the marked area can also be a circle, triangle or other areas of a specified shape. This application does not limit this.

[0171] Step 306: Display the annotation information of the target object on the annotation elevation of the target object in the display interface.

[0172] In one possible implementation, the annotation information of the target object can be presented on the labeled elevation of the target object displayed on the display interface based on the depth information.

[0173] In AR or VR scenarios, the model corresponding to the target object is a model constructed based on three-dimensional data. Therefore, the annotation information in the annotation area of ​​the annotation facade may have a depth attribute. That is, the annotation information presented in the annotation area corresponding to the annotation facade of the model corresponding to the target object can be a annotation information with three-dimensional attributes.

[0174] In one possible implementation, a three-dimensional model of the annotation information is generated based on the size of the annotation area; the three-dimensional model of the annotation information is presented on a plane parallel to the annotation area in the display interface; the parallel plane is a plane located in front of the annotation facade and parallel to the annotation facade.

[0175] In one possible implementation, the dimensions of the annotation information are generated based on the dimensions of the annotation area; based on the dimensions of the annotation information, a 3D model of the annotation information is displayed for the annotation area within a plane parallel to the annotation elevation, and presented on the display interface corresponding to the target location. That is, the dimensions of the annotation information are related to the dimensions of the annotation area; the larger the size of the annotation area, the larger the dimensions of the annotation information, and the larger the 3D model of the annotation information is presented on the display interface.

[0176] In one possible implementation, the display direction of the annotation information is determined based on the horizontal and vertical lengths of the annotation area on the annotation facade; based on the display direction of the annotation information, the annotation information of the target object is presented on the annotation facade of the target object displayed on the display interface.

[0177] In one possible implementation, when the annotation information is text-based, the presentation direction of the annotation information can be determined based on the horizontal and vertical lengths of the annotation area on the annotation facade. For example, when the horizontal length of the annotation area is greater than its vertical length, the annotation information can be presented horizontally on the annotation area displayed on the screen; when the vertical length of the annotation area is greater than its horizontal length, the annotation information can be presented vertically on the annotation area displayed on the screen.

[0178] In one possible implementation, the distance information between the target object and the display interface of the specified scene is obtained; when the distance information is less than a threshold, the annotation information corresponding to the target object is displayed on the annotation area corresponding to the annotation facade in the display interface.

[0179] When the target object is too far away from the display interface of the specified scene, the field of view when observing the target object from the target position is small, and the annotation information is difficult to identify. In this case, the annotation information is not displayed on the annotation surface corresponding to the target object in the display interface.

[0180] In one possible implementation, when the area of ​​the projected area of ​​the visible region of the marked facade on the display interface is greater than a specified area threshold, the marked information of the target object is displayed on the display interface for the visible region of the marked facade.

[0181] When the area of ​​the labeled elevation is too small, the labeled area displayed on the display interface may be difficult to identify. In this case, the labeled information will not be displayed on the labeled elevation corresponding to the target object displayed on the display interface.

[0182] Please refer to Figure 9 This illustrates a data resource flowchart related to an embodiment of this application. For example... Figure 9 As shown in the embodiments of this application, the solution can be implemented on a terminal that deploys AR SDK (Software Development Kit) platform software and a 3D building data model. During runtime, the program code runs in the host memory and / or GPU (Graphics Processing Unit) memory of the terminal device, loading building models of the surrounding area from the server. Based on the underlying support provided by the terminal, the terminal can perform text annotation calculations and rendering, including calculating the visible area of ​​the building, the area of ​​the projected visible area, and the display range of the text annotations, as well as displaying and handling text annotation conflicts.

[0183] Please refer to Figure 10 The diagram illustrates a structural diagram of an annotation rendering method according to an embodiment of this application. Figure 10 As shown, taking text annotation display as an example, the method structure consists of underlying support 1001, annotation text calculation 1002, and rendering display 1003, and the method is executed by an electronic device. The method shown in this application can be developed on the Unity 3D rendering engine software, where Unity software provides a packaged 3D rendering pipeline, providing functions such as collision detection and 3D rendering, and loading the 3D model of city buildings according to the data corresponding to the 3D buildings, thereby providing the underlying support for this method.

[0184] The electronic device determines the visible facade of the building based on the minimum bounding box; then, based on the minimum bounding box of the building, it determines the visible vertex corresponding to the visible facade and the visible area range of the facade; and selects the facade with the largest area projection as the display facade by projecting the area of ​​the visible area in the direction corresponding to the user.

[0185] Based on the determined display facade, the maximum text annotation display area on that facade is calculated, and the text annotations are displayed within the text annotation display area corresponding to the 3D building model of the building with the maximum text annotation. Furthermore, after the text annotation content is rendered and displayed on the facade, occlusion may occur when projected onto the screen. For example, a text annotation that is close to the others may obscure a text annotation behind it, creating a conflict. Therefore, it is necessary to handle the display conflict of these text annotations.

[0186] Furthermore, during the execution of the above method, the camera begins real-time pose simulation, using electronic devices to simulate the user's real-time position and angle, thereby verifying the correctness of the displayed text annotation.

[0187] Please refer to Figure 11 This illustrates a flowchart of the visible facade calculation for a building according to an embodiment of this application. Figure 11 As shown:

[0188] S1101, Electronic equipment determines the buildings that need to be labeled.

[0189] S1102. Based on the 3D model corresponding to the marked building, obtain the triangular mesh structure corresponding to the 3D model of the building, obtain the coordinates of each vertex of the building, calculate the minimum bounding box structure (i.e. cuboid structure) corresponding to the building based on the coordinates of each vertex of the building, and obtain the minimum bounding box as the target model corresponding to the building.

[0190] S1103, obtain the vector Vcam from the center of the building to the camera, calculate the normal vector Vnor of each facade of the target model, and calculate the projection length L of Vnor onto the Vcam vector.

[0191] S1104, When L is greater than 0, it means that the projection vector of the Vnor vector onto the Vcam is in the same direction as the Vcam vector. In this case, the facade corresponding to the Vnor vector is visible in the Vcam direction. When L is less than 0, it means that the projection vector of the Vnor vector onto the Vcam is in the opposite direction to the Vcam vector. In this case, the facade corresponding to the Vnor vector is invisible in the Vcam direction. S1105, Add the visible facades to the set of visible facades.

[0192] Please refer to Figure 12 The diagram illustrates a flowchart of a method for calculating a visible area according to an embodiment of this application.

[0193] S1201, Electronic equipment determines the buildings that need to be labeled.

[0194] S1202. Based on the 3D model corresponding to the marked building, obtain the triangular mesh structure corresponding to the 3D model of the building, obtain the coordinates of each vertex of the building, calculate the minimum bounding box structure (i.e. cuboid structure) corresponding to the building based on the coordinates of each vertex of the building, and obtain the minimum bounding box as the target model corresponding to the building.

[0195] S1203, traverse the visible facades in the set, and based on the positional relationship between the camera and the facade vertices corresponding to the building, determine whether the line connecting the camera and the visible facade vertices of the building is blocked by the building, and determine whether the facade vertices are visible in the camera's field of view.

[0196] S1204: When a vertex is visible, directly input the vertex into the set of visible vertices. When a vertex is invisible, use a binary search method to determine the boundary between the occluded and unoccluded regions on the adjacent edges of the vertex.

[0197] S1205, input the boundary point as a visible vertex into the set of visible vertices.

[0198] S1206 After identifying all the visible vertices corresponding to the building, the visible area corresponding to the building model is formed based on the visible vertices, and the area of ​​the visible area is calculated based on the occlusion status of the vertices.

[0199] S1207, based on the projection of the visible area onto the camera's line of sight, obtain the size of the visible area on the screen, which is the size of the area that the user can actually observe on the screen.

[0200] Please refer to Figure 13 The diagram illustrates a calculation flowchart for a text annotation range according to an embodiment of this application. The above-mentioned visible area is the size of the area observed on the screen (i.e., the projected area of ​​the visible facade).

[0201] S1301. Compare the visible area areas and select the facade with the largest visible area as the labeled facade.

[0202] S1302, based on the occlusion of the vertex, calculate the rectangle with the largest area in the visible region, and use the area corresponding to the rectangle as the annotation range, that is, the annotation information corresponding to the building can be displayed in the annotation range.

[0203] S1303 Before displaying the annotation information corresponding to the building within the marked area, it is necessary to determine whether to display the annotation information. This involves evaluating the area of ​​the marked area and the distance between the building and the camera to determine whether to display the annotation information. For example, when the marked area is small, the annotation information displayed within that area will be relatively small, and the user may have difficulty seeing it clearly. Therefore, the marked area may not be displayed. Similarly, when the building is far from the camera, the display area of ​​the building on the terminal may also be small, and the user may also be unable to see the displayed annotation information clearly. Therefore, the marked area may not be displayed.

[0204] Please refer to Figure 14 The diagram illustrates a real-time camera pose simulation flowchart according to an embodiment of this application. Figure 14 As shown, during the display of the above annotations, it is possible to run in real time as follows: Figure 14 The steps shown involve inputting corresponding commands to the terminal to control camera movement, viewpoint rotation, and other response events, simulating human scene browsing to verify the building information annotation method shown in this application embodiment. A personal computer is used as an example of the terminal implementation.

[0205] With the S1401, users can input corresponding commands through external devices such as keyboards and mice to control camera objects in the simulated 3D scene.

[0206] S1402, the electronic device responds in real time to user input such as keyboard and mouse input, allowing the camera to perform panning and view rotation actions.

[0207] S1403, during operation, the external input is controlled to allow the camera to simulate a human perspective to view the city from a human perspective, respond to the building annotation information in real time, and determine whether the information annotation method corresponding to the embodiment of this application can be displayed normally based on the display of the building annotation information.

[0208] This application can be used not only for text annotation on buildings, but also for information annotation in various forms such as images and videos. For example, it can be used in AR scenarios to project video advertisements onto building facades, display new car models, introduce store information on shopping mall buildings, and introduce the history of museum collections, among other applications. By identifying the visible plane of a building in an AR scene, multimedia information can be displayed within the scene, and users can receive information from the AR scene from different angles, enriching the AR experience.

[0209] This application also proposes to calculate the visible area of ​​a building model in a 3D scene and the projected area of ​​the visible area in the direction of human eye line of sight, so as to determine the facade direction of the building annotation display.

[0210] First, the visible area of ​​the building model in the 3D scene is calculated. The occlusion of building facade vertices is categorized and calculated, and the area within the visible area is calculated. This visible area is then projected onto the viewer's line of sight to obtain the size of the visible area on the screen. Next, the text annotation is displayed within this visible area. The text annotation is now positioned within the building's visible area, and its direction aligns with the building's facade. The text annotation is 3D and will not be occluded by buildings in the current frame. The 3D text annotation displays with varying distances on the screen; text closer to the user appears larger, and the text direction aligns with the building's direction, better expressing the relationship between the text annotation and the annotated building.

[0211] Furthermore, by calculating the display range of text annotations within the visible area, the annotation text is adaptively displayed according to different viewing angles. The solution shown in this application dynamically calculates the range of the visible area, determining the rectangle with the largest area. The visible area is a two-dimensional plane, and its shape changes with the user's viewing angle. The base point of the rectangle needs to be determined based on the visibility of the facade vertices and the vertex relationships, and the position of the rectangle's symmetrical points is judged to determine the rectangle with the largest area. When the user browses using the AR application, the building content in the scene changes, and the building text annotations change with the scene, presenting the annotations to the user from a better viewing angle.

[0212] Please refer to Figure 15 The diagram illustrates a comparison between an embodiment of this application and an AR map technology. Figure 15 As shown in the embodiment of this application, in scheme 1501, the building annotations change under different viewing angles. The building annotations are within the visible range of the scene, and their direction is in the direction of the building facade. Users can easily associate the annotations with the annotated buildings, and the annotations provide users with changes in distance. The annotations can provide a certain degree of location and direction guidance, bringing users a better AR experience. However, in the AR map technology shown in 1502, the association between the annotations and the annotated buildings is not strong.

[0213] Annotation in AR scenes has the following characteristics: 1. When a user browses, the number of building annotations in the current frame should not be excessive. Annotations should be prioritized based on distance from the user and the size of the visible area; the closer the annotation and the larger the visible area, the higher the priority. 2. When the vertical axis of the visible area is longer than the horizontal axis, building information can be displayed vertically. 3. When the text annotation area is too small to effectively display building information, it can be ignored.

[0214] The AR map technology 1502 uses an information annotation method in AR scenes that identifies the range of buildings on the screen and then tiles the text annotations onto the screen range of the buildings. The information annotations are still two-dimensional planes and do not utilize the depth information in the AR scene. Moreover, the tiled information annotations are not strongly related to the buildings they belong to, and may obscure other content in the scene.

[0215] The difference between Embodiment 1501 of this application and AR map technology 1502 is that it uses three-dimensional information annotations, which are displayed in the visible area of ​​the building. The normal direction of the information annotations is consistent with the normal of the facade, making the information annotations more related to the building and becoming part of the AR scene. Moreover, it obstructs other content in the scene less, giving users a better experience when experiencing AR.

[0216] In AR map technology 1501, the annotations of buildings in a 3D scene are static. Blind spots or incorrect annotation information may occur when users browse the annotations from different perspectives. By calculating the visible area of ​​the building from the user's perspective in the current frame, the text content is displayed within that visible area. In the solution shown in this embodiment, the annotation information of the building automatically adjusts as the user browses the building from different perspectives, changing with the perspective. This method allows the annotation information to be presented dynamically, making information annotation in a 3D scene more flexible and intelligent. Building annotations can also be expressed using multimedia methods such as images and videos. Annotation information is an important interactive entry point in AR applications.

[0217] In AR map technology 1502, the annotations of buildings in a 3D scene are tiled on the screen, which has a weak correlation with the building and may obstruct other scene content. However, in the solution shown in the embodiment of this application, the 3D annotations in the 3D scene are calculated so that the annotation direction is consistent with the facade direction of the building. This allows the annotation information to be displayed on the screen as it changes in distance. Therefore, by implementing 3D building annotations, the position and direction of the annotations can be expressed in a directional manner, which enhances the correlation between the annotations and the annotated buildings. Moreover, the text is expressed linearly, which can indicate the orientation information of the buildings to a certain extent, further enriching the user's 3D experience in the AR scene.

[0218] In summary, the solution presented in this application involves obtaining the visible facade of a target object within a virtual reality or augmented reality scene, determining the labeled facade based on the projection of the visible facade onto the display interface, and displaying the labeled information of the target object in the area corresponding to the visible area of ​​the labeled facade. Through this solution, when displaying the label of a target object, the labeled facade can be selected based on the target location and the projection of the visible facade onto the display interface of the specified scene. This allows for the dynamic selection of facades with larger visible areas to display the labeled information within the virtual reality / augmented reality scene, thereby improving the display effect of the labeled information.

[0219] Please refer to Figure 16 This is a flowchart illustrating a method for presenting object annotation information according to an exemplary embodiment. The method can be executed by an electronic device, which can be a terminal or a server, or the electronic device can include both a terminal and a server, wherein the terminal can be one of the aforementioned... Figure 1 The user terminal 140 in the illustrated embodiment can be the server described above. Figure 1 Server 120 in the illustrated embodiment. (As shown) Figure 3 As shown, taking a VR or AR map application scenario where the target object is a building as an example, the process of presenting the object's annotation information can include the following steps:

[0220] Step 1601: Obtain the target object in the specified scene.

[0221] In one possible implementation, when the specified scene is a VR scene, the target object can be an object constructed by the VR device based on 3D model data; when the specified scene is an AR scene, the target object can be an object captured by the AR device through a camera component.

[0222] Step 1602: Determine candidate annotation areas from the visible areas of the at least two visible facades respectively.

[0223] The candidate labeling area is the largest area among the regions of the second shape contained in the visible area of ​​the corresponding facade.

[0224] The method for determining this candidate labeled region is similar to... Figure 3 The method for confirming the annotation area in the annotation elevation shown in step 305 is similar and will not be repeated here.

[0225] Step 1603: Obtain the candidate annotation areas corresponding to each of the at least two visible facades as the projection areas of the at least two visible facades.

[0226] The candidate annotation area is the second largest shape area in the visible area of ​​the corresponding facade. In other words, in this embodiment, a portion of the visible area of ​​the visible facade can be used as the projection area, i.e., the candidate annotation area can be used as the projection area.

[0227] In one possible implementation, the projection areas of the at least two visible facades are projected onto the display interface of the specified scene to obtain the projection area of ​​each of the at least two visible facades on the display interface of the specified scene.

[0228] In one possible implementation, the candidate annotation area is used as the projection area and projected onto the display interface of the specified scene to obtain the projection area of ​​each of the candidate annotation areas of the at least two visible facades on the display interface of the specified scene.

[0229] Step 1604: Determine the marked facade from the at least two visible facades based on the projection area of ​​each of the visible areas of the at least two visible facades on the display interface of the specified scene.

[0230] In one possible implementation, the labeled facade is determined from the at least two visible facades based on the projection area of ​​each of the candidate labeled areas of the at least two visible facades on the display interface of the specified scene.

[0231] In one possible implementation, the labeled facade is determined from the at least two visible facades based on the size of the projected area corresponding to the projection area of ​​each of the candidate labeled areas on the display interface of the specified scene.

[0232] Before determining the labeled facade of the target object, candidate labeled areas of a specified shape for each visible facade can be obtained first. These candidate labeled areas are then used as projection areas and projected onto the display interface of the specified scene. The labeled facade is then determined based on the size of the projected area on the display interface of that scene. Since the angle between the target location and different visible facades varies, the projected area of ​​the visible region of a certain facade may be large, but the projected area of ​​its corresponding labeled region may be small. Therefore, the labeled facade can also be determined by first comparing the projected areas of the labeled facades of each facade and then using the comparison results.

[0233] Step 1605: For the candidate annotation area corresponding to the annotation facade, display the annotation information of the target object on the annotation facade presented on the display interface.

[0234] After determining the labeled facade based on the candidate labeled areas corresponding to at least two facades, the corresponding labeled information of the target object can be presented in the display area of ​​the candidate labeled area on the display interface. That is, firstly, the area of ​​the largest area of ​​a specified shape of each visible facade projected onto the display interface is compared, and the candidate labeled area of ​​the visible facade corresponding to the largest projected area is used to present the labeled information.

[0235] In summary, the solution presented in this application involves obtaining the visible facade of a target object within a specified scene, determining the labeled facade based on the projection of the visible facade onto the display interface, and then displaying the labeling information of the target object in the area corresponding to the visible area of ​​the labeled facade. This solution improves the display effect of the labeling information by allowing the selection of the labeled facade based on the target location and the projection of the visible facade onto the display interface of the specified scene when displaying the labeling of the target object.

[0236] Please refer to Figure 17 This illustrates a flowchart of a method for presenting object annotation information. For example... Figure 17 As shown, taking the method running on user terminal 1700 and the application scenario as an AR map as an example, user terminal 1700 constructs several three-dimensional building models 1702 in the direction corresponding to the user terminal based on the three-dimensional model data 1701 and the location and orientation information of the user terminal at this time. When identifying a target building model, the visible facade 1703 corresponding to the direction of the user terminal is obtained, and based on the visible facade corresponding to the direction of the target building, the visible area 1704 corresponding to each visible facade and the visible area area corresponding to each visible area 1704 are obtained.

[0237] Based on the projection 1705 of the visible area in the direction of the user terminal, the visible facade corresponding to the largest projected area of ​​the visible area in the direction of the user terminal is obtained as the labeled facade 1706. Based on the labeled facade, the area of ​​the largest region of a specified shape in the labeled facade is obtained as the labeled area 1707 corresponding to the labeled facade. Based on the labeled area 1707, the labeled information 1708 corresponding to the building is displayed on the 3D building model corresponding to the building and presented on the display interface.

[0238] Figure 18 This is a structural block diagram illustrating an object annotation information presentation device according to an exemplary embodiment. This object annotation information presentation device can achieve... Figure 2 , Figure 3 or Figure 16The illustrated embodiment provides all or part of the steps in the method. The object annotation information presentation device may include:

[0239] The target object acquisition unit 1801 is used to acquire a target object in a specified scene, wherein the specified scene is a scene presented at the target location;

[0240] The annotation information presentation unit 1802 is used to present the annotation information of the target object on the annotation facade of the target object presented on the display interface. The annotation facade is determined from the at least two visible facades based on the projection areas of each of the at least two visible facades of the target object presented on the display interface. The visible facade is the facade of the target object that is visible to the target position.

[0241] In one possible implementation, the designated scene is an augmented reality scene or a virtual reality scene presented at the target location.

[0242] In one possible implementation, the marked facade is the one with the largest projected area on the display interface among the at least two visible facades.

[0243] In one possible implementation, the device further includes:

[0244] The projection area acquisition unit is used to acquire the projection area of ​​the at least two visible facades based on the visible areas of the at least two visible facades; the visible area is the area of ​​the corresponding visible facade that is visible to the target position in the specified scene;

[0245] The projection area acquisition unit is used to project the projection areas of the at least two visible facades onto the display interface to obtain the projection areas of each of the at least two visible facades on the display interface.

[0246] In one possible implementation, the region to be projected acquisition unit is used for,

[0247] The entire visible area of ​​each of the at least two visible facades is obtained as the projection area of ​​the at least two visible facades.

[0248] In one possible implementation, the annotation information presentation unit 1802 includes:

[0249] A region determination subunit is used to determine a labeling region from the visible area of ​​the labeling facade; the labeling region is the region with the largest area among the regions of a first shape included in the visible area of ​​the labeling facade.

[0250] The annotation information presentation subunit is used to present the annotation information of the target object on the annotation elevation of the target object presented on the display interface.

[0251] In one possible implementation, the region defining sub-units includes:

[0252] An occlusion information acquisition subunit is used to acquire occlusion information of the marked facade, wherein the occlusion information is used to indicate the occluded vertices and occluded edges of the marked facade;

[0253] The annotation area determination subunit is used to determine the annotation area in the visible area of ​​the annotation facade based on the occlusion information.

[0254] In one possible implementation, the first shape is rectangular, and the labeled area defines sub-units for...

[0255] When the occlusion information indicates that there is an occluded vertex on the marked elevation, the diagonal vertex of the occluded vertex is taken as the first target point;

[0256] A first endpoint is determined on the non-adjacent edge corresponding to the first target point, such that the rectangle with the line segment between the first endpoint and the first target point as its diagonal is the rectangle with the largest area within the visible area of ​​the marked elevation.

[0257] The area containing the rectangle whose diagonal is the line segment between the first endpoint and the first target point is defined as the labeled area.

[0258] In one possible implementation, the first shape is rectangular, and the labeled area defines sub-units for...

[0259] When the occlusion information indicates that there are two occluded vertices on the labeled elevation, and the edge between the two occluded vertices is completely occluded, the vertex with the largest sum of the lengths of the unoccluded parts of its adjacent edges among the unoccluded vertices on the labeled elevation is obtained as the second target point.

[0260] A second endpoint is determined on the non-adjacent edge corresponding to the second target point; the second endpoint is located in the visible area of ​​the labeled facade, and the rectangle with the line segment between the second endpoint and the second target point as its diagonal is the rectangle with the largest area in the visible area of ​​the labeled facade.

[0261] The area containing the rectangle whose diagonal is the line segment between the second endpoint and the second target point is defined as the labeled area.

[0262] In one possible implementation, the first shape is rectangular, and the labeled area defines sub-units for...

[0263] When the occlusion information indicates that there are two occluded vertices on the labeled facade and no completely occluded edges, a target point set is obtained; the target point set includes the unoccluded vertices of the labeled facade and the boundary points on the adjacent edges of the two occluded vertices; the boundary points are used to distinguish between the occluded and unoccluded areas of the visible facade.

[0264] A third endpoint is determined within the visible area of ​​the marked facade; the rectangle with the line segment between the third endpoint and the third target point as its diagonal is the rectangle with the largest area within the visible area of ​​the marked facade; the third target point is one of the target points in the set.

[0265] The area containing the rectangle whose diagonal is the line segment between the third endpoint and the third target point is defined as the labeled area.

[0266] In one possible implementation, the first shape is rectangular, and the labeled area defines sub-units for...

[0267] When the occlusion information indicates that there are three occluded vertices on the marked elevation, the unoccluded vertices on the marked elevation are obtained as the fourth target point;

[0268] The area containing the rectangle formed by the fourth target point and the boundary points on its two adjacent sides is defined as the marked area; the boundary points are used to distinguish between the obscured and unobscured areas of the visible facade.

[0269] In one possible implementation, the annotation information presentation sub-unit includes:

[0270] The annotation information model generation subunit is used to generate a three-dimensional model of the annotation information based on the size of the annotation area;

[0271] The annotation information model presentation subunit is used to present a three-dimensional model of the annotation information on a parallel plane of the annotation area presented on the display interface; the parallel plane is a plane located in front of the annotation facade and parallel to the annotation facade.

[0272] In one possible implementation, the region to be projected acquisition unit includes:

[0273] A candidate annotation region determination subunit is used to determine candidate annotation regions from the visible regions of the at least two visible facades respectively; the candidate annotation region is the region with the largest area among the regions of a second shape contained in the visible region of the corresponding facade;

[0274] The sub-unit for obtaining the projection area is used to obtain the candidate annotation areas corresponding to each of the at least two visible facades as the projection areas of the at least two visible facades.

[0275] In one possible implementation, the annotation information presentation unit 1802 is further configured to,

[0276] The annotation information of the target object is presented on the annotation surface of the display interface, corresponding to the candidate annotation area of ​​the annotation facade.

[0277] In one possible implementation, the annotation information presentation unit 1802 is used for,

[0278] When the area of ​​the visible region of the marked facade projected onto the display interface is greater than a specified area threshold, the marking information of the target object is presented on the marked facade displayed on the display interface.

[0279] In summary, the solution presented in this application involves obtaining the visible facade of a target object within a virtual reality or augmented reality scene, determining the labeled facade based on the projection of the visible facade onto the display interface, and displaying the labeled information of the target object in the area corresponding to the visible area of ​​the labeled facade. Through this solution, when displaying the label of a target object, the labeled facade can be selected based on the target location and the projection of the visible facade onto the display interface of the specified scene. This allows for the dynamic selection of facades with larger visible areas to display the labeled information within the specified scene, thereby improving the display effect of the labeled information.

[0280] It is understood that, in order to achieve the above-mentioned functions, electronic devices include hardware structures and / or software modules (or units) corresponding to the execution of each function. Based on the units and algorithm steps described in the embodiments disclosed in this application, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of the embodiments of this application.

[0281] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0282] When using integrated units, Figure 19 A schematic diagram of a possible structure of the electronic device involved in the above embodiments is shown. The electronic device 1900 includes a processing unit 1902 and a communication unit 1903. The processing unit 1902 is used to control and manage the actions of the electronic device 1900. For example, when the electronic device 1900 is a user terminal, the processing unit 1902 is used to support the electronic device 1900 in performing... Figure 2 Steps 21 to 22 in the illustrated embodiment, Figure 3 Steps 301 to 306 in the illustrated embodiment Figure 16 Steps 1601 to 1605 in the illustrated embodiments, and / or other steps for performing the techniques described herein. The electronic device 1900 may also include a storage unit 1901 for storing program code and data of the electronic device 1900. For example, when the electronic device 1900 is a user terminal, the storage unit 1901 stores the three-dimensional model data described above.

[0283] The processing unit 1902 can be a processor or controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The communication unit 1903 can be a communication interface, transceiver, transceiver circuit, etc., where "communication interface" is a general term and may include one or more interfaces. The storage unit 1901 can be a memory.

[0284] When the processing unit 1902 is a processor, the communication unit 1903 is a communication interface, and the storage unit 1901 is a memory, the electronic device involved in the embodiments of this application can be... Figure 20 The electronic device shown.

[0285] See Figure 20 As shown, the electronic device 2010 includes a processor 2012, a communication interface 2013, and a memory 2011. Optionally, the electronic device 2010 may also include a bus 2014. The communication interface 2013, processor 2012, and memory 2011 can be interconnected via the bus 2014; the bus 2014 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 2014 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 20 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0286] The above Figure 19 or Figure 20 The electronic device shown can be a user terminal or a server.

[0287] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules (or units), which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs (compact disc read-only memory), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in an electronic device. Of course, the processor and storage medium can also exist as discrete components in an electronic device.

[0288] This application also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the aforementioned object annotation information presentation method.

[0289] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0290] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above descriptions are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A method for presenting object annotation information, characterized in that, The method includes: Obtain the target object in a specified scene, wherein the specified scene is a scene presented at the target location; Identify at least two visible facades of the target object presented on the display interface, wherein the visible facades are the facades of the target object that are visible from the viewpoint corresponding to the target position. Based on the visible areas of the at least two visible facades, obtain the projection areas of the at least two visible facades; the visible areas are the areas of the corresponding visible facades that are visible from the viewpoint corresponding to the target position in the specified scene. Projecting the projection areas of the at least two visible facades onto the display interface, thereby obtaining the projection areas of each of the at least two visible facades on the display interface. Based on the projection area of ​​each of the at least two visible facades on the display interface, the labeled facade of the target object is selected from the at least two visible facades. The annotation information of the target object is presented on the annotation elevation. The annotation elevation of the target object where the annotation information is located changes with the perspective change caused by the change of the target position, and the distance of the annotation information presented on the display interface changes.

2. The method according to claim 1, characterized in that, The specified scene is an augmented reality scene or a virtual reality scene presented at the target location.

3. The method according to claim 1, characterized in that, The marked facade is the one with the largest projected area on the display interface among the at least two visible facades.

4. The method according to claim 1, characterized in that, The step of obtaining the projection area of ​​the at least two visible facades based on their visible areas includes: The entire visible area of ​​each of the at least two visible facades is obtained as the projection area of ​​the at least two visible facades.

5. The method according to claim 4, characterized in that, The step of presenting the annotation information of the target object on the annotation elevation includes: The marking area is determined from the visible area of ​​the marking facade; the marking area is the largest area among the regions of a first shape included in the visible area of ​​the marking facade. The annotation information of the target object is displayed in the annotation area of ​​the annotation facade presented on the display interface.

6. The method according to claim 5, characterized in that, Determining the labeling area from the visible area of ​​the labeling facade includes: Obtain the occlusion information of the marked facade, wherein the occlusion information is used to indicate the occluded vertices and occluded edges of the marked facade; Based on the occlusion information, the marking area is determined within the visible area of ​​the marking facade.

7. The method according to claim 6, characterized in that, The first shape is a rectangle, and the step of determining the annotation area in the visible area of ​​the annotation facade based on the occlusion information includes: When the occlusion information indicates that there is an occluded vertex on the marked elevation, the diagonal vertex of the occluded vertex is taken as the first target point; A first endpoint is determined on the non-adjacent edge corresponding to the first target point, such that the rectangle with the line segment between the first endpoint and the first target point as its diagonal is the rectangle with the largest area within the visible area of ​​the marked elevation. The area containing the rectangle whose diagonal is the line segment between the first endpoint and the first target point is defined as the labeled area.

8. The method according to claim 6, characterized in that, The first shape is a rectangle, and the step of determining the annotation area in the visible area of ​​the annotation facade based on the occlusion information includes: When the occlusion information indicates that there are two occluded vertices on the labeled elevation, and the edge between the two occluded vertices is completely occluded, the vertex with the largest sum of the lengths of the unoccluded parts of its adjacent edges among the unoccluded vertices on the labeled elevation is obtained as the second target point. A second endpoint is determined on the non-adjacent edge corresponding to the second target point; the second endpoint is located in the visible area of ​​the labeled facade, and the rectangle with the line segment between the second endpoint and the second target point as its diagonal is the rectangle with the largest area in the visible area of ​​the labeled facade. The area containing the rectangle whose diagonal is the line segment between the second endpoint and the second target point is defined as the labeled area.

9. The method according to claim 6, characterized in that, The first shape is a rectangle, and the step of determining the annotation area in the visible area of ​​the annotation facade based on the occlusion information includes: When the occlusion information indicates that there are two occluded vertices on the labeled facade and no completely occluded edges, a target point set is obtained; the target point set includes the unoccluded vertices of the labeled facade and the boundary points on the adjacent edges of the two occluded vertices; the boundary points are used to distinguish between the occluded and unoccluded areas of the visible facade. A third endpoint is determined within the visible area of ​​the marked facade; the rectangle with the line segment between the third endpoint and the third target point as its diagonal is the rectangle with the largest area within the visible area of ​​the marked facade; the third target point is one of the target points in the set. The area containing the rectangle whose diagonal is the line segment between the third endpoint and the third target point is defined as the labeled area.

10. The method according to claim 6, characterized in that, The first shape is a rectangle, and the step of determining the annotation area in the visible area of ​​the annotation facade based on the occlusion information includes: When the occlusion information indicates that there are three occluded vertices on the marked elevation, the unoccluded vertices on the marked elevation are obtained as the fourth target point; The area containing the rectangle formed by the fourth target point and the boundary points on its two adjacent sides is defined as the marked area; the boundary points are used to distinguish between the obscured and unobscured areas of the visible facade.

11. The method according to claim 5, characterized in that, The presentation of the annotation information of the target object on the annotation elevation of the target object displayed on the display interface includes: Based on the dimensions of the labeled area, a three-dimensional model of the labeled information is generated; A three-dimensional model of the annotation information is presented on a plane parallel to the annotation area displayed on the display interface; the parallel plane is a plane located in front of the annotation facade and parallel to the annotation facade.

12. The method according to claim 1, characterized in that, The step of obtaining the projection area of ​​the at least two visible facades based on their visible areas includes: Candidate labeling regions are determined from the visible areas of the at least two visible facades; the candidate labeling region is the region with the largest area among the regions of the second shape contained in the visible area of ​​the corresponding facade. The candidate annotation areas corresponding to each of the at least two visible facades are obtained as the projection areas of the at least two visible facades.

13. The method according to claim 12, characterized in that, The presentation of the annotation information of the target object on the annotation elevation of the target object displayed on the display interface includes: The annotation information of the target object is presented on the annotation surface of the display interface, corresponding to the candidate annotation area of ​​the annotation facade.

14. The method according to any one of claims 1 to 13, characterized in that, The annotation information of the target object is presented on the annotation elevation, including: When the area of ​​the visible region of the marked facade projected onto the display interface is greater than a specified area threshold, the marking information of the target object is displayed on the marked facade presented on the display interface.

15. A device for presenting object annotation information, characterized in that, The device includes: The target object acquisition unit is used to acquire a target object in a specified scene, wherein the specified scene is a scene presented at the target location; The device is used to determine at least two visible facades of the target object presented on the display interface, wherein the visible facades are the facades of the target object that are visible from the viewpoint corresponding to the target position. The projection area acquisition unit is used to acquire the projection area of ​​the at least two visible facades based on the visible areas of the at least two visible facades; the visible area is the area of ​​the corresponding visible facade that is visible from the viewpoint corresponding to the target position in the specified scene; The projection area acquisition unit is used to project the projection areas of the at least two visible facades onto the display interface to obtain the projection areas of the visible areas of the at least two visible facades presented on the display interface. The device is also used to select the labeled facade of the target object from the at least two visible facades based on the projection area presented on the display interface by the visible areas of the at least two visible facades respectively. The annotation information presentation unit is used to present the annotation information of the target object on the annotation facade, wherein the annotation facade of the target object where the annotation information is located changes with the change of the viewing angle caused by the change of the target position, and the distance of the annotation information presented on the display interface changes.

16. The apparatus according to claim 15, characterized in that, The specified scene is an augmented reality scene or a virtual reality scene presented at the target location.

17. The apparatus according to claim 15, characterized in that, The marked facade is the one with the largest projected area on the display interface among the at least two visible facades.

18. The apparatus according to claim 15, characterized in that, The region to be projected acquisition unit is used for, The entire visible area of ​​each of the at least two visible facades is obtained as the projection area of ​​the at least two visible facades.

19. The apparatus according to claim 17, characterized in that, The annotation information presentation unit includes: A region determination subunit is used to determine a labeling region from the visible area of ​​the labeling facade; the labeling region is the region with the largest area among the regions of a first shape included in the visible area of ​​the labeling facade. The annotation information presentation subunit is used to present the annotation information of the target object on the annotation elevation presented on the display interface.

20. The apparatus according to claim 19, characterized in that, The region is defined as a sub-unit, including: An occlusion information acquisition subunit is used to acquire occlusion information of the marked facade, wherein the occlusion information is used to indicate the occluded vertices and occluded edges of the marked facade; The annotation area determination subunit is used to determine the annotation area in the visible area of ​​the annotation facade based on the occlusion information.

21. The apparatus according to claim 20, characterized in that, The first shape is a rectangle, and the labeled area defines the sub-units for use in... When the occlusion information indicates that there is an occluded vertex on the marked elevation, the diagonal vertex of the occluded vertex is taken as the first target point; A first endpoint is determined on the non-adjacent edge corresponding to the first target point, such that the rectangle with the line segment between the first endpoint and the first target point as its diagonal is the rectangle with the largest area within the visible area of ​​the marked elevation. The area containing the rectangle whose diagonal is the line segment between the first endpoint and the first target point is defined as the labeled area.

22. The apparatus according to claim 20, characterized in that, The first shape is a rectangle, and the labeled area defines the sub-units for use in... When the occlusion information indicates that there are two occluded vertices on the labeled elevation, and the edge between the two occluded vertices is completely occluded, the vertex with the largest sum of the lengths of the unoccluded parts of its adjacent edges among the unoccluded vertices on the labeled elevation is obtained as the second target point. A second endpoint is determined on the non-adjacent edge corresponding to the second target point; the second endpoint is located in the visible area of ​​the labeled facade, and the rectangle with the line segment between the second endpoint and the second target point as its diagonal is the rectangle with the largest area in the visible area of ​​the labeled facade. The area containing the rectangle whose diagonal is the line segment between the second endpoint and the second target point is defined as the labeled area.

23. The apparatus according to claim 20, characterized in that, The first shape is a rectangle, and the labeled area defines the sub-units for use in... When the occlusion information indicates that there are two occluded vertices on the labeled facade and no completely occluded edges, a target point set is obtained; the target point set includes the unoccluded vertices of the labeled facade and the boundary points on the adjacent edges of the two occluded vertices; the boundary points are used to distinguish between the occluded and unoccluded areas of the visible facade. A third endpoint is determined within the visible area of ​​the marked facade; the rectangle with the line segment between the third endpoint and the third target point as its diagonal is the rectangle with the largest area within the visible area of ​​the marked facade; the third target point is one of the target points in the set. The area containing the rectangle whose diagonal is the line segment between the third endpoint and the third target point is defined as the labeled area.

24. The apparatus according to claim 20, characterized in that, The first shape is a rectangle, and the labeled area defines the sub-units for use in... When the occlusion information indicates that there are three occluded vertices on the marked elevation, the unoccluded vertices on the marked elevation are obtained as the fourth target point; The area containing the rectangle formed by the fourth target point and the boundary points on its two adjacent sides is defined as the marked area; the boundary points are used to distinguish between the obscured and unobscured areas of the visible facade.

25. The apparatus according to claim 19, characterized in that, The annotation information presentation sub-unit includes: The annotation information model generation subunit is used to generate a three-dimensional model of the annotation information based on the size of the annotation area; The annotation information model presentation subunit is used to present a three-dimensional model of the annotation information on a parallel plane of the annotation area presented on the display interface; the parallel plane is a plane located in front of the annotation facade and parallel to the annotation facade.

26. The apparatus according to claim 15, characterized in that, The region to be projected acquisition unit includes: A candidate annotation region determination subunit is used to determine candidate annotation regions from the visible regions of the at least two visible facades respectively; the candidate annotation region is the region with the largest area among the regions of a second shape contained in the visible region of the corresponding facade; The sub-unit for obtaining the projection area is used to obtain the candidate annotation areas corresponding to each of the at least two visible facades as the projection areas of the at least two visible facades.

27. The apparatus according to claim 26, characterized in that, The annotation information presentation unit is also used for, The annotation information of the target object is presented on the annotation surface of the display interface, corresponding to the candidate annotation area of ​​the annotation facade.

28. The apparatus according to any one of claims 15 to 27, characterized in that, The annotation information presentation unit is used for, When the area of ​​the visible region of the marked facade projected onto the display interface is greater than a specified area threshold, the marking information of the target object is presented on the marked facade displayed on the display interface.

29. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing computer instructions which are loaded and executed by the processor to implement the object annotation information presentation method as described in any one of claims 1 to 14.

30. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which is executed by a processor to implement the object annotation information presentation method as described in any one of claims 1 to 14.

Images