Space comment method and system of panoramic VR and readable storage medium

Abstract

The invention provides a panoramic VR space comment method and system and a readable storage medium, and the method comprises the steps: obtaining two-dimensional plane position information of a to-be-commented target in a VR panoramic picture, the two-dimensional plane position information comprising horizontal coordinate information and vertical coordinate information of the to-be-commented target; depth coordinate information of the to-be-commented target is obtained based on demand information of comment adding of the user, and the depth coordinate, the second coordinate and the third coordinate are perpendicular to each other; converting the two-dimensional plane position information of the to-be-commented target into three-dimensional position information based on the depth coordinate information, the horizontal coordinate information and the vertical coordinate information; and adding a comment to the to-be-commented target in the VR panoramic picture based on the three-dimensional position information of the to-be-commented target. According to the method, the comments are directly displayed in the panoramic space, so that the panoramic VR comments are more accurate and effective, and the interactivity is higher.

Description

Technical Field

[0001] The present invention belongs to the field of multimedia technology, and in particular to a panoramic VR space review method, system and readable storage medium. Background Art

[0002] Usually, content platforms have a comment function, where users can post their own comments on articles, videos, etc. There are two types of comments. One is to comment on the content as a whole, usually with a comment input box at the bottom of the article or video; the other is to comment on a certain part of the content, such as annotating a sentence in an article or sending a bullet comment on a certain moment in a video. Article annotations and video bullet comments can be understood as comments on a certain part of the content.

[0003] Panoramic VR resource creation platforms usually support opening panoramic VR content directly in a web browser without the need for special VR equipment, making it very cheap for the public to watch VR content. Because the presentation effect is three-dimensional, which is different from ordinary planes, popular comment methods such as annotations and bullet screens cannot be directly applied to panoramic VR content, making the feedback effect and interactivity of such content very poor.

[0004] For example, in a panoramic VR scene containing a statue, if the user wants to comment on a statue, he or she can directly select the location of the statue in the panoramic scene and enter the comment content. After completion, the comment will be displayed at the same location of the statue, and the author or other users can see at a glance that the comment is directed at the statue. Summary of the invention

[0005] The present invention provides a spatial comment method, system and readable storage medium for panoramic VR, which can at least solve the technical problem that three-dimensional space is different from ordinary plane, and popular comment methods such as annotations and barrages cannot be directly applied to panoramic VR content, resulting in poor feedback effect and interactivity of such content.

[0006] The technical solution provided by the present invention is as follows: On the one hand, a spatial review method for panoramic VR is provided, the method comprising: Acquire two-dimensional plane position information of the target to be commented on in the VR panoramic picture, wherein the two-dimensional plane position information includes abscissa information and ordinate information of the target to be commented on; Based on the user's requirement information for adding a comment, obtaining depth coordinate information of the target to be commented on, wherein the depth coordinate, the second coordinate, and the third coordinate are perpendicular to each other; Converting the two-dimensional plane position information of the target to be reviewed into three-dimensional position information based on the depth coordinate information, the abscissa information and the ordinate information; Add comments to the target to be commented on in the VR panoramic picture based on the three-dimensional position information of the target to be commented on.

[0007] In an optional implementation manner, obtaining the depth coordinate information of the target to be reviewed includes: Acquire the sliding value of the mouse wheel by the user at the user end, and determine the depth coordinate information based on the sliding value of the mouse wheel by the user at the user end; Get the two-dimensional plane position information of the target to be reviewed in the VR panoramic picture, including: Acquire the width of the user terminal display screen, and use the width of the user terminal display screen as the horizontal coordinate information; The length of the user terminal display screen is obtained, and the length of the user terminal display screen is used as the vertical coordinate information.

[0008] In an optional implementation, the method further includes: storing the comments added to the target to be commented on in a comment layer.

[0009] In an optional implementation manner, the comments added to the target to be commented on are assigned identifiers, one comment corresponds to one identifier, and each identifier is different.

[0010] In an optional implementation, the comments are stored in a specific location of the comment layer according to the identifier of each comment.

[0011] In an optional implementation, the comment includes text, image, emoticon or link.

[0012] In an optional implementation, the method further includes: obtaining a comment display request from a user terminal; The corresponding comments are displayed based on the comment display request.

[0013] In an optional implementation manner, the comment display request of the user terminal includes: Obtaining the identifier of a comment in the comment display request of the user end, and determining a display target related to the identifier of the comment based on the identifier of the comment; All comments related to the display target are determined and displayed based on the display target.

[0014] On the other hand, a panoramic VR space review system is provided, and the panoramic VR space review system includes: a memory storing execution instructions; and A processor, wherein the processor executes the execution instructions stored in the memory, so that the processor executes any one of the above-mentioned panoramic VR spatial review methods.

[0015] On the other hand, a readable storage medium is provided, wherein the readable storage medium stores execution instructions, and when the execution instructions are executed by a processor, the execution instructions are used to implement any of the above-mentioned panoramic VR spatial review methods.

[0016] The method provided by the embodiment of the present invention has at least the following beneficial effects: The method provided by the embodiment of the present invention is based on converting the two-dimensional plane position information of the target to be commented on in the panoramic VR picture into three-dimensional position information, and adding comments or annotations to the target to be commented on based on the three-dimensional position information of the target to be commented on, so that when watching the panoramic VR content, the user can comment on the specific objects in the VR panoramic picture, and the comments are directly displayed in the panoramic space, making the panoramic VR comments more accurate, effective and interactive. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0018] Figure 1 A schematic diagram of a spatial review method for panoramic VR is shown; Figure 2 A schematic diagram of the overall framework of a panoramic VR space review system is shown.

[0019] Figure 3 A schematic diagram showing a review data structure in a spatial review method for panoramic VR.

[0020] Figure 4 A detailed framework diagram of a panoramic VR spatial review system is shown. DETAILED DESCRIPTION

[0021] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0022] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0023] Panoramic VR presents a three-dimensional space content. The existing forms of annotation and bullet screen are not suitable for panoramic scenes. Based on this, an embodiment of the present invention provides a spatial comment method and system for panoramic VR.

[0024] See also Figure 1 , an embodiment of the present invention provides a panoramic VR space review method, the method comprising: step S101, step S102, step S103 and step S104.

[0025] Step S101: Acquire two-dimensional plane position information of the target to be commented on in the VR panoramic picture, wherein the two-dimensional plane position information includes horizontal coordinate information and vertical coordinate information of the target to be commented on.

[0026] Step S102: based on the user's comment adding requirement information, obtain depth coordinate information of the target to be commented on, wherein the first coordinate, the second coordinate and the third coordinate are perpendicular to each other.

[0027] Step S103: convert the two-dimensional plane position information of the target to be reviewed into three-dimensional position information based on the depth coordinate information, the abscissa information and the ordinate information.

[0028] Step S104: adding comments to the target to be commented on in the VR panoramic picture based on the three-dimensional position information of the target to be commented on.

[0029] In step S101, the two-dimensional plane position information of the target to be reviewed in the VR panoramic picture is obtained.

[0030] It is understandable that when using it, users can rotate the panoramic VR screen by dragging the mouse. They can drag in any 360 directions and rotate it to where they want to add comments. The WebGL technology is used to update the angles of the three spatial axes to achieve screen rotation. The screen can obtain the current angle of the spatial axis at any given moment.

[0031] Based on the user's request to comment on the target to be commented on in the VR panorama, the server obtains the two-dimensional plane position information of the target to be commented on in the VR panorama, and the two-dimensional plane position information includes the coordinate information of the target to be commented on, such as the horizontal coordinate and the vertical coordinate information.

[0032] In an optional embodiment, step S101 obtains the two-dimensional plane position information of the target to be commented on in the VR panoramic picture, including: obtaining the width of the user terminal display screen, and using the width of the user terminal display screen as the horizontal coordinate information.

[0033] Get the length of the user-side display screen and use the length of the user-side display screen as the vertical coordinate information.

[0034] In step S102, the depth coordinate information of the object to be reviewed is obtained, wherein the first coordinate, the second coordinate and the third coordinate are perpendicular to each other.

[0035] In an optional embodiment, obtaining the depth coordinate information, the horizontal coordinate information, and the vertical coordinate information of the target to be commented on includes: obtaining the sliding value of the mouse wheel by the user at the user end, and determining the depth coordinate information based on the sliding value of the mouse wheel by the user at the user end; It is understandable that in a VR panoramic screen, the screen can be rotated 360° and enlarged or reduced. The user can adjust the size of the screen as needed, which is reflected on the screen, that is, the depth position of the target to be commented on is adjusted. For example, when the screen is far from the user's line of sight, the coordinate value displayed by the depth coordinate information is larger, and when the screen is close to the user's line of sight, the coordinate value displayed by the depth coordinate information is smaller. Therefore, the distance between the target to be commented on and the user's line of sight can be adjusted by adjusting the depth coordinate information, which can also be understood as adjusting the size of the target to be commented on in the screen.

[0036] Step S103: convert the two-dimensional plane position information of the target to be reviewed into three-dimensional position information based on the depth coordinate information, the abscissa information and the ordinate information.

[0037] Exemplarily, the user clicks on the interface of the Web end to trigger adding a comment and enter the comment mode. In the comment mode, the plane coordinates of the mouse click can be converted to three-dimensional coordinates, and the comment input component is rendered at the calculated coordinate position.

[0038] Step S104: adding a comment to the target to be commented on based on the three-dimensional position information of the target to be commented on.

[0039] For example, the user can use the mouse to click on the user interface to add a comment. The comment position in the screen is the position where the mouse is clicked. The depth of the comment in space is adjusted by using the mouse wheel. The plane coordinates of the mouse on the screen, the value of the mouse wheel and the spatial axis angle are used as the three-dimensional position information of the target to be commented on.

[0040] Using the browser WebGL (‌Web Graphics Library, 3D drawing protocol) technology, the specific location to be rendered in the Web browser screen is determined according to the calculated three-dimensional position information of the target to be commented on, and the comment input component is rendered at this location.

[0041] The position of the comment can also be changed by dragging the comment icon, that is, the position of the comment can be adjusted as needed. For example, when the position of the comment blocks other images, the depth position information of the target to be commented on can be calculated according to the sliding depth of the mouse after dragging the comment icon, and the position information of the target to be commented on can be determined according to the width and length of the display screen, thereby achieving dynamic update of the comment position.

[0042] The method provided by the embodiment of the present invention further includes: storing the comments added to the target to be commented on in the comment layer.

[0043] After the user enters the comment content, the Web-side program organizes the user's comment data into a comment data structure (such as the attached Figure 2 as shown) to the server for storage.

[0044] In an optional implementation, the comments added to the review target are assigned identifiers, one comment corresponds to one identifier, and each identifier is different.

[0045] See also Figure 2 In the embodiment of the present invention, a globally unique identifier is assigned to each comment, that is, the identifier of each comment is different. Based on this, the accuracy of comment search can be improved when displaying comments.

[0046] In an optional implementation, the comments are stored in a specific location of the comment layer according to the identifier of each comment.

[0047] See also Figure 2 In an optional implementation, the comment includes text, image, emoticon or link.

[0048] In an optional implementation, the method further includes: obtaining a comment display request from a user terminal; Display the corresponding comments based on the comment display request.

[0049] It should be noted that, in the method provided by the embodiment of the present invention, the comments added by the user are stored in the comment layer, and the comments in the comment layer are hidden and will not be displayed when the user does not want to view them, so as not to block other images and affect the user experience. When the user wants to view a certain image, a request to display the comments can be sent to the server, and the server displays the corresponding comments based on the comment display request.

[0050] In an optional implementation, the comment display request of the user terminal includes: obtaining the identifier of the comment in the comment display request of the user terminal, determining a display target related to the identifier of the comment based on the identifier of the comment; and determining and displaying all comments related to the display target based on the display target.

[0051] It is understandable that each comment provided by the embodiment of the present invention has a globally unique identifier, and the representation of each comment includes a parent node and a child node associated with the parent node. For example, when a user only needs to view a comment, only the comment can be displayed. When the user wants to view the reply or more branch comments of the comment, the user can send a request to the server to view the reply or more branch comments of the comment, and the server displays the comment and its related replies or more branch comments based on the request. In this way, it can be avoided that too many comments affect the display of the image and affect the user experience.

[0052] As an example, a user can use a web browser to open a panoramic VR screen, load the complete VR scene, and trigger the pulling of comment data for the scene from the server.

[0053] Create a comment layer. All comment components are rendered in this comment layer. The display / hiding of all comments is uniformly controlled by showing and hiding the comment layer. Comments can be hidden when necessary to avoid blocking the scene.

[0054] The embodiment of the present invention uses WebGL to render the comment component at the corresponding position according to the three-dimensional coordinate information of the comment.

[0055] In an optional implementation, an embodiment of the present invention adds a user click event listener to the comment component. When the server receives a user click and expands the comment component to display detailed comment content, the comment can be added to the panoramic resource on the Web and its position can be set, and the comment can be displayed in the panoramic VR three-dimensional space.

[0056] On the other hand, an embodiment of the present invention provides a panoramic VR space review system, the panoramic VR space review system comprising: a memory, the memory storing execution instructions; and The processor executes the execution instructions stored in the memory, so that the processor executes any of the above-mentioned panoramic VR spatial review methods.

[0057] For further information, see Figure 3 and Figure 4 The comment system of the present invention may include a web terminal and a server terminal (the server of the present invention). The web terminal can add comments and display comments. The server terminal (server) can store comment data and query comments.

[0058] The user can rotate the panoramic VR screen by dragging the mouse on the web, enter the comment mode after obtaining the three-dimensional position information of the position to be commented on, set the position to add the comment and click Add Comment. The user can zoom in or out the size of the target to be commented on by scrolling the mouse wheel, that is, set the depth of the target to be commented on, and obtain the mouse coordinates (the horizontal and vertical coordinates of the mouse on the user side), and add comments by calculating the spatial coordinates (calculating the three-dimensional position information to be commented on) through webGL technology.

[0059] The added comments are stored in the comment database on the server side. When the comments need to be displayed, the server side obtains the comment data from the comment database through the comment query module for display.

[0060] On the other hand, a readable storage medium is provided, in which execution instructions are stored. When the execution instructions are executed by a processor, they are used to implement any of the above-mentioned panoramic VR spatial review methods.

[0061] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A spatial review method for panoramic VR, characterized in that: The method comprises: Acquire two-dimensional plane position information of the target to be commented on in the VR panoramic picture, wherein the two-dimensional plane position information includes abscissa information and ordinate information of the target to be commented on; Based on the user's requirement information for adding a comment, obtaining depth coordinate information of the target to be commented on, wherein the depth coordinate, the second coordinate, and the third coordinate are perpendicular to each other; Converting the two-dimensional plane position information of the target to be reviewed into three-dimensional position information based on the depth coordinate information, the abscissa information and the ordinate information; Add comments to the target to be commented on in the VR panoramic picture based on the three-dimensional position information of the target to be commented on.

2. The spatial review method of panoramic VR according to claim 1, characterized in that: Obtaining the depth coordinate information of the target to be reviewed includes: Acquire the sliding value of the mouse wheel by the user at the user end, and determine the depth coordinate information based on the sliding value of the mouse wheel by the user at the user end; Get the two-dimensional plane position information of the target to be reviewed in the VR panoramic picture, including: Acquire the width of the user terminal display screen, and use the width of the user terminal display screen as the horizontal coordinate information; The length of the user terminal display screen is obtained, and the length of the user terminal display screen is used as the vertical coordinate information.

3. The spatial review method of panoramic VR according to claim 1, characterized in that: Also includes: The comments added to the target to be commented are stored in the comment layer.

4. The method for spatial review of panoramic VR according to claim 3, characterized in that: The comments added to the target to be commented on are assigned identifiers, one comment corresponds to one identifier, and each identifier is different.

5. The method for spatial review of panoramic VR according to claim 4, characterized in that: The comments are stored in a specific location of the comment layer according to the identifier of each comment.

6. The method for spatial review of panoramic VR according to any one of claims 1 to 5, characterized in that: The comment includes text, images, emoticons or links.

7. The method for spatial review of panoramic VR according to claim 4, characterized in that: Also includes: Get the comment display request from the user; The corresponding comments are displayed based on the comment display request.

8. The method for spatial review of panoramic VR according to claim 7, characterized in that: The comment display request of the user terminal includes: Obtaining the identifier of a comment in the comment display request of the user end, and determining a display target related to the identifier of the comment based on the identifier of the comment; All comments related to the display target are determined and displayed based on the display target.

9. A panoramic VR space review system, characterized in that: The panoramic VR space review system includes: a memory storing execution instructions; and A processor, wherein the processor executes the execution instructions stored in the memory, so that the processor executes the spatial review method of the panoramic VR according to any one of claims 1 to 8.

10. A readable storage medium, characterized in that: The readable storage medium stores execution instructions, which are used to implement the panoramic VR spatial review method according to any one of claims 1 to 8 when executed by the processor.

Images