Video processing method, device, electronic device and storage medium

Abstract

The disclosed embodiments provide a video processing method, apparatus, electronic device, and storage medium. The method comprises: displaying a perspective window in the current scene in response to an instruction to display a perspective image; determining the perspective image to be displayed in the perspective window based on observation point information of a target object and a pre-created perspective scene; and displaying the perspective image in the current scene. The technical solution of the disclosed embodiments enables the effect of perspective rendering of another scene through the perspective window while the user is in the current scene, thereby improving the interactive experience between the user and the virtual reality device, and thereby enhancing the user's immersive and user experience.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of image processing technology, and in particular to a video processing method, device, electronic device, and storage medium. Background Art

[0002] With the rapid development of virtual reality (VR) technology, it has become a common form of leisure and entertainment for users to immersively experience the virtual world by wearing VR devices.

[0003] Generally, when a user wears a VR device, he or she can walk around in the virtual scene displayed in the display area or perform other entertainment activities.

[0004] However, existing VR devices have certain limitations when displaying scenes, and are unable to interact with multiple virtual scenes simultaneously in the same display area, affecting the user experience. Summary of the Invention

[0005] The present disclosure provides a video processing method, device, electronic device, and storage medium to achieve the effect of perspective rendering of another scene through a perspective window while the user is in the current scene, thereby improving the interactive experience between the user and the virtual reality device, and further enhancing the user's immersive experience and usage experience.

[0006] In a first aspect, an embodiment of the present disclosure provides a video processing method, the method comprising:

[0007] In response to an operation instruction to display a perspective image, display a perspective window in the current scene;

[0008] Determining a perspective image displayed in the perspective window based on observation point information of the target object and a pre-made perspective scene; wherein the perspective image is at least a portion of the perspective scene;

[0009] The perspective picture is displayed in the current scene.

[0010] In a second aspect, an embodiment of the present disclosure further provides a video processing device, the device comprising:

[0011] A perspective window display module, configured to display a perspective window in the current scene in response to an operation instruction to display a perspective image;

[0012] a perspective picture determination module, configured to determine a perspective picture displayed in the perspective window based on observation point information of the target object and a pre-made perspective scene; wherein the perspective picture is at least a portion of the perspective scene;

[0013] A perspective picture display module is used to display the perspective picture in the current scene.

[0014] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0015] one or more processors;

[0016] a storage device for storing one or more programs,

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the video processing method as described in any one of the embodiments of the present disclosure.

[0018] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to execute the video processing method as described in any one of the embodiments of the present disclosure.

[0019] The technical solution of the embodiment of the present disclosure displays a perspective window in the current scene in response to an operation instruction to display a perspective picture. Furthermore, based on the observation point information of the target object and a pre-made perspective scene, the perspective picture displayed in the perspective window is determined. Finally, the perspective picture is displayed in the current scene. This solves the problem that the existing technology cannot realize the interaction with multiple virtual scenes in the same display area at the same time. It realizes the effect of perspective rendering of another scene through the perspective window under the premise that the user is in the current scene, thereby improving the interactive experience between the user and the virtual reality device, and further enhancing the user's immersive experience and usage experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0021] Figure 1 This is a flow chart of a video processing method provided by an embodiment of the present disclosure;

[0022] Figure 2 This is a flow chart of a video processing method provided by an embodiment of the present disclosure;

[0023] Figure 3 This is a flow chart of a video processing method provided by an embodiment of the present disclosure;

[0024] Figure 4 This is a schematic diagram of a method for determining display location information provided by an embodiment of the present disclosure;

[0025] Figure 5 This is a flow chart of a video processing method provided by an embodiment of the present disclosure;

[0026] Figure 6 This is a flow chart of a video processing method provided by an embodiment of the present disclosure;

[0027] Figure 7 is a schematic structural diagram of a video processing device provided by an embodiment of the present disclosure;

[0028] Figure 8 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0030] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0031] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0032] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0033] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0034] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0035] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0036] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0037] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0038] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0039] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0040] Before introducing the present technical solution, an example of an application scenario can be first described. The disclosed embodiment can be applied to a scene in which a user observes any picture in another virtual scene through a virtual window in a VR three-dimensional virtual scene. For example, a user wears a virtual reality device to move in the current virtual scene. When a user's operation instruction for displaying a perspective picture is detected, a perspective window for displaying the perspective picture can be determined in the current virtual scene. Furthermore, the observation point information when the user observes the perspective picture can be determined, so as to render the perspective picture in the perspective window based on the observation point information. Furthermore, it can be determined that the perspective picture in the perspective scene is viewed through the perspective window. Thus, the perspective picture can be displayed in the current scene, achieving the effect of perspective rendering of another scene through the perspective window under the premise that the user is in the current scene. At the same time, the effect of rendering and displaying two virtual scenes in the same display area is achieved.

[0041] Figure 1 This is a flow chart of a video processing method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to a situation where a user is in a current scene and observes another scene through a perspective window. The method can be executed by a video processing device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, PC or server, etc.

[0042] like Figure 1 As shown, the method includes:

[0043] S110 : In response to an operation instruction to display a perspective image, a perspective window is displayed in the current scene.

[0044] The apparatus for executing the video processing method provided in the embodiments of the present disclosure may be integrated into application software supporting video processing functions, and the software may be installed in an electronic device. Optionally, the electronic device may be a virtual reality (VR) device or other device. The application software may be a type of software for image / video processing, and its specific application software will not be described in detail here, as long as it can implement image / video processing.

[0045] In this embodiment, in an application software or program that supports video processing functions, a control for displaying a perspective image can be pre-developed, or a preset gesture action for displaying a perspective image can be pre-set. When it is detected that the user triggers the control, or when it is detected that the user's body movement information is consistent with the preset body movement, the operation instruction can be responded to, thereby displaying a perspective window in the current scene. In actual application, when a user wears a virtual reality device, the user's body movement information can be collected based on the handle, and when it is detected that the collected body movement information is consistent with the preset body movement, it can be determined that the operation instruction for displaying the perspective image has been triggered and a response is made to it; or, when a user wears a virtual reality device, the user's posture movement information, such as body movement information, can be collected based on a binocular camera set at the front end of the VR glasses. When it is detected that the collected body movement information is consistent with the preset body movement, it can be determined that the operation instruction for displaying the perspective image has been triggered and a response is made to it.

[0046] The perspective view can be a view obscured by the current scene and visible only through a window within a preset area. In practical applications, a user wears a virtual reality device, and the display area of the virtual reality device displays the current scene. When an instruction to display the perspective view is detected, a window within a preset area can be demarcated from the current scene and enabled to display the perspective view. This window then displays the view obscured by the current scene, which can be used as the perspective view. Accordingly, the current scene can be the virtual scene currently being viewed by the user wearing the virtual reality device, that is, the scene displayed within the display area of the virtual reality device when the user wears the virtual reality device. The current scene can be a three-dimensional scene. It should be noted that the current scene can be pre-created and integrated into the virtual reality device. The perspective window can be a window that enables viewing the perspective view from the current scene. It should be noted that the perspective window can be of any shape and size, and can optionally be circular, directional, elliptical, or a user-defined shape.

[0047] In actual applications, when a user wears a virtual reality device and the current scene is displayed in the display area of the virtual reality device, if it is detected that the user has triggered an operation instruction to display a perspective picture, a perspective window can be determined and displayed in any area in the current scene.

[0048] S120: Determine a perspective picture displayed in the perspective window based on the observation point information of the target object and a pre-made perspective scene.

[0049] The target object may be an object observing the perspective image. The target object may be any object, and optionally, may be a user. The observation point information may be the observation position information corresponding to the user's eyes when the user observes the perspective image through their eyes based on their own needs. In actual applications, when a trigger operation based on the perspective image input is detected, the perspective image will move or rotate accordingly based on the trigger operation to obtain the perspective image observation position required by the user. At this time, the information corresponding to the observation position can be used as the observation point information. Furthermore, based on the observation point information and a pre-made perspective scene, the perspective image displayed in the perspective window can be determined.

[0050] The perspective scene can be a pre-made scene that can be observed through the perspective window in the current scene, that is, any scene other than the current scene. The perspective scene can also be a three-dimensional scene. In actual applications, two scenes can be pre-made and integrated into the virtual reality device. When the user wears the virtual reality device, the scene displayed in the display area is the current scene. Furthermore, when an instruction to display a perspective picture is detected, the scene that can be observed through the perspective window is the perspective scene. Then, based on the observation point information and the perspective scene, the perspective picture displayed in the perspective window can be determined.

[0051] The perspective image is at least a portion of a perspective scene. That is, the image observed through the perspective window is a portion of or the entire perspective scene. In this embodiment, the perspective image observed by the user varies depending on the observation point information. That is, the rendering angle of the perspective scene varies depending on the observation point information. In actual applications, the perspective image displayed in the perspective window can be adjusted to represent the perspective scene at different rendering angles by changing the observation point information.

[0052] In actual applications, after determining the perspective window and displaying it in the current scene, the observation point information can be determined and the pre-made perspective scene can be retrieved from the memory space. Then, the rendering angle of the perspective scene in the perspective window can be determined based on the observation point information, thereby determining the perspective picture displayed in the perspective window.

[0053] S130: Display the perspective image in the current scene.

[0054] In this embodiment, after determining the perspective picture displayed in the perspective window, the determined perspective picture can be displayed in the perspective window in the current scene, so that another scene can be observed from the current scene through the perspective window, that is, the effect of perspective scene.

[0055] The technical solution of the embodiment of the present disclosure displays a perspective window in the current scene in response to an operation instruction to display a perspective picture. Furthermore, based on the observation point information of the target object and a pre-made perspective scene, the perspective picture displayed in the perspective window is determined. Finally, the perspective picture is displayed in the current scene. This solves the problem that the existing technology cannot realize the interaction with multiple virtual scenes in the same display area at the same time. It realizes the effect of perspective rendering of another scene through the perspective window under the premise that the user is in the current scene, thereby improving the interactive experience between the user and the virtual reality device, and further enhancing the user's immersive experience and usage experience.

[0056] Figure 2This is a flow chart of a video processing method provided by an embodiment of the present disclosure. Based on the aforementioned embodiment, the display position information of the see-through window can be first determined, and then the see-through window can be displayed in the current scene based on the display position information. For specific implementations, please refer to the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the aforementioned embodiments are not repeated here.

[0057] like Figure 2 As shown, the method specifically includes the following steps:

[0058] S210 : In response to an operation instruction for displaying a perspective image, determining display position information of the perspective window in the current scene, and displaying the perspective window based on the display position information.

[0059] The display position information may be information representing the position corresponding to the perspective window when displayed in the current scene. Optionally, the display position information may be determined based on pre-set parameter information or based on a user triggering operation in the current scene, which is not specifically limited in the present embodiment.

[0060] In actual applications, when it is detected that the user has triggered an operation instruction to display a perspective picture in the current scene, the operation instruction can be responded to. The display position information of the perspective window used to display the perspective picture in the current scene can be first determined, and then the perspective window can be displayed based on the display position information, so that the perspective window can be displayed in the current scene.

[0061] S220: Display a perspective window in the current scene.

[0062] S230: Determine a perspective picture displayed in the perspective window based on the observation point information of the target object and a pre-made perspective scene.

[0063] S240: Display the perspective image in the current scene.

[0064] The technical solution of the embodiment of the present disclosure determines the display position information of the perspective window in the current scene in response to an operation instruction to display the perspective picture, displays the perspective window based on the display position information, displays the perspective window in the current scene, further determines the perspective picture displayed in the perspective window based on the observation point information of the target object and the pre-made perspective scene, and finally displays the perspective picture in the current scene, which solves the problem that the existing technology cannot realize interaction with multiple virtual scenes in the same display area at the same time, realizes the dynamic construction effect of the perspective window, and further, realizes the effect of perspective rendering of another scene through the perspective window under the premise that the user is in the current scene, thereby enhancing the user's immersive experience and usage experience.

[0065] Figure 3 This is a flow chart of a video processing method provided by an embodiment of the present disclosure. Building on the previous embodiment, the display position information can be determined based on the pre-set motion trajectory of the auxiliary window and the preset position information of the perspective scene. For specific implementations, please refer to the technical solution of this embodiment. Technical terms that are identical or corresponding to those in the previous embodiment are not repeated here.

[0066] like Figure 3 As shown, the method specifically includes the following steps:

[0067] S310 , in response to an operation instruction to display a perspective image, determine display position information of the perspective window based on a preset motion trajectory of the auxiliary window and set position information of the perspective scene, so as to display the perspective window based on the display position information.

[0068] The auxiliary window can be a window corresponding to the perspective scene. That is, when the conditions for displaying a perspective image are met, the corresponding perspective scene can be observed through this window. This window can also be used to determine the rendering size of the perspective scene in the display interface. The auxiliary window can be a three-dimensional window of any shape and layout, and can optionally be a sphere. In this embodiment, the motion trajectory can be the trajectory corresponding to the auxiliary window's shape changes within the current scene. For example, the motion trajectory can be the trajectory corresponding to the auxiliary window's size expansion or contraction, or the trajectory corresponding to the auxiliary window's movement from a far distance to a near distance or from a near distance to a far distance from the user's observation point. The set position information can be pre-set information representing the location of the perspective scene. For example, if the perspective scene is a spherical scene, the set position information can be the center of the sphere. During the actual development phase, to determine the display position information of the perspective window, an auxiliary window can be introduced and a pre-set motion trajectory for the auxiliary window can be set. This allows the auxiliary window to intersect with the perspective scene as it moves along the motion trajectory, resulting in multiple intersection points. The display position information of the perspective window can then be determined based on these intersection points.

[0069] In practical applications, when determining the display position information of the perspective window, an auxiliary window may be introduced and the motion trajectory of the auxiliary window may be pre-set to determine the display position information based on the set motion trajectory and the set position information of the perspective scene.

[0070] Optionally, based on the pre-set auxiliary window motion trajectory and the set position information of the perspective scene, the display position information of the perspective window is determined, including: determining the center point position information of the auxiliary window during the movement of the auxiliary window according to the motion trajectory; determining the display position information of the perspective window based on the center point position information and the set position information.

[0071] The center point position information may be the spatial position information of the center point of the auxiliary window in space. For example, if the auxiliary window is a sphere, the center point position information may be the spatial position information of the center of the sphere in space.

[0072] It should be noted that the display position information changes with the movement trajectory. Therefore, when determining the display position information of the perspective window, the auxiliary window corresponding to one of the trajectory points in the movement trajectory may be used as an example for explanation.

[0073] In actual applications, the motion trajectory of the auxiliary window can first be set so that the auxiliary window can intersect with the perspective scene when it moves based on the motion trajectory. Then, the center point position information of the auxiliary window can be determined based on the motion trajectory when the auxiliary window intersects with the perspective scene. Further, the set position information of the perspective scene is obtained to determine the intersection of the perspective scene and the auxiliary window based on the set position information and the center point position information of the auxiliary window. Then, the display position information of the perspective window can be determined by analyzing the intersection. The advantage of this setting is that the display position information of the perspective window can be determined based on the intersection of the auxiliary window and the perspective scene, making the determination method of the perspective window more diverse. In addition, the relative distance between the perspective scene and the auxiliary window can be controlled to achieve the effect of dynamic change of the size and position of the perspective window.

[0074] In practical applications, the perspective window can correspond to various shapes, optionally, a sphere, a cuboid, or an ellipsoid. The following describes the process of determining the perspective window display position information using a sphere as an example.

[0075] Optionally, the display position information of the perspective window is determined based on the center point position information and the set position information, including: determining the intersection information of the auxiliary window and the perspective scene to which the set position information belongs based on the center point position information and the set position information; and determining the center coordinates and display radius in the display position information based on the intersection information, the center point position information and the set position information.

[0076] The intersection information may be the spatial position information of the intersection point obtained when the auxiliary window intersects the perspective scene to which the set position information belongs. In practical applications, after determining the center point position information and the set position information, the position of the auxiliary window in space can be determined based on the center point position information, and the position of the perspective scene in space can be determined based on the set position information. Furthermore, the intersection between the auxiliary window and the perspective scene can be determined, and based on this intersection, the intersection information of multiple intersection points between the auxiliary window and the perspective scene can be determined.

[0077] In this embodiment, after obtaining the intersection information of multiple intersection points between the auxiliary window and the perspective scene, the center coordinates and display radius in the display position information can be determined based on the multiple intersection information, the center point position information, and the set position information. The center coordinates may be the coordinates corresponding to the center point position of the perspective window. The display radius may be the radius corresponding to the perspective window.

[0078] In actual applications, when the perspective window corresponds to a sphere, when determining the display position information of the perspective window, the auxiliary window and the perspective scene can be projected onto a two-dimensional plane to obtain two intersecting two-dimensional figures. Furthermore, a triangle can be constructed based on any one of the multiple intersection points, the center point corresponding to the auxiliary window, and the set position point corresponding to the perspective scene, wherein the side lengths of the triangle correspond to the radius of the auxiliary window, the radius of the perspective scene, and the distance between the center point of the auxiliary window and the set position point of the perspective scene, respectively. Then, starting from the intersection point in the constructed triangle, a perpendicular line is drawn to the side of the triangle opposite to the intersection point to obtain the foot of the perpendicular. At this time, the foot of the perpendicular can be used as the center point in the display position information, and the distance corresponding to the perpendicular line can be used as the display radius in the display position information. The advantage of this setting is that the display position information of the perspective window can be determined more accurately, thereby improving the rendering efficiency of the perspective window.

[0079] For example, it can be combined with Figure 4 The above determination method is explained as follows: Take the perspective scene and the auxiliary window as an example, both of which are spheres. Then, project the perspective scene and the auxiliary window into a two-dimensional plane to obtain two intersecting circles. Among them, O1 and R1 correspond to the perspective scene, which are the center and radius of the perspective scene respectively; O2 and R2 correspond to the auxiliary window, which are the center and radius of the auxiliary window respectively; Q and P are the intersection points between the auxiliary window and the perspective scene; the coordinates corresponding to point c are the coordinates of the center point of the perspective window; and r is the display radius of the perspective window. Based on Figure 4 The figure shown in and combined with geometric knowledge can be obtained:

[0080] Let x = |O1-c|, y = |c-O2|, d = |O1-O2|, where || represents the modulus of the corresponding vector;

[0081] Furthermore, by arranging the above three formulas, we can obtain: x+y=d;

[0082] And, by Figure 4 It can be seen that r is the common side of the two right triangles, so we can get:

[0083] R1*R1-x*x=R2*R2-y*y;

[0084] After arranging the above two formulas, we can get:

[0085] x=((R1*R1-R2*R2) / d+d)*0.5, y=dx

[0086] Then, the coordinates of the display point c and r can be determined based on the following formula:

[0087] r=sqrt(R1*R1-x*x); c=O1+normalize(O2-O1)*x

[0088] Among them, sqrt() is the square root function, normalize(O2-O1) is the normalized direction vector from O1 to O2, and * represents the product.

[0089] It should be noted that based on the above steps, the center point coordinates and display radius of the perspective window when it corresponds to a sphere can be determined. Therefore, when creating a three-dimensional model corresponding to the perspective window, the nodes of each grid model can be determined based on the determined center point coordinates and display radius, and then, a three-dimensional model corresponding to the perspective window can be created. However, when determining the display position information of the perspective window, an auxiliary window is introduced. Therefore, the auxiliary window can be used as a proxy geometry of the perspective window for model creation to obtain the model corresponding to the perspective window. The advantage of this setting is that there is no need to add a new grid model to the scene, which reduces memory usage and thus improves the rendering efficiency of the perspective window.

[0090] In a specific implementation, when the number of intersections between the auxiliary window and the perspective scene is two, the distances between each intersection and the center point of the auxiliary window can be determined respectively, which are the first distance to be processed and the second distance to be processed. Then, the distance between the center point of the auxiliary window and the set position point of the perspective scene is determined, which is the center distance. Further, the cosine value between the first distance to be processed and the center distance is determined to obtain the first cosine value, and the cosine value between the second distance to be processed and the center distance is determined to obtain the second cosine value. At this time, the auxiliary window is divided based on the first cosine value and the second cosine value, and the geometric bodies within the cosine value range constructed based on the first cosine value and the second cosine value are retained, and this part of the geometric bodies is used as the geometric bodies corresponding to the perspective window.

[0091] For example, continue to refer to Figure 4 As shown, first calculate the direction vector between the intersection point P and O2 Direction vector between intersection point Q and O2 and the direction vector between O1 and O2 Further, calculation and The cosine value of the angle between the two is used to obtain the first cosine value. At the same time, calculate and The cosine value of the angle between the two is obtained to obtain the second cosine value. Thus, the auxiliary window can be divided based on the first cosine value and the second cosine value, and the geometric body within the cosine value range constructed based on the first cosine value and the second cosine value is retained to obtain the perspective window.

[0092] S320: Display a perspective window in the current scene.

[0093] S330: Determine the perspective picture displayed in the perspective window based on the observation point information of the target object and the pre-made perspective scene.

[0094] S340: Display the perspective image in the current scene.

[0095] The technical solution of the embodiment of the present disclosure, in response to an operation instruction to display a perspective picture, determines the display position information of the perspective window based on the pre-set motion trajectory of the auxiliary window and the set position information of the perspective scene, displays the perspective window based on the display position information, and displays the perspective window in the current scene. Furthermore, based on the observation point information of the target object and the pre-made perspective scene, the perspective picture displayed in the perspective window is determined. Finally, the perspective picture is displayed in the current scene, which solves the problem that the existing technology cannot realize interaction with multiple virtual scenes in the same display area at the same time, and realizes the effect of perspective rendering of another scene through the perspective window under the premise that the user is in the current scene. At the same time, by controlling the relative distance between the perspective scene and the auxiliary window, the effect of dynamic change of the size and position of the perspective window is realized, thereby improving the user experience.

[0096] Figure 5 This is a flow chart of a video processing method provided by an embodiment of the present disclosure. Building on the previous embodiment, the display position information of the perspective window can be determined by manipulating a user's drawing trajectory within the current scene. For specific implementations, please refer to the technical solution of this embodiment. Technical terms that are identical or corresponding to those in the previous embodiment are not repeated here.

[0097] like Figure 5 As shown, the method specifically includes the following steps:

[0098] S410 . In response to an operation instruction for displaying a perspective image, obtain a drawing trajectory of the operating user in the current scene, and determine display position information of the perspective window based on the drawing trajectory, so as to display the perspective window based on the display position information.

[0099] Among them, the operating user can be a user who wears and operates the virtual reality device. The drawing trajectory can be a trajectory formed by swiping the contact point in the current scene based on the input device or the user's finger. In actual applications, when drawing the corresponding drawing trajectory based on the current scene in the display area, any drawing point can be selected as the starting point of the drawing trajectory, and then, based on the input device or the user's finger, the drawing point is controlled to move in the display interface. When it is detected that the drawing point stays in any area for a preset time, the position of the drawing point at this time can be used as the end point of the drawing trajectory. At this time, the trajectory between the starting point and the end point is the drawing trajectory. Optionally, the input device can be a handle in the virtual reality device.

[0100] In actual application, there are at least two ways to determine the drawing trajectory of the operating user in the current scene. The following will respectively describe these two determination methods and the specific process of determining the display position information of the perspective window based on the drawing trajectory.

[0101] One method may be: obtaining the drawing trajectory of the operating user in the current scene, and determining the display position information of the perspective window based on the drawing trajectory, including: obtaining the gesture movement information of the operating user in the current scene based on a gesture recognition algorithm, and determining the drawing trajectory based on the gesture movement information; sampling the drawing trajectory according to a preset time sampling interval to obtain multiple sampling points; fitting the multiple sampling points based on a least squares fitting algorithm to obtain the center coordinates and display radius in the display position information.

[0102] The gesture recognition algorithm can be an algorithm that tracks and locates user gestures to determine the spatial position of the user's hand in each video frame. In this embodiment, when a change in the posture information of the operating user in the current scene is detected, multiple images of the operating user can be captured based on an image capture device pre-installed on the virtual reality device. Furthermore, these images can be pre-processed based on a gesture recognition algorithm pre-integrated in the virtual reality device to obtain segmented images including the operating user's gestures. Gesture features are then extracted from these segmented images to determine position change information of key points of the hand based on the extracted gesture features. Furthermore, based on this position change information, the gesture motion information of the operating user in the current scene can be determined. The gesture motion information can be information that characterizes the movement of the user's gesture in the current scene. For example, the gesture motion information can be information corresponding to the operating user drawing a closed shape with their fingers in the current scene.

[0103] In actual applications, when a change in the user's gesture information in the current scene is detected, the user's hand feature points can be tracked and detected based on a gesture recognition algorithm to determine the position change information of the hand feature points. Furthermore, based on the position change information, the user's gesture motion information in the current scene can be determined, and thus, the drawing trajectory can be determined based on the gesture motion information. For example, if the gesture motion information can be information corresponding to the user drawing a closed shape with their finger in the current scene, the corresponding drawing trajectory can be the trajectory corresponding to the closed shape.

[0104] Furthermore, after the drawing trajectory is obtained, the drawing trajectory may be sampled according to a preset time sampling interval to obtain a plurality of sampling points.

[0105] The preset time sampling interval can be pre-set and used to calculate the time difference between two samples when sampling the drawn trajectory. In practical applications, after obtaining the drawn trajectory, the spatial position information of each trajectory point included in the drawn trajectory can be sampled according to the preset time sampling interval. When the newly sampled sampling point approximately coincides with the first sampling point, the sampling process can be determined to be complete. At this point, multiple sampling points can be obtained. The trajectory points can include sampling points.

[0106] Furthermore, after obtaining the plurality of sampling points, a fitting process may be performed on the plurality of sampling points based on a least square fitting algorithm, thereby obtaining the center coordinates and the display radius in the display position information.

[0107] Those skilled in the art will appreciate that the least squares fitting algorithm is a mathematical optimization technique. It finds the best function matching the data by minimizing the sum of squared errors. Least squares can be used to easily obtain unknown data and minimize the sum of squared errors between the obtained data and the actual data. The least squares fitting algorithm can also be used for curve fitting, i.e., determining a curve such that all data points are within a certain range above or below the curve.

[0108] In this embodiment, the center coordinates may be coordinates corresponding to the center position of the perspective window, and the display radius may be the radius of the perspective window.

[0109] In practical applications, after obtaining multiple sampling points, these sampling points can be fitted based on a least squares fitting algorithm to obtain a fitting curve. Further, this fitting curve is analyzed to obtain the center coordinates and display radius in the display position information. For example, the fitting curve can be a fitting circle.

[0110] Another method may be: obtaining the drawing trajectory of the operating user in the current scene, and determining the display position information of the perspective window based on the drawing trajectory, including: obtaining the drawing trajectory of the target handle in space; sampling the drawing trajectory according to a preset time sampling interval to obtain multiple sampling points; fitting the multiple sampling points based on the least squares fitting algorithm to obtain the center coordinates and display radius in the display position information.

[0111] The target handle may be a handle that the operating user holds and uses to perform trajectory drawing actions in space.

[0112] In actual applications, when the operating user wears a virtual reality device, the rotation angle and movement displacement of multiple monitoring parts of the target object, such as the handle holding part, can be monitored based on the virtual reality device. When it is detected that the operating user holds any handle and makes a trajectory drawing action in space, the handle can be used as the target handle, and the spatial position information of the target handle in each frame can be collected. When it is detected that the target handle no longer moves, or the target handle stays in any area in space for a preset time, a drawing trajectory can be generated based on the collected spatial position information. Then, the drawing trajectory can be sampled and fitted to obtain the center coordinates and display radius in the display position information.

[0113] It should be noted that the benefits of determining the display location information based on the above two methods are: improving the interactivity between the user and the virtual reality device, and at the same time, improving the intelligence of the virtual reality device, thereby improving the user experience.

[0114] S420: Display a perspective window in the current scene.

[0115] S430: Determine the perspective picture displayed in the perspective window based on the observation point information of the target object and the pre-made perspective scene.

[0116] S440: Display the perspective image in the current scene.

[0117] The technical solution of the embodiment of the present disclosure obtains the drawing trajectory of the operating user in the current scene in response to an operation instruction to display a perspective picture, and determines the display position information of the perspective window based on the drawing trajectory, so as to display the perspective window based on the display position information and display the perspective window in the current scene. Furthermore, based on the observation point information of the target object and the pre-made perspective scene, the perspective picture displayed in the perspective window is determined. Finally, the perspective picture is displayed in the current scene, which solves the problem that the existing technology cannot realize the interaction with multiple virtual scenes in the same display area at the same time. It realizes the effect of perspective rendering of another scene through the perspective window under the premise that the user is in the current scene, thereby improving the interactive experience between the user and the virtual reality device, and further enhancing the user's immersive experience and usage experience.

[0118] Figure 6 This is a flow chart of a video processing method provided by an embodiment of the present disclosure. Building on the previous embodiment, observation point information can be obtained to determine the perspective image of a perspective scene viewed through a perspective window based on this observation point information. For specific implementations, please refer to the technical solution of this embodiment. Technical terms that are identical or corresponding to those in the previous embodiment are not repeated here.

[0119] like Figure 6 As shown, the method specifically includes the following steps:

[0120] S510: In response to an operation instruction to display a perspective image, a perspective window is displayed in the current scene.

[0121] S520. Obtain observation point information.

[0122] In actual applications, when a user's triggering operation on the perspective window is detected, so that the perspective window moves and rotates accordingly according to the triggering operation, or when it is detected that the observation angle of the VR glasses in the virtual reality device worn by the user changes, the spatial position information of the observation point corresponding to each change can be collected, so as to obtain observation point information that meets the user's needs.

[0123] It should be noted that when the perspective window moves and rotates accordingly based on the trigger operation, or when the observation angle of the VR glasses in the virtual reality device worn by the user changes, it may correspond to multiple observation points and obtain observation point information of multiple observation points. For the convenience of explanation, the observation point information of one of the observation points can be used as an example to explain the subsequent steps.

[0124] S530: Based on the observation point information, determine a perspective picture in the perspective scene to be viewed through the perspective window.

[0125] In this embodiment, after obtaining the observation point information, the rendering viewing angle of the perspective scene when it is rendered under the observation point information can be determined, and further, the perspective picture in the perspective scene can be determined.

[0126] It should be noted that in order to realize viewing the perspective picture in the perspective scene from the current scene through the perspective window, when rendering the perspective picture, the entire perspective scene will not be rendered, only the perspective picture corresponding to the perspective window will be rendered.

[0127] Based on this, before determining the perspective picture in the perspective scene to be viewed through the perspective window based on the observation point information, it also includes: making a perspective scene layer and a mask layer to display the perspective picture based on the perspective scene layer, the mask layer and the perspective window.

[0128] In this embodiment, the perspective scene layer can be a layer corresponding to the perspective scene, that is, a layer created based on the perspective scene. The mask layer can be a layer corresponding to the mask body, that is, a layer created based on the mask body. The mask body can be a three-dimensional model of any shape, optionally a sphere. It should be noted that the mask body can cover all information in the perspective scene except the perspective window. Accordingly, the size of the mask layer will also be larger than that of the perspective scene layer.

[0129] During the actual development stage, a perspective scene layer and a mask layer can be created in the current scene, and the size of the mask layer should be much larger than the perspective scene layer. Furthermore, in the actual application process, after determining the perspective window, the perspective picture can be rendered based on the perspective scene layer, the mask layer and the perspective window to obtain the perspective picture in the perspective window.

[0130] In practical applications, when determining to view the perspective picture in the perspective scene through the perspective window, the perspective picture and the mask layer can be rendered simultaneously, thereby achieving the effect of viewing the perspective picture only in the perspective window.

[0131] Optionally, based on the observation point information, determining the perspective picture in the perspective scene viewed through the perspective window includes: determining the rendering angle of the perspective scene according to the observation point information; rendering the mask layer and the perspective picture based on the rendering angle, a preset reference pixel value and a preset depth value; and displaying the mask layer and the perspective picture in the perspective scene layer.

[0132] The rendering angle can be the angle of the perspective scene presented in the perspective window, or can be understood as the angle corresponding to the perspective scene when facing the operating user. The preset reference pixel value can be a pre-set reference pixel value used to perform the template test step in the rendering process. The preset reference pixel value can be any pixel value between 0 and 255. The preset depth value can be a pre-set depth value used to perform the depth test step in the rendering process.

[0133] In actual applications, after obtaining the observation point information, the observation angle of the operating user for the perspective window can be determined, and then, based on the observation angle, the rendering angle with respect to the perspective scene can be determined. Thus, the mask layer and the perspective picture can be rendered according to the rendering angle, the preset reference pixel value and the preset depth value.

[0134] It should be noted that when rendering, the corresponding rendering order may be to render the perspective image first and then the mask layer. The advantage of this setting is that it can ensure that the template writing process of the perspective image is before the template writing process of the mask layer.

[0135] In practical applications, when rendering a perspective image, the rendering can be performed by a graphics processing unit (GPU). During the rendering process, the rendering process of the perspective image can be implemented by setting relevant parameters. Specifically, first, the template test function module in the rendering pipeline is enabled, the template test function is set to GL_ALWAYS (always pass the test), and at the same time, a preset reference pixel value is written to the template buffer. Among them, GL_ALWAYS means that regardless of whether the pixel value of the pixel point is equal to the preset reference pixel value, the pixel point can be rendered. Further, the depth test function module in the rendering pipeline is enabled to perform a depth test. Based on the observation angle, the depth value of each pixel point in the perspective image to the camera is determined. Each depth value is compared with the preset depth value. Based on the comparison result, the perspective image is rendered, thereby ensuring that the occlusion relationship between each object in the rendered perspective image is correct. At the same time, the depth write function is disabled so that the depth value of the object that completes the rendering process first cannot be written to the depth buffer, ensuring that the depth value is always compared with the preset depth value during the rendering process. Finally, turn off the color writing function of the perspective window to ensure that the perspective window model itself will not be rendered into the final displayed image.

[0136] In actual applications, when rendering the mask layer, it can also be rendered through the Graphics Processing Unit (GPU). During the rendering process, the rendering process of the mask layer can be realized by setting relevant parameters. Specifically, first, turn off the back face culling function; wherein, back face culling (Face Culling) can be understood as when rendering any three-dimensional model, for the faces that the user cannot observe, there is no need to render, and only the planes that the user can observe are rendered. Then, turn on the template test function module in the rendering pipeline, and set the template test function to GL_NOY_ALWAYS (not equal to the preset reference pixel value to pass the test). The corresponding meaning is that when the pixel value of any pixel point in the mask layer is not equal to the preset reference pixel value, the pixel point can be rendered, and the preset reference pixel value applied at this time is the same as the preset reference pixel value corresponding to the perspective picture. It should be noted that the benefit of configuring the template test parameters in this way is that the template test of the mask layer will fail in the pixel area covered by the perspective picture. Therefore, it will not enter the subsequent depth writing link, and will not affect the rendering of the perspective picture, thereby achieving the effect of digging perspective. Furthermore, the depth test function module in the rendering pipeline is turned on to perform a depth test. Based on the observation angle, the depth value of each pixel in the mask layer to the camera is determined, and each depth value is compared with the preset depth value. Based on the comparison result, the mask layer is rendered, thereby ensuring that the rendered occlusion relationship is correct. At the same time, the depth writing function is turned on to make the depth test of the perspective scene with a larger depth value fail, thereby achieving the hiding effect of the perspective scene. Finally, the color writing function of the perspective window is turned off to ensure that the mask model itself will not be rendered into the final displayed image.

[0137] Furthermore, once the mask layer and perspective image are rendered, they can be displayed in the perspective scene layer. This ensures that while the perspective image is displayed, the mask layer obscures other information in the perspective scene, hiding everything except the perspective image. This arrangement provides the advantage of achieving a correct spatial perspective effect and spatial occlusion, and further, allows viewing another scene through the perspective window within the current scene.

[0138] S540: Display the perspective image in the current scene.

[0139] The technical solution of the embodiment of the present disclosure is to display a perspective window in the current scene in response to an operation instruction to display a perspective picture, further obtain observation point information, and determine the perspective picture in the perspective scene viewed through the perspective window based on the observation point information. Finally, the perspective picture is displayed in the current scene, which solves the problem that the existing technology cannot realize the interaction with multiple virtual scenes in the same display area at the same time. It realizes the effect of perspective rendering of another scene through the perspective window under the premise that the user is in the current scene. At the same time, during the rendering process, the correct spatial perspective effect and spatial occlusion effect are guaranteed.

[0140] Figure 7 is a schematic diagram of the structure of a video processing device provided by an embodiment of the present disclosure, such as Figure 7 As shown, the apparatus is configured in a virtual reality device, and includes: a perspective window display module 610 , a perspective picture determination module 620 and a perspective picture display module 630 .

[0141] The perspective window display module 610 is configured to display a perspective window in the current scene in response to an operation instruction to display a perspective image.

[0142] A perspective picture determination module 620 is configured to determine a perspective picture to be displayed in the perspective window based on the observation point information of the target object and a pre-made perspective scene; wherein the perspective picture is at least a portion of the perspective scene;

[0143] The perspective image display module 630 is configured to display the perspective image in the current scene.

[0144] Based on the above technical solutions, the device further includes: a display position information determination module.

[0145] The display position information determining module is used to determine the display position information of the perspective window in the current scene before displaying the perspective window in the current scene, so as to display the perspective window based on the display position information.

[0146] On the basis of the above technical solutions, the display position information determination module is specifically configured to determine the display position information of the perspective window based on a preset motion trajectory of the auxiliary window and the set position information of the perspective scene.

[0147] On the basis of the above technical solutions, the display position information determination module includes: a center point position information determination unit and a display position information determination unit.

[0148] a center point position information determining unit, configured to determine the center point position information of the auxiliary window during the movement of the auxiliary window according to the motion trajectory;

[0149] A display position information determining unit is configured to determine the display position information of the perspective window according to the center point position information and the set position information; wherein the display position information changes as the motion trajectory changes.

[0150] On the basis of the above technical solutions, the perspective window corresponds to a sphere, and the display position information determination unit includes: an intersection information determination unit and a center coordinate determination unit.

[0151] an intersection information determining unit, configured to determine, based on the center point position information and the set position information, the intersection information of the auxiliary window and the perspective scene to which the set position information belongs;

[0152] The center coordinate determining unit is configured to determine the center coordinates and the display radius in the display position information based on the intersection information, the center point position information and the set position information.

[0153] On the basis of the above technical solutions, the display position information determination module is specifically used to obtain the drawing trajectory of the operating user in the current scene, and determine the display position information of the perspective window based on the drawing trajectory.

[0154] On the basis of the above technical solutions, the display position information determination module further includes: a drawing trajectory determination unit, a drawing trajectory acquisition unit, a sampling point determination unit, and a display position information determination unit.

[0155] a drawing trajectory determining unit, configured to obtain gesture motion information of the operating user in the current scene based on a gesture recognition algorithm, and determine the drawing trajectory based on the gesture motion information; or

[0156] A drawing trajectory acquisition unit, used to acquire the drawing trajectory of the target handle in space;

[0157] a sampling point determination unit, configured to sample and process the drawing trajectory according to a preset time sampling interval to obtain a plurality of sampling points;

[0158] The display position information determining unit is configured to perform fitting processing on the plurality of sampling points based on a least square fitting algorithm to obtain the center coordinates and the display radius in the display position information.

[0159] On the basis of the above technical solutions, the perspective picture determination module 620 includes: an observation point information acquisition unit and a perspective picture determination unit.

[0160] An observation point information acquisition unit, configured to acquire the observation point information;

[0161] A perspective picture determining unit is used to determine a perspective picture in the perspective scene viewed through the perspective window based on the observation point information.

[0162] On the basis of the above technical solutions, the device further includes: a mask layer making module.

[0163] The mask layer production module is used to produce a perspective scene layer and a mask layer before determining the perspective picture in the perspective scene to be viewed through the perspective window based on the observation point information, so as to display the perspective picture based on the perspective scene layer, the mask layer and the perspective window.

[0164] On the basis of the above technical solutions, the perspective picture determination unit includes: a rendering angle determination subunit, a perspective picture rendering subunit and a perspective picture display subunit.

[0165] a rendering angle determination subunit, configured to determine a rendering angle for the perspective scene based on the observation point information;

[0166] a perspective picture rendering subunit, configured to render the mask layer and the perspective picture based on the rendering angle, a preset reference pixel value, and a preset depth value;

[0167] The perspective picture display subunit is used to display the mask layer and the perspective picture in the perspective scene layer.

[0168] The technical solution of the embodiment of the present disclosure displays a perspective window in the current scene in response to an operation instruction to display a perspective picture. Furthermore, based on the observation point information of the target object and a pre-made perspective scene, the perspective picture displayed in the perspective window is determined. Finally, the perspective picture is displayed in the current scene. This solves the problem that the existing technology cannot realize the interaction with multiple virtual scenes in the same display area at the same time. It realizes the effect of perspective rendering of another scene through the perspective window under the premise that the user is in the current scene, thereby improving the interactive experience between the user and the virtual reality device, and further enhancing the user's immersive experience and usage experience.

[0169] The video processing device provided in the embodiments of the present disclosure can execute the video processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0170] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0171] Figure 8 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 8 , which shows an electronic device (eg Figure 8 The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0172] like Figure 8 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An edit / output (I / O) interface 705 is also connected to the bus 704.

[0173] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0174] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0175] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0176] The electronic device provided by the embodiment of the present disclosure and the video processing method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0177] An embodiment of the present disclosure provides a computer storage medium on which a computer program is stored. When the program is executed by a processor, the video processing method provided by the above embodiment is implemented.

[0178] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0179] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0180] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0181] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0182] In response to an operation instruction to display a perspective image, display a perspective window in the current scene;

[0183] Determining a perspective image displayed in the perspective window based on observation point information of the target object and a pre-made perspective scene; wherein the perspective image is at least a portion of the perspective scene;

[0184] The perspective picture is displayed in the current scene.

[0185] Alternatively, the computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device:

[0186] In response to an operation instruction to display a perspective image, display a perspective window in the current scene;

[0187] Determining a perspective image displayed in the perspective window based on observation point information of the target object and a pre-made perspective scene; wherein the perspective image is at least a portion of the perspective scene;

[0188] The perspective picture is displayed in the current scene.

[0189] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0190] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0191] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0192] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0193] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0194] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0195] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0196] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A video processing method, characterized in that: include: In response to an operation instruction to display a perspective image, display a perspective window in the current scene; Determining a perspective image displayed in the perspective window based on observation point information of the target object and a pre-made perspective scene; wherein the perspective image is at least a portion of the perspective scene; Displaying the perspective picture in the current scene; The display position information of the perspective window is determined based on the following method: Determining the display position information of the perspective window based on the preset motion trajectory of the auxiliary window and the set position information of the perspective scene; or, A drawing trajectory of an operating user in the current scene is obtained, and display position information of the perspective window is determined based on the drawing trajectory.

2. The method according to claim 1, characterized in that Before displaying the perspective window in the current scene, the method further includes: Determine display position information of the perspective window in the current scene, and display the perspective window based on the display position information.

3. The method according to claim 1, characterized in that The determining of the display position information of the perspective window based on the preset auxiliary window motion trajectory and the set position information of the perspective scene includes: During the movement of the auxiliary window according to the motion trajectory, determining the center point position information of the auxiliary window; Determining display position information of the perspective window according to the center point position information and the set position information; The display position information changes with the change of the movement trajectory.

4. The method according to claim 3, characterized in that The perspective window corresponds to a sphere, and determining the display position information of the perspective window according to the center point position information and the set position information includes: determining, according to the center point position information and the set position information, intersection information of the auxiliary window and the perspective scene to which the set position information belongs; Based on the intersection information, the center point position information and the set position information, the center coordinates and the display radius in the display position information are determined.

5. The method according to claim 1, wherein The acquiring operation of the user's drawing trajectory in the current scene and determining the display position information of the perspective window based on the drawing trajectory includes: Acquiring gesture motion information of the operating user in the current scene based on a gesture recognition algorithm, and determining the drawing trajectory based on the gesture motion information; or, Get the drawing trajectory of the target handle in space; Sampling the drawn trajectory according to a preset time sampling interval to obtain a plurality of sampling points; The plurality of sampling points are fitted based on a least square fitting algorithm to obtain the center coordinates and the display radius in the display position information.

6. The method according to claim 1, characterized in that The determining of the perspective picture displayed in the perspective window based on the observation point information of the target object and the pre-made perspective scene includes: Obtaining the observation point information; Based on the observation point information, it is determined to view a perspective picture in the perspective scene through the perspective window.

7. The method according to claim 6, characterized in that Before determining to view the perspective image in the perspective scene through the perspective window based on the observation point information, the method further includes: A perspective scene layer and a mask layer are produced to display the perspective picture based on the perspective scene layer, the mask layer and the perspective window.

8. The method according to claim 7, characterized in that The determining, based on the observation point information, to view a perspective picture in the perspective scene through the perspective window includes: determining a rendering angle for the perspective scene according to the observation point information; Rendering the mask layer and the perspective picture based on the rendering angle, the preset reference pixel value, and the preset depth value; The mask layer and the perspective picture are displayed in the perspective scene layer.

9. A video processing device, characterized in that: include: The perspective window display module is used to display the perspective window in the current scene in response to the operation instruction of displaying the perspective picture: A perspective picture determination module is configured to determine a perspective picture displayed in the perspective window based on the observation point information of the target object and a pre-made perspective scene; wherein the perspective picture is at least a portion of the perspective scene; A perspective picture display module, configured to display the perspective picture in the current scene; The video processing device also includes: a display position information determination module; the display position information determination module is specifically used to determine the display position information of the perspective window based on the pre-set motion trajectory of the auxiliary window and the set position information of the perspective scene; or, obtain the drawing trajectory of the operating user in the current scene, and determine the display position information of the perspective window based on the drawing trajectory.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the video processing method according to any one of claims 1 to 8.

11. A storage medium comprising computer-executable instructions, wherein the computer-executable instructions are used to perform the video processing method according to any one of claims 1 to 8 when executed by a computer processor.

Images