Method, system and storage medium for enlarging and displaying a region in a scene based on picture-in-picture

Abstract

The present invention discloses a method, system and storage medium for zooming in and displaying an area in a scene based on picture-in-picture, which belongs to the field of virtual and reality technology. The method includes: connecting a user camera to obtain a user's field of view in a virtual scene; obtaining a selection box drawn by a user on the field of view, projecting the selection box to the direction of the current user's camera, recording the position of the projection of the selection box when it touches the model and generating a projection frame on the plane where the position is located; calculating the center point coordinates of the projection frame; setting an auxiliary camera in the virtual scene, facing the projection frame, adjusting the distance between the position of the auxiliary camera and the center point coordinates so that the field of view of the auxiliary camera includes the projection frame; generating a second window on the user's display interface, and the second window is used to play the content collected by the auxiliary camera. The present invention collects the scene content in the selection box drawn by the user by selecting an auxiliary camera, so as to achieve the effect of zooming in and displaying a local area in the scene.

Description

Technical Field

[0001] The present invention relates to the field of virtual and reality technology, and in particular to a method, system and storage medium for zooming in and displaying an area within a scene based on picture-in-picture. Background Art

[0002] Virtual reality technology (English name: Virtual Reality, abbreviated as VR) encompasses computers, electronic information, and simulation technologies. Its basic implementation method is based on computer technology, utilizing and integrating the latest developments in three-dimensional graphics technology, multimedia technology, simulation technology, display technology, servo technology, and other high-tech technologies. With the help of computers and other equipment, it creates a realistic three-dimensional virtual world with multiple sensory experiences such as vision, touch, and smell, thereby giving people in the virtual world an immersive feeling.

[0003] With the development of virtual reality technology, virtual scenes are gradually applied in various activities. For example, in some virtual exhibitions, by constructing simulated virtual scenes, users can experience the exhibition contents of different exhibition halls without leaving home. When managers display scenes to other users, they occasionally need to enlarge and display local areas in the scene without moving. For example, in the process of displaying the entire panoramic picture, it is necessary to explain a small area in the panoramic picture in a targeted manner. In order not to lose the display effect of the entire panoramic picture, it is necessary to enlarge the small area. However, traditional enlargement generally captures the current scene picture as a picture, and then uses the method of enlarging a certain area in the picture to achieve this local enlargement display function. This method has certain disadvantages. For example, after the administrator's perspective rotates, the current scene picture changes. Since the enlarged area is determined by the position of the scene picture, when the current picture changes, the locally enlarged content will also change, which is not conducive to the administrator's use. In addition, this way of enlarging the picture will cause pixel loss, and the user's perception will also deteriorate. Therefore, the existing enlarged display still has certain defects. Summary of the invention

[0004] The purpose of the present invention is to solve the problem of poor effect of traditional scene area magnification by proposing a scene area magnification display method, system and storage medium based on picture-in-picture, which has the advantages of using an auxiliary camera to independently display the magnified area, is not affected by the current scene picture and pixels, and has a better display effect.

[0005] In a first aspect, the present invention achieves the above-mentioned purpose through the following technical scheme, a method for enlarging and displaying an area in a scene based on picture-in-picture, the method comprising the following steps:

[0006] Connect to the user's camera to obtain the user's field of view in the virtual scene;

[0007] Obtaining a selection box drawn by the user on the field of view, projecting the selection box in the direction of the current direction of the user's camera, recording the position of the projection of the selection box when it touches the model, and generating a projection box on the plane where the position is located;

[0008] Calculate the center point coordinates of the projection frame;

[0009] An auxiliary camera is set in the virtual scene and faces the projection frame, and the distance between the position of the auxiliary camera and the coordinates of the center point is adjusted so that the field of view of the auxiliary camera includes the projection frame;

[0010] A second window is generated on the user's display interface, where the second window is used to play the content captured by the auxiliary camera.

[0011] Preferably, the selection box is a closed figure drawn by the user using a brush function in the virtual scene, and the closed figure is a two-dimensional figure.

[0012] Preferably, the method of recording the position of the projection of the selection box when it touches the model and generating a projection box on the plane where the position is located is:

[0013] A three-dimensional coordinate system is established with the plane where the selection box is located, with the horizontal and vertical directions of the selection box as the X-axis and Y-axis of the three-dimensional coordinate system, and the direction of the current user's camera is set as the Z-axis of the three-dimensional coordinate system;

[0014] Get all edge points of the selection box, and emit rays starting from the edge points in the direction of the current user's camera;

[0015] Record the coordinates of the collision point when the ray collides with the model (Xn, Yn, Zn);

[0016] Select the collision point coordinate P0 with the smallest Zn value among all the collision point coordinates, and generate a plane parallel to the selection box at the position of P0;

[0017] Project the selection box onto this plane to generate the projection box.

[0018] Preferably, the method for calculating the coordinates of the center point of the projection frame is:

[0019] Get the coordinates of all vertices of the projection frame;

[0020] Select the minimum value min_x and maximum value max_x of all vertex coordinates in the x direction, the minimum value min_y and maximum value max_y in the y direction, and the minimum value min_z and maximum value max_z in the z direction;

[0021] Calculate the center point coordinates (center_x, center_y, center_z),

[0022] center_x=(min_x+max_x) / 2;

[0023] center_y=(min_y+max_y) / 2;

[0024] center_z=(min_z+max_z) / 2.

[0025] Preferably, the method of adjusting the distance between the position of the auxiliary camera and the center point coordinates so that the field of view of the auxiliary camera includes the projection frame is:

[0026] Set the field of view angle fov of the auxiliary camera;

[0027] Calculate the distance between the camera and the center point in coordinates: dist=diagonal / (2*tan(fov / 2)), where diagonal is the diagonal length of the projection frame.

[0028] .

[0029] Preferably, the method also includes setting a control interface for the second window, the control interface including movement and zooming functions, the movement function is achieved by translating the auxiliary camera or rotating the auxiliary camera, and the zooming function is achieved by adjusting the distance between the auxiliary camera and the projection frame.

[0030] In a second aspect, the present invention achieves the above-mentioned purpose through the following technical solutions: a system for magnifying and displaying a region in a scene based on picture-in-picture, the system comprising:

[0031] A camera connection unit, used to connect to a user's camera to obtain a user's field of view in a virtual scene;

[0032] An area selection unit, used for obtaining a selection box drawn by the user on the field of view, projecting the selection box in the direction of the current direction of the user's camera, recording the position of the projection of the selection box when it touches the model and generating a projection box on the plane where the position is located;

[0033] A center point calculation unit, used to calculate the center point coordinates of the projection frame;

[0034] An auxiliary camera setting unit, used to set an auxiliary camera in the virtual scene and face the projection frame, and adjust the distance between the position of the auxiliary camera and the center point coordinates so that the field of view of the auxiliary camera includes the projection frame;

[0035] The display unit is used to generate a second window on the user's display interface, and the second window is used to play the content captured by the auxiliary camera.

[0036] Preferably, the method for recording the position of the projection of the selection box when it touches the model in the area selection unit and generating a projection box on the plane where the position is located is:

[0037] A three-dimensional coordinate system is established with the plane where the selection box is located, with the horizontal and vertical directions of the selection box as the X-axis and Y-axis of the three-dimensional coordinate system, and the direction of the current user's camera is set as the Z-axis of the three-dimensional coordinate system;

[0038] Get all edge points of the selection box, and emit rays starting from the edge points in the direction of the current user's camera;

[0039] Record the coordinates of the collision point when the ray collides with the model (Xn, Yn, Zn);

[0040] Select the collision point coordinate P0 with the smallest Zn value among all the collision point coordinates, and generate a plane parallel to the selection box at the position of P0;

[0041] Project the selection box onto this plane to generate the projection box.

[0042] Preferably, the system also includes a control unit for setting a control interface for the second window, the control interface including movement and zooming functions, the movement function is achieved by translating the auxiliary camera or rotating the auxiliary camera, and the zooming function is achieved by adjusting the distance between the auxiliary camera and the projection frame.

[0043] In a third aspect, the present invention achieves the above-mentioned purpose through the following technical scheme: a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for enlarging and displaying an area within a scene based on picture-in-picture as described in the first aspect.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention utilizes a selection box drawn by a user to select an area that needs to be displayed separately, projects the selection box to generate a projection frame, and uses the projection frame to determine the position of the auxiliary camera, so that the auxiliary camera can completely capture the content in the projection frame, and then use the second window to play the picture of the auxiliary camera to achieve the effect of enlarging and displaying the local area in the virtual scene. Using this method to enlarge the display will not lose pixels, allowing users to watch more clearly, and even if the field of view of the user's camera changes, since the auxiliary camera is actually present in the scene, it will not affect the playback content of the second window. This independent display method can also display different angles of the selected area by adjustment, and more flexible use can meet the needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of the method for enlarging and displaying an area within a scene based on picture-in-picture according to the present invention.

[0047] Figure 2 It is a schematic diagram of projecting the selection box on the transparent canvas of the present invention onto the model.

[0048] Figure 3 It is a schematic diagram of placing the auxiliary camera of the present invention.

[0049] Figure 4 The present invention is a schematic diagram of the position of the projection frame generated when the selection frame of the present invention is projected onto multiple models.

[0050] Figure 5 It is a schematic diagram of the composition of the system for enlarging and displaying an area within a scene based on picture-in-picture according to the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Example 1

[0053] like Figure 1 As shown, a method for enlarging and displaying an area in a scene based on picture-in-picture includes the following steps:

[0054] Step S1, connect the user camera to obtain the user's field of view in the virtual scene. The content displayed by the user in the virtual scene is obtained by playing the picture captured by the user's camera on the display screen. Therefore, by connecting the user's camera, the virtual scene picture that the current user can see can be obtained in real time.

[0055] Step S2, obtain the selection box drawn by the user on the field of view screen, project the selection box in the direction of the current user's camera, record the position of the projection of the selection box when it touches the model, and generate a projection box on the plane where the position is located. In a virtual scene, a brush function is usually provided. This solution uses the drawing operation provided by the screen function to implement the selection box drawn by the user. Different brush colors, line thicknesses, clear and other operation buttons are set on the screen function. In addition, since the virtual scene is a three-dimensional scene, and the screen wants to be drawn in the three-dimensional scene, it is necessary to set a canvas. Usually, in order not to affect the display of the virtual scene, the canvas is set to a transparent state and can only display the drawn lines. Therefore, the user of step S2 can draw a selection box. Since the canvas is pre-set and has a certain distance from the user, the selection box cannot be 100% close to the model that wants to be framed. In this way, when displaying the model, if the distance is too far, it will The display clarity of the model is affected. To address the problem that the distance between the selection box and the model cannot be controlled, the selection box needs to be projected. The selection box is drawn on the current field of view, so the area the user wants to select must be an area that can also be seen in the current field of view. Therefore, the projection direction of the selection box is determined according to the direction of the current user's camera. In this way, the selection box can be accurately projected onto the target area. At this time, the model position touched by the selection box during projection is recorded, and a projection box consistent with the selection box is regenerated at this position. This ensures that the projection box is close to the model, so that the content of the projection box can be displayed with better clarity due to its close distance to the model. For example, Figure 2 As shown, from the perspective of the user's camera, the transparent canvas is at a certain distance from the user's camera, so the selection box drawn on the transparent canvas is still at a certain distance from the model. If the selection box is used as the area for subsequent zoomed display, the distance from the target model will result in poor display effect. After projection, the projection box generated by the projection can be seen on the model. At this time, if the projection box is used as the area for subsequent zoomed display, the observed model will be clearer.

[0056] Step S3, calculate the center point coordinates of the projection frame. According to step S2, the final displayed area is the selected area of ​​the projection frame. Therefore, in order to ensure that the display area will not be offset, the camera used for acquisition needs to face the projection frame to be able to fully display the selected area of ​​the projection frame. Therefore, the center point coordinates of the projection frame are calculated to determine the center position of the projection frame. The center position is used to determine the placement of the camera. The specific placement will be explained in detail in subsequent steps. In addition, the shape of the drawn selection box is different, which will cause the shape of the projection frame to change. Therefore, each time a selection box is drawn, the center point coordinates of the projection frame need to be calculated to ensure the center position of the projection frame.

[0057] Step S4, setting an auxiliary camera in the virtual scene and facing the projection frame, adjusting the distance between the position of the auxiliary camera and the center point coordinates so that the field of view of the auxiliary camera includes the projection frame. The auxiliary camera is used to collect the display content of the area selected by the projection frame. Therefore, the auxiliary camera must include the projection frame to fully display the content of the projection frame. According to the center point coordinates calculated in step S3, the position of the auxiliary camera can be determined, such as Figure 3 As shown, the auxiliary camera is only used to capture the image of the projection frame area, and the user camera is responsible for capturing the image in the entire scene. Even if the image of the auxiliary camera is played to achieve the effect of displaying the enlarged area, because the main image and the enlarged display image are both captured by the camera in the virtual scene, there is no sense of disobedience between the two images in terms of color tone and resolution. When the auxiliary camera plays content, it also plays a three-dimensional image, which is more realistic. In addition, we use the auxiliary camera to capture the content of the projection frame area instead of using the captured image of the projection frame area for display. Apart from the two-dimensional display effect of the image, the auxiliary camera is more operable. For example, the function of controlling the auxiliary camera can be added later to realize the display of different angles in the projection frame area, and it is easier to adjust and move the enlarged display area locally.

[0058] Step S5, generating a second window on the user's display interface, the second window is used to play the content captured by the auxiliary camera. It can be seen from step S4 that the main display screen in the scene is realized by playing the content captured by the user's camera, and the second window can be set to be a floating window suspended on the main display screen. It will only be suspended on the main display screen when the enlarged display function is turned on. When the function is turned off, the second window disappears, and it will not affect the normal use of the entire virtual scene. This uses the second window to play the content captured by the auxiliary camera. It is simple and convenient to implement, and it is also easy to close. In actual use, the size of the second window also needs to provide an adjustable function. The display ratio and size of the second window can be adjusted by user customization, and the display enlargement area can be customized, which is more flexible.

[0059] In step S2, the selection box is drawn by the user. In order to avoid being unable to recognize the area that the user wants to select, the selection box is a closed figure drawn by the user using the brush function in the virtual scene, and the closed figure is a two-dimensional figure. Only when the user draws a closed figure can a selection box be generated, and the shape of the selection box is not fixed, and the user can customize the area to be enlarged and displayed. The two-dimensional figure is also for the convenience of the user to smoothly draw the selection box that he wants to select without rotating the viewing angle, because if the user wants to enlarge a certain area, it must be an area that can be seen under the current viewing angle, so even if a two-dimensional figure is used, the area that the user wants to enlarge and display can be selected.

[0060] like Figure 2 As shown, the projection in step S2 is mainly used when the selection box can be projected onto the model surface to form a projection box. However, in actual applications, when the scene content is complex and the drawing skills of each user are different, it is difficult to draw a selection box that can be directly projected onto the model surface. Therefore, the projection method needs to be optimized. The method of recording the position of the projection of the selection box when it touches the model and generating a projection box on the plane where the position is located is:

[0061] A 3D coordinate system is established based on the plane where the selection box is located. The horizontal and vertical directions of the selection box are the X-axis and Y-axis of the 3D coordinate system, and the direction of the current user camera is set as the Z-axis of the 3D coordinate system. Since the 3D coordinate system is established based on the plane of the selection box, the Z-axis coordinates of the edge points on the selection box are all 0. The direction of the user camera here is also the projection direction of the selection box.

[0062] Get all the edge points of the selection box, and use the edge points as the starting point to emit rays in the direction of the current user's camera. Projection is achieved through rays because the rays will generate collision signals when they touch obstacles, so the collision points with the model can be detected while projecting;

[0063] Record the coordinates of the collision point when the ray collides with the model (Xn, Yn, Zn);

[0064] Select the collision point coordinate P0 with the smallest Zn value among all the collision point coordinates, and generate a plane parallel to the selection box at the position of P0;

[0065] Project the selection box onto this plane to generate the projection box.

[0066] like Figure 4As shown in the figure, it is a schematic diagram of the projection of the projection frame when the projection method is used for projection. From the figure, it can be seen that the projection direction of the selection frame has two models, the first model and the second model, and the collision points generated by the projection exist not only on the surface of the first model, but also on the surface of the second model. If all collision points are directly selected to form a projection frame, then the projection frame is not a two-dimensional closed figure, and the center point coordinates cannot be calculated, and the position of the auxiliary camera cannot be placed subsequently. Therefore, after obtaining the coordinates of all collision points using this method, the minimum collision point coordinates need to be calculated. In the figure, the distance between the second model and the selection frame is relatively close, so the value of the collision point coordinates on the surface of the second model in the Z-axis direction is the smallest. Therefore, the second model surface is selected as the generated plane, and the selection frame is finally projected to the plane shape projection frame. This can avoid the problem that the selection frame cannot project a two-dimensional plane projection frame when there are many models in the scene.

[0067] The method for calculating the center point coordinates of the projection frame is:

[0068] Get the coordinates of all vertices of the projection frame;

[0069] Select the minimum value min_x and maximum value max_x of all vertex coordinates in the x direction, the minimum value min_y and maximum value max_y in the y direction, and the minimum value min_z and maximum value max_z in the z direction;

[0070] Calculate the center point coordinates (center_x, center_y, center_z),

[0071] center_x=(min_x+max_x) / 2;

[0072] center_y=(min_y+max_y) / 2;

[0073] center_z=(min_z+max_z) / 2. This calculation method is to determine the center point coordinates by calculating half of the distance between the maximum and minimum values ​​of the edge points of the selection box in the three directions of xyz.

[0074] The method of adjusting the distance between the position of the auxiliary camera and the center point coordinates so that the field of view of the auxiliary camera includes the projection frame is:

[0075] Set the field of view angle fov of the auxiliary camera;

[0076] Calculate the distance between the camera and the center point in coordinates: dist=diagonal / (2*tan(fov / 2)), where diagonal is the diagonal length of the projection frame.

[0077] .

[0078] Since the auxiliary camera's field of view angle is fixed, and its direction is consistent with the direction of the current user's camera, the size of the field of view can be adjusted by adjusting the distance between the auxiliary camera and the projection frame. Therefore, the above method is to calculate the appropriate distance so that the field of view can include the projection frame and the image in the projection frame can be fully displayed.

[0079] The method also includes setting a control interface for the second window, the control interface includes movement and zooming functions, the movement function is realized by translating the auxiliary camera or rotating the auxiliary camera, and the zooming function is realized by adjusting the distance between the auxiliary camera and the projection frame. In order to ensure that the content displayed in the second window is more accurate or can be freely adjusted according to the user, a control interface for operating the auxiliary camera is provided. For example, by moving the auxiliary camera, the content of the second window can be changed, such as moving the collected content up, down, left, and right, selecting a suitable area for fine-tuning, or rotating the auxiliary camera to change the collection angle, or moving the relative position between the auxiliary camera and the projection frame, so as to zoom in and out the collection area.

[0080] Example 2

[0081] like Figure 5 As shown, a system for zooming in and displaying a region in a scene based on picture-in-picture, the system comprising:

[0082] A camera connection unit, used to connect to a user's camera to obtain a user's field of view in a virtual scene;

[0083] An area selection unit, used for obtaining a selection box drawn by the user on the field of view, projecting the selection box in the direction of the current direction of the user's camera, recording the position of the projection of the selection box when it touches the model and generating a projection box on the plane where the position is located;

[0084] A center point calculation unit, used to calculate the center point coordinates of the projection frame;

[0085] An auxiliary camera setting unit, used to set an auxiliary camera in the virtual scene and face the projection frame, and adjust the distance between the position of the auxiliary camera and the center point coordinates so that the field of view of the auxiliary camera includes the projection frame;

[0086] The display unit is used to generate a second window on the user's display interface, and the second window is used to play the content captured by the auxiliary camera.

[0087] The method for recording the position of the projection of the selection box when it touches the model in the area selection unit and generating a projection box on the plane where the position is located is:

[0088] A three-dimensional coordinate system is established with the plane where the selection box is located, with the horizontal and vertical directions of the selection box as the X-axis and Y-axis of the three-dimensional coordinate system, and the direction of the current user's camera is set as the Z-axis of the three-dimensional coordinate system;

[0089] Get all edge points of the selection box, and emit rays starting from the edge points in the direction of the current user's camera;

[0090] Record the coordinates of the collision point when the ray collides with the model (Xn, Yn, Zn);

[0091] Select the collision point coordinate P0 with the smallest Zn value among all the collision point coordinates, and generate a plane parallel to the selection box at the position of P0;

[0092] Project the selection box onto this plane to generate the projection box.

[0093] The system also includes a control unit for setting a control interface for the second window. The control interface includes movement and zooming functions. The movement function is achieved by translating the auxiliary camera or rotating the auxiliary camera, and the zooming function is achieved by adjusting the distance between the auxiliary camera and the projection frame.

[0094] The selection box is a closed figure drawn by the user using a brush function in the virtual scene, and the closed figure is a two-dimensional figure.

[0095] The method for calculating the center point coordinates of the projection frame is:

[0096] Get the coordinates of all vertices of the projection frame;

[0097] Select the minimum value min_x and maximum value max_x of all vertex coordinates in the x direction, the minimum value min_y and maximum value max_y in the y direction, and the minimum value min_z and maximum value max_z in the z direction;

[0098] Calculate the center point coordinates (center_x, center_y, center_z),

[0099] center_x=(min_x+max_x) / 2;

[0100] center_y=(min_y+max_y) / 2;

[0101] center_z=(min_z+max_z) / 2.

[0102] The method of adjusting the distance between the position of the auxiliary camera and the center point coordinates so that the field of view of the auxiliary camera includes the projection frame is:

[0103] Set the field of view angle fov of the auxiliary camera;

[0104] Calculate the distance between the camera and the center point in coordinates: dist=diagonal / (2*tan(fov / 2)), where diagonal is the diagonal length of the projection frame.

[0105] .

[0106] Embodiment 2 is essentially the same as Embodiment 1, so the specific working principles of each unit will not be described in detail.

[0107] Example 3

[0108] This embodiment proposes a storage medium, which includes a program storage area and a data storage area, wherein the program storage area can store an operating system and programs required for running an instant messaging function, etc.; the data storage area can store various instant messaging information and operation instruction sets, etc. The computer program is stored in the program storage area, and when the computer program is executed by the processor, the method for enlarging and displaying a region in a scene based on a picture-in-picture as described in Embodiment 1 is implemented. The processor may include one or more central processing units, or a digital processing unit, etc.

[0109] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be considered exemplary and non-restrictive in all respects, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present invention.

[0110] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for enlarging and displaying a region in a scene based on picture-in-picture, characterized in that: The method comprises the following steps: Connect to the user's camera to obtain the user's field of view in the virtual scene; Obtaining a selection box drawn by the user on the field of view, projecting the selection box in the direction of the current direction of the user's camera, recording the position of the projection of the selection box when it touches the model, and generating a projection box on the plane where the position is located; Calculate the center point coordinates of the projection frame; An auxiliary camera is set in the virtual scene and faces the projection frame, the auxiliary camera is used to collect display content of the area selected by the projection frame, and the distance between the position of the auxiliary camera and the coordinates of the center point is adjusted so that the field of view of the auxiliary camera includes the projection frame; A second window is generated on the user's display interface, where the second window is used to play the content captured by the auxiliary camera.

2. The method for enlarging and displaying a region in a scene based on picture-in-picture according to claim 1, characterized in that: The selection box is a closed figure drawn by the user using a brush function in the virtual scene, and the closed figure is a two-dimensional figure.

3. The method for enlarging and displaying a region in a scene based on picture-in-picture according to claim 1, characterized in that: The method of recording the position of the projection of the selection box when it touches the model and generating a projection box on the plane where the position is located is: A three-dimensional coordinate system is established with the plane where the selection box is located, with the horizontal and vertical directions of the selection box as the X-axis and Y-axis of the three-dimensional coordinate system, and the direction of the current user's camera is set as the Z-axis of the three-dimensional coordinate system; Get all edge points of the selection box, and emit rays starting from the edge points in the direction of the current user's camera; Record the coordinates of the collision point when the ray collides with the model (Xn, Yn, Zn); Select the collision point coordinate P0 with the smallest Zn value among all the collision point coordinates, and generate a plane parallel to the selection box at the position of P0; Project the selection box onto this plane to generate the projection box.

4. The method for enlarging and displaying a region in a scene based on picture-in-picture according to claim 1, characterized in that: The method for calculating the center point coordinates of the projection frame is: Get the coordinates of all vertices of the projection frame; Select the minimum value min_x and maximum value max_x of all vertex coordinates in the x direction, the minimum value min_y and maximum value max_y in the y direction, and the minimum value min_z and maximum value max_z in the z direction; Calculate the center point coordinates (center_x, center_y, center_z), center_x=(min_x+max_x) / 2; center_y=(min_y+max_y) / 2; center_z=(min_z+max_z) / 2.

5. The method for enlarging and displaying a region in a scene based on picture-in-picture according to claim 4, characterized in that: The method of adjusting the distance between the position of the auxiliary camera and the center point coordinates so that the field of view of the auxiliary camera includes the projection frame is: Set the field of view angle fov of the auxiliary camera, where fov is the angle value of the horizontal field of view angle of the auxiliary camera; Calculate the distance between the camera and the center point in coordinates: dist=diagonal / (2*tan(fov / 2)), where diagonal is the diagonal length of the projection frame. 。 6. The method for enlarging and displaying a region in a scene based on picture-in-picture according to claim 1, characterized in that: The method also includes setting a control interface for the second window, the control interface including movement and zooming functions, the movement function is achieved by translating the auxiliary camera or rotating the auxiliary camera, and the zooming function is achieved by adjusting the distance between the auxiliary camera and the projection frame.

7. A system for magnifying and displaying a region in a scene based on picture-in-picture, characterized in that: The system includes: A camera connection unit, used to connect to a user's camera to obtain a user's field of view in a virtual scene; An area selection unit, used for obtaining a selection box drawn by the user on the field of view, projecting the selection box in the direction of the current direction of the user's camera, recording the position of the projection of the selection box when it touches the model and generating a projection box on the plane where the position is located; A center point calculation unit, used to calculate the center point coordinates of the projection frame; an auxiliary camera setting unit, used to set an auxiliary camera in the virtual scene and face the projection frame, the auxiliary camera is used to collect display content of the area selected by the projection frame, and adjust the distance between the position of the auxiliary camera and the center point coordinates so that the field of view of the auxiliary camera includes the projection frame; The display unit is used to generate a second window on the user's display interface, and the second window is used to play the content captured by the auxiliary camera.

8. The system for enlarging and displaying a region in a scene based on picture-in-picture according to claim 7, characterized in that: The method for recording the position of the projection of the selection box when it touches the model in the area selection unit and generating a projection box on the plane where the position is located is: A three-dimensional coordinate system is established with the plane where the selection box is located, with the horizontal and vertical directions of the selection box as the X-axis and Y-axis of the three-dimensional coordinate system, and the direction of the current user's camera is set as the Z-axis of the three-dimensional coordinate system; Get all edge points of the selection box, and emit rays starting from the edge points in the direction of the current user's camera; Record the coordinates of the collision point when the ray collides with the model (Xn, Yn, Zn); Select the collision point coordinate P0 with the smallest Zn value among all the collision point coordinates, and generate a plane parallel to the selection box at the position of P0; Project the selection box onto this plane to generate the projection box.

9. The system for zooming in and displaying a region in a scene based on picture-in-picture according to claim 7, characterized in that: The system also includes a control unit for setting a control interface for the second window. The control interface includes movement and zooming functions. The movement function is achieved by translating the auxiliary camera or rotating the auxiliary camera, and the zooming function is achieved by adjusting the distance between the auxiliary camera and the projection frame.

10. A storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for enlarging and displaying an area within a scene based on picture-in-picture as described in any one of claims 1 to 6 is implemented.

Images