Transition switching method of image

Abstract

The embodiment of the invention discloses an image transition switching method. The method comprises the following steps: determining a matching point pair between a local image in a panorama of a preset scene and a high-definition image; obtaining related information of a display device, and respectively converting the local image and the high-definition image to a screen coordinate system corresponding to the display device; determining a first homography matrix between the local image and the high-definition image based on coordinate information of the matching point pair under the screen coordinate system; on the basis of the first homography matrix and the second homography matrix, smooth transition from the local image to the high-definition image is achieved, and the high-definition image is displayed on the display device; according to the embodiment of the invention, the smooth transition from the local image to the high-definition image is realized through the first homography matrix and the second homography matrix, and the requirement of a user for checking local details in the panoramic image is met.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and particularly to a transition switching method of images. BACKGROUND

[0002] Step-by-step panoramic VR is a virtual reality (VR) experience form based on real panoramic images. Its core feature is to realize the "step-by-step exploration" of users in a virtual space by combining interactive logic with panoramic images collected and fused in steps. It has the advantages of high real scene restoration and controllable production cost, and is widely used in virtual house viewing, museum tour, and tourism scenic spot roaming. When browsing panoramic images, users often have the need to view the details of certain specific objects or scenes, which requires further display of the objects or scenes, such as providing high-definition photos. SUMMARY

[0003] Embodiments of the present disclosure provide a transition switching method of images.

[0004] According to an aspect of an embodiment of the present disclosure, a transition switching method of images is provided, comprising: determining a matching point pair between a local image in a panoramic image of a preset scene and a high-definition image; obtaining relevant information of a display device, and converting the local image and the high-definition image into screen coordinate systems corresponding to the display device, respectively; determining a first homography matrix between the local image and the high-definition image based on coordinate information of the matching point pair in the screen coordinate system; realizing smooth transition from the local image to the high-definition image based on the first homography matrix and a second homography matrix, and displaying the high-definition image on the display device.

[0005] Optionally, the obtaining relevant information of a display device, and converting the local image and the high-definition image into screen coordinate systems corresponding to the display device, respectively, comprises: determining world space coordinates of the local image in a world coordinate system; converting the world space coordinates of the local image into local plane coordinates corresponding to the local image in the screen coordinate system based on an extrinsic parameter matrix of a virtual camera corresponding to the display device; performing adaptive scaling on the high-definition image based on screen information of the display device to determine plane coordinates of the high-definition image in the screen coordinate system.

[0006] Optionally, the determining world space coordinates of the local image in a world coordinate system comprises: performing UV normalization processing on the local map in the spherical coordinate system to obtain a three-dimensional spherical coordinate corresponding to the local map; convert the three-dimensional spherical coordinate into a world coordinate system to obtain the world space coordinate corresponding to the local map.

[0007] Optionally, the determining the first homography matrix between the local map and the high-definition image based on the coordinate information of the matching point pairs in the screen coordinate system comprises: constructing a homogeneous equation set based on the coordinates of the matching point pairs in the screen coordinate system; wherein, each matching point pair generates two rows of constraint equations in the homogeneous equation set; solving the homogeneous equation set to determine the first homography matrix between the local map and the high-definition image.

[0008] Optionally, the solving the homogeneous equation set to determine the first homography matrix between the local map and the high-definition image comprises: performing singular value decomposition on the homogeneous equation set to obtain coefficient values in the homogeneous equation set and obtain an initial matrix; performing scale normalization on the initial matrix to obtain the first homography matrix.

[0009] Optionally, the second homography matrix is a unit matrix, and the realizing smooth transition from the local map to the high-definition image based on the first homography matrix and the second homography matrix and displaying the high-definition image on the display device comprises: constructing an off-screen rendering channel, and rendering rendering parameters of the local map to an off-screen texture through the off-screen rendering channel; performing interpolation processing on the first homography matrix and the second homography matrix to obtain at least one intermediate matrix; smoothly transitioning from the local map to the high-definition image based on the at least one intermediate matrix; rendering the high-definition image to the display device for display through a texture shader.

[0010] Optionally, before the realizing smooth transition from the local map to the high-definition image based on the first homography matrix and the second homography matrix and displaying the high-definition image on the display device, the method further comprises: determining a camera viewing pyramid corresponding to the display device based on intrinsic parameters and extrinsic parameters of a virtual camera corresponding to the display device; projecting the local map to the camera viewing pyramid through inverse projection; adjusting a field of view angle corresponding to the virtual camera so that feature points in the matching point pairs included in the local map are projected within an effective area of a screen to obtain a target parameter matrix of the virtual camera.

[0011] Optionally, the field of view corresponding to the virtual camera is adjusted so that the feature points in the matching point pair included in the local image are projected within the effective area of the screen, to obtain a target parameter matrix of the virtual camera, comprising: Within a preset field of view adjustment range, a binary search method is performed on the field of view with a preset step value to dynamically adjust the field of view so that the feature points in the matching point pair included in the local image are projected within the effective area of the screen; Based on the dynamically adjusted field of view, the target parameter matrix is determined.

[0012] Optionally, the matching point pair between the local image in the panoramic image of the preset scene and the high-definition image comprises: Feature extraction is performed on the local image and the high-definition image respectively, to obtain at least one first feature point corresponding to the local image and at least one second feature point corresponding to the high-definition image; Through feature matching, the matching point pair is determined from the at least one first feature point and the at least one second feature point.

[0013] Optionally, it further comprises: In response to triggering an exit operation, based on the first homography matrix and the second homography matrix, a smooth transition from the high-definition image to the local image is realized, and the local image is displayed on the display device.

[0014] According to another aspect of the embodiments of the present disclosure, an image transition switching device is provided, comprising: A feature point matching module is configured to determine a matching point pair between a local image in a panoramic image of a preset scene and a high-definition image; A coordinate system conversion module is configured to obtain relevant information of a display device, and convert the local image and the high-definition image into a screen coordinate system corresponding to the display device, respectively; A mapping relationship module is configured to determine a first homography matrix between the local image and the high-definition image based on coordinate information of the matching point pair in the screen coordinate system; An image transition module is configured to realize a smooth transition from the local image to the high-definition image based on the first homography matrix and a second homography matrix, and display the high-definition image on the display device.

[0015] Optionally, the coordinate system conversion module is specifically configured to determine world space coordinates of the local map in a world coordinate system; convert the world space coordinates of the local map to the screen coordinate system based on an extrinsic parameter matrix of a virtual camera corresponding to the display device, to obtain local plane coordinates corresponding to the local map; and perform adaptive scaling on the high-definition image based on screen information of the display device, to determine plane coordinates of the high-definition image in the screen coordinate system.

[0016] Optionally, when determining the world space coordinates of the local map in the world coordinate system, the coordinate system conversion module is configured to perform UV normalization processing on the local map in a spherical coordinate system, to obtain three-dimensional spherical coordinates corresponding to the local map; and convert the three-dimensional spherical coordinates to the world coordinate system, to obtain the world space coordinates corresponding to the local map.

[0017] Optionally, the mapping relationship module is specifically configured to construct a homogeneous equation set based on the coordinates of the matching point pairs in the screen coordinate system; wherein, two rows of constraint equations in the homogeneous equation set are generated for each matching point pair; and solve the homogeneous equation set, to determine a first homography matrix between the local map and the high-definition image.

[0018] Optionally, when solving the homogeneous equation set to determine the first homography matrix between the local map and the high-definition image, the mapping relationship module is configured to perform singular value decomposition on the homogeneous equation set, to obtain coefficient values in the homogeneous equation set, to obtain an initial matrix; and perform scale normalization on the initial matrix, to obtain the first homography matrix.

[0019] Optionally, the image transition module is specifically configured to construct an off-screen rendering channel, to render rendering parameters of the local map to an off-screen texture through the off-screen rendering channel; perform interpolation processing on the first homography matrix and the second homography matrix, to obtain at least one intermediate matrix; based on the at least one intermediate matrix, smoothly transition from the local map to the high-definition image; and render the high-definition image to the display device for display through a texture shader.

[0020] Optionally, the apparatus further comprises: a frustum determination module configured to determine a camera frustum corresponding to the display device based on intrinsic and extrinsic parameters of a virtual camera corresponding to the display device; a reverse projection module configured to project the local map to the camera frustum through reverse projection; a field of view angle adjustment module configured to adjust a field of view angle corresponding to the virtual camera, so that feature points in the matching point pairs included in the local map are projected within an effective area of a screen, to obtain a target parameter matrix of the virtual camera.

[0021] Optionally, the field of view angle adjusting module is specifically configured to perform a binary search method on the field of view angle with a preset step value in a preset field of view angle adjustment range, so that the feature points in the matching point pair included in the local image are projected in the effective area of the screen; and determine the target parameter matrix based on the dynamically adjusted field of view angle.

[0022] Optionally, the feature point matching module is specifically configured to perform feature extraction on the local image and the high-definition image respectively to obtain at least one first feature point corresponding to the local image and at least one second feature point corresponding to the high-definition image; and determine the matching matching point pair from the at least one first feature point and the at least one second feature point through feature matching.

[0023] Optionally, the apparatus further comprises: a smooth exit module configured to, in response to triggering an exit operation, realize a smooth transition from the high-definition image to the local image based on the first homography matrix and the second homography matrix, and display the local image on the display device.

[0024] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, and the electronic device comprises: a memory configured to store a computer program product; a processor configured to execute the computer program product stored in the memory, and when the computer program product is executed, implement the transition switching method of the image according to any one of the above embodiments.

[0025] According to another aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, and the computer readable storage medium stores computer program instructions, and the computer program instructions are executed by a processor to implement the transition switching method of the image according to any one of the above embodiments.

[0026] According to another aspect of the embodiments of the present disclosure, a computer program product is provided, and the computer program product comprises computer program instructions, and the computer program instructions are executed by a processor to implement the transition switching method of the image according to any one of the above embodiments.

[0027] Based on the transition switching method of the image provided by the above-mentioned embodiments of the present disclosure, the matching point pairs between the local map in the panoramic map of the preset scene and the high-definition image are determined; the related information of the display device is obtained, and the local map and the high-definition image are respectively converted to the screen coordinate system corresponding to the display device; based on the coordinate information of the matching point pairs in the screen coordinate system, the first homography matrix between the local map and the high-definition image is determined; based on the first homography matrix and the second homography matrix, the smooth transition from the local map to the high-definition image is realized, and the high-definition image is displayed on the display device; in the embodiments of the present disclosure, the corresponding high-definition image of the local map is provided, the high-definition image provides more detailed information, the local map and the high-definition image are matched through the matching point pairs, and the smooth transition from the local map to the high-definition image is realized through the first homography matrix and the second homography matrix, which meets the needs of users for viewing the local details in the panoramic map, and the display process of the high-definition image is not disconnected from the panoramic map through the smooth transition, thereby improving the user experience.

[0028] The technical solutions of the present disclosure will be described in further detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0029] The accompanying drawings, which form a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0030] The present disclosure can be more clearly understood and appreciated from the following detailed description, taken in conjunction with the accompanying drawings, in which: Figure 1 FIG. 1 is a flowchart of a transition switching method of an image provided by an exemplary embodiment of the present disclosure; Figure 2 FIG. 2 is a flowchart of step 102 in the embodiment shown in FIG. 1; Figure 1 FIG. 3 is a flowchart of step 104 in the embodiment shown in FIG. 1; Figure 3 Figure 1 FIG. 4 is a flowchart of step 106 in the embodiment shown in FIG. 1; Figure 4 FIG. 5 is a flowchart of step 108 in the embodiment shown in FIG. 1; Figure 1 FIG. 6 is a structural diagram of a transition switching device of an image provided by an exemplary embodiment of the present disclosure; Figure 5 Figure 1 FIG. 7 is a block diagram of an electronic device according to an embodiment of the present disclosure. Figure 6 The present disclosure can be more clearly understood and appreciated from the following detailed description, taken in conjunction with the accompanying drawings, in which: Figure 7 FIG. 7 is a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] ​​Hereinafter, example embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, and are not all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the example embodiments described herein.

[0032] It should be noted that the relative arrangement, numerical expressions, and numerical values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0033] Those skilled in the art can understand that the terms "first", "second", and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, and do not represent any specific technical meaning, nor indicate their logical order.

[0034] It should also be understood that in the embodiments of the present disclosure, "a plurality of" can mean two or more, and "at least one" can mean one, two or more.

[0035] It should also be understood that for any component, data or structure mentioned in the embodiments of the present disclosure, it can be understood as one or more in general, without explicit limitation or in the context of the opposite indication.

[0036] In addition, the term "and / or" in the present disclosure is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the character " / " in the present disclosure generally represents an "or" relationship between the front and rear associated objects. The data referred to in the present disclosure can include unstructured data such as text, images, and videos, and can also be structured data.

[0037] It should also be understood that the description of the embodiments of the present disclosure focuses on the differences between the embodiments, and the same or similar parts can be referred to each other, and for the sake of brevity, will not be repeated.

[0038] At the same time, it should be understood that in order to facilitate description, the size of each part shown in the drawings is not drawn in accordance with the actual proportional relationship.

[0039] The following description of at least one example embodiment is merely illustrative in nature and is in no way limiting to the disclosure and its application or uses.

[0040] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but where appropriate, the described techniques, methods, and devices should be considered as part of the specification.

[0041] It should be noted that like reference numerals and characters refer to like elements throughout the following description and the claims. Thus, once any certain element is defined in one drawing, it should not have to be discussed further in subsequent drawings.

[0042] Embodiments of the present disclosure can be applied to terminal devices, computer systems, servers, and other electronic devices, which can operate with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations that can be suitable for use with terminal devices, computer systems, servers, and other electronic devices include, but are not limited to: personal computers, server computers, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputers, mainframe computers, and distributed cloud computing environments that include any of the above systems or the like.

[0043] Terminal devices, computer systems, servers, and other electronic devices can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, and the like, that perform particular tasks or implement particular abstract data types. Computer systems / servers can be practiced in distributed cloud-computing environments with remote processing devices that are linked through a communications network. In a distributed cloud-computing environment, program modules can be located in local or remote computer system storage media including memory storage devices.

[0044] SUMMARY In the process of implementing the present disclosure, the inventors found that, for the detailed display of an object or a scene in a panorama, a user usually clicks a button when browsing the panorama, and then a high-definition picture is popped up. This way at least has the following problems: the user will feel that the image and the scene are split, and there is a strong sense of fragmentation.

[0045] Exemplary method Figure 1 FIG. 1 is a flow diagram of an image transition switching method provided by an exemplary embodiment of the present disclosure. The present embodiment can be applied to an electronic device, such as a terminal device, a computer system, a server, or the like. Figure 1 As shown in FIG. 1, the method includes the following steps: Step 102, determining a matching point pair between a local image in a panorama of a preset scene and a high-definition image.

[0046] The high-definition image is an image with higher resolution and pixel density, and can present more details in visual experience. There is at least one matching feature point between the high-definition image and the local image. A matching point pair includes a first feature point in the local image and a second feature point in the high-definition image. The first feature point and the second feature point in a matching point pair correspond to a local feature at the same position in the physical space, and can be accurately described by a key point attribute and a descriptor to ensure the uniqueness and accuracy of the matching. Optionally, any existing feature point matching method can be used to obtain the matching point pair, for example, a deep neural network model or an ORB feature matching.

[0047] The high-definition image in the embodiment can be an image with higher precision obtained by separately shooting some objects or scenes in the panoramic image.

[0048] In step 104, the relevant information of the display device is obtained, and the local image and the high-definition image are respectively converted to the screen coordinate system corresponding to the display device.

[0049] In an embodiment, the display device at least includes a display screen, which can be any device with display function, such as a mobile phone, a computer, a tablet computer, etc.; the related information of the display device can include, but is not limited to, at least one of the following: pixel size of the screen (i.e., horizontal pixel number x vertical pixel number, such as 1920 x 1080, 3840 x 2160, etc.), field of view (FOV), parameter matrix of a virtual camera, etc. Among them, the virtual camera (Virtual Camera) is a software abstract concept in computer graphics, 3D rendering and virtual / augmented reality (VR / AR) that simulates the function of a real camera, used to define the “viewing angle” and project a 3D scene (such as a panoramic map, a 3D model) onto a 2D screen. It does not have a physical entity, but simulates the optical properties, position and pose of a real camera through parameter settings to determine the final picture content presented to the user. Optionally, in an optional example, the process of converting the local map in the panoramic map to the screen coordinate system can include: converting the local map from the model coordinate system to the world coordinate system: the objects in the 3D scene are first converted to a unified world coordinate system through their own model matrix. Convert from the world coordinate system to the camera coordinate system: convert the points in the world coordinate system to the camera view (with the camera as the origin and the Z axis as the line of sight) through the extrinsic matrix (rotation matrix R and translation matrix T). Convert from the camera coordinate system to the screen coordinate system: project the 3D points to the 2D screen through the intrinsic matrix (K) and the projection transformation (perspective or orthogonal) to get the final display pixel coordinates. Optionally, the process of converting the high-definition image to the screen coordinate system is relatively simple and can include: size adaptation: scaling the high-definition image proportionally to ensure that the aspect ratio is compatible with the screen; coordinate mapping: calculating the scaling factor and offset to map the high-definition image pixels one by one to the screen location.

[0050] Step 106, based on the coordinate information of the matched point pairs in the screen coordinate system, determine the first homography matrix between the local map and the high-definition image.

[0051] The homography matrix (Homography Matrix) is a 3x3 homogeneous matrix in computer vision and graphics that describes the projection transformation relationship between two planes, used to represent the mapping from a point on one plane to the corresponding point on another plane. It is widely used in image stitching, perspective conversion, augmented reality (AR), etc. The core role is to establish the coordinate conversion relationship between two 2D planes. In this embodiment, after the local map and the high-definition image are both converted to the screen coordinate system, they are both in 2D planes, and the conversion between them is the coordinate conversion relationship between two 2D planes.

[0052] In some optional examples, a homogeneous equation can be constructed first, and then the coefficients in the homogeneous equation can be solved by substituting the two-dimensional coordinates of the matching point pair in the screen coordinate system into the homogeneous equation to obtain the first homography matrix.

[0053] Step 108: Based on the first and second homography matrices, a smooth transition from the local image to the high-definition image is achieved, and the high-definition image is displayed on the display device.

[0054] Optionally, the second homography matrix can be an identity matrix. All elements of the identity matrix satisfy the following condition: the coordinates of any point on the image remain unchanged after transformation by this matrix, indicating the correspondence between local and high-definition images. In this embodiment, when transitioning from a local image to a high-definition image, linear interpolation can be performed between the second and first homography matrices to weighted mix the homography matrix elements, achieving a smooth transition from the local image to the high-definition image. Linear interpolation of the homography matrix ensures the continuity of image transformation, avoids direct image jumps, achieves a natural image transition effect, and enhances the visual experience.

[0055] The image transition switching method provided in the above embodiments of this disclosure determines matching point pairs between a partial image and a high-definition image in a panoramic image of a preset scene; obtains relevant information of the display device, and converts the partial image and the high-definition image to the screen coordinate system corresponding to the display device; determines a first homography matrix between the partial image and the high-definition image based on the coordinate information of the matching point pairs in the screen coordinate system; and achieves a smooth transition from the partial image to the high-definition image based on the first homography matrix and the second homography matrix, displaying the high-definition image on the display device. This disclosure provides all or part of the high-definition image corresponding to the partial image, with the high-definition image providing more detailed information. By matching the partial image and the high-definition image through matching point pairs, and by achieving a smooth transition from the partial image to the high-definition image through the first homography matrix and the second homography matrix, it satisfies the user's need to view local details in the panoramic image. Furthermore, the smooth transition ensures that the display process of the high-definition image does not feel disjointed with the panoramic image, improving the user experience. This disclosure also allows users to switch the current screen content from the panoramic VR image to an additionally captured high-definition image in a smooth transition manner after clicking on a part of the scene while browsing the panoramic image.

[0056] like Figure 2 As shown above, in the above Figure 1 Based on the illustrated embodiment, step 102 may include the following steps: Step 1021: Perform feature extraction on the local image and the high-resolution image respectively to obtain at least one first feature point corresponding to the local image and at least one second feature point corresponding to the high-resolution image.

[0057] Optionally, the key points and descriptors are obtained by ORB feature extraction, which can specifically include: 1, key point detection (based on FAST corner, enhanced rotation and scale invariance): FAST corner detection: quickly locate the "corner points" (regions with sharp changes in pixel values, such as edge, texture junction) in the image, the specific method is: for each pixel, check its 16 neighborhood pixels, if the gray value of more than 9 consecutive pixels is different from the pixel by more than a threshold, it is determined as a corner point. Scale invariance optimization: by constructing an image pyramid (scaled versions of different scales), detect FAST corner points on each layer of the pyramid, to ensure that the same object features at different scales can be matched. Rotation invariance optimization: calculate the "main direction" for each key point (by the gray centroid method: calculate the centroid of the gray values in the neighborhood of the key point, and the direction of the connecting line between the centroid and the key point is the direction of the point), so that the subsequent descriptor can adapt to image rotation (such as single image and panoramic image view rotation can still be matched). 2, descriptor generation (enhanced rotation robustness): BRIEF descriptor: binary comparison of pixel pairs in the neighborhood of the key point (such as "pixel A gray > pixel B gray, then record 1, otherwise 0"), generate a 256-bit binary string (vector) as the "feature code" of the key point. Rotation adaptation: since the main direction of the key point has been calculated, the neighborhood will be rotated to the main direction before generating the BRIEF descriptor, to ensure that the descriptor is not sensitive to image rotation.

[0058] In another example, the local image and the high-definition image can be respectively subjected to feature extraction by a deep neural network (for example, a convolutional neural network), to respectively obtain corresponding image features. The image features obtained by the deep neural network can not have physical meaning, but are only intermediate products in the feature matching process.

[0059] In step 1022, matched matching point pairs are determined from the at least one first feature point and the at least one second feature point through feature matching.

[0060] Optionally, based on the above feature extraction process, the local map and the high-definition image respectively obtain a set of key points (coordinates + scales + directions) and corresponding ORB descriptors (256-bit binary strings). Feature matching is realized based on the similarity measurement of the descriptors, that is, by comparing the descriptors of the key points in the two images, the "similar features" are found to determine the points that may come from the same physical location. The matching process can include: brute-force matching: for each key point of a single image, all key points of the panoramic image are traversed, the Hamming distance is calculated, and the candidate matching with the smallest distance is retained. Screening optimization: using "nearest neighbor distance ratio (NNDR)" filtering: for each key point, take the two closest matching points in the other image, if the nearest distance / second nearest distance is less than the threshold (usually 0.75), the matching is retained (fuzzy matching is removed). Through screening, the matching point pairs can be obtained. In order to achieve better matching effect, a geometric verification process can also be added to remove false matches, for example, using the spatial relationship (such as the projection transformation of the overlapping area) between the panoramic image and the high-definition image to remove the false matching point pairs. The ORB feature matching feature point pairs are used, the calculation process is simple, the hardware requirement is low, and it is easier to implement.

[0061] Optionally, in some examples, feature matching can also be realized based on the image features extracted by the above deep neural network, based on the feature matching network model to obtain matching point pairs. Through deep learning, the matching efficiency of the matching points can be improved.

[0062] As shown in the above Figure 3 , based on the above-mentioned embodiment Figure 1 , step 104 can include the following steps: Step 1041, determining the world space coordinates of the local map in the world coordinate system.

[0063] In some optional embodiments, UV normalization processing is performed on the local map in the spherical coordinate system to obtain the three-dimensional spherical coordinates corresponding to the local map; The spherical coordinate system is the initial coordinate system of the panoramic image, which directly corresponds to the intuitive cognition of human beings to the 360° space. It takes the "collection center point" (such as the camera optical center of the collection point) of the panoramic image as the origin, and uses two angle parameters to describe the position of any point in the space, perfectly covering the full space range of "from the zenith to the ground seam, 360° horizontally"; including two core angle parameters: azimuth (Azimuth, denoted as ): describing the angle of the point on the "horizontal plane", corresponding to the "horizontal direction (Width)" of the panoramic image; and elevation (Elevation, denoted as ): describes the angle of a point on the "vertical plane", corresponding to the "vertical direction (Height)" of the panorama. Through UV normalization processing, the panorama can be converted from the spherical coordinate system to the three-dimensional spherical coordinate (azimuth [0, 2pi], pitch angle [0, pi]), UV normalization itself does not change the "spatial mapping relationship" of the panorama (still corresponding to the spherical coordinate system), but will produce three key results: result 1: uniform coordinate range, eliminating resolution dependence; result 2: establish a direct mapping between "normalized UV" and "spherical angle"; result 3: adapt to graphics / 3D algorithms, support texture sampling and spatial conversion, when the spatial direction of the panorama needs to be converted to the world coordinate system (X, Y, Z), it is necessary to first get , and and through normalized UV, combined with camera extrinsic parameters (position, attitude) calculation, UV normalization is the "standard input" of this process, this embodiment supports texture sampling and spatial conversion through UV normalization to realize the conversion of the local map to the world coordinate system.

[0064] UV coordinates are "pixel position descriptions" of planar images: U corresponds to horizontal direction pixel index (such as from left to right ), V corresponds to vertical direction pixel index (such as from bottom to top ); and the core of UV normalization is to convert these two "absolute pixel indices" into "relative normalized values" (usually mapped to [0, 1] or [-1, 1] interval), eliminating the influence of image resolution (pixel number of width and height).

[0065] Convert the three-dimensional spherical coordinate to the world coordinate system to get the world space coordinate corresponding to the local map.

[0066] Before performing the coordinate conversion of the three-dimensional spherical coordinate to the world coordinate system, the extrinsic matrix of the image acquisition device (for example, the camera) corresponding to the acquisition of the panorama is obtained, and after UV normalization, the three-dimensional spherical coordinate ( , The vector describes the direction from the camera to the target pixel, but it needs to be quantized into a 3D vector in the camera coordinate system before subsequent coordinate transformations can be performed; for example, it can be converted into a unit direction vector in the camera coordinate system using trigonometric functions. Then, the unit vector in the camera coordinate system is transformed to the world coordinate system. The camera coordinate system is a "local coordinate system," while the world coordinate system is a "global coordinate system." The pose differences between the two (such as camera tilt and rotation) need to be corrected using a rotation matrix R—the role of the rotation matrix is ​​to convert the "local camera direction" into the "global world direction."

[0067] Step 1042: Based on the extrinsic parameter matrix of the virtual camera corresponding to the display device, transform the world space coordinates of the local image to the screen coordinate system to obtain the local plane coordinates corresponding to the local image.

[0068] In this embodiment, the extrinsic parameter matrix of the virtual camera can describe the pose and position of the virtual camera in the world coordinate system. Therefore, based on the extrinsic parameter matrix, the transformation relationship between the camera coordinate system and the world coordinate system corresponding to the virtual camera can be determined, and the local image can be transformed from the world coordinate system to the camera coordinate system, and then from the camera coordinate system to the screen coordinate system. Optionally, the image can be transformed from the camera coordinate system to the screen coordinate system by converting the 3D camera coordinates to 2D image coordinates based on the memory of the virtual camera, that is, transforming the camera coordinate system to the screen coordinate system.

[0069] Step 1043: Based on the screen information of the display device, perform adaptive scaling on the high-definition image to determine the planar coordinates of the high-definition image in the screen coordinate system.

[0070] Optionally, in this embodiment, the screen information of the display device includes at least the physical size of the screen. Since a high-definition image is itself a two-dimensional image, when it needs to be displayed on the display screen, it is only necessary to adaptively scale the high-definition image according to the physical size of the screen so that the scaled high-definition image can be fully displayed on the display screen. This embodiment transforms the partial image corresponding to the high-definition image in the panoramic image and the high-definition image to the screen coordinate system, providing a basis for realizing the transition from the partial image to the high-definition image on the screen of the display device.

[0071] like Figure 4 As shown above, in the above Figure 1 Based on the illustrated embodiment, step 106 may include the following steps: Step 1061: Construct a homogeneous system of equations based on the coordinates of the matching point pairs in the screen coordinate system.

[0072] Each pair of matching points generates constraint equations in two rows of homogeneous equations, with the x-axis coordinates and y-axis coordinates each forming a row of constraint equations.

[0073] In this embodiment, there are multiple pairs of matching points between the local map and the high-definition image. Since the homography matrix is a 3x3 homogeneous matrix, a blank 3x3 homogeneous matrix is first constructed, and a constraint equation in the homogeneous equation is generated through each pair of matching points. The homography matrix actually has 8 degrees of freedom (not 9), and needs to be solved through 8 equations, that is, at least 4 pairs of matching points are needed to generate 8 equations, which are sufficient to solve the 8-degree-of-freedom matrix elements. In practical applications, in order to reduce the influence of noise, more than 4 pairs of matching points are usually used. Alternatively, the RANSAC algorithm can also be used to remove the mis-matching points, and then the least squares method is used to solve the optimal matrix.

[0074] In step 1062, the homogeneous equation set is solved to determine the first homography matrix between the local map and the high-definition image.

[0075] Alternatively, singular value decomposition is performed on the homogeneous equation set to obtain the coefficient values in the homogeneous equation set and obtain the initial matrix.

[0076] In an optional example, if 4 pairs of matching points are known where i takes values 1, 2, 3, and 4; each pair of matching points can list the equation shown in formula (1) as follows: Formula (1) Taking i in formula (1) as 1, 2, 3, and 4 respectively, 8 equations are obtained, and the unknown is substituted to obtain an 8x9 matrix A to form an equation set Ah=0.

[0077] SVD decomposition is performed on the matrix A through a numerical calculation library (such as NumPy, OpenCV): to obtain a singular value matrix and a right singular matrix V. The smallest singular value in is found, and the column vector of V corresponding to the smallest singular value (usually the last column of V) is the solution vector h of the equation set.

[0078] The initial matrix is proportionally normalized to obtain the first homography matrix.

[0079] Due to the scale independence of the homogeneous solution, in this embodiment, in order to eliminate the scale uncertainty, the solution vector h is normalized after obtaining the initial matrix, for example, After normalization, the first homography matrix is obtained. The method for determining the first homography matrix in this embodiment has high stability and good precision, and is more suitable for processing redundant equations (since the number of matching points is large, the number of equations > unknowns).

[0080] As shown in formula (2) as follows, Figure 5 where h is the solution vector of the equation set, and the first homography matrix is obtained by normalizing h. Figure 1On the basis of the embodiment shown, the second homography matrix is a unit matrix, and step 108 can include the following steps: including: Step 1081, constructing an off-screen rendering channel, and rendering the rendering parameters of the local map to an off-screen texture through the off-screen rendering channel.

[0081] Optionally, a WebGLRenderTarget object is created to take over the main rendering stream, and the panorama picture (local map) that is originally rendered to the screen is first rendered to the off-screen WebGLRenderTarget for storage (specifically including: creating a WebGLRenderTarget, rendering the picture that is originally rendered to the screen to an off-screen texture first, and implementing "taking over the main rendering stream"); then it is decided whether to display or further process, to realize the preservation of the original panorama picture, and the rendering target is temporarily switched to the off-screen texture by modifying the setRenderTarget method of the renderer, to complete the preservation of the original panorama picture (local map). The WebGLRenderTarget is a core object in Three.js for off-screen rendering, which allows developers to render a 3D scene to a texture in memory, rather than directly displaying it on the screen's canvas. Through off-screen rendering, the original picture is preserved in the off-screen texture, avoiding the problem of being covered by subsequent pictures when directly rendered to the screen, and enabling complex visual effects to be achieved without damaging the original picture.

[0082] Step 1082, performing interpolation processing on the first homography matrix and the second homography matrix to obtain at least one intermediate matrix.

[0083] In this embodiment, the second homography matrix is a unit matrix, i.e., the second homography matrix represents the projection transformation relationship between the local map and the local map; and the first homography matrix represents the projection transformation relationship between the local map and the high-definition image. In order to realize the non-linear transition from the local map to the high-definition image, the non-linear transition of the homography matrix needs to be realized, which requires the combination of linear interpolation (lerp) and cubic easing function (Cubic Easing Function). The linear interpolation function (lerpHomography) of the homography matrix is used to weight and mix the elements in the first homography matrix and the second homography matrix, and at least one intermediate matrix can be obtained.

[0084] Step 1083, based on the at least one intermediate matrix, smoothly transitioning from the local map to the high-definition image.

[0085] Optionally, the smooth transition can be implemented based on a cubic easing function, where the cubic easing function is a kind of non-linear time interpolation function, and the core function is to convert "linear elapsed time" into "non-linearly changing state progress". In this embodiment, the homography matrix transition is implemented through the cubic easing function.

[0086] At step 1084, the high-definition image is rendered to the display device for display through the texture shader.

[0087] Optionally, the texture shader can be a multi-texture blending shader. The working principle of the rendering process includes: 1. Double-channel UV mapping implementation: the vertex shader transmits the basic UV coordinates to the fragment shader, and the fragment shader uses the imported H matrix and its inverse matrix to perform coordinate transformation through the applyHomography function. The transformation process takes into account the homogeneous coordinate characteristics to ensure correct UV coordinate mapping; 2. Dynamic blending control: receive the progress parameter (0.0 to 1.0) to control the blending progress, and use the cubic easing function to calculate the alpha value to make the transition more natural, and realize the alpha blending formula where C1 is the color of the high-definition image, and C2 is the color of the local image in the panoramic image; 3. Edge processing mechanism: the edge mask of the close-up image is realized through the getFeatureMask function, the alpha channel of the close-up image and the edge distance are combined to calculate the mask, and a smooth transition is added to the edge of the high-definition image (for example, within a range of 0.02), thereby enhancing the visual continuity with the panoramic image. This embodiment enhances the user experience through the smooth transition while maintaining the visual continuity.

[0088] In some optional embodiments, before step 108 is performed, the following steps can also be included: Based on the intrinsic and extrinsic parameters of the virtual camera corresponding to the display device, the camera viewing pyramid corresponding to the display device is determined.

[0089] The camera frustum needs to combine the intrinsic parameters (such as focal length, resolution) and extrinsic parameters (such as position, orientation) of the virtual camera to define the "3D space range that the camera can see" through geometric calculation, which is the "viewfinder" connecting the virtual 3D scene and the 2D display. The structure of the camera frustum is a prism structure surrounded by the virtual camera position (vertex) and the "near clipping plane" and "far clipping plane", which contains 6 planes: the near clipping plane: a virtual plane in front of the camera, with a distance of near (such as 0.1m) from the camera optical center, all objects in front of the near clipping plane will be culled; the far clipping plane: a virtual plane further in front of the camera, with a distance of far (such as 1000m) from the camera optical center, all objects behind the far clipping plane will be culled; the upper, lower, left and right clipping planes: the side planes of the frustum, determined by the "field of view angle" of the camera and the aspect ratio of the screen, objects beyond these planes will be culled. Optionally, based on the determination of the intrinsic and extrinsic parameters of the virtual camera, the aspect ratio is calculated according to the resolution of the display device; the "opening angle" of the frustum is determined in combination with the expected field of view angle (or derived through the intrinsic parameters); the near / far clipping plane distance is set, and the boundary size of the frustum is calculated; through the visualization tool, the adjustment is verified to ensure that the frustum matches the display device and the scene, and the camera frustum corresponding to the display device is determined.

[0090] The local map is projected into the camera frustum by inverse projection.

[0091] In this embodiment, it is necessary to establish the inverse mapping between the pixels in the camera frustum and the panorama texture. The panorama is usually stored in equirectangular projection, which essentially maps the 3D points (x, y, z) of the spherical panorama to the (u, v) coordinates of the 2D texture. For each pixel in the camera frustum (screen coordinates), its corresponding point on the panorama sphere is calculated inversely through the camera parameters, and then converted to the texture coordinates of the panorama, and finally the panorama color is sampled.

[0092] The field of view angle corresponding to the virtual camera is adjusted so that the feature points in the matching point pairs included in the local map are projected within the effective area of the screen, and the target parameter matrix of the virtual camera is obtained.

[0093] Optionally, within a preset field of view angle adjustment range, a binary search method is performed with a preset step value to dynamically adjust the field of view angle, so that the feature points in the matching point pairs included in the local map are projected within the effective area of the screen. Based on the dynamically adjusted field of view angle, the target parameter matrix is determined.

[0094] Optionally, the preset step size can be set according to specific application scenarios, for example, 0.5°, etc.; the larger the field of view (FOV) is, the wider the camera frustum is, and the more feature points can be accommodated (the less likely to exceed the screen); the smaller the FOV is, the narrower the frustum is, and the more likely the feature points are to exceed. Therefore, the goal is to find the minimum FOV such that all feature points are projected within the screen. To determine whether the feature points are in the effective area of the screen, the three-dimensional points are projected to the screen coordinate system through the FOV corresponding frustum, and the FOV adjustment threshold (FOV_adjust_threshold) is set in the embodiment to prevent numerical overflow, and the camera parameter matrix at the optimal solution is recorded; on the basis of determining the adjustment threshold, the iterative search is performed through the bisection method until the minimum field of view angle at which all feature points are projected within the effective area of the screen is determined, the field of view angle in the initial parameter matrix corresponding to the display device is replaced based on the field of view angle, and the target parameter matrix is obtained.

[0095] In some optional embodiments, the embodiment can further include: In response to triggering the exit operation, a smooth transition from the high-definition image to the local image is implemented based on the first homography matrix and the second homography matrix, and the local image is displayed on the display device.

[0096] The embodiment realizes switching from the panoramic image to the high-definition image corresponding to the local in the panoramic image, and realizes visual transition in the switching process. The geometric correction of the homography matrix provides a mathematical basis for the visual transition, the FOV adaptive algorithm ensures the visual consistency of feature matching, and the matrix interpolation and the shader mixing jointly realize the smooth switching effect without perception. After switching to the high-definition image, the inverse process of the above switching can be performed according to the exit operation (for example, receiving the exit instruction of the user), the camera pose matrix, the FOV parameter, and the renderer original state are cached at the switching initial time; when the exit operation is triggered, the reverse homography matrix interpolation calculation is started, the first homography matrix is gradually restored to the unit matrix, and the FOV is synchronously rolled back to the initial value; the high-definition image is switched back to the panoramic image, after the switching, the off-screen rendering binding is released, the custom shader material is removed, the camera matrix automatic update logic is restarted, and the user interaction function in the panoramic image scene is restored. After the interaction function is restored, in some optional embodiments, the method provided by the embodiment can further include: In response to receiving a movement instruction through the display device, the panoramic image is controlled to perform perspective conversion according to the movement instruction.

[0097] In the embodiment, after the exit operation is executed, the panoramic image is displayed on the display device. At this time, the same scene as displaying the panoramic image on the display device can receive a movement instruction, and the perspective of the panoramic image displayed on the display device is switched according to the movement instruction.

[0098] ​It should be noted that, in the case of no conflict, the technical features described in different optional implementations can be included in the same embodiment. For brevity, the description is not repeated here.

[0099] Any one of the image transition switching methods provided by the embodiments of the present disclosure can be executed by any appropriate device with data processing capability, including but not limited to: terminal devices and servers, etc. Alternatively, any one of the image transition switching methods provided by the embodiments of the present disclosure can be executed by a processor, such as a processor executing any one of the image transition switching methods mentioned by the embodiments of the present disclosure by calling corresponding instructions stored in a memory. The following will not be repeated.

[0100] Exemplary device Figure 6 is a structural schematic diagram of an image transition switching device provided by an exemplary embodiment of the present disclosure. As shown in Figure 6 The device provided by the embodiment includes: The feature point matching module 61 is configured to determine a matching point pair between a local image in a panorama of a preset scene and a high-definition image.

[0101] The coordinate system conversion module 62 is configured to obtain relevant information of a display device, and convert the local image and the high-definition image into screen coordinate systems corresponding to the display device respectively.

[0102] The mapping relationship module 63 is configured to determine a first homography matrix between the local image and the high-definition image based on coordinate information of the matching point pair in the screen coordinate system.

[0103] The image transition module 64 is configured to realize smooth transition from the local image to the high-definition image based on the first homography matrix and a second homography matrix, and display the high-definition image on the display device.

[0104] The transition switching device of the image provided by the above-mentioned embodiment of the present disclosure determines a matching point pair between a local graph in a panoramic graph of a preset scene and a high-definition image; obtains relevant information of a display device, and converts the local graph and the high-definition image to a screen coordinate system corresponding to the display device respectively; determines a first homography matrix between the local graph and the high-definition image based on coordinate information of the matching point pair in the screen coordinate system; and realizes smooth transition from the local graph to the high-definition image based on the first homography matrix and a second homography matrix, and displays the high-definition image on the display device. According to the embodiment of the present disclosure, a high-definition image corresponding to all or part of a local graph is provided, the high-definition image provides more detailed information, the local graph and the high-definition image are matched through a matching point pair, and smooth transition from the local graph to the high-definition image is realized through a first homography matrix and a second homography matrix, thereby meeting the needs of users for viewing local details in a panoramic graph, and the display process of the high-definition image is not disconnected from the panoramic graph through smooth transition, thereby improving the user experience. According to the embodiment of the present disclosure, when a user clicks on a certain part of a scene during browsing of a panoramic graph, the current screen content is switched from a panoramic graph VR to an extra high-definition image in a smooth and fluent transition manner.

[0105] In some optional embodiments, the feature point matching module 61 is specifically configured to perform feature extraction on the local graph and the high-definition image respectively to obtain at least one first feature point corresponding to the local graph and at least one second feature point corresponding to the high-definition image; and determine the matching point pair by matching the at least one first feature point and the at least one second feature point.

[0106] In some optional embodiments, the coordinate system conversion module 62 is specifically configured to determine a world space coordinate of the local graph in a world coordinate system; convert the world space coordinate of the local graph to a screen coordinate system based on an extrinsic parameter matrix of a virtual camera corresponding to the display device to obtain a local plane coordinate corresponding to the local graph; and perform adaptive scaling on the high-definition image based on screen information of the display device to determine a plane coordinate of the high-definition image in the screen coordinate system.

[0107] Optionally, when determining the world space coordinate of the local graph in the world coordinate system, the coordinate system conversion module 62 is configured to perform UV normalization processing on the local graph in a spherical coordinate system to obtain a three-dimensional spherical coordinate corresponding to the local graph; and convert the three-dimensional spherical coordinate to the world coordinate system to obtain the world space coordinate corresponding to the local graph.

[0108] In some optional embodiments, the mapping relationship module 63 is specifically configured to construct a system of homogeneous equations based on the coordinates of the matching point pair in the screen coordinate system; wherein each matching point pair generates a constraint equation in two rows of the system of homogeneous equations; and solve the system of homogeneous equations to determine the first homography matrix between the local graph and the high-definition image.

[0109] Optionally, the mapping relationship module 63 is configured to perform singular value decomposition on the alignment equation set to obtain coefficient values in the alignment equation set and obtain an initial matrix when determining the first homography matrix between the local image and the high-definition image; and perform scale normalization on the initial matrix to obtain the first homography matrix.

[0110] In some optional embodiments, the image transition module 64 is specifically configured to construct an off-screen rendering channel, render the rendering parameters of the local image to an off-screen texture through the off-screen rendering channel, perform interpolation processing on the first homography matrix and the second homography matrix to obtain at least one intermediate matrix, and based on the at least one intermediate matrix, smoothly transition from the local image to the high-definition image, and render the high-definition image to a display device through a texture shader for display.

[0111] In some optional embodiments, the apparatus provided in the embodiments can further include: The frustum determination module is configured to determine a camera frustum corresponding to the display device based on the intrinsic parameters and extrinsic parameters of the virtual camera corresponding to the display device.

[0112] The inverse projection module is configured to project the local image to the camera frustum through inverse projection.

[0113] The field of view angle adjustment module is configured to adjust the field of view angle corresponding to the virtual camera so that the feature points in the matching point pairs included in the local image are projected within the effective area of the screen to obtain a target parameter matrix of the virtual camera.

[0114] Optionally, the field of view angle adjustment module is specifically configured to perform dynamic adjustment on the field of view angle within a preset field of view angle adjustment range at a preset step value through a binary search method so that the feature points in the matching point pairs included in the local image are projected within the effective area of the screen; and determine the target parameter matrix based on the dynamically adjusted field of view angle.

[0115] In some optional embodiments, the apparatus provided in the embodiments can further include: The smooth exit module is configured to, in response to triggering an exit operation, based on the first homography matrix and the second homography matrix, realize smooth transition from the high-definition image to the local image, and display the local image on the display device.

[0116] Exemplary electronic device Below, with reference to Figure 7 an electronic device according to embodiments of the present disclosure will be described. The electronic device can be either or both of the first device and the second device, or a single device independent of them, which can communicate with the first device and the second device to receive the collected input signals therefrom.

[0117] Figure 7FIG. 1 illustrates a block diagram of an electronic device according to an embodiment of the disclosure.

[0118] As shown in Figure 7 The electronic device includes one or more processors and a memory.

[0119] The processor can be a central processing unit (CPU) or other form of processing unit having data processing and / or instruction execution capabilities, and can control other components in the electronic device to perform desired functions.

[0120] The memory can store one or more computer program products, which can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), cache, and / or the like. The non-volatile memory, for example, can include read-only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program products can be stored on the computer-readable storage media, and the processor can execute the computer program products to implement the transition switching method of an image and / or other desired functions of various embodiments of the disclosure described above.

[0121] In one example, the electronic device can further include an input device and an output device, which are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0122] In addition, the input device can further include, for example, a keyboard, a mouse, and / or the like.

[0123] The output device can output various information, including determined distance information, direction information, and / or the like, to the outside. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and / or the like.

[0124] Of course, in order to simplify, Figure 7 In FIG. 1, only some of the components related to the disclosure among the electronic device are shown, and components such as a bus, an input / output interface, and / or the like are omitted. In addition to this, the electronic device can further include any other appropriate components according to a specific application.

[0125] In addition to the above-described method and device, an embodiment of the disclosure can be a computer program product including computer program instructions, which, when executed by a processor, cause the processor to perform steps of the transition switching method of an image according to various embodiments of the disclosure described in the above parts of the specification.

[0126] The computer program product can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server.

[0127] Furthermore, embodiments of the present disclosure can also be a computer readable storage medium, having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the steps described in the foregoing disclosure of the method for transition switching of images according to various embodiments of the present disclosure.

[0128] The computer readable storage medium can be any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0129] The above describes the basic principles of the present disclosure in combination with specific embodiments, but it should be noted that the advantages, benefits, effects and the like mentioned in the present disclosure are only examples and are not limiting, and these advantages, benefits, effects and the like cannot be considered as the must-haves of each embodiment of the present disclosure. In addition, the above specific details are only for the purpose of example and understanding, and the above details do not limit the present disclosure to the must-haves of the above specific details.

[0130] Each embodiment in the present specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between each embodiment can be understood by mutual reference. For system embodiments, since they basically correspond to method embodiments, the description is relatively simple, and the relevant parts can be understood by referring to the part of the method embodiment.

[0131] The block diagrams of devices, apparatuses, equipment, systems referred to in this disclosure are merely illustrative examples and are not intended to require or imply that the connection, arrangement, configuration must be as shown in the block diagrams. These devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner as will be appreciated by those skilled in the art. Words such as "include," "contain," "have," and the like are open-ended words that are to be interpreted to mean "including but not limited to," and are not to be interpreted as limiting the described embodiment to features, elements, and / or steps disclosed herein. The words "or" and "and" as used herein are to be interpreted as the word "and / or," and are not to be interpreted as requiring both features, elements, and / or steps disclosed herein. The word "such as" as used herein is to be interpreted as the phrase "such as but not limited to," and is not to be interpreted as limiting the described embodiment to features, elements, and / or steps disclosed herein.

[0132] The methods and apparatuses of this disclosure can be implemented in a number of ways. For example, the methods and apparatuses of this disclosure can be implemented using software, hardware, firmware, or any combination of these. The above described order of steps for the methods is merely illustrative, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, the disclosure can also be implemented as a program recorded in a recording medium, which includes machine readable instructions for implementing the methods according to the disclosure. Thus, the disclosure also covers a recording medium storing a program for executing the methods according to the disclosure.

[0133] It is also important to note that the devices, equipment, and methods of this disclosure can be embodied in a variety of ways. These variations are contemplated as being within the scope of the present disclosure.

[0134] The above description of the disclosed aspects is given for illustrative purposes and is not intended to limit the scope of the disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0135] The above description has been given for illustrative and descriptive purposes. In addition, this description is not intended to limit embodiments of the disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain modifications, permutations, additions, and sub-combinations thereof.

Claims

1. A method for transitioning images, characterized in that, include: Identify matching point pairs between a local view in the panoramic view of a preset scene and a high-resolution image; Obtain relevant information about the display device, and convert the partial image and the high-definition image to the screen coordinate system corresponding to the display device; Based on the coordinate information of the matching point pairs in the screen coordinate system, a first homography matrix between the local image and the high-definition image is determined. Based on the first and second homography matrices, a smooth transition from the local image to the high-definition image is achieved, and the high-definition image is displayed on the display device.

2. The method according to claim 1, characterized in that, The step of acquiring relevant information about the display device and converting the partial image and the high-definition image to the screen coordinate system corresponding to the display device includes: Determine the world space coordinates of the local graph in the world coordinate system; Based on the extrinsic parameter matrix of the virtual camera corresponding to the display device, the world space coordinates of the local image are transformed to the screen coordinate system to obtain the local plane coordinates corresponding to the local image. Based on the screen information of the display device, adaptive scaling is performed on the high-definition image to determine the planar coordinates of the high-definition image in the screen coordinate system.

3. The method according to claim 2, characterized in that, Determining the world space coordinates of the local graph in the world coordinate system includes: Perform UV normalization on the local image in the spherical coordinate system to obtain the three-dimensional spherical coordinates corresponding to the local image; The three-dimensional spherical coordinates are transformed to the world coordinate system to obtain the world space coordinates corresponding to the local graph.

4. The method according to any one of claims 1-3, characterized in that, Determining the first homography matrix between the local image and the high-resolution image based on the coordinate information of the matching point pairs in the screen coordinate system includes: A homogeneous system of equations is constructed based on the coordinates of the matching point pairs in the screen coordinate system; wherein, each pair of matching points generates two rows of constraint equations in the homogeneous system. Solve the homogeneous system of equations to determine the first homography matrix between the local image and the high-resolution image.

5. The method according to claim 4, characterized in that, Solving the homogeneous system of equations to determine the first homography matrix between the local image and the high-resolution image includes: Singular value decomposition is performed on the homogeneous system of equations to obtain the coefficient values ​​in the homogeneous system of equations, thus obtaining the initial matrix; The initial matrix is ​​proportionally normalized to obtain the first homography matrix.

6. The method according to any one of claims 1-5, characterized in that, The second homography matrix is ​​an identity matrix. The step of achieving a smooth transition from the local image to the high-definition image based on the first and second homography matrices, and displaying the high-definition image on the display device, includes: Construct an off-screen rendering channel, and render the rendering parameters of the local image to an off-screen texture through the off-screen rendering channel; Interpolation is performed on the first homography matrix and the second homography matrix to obtain at least one intermediate matrix; Based on the at least one intermediate matrix, a smooth transition is achieved from the local image to the high-resolution image; The high-definition image is rendered onto the display device using a texture shader for display.

7. The method according to any one of claims 1-6, characterized in that, Before displaying the high-definition image on the display device, and achieving a smooth transition from the local image to the high-definition image based on the first and second homography matrices, the method further includes: Based on the intrinsic and extrinsic parameters of the virtual camera corresponding to the display device, the view frustum of the camera corresponding to the display device is determined; The local image is projected onto the camera's field of view using reverse projection; Adjust the field of view of the virtual camera so that the feature points in the matching point pairs included in the local image are projected onto the effective area of ​​the screen, thereby obtaining the target parameter matrix of the virtual camera.

8. The method according to claim 7, characterized in that, The step of adjusting the field of view of the virtual camera, so that the feature points in the matching point pairs included in the local image are projected onto the effective area of ​​the screen, to obtain the target parameter matrix of the virtual camera, includes: Within the preset field of view adjustment range, the field of view is dynamically adjusted by performing a binary search method with a preset step size value, so that the feature points in the matching point pairs included in the local image are projected into the effective area of ​​the screen. The target parameter matrix is ​​determined based on the dynamically adjusted field of view.

9. The method according to any one of claims 1-8, characterized in that, The process of determining the matching point pairs between the partial image in the panoramic image of the preset scene and the high-definition image includes: Feature extraction is performed on the local image and the high-resolution image respectively to obtain at least one first feature point corresponding to the local image and at least one second feature point corresponding to the high-resolution image; By feature matching, the matching point pair is determined from the at least one first feature point and the at least one second feature point.

10. The method according to any one of claims 1-9, characterized in that, Also includes: In response to a triggered exit operation, a smooth transition from the high-definition image to the partial image is achieved based on the first and second homography matrices, and the partial image is displayed on the display device.

Images