Image processing method, device, electronic device, storage medium and program product

Abstract

The embodiments of the present application provide an image processing method, device, electronic device, storage medium and program product, which relate to the field of image processing. The method includes: receiving an image viewing request, the image viewing request includes an angle identifier of a viewing angle to be viewed; loading a compressed first sub-image and a second sub-image from a pre-stored image set according to the angle identifier, obtaining a loaded target first sub-image and a target second sub-image, the image set including at least one compressed first sub-image and at least one second sub-image corresponding to at least one first sub-image before compression; rendering the target first sub-image, and splicing the rendered target first sub-image with the target second sub-image to obtain a viewing image and display it. In the present application, since the image is divided into multiple first sub-images for loading, data request acceleration is achieved, and only the target first sub-image can be rendered, which improves rendering efficiency.

Description

Technical Field

[0001] The present application relates to technical fields such as image cutting technology, image compression technology, network resource loading technology, and three-dimensional rendering technology in IT (Internet Technology). Specifically, the present application relates to an image processing method, device, electronic device, storage medium, and program product. Background Art

[0002] At present, users usually use images (such as panoramic images) to better record their lives. There are currently many ways to load high-definition panoramas, such as directly loading and rendering a single original high-definition panorama, or requesting a low-resolution blurred panorama for the first load, and then replacing the original low-resolution blurred panorama after the high-definition panorama is loaded, or cutting the high-definition panorama into multiple small images. Each time an image is loaded, the browser supports the feature of concurrent loading of network resources to request multiple images at the same time, and then the cut images are stitched into a complete high-definition panorama according to the agreed rules. However, research has found that the above methods have problems such as long waiting time and easy frame drops in the picture, which reduces the user experience. Summary of the invention

[0003] The embodiments of the present application provide an image processing method, device, electronic device, storage medium and program product, which can solve the problems of long waiting time and easy frame drop in the picture when loading high-definition panoramic images in the prior art. The technical solution is as follows:

[0004] According to one aspect of an embodiment of the present application, there is provided an image processing method, the method comprising:

[0005] receiving an image viewing request, wherein the image viewing request includes an angle identifier of a viewing angle to be viewed;

[0006] Loading a compressed first sub-image and a second sub-image from a pre-stored image set according to the angle identifier to obtain a loaded target first sub-image and a target second sub-image, wherein the image set includes at least one compressed first sub-image and at least one second sub-image corresponding to at least one first sub-image before compression, and each first sub-image before compression corresponds to an angle;

[0007] The target first sub-image is rendered, and the rendered target first sub-image is spliced ​​with the target second sub-image to obtain a viewing image and display it.

[0008] Optionally, before receiving the image viewing request, the method further includes:

[0009] Acquire an image to be processed and a blurred image corresponding to the image to be processed, wherein the blurred image is an image obtained by reducing the definition of the image to be processed;

[0010] Performing at least one image segmentation on the image to be processed to obtain at least one first sub-image;

[0011] Performing at least one image segmentation on the blurred image according to at least one first sub-image to obtain at least one second sub-image;

[0012] At least one first sub-image and at least one second sub-image are stored to obtain an image set.

[0013] Optionally, the image to be processed is an unfolded plane image of a sphere model, and the image to be processed is segmented at least once to obtain at least one first sub-image, including:

[0014] Mapping the image to be processed into a spherical model, and projecting the image to be processed mapped into the spherical model at at least one angle to obtain a projection result corresponding to the at least one angle;

[0015] The image to be processed is segmented according to the projection result corresponding to at least one angle to obtain at least one first sub-image.

[0016] Optionally, at least one compressed first sub-image is obtained by:

[0017] At least one first sub-image is compressed into a file of a set format to obtain at least one compressed first sub-image.

[0018] Optionally, loading the compressed first sub-image and the second sub-image from a pre-stored image set according to the angle identifier to obtain the loaded target first sub-image and the target second sub-image includes:

[0019] Loading a compressed first sub-image corresponding to the angle identifier from a pre-stored image set as a target first sub-image;

[0020] The other second sub-images except the second sub-image corresponding to the target first sub-image in the at least one second sub-image are loaded from the pre-stored image set as the target second sub-image.

[0021] Optionally, rendering the target first sub-image and splicing the rendered target first sub-image with the target second sub-image to obtain a viewing image includes:

[0022] Decoding and converting the target first sub-image to obtain the target first sub-image in a specified format, and rendering the target first sub-image in the specified format to obtain a rendered target first sub-image;

[0023] The rendered target first sub-image and target second sub-image are spliced ​​to obtain a viewing image.

[0024] According to another aspect of an embodiment of the present application, there is provided an image processing device, the device comprising:

[0025] A request receiving module, used to receive an image viewing request, wherein the image viewing request includes an angle identifier of a viewing angle to be viewed;

[0026] An image loading module is used to load the compressed first sub-image and the second sub-image from a pre-stored image set according to the angle identifier to obtain a loaded target first sub-image and a target second sub-image, wherein the image set includes at least one compressed first sub-image and at least one second sub-image corresponding to at least one first sub-image before compression, and each first sub-image before compression corresponds to an angle;

[0027] The image stitching module is used to render the target first sub-image and stitch the rendered target first sub-image with the target second sub-image to obtain and display a viewing image.

[0028] Optionally, the device further includes an image segmentation module, which is used to:

[0029] Before receiving an image viewing request, obtaining an image to be processed and a blurred image corresponding to the image to be processed, wherein the blurred image is an image obtained by reducing the definition of the image to be processed;

[0030] Performing at least one image segmentation on the image to be processed to obtain at least one first sub-image;

[0031] Performing at least one image segmentation on the blurred image according to at least one first sub-image to obtain at least one second sub-image;

[0032] At least one first sub-image and at least one second sub-image are stored to obtain an image set.

[0033] Optionally, the image to be processed is an unfolded plane image of a sphere model, and when the image segmentation module performs at least one image segmentation on the image to be processed to obtain at least one first sub-image, it is specifically used to:

[0034] Mapping the image to be processed into a spherical model, and projecting the image to be processed mapped into the spherical model at at least one angle to obtain a projection result corresponding to the at least one angle;

[0035] The image to be processed is segmented according to the projection result corresponding to at least one angle to obtain at least one first sub-image.

[0036] Optionally, at least one compressed first sub-image is obtained by:

[0037] At least one first sub-image is compressed into a file of a set format to obtain at least one compressed first sub-image.

[0038] Optionally, when the image loading module loads the compressed first sub-image and the second sub-image from the pre-stored image set according to the angle identifier to obtain the loaded target first sub-image and the target second sub-image, it is specifically used to:

[0039] Loading a compressed first sub-image corresponding to the angle identifier from a pre-stored image set as a target first sub-image;

[0040] The other second sub-images except the second sub-image corresponding to the target first sub-image in the at least one second sub-image are loaded from the pre-stored image set as the target second sub-image.

[0041] Optionally, when the image stitching module renders the target first sub-image and stitches the rendered target first sub-image with the target second sub-image to obtain the viewing image, it is specifically used to:

[0042] Decoding and converting the target first sub-image to obtain the target first sub-image in a specified format, and rendering the target first sub-image in the specified format to obtain a rendered target first sub-image;

[0043] The rendered target first sub-image and target second sub-image are spliced ​​to obtain a viewing image.

[0044] According to another aspect of an embodiment of the present application, an electronic device is provided. The electronic device includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement any step in the image processing method.

[0045] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any step in the image processing method is implemented.

[0046] According to one aspect of an embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, any step in the image processing method is implemented.

[0047] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0048] In an embodiment of the present application, when loading an image, the compressed first sub-image and the second sub-image can be loaded from the pre-stored image set according to the received angle identifier, and then the target first sub-image is rendered, and the rendered target first sub-image and the target second sub-image are spliced ​​to obtain a viewing image and then displayed to the user. In this process, since the image is divided into multiple first sub-images for loading, data request acceleration is achieved, which effectively solves the problem of long user waiting time, and since the first sub-image obtained after division is a compressed image, only the compressed target first sub-image can be rendered during subsequent image rendering, which improves rendering efficiency, and the image loading will be smoother, further improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0050] Figure 1 A schematic diagram of a system architecture for implementing an image processing method provided in an embodiment of the present application;

[0051] Figure 2 A flowchart of an image processing method provided in an embodiment of the present application;

[0052] Figure 3a A schematic diagram of an image to be processed provided in an embodiment of the present application;

[0053] Figure 3b A schematic diagram of a sphere model provided in an embodiment of the present application;

[0054] Figure 3c A schematic diagram of a projection result provided in an embodiment of the present application;

[0055] Figure 3d A schematic diagram of image segmentation provided in an embodiment of the present application;

[0056] Figure 4 A schematic diagram of image compression provided by an embodiment of the present application;

[0057] Figure 5 A schematic diagram of image loading provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of image stitching provided in an embodiment of the present application;

[0059] Figure 7 A schematic diagram of decoding a basis format file provided in an embodiment of the present application;

[0060] Figure 8A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0061] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0063] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" may be implemented as "A", or as "B", or as "A and B".

[0064] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0065] Currently, users usually use images (such as panoramic images) to better record their lives. There are many solutions for loading high-definition panoramic images in the prior art. However, research has found that the following problems exist in loading high-definition panoramic images in the prior art:

[0066] 1. The file size of a single high-definition panoramic image is large, and the network request time is very long. Users often have to wait for a long time to see the expected picture.

[0067] 2. Although low-resolution images can be loaded and displayed quickly on the screen, users will still see blurry images for a long time.

[0068] 3. Image cutting solves the problem of long loading time of high-definition panoramas to a certain extent, but it increases the transmission steps of rendering data during 3D rendering, which will cause frame drops in the picture content.

[0069] Based on this, the present application provides an image processing method, device, electronic device, storage medium and program product, aiming to solve the above technical problems of the prior art.

[0070] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.

[0071] Optionally, the image processing method provided by the present application can be executed by any electronic device, including but not limited to a terminal device or a server. Optionally, the image processing method provided by the embodiment of the present application can be implemented as an independent application or as a functional module / plug-in of an existing application. By applying the image processing provided by the embodiment of the present application to the application, when receiving a user's image viewing request, the server of the application can determine the display image by executing the computer program corresponding to the functional module / plug-in, and then send the display image to the terminal device, so that the display image can be displayed to the user through the terminal device.

[0072] In the embodiment of the present application, the specific terminal device is not limited in the embodiment of the present application, and may include but not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, wearable electronic devices, AR / VR devices, etc. The server may be a cloud server or a physical server, and may be a single server or a server cluster.

[0073] Optional, Figure 1 A schematic diagram of an architecture applicable to the image processing method provided in the embodiment of the present application, such as Figure 1 As shown in , the system may include a terminal device 10 (i.e., a terminal device that displays an image) and a server 20. The terminal device 10 and the server 20 exchange data through a network. A database 30 may be configured on one side of the server 20. The data in the database 30 may be stored in the storage space of the server 20 itself, or in an independent storage device (such as a cloud server) of the server 20. If it is an independent storage device, the server 20 may communicate with the storage device to exchange data. The database 30 may store a compressed first sub-image and at least one second sub-image.

[0074] Correspondingly, when the terminal device 10 receives an image viewing request triggered by the user, the image viewing request is sent to the server 20. The server 20 loads the compressed first sub-image and the second sub-image from the database 30 according to the angle identifier included in the image viewing request, and then renders the target first sub-image, and splices the rendered target first sub-image with the target second sub-image, and after obtaining the viewing image, sends it to the terminal device 10, so that the image can be displayed to the user through the terminal device 10.

[0075] Optionally, in actual applications, the terminal device 10 may also directly communicate data with the database 30, and when the terminal device 10 receives an image viewing request triggered by a user, the terminal device 10 directly loads the compressed first sub-image and the second sub-image from the database 30 according to the angle identifier included in the image viewing request, and then renders the target first sub-image, and splices the rendered target first sub-image with the target second sub-image to obtain the viewing image and then display the image to the user. Optionally, a browser application may be running in the terminal device 10, and when the terminal device 10 loads the compressed first sub-image and the second sub-image from the database 30 according to the angle identifier included in the image viewing request, the browser's concurrent network request capability may be used to split the loaded image into multiple small files for loading, thereby accelerating data requests and reducing the user's waiting time.

[0076] The present application provides an image processing method, such as Figure 2 As shown, the method includes:

[0077] Step S201: receiving an image viewing request, wherein the image viewing request includes an angle identifier of a viewing angle to be viewed.

[0078] Among them, the specific type of the image is not limited in the embodiment of the present application, such as a panoramic image or an ordinary plane image, the image can be a frame image in a video, or a separate image, etc. The embodiment of the present application does not limit the source of the image. Optionally, in order to better understand the method provided in the embodiment of the present application, the following text takes a high-definition panoramic image (a panoramic image with a clarity greater than a set threshold) as an example for explanation.

[0079] Optionally, when the user wants to view an image, an image viewing request may be triggered, and the image viewing request includes an angle identifier of the user's viewing angle to be viewed. The viewing angle to be viewed refers to the specific angle at which the user wants to view the image. For example, when the user wants to view the image at a level angle, the viewing angle to be viewed is 0 degrees. For another example, when the image is a panoramic image, the user can view the panoramic image at six angles, namely, front, back, top, bottom, left, and right. At this time, the corresponding viewing angles to be viewed may include six angles, namely, front, back, top, bottom, left, and right.

[0080] Step S202, loading the compressed first sub-image and the second sub-image from the pre-stored image set according to the angle identifier, obtaining the loaded target first sub-image and the target second sub-image, the image set including at least one compressed first sub-image and at least one second sub-image corresponding to at least one first sub-image before compression, each first sub-image before compression corresponds to an angle.

[0081] Optionally, in an embodiment of the present application, an image set may be pre-stored, the image set including at least one compressed first sub-image and at least one second sub-image corresponding to at least one first sub-image before compression. The first sub-image refers to an image obtained after segmenting the image, and corresponds to an angle. Accordingly, for a first sub-image, since the first sub-image corresponds to a second sub-image and corresponds to an angle, the second sub-image also corresponds to the angle at this time, that is, the corresponding first sub-image and second sub-image correspond to the same angle.

[0082] Furthermore, after receiving an image viewing request, the compressed first sub-image and second sub-image can be loaded from a pre-stored image set according to the angle identifier included in the image viewing request and the angle corresponding to each first sub-image and second sub-image, so as to obtain the loaded target first sub-image and target second sub-image.

[0083] Step S203, rendering the target first sub-image, and splicing the rendered target first sub-image with the target second sub-image to obtain and display a viewing image.

[0084] Optionally, after the target first sub-image and the target second sub-image are loaded, the target first sub-image obtained can be rendered, and the rendered target first sub-image and the target second sub-image can be spliced ​​to obtain a viewing image. The method of rendering the target first sub-image is not limited in the embodiment of the present application, such as using WebGL (Web Graphics Library) technology for rendering.

[0085] In an embodiment of the present application, when loading an image, the compressed first sub-image and the second sub-image can be loaded from the pre-stored image set according to the received angle identifier, and then the target first sub-image is rendered, and the rendered target first sub-image and the target second sub-image are spliced ​​to obtain a viewing image and then displayed to the user. In this process, since the image is divided into multiple first sub-images for loading, data request acceleration is achieved, which effectively solves the problem of long user waiting time, and since the first sub-image obtained after division is a compressed image, only the compressed target first sub-image can be rendered during subsequent image rendering, which improves rendering efficiency, and the image loading will be smoother, further improving the user experience.

[0086] A possible implementation manner is provided in an embodiment of the present application. Before receiving an image viewing request, the method further includes:

[0087] Acquire an image to be processed and a blurred image corresponding to the image to be processed, wherein the blurred image is an image obtained by reducing the definition of the image to be processed;

[0088] Performing at least one image segmentation on the image to be processed to obtain at least one first sub-image;

[0089] Performing at least one image segmentation on the blurred image according to at least one first sub-image to obtain at least one second sub-image;

[0090] At least one first sub-image and at least one second sub-image are stored to obtain an image set.

[0091] Optionally, the image set mentioned above may be pre-stored before receiving the image viewing request, and when the image set is obtained, the image to be processed and the blurred image obtained by reducing the clarity of the image to be processed may be obtained. Further, the image to be processed may be segmented at least once to obtain at least one first sub-image, and then the blurred image may be segmented at least once based on the at least one first sub-image obtained to obtain at least one second sub-image, and the second sub-image obtained at this time corresponds to the first sub-image one by one, and for each corresponding first sub-image and second sub-image, the image content between the two is the same, but the clarity is different, and the clarity of the first sub-image is higher than that of the second sub-image.

[0092] Correspondingly, after obtaining at least one first sub-image and at least one first sub-image, the at least one first sub-image and at least one first sub-image can be stored to obtain an image set. For example, the at least one first sub-image and at least one first sub-image can be stored in the cloud to obtain a stored image set.

[0093] A possible implementation method is provided in an embodiment of the present application. The image to be processed is an unfolded plane image of a sphere model. The image to be processed is segmented at least once to obtain at least one first sub-image, including:

[0094] Mapping the image to be processed into a spherical model, and projecting the image to be processed mapped into the spherical model at at least one angle to obtain a projection result corresponding to the at least one angle;

[0095] The image to be processed is segmented according to the projection result corresponding to at least one angle to obtain at least one first sub-image.

[0096] Optionally, the image to be processed in the embodiment of the present application may refer to the unfolded plane image corresponding to the spherical model. For example, the image to be processed may be a high-definition panoramic image with an aspect ratio of 2:1, which can be used as the unfolded plane image of the spherical model. When the image to be processed is used as a spherical model, a basic panoramic spherical display effect can be achieved.

[0097] Furthermore, the image to be processed can be mapped into a spherical model, and the image to be processed mapped into the spherical model can be projected at at least one angle, where each angle corresponds to a projection result. Then, the image to be processed can be segmented according to the projection result corresponding to at least one angle to obtain at least one first sub-image.

[0098] In one example, assuming that the image to be processed is a 2:1 high-definition panoramic image, the specific processing may be as follows: Figure 3a As shown, the image to be processed can be mapped into a spherical model (such as Figure 3b Further, the spherical model can be projected upward, downward, left, right, front, and back (as shown in Figure 3c Then, the projection mapping is used to calculate the image contents (i.e., projection results) corresponding to the upper, lower, left, right, front, and rear of the sphere model. At this time, the image to be processed can be segmented according to the obtained image contents corresponding to the upper, lower, left, right, front, and rear to obtain 6 first sub-images (as shown in FIG. Figure 3d shown).

[0099] Optionally, when performing image segmentation on the image to be processed according to the projection results corresponding to at least one angle, at least one initial first sub-image can be obtained, and then the at least one initial first sub-image obtained can be segmented again, and the sub-image obtained after the segmentation is used as the first sub-image stored in the image set. It can be understood that for an initial first sub-image, when the initial first sub-image is segmented again, the angles corresponding to all images obtained by segmentation are the same, that is, they are all angles corresponding to the initial first sub-image.

[0100] An embodiment of the present application provides a possible implementation method, in which at least one compressed first sub-image is obtained by the following method:

[0101] At least one first sub-image is compressed into a file of a set format to obtain at least one compressed first sub-image.

[0102] Optionally, after performing image segmentation on the image to be processed according to the projection result corresponding to at least one angle to obtain at least one first sub-image, the at least one first sub-image is compressed into a file of a set format to obtain at least one compressed first sub-image and store it in an image set. Among them, the specific compression technology and set format used when compressing at least one first sub-image can be pre-configured according to actual needs, and are not limited in the embodiments of the present application. For example, the Basis Universal (compressed GPU (Graphics Processing Unit) texture data exchange system) image compression technology can be used to compress common jpg (Joint Photographic Experts Group, image file format) / png (Portable Network Graphics, lossless compressed bitmap graphics format) images (i.e., first sub-images) into basis (universal texture format) format files.

[0103] In an embodiment of the present application, a possible implementation method is provided, which loads a compressed first sub-image and a second sub-image from a pre-stored image set according to an angle identifier to obtain a loaded target first sub-image and a target second sub-image, including:

[0104] Loading a compressed first sub-image corresponding to the angle identifier from a pre-stored image set as a target first sub-image;

[0105] The other second sub-images except the second sub-image corresponding to the target first sub-image in the at least one second sub-image are loaded from the pre-stored image set as the target second sub-image.

[0106] Optionally, after receiving the angle identifier, since each compressed first sub-image corresponds to an angle, the compressed first sub-image corresponding to the received angle identifier is found from the pre-stored image set, and the first sub-image can be loaded as the target first sub-image. Furthermore, since each first sub-image and the second sub-image are one-to-one corresponding, at least one second sub-image other than the second sub-image corresponding to the target first sub-image can be loaded from the pre-stored image set as the target second sub-image. In other words, in the embodiment of the present application, when viewing an image, only the image portion of the angle to be viewed is loaded, and for the other portions of the image, a low-definition image is used instead, thereby reducing the image loading time.

[0107] In an embodiment of the present application, a possible implementation method is provided, which renders a target first sub-image, and splices the rendered target first sub-image with a target second sub-image to obtain a viewing image, including:

[0108] Decoding and converting the target first sub-image to obtain the target first sub-image in a specified format, and rendering the target first sub-image in the specified format to obtain a rendered target first sub-image;

[0109] The rendered target first sub-image and target second sub-image are spliced ​​to obtain a viewing image.

[0110] Optionally, after obtaining the target first sub-image, since the target first sub-image is a set format file, the target first sub-image can be decoded and converted to obtain the target first sub-image in the specified format, and then the rendered target first sub-image and the target second sub-image are spliced ​​to obtain the viewing image. Among them, decoding and converting the target first sub-image to obtain the target first sub-image in the specified format refers to decoding the basis file and converting it into GPUTexture (GPU texture data).

[0111] Optionally, in order to better understand the method provided by the embodiment of the present application, the following is described in conjunction with a specific application scenario. In this example, the image to be processed is as follows Figure 3a The high-definition panoramic image shown has an aspect ratio of 2:1. A browser is running in the terminal device and a panoramic VR application is available.

[0112] Optionally, after obtaining the image to be processed, the image to be processed can be mapped into a spherical model (such as Figure 3b As shown, the spherical model is projected upward, downward, left, right, front, and back (as shown Figure 3c Then, the projection mapping is used to calculate the image contents (i.e., projection results) corresponding to the upper, lower, left, right, front, and rear of the sphere model. At this time, the image to be processed can be segmented according to the obtained image contents corresponding to the upper, lower, left, right, front, and rear to obtain 6 first sub-images (as shown in FIG. Figure 3d Then, the obtained image to be processed is subjected to a definition reduction process to obtain a blurred image, and then the blurred image is segmented according to the obtained 6 first sub-images to obtain second sub-images corresponding to the upper, lower, left, right, front, and rear picture contents, respectively.

[0113] Further, such as Figure 4As shown, the segmented 6 first sub-images can be imported, and the Basis Universal image compression technology can be used to compress the 6 first sub-images into basis format files, and then the compressed 6 first sub-images and 6 second sub-images can be stored in the cloud.

[0114] Accordingly, when the user wants to view the image to be processed, an image viewing request can be triggered. For example, when the user wants to view the image at the front angle, the angle identifier included in the image viewing request corresponds to the front angle. Figure 5 As shown, the panoramic VR application can be opened in the browser in the terminal device, and the compressed first sub-image corresponding to the front and the second sub-images corresponding to the top, bottom, left, right, and back can be loaded from the cloud by utilizing the principle that the browser can make concurrent requests.

[0115] Further, such as Figure 6 As shown, the compressed first sub-image loaded into the memory (i.e., the basis format file) is converted into GPU Texture data (i.e., decoding in the figure), and then the GPU Texture data is rendered through WebGL technology (i.e., rendering in the figure), and is spliced ​​with the second sub-images corresponding to the top, bottom, left, right, and back to form a complete panoramic image (i.e., splicing in the figure), which is then displayed to the user through the terminal device.

[0116] The process of decoding the basis format file into GPU Texture data can be as follows: Figure 7 As shown, it specifically includes two parts: Mobile (mobile terminal) and Desktop (Desktop). Among them, Mobile includes ETC (Ericsson Texture Compression) 1, ETC2, PVRTC (PowerVR Texture Compression) 1, ASTC (Adaptive scalable texture compression), and Desktop includes BC (Block Compression) 1-5 and BC7.

[0117] In this example, the existing high-definition panorama cutting and loading is implemented, and the latest image compression technology is integrated, which is suitable for the front-end three-dimensional panorama VR display scene. And the browser's concurrent network request capability is effectively utilized to split large files into multiple small files for loading, realizing data request acceleration; at the same time, the file size is compressed through GPU Texture technology, reducing the steps of converting image data into Texture, and improving rendering efficiency. Accordingly, the loading speed when displaying high-definition panoramas can be greatly improved, and the efficiency of three-dimensional rendering is also improved, reducing performance overhead. Users will be able to clearly feel that the loading and switching speed of the panorama is smoother.

[0118] The present application embodiment provides an image processing device, such as Figure 8 As shown, the image processing device 70 may include: a request receiving module 701, an image loading module 702 and an image stitching module 703, wherein:

[0119] A request receiving module 701 is used to receive an image viewing request, wherein the image viewing request includes an angle identifier of a viewing angle to be viewed;

[0120] An image loading module 702 is used to load the compressed first sub-image and the second sub-image from a pre-stored image set according to the angle identifier to obtain a loaded target first sub-image and a target second sub-image, wherein the image set includes at least one compressed first sub-image and at least one second sub-image corresponding to at least one first sub-image before compression, and each first sub-image before compression corresponds to an angle;

[0121] The image stitching module 703 is used to render the target first sub-image, and stitch the rendered target first sub-image with the target second sub-image to obtain and display a viewing image.

[0122] Optionally, the device further includes an image segmentation module, which is used to:

[0123] Before receiving an image viewing request, obtaining an image to be processed and a blurred image corresponding to the image to be processed, wherein the blurred image is an image obtained by reducing the definition of the image to be processed;

[0124] Performing at least one image segmentation on the image to be processed to obtain at least one first sub-image;

[0125] Performing at least one image segmentation on the blurred image according to at least one first sub-image to obtain at least one second sub-image;

[0126] At least one first sub-image and at least one second sub-image are stored to obtain an image set.

[0127] Optionally, the image to be processed is an unfolded plane image of a sphere model, and when the image segmentation module performs at least one image segmentation on the image to be processed to obtain at least one first sub-image, it is specifically used to:

[0128] Mapping the image to be processed into a spherical model, and projecting the image to be processed mapped into the spherical model at at least one angle to obtain a projection result corresponding to the at least one angle;

[0129] The image to be processed is segmented according to the projection result corresponding to at least one angle to obtain at least one first sub-image.

[0130] Optionally, at least one compressed first sub-image is obtained by:

[0131] At least one first sub-image is compressed into a file of a set format to obtain at least one compressed first sub-image.

[0132] Optionally, when the image loading module loads the compressed first sub-image and the second sub-image from the pre-stored image set according to the angle identifier to obtain the loaded target first sub-image and the target second sub-image, it is specifically used to:

[0133] Loading a compressed first sub-image corresponding to the angle identifier from a pre-stored image set as a target first sub-image;

[0134] The other second sub-images except the second sub-image corresponding to the target first sub-image in the at least one second sub-image are loaded from the pre-stored image set as the target second sub-image.

[0135] Optionally, when the image stitching module renders the target first sub-image and stitches the rendered target first sub-image with the target second sub-image to obtain the viewing image, it is specifically used to:

[0136] Decoding and converting the target first sub-image to obtain the target first sub-image in a specified format, and rendering the target first sub-image in the specified format to obtain a rendered target first sub-image;

[0137] The rendered target first sub-image and target second sub-image are spliced ​​to obtain a viewing image.

[0138] The device of the embodiments of the present application can execute the method provided by the embodiments of the present application, and the implementation principles are similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0139] An embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, and the processor executes the above-mentioned computer program to implement the steps of the image processing method.

[0140] In an alternative embodiment, an electronic device is provided, such as Fig. 9 As shown, Fig. 9 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0141] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0142] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0143] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.

[0144] The memory 4003 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the above method embodiment.

[0145] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0146] The embodiment of the present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.

[0147] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described in the drawings.

[0148] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0149] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. An image processing method, characterized in that: include: Acquire an image to be processed and a blurred image corresponding to the image to be processed, wherein the blurred image is an image obtained by reducing the definition of the image to be processed; the image to be processed is an unfolded plane image of a sphere model; Mapping the image to be processed into a spherical model, and performing at least one angle projection on the image to be processed mapped into the spherical model to obtain a projection result corresponding to the at least one angle; Performing image segmentation on the image to be processed according to the projection result corresponding to the at least one angle to obtain at least one first sub-image; wherein each of the first sub-images corresponds to one angle; Performing at least one image segmentation on the blurred image according to the at least one first sub-image to obtain at least one second sub-image; wherein the first sub-image and its corresponding second sub-image have the same image content and a preset angle, and the clarity of the first sub-image is higher than that of the second sub-image; storing at least one compressed first sub-image and at least one second sub-image corresponding to the at least one first sub-image before compression to obtain a pre-stored image set; receiving an image viewing request, wherein the image viewing request includes an angle identifier of a viewing angle to be viewed; Loading a compressed first sub-image corresponding to the angle identifier from the pre-stored image set as a target first sub-image; Loading, from the pre-stored image set, the other second sub-images except the second sub-image corresponding to the target first sub-image among the at least one second sub-image as the target second sub-image; The target first sub-image is rendered, and the rendered target first sub-image is spliced ​​with the target second sub-image to obtain a viewing image and display it.

2. The method according to claim 1, characterized in that The at least one compressed first sub-image is obtained by: The at least one first sub-image is respectively compressed into a file of a set format to obtain the at least one compressed first sub-image.

3. The method according to claim 1, characterized in that The step of rendering the target first sub-image and splicing the rendered target first sub-image with the target second sub-image to obtain a viewing image includes: Decoding and converting the target first sub-image to obtain a target first sub-image in a specified format, and rendering the target first sub-image in the specified format to obtain a rendered target first sub-image; The rendered target first sub-image and the target second sub-image are spliced ​​to obtain a viewing image.

4. An image processing device, characterized in that: include: An image segmentation module, used for acquiring an image to be processed and a blurred image corresponding to the image to be processed, wherein the blurred image is an image obtained by reducing the definition of the image to be processed; the image to be processed is an unfolded plane image of a sphere model; Mapping the image to be processed into a spherical model, and performing at least one angle projection on the image to be processed mapped into the spherical model to obtain a projection result corresponding to the at least one angle; Performing image segmentation on the image to be processed according to the projection result corresponding to the at least one angle to obtain at least one first sub-image; wherein each of the first sub-images corresponds to one angle; Performing at least one image segmentation on the blurred image according to the at least one first sub-image to obtain at least one second sub-image; wherein the first sub-image and its corresponding second sub-image have the same image content and a preset angle, and the clarity of the first sub-image is higher than that of the second sub-image; storing at least one compressed first sub-image and at least one second sub-image corresponding to the at least one first sub-image before compression to obtain a pre-stored image set; A request receiving module, used for receiving an image viewing request, wherein the image viewing request includes an angle identifier of a viewing angle to be viewed; an image loading module, configured to load a compressed first sub-image corresponding to the angle identifier from the pre-stored image set as a target first sub-image; and load other second sub-images of the at least one second sub-image except the second sub-image corresponding to the target first sub-image from the pre-stored image set as the target second sub-image; The image stitching module is used to render the target first sub-image, and stitch the rendered target first sub-image with the target second sub-image to obtain and display a viewing image.

5. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images