Panorama generation method and device, storage medium and product

Abstract

The invention provides a panorama generation method and device, a storage medium and a product, and relates to the technical field of image processing. The panorama generation method comprises the following steps: initializing a scene and a cubic camera, wherein the cubic camera is arranged in an area center corresponding to the scene; if it is determined that the model of which the material is the preset material exists in the scene, the transparency of the preset material is converted into preset transparency; and rendering based on the transparency and the cubic camera to obtain a target panorama. According to the embodiment of the invention, when the material of the model is the preset material, transparency conversion can be carried out on the model, so that transparency calculation and rendering sequence arrangement can be carried out correctly, black blocks are avoided, the completeness and visual effect of the panorama are improved, the cubic camera can be located in the space bounding box corresponding to the scene, and the user experience is improved. And the shooting effect of the panorama is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a panoramic image generation method, apparatus, and storage medium. Background Technology

[0002] A panoramic image, also known as a panorama, typically refers to a photograph capturing a complete 360-degree scene that falls within the normal effective visual range of human eyes (approximately 90 degrees horizontally and 70 degrees vertically) or includes peripheral vision (approximately 180 degrees horizontally and 90 degrees vertically). Panoramic imaging is a new image compositing technology that combines images of a scene from various angles into a single panoramic image. This allows users to view images of a scene from all directions within a single panoramic view.

[0003] In existing technologies, the position of the cube camera is not specifically restricted, which can easily result in the cube camera being located outside the spatial bounding box. This can lead to inaccurate panoramic images that fail to achieve the desired shooting effect. Furthermore, when the models in the scene have preset materials and their transparency is enabled, it is difficult to correctly calculate the transparency and arrange the rendering order. This can cause a face in the panoramic image to fail to display properly, resulting in black blocks that affect the integrity and visual effect of the panoramic image. Summary of the Invention

[0004] This application provides a panoramic image generation method, device, storage medium, and product, which can solve the problems of inaccurate cube positioning, affecting shooting effect, and black blocks appearing in model materials, thus affecting the integrity and visual effect of the panoramic image. To achieve this objective, this application provides the following solutions.

[0005] According to one aspect of the embodiments of this application, a panoramic image generation method is provided, the method comprising: Initialize the scene and the cube camera, wherein the cube camera is positioned at the center of the area corresponding to the scene; If it is determined that there is a model in the scene with a preset material, then the transparency of the preset material is converted to a predetermined transparency. Rendering is performed based on the transparency and the cube camera to obtain a panoramic image of the target.

[0006] In one possible implementation, the initialization of the scene and cube camera includes: In response to a scene creation request, create a scene and obtain the scene parameters corresponding to the scene; A cube camera is set according to the center of the area corresponding to the scene.

[0007] In one possible implementation, setting the cube camera according to the area center corresponding to the scene includes: Obtain the model corresponding to the scene, and calculate the area center based on the geometry of the model; The position coordinates of the cube camera are determined based on the position coordinates corresponding to the center of the area.

[0008] In one possible implementation, rendering based on the transparency and the cube camera includes: The model with replaced transparency is rendered onto six rendering targets using the cube camera to obtain a rendered image, where the six rendering targets correspond to the six faces of the cube.

[0009] In one possible implementation, obtaining the rendered image includes: Identify the horizontal pixels in the rendered image and invert them.

[0010] In one possible implementation, the method includes: In response to a download command, display image format selection information and retrieve the selection command; Download the target panoramic image and save the downloaded target panoramic image as an image, the image format of which corresponds to the selection command.

[0011] In one possible implementation, the method includes: In response to a color compression command, obtain the color to be converted for each color in the image; Color compression is performed based on the color to be converted.

[0012] In one possible implementation, the response to the audio upload request includes: The audio upload entry point is displayed on the upload interface; If an audio file upload operation of the target object is detected through the audio upload entry, it is determined that an audio upload request has been received, and the audio upload request is responded to. The method includes: Obtain the audio file processing information transmitted from the backend, and display the audio file processing information using the upload interface. The audio file processing information includes at least one of audio file upload progress and conversion status.

[0013] According to one aspect of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the method described above.

[0014] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method described above.

[0015] According to one aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0016] The beneficial effects of the technical solutions provided in this application are: The panoramic image generation method provided in this application initializes the scene and a cube camera, with the cube camera positioned at the center of the area corresponding to the scene. If it is determined that there is a model in the scene with a preset material, the transparency of the preset material is converted to a predetermined transparency. Based on the transparency and the cube camera, rendering is performed to obtain the target panoramic image. The embodiments of this application can convert the transparency of the model when the model's material is a preset material, which facilitates the correct calculation of transparency and the arrangement of rendering order, avoids the appearance of black blocks, improves the completeness and visual effect of the panoramic image, and enables the cube camera to be located inside the spatial bounding box corresponding to the scene, thus improving the shooting effect of the panoramic image. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0018] Figure 1 A flowchart of the panoramic image generation method provided in the embodiments of this application; Figure 2 Rendering diagram provided for embodiments of this application; Figure 3 A flowchart illustrating the panorama generation method provided in this application embodiment; Figure 4 A schematic diagram of a panoramic image generated by the panoramic image generation method provided in the embodiments of this application; Figure 5 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0020] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” indicates implementation as “A,” or implementation as “A,” or implementation as “A and B.”

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0022] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0023] The panoramic image generation method, device, storage medium, and product provided in this application are intended to solve at least one technical problem existing in the prior art.

[0024] This application provides a method for generating panoramic images, which can be used on mobile phones, computers, smartwatches, and other devices capable of generating panoramic images. Specifically, panoramic images can be generated using a specific panoramic image generation tool within a webpage in a browser. This panoramic image generation tool can be an existing 3D engine capable of generating panoramic images, through which the panoramic image generation operation is performed.

[0025] Optionally, such as Figures 1-4 As shown, the panoramic image generation method of this application includes: S101: Initialize the scene and cube camera.

[0026] Optionally, the cube camera is positioned at the center of the area corresponding to the scene. Initializing the scene and the cube camera includes: in response to a scene creation request, creating the scene and obtaining the scene parameters corresponding to the scene; and setting the cube camera according to the center of the area corresponding to the scene. The scene can form a bounding box, and the model to be rendered and the cube camera are located within the bounding box corresponding to the scene.

[0027] Alternatively, scene and cube camera initialization can be performed using three.js, or using Cesium, Mapbox GL, Unreal Engine, Babylon.js, and other tools that can be used for panoramic image generation.

[0028] Optionally, scene parameters may include scene size, background color, lighting parameters, fog, and other scene-related parameters, and the types of scene parameters can be determined based on the object used for initialization.

[0029] Optionally, a cube camera is set according to the area center corresponding to the scene, including: acquiring the model corresponding to the scene, calculating the area center according to the geometry of the model, and determining the position coordinates of the cube camera according to the position coordinates corresponding to the area center.

[0030] In one embodiment, a scene is created using three.js, and scene parameters are set. These parameters may include scene size, background color, lighting parameters, etc., and the specific values ​​can be determined based on the style, brightness, and other requirements of the panoramic image. After creating the scene, a cube camera is created, and the cube's position can be the center of the area of ​​the model in the scene from which the panoramic image is to be generated.

[0031] Optionally, a top view or bottom view of the model can be obtained, the area center of the top view can be calculated, and the area center can be determined as the planar coordinates of the cube camera. The height coordinates of the cube can be half the height value of the model. In this way, the cube camera is set inside the graphic formed by the scene.

[0032] S102: If it is determined that there is a model in the scene with a preset material, then convert the transparency of the preset material to the predetermined transparency.

[0033] Optionally, when the object used to generate the panoramic image is three.js, the preset material can be a physical mesh material, and the preset transparency can be the transparency of a standard material. This transparency substitution addresses a logical flaw in three.js when processing cube panoramic images, where it fails to correctly calculate transparency and arrange the rendering order, resulting in a black block appearing on one face, affecting the integrity and visual effect of the panoramic image. When the object generating the panoramic image is another object besides three.js, the corresponding material and the required transparency conversion can be determined based on the object's logical flaws to avoid the panoramic image failing to display.

[0034] In one embodiment, the object generating the panoramic image is three.js. Due to a bug in three.js, when the model's material is a physical mesh material, rendering errors (black blocks) occur when rendering to the RenderTarget using CubeCamera. Therefore, after completing scene initialization and cube camera initialization, the material of the models in the scene is detected. If any model's material is a physical mesh material, the transparency of the standard material is used to simulate the transparency of the physical mesh material (this transparency replacement can be achieved by converting the physical mesh material (MeshPhysicalMaterial) to the standard mesh material (MeshStandardMaterial)) to avoid the black block problem.

[0035] S103: Rendering based on transparency and a cube camera to obtain a panoramic view of the target.

[0036] Optionally, rendering is performed based on transparency and a cube camera, including: using the cube camera to render the model with replaced transparency to six rendering targets to obtain a rendered image, the six rendering targets corresponding to the six faces of a cube.

[0037] In one embodiment, such as Figure 2 As shown, the six rendering targets are located in the six directions of the cube camera (i.e., front, back, left, right, up, down, which can be represented as +Z, -Z, -X, +X, +Y, -Y). The cube camera performs rendering operations based on these six directions to obtain the rendered image.

[0038] Optionally, since the cube camera is located inside the cube corresponding to the scene, the generated rendered image is an image inside the cube. The six generated panoramic images are tiled inside the cube and need to be flipped. Therefore, obtaining the rendered image includes: determining the horizontal pixels in the rendered image and flipping the horizontal pixels.

[0039] In one embodiment, a cube camera is located inside a cube corresponding to the scene. Rendering is performed using this cube camera located at the center of the area, targeting six rendering objects, each corresponding to one of the six faces inside the cube. This rendering method ensures that the image direction captured by each face is opposite to the direction corresponding to the actual panoramic image in the scene. For example, an image captured from the front is rendered with the cube as the center, facing the back of the object, which is opposite to the desired panoramic image direction (i.e., from the front view of the object). For the rendered image on each rendering target, the horizontal pixels in the image are inverted to ensure that the generated panoramic image has the correct horizontal orientation (changing to a front view of the object). This can be achieved by multiplying the Z-axis coordinate of the pixels by -1 to obtain the inverted pixels. The generated panoramic image can be as follows: Figure 4 As shown.

[0040] Optionally, to meet the need for flexible selection of different image formats, the method of this application further includes: responding to a download instruction, displaying image format selection information, obtaining the selection instruction; downloading the target panoramic image, and saving the downloaded target panoramic image as an image, wherein the image format corresponds to the selection instruction.

[0041] Optionally, the panoramic image is generated in a browser. After obtaining the rendered images in six directions corresponding to the panoramic image, the user can download the panoramic image through the browser. When downloading the panoramic image according to user instructions, an image format selection function can be provided (such as displaying an image format selection interface or window), the user's selection instruction can be obtained, the selected image format can be determined based on the selection instruction, and the downloaded panoramic image can be saved in that format.

[0042] Alternatively, the panoramic image can be downloaded via a browser by creating a link using the DOM and then downloading the panoramic image through that link.

[0043] Optionally, the image format can be PNG formats with different bit depths, such as JPG, PNG8, PNG16, and PNG32. The image format type can be determined based on the browser type and the object being generated as the panorama.

[0044] Optionally, the method of this application further includes: in response to a color compression instruction, obtaining the color to be converted corresponding to each color in the image; and performing color compression based on the color to be converted.

[0045] Optionally, for obtaining the color to be converted, the image contains three color channels: red (R), green (G), and blue (B). The closest target color in the image can be found based on Euclidean distance, and this target color is determined as the color to be converted. Specifically, a 256-color palette can be generated based on the image's colors. The image's color space can be viewed as a three-dimensional coordinate system, where the X, Y, and Z axes represent the red (R), green (G), and blue (B) values ​​of the colors, respectively. Each color can be represented by a point in this three-dimensional space. Given a target color, each color in the 256-color palette is traversed, and the Euclidean distance between each color and each target color is calculated. Then, the target color with the smallest distance is found and taken as the closest color. This method is used to obtain the color to be converted for each color in the image.

[0046] In one embodiment, for PNG format images, open-source libraries such as png-js can be used to perform color compression to reduce the image size, achieving a balance between image quality and image size, expanding the application of images in different scenarios, and improving transmission efficiency.

[0047] Optionally, when saving the panoramic image as a JPG, the image quality (such as image size, resolution, and other quality parameters) can be determined according to the user's instructions to compress the image and meet the requirements for image quality and file size in different scenarios.

[0048] The panoramic image generation method provided in this application has the following advantages: 1. It solves the problem of black blocks appearing in the model under physical mesh material, ensuring the integrity and visual effect of the panoramic image, improving the user experience, and enabling the generated panoramic image to have a wider range of applications in fields with high requirements for image accuracy.

[0049] 2. By inverting the horizontal pixels, the horizontal image orientation of the panoramic image is ensured to be correct, meeting the requirements for the accuracy of the panoramic image orientation in practical applications and avoiding the inconvenience caused by image display errors.

[0050] 3. Added image compression function, supports multiple image formats and quality options, and can flexibly adjust image size and quality according to actual needs, improving the application flexibility and transmission efficiency of images in different scenarios, and saving storage space and bandwidth.

[0051] 4. More accurate cube camera positioning. By calculating the center of the area to determine the camera position, the problem of the cube camera corresponding to the scene potentially appearing outside the space is solved. This makes the panoramic image's shooting perspective more in line with actual needs and improves the display effect of the panoramic image.

[0052] Based on the same inventive concept, embodiments of this application provide an electronic device, such as... Figure 5 As shown, Figure 5 The illustrated electronic device 2000 includes a processor 2001 and a memory 2003. The processor 2001 and the memory 2003 are communicatively connected, for example, via a bus 2002.

[0053] Processor 2001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 2001 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0054] Bus 2002 may include a pathway for transmitting information between the aforementioned components. Bus 2002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 2002 can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the figure, but this does not indicate that there is only one bus or one type of bus.

[0055] The memory 2003 may be ROM (Read-Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read-Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0056] Optionally, the electronic device 2000 may also include a communication unit 2004. The communication unit 2004 can be used for receiving and transmitting signals. The communication unit 2004 allows the electronic device 2000 to communicate wirelessly or wiredly with other devices to exchange data. It should be noted that in practical applications, the communication unit 2004 is not limited to one.

[0057] Optionally, the electronic device 2000 may further include an input unit 2005. The input unit 2005 can be used to receive input numbers, characters, images, and / or sound information, or to generate key signal inputs related to user settings and function control of the electronic device 2000. The input unit 2005 may include, but is not limited to, one or more of the following: a touchscreen, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, a joystick, a camera, a microphone, etc.

[0058] Optionally, the electronic device 2000 may also include an output unit 2006. The output unit 2006 can be used to output or display information processed by the processor 2001. The output unit 2006 may include, but is not limited to, one or more of a display device, a speaker, a vibration device, etc.

[0059] Although the electronic device 2000 with various devices is shown in the figure, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0060] Optionally, the memory 2003 is used to store a computer program for executing the scheme of this application, and its execution is controlled by the processor 2001. The processor 2001 is used to execute the computer program stored in the memory 2003 to implement the steps of any method provided in the embodiments of this application.

[0061] This application provides a computer-readable storage medium including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0062] This application provides a computer program product, which includes a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0063] The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the illustrations or text descriptions.

[0064] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0065] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.

Claims

1. A method for generating panoramic images, characterized in that, The method includes: Initialize the scene and the cube camera, wherein the cube camera is positioned at the center of the area corresponding to the scene; If it is determined that there is a model in the scene with a preset material, then the transparency of the preset material is converted to a predetermined transparency. Rendering is performed based on the transparency and the cube camera to obtain a panoramic image of the target.

2. The panoramic image generation method according to claim 1, characterized in that, The initialization of the scene and cube camera includes: In response to a scene creation request, create a scene and obtain the scene parameters corresponding to the scene; A cube camera is set according to the center of the area corresponding to the scene.

3. The panoramic image generation method according to claim 2, characterized in that, The step of setting up a cube camera according to the center of the area corresponding to the scene includes: Obtain the model corresponding to the scene, and calculate the area center based on the geometry of the model; The position coordinates of the cube camera are determined based on the position coordinates corresponding to the center of the area.

4. The panoramic image generation method according to claim 2, characterized in that, The rendering based on the transparency and the cube camera includes: The model with replaced transparency is rendered onto six rendering targets using the cube camera to obtain a rendered image, where the six rendering targets correspond to the six faces of the cube.

5. The panoramic image generation method according to claim 4, characterized in that, The obtained rendered image includes: Identify the horizontal pixels in the rendered image and invert them.

6. The panoramic image generation method according to claim 1, characterized in that, The method includes: In response to a download command, display image format selection information and retrieve the selection command; Download the target panoramic image and save the downloaded target panoramic image as an image, the image format of which corresponds to the selection command.

7. The panoramic image generation method according to claim 6, characterized in that, The method includes: In response to a color compression command, obtain the color to be converted for each color in the image; Color compression is performed based on the color to be converted.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the steps of the method as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-7.

Images