Image Processing Method, Apparatus, Electronic Device, and Storage Medium

Abstract

Embodiments of the present disclosure provide an image processing method, apparatus, electronic device, and storage medium. The method includes: obtaining a first depth image and a second depth image including a target three-dimensional object; processing the first depth image based on a preset algorithm to determine a first transition map to be fused under a target camera view; and processing the second depth image based on a preset algorithm to determine a second transition map to be fused under the target camera view; determining a target transition image for transitioning from the first depth image to the second depth image based on the first transition map to be fused and the second transition map to be fused. The technical solution provided by the embodiments of the present disclosure achieves the technical effect that view transition can be performed when the view is switched.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of image processing technologies, and in particular, to an image processing method, apparatus, electronic device, and storage medium. Background Art

[0002] Generally, an object is three-dimensional. When displaying an object on a display interface, a user may want to view views from multiple angles. At this time, views from multiple angles can be collected and displayed on the display interface, but there is a problem of resource occupation.

[0003] If the number of collected images is small, multi-angle display may not be possible, resulting in a poor image display effect. Further, the visual angle difference between two adjacent images is large. When switching views, there is a problem of uneven transition, which not only provides a poor user experience but also results in a poor display effect. Summary of the Invention

[0004] Embodiments of the present disclosure provide an image processing method, apparatus, electronic device, and storage medium, which achieve smooth transition based on a determined transition image when switching images, improving the image display effect and user experience.

[0005] In a first aspect, embodiments of the present disclosure provide an image processing method, which includes:

[0006] Obtain a first depth image and a second depth image including a target three-dimensional object; wherein, a first camera view angle of the first depth image is adjacent to a second camera view angle of the second depth image;

[0007] Process the first depth image based on a preset algorithm to determine a first to-be-fused transition map at a target camera view angle; and,

[0008] Process the second depth image based on the preset algorithm to determine a second to-be-fused transition map at the target camera view angle; wherein, the preset algorithm is used to determine a corresponding to-be-fused transition map based on the depth value of the depth image;

[0009] Determine a target transition image from the first depth image to the second depth image based on the first to-be-fused transition map and the second to-be-fused transition map;

[0010] wherein, the target camera view angle is between the first camera view angle and the second camera view angle.

[0011] In a second aspect, embodiments of the present disclosure further provide an image processing apparatus, which includes:

[0012] A view acquisition module, configured to acquire a first depth image and a second depth image including a target three-dimensional object; wherein, a first camera view of the first depth image is adjacent to a second camera view of the second depth image;

[0013] A first transition map determination module, configured to process the first depth image based on a preset algorithm to determine a first transition map to be fused under a target camera view; and,

[0014] A second transition map determination module, configured to process the second depth image based on the preset algorithm to determine a second transition map to be fused under the target camera view; wherein, the preset algorithm is used to determine a corresponding transition map to be fused based on the depth value of the depth image;

[0015] A target transition map determination module, configured to determine a target transition image from the first depth image to the second depth image based on the first transition map to be fused and the second transition map to be fused;

[0016] Wherein, the target camera view is located between the first camera view and the second camera view.

[0017] In a third aspect, an embodiment of the present disclosure further provides an electronic device, where the electronic device includes:

[0018] One or more processors;

[0019] A storage device, configured to store one or more programs,

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method according to any one of the embodiments of the present disclosure.

[0021] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium containing computer-executable instructions, where the computer-executable instructions are used to execute the image processing method according to any one of the embodiments of the present disclosure when executed by a computer processor.

[0022] The technical solution of the embodiment of the present disclosure obtains a first depth image and a second depth image including a target three-dimensional object, processes the first depth image based on a preset algorithm to determine a first transition map to be fused under the target camera view, and processes the second depth image based on a preset algorithm to determine a second transition map to be fused under the target camera view. Based on the first transition map to be fused and the second transition map to be fused, a target transition image from the first depth image to the second depth image is determined, which solves the problem in the prior art that when the number of perspective images is limited, the display effect and user experience are poor during view switching, and realizes determining the transition image between two adjacent camera views so that the corresponding transition image can be displayed during image switching, not only improving the image display effect but also improving the user viewing experience effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages and aspects of the embodiments of the present disclosure will become more obvious. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the original and elements are not necessarily drawn to scale.

[0024] Figure 1 It is a flowchart of an image processing method provided by an embodiment of the present disclosure;

[0025] Figure 2 It is a flowchart of an image processing method provided by an embodiment of the present disclosure;

[0026] Figure 3 It is a flowchart of an image processing method provided by an embodiment of the present disclosure;

[0027] Figure 4 It is a flowchart of an image processing method provided by an embodiment of the present disclosure;

[0028] Figure 5 It is a flowchart of an image processing method provided by an embodiment of the present disclosure;

[0029] Figure 6 It is a depth image to be applied applicable to an image processing method provided by an embodiment of the present disclosure.

[0030] Figure 7 It is a depth image to be applied applicable to an image processing method provided by an embodiment of the present disclosure.

[0031] Figure 8 It is a pyramid diagram applicable to an image processing method provided by an embodiment of the present disclosure;

[0032] Figure 9Schematic diagram of depth information acquisition applicable to an image processing method provided by an embodiment of the present disclosure;

[0033] Figure 10 Block diagram of the structure of an image processing device provided by an embodiment of the present disclosure;

[0034] Figure 11 Schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0035] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0036] It should be understood that the steps recited in the method embodiments of the present disclosure can be executed in a different order and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0037] As used herein, the term "including" and its variations are open-ended, that is, "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0038] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependent relationships.

[0039] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0040] Before introducing the technical solution, an exemplary description of the application scenario can be given first. The technical solution of the present disclosure can be applied to any scenario for displaying three-dimensional objects. For example, if you want to display object images from different angles on a display interface, generally, it is necessary to collect the views corresponding to the object at different angles, so as to display based on the user's trigger operation. Different angles can be based on the camera view angle of 0° as the initial angle, and the shooting step size is 5° or 6° to shoot the object views corresponding to different camera view angles. Among them, the shooting step size can be set according to the actual situation. Based on this method, views from multiple camera view angles can be obtained, but at this time, there may be a problem that the shooting step size is relatively large, that is, when switching between adjacent camera view angles, there is a problem of a view blind area. Based on this, the shooting step size can be adjusted to 1°, 2°, or 3°, etc., but at this time, more views will be shot, that is, there is a problem of occupying more memory resources. Based on this, the technical solution provided by the embodiments of the present disclosure can be adopted. Under the condition that the shooting interval step size increases or remains unchanged, corresponding views can be shot, and at least one transition image between two adjacent view angles can be determined, so that when switching views, the transition can be performed based on at least one transition image, that is, the current view can be switched to the target view based on at least one transition view.

[0041] Figure 1 As shown in the flowchart of an image processing method provided by an embodiment of the present disclosure, this embodiment is applicable to the situation where when switching views between two adjacent view angles, at least one transition view determined based on the two adjacent views can be used for transition to display the switched view. This method can be executed by an image processing device, and the device can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC, or a server, etc. Any three-dimensional object display scenario is usually implemented in cooperation with a client and a server. The method provided by this embodiment can be executed by the server, the client, or the cooperation between the client and the server.

[0042] As Figure 1 , the method of this embodiment includes:

[0043] S110. Obtain a first depth image and a second depth image including a target three-dimensional object.

[0044] Among them, the apparatus for implementing the image processing method provided in the embodiments of the present disclosure may be integrated in an application software supporting the three-dimensional display function, and the software may be installed in an electronic device. Optionally, the electronic device may be a mobile terminal or a PC terminal, etc. The application software may be a type of software for image / video processing. Specific application software will not be elaborated here one by one, as long as it can implement image / video processing. It may also be a specially developed application program, which may be software and a client for determining a transition view, or integrated in a corresponding page. The user may determine the transition view through the page integrated in the PC terminal.

[0045] Among them, the target three-dimensional object may be any object that needs to be three-dimensionally displayed. The first depth image and the second depth image are relative. The image displayed before the switch is taken as the first depth image, and the image to which the switch is made is taken as the second depth image. The first depth image and the second depth image are taken from different camera perspectives. The shooting of the first depth image and the second depth image is completed based on a depth camera. Correspondingly, the captured images are images including the depth information and color information of the target three-dimensional object from the corresponding camera perspectives. That is to say, both the first depth image and the second depth image are RGBD images, that is, images including color information and depth information. Denote the camera perspective corresponding to the first depth image as the first camera perspective. Correspondingly, the camera perspective corresponding to the second depth image is the second camera perspective, and the first camera perspective and the second camera perspective are adjacent. For example, there are mainly three captured images, namely the view at a camera perspective of 0°, the view at a camera perspective of 6°, and the view at a camera perspective of 12°. At this time, 0° and 6° are adjacent camera perspectives, 6° and 12° are adjacent camera perspectives, and 12° and 0° are adjacent camera perspectives.

[0046] Exemplarily, before the view corresponding to the target three-dimensional object is displayed, views from multiple camera perspectives may be captured and the captured views may be displayed on the display interface. When it is detected that the user swipes the view, it indicates that the user wants to switch the view, that is, the depth image corresponding to the currently displayed view is taken as the first depth image, and the depth image corresponding to the next view to be displayed is taken as the second depth image.

[0047] Specifically, images of the target three-dimensional object from different angles may be captured in advance by a depth camera, and the images from different angles may be uploaded on the upload page of the client for display on the client.

[0048] S120. Process the first depth image based on a preset algorithm to determine a first to-be-fused transition map from the target camera perspective; and process the second depth image based on the preset algorithm to determine a second to-be-fused transition map from the target camera perspective.

[0049] Among them, the preset algorithm is used to determine the corresponding transition graph to be fused based on the depth value of the depth image; the preset algorithm can be understood as any algorithm that can determine the corresponding view according to the image depth information. For example, the preset algorithm is the ray casting algorithm, and the depth value of the image can be obtained through the ray casting algorithm. The implementation process of the ray casting algorithm mainly uses the camera position as the origin, and emits rays step by step to each pixel point. When the ray collides with an object, the ray can return the pixel depth information. If no information is returned, it can be determined that there is no pixel in the direction corresponding to the ray. Finally, based on the returned depth information, the color value and depth value corresponding to the collision pixel point are calculated according to the method of ray tracing. The target camera view is the view between the first camera view and the second camera view. For example, the first camera view is 0°, the second camera view is 6°, and the target camera view can be any one or more of 2°, 3°, and 4°. The first transition graph to be fused is the transition image obtained by processing the first depth image based on the ray casting algorithm. The second transition graph to be fused is the transition image obtained by processing the second depth image through the ray casting algorithm.

[0050] It should be noted that the number of the first transition graph to be fused and the second transition graph to be fused can be one or more, and the specific number is related to the accuracy required in the final application. No matter how many images there are, the method provided by the embodiments of the present disclosure can be used to determine their transition images.

[0051] Specifically, based on the ray casting algorithm, light beams are emitted to the first depth image and the second depth image. If the emitted light beam has an intersection with the first depth image and the second depth image, it will pass through the pixel points in the image, and the pixel points can be processed to obtain the first transition graph to be fused and the second transition graph to be fused.

[0052] In practical applications, in order to improve the transition effect from the first depth image to the second depth image, the second depth image can also be processed based on the ray casting algorithm to determine the second transition graph to be fused under the target camera view. The determination method of the second transition graph to be fused is the same as that of the first transition graph to be fused, and the specific implementation method will not be elaborated here.

[0053] S130. Based on the first transition graph to be fused and the second transition graph to be fused, determine the target transition image from the first depth image to the second depth image.

[0054] Among them, the target transition image is at least one transition image determined based on the first depth image and the second depth image when switching from the first depth image to the second depth image.

[0055] In this technical solution, determining the target transition image can be: performing alignment processing on the first transition graph to be fused and the second transition graph to be fused to determine the target transition image.

[0056] Specifically, at least one feature point of the first transition graph to be fused can be obtained through a feature extraction algorithm, and at the same time, at least one feature point of the second transition graph to be fused can be determined. Based on the at least one feature point, corresponding processing is performed to determine the target transition image. It can be understood that based on the feature points, information related to the rotation, translation, and matrix of the image can be determined. Furthermore, based on information such as translation, rotation, and matrix, the corresponding pixel points in the two images are matched. Further, the color information and depth information of the corresponding pixel points are averaged to determine the target transition image.

[0057] It should be noted that the number of target transition images can be multiple, and the specific determination method can be determined by repeatedly executing the above steps.

[0058] The technical solution of the embodiments of the present disclosure obtains a first depth image and a second depth image including a target three-dimensional object, processes the first depth image based on a preset algorithm to determine a first transition graph to be fused under a target camera view, and, based on a preset algorithm, processes the second depth image to determine a second transition graph to be fused under the target camera view. Based on the first transition graph to be fused and the second transition graph to be fused, a target transition image from the first depth image to the second depth image is determined, which solves the problem in the prior art that when the number of perspective images is limited, during view switching, the display effect and user experience are poor, and realizes determining the transition image between two adjacent camera views so that when the images are switched, the corresponding transition images can be displayed, which not only improves the image display effect but also improves the user viewing experience.

[0059] Figure 2 It is a schematic flowchart of an image processing method provided by the embodiments of the present disclosure. On the basis of the foregoing embodiments, the view transition conditions and the determination method of the target transition image can be further refined, and the specific display method can refer to the specific content of this embodiment. Among them, the same or corresponding technical terms as those in the above embodiments will not be described in detail here.

[0060] As Figure 2 shown, the method specifically includes the following steps:

[0061] S210. Obtain depth images under each camera view based on a depth camera, so as to determine a target transition image based on two adjacent depth images of the camera views.

[0062] Among them, the depth camera is a camera used to capture a three-dimensional object of interest. The image captured by the depth camera includes not only the depth information of the three-dimensional object of interest but also the color information of the three-dimensional object of interest. When it is desired to obtain views of the object from many angles, the shooting angle of the depth camera relative to the object can be adjusted, and based on the relative shooting angle, the depth image corresponding to the object is determined.

[0063] Specifically, for the convenience of shooting, the depth camera can be placed on a rotating platform. The angle of each rotation of the rotating platform can be 6°, and the rotation center is the object of interest. Based on this, the depth images corresponding to the object from many angles can be captured. Furthermore, based on two depth images of the three-dimensional object of interest captured from two adjacent camera viewpoints, the target transition image can be determined.

[0064] The depth images can be stacked and displayed on the display interface so that when the image switching control is triggered, the corresponding depth images are displayed in sequence.

[0065] S220. When it is detected that the view transition condition is satisfied, the current depth image is used as the first depth image, and the second depth image of the next camera viewpoint adjacent to the first camera viewpoint is obtained.

[0066] Among them, the view transition condition can be a condition preset for determining whether image processing is to be performed. The next camera viewpoint corresponds to the second camera viewpoint. The current depth image is the image displayed when the view transition condition is satisfied, and the second depth image is the image switched to.

[0067] In this embodiment, the view transition condition includes at least one of the following: detecting that the view transition switching control is triggered; detecting that the display duration of the current depth image reaches a preset display duration threshold.

[0068] Among them, the view transition switching control can be a control corresponding to a sliding operation or a control corresponding to a click operation. The preset display duration threshold is the display duration of the current angle view preset. Optionally, the preset display duration is 1 s or 2 s.

[0069] Specifically, the image of the target three-dimensional object can be displayed on the display interface of the client. When the user performs a sliding operation on the current display interface, it indicates that the user wants to switch from the image of the current perspective to the image of the next perspective. Then, the currently displayed image is used as the first depth image, and the depth image of the perspective to be switched is used as the second depth image. The user can also preset a display duration threshold in advance. For example, it can be 1 second. To better display the target three-dimensional object, when the display duration of the target three-dimensional object in the current view reaches 1 second, if no sliding operation or click operation by the user is detected, it indicates that the target three-dimensional object can be switched to the next angle for display. At this time, the client can determine that the target three-dimensional object meets the condition for view switching and generate the target transition image required for view switching.

[0070] S230. Process the first depth image based on a preset algorithm to determine the first transitional image to be fused from the perspective of the target camera; and, process the second depth image based on a preset algorithm to determine the second transitional image to be fused from the perspective of the target camera.

[0071] Among them, the preset algorithm is used to determine the corresponding transitional image to be fused based on the depth value of the depth image. It can be understood that determining the target transition image can be real-time or pre-determined. If it is a pre-determined target transition image, then during view switching, as long as the camera perspective corresponding to the switching is combined, the corresponding target transition image can be retrieved.

[0072] Specifically, in practical applications, multiple depth images from different camera perspectives can be pre-shot and uploaded to the client. The client processes two adjacent depth images from different camera perspectives to obtain the transitional image between the two images and saves it. When it is detected that the view transition condition is met, the client can search for the pre-saved transitional image based on the current first depth image and the second depth image of the next perspective as the target transition image to achieve the switch from the first depth image to the second depth image based on the target transition image. The advantage of doing this is that it can improve the image transition efficiency.

[0073] Exemplarily, depth images a, b, c, d, and e of a target three-dimensional object can be captured at viewing angles of 0°, 6°, 12°, 18°, and 24° respectively. Among them, the above viewing angles are adjacent camera viewing angles. The above depth images are uploaded to the client, and the client processes the above images to determine the transition image 1 between depth image a and depth image b, the transition image 2 between depth image b and depth image c, the transition image 3 between depth image c and depth image d, and the transition image 3 between depth image d and depth image e, and saves them. Among them, the camera shooting angle corresponding to the transition image 1 is between 0° and 6°, the camera shooting angle corresponding to the transition image 1 is between 0° and 6°, the camera shooting angle corresponding to the transition image 2 is between 6° and 12°, the camera shooting angle corresponding to the transition image 3 is between 12° and 18°, and the camera shooting angle corresponding to the transition image 3 is between 18° and 24°. When it is necessary to switch from depth image a to depth image b, the transition image 1 can be directly obtained and used as the target transition image. It should also be noted that the determination of the transition image can be real-time or pre-determined.

[0074] Another way is to determine it in real time. The advantage of doing this is that it is possible that not all views at all viewing angles are previewed by the user. Therefore, when storing views at different viewing angles, there is a problem of resource occupation. Therefore, the target transition image during switching can be determined in real time, that is, when it is detected that the view transition condition is met, the corresponding target transition view is generated in real time based on the first depth image and the second depth image, and the switching from the first depth image to the second depth image is realized based on the target transition image.

[0075] S240. Based on the first transition graph to be fused and the second transition graph to be fused, determine the target transition image for transitioning from the first depth image to the second depth image, and perform view switching based on the target transition image.

[0076] The technical solution of the embodiments of the present disclosure, by obtaining the first depth image and the second depth image including the target three-dimensional object, processing the first depth image based on a preset algorithm to determine the first transition graph to be fused at the target camera viewing angle, and, processing the second depth image based on a preset algorithm to determine the second transition graph to be fused at the target camera viewing angle, and based on the first transition graph to be fused and the second transition graph to be fused, determining the target transition image for transitioning from the first depth image to the second depth image, solves the problem in the prior art that when the number of perspective images is limited, during view switching, there are problems of poor display effects and user experience, and realizes determining the transition image between two adjacent camera viewing angles, so that when the image is switched, the corresponding transition image can be displayed, which not only improves the image display effect but also improves the user viewing experience effect.

[0077] Figure 3The flowchart of an image processing method provided by an embodiment of the present disclosure. Based on the foregoing embodiment, pyramid graphs corresponding to the first depth image and the second depth image can be determined respectively, and a target transition image can be determined based on the pyramid graphs. For the specific implementation manner, reference can be made to the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiment will not be elaborated herein.

[0078] As Figure 3 shown, the method specifically includes the following steps:

[0079] S310. Obtain a first depth image and a second depth image including a target three-dimensional object.

[0080] Among them, the first camera view of the first depth image is adjacent to the second camera view of the second depth image.

[0081] S320. For a depth image, determine at least one depth image to be applied corresponding to the current depth image with at least one preset pixel segmentation region as the resolution adjustment ratio.

[0082] Among them, the resolution adjustment ratio can include at least one. Correspondingly, the preset pixel segmentation region can also include at least one. For example, the resolution adjustment ratio is based on the original depth image, and the resolution is reduced Correspondingly, the preset pixel segmentation region is two pixel points as a preset pixel segmentation region, four pixel points as a preset pixel segmentation region, and eight pixel points as a preset pixel segmentation region.

[0083] Correspondingly, each depth image contains at least one depth image to be applied, and the number of corresponding depth images to be applied is related to the resolution adjustment ratio. For example, the adjustment ratio of a certain depth image is Correspondingly, the depth image to be applied corresponding to this depth image is 3 images.

[0084] In this embodiment, each depth image can be processed, or one or more of them can be processed.

[0085] Based on the above technical solutions, the following details the determination method of the depth image to be applied corresponding to each depth image. It can be that, according to the resolution adjustment ratio, at least one preset pixel segmentation region is determined. For the preset pixel segmentation region, at least one segmentation region to be processed corresponding to the current depth image is determined according to the current preset pixel segmentation region, and the pixel points of each segmentation region to be processed are aggregated respectively to determine the depth image to be applied corresponding to the current depth image.

[0086] Among them, the to-be-processed segmentation region can be multiple regions that need pixel aggregation processing after segmenting the to-be-applied depth image according to a preset pixel segmentation region. The to-be-applied depth image is a depth image obtained by adjusting each depth image according to a resolution adjustment ratio.

[0087] Specifically, for each depth image, setting multiple preset pixel segmentation regions can be taking the region corresponding to two pixel points in the depth image as a preset segmentation region, fusing the pixel points within the preset segmentation region to obtain one pixel point, and further obtaining a depth image with reduced resolution, that is, the to-be-applied depth image. It is also possible to take the region corresponding to four pixel points as a preset segmentation region, fuse the four pixel points into one pixel point, and further obtain a depth image with reduced resolution as another to-be-applied depth image.

[0088] It can be understood that both the first depth image and the second depth image correspond to multiple resolution adjustment ratios. Through the above steps, multiple to-be-applied depth images with different resolutions corresponding to each depth image can be obtained.

[0089] Exemplarily, as Figure 6 shown, the resolution of this depth image is 8*8, and the resolution adjustment ratio corresponding to the preset pixel segmentation region is Correspondingly, Figure 6 the region circled by the dotted line in Figure 7 is the to-be-processed segmentation region. All regions containing four pixel points are to-be-processed segmentation regions. In this embodiment, only one to-be-processed segmentation region is circled and described, and the processing methods of other to-be-processed segmentation regions are the same. In this depth image, the four pixel points in each to-be-processed segmentation region are aggregated to obtain one pixel point. The specific method can be taking the color average value and the maximum depth value of multiple original pixel points in the region as the color value and depth value of the aggregated pixel point. Further, the obtained 4*4 resolution image, that is, the image corresponding to

[0090] It can be understood that in addition to taking two or four pixel points in the depth image as the preset pixel segmentation region, it is also possible to take the region corresponding to eight pixel points as the preset segmentation region and determine its corresponding resolution ratio as The specific processing method is the same as the image processing method of taking four pixel points as the preset pixel segmentation region above, and will not be elaborated here.

[0091] S330. Perform depth marking on each pixel point in the applied depth image to obtain a pyramid image with a pyramid-shaped resolution arrangement of the applied depth image, and determine a first transition image to be fused from the pyramid image of the first depth image and a second transition image to be fused from the pyramid image of the second depth image based on the target camera view.

[0092] Among them, the resolution of the applied depth image in the pyramid image shows a gradually increasing trend from the top level to the bottom level, as Figure 8 shown.

[0093] Exemplarily, perform resolution adjustment on the depth image to obtain applied depth images with different resolutions corresponding to the depth image. Such as applied depth image 1, applied depth image 2, and applied depth image 3. Sort applied depth images 1, 2, and 3 in ascending order of resolution from high to low to form a pyramid image. The top of the pyramid is applied image 1, followed by applied image 2, then applied image 3, and finally the bottom is the depth image.

[0094] It should be noted that the above method can be used to determine the corresponding pyramid image for each depth image.

[0095] Specifically, it can be to mark the pixel point values in the applied depth image, and form a pyramid image corresponding to the first depth image and the second depth image by arranging multiple applied images with pixel point depth marking in the order of resolution size. Further, according to the step algorithm, the depth camera emits step light rays to the pyramid image in the target camera view to obtain depth information, and then draw the first and second transition images to be fused based on the depth information.

[0096] In this embodiment, performing depth marking on each pixel point in the applied depth image to obtain a pyramid image with a pyramid-shaped resolution arrangement of the applied depth image includes: for the applied depth image, obtaining the maximum depth value of the pixel points in each to-be-processed segmentation region in the current applied depth image, and using it as the depth value of the aggregated pixel points in the corresponding to-be-processed segmentation region to update the current applied depth image. Based on the updated applied depth image, determine a pyramid image with a pyramid-shaped resolution arrangement of the applied depth image.

[0097] Among them, when the first depth image and the second depth image are segmented and processed according to the preset pixel segmentation region, the currently processed segmentation region is the to-be-processed segmentation region.

[0098] Specifically, there are multiple pixel points in the area to be processed and segmented. The depth values of each pixel point can be obtained, and the maximum depth value is used as the depth value of the aggregated pixel point and marked. In this way, the depth values of the pixel points in the depth image to be applied are updated.

[0099] S340. Based on the first transition graph to be fused and the second transition graph to be fused, determine the target transition graph for transitioning from the first depth image to the second depth image.

[0100] Among them, the target camera view is located between the first camera view and the second camera view.

[0101] The technical solution of the embodiment of the present disclosure, by obtaining the first depth image and the second depth image including the target three-dimensional object, processing the first depth image based on a preset algorithm to determine the first transition graph to be fused under the target camera view, and, processing the second depth image based on a preset algorithm to determine the second transition graph to be fused under the target camera view, and determining the target transition graph for transitioning from the first depth image to the second depth image based on the first transition graph to be fused and the second transition graph to be fused, solves the problem in the prior art that when the number of view images is limited and the view is switched, the display effect and the user experience are not good, and realizes determining the transition image between two adjacent camera views so that when the image is switched, the corresponding transition image can be displayed, which not only improves the image display effect but also improves the user viewing experience effect.

[0102] Figure 4 It is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure. On the basis of the foregoing embodiment, this embodiment introduces a specific method for determining the transition graph to be fused based on the intersection point of the emission beam and the depth image. The specific implementation manner can refer to the technical solution of this embodiment. Among them, the same or corresponding technical terms as those in the above embodiment will not be described in detail here.

[0103] As Figure 4 shown, the method specifically includes the following steps:

[0104] S410. Obtain the first depth image and the second depth image including the target three-dimensional object.

[0105] Among them, the first camera view of the first depth image is adjacent to the second camera view of the second depth image.

[0106] S420. Based on a preset algorithm, determine the intersection point information of the emission beam and the first depth image, and determine the first transition graph to be fused under the camera view according to the pixel depth information of the intersection point information.

[0107] Specifically, in the perspective of the target camera, step rays can be emitted to each pixel point in the first depth image to determine the intersection information between the step rays and the first depth image. If there is an intersection with the pixels in the depth image, the depth information of the intersection pixel can be obtained. Based on the depth information of this pixel, the color information of this pixel point can be determined. If there is no intersection, it means that this pixel point has no depth information in the perspective of the target camera, that is, no color information. Based on the determined color information of the pixel points, a first fusion transition map of the first depth image in the perspective of the target camera is drawn.

[0108] S430. Process the second depth image based on a preset algorithm to determine a second fusion transition map in the perspective of the target camera.

[0109] It should be noted that the method of processing the second depth image based on a preset algorithm to obtain the second fusion transition map is the same as the processing method in step S420 above.

[0110] S440. Based on the first fusion transition map and the second fusion transition map, determine a target transition image from the first depth image to the second depth image.

[0111] Among them, the target camera perspective is located between the first camera perspective and the second camera perspective.

[0112] The technical solution of the embodiment of the present disclosure obtains a first depth image and a second depth image including a target three-dimensional object, processes the first depth image based on a preset algorithm to determine a first fusion transition map in the perspective of the target camera, and processes the second depth image based on a preset algorithm to determine a second fusion transition map in the perspective of the target camera. Based on the first fusion transition map and the second fusion transition map, a target transition image from the first depth image to the second depth image is determined, which solves the problem that in the prior art, when the number of perspective images is limited, during view switching, the display effect and user experience are poor. It realizes determining the transition image between two adjacent camera perspectives so that when the image is switched, the corresponding transition image can be displayed, which not only improves the image display effect but also improves the user viewing experience.

[0113] Figure 5 It is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure. On the basis of the foregoing embodiment, this embodiment introduces a specific method for fusing and transitioning an image based on the intersection information between the emitted light beam and the depth image to be applied in the pyramid diagram. The specific implementation manner can refer to the technical solution of this embodiment. Among them, the same or corresponding technical terms as those in the above embodiment will not be described in detail here.

[0114] As Figure 5 shown, the method specifically includes the following steps:

[0115] S510. Obtain a first depth image and a second depth image including the target three-dimensional object.

[0116] Wherein, the first camera view angle of the first depth image is adjacent to the second camera view angle of the second depth image.

[0117] S520. Determine the emission light beam under the target camera view angle based on a preset algorithm.

[0118] S530. Determine a first transition graph to be fused based on the emission light beam and the pyramid graph of the first depth image.

[0119] Specifically, it can be that the camera sends the emission light beam to the pyramid graph under the target camera view angle, and based on the step algorithm, obtains the depth information of the pyramid graph returned by the emission light beam, obtains the pixel point depth information in the pyramid graph based on the depth information, and draws the first transition graph to be fused based on the depth information.

[0120] In this embodiment, at least one depth image to be applied is included in the pyramid graph. The specific determination method for determining the transition image under a certain view angle can be referred to as follows:

[0121] Determine whether the emission light beam has an intersection with the depth image to be applied corresponding to the top layer; if so, search downward based on the intersection point for the intersection information of the emission light beam and the depth image to be applied in the next layer, and repeatedly execute searching downward for the intersection information of the emission light beam and the depth image to be applied in the next layer based on the intersection information until the depth and color of the pixel point corresponding to the absence of intersection information are used as the depth and color of the emission light beam; determine the first transition graph to be fused based on at least one color and depth corresponding to the emission light beam.

[0122] It should be noted that there is a corresponding first pyramid graph for the first depth image, and there is a corresponding second pyramid graph for the second depth image. The first pyramid graph includes multiple depth images to be applied corresponding to the first depth image, and the second pyramid graph includes multiple depth images to be applied corresponding to the second depth image. As Figure 8 shown, in the pyramid graph, the resolution of the depth images to be applied is arranged in ascending order from top to bottom. As Figure 9As shown in the figure, in practical applications, if there is an intersection point between the emitted light beam and the depth image to be applied at the first level of the pyramid, based on this intersection point, it is possible to search downward to determine whether there is an intersection point between the depth image to be applied at the second level and the emitted light beam. If there is an intersection point, based on the intersection point information between the depth image to be applied at the second level and the emitted light beam, continue to search downward to determine whether there is an intersection point between the emitted light beam and the depth image to be applied at the third level. Repeat the above operations until there is no intersection point between the emitted light beam and the depth image to be applied at a certain level. Then, use the depth information and color information of the pixel point corresponding to the intersection point between the depth image to be applied at the upper level of this level and the emitted light beam as the depth and color of the emitted light beam, and based on this color and depth, draw the first fusion transition diagram.

[0123] Based on the above embodiments, determining whether there is an intersection point between the emitted light beam and the depth image to be applied corresponding to the top level includes: determining the maximum depth value in the depth image to be applied, and determining whether there is an intersection point between the emitted light beam and the pixel point corresponding to the maximum depth value.

[0124] In practical applications, usually if a light beam does not intersect with the pixel point with the maximum depth value in the depth image to be applied at the first level, it means that this light beam will not intersect with the pixel points in the depth images to be applied corresponding to the levels below the first level. Based on this, it is not necessary to calculate the intersection points of this light beam with the second, third, fourth... and other levels, which greatly reduces the number of intersection calculations and improves the processing efficiency.

[0125] Specifically, it can be to determine the maximum depth value in the depth image to be applied, and determine whether there is an intersection point between the emitted light beam and the maximum depth value. If there is no intersection point between the emitted light beam and the pixel point of the maximum depth value, it means that this light beam has no intersection point with the top level and all the lower levels of the top level. It is possible to determine whether there is an intersection point between the depth image to be applied at the next level and the emitted light beam, which can reduce the number of intersection calculations.

[0126] Optionally, in the case where there is no intersection point between the emitted light beam and the pixel point corresponding to the maximum depth value, the emitted light beam is removed, and the next emitted light beam is obtained and the determination of whether there is an intersection point between the emitted light beam and the depth image to be applied is repeated until the emitted light beam is the last emitted light beam.

[0127] It can be understood that if there is no intersection point between the emitted light beam and the depth image to be applied in the pyramid diagram, it means that this light ray has no corresponding depth information. There are multiple light beams emitted by the camera. If the current light beam has no intersection point with the depth image to be applied, the current light beam is removed to improve the determination efficiency of the intersection point. Obtain whether the next light beam has an intersection point with the depth image to be applied until it is determined whether all the emitted light beams have an intersection point with the depth image to be applied, and the above operations are completed.

[0128] S540. Process the second depth image based on a preset algorithm to determine a second transitional image to be fused from the perspective of the target camera.

[0129] S550. Align the first transitional image to be fused and the second transitional image to be fused to determine a target transitional image.

[0130] The technical solution of the embodiments of the present disclosure obtains a first depth image and a second depth image including a target three-dimensional object, processes the first depth image based on a preset algorithm to determine a first transitional image to be fused from the perspective of the target camera, and processes the second depth image based on the preset algorithm to determine a second transitional image to be fused from the perspective of the target camera. Based on the first transitional image to be fused and the second transitional image to be fused, a target transitional image for transitioning from the first depth image to the second depth image is determined, solving the problem in the prior art that when the number of perspective images is limited and the view is switched, the display effect and the user experience are poor. It realizes determining the transitional image between two adjacent camera perspectives so that when the image is switched, the corresponding transitional image can be displayed, not only improving the image display effect but also improving the user viewing experience.

[0131] Figure 10 It is a structural block diagram of an image processing device provided by the embodiments of the present disclosure, which can execute the image processing method provided by any embodiment of the present disclosure and has the corresponding functional modules and beneficial effects for executing the method. As Figure 10 shown, the device specifically includes: a view acquisition module 610, a first transitional image determination module 620, a second transitional image determination module 630, and a target transitional image determination module 640.

[0132] The view acquisition module 610 is configured to acquire a first depth image and a second depth image including a target three-dimensional object; wherein, the first camera perspective of the first depth image is adjacent to the second camera perspective of the second depth image;

[0133] The first transitional image determination module 620 is configured to process the first depth image based on a preset algorithm to determine a first transitional image to be fused from the perspective of the target camera; and

[0134] The second transitional image determination module 630 is configured to process the second depth image based on the preset algorithm to determine the second transitional image to be fused from the perspective of the target camera; wherein, the preset algorithm is used to determine the corresponding transitional image to be fused based on the depth value of the depth image;

[0135] The target transitional image determination module 640 is configured to determine a target transitional image for transitioning from the first depth image to the second depth image based on the first transitional image to be fused and the second transitional image to be fused;

[0136] Among them, the target camera view is located between the first camera view and the camera view.

[0137] Based on the above technical solutions, the image processing device further includes:

[0138] A transition condition detection module, configured to, when detecting that a view transition condition is satisfied, use the current depth image as the first depth image and obtain a second depth image of the next camera view adjacent to the first camera view; wherein, the next camera view corresponds to the second camera view;

[0139] A transition image acquisition module, configured to acquire a target transition image between the first depth image and the second depth image, so as to switch from the first depth image to the second depth image based on the target transition image; or,

[0140] A transition image determination module, configured to determine a target transition image for transitioning from the first depth image to the second depth image based on the first depth image and the second depth image.

[0141] Based on the above technical solutions, the view transition condition includes at least one of the following:

[0142] Detecting that a view transition switch control is triggered;

[0143] Detecting that the display duration of the current depth image reaches a preset display duration threshold.

[0144] Based on the above technical solutions, the image processing device further includes:

[0145] A depth image acquisition module, configured to, before acquiring the first depth image and the second depth image including the target three-dimensional object, acquire depth images at each camera view based on a depth camera, so as to determine a target transition image based on two adjacent depth images of the camera views.

[0146] Based on the above technical solutions, the image processing device further includes:

[0147] A depth map to be applied determination module, configured to, before processing the first depth image and / or the second depth image based on a preset algorithm, for the depth image, determine at least one depth image to be applied corresponding to the current depth image with at least one preset pixel segmentation region as a resolution adjustment ratio;

[0148] A depth marking module, configured to perform depth marking on each pixel point in the depth image to be applied, so as to obtain a pyramid image with a pyramid-shaped resolution arrangement of the depth image to be applied, and based on the pyramid image of the first depth image, determine a first transition image to be fused under the target camera view, and based on the pyramid image of the second depth image, determine a second transition image to be fused under the target camera view;

[0149] Wherein, in the pyramid image, the resolution of the depth image to be applied shows a gradually increasing trend from the top level to the bottom level.

[0150] Based on the above technical solutions, the depth map to be applied determining module includes:

[0151] A preset segmentation area determining module, configured to determine at least one preset pixel segmentation area according to the resolution adjustment ratio; wherein, the number of pixel points in each preset pixel segmentation area is different;

[0152] A pixel point aggregation module, configured to, for a preset pixel segmentation area, determine at least one segmentation area to be processed corresponding to the current depth image according to the current preset pixel segmentation area, and perform pixel point aggregation processing on each segmentation area to be processed, so as to determine the depth image to be applied corresponding to the current depth image.

[0153] Based on the above technical solutions, the depth marking module includes:

[0154] A depth maximum value determining module, configured to, for the depth image to be applied, obtain the depth maximum value of the pixel points in each segmentation area to be processed in the current depth image to be applied, and use it as the depth value mark of the aggregated pixel points in the corresponding segmentation area to be processed, so as to update the current depth image to be applied;

[0155] An update module, configured to determine a pyramid image with a pyramid-shaped resolution arrangement of the depth image to be applied based on the updated depth image to be applied.

[0156] Based on the above technical solutions, the first transition image determining module includes:

[0157] An intersection point information determining module, configured to determine the intersection point information of the emission beam and the first depth image based on the preset algorithm, and determine the first transition image to be fused under the camera view according to the pixel depth information of the intersection point information.

[0158] Based on the above technical solutions, the first transition image determining module further includes:

[0159] An emission beam determining module, configured to determine the emission beam under the target camera view based on the preset algorithm;

[0160] The first transition graph to be fused determining module is configured to determine the first transition graph to be fused based on the emission light beam and the pyramid graph of the first depth image.

[0161] Based on the above technical solutions, at least one level is included in the pyramid graph, and each level corresponds to a depth image to be applied. The first transition graph to be fused determining module includes:

[0162] The intersection point determining unit is configured to determine whether the emission light beam intersects with the depth image to be applied corresponding to the top level;

[0163] The downward search unit is configured to, if so, search downward based on the intersection point for the intersection point information of the emission light beam and the depth image to be applied at the next level, and repeatedly execute the search for the intersection point information of the emission light beam and the depth image to be applied at the next level based on the intersection point information until the depth and color of the pixel point corresponding to the absence of intersection point information are used as the depth and color of the emission light beam.

[0164] The first transition graph to be fused determining unit is configured to determine the first transition graph to be fused based on at least one color and depth corresponding to the emission light beam.

[0165] Based on the above technical solutions, the intersection point determining unit includes:

[0166] The maximum depth value determining unit is configured to determine the maximum depth value in the depth image to be applied and determine whether the emission light beam intersects with the pixel point corresponding to the maximum depth value.

[0167] Based on the above technical solutions, the image processing device further includes:

[0168] The light beam culling module is configured to cull the emission light beam when the emission light beam does not intersect with the pixel point corresponding to the maximum depth value, and obtain the next emission light beam and repeatedly execute the determination of whether the emission light beam intersects with the depth image to be applied until the emission light beam is the last emission light beam.

[0169] Based on the above technical solutions, the target transition graph determining module 740 further includes:

[0170] The alignment module is configured to perform alignment processing on the first transition graph to be fused and the second transition graph to be fused to determine the target transition image.

[0171] The technical solution of the embodiment of the present disclosure obtains a first depth image and a second depth image including a target three-dimensional object, processes the first depth image based on a preset algorithm to determine a first transition map to be fused under the target camera view, and processes the second depth image based on the preset algorithm to determine a second transition map to be fused under the target camera view. Based on the first transition map to be fused and the second transition map to be fused, a target transition image from the first depth image to the second depth image is determined, which solves the problem in the prior art that when the number of perspective images is limited, the display effect and user experience are poor during view switching, and realizes determining a transition image between two adjacent camera views so that when the images are switched, the corresponding transition image can be displayed, which not only improves the image display effect but also improves the user viewing experience effect.

[0172] The transition view determination device provided by the embodiment of the present disclosure can execute the image processing method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

[0173] It should be noted that the various units and modules included in the above device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the embodiments of the present disclosure.

[0174] Figure 11 It is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. Referring below to Figure 11 , which shows a schematic structural diagram of an electronic device 700 suitable for implementing the embodiment of the present disclosure (such as Figure 11 the terminal device or server in). The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 11 The electronic device shown is only an example and should not bring any limitation to the functions and usage scope of the embodiments of the present disclosure.

[0175] As Figure 11As shown, the electronic device 700 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 701, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 706 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An editing / output (I / O) interface 705 is also connected to the bus 704.

[0176] Generally, the following devices may be connected to the I / O interface 705: an editing device 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 11 the electronic device 700 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had.

[0177] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device 709, or installed from the storage device 706, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above functions defined in the method of the embodiment of the present disclosure are executed.

[0178] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0179] The electronic device provided by the embodiment of the present disclosure and the special effect image processing method provided by the above embodiment belong to the same inventive concept. The technical details not described in detail in this embodiment may be referred to in the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0180] The embodiment of the present disclosure provides a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, the image processing method provided by the above embodiment is implemented.

[0181] It should be noted that the above-mentioned computer-readable medium in this disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, and this computer-readable signal medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0182] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed network.

[0183] The above-mentioned computer-readable medium can be included in the above-mentioned electronic device; it can also exist separately without being assembled into the electronic device.

[0184] The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device is caused to:

[0185] Obtain a first depth image and a second depth image including a target three-dimensional object; wherein, a first camera view angle of the first depth image is adjacent to a second camera view angle of the second depth image;

[0186] Process the first depth image based on a preset algorithm to determine a first transition graph to be fused under a target camera view angle; and,

[0187] Process the second depth image based on the preset algorithm to determine a second transition graph to be fused under the target camera view angle; wherein, the preset algorithm is used to determine a corresponding transition graph to be fused based on the depth value of the depth image;

[0188] Determine a target transition image from the first depth image to the second depth image based on the first transition graph to be fused and the second transition graph to be fused;

[0189] Wherein, the target camera view angle is located between the first camera view angle and the second camera view angle.

[0190] Computer program code for performing the operations of the present disclosure can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include, but are not limited to, object-oriented programming languages - such as Java, Smalltalk, C++, and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0191] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0192] The units involved in the embodiments described in the present disclosure can be implemented in software or in hardware. Among them, the name of the unit does not constitute a limitation on the unit itself in some cases. For example, the first acquisition unit can also be described as "the unit for acquiring at least two Internet protocol addresses".

[0193] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and so on.

[0194] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0195] According to one or more embodiments of the present disclosure, [Example 1] provides an image processing method, the method comprising:

[0196] Obtain a first depth image and a second depth image including a target three-dimensional object; wherein, a first camera view angle of the first depth image is adjacent to a second camera view angle of the second depth image;

[0197] Process the first depth image based on a preset algorithm to determine a first transition graph to be fused under a target camera view angle; and,

[0198] Process the second depth image based on the preset algorithm to determine a second transition graph to be fused under the target camera view angle;

[0199] Determine a target transition image from the first depth image to the second depth image based on the first transition graph to be fused and the second transition graph to be fused;

[0200] Wherein, the target camera view angle is located between the first camera view angle and the second camera view angle.

[0201] According to one or more embodiments of the present disclosure, [Example 2] provides an image processing method, further including:

[0202] When it is detected that a view transition condition is satisfied, use the current depth image as the first depth image, and obtain a second depth image of the next camera view angle adjacent to the first camera view angle; wherein, the next camera view angle corresponds to the second camera view angle

[0203] Obtain a target transition image between the first depth image and the second depth image to switch from the first depth image to the second depth image based on the target transition image; or,

[0204] Determine a target transition image from the first depth image to the second depth image based on the first depth image and the second depth image.

[0205] According to one or more embodiments of the present disclosure, [Example 3] provides an image processing method, further including:

[0206] Optionally, the view transition condition includes at least one of the following:

[0207] Detect that a view transition switch control is triggered;

[0208] Detect that the display duration of the current depth image reaches a preset display duration threshold.

[0209] According to one or more embodiments of the present disclosure, [Example 4] provides an image processing method, further including:

[0210] Optionally, before obtaining the first depth image and the second depth image including the target three-dimensional object, it further includes:

[0211] Based on the depth camera, obtain depth images at each camera view, and based on two adjacent depth images at the camera views, determine the target transition image.

[0212] According to one or more embodiments of the present disclosure, [Example Five] provides an image processing method, which further includes:

[0213] Optionally, before processing the first depth image and / or the second depth image based on a preset algorithm, it further includes:

[0214] For the depth image, use at least one preset pixel segmentation area as the resolution adjustment ratio to determine at least one depth image to be applied corresponding to the current depth image;

[0215] Perform depth marking on each pixel point in the depth image to be applied to obtain a pyramid diagram with a pyramid-shaped resolution arrangement of the depth image to be applied, and based on the pyramid diagram of the first depth image, determine the first fusion transition diagram at the target camera view, and based on the pyramid diagram of the second depth image, determine the second fusion transition diagram at the target camera view;

[0216] Wherein, the resolution of the depth image to be applied in the pyramid diagram shows a gradually increasing trend from the top level to the bottom level. According to one or more embodiments of the present disclosure, [Example Six] provides an image processing method, which further includes:

[0217] Optionally, the step of using at least one preset pixel segmentation area as the resolution adjustment ratio to determine at least one depth image to be applied corresponding to the current depth image includes:

[0218] According to the resolution adjustment ratio, determine at least one preset pixel segmentation area; wherein, the number of pixel points in each preset pixel segmentation area is different;

[0219] For the preset pixel segmentation area, determine at least one segmentation area to be processed corresponding to the current depth image according to the current preset pixel segmentation area, and perform aggregation processing on the pixel points in each segmentation area to be processed to determine the depth image to be applied corresponding to the current depth image.

[0220] According to one or more embodiments of the present disclosure, [Example Seven] provides an image processing method, which further includes:

[0221] Optionally, the step of performing depth marking on each pixel point in the depth image to be applied to obtain a pyramid diagram with a pyramid-shaped resolution arrangement of the depth image to be applied includes:

[0222] For the depth image to be applied, obtain the maximum depth value of the pixel points in each to-be-processed segmentation region in the current depth image to be applied, and use it as the depth value of the aggregated pixel points in the corresponding to-be-processed segmentation region to update the current depth image to be applied;

[0223] Based on the updated depth image to be applied, determine a pyramid image in which the resolution arrangement of the depth image to be applied is in a pyramid shape.

[0224] According to one or more embodiments of the present disclosure, [Example VIII] provides an image processing method, further including:

[0225] Optionally, the processing the first depth image based on a preset algorithm to determine a first to-be-fused transition image under a target camera view includes:

[0226] Determine the intersection information of the emission beam and the first depth image based on the preset algorithm, and determine the first to-be-fused transition image under the camera view according to the pixel depth information of the intersection information.

[0227] According to one or more embodiments of the present disclosure, [Example IX] provides an image processing method, further including:

[0228] Optionally, the processing the first depth image based on a preset algorithm to determine a first to-be-fused transition image under a target camera view includes:

[0229] Determine the emission beam under the target camera view based on the preset algorithm;

[0230] Based on the emission beam and the pyramid image of the first depth image, determine the first to-be-fused transition image.

[0231] According to one or more embodiments of the present disclosure, [Example X] provides an image processing method, further including:

[0232] Optionally, the pyramid image includes at least one level, and each level corresponds to a depth image to be applied. The determining the first to-be-fused transition image based on the emission beam and the pyramid image of the first depth image includes:

[0233] Determine whether the emission beam intersects with the depth image to be applied corresponding to the top level;

[0234] If so, downward search for the intersection information of the emission beam and the depth image to be applied in the next level based on the intersection point, and repeatedly execute downward search for the intersection information of the emission beam and the depth image to be applied in the next level based on the intersection information until the depth and color of the pixel point corresponding to the absence of intersection information are used as the depth and color of the emission beam;

[0235] Determine the first transitional graph to be fused based on at least one color and depth corresponding to the emitted light beam.

[0236] According to one or more embodiments of the present disclosure, [Example XI] provides an image processing method, further including:

[0237] Optionally, the determining whether the emitted light beam intersects with the depth image to be applied corresponding to the top level includes:

[0238] Determine the maximum depth value in the depth image to be applied, and determine whether the emitted light beam intersects with the pixel point corresponding to the maximum depth value.

[0239] According to one or more embodiments of the present disclosure, [Example XII] provides an image processing method, further including:

[0240] Optionally, in the case where the emitted light beam does not intersect with the pixel point corresponding to the maximum depth value, eliminate the emitted light beam, and obtain the next emitted light beam to repeat the determination of whether the emitted light beam intersects with the depth image to be applied until the emitted light beam is the last emitted light beam.

[0241] According to one or more embodiments of the present disclosure, [Example XIII] provides an image processing method, further including:

[0242] Optionally, the determining the target transitional image for transitioning from the first depth image to the second depth image based on the first transitional graph to be fused and the second transitional graph to be fused includes:

[0243] Align the first transitional graph to be fused and the second transitional graph to be fused to determine the target transitional image.

[0244] According to one or more embodiments of the present disclosure, [Example XIV] provides an image processing apparatus, including:

[0245] A view acquisition module, configured to acquire a first depth image and a second depth image including a target three-dimensional object; wherein, a first camera view of the first depth image is adjacent to a second camera view of the second depth image;

[0246] A first transitional graph determination module, configured to process the first depth image based on a preset algorithm to determine a first transitional graph to be fused in a target camera view; and,

[0247] A second transition map determination module, configured to process the second depth image based on the preset algorithm to determine a second transition map to be fused in the target camera view; wherein, the preset algorithm is used to determine a corresponding transition map to be fused based on the depth value of the depth image; A target transition map determination module, configured to determine a target transition image for transitioning from the first depth image to the second depth image based on the first transition map to be fused and the second transition map to be fused;

[0248] Wherein, the target camera view is located between the first camera view and the camera view.

[0249] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0250] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although a number of specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0251] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms for implementing the claims.

Claims

1. An image processing method, characterized in that, Including: Obtaining a first depth image and a second depth image including a target three-dimensional object; wherein, a first camera view angle of the first depth image is adjacent to a second camera view angle of the second depth image; Processing the first depth image based on a preset algorithm to determine a first transition map to be fused under a target camera view angle, including: determining intersection information of a transmitted light beam and the first depth image based on the preset algorithm, and determining the first transition map to be fused under the camera view angle according to pixel depth information of the intersection information; and, Processing the second depth image based on the preset algorithm to determine a second transition map to be fused under the target camera view angle; wherein, the preset algorithm is used to determine a corresponding transition map to be fused based on the depth value of the depth image; the processing method of the second depth image based on the preset algorithm is the same as that of the first depth image; Determining a target transition image from the first depth image to the second depth image based on the first transition map to be fused and the second transition map to be fused, including: performing alignment processing on the first transition map to be fused and the second transition map to be fused to determine the target transition image; Wherein, the target camera view angle is located between the first camera view angle and the second camera view angle.

2. The method according to claim 1, wherein Further including: When it is detected that a view transition condition is satisfied, taking the current depth image as the first depth image, and obtaining a second depth image of the next camera view angle adjacent to the first camera view angle; wherein, the next camera view angle corresponds to the second camera view angle; Obtaining a target transition image between the first depth image and the second depth image to switch from the first depth image to the second depth image based on the target transition image; or, Determining a target transition image from the first depth image to the second depth image based on the first depth image and the second depth image.

3. The method according to claim 2, wherein The view transition condition includes at least one of the following: Detecting that a view transition switch control is triggered; Detecting that the display duration of the current depth image reaches a preset display duration threshold.

4. The method according to claim 1, characterized in that Before obtaining the first depth image and the second depth image including the target three-dimensional object, further including: Obtaining depth images under each camera view angle based on a depth camera to determine a target transition image based on two depth images with adjacent camera view angles.

5. The method according to claim 1, characterized in that, Before processing the first depth image and / or the second depth image based on the preset algorithm, further including: For the depth image, determining at least one depth image to be applied corresponding to the current depth image with at least one preset pixel segmentation area as a resolution adjustment ratio; Performing depth marking on each pixel point in the depth image to be applied to obtain a pyramid image with the resolution arrangement of the depth image to be applied in a pyramid shape, to determine a first transition map to be fused under the target camera view angle based on the pyramid image of the first depth image, and to determine the second transition map to be fused under the target camera view angle based on the pyramid image of the second depth image; Wherein, the resolution of the depth image to be applied in the pyramid image shows a gradually increasing trend from the top layer to the bottom layer.

6. The method according to claim 5, wherein Determining at least one depth image to be applied corresponding to the current depth image with at least one preset pixel segmentation area as the resolution adjustment ratio includes: Determining at least one preset pixel segmentation area according to the resolution adjustment ratio; wherein, the number of pixel points in each preset pixel segmentation area is different; For the preset pixel segmentation area, determining at least one segmentation area to be processed corresponding to the current depth image according to the current preset pixel segmentation area, and respectively performing pixel aggregation processing on the pixel points of each segmentation area to be processed to determine the depth image to be applied corresponding to the current depth image.

7. The method according to claim 6, characterized in that, Performing depth marking on each pixel point in the depth image to be applied to obtain a pyramid image in which the resolution arrangement of the depth image to be applied is in a pyramid shape, including: For the depth image to be applied, obtaining the maximum depth value of the pixel points in each segmentation area to be processed in the current depth image to be applied, and using it as the depth value of the aggregated pixel points in the corresponding segmentation area to be processed to update the current depth image to be applied; Based on the updated depth image to be applied, determining a pyramid image in which the resolution arrangement of the depth image to be applied is in a pyramid shape.

8. The method according to claim 5, characterized in that Processing the first depth image based on the preset algorithm to determine a first transition image to be fused under the target camera view, including: Determining the emission beam under the target camera view based on the preset algorithm; Determining the first transition image to be fused based on the emission beam and the pyramid image of the first depth image.

9. The method according to claim 8, wherein The pyramid image includes at least one level, and each level corresponds to a depth image to be applied. Determining the first transition image to be fused based on the emission beam and the pyramid image of the first depth image includes: Determining whether the emission beam intersects with the depth image to be applied corresponding to the top level; If so, searching downward based on the intersection point for the intersection point information of the emission beam and the depth image to be applied at the next level, and repeatedly performing downward search for the intersection point information of the emission beam and the depth image to be applied at the next level based on the intersection point information until the depth and color of the pixel points corresponding to the absence of intersection point information are used as the depth and color of the emission beam; Determining the first transition image to be fused based on at least one color and depth corresponding to the emission beam.

10. The method according to claim 9, wherein Determining whether the emission beam intersects with the depth image to be applied corresponding to the top level includes: Determining the maximum depth value in the depth image to be applied, and determining whether the emission beam intersects with the pixel point corresponding to the maximum depth value.

11. The method according to claim 10, wherein Further including: In the case where the emission beam does not intersect with the pixel point corresponding to the maximum depth value, eliminating the emission beam, and obtaining the next emission beam and repeating the determination of whether the emission beam intersects with the depth image to be applied until the emission beam is the last emission beam.

12. The method according to claim 1, characterized in that, Determining a target transition image for transitioning from the first depth image to the second depth image based on the first transition image to be fused and the second transition image to be fused, including: Align the first transition graph to be fused and the second transition graph to be fused to determine the target transition image.

13. An image processing apparatus, characterized in that, Including: A view acquisition module for acquiring a first depth image and a second depth image including a target three-dimensional object; wherein, a first camera view of the first depth image is adjacent to a second camera view of the second depth image; A first transition graph determination module for processing the first depth image based on a preset algorithm to determine a first transition graph to be fused under a target camera view; and, A second transition graph determination module for processing the second depth image based on the preset algorithm to determine a second transition graph to be fused under the target camera view; wherein, the preset algorithm is used to determine a corresponding transition graph to be fused based on the depth value of the depth image; the processing methods of the preset algorithm for the first depth image and the second depth image are the same; A target transition graph determination module for determining a target transition image from the first depth image to the second depth image based on the first transition graph to be fused and the second transition graph to be fused; wherein, the target camera view is located between the first camera view and the camera view; The first transition graph determination module includes: an intersection information determination module for determining intersection information of an emission beam and the first depth image based on the preset algorithm, and determining the first transition graph to be fused under the camera view according to the pixel depth information of the intersection information; The target transition graph determination module further includes: an alignment module for aligning the first transition graph to be fused and the second transition graph to be fused to determine the target transition image.

14. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method according to any one of claims 1-12.

15. A storage medium containing computer-executable instructions, the computer-executable instructions being used to execute the image processing method according to any one of claims 1-12 when executed by a computer processor.

Images