Panoramic image processing method and device, computer device, medium and program product

Abstract

The application relates to a panoramic image processing method and device, computer equipment, a medium and a program product. The method comprises the following steps: acquiring observation angles for observing a panoramic image at multiple preset time points; determining a first angle variation range according to N observation angles; determining an actual angle sampling range of the panoramic image according to a blur intensity parameter and the first angle variation range; sequentially extracting the N observation angles in the actual angle sampling range, and obtaining N observation images corresponding to the N observation angles in the panoramic image; wherein N is a positive integer; and calculating a target blur image according to the N observation images. The method can realize the blur effect of image additional residual image and the visual effect of a bionic human eye.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a panoramic image processing method, apparatus, computer equipment, medium, and program product. Background Technology

[0002] When observing a landscape with the human eye, due to the phenomenon of visual persistence, the electrochemical phenomena of the retina cause a visual reaction time. This means that when the human eye sees a scene at a certain moment, the scene will not disappear from the brain's vision for a short period of time. Therefore, during the rapid change of the human eye's field of vision, the scene seen by the human eye is often blurry and has afterimages.

[0003] For any panoramic image, the viewing angle is constantly changing during camera movement, which causes the observed planar image to also constantly change. Each change in the planar image is equivalent to a momentary scene seen by the human eye.

[0004] However, when performing general camera movement editing on panoramic footage, each frame during the movement is very sharp, resulting in a very sharp edited image, which does not match real human vision. Therefore, how to give the edited image an additional blurring effect with afterimages and a biomimetic human eye perspective has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] Therefore, it is necessary to provide a panoramic image processing method, apparatus, computer equipment, medium, and program product that can add a blurring effect of afterimages and a biomimetic human eye visual effect to images to address the above-mentioned technical problems.

[0006] Firstly, this application provides a panoramic image processing method. The method includes:

[0007] Obtain the observation perspectives of the panoramic image at multiple preset times;

[0008] The range of change of the first perspective is determined based on multiple observation perspectives;

[0009] The actual view sampling range of the panoramic image is determined based on the blur intensity parameter and the range of the first view variation.

[0010] Within the actual view sampling range, N observation viewpoints are sequentially sampled, and N observation images corresponding to the N observation viewpoints in the panoramic image are obtained; where N is a positive integer;

[0011] The blurred image of the target is calculated based on N observed images.

[0012] In one embodiment, the first viewing angle variation range is the viewing angle variation range between a first time point and a target time point and / or the viewing angle variation range between the target time point and a second time point; wherein, the first time point, the target time point, and the second time point are the times observed sequentially among the preset times, and the time interval between the first time point and the target time point is equal to the time interval between the target time point and the second time point;

[0013] Determining the actual view sampling range of the panoramic image based on the blur intensity parameter and the first view variation range includes:

[0014] Among the multiple observation perspectives, the target observation perspective corresponding to the target time is marked as the second sampling perspective;

[0015] The first sampling perspective is calculated based on the target observation perspective, the first observation perspective corresponding to the first moment, and the fuzziness intensity parameter.

[0016] The third sampling perspective is calculated based on the target observation perspective, the second observation perspective corresponding to the second time, and the fuzziness intensity parameter.

[0017] The range of perspective change from the first sampling viewpoint to the second sampling viewpoint and / or the range of perspective change from the second sampling viewpoint to the third sampling viewpoint is marked as the actual perspective sampling range.

[0018] In one embodiment, the actual view sampling range is the view variation range from the first sampling view to the second sampling view and the view variation range from the second sampling view to the third sampling view.

[0019] Within the actual view sampling range, the N observation viewpoints are sequentially sampled, and N observation images corresponding to the N observation viewpoints in the panoramic image are obtained, including:

[0020] Within the range of perspective change from the first sampling perspective to the second sampling perspective, n1 observation perspectives are sequentially and equally sampled; where n1 = N / 2, and n1 is a positive integer;

[0021] Within the range of perspective change from the second sampling perspective to the third sampling perspective, n2 observation perspectives are sequentially and equally sampled; where n2 = N / 2, and n2 is a positive integer;

[0022] The n1 and n2 observation viewpoints are combined to form N observation viewpoints. The panoramic image is sampled based on the N observation viewpoints to obtain N observation images.

[0023] In one embodiment, calculating the target blurred image based on N observed images includes:

[0024] Assign corresponding weights to each of the observed images;

[0025] The target blurred image is obtained by performing a weighted summation process based on the pixel values ​​of each observed image and the corresponding weights.

[0026] In one embodiment, assigning corresponding weights to each of the observed images includes:

[0027] According to the extraction order of the observed images, the initial values ​​of each observed image are set; wherein, the initial values ​​of each observed image are increased sequentially according to the extraction order;

[0028] The initial values ​​are normalized to obtain the weights corresponding to each observed image.

[0029] In one embodiment, the step of performing a weighted summation process based on the pixel values ​​of each observed image and the corresponding weights to obtain the target blurred image includes:

[0030] Obtain the initial pixel value of each pixel in each of the observed images;

[0031] The initial pixel values ​​and corresponding weights of each pixel in each observed image are weighted and summed to obtain the target pixel values ​​of each pixel in the target blurred image.

[0032] Secondly, this application also provides a panoramic image processing apparatus. The apparatus includes:

[0033] The observation perspective acquisition module is used to acquire the observation perspectives of the panoramic image at multiple preset times.

[0034] The first-viewpoint variation range determination module is used to determine the first-viewpoint variation range based on multiple observation viewpoints.

[0035] The actual view sampling range determination module is used to determine the actual view sampling range of the panoramic image based on the blur intensity parameter and the first view variation range;

[0036] The observation image acquisition module is used to sequentially extract N observation perspectives within the actual view sampling range and obtain N observation images corresponding to the N observation perspectives in the panoramic image; where N is a positive integer;

[0037] The target blurred image calculation module is used to calculate the target blurred image based on N observed images.

[0038] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the first aspect embodiment.

[0039] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the first aspect embodiment.

[0040] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the first aspect embodiment.

[0041] The aforementioned panoramic image processing method, apparatus, computer equipment, medium, and program product acquire observation angles of a panoramic image at multiple preset times, then determine the range of change of a first viewpoint based on the observation angles, and then determine the actual viewpoint sampling range of the panoramic image based on the blur intensity parameter and the range of change of the first viewpoint. Next, N observation angles are sequentially sampled within the actual viewpoint sampling range, and N observation images corresponding to these N observation angles in the panoramic image are obtained. Finally, a target blurred image is calculated based on these N observation images. The technical solution of this application determines the range of change of the first viewpoint based on the observation angle to simulate the change of the human eye's observation angle. Then, N observation angles are sequentially sampled within the actual viewpoint sampling range, and N observation images corresponding to these N observation angles in the panoramic image are obtained. This facilitates the calculation of the target blurred image based on the N observation images, generating a blurred image that simulates the change in the human eye's observation angle. This achieves the effect of adding a blurring effect to an edited image and adds a biomimetic human eye visual effect. Attached Figure Description

[0042] Figure 1 This is an application environment diagram of a panoramic image processing method in one embodiment;

[0043] Figure 2 This is a flowchart illustrating a panoramic image processing method in one embodiment;

[0044] Figure 3 This is a flowchart illustrating the calculation of the range of change of the first viewpoint and the range of change of the second viewpoint in one embodiment.

[0045] Figure 4 This is a flowchart illustrating the process of calculating the actual view sampling range in one embodiment;

[0046] Figure 5 This is a flowchart illustrating the process of calculating a blurred target image in one embodiment;

[0047] Figure 6 This is a flowchart illustrating a panoramic image processing method in another embodiment;

[0048] Figure 7 This is a structural block diagram of a panoramic image processing device in one embodiment;

[0049] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] The panoramic image processing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Terminal 102 acquires observation angles of the panoramic image observed at multiple preset times, then determines the range of change of the first viewpoint based on the multiple observation angles; then, based on the blur intensity parameter and the range of change of the first viewpoint, it determines the actual viewpoint sampling range of the panoramic image; then, within the actual viewpoint sampling range, it sequentially samples N observation angles and obtains N observation images corresponding to the N observation angles in the panoramic image; finally, it calculates the target blurred image based on the N observation images. Terminal 102 can be various personal computers, laptops, smartphones, tablets, cameras, or other electronic devices with shooting and image processing functions. Server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0052] In some embodiments, such as Figure 2 As shown, a panoramic image processing method is provided, which can be applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps:

[0053] Step 202: Obtain the observation angles for observing the panoramic image at multiple preset times.

[0054] The panoramic image can be a directly input image or a video frame from a panoramic video. It should be noted that if the panoramic image is a video frame from a panoramic video, then the panoramic image processing method described in this embodiment is applied to each video frame in the panoramic video.

[0055] It should be noted that panoramic images can be composed of stitched images from multiple sub-planar images, or they can be captured by electronic devices with shooting capabilities. This application does not impose specific limitations on this. For example, panoramic images can be obtained by shooting devices with front and rear fisheye lenses.

[0056] The preset time can refer to a time that is set in advance.

[0057] The viewing angle refers to the angle formed at the optical center of the observer's eye when observing a panoramic image, with light rays emanating from both ends of the panoramic image visible to the observer. The smaller the size of the panoramic image and the greater its distance from the observer, the smaller the viewing angle. It's important to understand that while the viewing angle of a panoramic image is numerically the same at different times, the specific image observed can differ.

[0058] For example, through a specific mapping relationship, the pixel information in a panoramic image can be transformed onto a sphere in a three-dimensional coordinate system, with the observer positioned at the center of this sphere. At any given moment, the observer can only observe a portion of the content on this sphere. The angle formed by the light rays emanating from both ends of this portion at the observer's optical center is the viewing angle.

[0059] When processing an image whose camera movement process is known within a given timeframe, the viewing angle of that image at every moment in the timeframe is known. In this embodiment, the viewing angle of the panoramic image is known at every moment, meaning the viewing angle of the panoramic image is known at multiple preset moments.

[0060] In some embodiments, the panoramic image may be an image pre-stored on a server or other storage device. For example, the panoramic image may be an image pre-stored on a server, and the panoramic image stored on the server may be obtained through a network or other communication methods, and the observation angles of the panoramic image observed at multiple preset times may be obtained.

[0061] In some embodiments, the panoramic image can be an image captured in real time by an electronic device with shooting capabilities. For example, a camera is used to take a panoramic picture of a target object to form a panoramic image, and the captured panoramic image is transmitted to a terminal (which can be a personal computer, laptop, etc.) via a network or other communication method. The terminal receives the captured panoramic image and obtains the shooting angle of the camera at each moment when it captures the target object, thereby obtaining the observation angle of observing the panoramic image at multiple preset moments.

[0062] Step 204: Determine the range of change of the first perspective based on multiple observation perspectives.

[0063] Among them, the range of change of the first-person perspective can refer to the range of change of the observation perspective of the observer when observing the panoramic image at multiple preset times.

[0064] For example, the sequential changes of multiple perspectives in time can be used as the range of changes of the first perspective.

[0065] For example, suppose there are five preset times, which occur sequentially in time sequence, and are named a, b, c, d, and e respectively. The range of change of the observation viewpoint corresponding to time a to the observation viewpoint corresponding to time e can be taken as the first perspective range.

[0066] In one embodiment, assume there are three preset times, which occur sequentially in time and are named time a, time b, and time c, respectively. Time a occurs before times b and c, time b occurs between times a and c, and time c occurs after times a and b. Each preset time corresponds to an observation perspective for observing the panoramic image. Therefore, the range of the first viewpoint can be the range of viewpoint variation between time a and time b, and the range of viewpoint variation between time b and time c.

[0067] The technical solution of this application embodiment determines the range of change of the first perspective according to the sequential change of the observation perspective, which can sense the motion state and direction of the lens, thereby realizing the observation of the simulated human eye, so that the subsequently generated blurred image is more in line with the actual observation effect of the human eye, and improves the accuracy of the generated bionic human eye visual effect.

[0068] In one embodiment, the first perspective change range can be the perspective change range between a first moment and a target moment and / or the perspective change range between a target moment and a second moment; wherein, the first moment, the target moment, and the second moment are moments observed sequentially within a preset time, and the time interval between the first moment and the target moment is equal to the time interval between the target moment and the second moment.

[0069] For example, suppose that in multiple preset times, the time interval between each time is equal, and there are two times a1 and a2 before the target time, and two times b1 and b2 after the target time.

[0070] When the first time point is a1, the second time point is b1. The time interval between a1 and the target time point is equal to the time interval between the target time point and b1. Therefore, the first viewpoint range can be the viewpoint variation range between a1 and the target time point and the viewpoint variation range between the target time point and b1.

[0071] When the first time point is a2, the second time point is b2. The time interval between a2 and the target time point is equal to the time interval between the target time point and b2. Therefore, the first viewpoint range can be the viewpoint change range between a2 and the target time point and the viewpoint change range between the target time point and b2. The time interval between a2 and the target time point is greater than the time interval between a1 and the target time point.

[0072] The technical solution of this application embodiment sets the time interval between the first time and the target time to be equal to the time interval between the target time and the second time, so that the range of change of the first viewpoint can be changed with the selection of the first time and the second time, thereby facilitating the change of the value of the range of change of the first viewpoint, and thus facilitating the change of the accuracy of the subsequent actual viewpoint sampling range, thereby improving the adaptability of the panoramic image processing method.

[0073] In one embodiment, please refer to Figure 3 Step 204 includes, but is not limited to, the following steps:

[0074] Step 302: Among multiple observation perspectives, determine the first observation perspective corresponding to the first moment, the target observation perspective corresponding to the target moment, and the second observation perspective corresponding to the second moment.

[0075] In some embodiments, A t Let A represent the target observation perspective at the target time. Then, the first observation perspective at the first time can be represented by A. t-1 This indicates that the second observation perspective corresponding to the second moment can be represented by A. t+1 express.

[0076] Given that the observation perspective at each moment in the time series is known, the first observation perspective corresponding to the first moment is obtained based on the first moment and the known observation perspective. Similarly, the target observation perspective corresponding to the target moment and the second observation perspective corresponding to the second moment can be obtained.

[0077] Step 304: Determine the range of change of the second perspective based on the first observation perspective and the target observation perspective.

[0078] The range of change in the second perspective can be represented by the first observation perspective and the target observation perspective. For example, the range of change in the second perspective can be represented by: A t-1 →A t The range of perspective change from the first moment to the target moment can be represented by: A t-1 →A t express.

[0079] In some embodiments, once the first observation viewpoint and the target observation viewpoint are determined, the range of change in the observer's viewpoint during the process of switching from the first observation viewpoint to the target observation viewpoint can be used as the range of change in the second viewpoint.

[0080] Step 306: Determine the range of change of the third perspective based on the target observation perspective and the second observation perspective.

[0081] The range of change in the third-person perspective can be represented by the target observation perspective and the second observation perspective. For example, the range of change in the third-person perspective can be represented by: A t →A t+1 The range of perspective change between the target time and the second time can be represented by: A t →A t+1 express.

[0082] In some embodiments, after determining the target observation viewpoint and the second observation viewpoint, the range of viewpoint change of the observer during the process of switching from the target observation viewpoint to the second observation viewpoint can be used as the third viewpoint range of change, and the first viewpoint range of change can be the viewpoint range of change after merging the second viewpoint range of change and the third viewpoint range of change.

[0083] Specifically, when the target time is the initial time (i.e., the target time is the 0th time), the corresponding first-viewpoint change range is the viewpoint change range from the target time to the second time. When the target time is the end time (i.e., the target time is the last time in the time sequence), the corresponding first-viewpoint change range is the viewpoint change range from the first time to the target time.

[0084] For example, if the observer rotates horizontally at a uniform angular velocity, the range of change in the first-person perspective can be obtained in the following way:

[0085] By obtaining the observer's viewing angle at the first moment, the target moment, and the second moment, the range of change of the first viewing angle can be determined as follows: Obtain the first duration difference between the target moment and the first moment. If this first duration difference is large, multiply it by the angular velocity of rotation to obtain the rotation angle. Add this rotation angle to the viewing angle at the first moment to obtain the range of change of the second viewing angle. Then, obtain the second duration difference between the target moment and the second moment. If this second duration difference is large, multiply it by the angular velocity of rotation to obtain the rotation angle. Add this rotation angle to the viewing angle at the target moment to obtain the range of change of the third viewing angle. After determining the ranges of change of the second and third viewing angles, the range of change of the first viewing angle can be determined.

[0086] Step 206: Determine the actual view sampling range of the panoramic image based on the blur intensity parameter and the range of change of the first view.

[0087] The blur intensity parameter refers to the sampling intensity parameter used to sample the panoramic image. This blur intensity parameter is a pre-set adjustable parameter, which can be preset by the user or automatically set by the processor. The specific value of the blur intensity parameter can be set according to specific circumstances, and this application does not impose specific restrictions on it. The blur intensity parameter can be represented by K, where K∈[0,1]. When the blur intensity parameter K is closer to 0, the corresponding actual viewpoint sampling range is smaller; when the blur intensity parameter is closer to 1, the corresponding actual viewpoint sampling range is larger.

[0088] The actual view sampling range can refer to the range of view sampling performed on a panoramic image.

[0089] In some embodiments, the actual viewpoint sampling range can be controlled by controlling the value of the blur intensity parameter.

[0090] For example, by merging the second-view and third-view variation ranges and then setting the blur intensity parameter to 0.5, the actual view sampling range is half the sum of the second and third-view variation ranges, meaning the actual view sampling range is half the first-view variation range. Specifically, when the second-view and third-view variation ranges are numerically equal, the actual view sampling range is numerically equal to the second-view variation range. In this case, half of the second-view variation range and half of the third-view variation range can be used to obtain the actual view sampling range. It should be understood that other methods can also be used to obtain the actual view sampling range; this application does not impose specific limitations on this.

[0091] Step 208: Within the actual view sampling range, N observation viewpoints are sequentially sampled, and N observation images corresponding to the N observation viewpoints in the panoramic image are obtained; where N is a positive integer.

[0092] Among them, the observation image can refer to the image of the panoramic image observed by the observer from the observation perspective.

[0093] Within the aforementioned actual viewpoint sampling range, N observation viewpoints are sequentially extracted from the panoramic image, and the corresponding observation images of each observation viewpoint in the panoramic image are obtained, resulting in N observation images.

[0094] For example, an unequal sampling method can be adopted, in which N observation views are sequentially and unequally sampled from the panoramic image within the actual view sampling range, resulting in N observation images. For instance, a first view is randomly selected from the panoramic image, and then a second view is selected from the panoramic image after the first view at a randomly obtained sampling interval.

[0095] For example, an equal sampling method can also be adopted, in which N observation views are equally sampled from the panoramic image within the actual view sampling range, and then N observation images corresponding to each observation view in the panoramic image are obtained.

[0096] Step 210: Calculate the blurred target image based on N observed images.

[0097] Among them, the target blurred image can refer to the image after processing the panoramic image, with the addition of afterimage blur effect and bionic human eye perspective effect.

[0098] In some embodiments, the blurred image of the target can be obtained by weighted summation of N observed images.

[0099] In the above panoramic image processing method, the range of change of the first viewpoint is determined according to the observation viewpoint to simulate the change of the human eye's observation viewpoint. Then, N observation viewpoints are sequentially sampled within the actual viewpoint sampling range, and N observation images corresponding to the N observation viewpoints in the panoramic image are obtained. This facilitates the calculation of the target blurred image based on the N observation images, so as to generate a blurred image that simulates the human eye's observation viewpoint during the change process. This achieves the blurring effect of adding afterimages to the edited image, as well as the addition of a bionic human eye's visual effect.

[0100] In some embodiments, such as Figure 4 As shown, step 206 includes, but is not limited to, the following steps:

[0101] Step 402: Among multiple observation perspectives, mark the target observation perspective corresponding to the target time as the second sampling perspective.

[0102] Here, B1 can be used to represent the second sampling perspective, then B1 = A t The observation perspective corresponding to the target time is used as the second sampling perspective corresponding to the target time.

[0103] Step 404: Calculate the first sampling perspective based on the target observation perspective, the first observation perspective corresponding to the first moment, and the fuzzy intensity parameter.

[0104] Here, B0 can be used to represent the first sampling perspective, and the first sampling perspective B0 can be represented by the following formula (1), which is as follows:

[0105] B0 = At -(A t -A t-1 )*K (1)

[0106] In formula (1), K represents the fuzzy intensity parameter, and A t A represents the target observation perspective at the target time. t-1 This indicates the first observation perspective corresponding to the first moment.

[0107] By substituting the pre-set fuzzy intensity parameters, target observation angle, and first observation angle into formula (1), the first sampling angle can be calculated.

[0108] Step 406: Calculate the third sampling perspective based on the target observation perspective, the second observation perspective corresponding to the second time point, and the fuzzy intensity parameter.

[0109] In some embodiments, B2 can be used to represent the third sampling perspective, and the third sampling perspective B2 can be represented by the following formula (2), which is as follows:

[0110] B2 = A t +(A t+1 -A t )*K (2)

[0111] In formula (2), K represents the fuzzy intensity parameter, and A t A represents the target observation perspective at the target time. t+1 This indicates the second observation perspective corresponding to the second moment.

[0112] By substituting the pre-set fuzzy intensity parameter, the target observation angle, and the second observation angle into formula (2), the third sampling angle can be calculated.

[0113] Step 408: Mark the range of view variation from the first sampling viewpoint to the second sampling viewpoint and / or the range of view variation from the second sampling viewpoint to the third sampling viewpoint as the actual viewpoint sampling range.

[0114] In some embodiments, the actual view sampling range can be the view variation range from the first sampling view to the second sampling view, the view variation range from the second sampling view to the third sampling view, or the view variation range from the first sampling view to the second sampling view and the view variation range from the second sampling view to the third sampling view.

[0115] For example, when the actual sampling range is the range of perspective change from the first sampling viewpoint to the second sampling viewpoint and the range of perspective change from the second sampling viewpoint to the third sampling viewpoint, the actual sampling range can be represented by B0→B1→B2. Here, B0→B1 represents the change from the first sampling viewpoint to the second sampling viewpoint; and B1→B2 represents the change from the second sampling viewpoint to the third sampling viewpoint.

[0116] In this embodiment, the actual view sampling range can be controlled by controlling the range of change of the first view and the blur intensity parameter.

[0117] In one embodiment, when the target time is the initial time (i.e., the target time is the 0th time), the corresponding first perspective change range is the perspective change range from the target time to the second time, and the actual perspective sampling range is the perspective change range from the second sampling perspective to the third sampling perspective. When the target time is the end time (i.e., the target time is the last time in the time sequence), the corresponding first perspective change range is the perspective change range from the first time to the target time, and the actual perspective sampling range is the perspective change range from the first sampling perspective to the third sampling perspective.

[0118] In some embodiments, the actual view sampling range is the view variation range from the first sampling view to the second sampling view and the view variation range from the second sampling view to the third sampling view. Step 208 includes, but is not limited to, the following steps: within the view variation range from the first sampling view to the second sampling view, n1 observation views are sequentially and equally sampled; where n1 = N / 2, and n1 is a positive integer; within the view variation range from the second sampling view to the third sampling view, n2 observation views are sequentially and equally sampled; where n2 = N / 2, and n2 is a positive integer; the n1 and n2 observation views are combined to form N observation views, and the panoramic image is sampled according to the sampled N observation views to obtain N observation images.

[0119] For example, an equal sampling method can be adopted to perform equal sampling within the range of perspective change from the first sampling perspective to the second sampling perspective, resulting in n1 observation perspectives.

[0120] For example, we can use B0 to represent the first sampling perspective, B1 to represent the second sampling perspective, and B2 to represent the third sampling perspective. Then, within the range of perspective change from B0 to B1, if n1 perspectives are equally sampled, it can be represented by the following formula (3):

[0121]

[0122] In formula (3), n1 represents the number of extracted viewpoints, n1 = N / 2, n1 is an integer, and C iThis represents the viewpoint extracted within the range of viewpoint change from B0 to B1, where i = 1, 2, ..., n1.

[0123] Similarly, within the range of perspective change from B1 to B2, n2 perspectives are equally sampled, which can be represented by the following formula (4), specifically:

[0124]

[0125] In formula (4), n2 represents the number of extracted viewpoints, n2 = N / 2, where n2 is an integer, and D j This represents the viewpoint extracted within the range of viewpoint change from B1 to B2, where j = 1, 2, ..., n2.

[0126] For example, when N=30, within the range of perspective change from B0 to B1, 15 observation perspectives that are sequentially continuous in time are extracted, and within the range of perspective change from B1 to B2, 15 observation perspectives that are sequentially continuous in time are extracted.

[0127] In summary, within the actual view sampling range B0→B1→B2, N observation views that are sequentially continuous were extracted. Then, the panoramic image was sampled based on the extracted N observation views to obtain the observation image corresponding to each observation view.

[0128] In one embodiment, when the actual view sampling range is the view variation range from the first sampling view to the second sampling view or the view variation range from the second sampling view to the third sampling view, N observation views can be directly sampled from the actual view sampling range.

[0129] like Figure 5 As shown, in some embodiments, step 210 includes, but is not limited to, the following steps:

[0130] Step 502: Assign corresponding weights to each observed image.

[0131] Step 504: Perform weighted summation based on the pixel values ​​and corresponding weights of each observed image to obtain the blurred target image.

[0132] For example, a weight is assigned to each of the N observed images. For instance, W1 represents the weight corresponding to the first observed image in time series, W2 represents the weight corresponding to the second observed image in time series, and so on, with W... N This represents the weight corresponding to the Nth observation image in the time series.

[0133] It should be noted that the sum of the weights is set to 1, that is... Where i = 1, 2, ..., N.

[0134] After determining the weight corresponding to each observed image, a weighted summation is performed based on the pixel value of each observed image and its corresponding weight to obtain the target blurred image.

[0135] In some embodiments, step 502 includes, but is not limited to, the following steps: setting initial values ​​for each observation image according to the extraction order of the observation images; wherein the initial values ​​of each observation image are incremented according to the extraction order; and normalizing each initial value to obtain the weight corresponding to each observation image.

[0136] In this embodiment, by setting incremental weights, the proportion of the observed image corresponding to the observation viewpoint that is earlier in the time sequence is smaller, and the proportion of the observed image corresponding to the observation viewpoint that is later in the time sequence is larger. This makes the camera movement blur effect have a strong sense of direction, allowing users to feel the specific movement and thus improving the user experience.

[0137] For example, let the initial value W i =i, then for W i After normalization, a weight that monotonically increases over time is obtained. That is, let W1 = 1, W2 = 2, ..., W... N =N, and then normalize each initial value to obtain a weight that monotonically increases over time. Then, perform a weighted summation based on the pixel values ​​of each observed image and the normalized weight to obtain the blurred target image.

[0138] In some embodiments, step 504 includes, but is not limited to, the following steps: obtaining the initial pixel value corresponding to each pixel in each observation image; performing weighted summation on the initial pixel value and corresponding weight of each pixel in each observation image to obtain the target pixel value of each pixel in the target blurred image.

[0139] A blurred target image can be composed of several pixels. Once the pixel value of each pixel is determined, the blurred target image is determined.

[0140] First, obtain the initial pixel value corresponding to each pixel in each observation image. Then, use weights to weight the initial pixel values ​​of each observation image to obtain the weighted pixel value. Next, obtain the sum of the weighted pixel values ​​of pixels at the same position in each observation image to obtain the target pixel value. After determining the target pixel value corresponding to each pixel in the observation image, the target blurred image is obtained.

[0141] For example, let I represent the final blurred target image, and let the target pixel value of the first pixel in the first row and first column of the blurred target image I be denoted by I. 11 In other words, I 11 It can be calculated using the following formula (5), which is as follows:

[0142]

[0143] In formula (5), a i This represents the initial pixel value in the first row and first column of the i-th observation image out of N observation images.

[0144] Similarly, the target pixel values ​​of other pixels in the target blurred image are calculated using the same method as in formula (5) to obtain the final target blurred image.

[0145] In some embodiments, such as Figure 6 As shown, the panoramic image processing method includes, but is not limited to, the following steps:

[0146] Step 602: Obtain the observation angles for observing the panoramic image at multiple preset times.

[0147] Step 604: Determine the range of change of the first viewing angle based on multiple observation perspectives; the range of change of the first viewing angle is the range of change of the viewing angle from the first moment to the target moment and / or the range of change of the viewing angle from the target moment to the second moment; wherein, the first moment, the target moment and the second moment are the moments observed sequentially in the preset moments, and the time interval between the first moment and the target moment is equal to the time interval between the target moment and the second moment.

[0148] Step 606: Among multiple observation perspectives, mark the target observation perspective corresponding to the target time as the second sampling perspective.

[0149] Step 608: Calculate the first sampling perspective based on the target observation perspective, the first observation perspective corresponding to the first moment, and the fuzzy intensity parameter.

[0150] Step 610: Calculate the third sampling perspective based on the target observation perspective, the second observation perspective corresponding to the second time point, and the fuzzy intensity parameter.

[0151] Step 612: Mark the range of view variation from the first sampling viewpoint to the second sampling viewpoint and / or the range of view variation from the second sampling viewpoint to the third sampling viewpoint as the actual viewpoint sampling range.

[0152] Step 614: Within the range of perspective change from the first sampling perspective to the second sampling perspective, n1 observation perspectives are sequentially and equally sampled; where n1 = N / 2, and n1 is a positive integer.

[0153] Step 616: Within the range of perspective change from the second to the third sampling perspective, n2 observation perspectives are sequentially and equally sampled; where n2 = N / 2, and n2 is a positive integer.

[0154] Step 618: Combine n1 and n2 observation viewpoints to form N observation viewpoints. Sample the panoramic image based on the extracted N observation viewpoints to obtain N observation images.

[0155] Step 620: Set the initial values ​​for each observation image according to the extraction order; wherein the initial values ​​of each observation image increase sequentially according to the extraction order.

[0156] Step 622: Normalize each initial value to obtain the weight corresponding to each observed image.

[0157] Step 624: Perform weighted summation based on the pixel values ​​and corresponding weights of each observed image to obtain the blurred target image.

[0158] It should be noted that the embodiments of steps 602 to 624 are described in the foregoing specific steps.

[0159] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0160] Based on the same inventive concept, this application also provides a panoramic image processing apparatus for implementing the panoramic image processing method described above.

[0161] In some embodiments, such as Figure 7 As shown, a panoramic image processing device is provided, including: an observation view acquisition module 702, a first view change range determination module 704, an actual view sampling range determination module 706, an observation image acquisition module 708, and a target blurred image calculation module 710, wherein:

[0162] The observation perspective acquisition module 702 is used to acquire the observation perspectives of the panoramic image at multiple preset times.

[0163] The first-view variation range determination module 704 is used to determine the first-view variation range based on multiple observation viewpoints.

[0164] The actual view sampling range determination module 706 is used to determine the actual view sampling range of the panoramic image based on the blur intensity parameter and the range of change of the first view.

[0165] The observation image acquisition module 708 is used to sequentially extract N observation viewpoints within the actual viewpoint sampling range and obtain N observation images corresponding to the N observation viewpoints in the panoramic image; where N is a positive integer.

[0166] The target blurred image calculation module 710 is used to calculate the target blurred image based on N observed images.

[0167] In some embodiments, the first perspective change range is the perspective change range between a first time point and a target time point and / or the perspective change range between a target time point and a second time point; wherein, the first time point, the target time point, and the second time point are the times observed sequentially within a preset time period, and the time interval between the first time point and the target time point is equal to the time interval between the target time point and the second time point.

[0168] The actual perspective sampling range determination module 706 includes:

[0169] The second sampling perspective determination unit is used to mark the target observation perspective corresponding to the target time as the second sampling perspective among multiple observation perspectives.

[0170] The first sampling perspective calculation unit is used to calculate the first sampling perspective based on the target observation perspective, the first observation perspective corresponding to the first moment, and the fuzzy intensity parameter.

[0171] The third sampling perspective calculation unit is used to calculate the third sampling perspective based on the target observation perspective, the second observation perspective corresponding to the second time moment, and the fuzzy intensity parameter.

[0172] The first marking unit is used to mark the range of view variation from the first sampling viewpoint to the second sampling viewpoint and / or the range of view variation from the second sampling viewpoint to the third sampling viewpoint as the actual viewpoint sampling range.

[0173] In some embodiments, the actual view sampling range is the view variation range from the first sampling view to the second sampling view and the view variation range from the second sampling view to the third sampling view. The observation image acquisition module 708 includes:

[0174] The first perspective extraction unit is used to sequentially and equally extract n1 observation perspectives within the perspective change range from the first sampling perspective to the second sampling perspective; where n1 = N / 2, and n1 is a positive integer.

[0175] The second perspective extraction unit is used to sequentially and equally extract n2 observation perspectives within the perspective variation range from the second sampling perspective to the third sampling perspective; where n2 = N / 2, and n2 is a positive integer.

[0176] The image sampling unit is used to combine n1 and n2 observation viewpoints to form N observation viewpoints, and to sample the panoramic image based on the extracted N observation viewpoints to obtain N observation images.

[0177] In some embodiments, the target blurred image calculation module 710 includes:

[0178] The weight setting unit is used to set the corresponding weights for each observed image.

[0179] The weighted summation unit is used to perform weighted summation based on the pixel values ​​of each observed image and their corresponding weights to obtain the blurred target image.

[0180] In some embodiments, the weight setting unit includes:

[0181] The initial value setting subunit is used to set the initial value of each observation image according to the extraction order of the observation images; wherein the initial value of each observation image increases in the extraction order.

[0182] The normalization processing subunit is used to normalize each initial value to obtain the weight corresponding to each observed image.

[0183] In some embodiments, the weighted summation unit includes:

[0184] The initial pixel value acquisition subunit is used to acquire the initial pixel value corresponding to each pixel in each observed image.

[0185] The weighted processing subunit is used to perform weighted summation on the initial pixel values ​​and corresponding weights of each pixel in each observed image to obtain the target pixel values ​​of each pixel in the target blurred image.

[0186] Each module in the aforementioned panoramic image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0187] In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, communication interface, display unit, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a panoramic image processing method. The display unit can be a liquid crystal display (LCD) or an e-ink display. The input device can be a touch layer covering the display unit, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0188] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0189] In some embodiments, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the panoramic image processing method described above.

[0190] In some embodiments, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of a panoramic image processing method.

[0191] In some embodiments, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the panoramic image processing method described above.

[0192] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0193] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0194] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A panoramic image processing method, characterized in that, The method includes: Obtain the observation perspectives of the panoramic image at multiple preset times; The range of change of the first perspective is determined based on multiple observation perspectives; The actual view sampling range of the panoramic image is determined based on the blur intensity parameter and the range of the first view change. Within the actual view sampling range, N observation viewpoints are sequentially sampled, and N observation images corresponding to the N observation viewpoints in the panoramic image are obtained; where N is a positive integer; The blurred image of the target is calculated based on N observed images.

2. The method according to claim 1, characterized in that, The first viewing angle variation range is the viewing angle variation range between the first time point and the target time point and / or the viewing angle variation range between the target time point and the second time point; wherein, the first time point, the target time point and the second time point are the times observed sequentially in the preset time points, and the time interval between the first time point and the target time point is equal to the time interval between the target time point and the second time point; Determining the actual view sampling range of the panoramic image based on the blur intensity parameter and the first view variation range includes: Among the multiple observation perspectives, the target observation perspective corresponding to the target time is marked as the second sampling perspective; The first sampling perspective is calculated based on the target observation perspective, the first observation perspective corresponding to the first moment, and the fuzziness intensity parameter. The third sampling perspective is calculated based on the target observation perspective, the second observation perspective corresponding to the second time, and the fuzziness intensity parameter. The range of perspective change from the first sampling viewpoint to the second sampling viewpoint and / or the range of perspective change from the second sampling viewpoint to the third sampling viewpoint is marked as the actual perspective sampling range.

3. The method according to claim 2, characterized in that, The actual view sampling range is the view variation range from the first sampling view to the second sampling view and the view variation range from the second sampling view to the third sampling view. Within the actual view sampling range, the N observation viewpoints are sequentially sampled, and N observation images corresponding to the N observation viewpoints in the panoramic image are obtained, including: Within the range of perspective change from the first sampling perspective to the second sampling perspective, n1 observation perspectives are sequentially and equally sampled; where n1 = N / 2, and n1 is a positive integer; Within the range of perspective change from the second sampling perspective to the third sampling perspective, n2 observation perspectives are sequentially and equally sampled, where n2 = N / 2 and n2 is a positive integer; The n1 and n2 observation viewpoints are combined to form N observation viewpoints. The panoramic image is sampled based on the N observation viewpoints to obtain N observation images.

4. The method according to any one of claims 1 to 3, characterized in that, The step of calculating the blurred target image based on N observed images includes: Assign corresponding weights to each of the observed images; The target blurred image is obtained by performing a weighted summation process based on the pixel values ​​of each observed image and the corresponding weights.

5. The method according to claim 4, characterized in that, The step of assigning corresponding weights to each of the observed images includes: According to the extraction order of the observed images, the initial values ​​of each observed image are set; wherein, the initial values ​​of each observed image are increased sequentially according to the extraction order; The initial values ​​are normalized to obtain the weights corresponding to each observed image.

6. The method according to claim 4, characterized in that, The step of performing a weighted summation process based on the pixel values ​​of each observed image and the corresponding weights to obtain the target blurred image includes: Obtain the initial pixel value of each pixel in each of the observed images; The initial pixel values ​​and corresponding weights of each pixel in each observed image are weighted and summed to obtain the target pixel values ​​of each pixel in the target blurred image.

7. A panoramic image processing device, characterized in that, The device includes: The observation perspective acquisition module is used to acquire the observation perspectives of the panoramic image at multiple preset times. The first-viewpoint variation range determination module is used to determine the first-viewpoint variation range based on multiple observation viewpoints. The actual view sampling range determination module is used to determine the actual view sampling range of the panoramic image based on the blur intensity parameter and the first view variation range; The observation image acquisition module is used to sequentially extract N observation perspectives within the actual view sampling range and obtain N observation images corresponding to the N observation perspectives in the panoramic image; where N is a positive integer; The target blurred image calculation module is used to calculate the target blurred image based on N observed images.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images