Video processing method, device and readable storage medium

Abstract

The present invention discloses a video processing method, device and readable storage medium, and relates to the field of video processing technology. The method comprises: selecting at least two areas from an image frame to be processed, wherein the first area includes a target object, the second area includes the first area, and the width of the image frame to be processed is greater than the height; performing spherical mapping on the first area and the third area respectively to obtain a mapping image of the image frame to be processed, wherein the third area is an area of ​​the second area excluding the first area, and the height of the mapping image is greater than the width; and displaying the mapping image. The embodiment of the present invention can display the mapping image corresponding to the image frame to be processed.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a video processing method, device and readable storage medium. Background Art

[0002] At present, when playing a video, an intelligent algorithm is usually used to identify the main object in the video, and then the main object is cut out from the original video and displayed, thereby converting the horizontal screen playback mode to the vertical screen playback mode. However, this processing method will lose a lot of information in the original video when converting from the horizontal screen playback mode to the vertical screen playback mode, causing the video to be distorted, affecting the user's viewing experience. Summary of the invention

[0003] Embodiments of the present invention provide a video processing method, device, and readable storage medium to improve a user's viewing experience when a video is converted from a horizontal screen playback mode to a vertical screen playback mode.

[0004] In a first aspect, an embodiment of the present invention provides a video processing method, including:

[0005] Selecting at least two regions from the image frame to be processed, wherein the first region includes the target object, the second region includes the first region, and the width of the image frame to be processed is greater than the height;

[0006] Performing spherical mapping on the first region and the third region respectively to obtain a mapping image of the image frame to be processed, wherein the third region is a region of the second region excluding the first region, and the height of the mapping image is greater than the width;

[0007] The map image is displayed.

[0008] Optionally, performing spherical mapping on the first area and the third area respectively to obtain a mapping image of the image frame to be processed includes:

[0009] When the ratio of the width of the second area to the height of the screen of the terminal is greater than a preset value, determining a first mapping angle from the first area to a spherical mapping surface, wherein the spherical mapping surface takes the center of the target object as an origin;

[0010] Determining a second mapping angle from the third area to the spherical mapping surface;

[0011] Performing spherical mapping on the first area using the first mapping angle, and performing spherical mapping on the third area using the second mapping angle;

[0012] A first mapping image is obtained according to the mapping result of the first area and the mapping result of the third area.

[0013] Optionally, the first mapping angle includes a first horizontal mapping angle from the first area to the spherical mapping surface in a horizontal direction, and a first vertical mapping angle from the first area to the spherical mapping surface in a vertical direction;

[0014] The second mapping angle includes a second horizontal mapping angle of the fourth area to the spherical mapping surface in the horizontal direction, a second vertical mapping angle of the fourth area to the spherical mapping surface in the vertical direction, a third horizontal mapping angle of the fifth area to the spherical mapping surface in the horizontal direction, and a third vertical mapping angle of the fifth area to the spherical mapping surface in the vertical direction, wherein the fifth area is an area in the third area that is at the same height as the first area, and the fourth area is an area in the third area excluding the fifth area.

[0015] Optionally, the above method further includes:

[0016] The horizontal mapping angle is determined according to the mapping scale factor and the width of each region, and the vertical mapping angle is determined according to the mapping scale factor and the height of each region.

[0017] Optionally, when the ratio of the width of the second area to the height of the screen of the terminal is greater than a preset value, the method further includes:

[0018] Selecting a sixth region from the image frame to be processed, wherein the sixth region is a region adjacent to the second region in a vertical direction, and a sum of heights of the sixth region and the second region is equal to a height of the image frame to be processed;

[0019] The performing spherical mapping on the first area and the third area respectively to obtain the mapping image of the image frame to be processed includes:

[0020] Performing spherical mapping on the first area, the third area, and the sixth area respectively to obtain a second mapping image of the image frame to be processed;

[0021] The displaying of the mapping image comprises:

[0022] The second mapped image is displayed.

[0023] Optionally, performing spherical mapping on the first area, the third area, and the sixth area respectively to obtain a second mapped image of the image frame to be processed includes:

[0024] Determining a first mapping angle from the first area to a spherical mapping surface, wherein the spherical mapping surface takes the center of the target object as an origin;

[0025] Determining a second mapping angle from the third area to the spherical mapping surface;

[0026] Determining a third mapping angle from the sixth area to the spherical mapping surface;

[0027] Performing spherical mapping on the first area using the first mapping angle, performing spherical mapping on the third area using the second mapping angle, and performing spherical mapping on the sixth area using the third mapping angle;

[0028] The second mapping image is obtained according to the mapping result of the first area, the mapping result of the third area, and the mapping result of the sixth area.

[0029] Optionally, the first mapping angle includes a first horizontal mapping angle from the first area to the spherical mapping surface in a horizontal direction, and a first vertical mapping angle from the first area to the spherical mapping surface in a vertical direction;

[0030] The second mapping angle includes a second horizontal mapping angle of a fourth area to the spherical mapping surface in a horizontal direction, a second vertical mapping angle of the fourth area to the spherical mapping surface in a vertical direction, a third horizontal mapping angle of a fifth area to the spherical mapping surface in a horizontal direction, and a third vertical mapping angle of the fifth area to the spherical mapping surface in a vertical direction, wherein the fifth area is an area in the third area that is located at the same height as the first area, and the fourth area is an area in the third area excluding the fifth area;

[0031] The third mapping angle includes a fourth horizontal mapping angle of the sixth area to the spherical mapping surface in a horizontal direction, and a fourth vertical mapping angle of the sixth area to the spherical mapping surface in a vertical direction.

[0032] Optionally, the above method further includes:

[0033] receiving a first input from a user on the currently displayed mapping image;

[0034] In response to the first input, the currently displayed mapping image is switched to a target mapping image corresponding to the first input in a display form of a rotating sphere.

[0035] In a second aspect, an embodiment of the present invention further provides a video processing device, comprising: a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps in the video processing method described in the first aspect above.

[0036] In a third aspect, an embodiment of the present invention further provides a video processing device, including:

[0037] A first selection module is used to select at least two areas from the image frame to be processed, wherein the first area includes the target object, the second area includes the first area, and the width of the image frame to be processed is greater than the height;

[0038] a mapping module, configured to perform spherical mapping on the first region and the third region respectively to obtain a mapping image of the image frame to be processed, wherein the third region is a region of the second region excluding the first region, and a height of the mapping image is greater than a width;

[0039] A display module is used to display the mapping image.

[0040] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the video processing method described in the first aspect above are implemented.

[0041] In an embodiment of the present invention, at least two areas are selected from an image frame to be processed, wherein the first area includes a target object, the second area includes the first area, and the width of the image frame to be processed is greater than the height; spherical mapping is performed on the first area and the third area respectively to obtain a mapping image of the image frame to be processed, wherein the third area is an area of ​​the second area excluding the first area, and the height of the mapping image is greater than the width; displaying the mapping image can display more image information on a vertical screen, and in the process of converting the image frame to be processed into the mapping image, the image conversion is smoother, thereby improving the viewing effect of the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0043] Figure 1 is a flow chart of a video processing method provided by an embodiment of the present invention;

[0044] Figure 2 is a schematic diagram of a spherical mapping surface provided by an embodiment of the present invention;

[0045] Figure 3 It is one of the regional schematic diagrams provided in the embodiment of the present invention;

[0046] Figure 4 This is the second regional schematic diagram provided by the embodiment of the present invention;

[0047] Figure 5 yes Figure 4 Schematic diagram of parameter marking;

[0048] Figure 6 This is the third regional schematic diagram provided by the embodiment of the present invention;

[0049] Figure 7 yes Figure 6 Schematic diagram of parameter marking;

[0050] Figure 8 is a structural block diagram of a video processing device provided by an embodiment of the present invention;

[0051] Fig. 9 It is a schematic diagram of the hardware structure of a video processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] See also Figure 1 , Figure 1 is a flow chart of a video processing method provided by an embodiment of the present invention. Figure 1 As shown, the following steps are included:

[0054] Step 101, selecting at least two regions from an image frame to be processed, wherein a first region includes a target object, a second region includes the first region, and a width of the image frame to be processed is greater than a height;

[0055] In this step, the width of the image frame to be processed refers to the width of the terminal screen when the terminal is in the horizontal screen playback state, and the height of the image frame to be processed refers to the height of the terminal screen when the terminal is in the horizontal screen playback state.

[0056] First, a target detection algorithm is used to identify the main content in the image frame to be processed, a second area (i.e., the main area) including the main content is selected, and the coordinate position information of the second area is recorded. The main content can be a person, an object, etc. If the number of recognized characters is greater than or equal to a preset number, the main character is selected. The preset number is determined based on experience or experimental calibration, and is generally 3. Further, the key parts of the main content are identified, and a first area (i.e., the key area) including the key parts is selected. The key parts are the target object, such as a face. Generally, a face detection algorithm can be used for identification, and the coordinate position information of the target object and the first area is recorded. Further, based on the coordinate position information, the center of the target object and the second area (here the center of the first area and the center of the target object can be the same), the width and height of the first area, and the width and height of the second area can be determined.

[0057] It should be pointed out that when less main content is identified in the image frame to be processed, for example, when there is only one main character, the second area including the main content is adjusted to a preset size, and the coordinate position information of the second area is recorded. Then, the main character is taken as the key part, i.e., the target object, the first area is selected, and the coordinate position information of the target object and the first area is recorded. Among them, the second area can be dynamically adjusted, and the coordinate position information of the adjusted second area can be calculated using a linear method. In this way, when the main content is less, more picture information in the image frame to be processed can be retained, so that the user's viewing experience is improved.

[0058] Step 102, performing spherical mapping on the first region and the third region respectively to obtain a mapping image of the image frame to be processed, wherein the third region is a region of the second region excluding the first region, and the height of the mapping image is greater than the width;

[0059] In this step, the height of the mapped image refers to the height of the terminal screen when the terminal is in a vertical screen playback state, and the width of the mapped image refers to the width of the terminal screen when the terminal is in a vertical screen playback state.

[0060] The principle of spherical projection is adopted to perform spherical mapping on the first area, and spherical mapping is performed on the third area in the second area except the first area, so as to deform the image frame to be processed to obtain a mapped image. The mapped image can retain more picture information, reduce information loss, and has less deformation, thereby improving the user's viewing effect.

[0061] Step 103: display the mapping image.

[0062] In this step, a 3D rendering tool (such as Unity 3D rendering editing software) may be used to render and play the mapping image.

[0063] It should be pointed out that the image frame to be processed is displayed in horizontal screen playback mode, and the mapped image is displayed in vertical screen playback mode, so as to realize the switch from horizontal screen playback mode to vertical screen playback mode. Since the mapped image is displayed in vertical screen playback mode, the mapped image can retain more picture information and has less deformation. Therefore, in the process of switching from horizontal screen playback mode to vertical screen playback mode, the switching process is smooth and smooth, without abrupt experience, achieving the effect of gradually zooming in the lens, and improving the user's viewing experience.

[0064] In this embodiment, at least two areas are selected from the image frame to be processed, wherein the first area includes the target object, the second area includes the first area, and the width of the image frame to be processed is greater than the height; spherical mapping is performed on the first area and the third area respectively to obtain a mapping image of the image frame to be processed, wherein the third area is an area of ​​the second area excluding the first area, and the height of the mapping image is greater than the width; displaying the mapping image can display more image information on the vertical screen, and in the process of converting the image frame to be processed into the mapping image, the image conversion is smoother, and the image area of ​​interest to the user can be rotated to the center of the screen for display, thereby improving the user's viewing effect.

[0065] In a specific embodiment, step 102 includes:

[0066] When the ratio of the width of the second area to the height of the screen of the terminal is greater than a preset value, determining a first mapping angle from the first area to a spherical mapping surface, wherein the spherical mapping surface takes the center of the target object as an origin;

[0067] Determining a second mapping angle from the third area to the spherical mapping surface;

[0068] Performing spherical mapping on the first area using the first mapping angle, and mapping the third area using the second mapping angle;

[0069] A first mapping image is obtained according to the mapping result of the first area and the mapping result of the third area.

[0070] The height of the terminal screen refers to the height of the terminal screen when the terminal is in a horizontal screen playback state. It should be known that the terminal can be a smart phone and a tablet computer, etc. The preset value is determined according to the ratio of the height and width of the terminal screen when the terminal is in a horizontal screen playback state, which is generally 1:2.

[0071] It should be noted that the spherical mapping surface ( Figure 2 The UV plane shown in FIG. 1 takes the center of the target object as its origin, so that after switching to the vertical screen playback mode, the target object can be displayed in the center of the screen.

[0072] Specifically, the first area and the third area are first projected onto an ellipsoid, the vertical lines are mapped to the longitudes of the ellipsoid, and the horizontal lines are mapped to the latitudes of the ellipsoid to form a spherical mapping surface. Here, in order to ensure that the first area is not deformed, the vertical lines are equally spaced and mapped to the longitudes on the spherical surface, and the horizontal lines are unequally spaced and mapped to the latitudes on the spherical surface. Then, the first area is mapped to the spherical mapping surface using the first mapping angle, and the third area is mapped to the spherical mapping surface using the second mapping angle, that is, spherical mapping is achieved, and the first mapping image is obtained based on the mapping results of the first area and the mapping results of the third area, that is, the plane image is stretched and deformed and mapped to the spherical surface, and then the 3D rendering tool is used to render the image within the viewing angle corresponding to the screen size, so that more pixel information in the image frame to be processed can be displayed in the same image area, thereby improving the viewing effect of the user.

[0073] It should also be pointed out that when the ratio of the width of the second area to the height of the terminal screen is less than a preset value, the video images corresponding to the first area and the third area are directly displayed in a vertical screen playback mode without stretching or deformation.

[0074] In one embodiment, the first mapping angle includes a first horizontal mapping angle of the first area to the spherical mapping surface in a horizontal direction, and a first vertical mapping angle of the first area to the spherical mapping surface in a vertical direction;

[0075] like Figure 3 As shown, the second mapping angle includes a second horizontal mapping angle of the fourth area to the spherical mapping surface in the horizontal direction, a second vertical mapping angle of the fourth area to the spherical mapping surface in the vertical direction, a third horizontal mapping angle of the fifth area to the spherical mapping surface in the horizontal direction, and a third vertical mapping angle of the fifth area to the spherical mapping surface in the vertical direction, wherein the fifth area is an area in the third area that is located at the same height as the first area, and the fourth area is an area in the third area excluding the fifth area.

[0076] That is to say, the mapping angle of each area to the spherical mapping surface includes a mapping angle in the horizontal direction and a mapping angle in the vertical direction.

[0077] In one embodiment, the above method further includes:

[0078] The horizontal mapping angle is determined according to the mapping scale factor and the width of each region, and the vertical mapping angle is determined according to the mapping scale factor and the height of each region.

[0079] It should be noted that the first horizontal mapping angle is equal to the product of the width of the first area and the first mapping scale coefficient;

[0080] The second horizontal mapping angle is equal to the product of the width of the fourth area and the first mapping scale coefficient;

[0081] The third horizontal mapping angle is equal to the product of the width of the fifth area and the first mapping proportional coefficient;

[0082] The first vertical mapping angle is equal to the product of the height of the first area and the first mapping scale factor;

[0083] The second vertical mapping angle is equal to the product of the height of the fourth area (that is, the height difference between the second area and the first area) and the second mapping scale coefficient;

[0084] The third vertical mapping angle is equal to the product of the height of the fifth area (that is, the height of the first area) and the second mapping scale factor;

[0085] The first mapping scale factor is smaller than the second mapping scale factor.

[0086] The following Figure 4 and Figure 5 As shown, the calculation method of each mapping angle is explained:

[0087] The center coordinates of the first area (i.e. the center coordinates of the target object) are (x1, y1); the width of the first area is w1 and the height is h1; the center coordinates of the second area are (x0, y0), the width is w0 and the height is h0; the width of the terminal when in horizontal screen playback state is w and the height is h.

[0088] It should also be noted that in the vertical direction, the mapping angle from the height h of the terminal in the horizontal screen playback state to the spherical mapping surface is 2θ, and needs to satisfy formula (1):

[0089] 2θ<π (1)

[0090] Assuming that the horizontal mapping angle of the second area is θ, since the horizontal mapping angle θ of the second area is equal to the product of the width w0 of the second area and the first mapping proportional coefficient Δ, the first mapping proportional coefficient is

[0091] Furthermore, the first horizontal mapping angle The calculation method of the horizontal mapping angles of other areas is the same as that of the first horizontal mapping angle, which is equal to the product of the width of each area and the first mapping proportional coefficient, and will not be repeated here.

[0092] In summary, each region adopts the same mapping ratio coefficient in the horizontal direction, that is, in the horizontal direction, the unit pixel mapping ratio is the same, which can be understood as no deformation in the horizontal direction of each region.

[0093] Then, the first area adopts the same mapping scale coefficient in the vertical direction as in the horizontal direction, that is, the first mapping scale coefficient Δ, and the third area close to the poles of the sphere adopts a second mapping scale coefficient in the vertical direction that is larger than the first mapping scale coefficient, so that the image of the first area is not deformed in the horizontal direction and the vertical direction, and the image of the fourth area close to the poles is gradually stretched and deformed toward the poles, thereby ensuring the best image deformation effect without affecting the user's viewing effect.

[0094] Here, the second mapping ratio coefficient may be kΔ (or k 2 Δ, akΔ, etc., where k≥1, a>1).

[0095] After determining each mapping ratio coefficient, the vertical mapping angle of each area can be obtained, wherein the first vertical mapping angle is equal to h1×Δ; the second vertical mapping angle is equal to (h0-h1)×kΔ; and the third vertical mapping angle is equal to h1×kΔ.

[0096] In another specific embodiment, when the ratio of the width of the second area to the height of the screen of the terminal is greater than a preset value, the method further includes:

[0097] Selecting a sixth region from the image frame to be processed, wherein the sixth region is a region adjacent to the second region in a vertical direction, and a sum of heights of the sixth region and the second region is equal to a height of the image frame to be processed;

[0098] Step 102 includes:

[0099] Performing spherical mapping on the first area, the third area, and the sixth area respectively to obtain a second mapping image of the image frame to be processed;

[0100] Step 103 includes:

[0101] The second mapped image is displayed.

[0102] That is to say, the above method can also select the sixth area adjacent to the second area in the vertical direction, and perform spherical mapping on the first area, the third area and the sixth area respectively to obtain the second mapping image of the image frame to be processed, thereby retaining more image information in the image frame to be processed and improving the user's viewing experience.

[0103] In one embodiment, performing spherical mapping on the first area, the third area, and the sixth area respectively to obtain a second mapped image of the image frame to be processed includes:

[0104] Determining a first mapping angle from the first area to a spherical mapping surface, wherein the spherical mapping surface takes the center of the target object as an origin;

[0105] Determining a second mapping angle from the third area to the spherical mapping surface;

[0106] Determining a third mapping angle from the sixth area to the spherical mapping surface;

[0107] Performing spherical mapping on the first area using the first mapping angle, performing spherical mapping on the third area using the second mapping angle, and performing spherical mapping on the sixth area using the third mapping angle;

[0108] The second mapping image is obtained according to the mapping result of the first area, the mapping result of the third area, and the mapping result of the sixth area.

[0109] The height of the terminal screen refers to the height of the terminal when it is in a horizontal screen playback state. It should be known that the terminal can be a smart phone and a tablet computer, etc. The preset value is determined according to the ratio of the height and width of the screen when the terminal is in a horizontal screen playback state, which is generally 1:2.

[0110] It should be noted that the spherical mapping surface ( Figure 2 The UV plane shown in FIG. 1 takes the center of the target object as its origin, so that after switching to the vertical screen playback mode, the target object can be displayed in the center of the screen.

[0111] Specifically, the first area, the third area and the sixth area are projected onto an ellipsoid, the vertical lines are mapped as longitudes on the sphere, the horizontal lines are mapped as latitudes on the sphere, the first area is spherically mapped using the first mapping angle, the third area is spherically mapped using the second mapping angle, and the sixth area is spherically mapped using the third mapping angle, and the second mapping image is obtained based on the mapping results of the first area, the mapping results of the third area and the mapping results of the sixth area, that is, the plane image is stretched and deformed and mapped onto the sphere, and then a 3D rendering tool (such as unity 3D rendering editing software) is used to render an image within the viewing angle corresponding to the screen size, so that more pixel information in the image frame to be processed can be displayed in the same image area, thereby improving the user's viewing effect.

[0112] It should also be pointed out that when the ratio of the width of the second area to the height of the terminal screen is less than a preset value, the video images corresponding to the first area and the third area are directly displayed in vertical screen playback mode without stretching or deformation, and the sixth area that does not include the main content may not be displayed.

[0113] In one embodiment, the first mapping angle includes a first horizontal mapping angle of the first area to the spherical mapping surface in a horizontal direction, and a first vertical mapping angle of the first area to the spherical mapping surface in a vertical direction;

[0114] like Figure 6 As shown, the second mapping angle includes a second horizontal mapping angle of the fourth area to the spherical mapping surface in the horizontal direction, a second vertical mapping angle of the fourth area to the spherical mapping surface in the vertical direction, a third horizontal mapping angle of the fifth area to the spherical mapping surface in the horizontal direction, and a third vertical mapping angle of the fifth area to the spherical mapping surface in the vertical direction, wherein the fifth area is an area in the third area that is located at the same height as the first area, and the fourth area is an area in the third area except the fifth area;

[0115] The third mapping angle includes a fourth horizontal mapping angle of the sixth area to the spherical mapping surface in a horizontal direction, and a fourth vertical mapping angle of the sixth area to the spherical mapping surface in a vertical direction.

[0116] That is to say, the mapping angle of each area to the spherical mapping surface includes a mapping angle in the horizontal direction and a mapping angle in the vertical direction.

[0117] It should be noted that the third mapping angle is determined here under the assumption that the width of the sixth area is the same as that of the second area. If the width of the sixth area is different from that of the second area, the third mapping angle can be determined by referring to the method for determining the second mapping angle.

[0118] In one embodiment, the first horizontal mapping angle is equal to the product of the width of the first area and a first mapping scale coefficient;

[0119] The second horizontal mapping angle is equal to the product of the width of the fourth area and the first mapping scale coefficient;

[0120] The third horizontal mapping angle is equal to the product of the width of the fifth area and the first mapping proportional coefficient;

[0121] The fourth horizontal mapping angle is equal to the product of the width of the sixth area and the first mapping scale coefficient;

[0122] The first vertical mapping angle is equal to the product of the height of the first area and the first mapping scale factor;

[0123] The second vertical mapping angle is equal to the product of the height of the fourth area (that is, the height difference between the second area and the first area) and the second mapping scale coefficient;

[0124] The third vertical mapping angle is equal to the product of the height of the fifth area (that is, the height of the first area) and the second mapping scale factor;

[0125] The fourth vertical mapping angle is equal to the product of the height of the sixth area (that is, the height difference between the height of the terminal screen and the height of the second area) and the third mapping proportional coefficient;

[0126] The first mapping scale factor is less than or equal to the second mapping scale factor, and the second mapping scale factor is less than the third mapping scale factor.

[0127] The following Figure 6 and Figure 7 As shown, the calculation method of each mapping angle is explained:

[0128] The center coordinates of the first area (i.e., the center coordinates of the target object) are (x1, y1); the width of the first area is w1, and the height is h1; the center coordinates of the second area are (x0, y0), the width is w0, and the height is h0; the width of the terminal when in horizontal screen playback state is w, and the height is h; it is assumed that the height of the sixth area is equal to h-h0, and the width is equal to the width w0 of the second area.

[0129] It should be noted that in the vertical direction, the mapping angle from the height h of the terminal in the horizontal screen playback state to the spherical mapping surface is 2θ, and needs to satisfy formula (1):

[0130] 2θ<π (1)

[0131] Assuming that the horizontal mapping angle of the second area is θ, since the horizontal mapping angle θ of the second area is equal to the product of the width w0 of the second area and the first mapping proportional coefficient Δ, the first mapping proportional coefficient is

[0132] Furthermore, the first horizontal mapping angle Since the calculation method of the horizontal mapping angles of other regions is the same as the calculation method of the first horizontal mapping angle, which is equal to the product of the width of each region and the first mapping proportional coefficient, it will not be repeated here.

[0133] In summary, each region adopts the same mapping ratio coefficient in the horizontal direction, that is, in the horizontal direction, the unit pixel mapping ratio is the same, which can be understood as no deformation in the horizontal direction of each region.

[0134] Then, the first area adopts the same mapping scale coefficient in the vertical direction as in the horizontal direction, that is, the first mapping scale coefficient Δ, while the fourth and sixth areas close to the poles of the ellipsoid adopt the second and third mapping scale coefficients in the vertical direction that are larger than the first mapping scale coefficient, so that the image of the first area is not deformed in the horizontal and vertical directions, and the images of the fourth and sixth areas close to the poles are gradually stretched and deformed toward the poles, thereby ensuring the best image deformation effect without affecting the user's viewing effect.

[0135] Here, the second mapping ratio coefficient may be kΔ, and the third mapping ratio coefficient may be k 2 Δ (or other linear relationship expressions such as akΔ, where k≥1, a>1) can be understood as tensile deformation in the vertical direction.

[0136] After determining each mapping ratio coefficient, the vertical mapping angle of each area can be obtained, where the first vertical mapping angle is equal to h1×Δ; the second vertical mapping angle is equal to (h0-h1)×kΔ; the third vertical mapping angle is equal to h1×kΔ; the fourth vertical mapping angle is equal to (h-h0)×k 2 Δ.

[0137] Since in the vertical direction, the mapping angle from the height h of the terminal in the horizontal screen playback state to the spherical mapping surface is 2θ, there is formula (2):

[0138] h1Δ+(h0-h1)kΔ+(h-h0)k 2 Δ=2θ (2)

[0139] It should also be noted that since the center of the first area is not necessarily located at the center of the screen of the terminal in the vertical direction, there are the following two situations:

[0140] In the case where the center of the first area is located above the center of the screen in the vertical direction, the mapping angle corresponding to the height difference between the terminal screen and the center of the first area is smaller than the mapping angle corresponding to half the height of the terminal screen (ie, θ), and thus smaller than There is the following formula (3):

[0141]

[0142] In the case where the center of the first area is located below the center of the screen in the vertical direction, the mapping angle corresponding to the center height of the first area is smaller than the mapping angle corresponding to half the height of the terminal screen (ie, θ), and thus smaller than There is the following formula (4):

[0143]

[0144] Finally, according to formula (1) to formula (4), k (k ≥ 1) is solved to obtain the minimum θ. The minimum θ can reduce the deformation caused by spherical mapping, so that the above-mentioned mapping angles can be obtained according to the minimum θ.

[0145] Furthermore, the above method also includes:

[0146] receiving a first input from a user on the currently displayed mapping image;

[0147] In response to the first input, the currently displayed mapping image is switched to a target mapping image corresponding to the first input in a display form of a rotating sphere.

[0148] The first input may include rotation parameter information, indicating the angle of the user to rotate the screen left or right or to long press the left or right side of the screen. That is, the user can rotate the screen left or right or long press the left or right side of the screen on the mapping image currently displayed in the vertical screen playback mode to switch the viewing angle and display the target mapping image that is not currently played in the vertical screen in the image frame to be processed, that is, similar to the rotation of a sphere, a smooth transition image can be achieved.

[0149] Furthermore, the above method also includes:

[0150] receiving a second input from the user;

[0151] In response to the second input, the image frame to be processed is displayed in a horizontal screen playback mode.

[0152] The second input may be a click information of the user on the screen, and in response to the second input, the image frame to be processed in a horizontal screen playback mode is displayed to realize the display of the complete screen image. Then the image frame to be processed may receive the third input of the user, and in response to the third input, the display is switched from the currently displayed mapping image to the third mapping image corresponding to the third input, wherein the third input indicates the center coordinates of the user's area of ​​interest, and according to the center coordinates, the area size is defined, the area of ​​interest is generated, and the area of ​​interest is mapped spherically to obtain the third mapping image of the image frame to be processed.

[0153] It should be noted that when the ratio of the width of the second area to the height of the terminal screen gradually decreases to a target preset value, the deformation of the mapped image will gradually decrease, so that the user experience is further improved. Here, the target preset value is equal to the reciprocal of the preset value.

[0154] To sum up, the above method can ensure that the key area of ​​the image (i.e., the first area including the target object) remains unchanged when switching from horizontal screen playback to vertical screen playback, reduce the amount of image information loss, thereby retaining more image information in the image frame to be processed and improving the user viewing experience.

[0155] like Figure 8 As shown, an embodiment of the present invention further provides a video processing device, including:

[0156] A first selection module 801 is used to select at least two regions from the image frame to be processed, wherein the first region includes the target object, the second region includes the first region, and the width of the image frame to be processed is greater than the height;

[0157] A mapping module 802 is used to perform spherical mapping on the first area and the third area respectively to obtain a mapping image of the image frame to be processed, wherein the third area is an area of ​​the second area except the first area, and the height of the mapping image is greater than the width;

[0158] The display module 803 is used to display the mapping image.

[0159] Optionally, the mapping module 802 is specifically configured to:

[0160] When the ratio of the width of the second area to the height of the screen of the terminal is greater than a preset value, determining a first mapping angle from the first area to a spherical mapping surface, wherein the spherical mapping surface takes the center of the target object as an origin;

[0161] Determining a second mapping angle from the third area to the spherical mapping surface;

[0162] Performing spherical mapping on the first area using the first mapping angle, and performing spherical mapping on the third area using the second mapping angle;

[0163] A first mapping image is obtained according to the mapping result of the first area and the mapping result of the third area.

[0164] Optionally, the first mapping angle includes a first horizontal mapping angle from the first area to the spherical mapping surface in a horizontal direction, and a first vertical mapping angle from the first area to the spherical mapping surface in a vertical direction;

[0165] The second mapping angle includes a second horizontal mapping angle of the fourth area to the spherical mapping surface in the horizontal direction, a second vertical mapping angle of the fourth area to the spherical mapping surface in the vertical direction, a third horizontal mapping angle of the fifth area to the spherical mapping surface in the horizontal direction, and a third vertical mapping angle of the fifth area to the spherical mapping surface in the vertical direction, wherein the fifth area is an area in the third area that is at the same height as the first area, and the fourth area is an area in the third area excluding the fifth area.

[0166] Optionally, the above device further includes:

[0167] The determination module is used to determine the horizontal mapping angle according to the mapping ratio coefficient and the width of each area, and to determine the vertical mapping angle according to the mapping ratio coefficient and the height of each area.

[0168] Optionally, when the ratio of the width of the second area to the height of the screen of the terminal is greater than a preset value, the apparatus further includes:

[0169] a second selection module, configured to select a sixth region from the image frame to be processed, wherein the sixth region is a region adjacent to the second region in a vertical direction, and a sum of heights of the sixth region and the second region is equal to a height of the image frame to be processed;

[0170] The mapping module 802 includes:

[0171] A mapping unit, used for performing spherical mapping on the first area, the third area and the sixth area respectively to obtain a second mapping image of the image frame to be processed;

[0172] The display module 803 includes:

[0173] A display unit is configured to display the second mapping image.

[0174] Optionally, the mapping unit is specifically used for:

[0175] Determining a first mapping angle from the first area to a spherical mapping surface, wherein the spherical mapping surface takes the center of the target object as an origin;

[0176] Determining a second mapping angle from the third area to the spherical mapping surface;

[0177] Determining a third mapping angle from the sixth area to the spherical mapping surface;

[0178] Performing spherical mapping on the first area using the first mapping angle, performing spherical mapping on the third area using the second mapping angle, and performing spherical mapping on the sixth area using the third mapping angle;

[0179] The second mapping image is obtained according to the mapping result of the first area, the mapping result of the third area, and the mapping result of the sixth area.

[0180] Optionally, the first mapping angle includes a first horizontal mapping angle from the first area to the spherical mapping surface in a horizontal direction, and a first vertical mapping angle from the first area to the spherical mapping surface in a vertical direction;

[0181] The second mapping angle includes a second horizontal mapping angle of a fourth area to the spherical mapping surface in a horizontal direction, a second vertical mapping angle of the fourth area to the spherical mapping surface in a vertical direction, a third horizontal mapping angle of a fifth area to the spherical mapping surface in a horizontal direction, and a third vertical mapping angle of the fifth area to the spherical mapping surface in a vertical direction, wherein the fifth area is an area in the third area that is located at the same height as the first area, and the fourth area is an area in the third area excluding the fifth area;

[0182] The third mapping angle includes a fourth horizontal mapping angle of the sixth area to the spherical mapping surface in a horizontal direction, and a fourth vertical mapping angle of the sixth area to the spherical mapping surface in a vertical direction.

[0183] Optionally, the above device further includes:

[0184] A receiving module, configured to receive a first input from a user on the currently displayed mapping image;

[0185] The response module is used to switch the display from the currently displayed mapping image to the target mapping image corresponding to the first input in response to the first input in the display form of a rotating sphere.

[0186] The device provided in the embodiment of the present invention can execute the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0187] like Fig. 9 As shown, the video processing device of an embodiment of the present invention includes: a processor 900; and a memory 920 connected to the processor 900 through a bus interface, wherein the memory 920 is used to store programs and data used by the processor 900 when performing operations, and the processor 900 calls and executes the programs and data stored in the memory 920.

[0188] The processor 900 is used to read the program in the memory 920 and execute the following process:

[0189] Selecting at least two regions from the image frame to be processed, wherein the first region includes the target object, the second region includes the first region, and the width of the image frame to be processed is greater than the height;

[0190] Performing spherical mapping on the first region and the third region respectively to obtain a mapping image of the image frame to be processed, wherein the third region is a region of the second region excluding the first region, and the height of the mapping image is greater than the width;

[0191] The map image is displayed.

[0192] The transceiver 910 is configured to receive and send data under the control of the processor 900 .

[0193] Among them, Fig. 9 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 900 and various circuits of memory represented by memory 920 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 910 may be a plurality of components, namely, a transmitter and a transceiver, providing a unit for communicating with various other devices on a transmission medium. For different user devices, the user interface 930 may also be an interface capable of externally and internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0194] The processor 900 is responsible for managing the bus architecture and general processing, and the memory 920 can store data used by the processor 900 when performing operations.

[0195] Optionally, the processor 900 is further configured to read the computer program and execute the following steps:

[0196] When the ratio of the width of the second area to the height of the screen of the terminal is greater than a preset value, determining a first mapping angle from the first area to a spherical mapping surface, wherein the spherical mapping surface takes the center of the target object as an origin;

[0197] Determining a second mapping angle from the third area to the spherical mapping surface;

[0198] Performing spherical mapping on the first area using the first mapping angle, and performing spherical mapping on the third area using the second mapping angle;

[0199] A first mapping image is obtained according to the mapping result of the first area and the mapping result of the third area.

[0200] Optionally, the first mapping angle includes a first horizontal mapping angle from the first area to the spherical mapping surface in a horizontal direction, and a first vertical mapping angle from the first area to the spherical mapping surface in a vertical direction;

[0201] The second mapping angle includes a second horizontal mapping angle of the fourth area to the spherical mapping surface in the horizontal direction, a second vertical mapping angle of the fourth area to the spherical mapping surface in the vertical direction, a third horizontal mapping angle of the fifth area to the spherical mapping surface in the horizontal direction, and a third vertical mapping angle of the fifth area to the spherical mapping surface in the vertical direction, wherein the fifth area is an area in the third area that is at the same height as the first area, and the fourth area is an area in the third area excluding the fifth area.

[0202] Optionally, the processor 900 is further configured to read the computer program and execute the following steps:

[0203] The horizontal mapping angle is determined according to the mapping scale factor and the width of each region, and the vertical mapping angle is determined according to the mapping scale factor and the height of each region.

[0204] Optionally, the processor 900 is further configured to read the computer program and execute the following steps:

[0205] Selecting a sixth region from the image frame to be processed, wherein the sixth region is a region adjacent to the second region in a vertical direction, and a sum of heights of the sixth region and the second region is equal to a height of the image frame to be processed;

[0206] Performing spherical mapping on the first area, the third area, and the sixth area respectively to obtain a second mapping image of the image frame to be processed;

[0207] The second mapped image is displayed.

[0208] Optionally, the processor 900 is further configured to read the computer program and execute the following steps:

[0209] Determining a first mapping angle from the first area to a spherical mapping surface, wherein the spherical mapping surface takes the center of the target object as an origin;

[0210] Determining a second mapping angle from the third area to the spherical mapping surface;

[0211] Determining a third mapping angle from the sixth area to the spherical mapping surface;

[0212] Performing spherical mapping on the first area using the first mapping angle, performing spherical mapping on the third area using the second mapping angle, and performing spherical mapping on the sixth area using the third mapping angle;

[0213] The second mapping image is obtained according to the mapping result of the first area, the mapping result of the third area, and the mapping result of the sixth area.

[0214] Optionally, the first mapping angle includes a first horizontal mapping angle from the first area to the spherical mapping surface in a horizontal direction, and a first vertical mapping angle from the first area to the spherical mapping surface in a vertical direction;

[0215] The second mapping angle includes a second horizontal mapping angle of a fourth area to the spherical mapping surface in a horizontal direction, a second vertical mapping angle of the fourth area to the spherical mapping surface in a vertical direction, a third horizontal mapping angle of a fifth area to the spherical mapping surface in a horizontal direction, and a third vertical mapping angle of the fifth area to the spherical mapping surface in a vertical direction, wherein the fifth area is an area in the third area that is located at the same height as the first area, and the fourth area is an area in the third area excluding the fifth area;

[0216] The third mapping angle includes a fourth horizontal mapping angle of the sixth area to the spherical mapping surface in a horizontal direction, and a fourth vertical mapping angle of the sixth area to the spherical mapping surface in a vertical direction.

[0217] Optionally, the processor 900 is further configured to read the computer program and execute the following steps:

[0218] receiving a first input from a user on the currently displayed mapping image;

[0219] In response to the first input, the currently displayed mapping image is switched to a target mapping image corresponding to the first input in a display form of a rotating sphere.

[0220] The device provided in the embodiment of the present invention can execute the above method embodiment, and its implementation principle and technical effects are similar, which will not be repeated in this embodiment.

[0221] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing relevant hardware through a computer program, wherein the computer program includes instructions for executing part or all of the steps of the above method; and the computer program may be stored in a readable storage medium, and the storage medium may be any form of storage medium.

[0222] In addition, a specific embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the above-mentioned video processing method are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0223] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0224] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0225] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the sending and receiving method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0226] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A video processing method, characterized in that: include: Selecting at least two regions from the image frame to be processed, wherein the first region includes the target object, the second region includes the first region, and the width of the image frame to be processed is greater than the height; Performing spherical mapping on the first region and the third region respectively to obtain a mapping image of the image frame to be processed, wherein the third region is a region of the second region excluding the first region, and the height of the mapping image is greater than the width; The map image is displayed.

2. The video processing method according to claim 1, characterized in that: The performing spherical mapping on the first area and the third area respectively to obtain the mapping image of the image frame to be processed includes: When the ratio of the width of the second area to the height of the screen of the terminal is greater than a preset value, determining a first mapping angle from the first area to a spherical mapping surface, wherein the spherical mapping surface takes the center of the target object as an origin; Determining a second mapping angle from the third area to the spherical mapping surface; Performing spherical mapping on the first area using the first mapping angle, and performing spherical mapping on the third area using the second mapping angle; A first mapping image is obtained according to the mapping result of the first area and the mapping result of the third area.

3. The video processing method according to claim 2, characterized in that: The first mapping angle includes a first horizontal mapping angle from the first area to the spherical mapping surface in a horizontal direction, and a first vertical mapping angle from the first area to the spherical mapping surface in a vertical direction; The second mapping angle includes a second horizontal mapping angle of the fourth area to the spherical mapping surface in the horizontal direction, a second vertical mapping angle of the fourth area to the spherical mapping surface in the vertical direction, a third horizontal mapping angle of the fifth area to the spherical mapping surface in the horizontal direction, and a third vertical mapping angle of the fifth area to the spherical mapping surface in the vertical direction, wherein the fifth area is an area in the third area that is at the same height as the first area, and the fourth area is an area in the third area excluding the fifth area.

4. The video processing method according to claim 3, characterized in that: The method further comprises: Determining the horizontal mapping angle according to the mapping scale factor and the width of each region, and determining the vertical mapping angle according to the mapping scale factor and the height of each region; The mapping scale coefficient used by each region in the horizontal direction is the first mapping scale coefficient, and the mapping scale coefficient used by the first region in the vertical direction is the first mapping scale coefficient, and the first mapping scale coefficient is equal to the ratio of the horizontal mapping angle of the second region to the width of the second region; The mapping scale coefficient used by the fourth area and the fifth area in the vertical direction is a second mapping scale coefficient, and the second mapping scale coefficient is equal to k multiplied by the first mapping scale coefficient, and k≥1.

5. The video processing method according to claim 1, characterized in that: In a case where a ratio of a width of the second area to a height of a screen of the terminal is greater than a preset value, the method further includes: Selecting a sixth region from the image frame to be processed, wherein the sixth region is a region adjacent to the second region in a vertical direction, and a sum of heights of the sixth region and the second region is equal to a height of the image frame to be processed; The performing spherical mapping on the first area and the third area respectively to obtain the mapping image of the image frame to be processed includes: Performing spherical mapping on the first area, the third area, and the sixth area respectively to obtain a second mapping image of the image frame to be processed; The displaying of the mapping image comprises: The second mapped image is displayed.

6. The video processing method according to claim 5, characterized in that: The performing spherical mapping on the first area, the third area, and the sixth area respectively to obtain the mapping image of the image frame to be processed includes: Determining a first mapping angle from the first area to a spherical mapping surface, wherein the spherical mapping surface takes the center of the target object as an origin; Determining a second mapping angle from the third area to the spherical mapping surface; Determining a third mapping angle from the sixth area to the spherical mapping surface; Performing spherical mapping on the first area using the first mapping angle, performing spherical mapping on the third area using the second mapping angle, and performing spherical mapping on the sixth area using the third mapping angle; The second mapping image is obtained according to the mapping result of the first area, the mapping result of the third area, and the mapping result of the sixth area.

7. The video processing method according to claim 6, characterized in that: The first mapping angle includes a first horizontal mapping angle from the first area to the spherical mapping surface in a horizontal direction, and a first vertical mapping angle from the first area to the spherical mapping surface in a vertical direction; The second mapping angle includes a second horizontal mapping angle of a fourth area to the spherical mapping surface in a horizontal direction, a second vertical mapping angle of the fourth area to the spherical mapping surface in a vertical direction, a third horizontal mapping angle of a fifth area to the spherical mapping surface in a horizontal direction, and a third vertical mapping angle of the fifth area to the spherical mapping surface in a vertical direction, wherein the fifth area is an area in the third area that is located at the same height as the first area, and the fourth area is an area in the third area excluding the fifth area; The third mapping angle includes a fourth horizontal mapping angle of the sixth area to the spherical mapping surface in a horizontal direction, and a fourth vertical mapping angle of the sixth area to the spherical mapping surface in a vertical direction.

8. The video processing method according to claim 1, characterized in that: Also includes: receiving a first input from a user on the currently displayed mapping image; In response to the first input, the currently displayed mapping image is switched to a target mapping image corresponding to the first input in a display form of a rotating sphere.

9. A video processing device, comprising: A transceiver, a memory, a processor and a computer program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the steps in the video processing method as described in any one of claims 1 to 8.

10. A computer-readable storage medium for storing a computer program, characterized in that: When the computer program is executed by a processor, the steps in the video processing method according to any one of claims 1 to 8 are implemented.

Images