Calibration-free image processing method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a calibration-free image processing method, device, electronic equipment and storage medium. The calibration-free image processing method comprises: acquiring a fisheye image; determining an image parameter of the fisheye image and a projection model of the fisheye image; determining a camera imaging parameter according to the image parameter and the projection model of the fisheye image; and converting the fisheye image into a panoramic image of a preset angle according to the camera imaging parameter. The present disclosure does not need calibration, reduces the user operation threshold, and improves the use experience.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a calibration-free image processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the popularity of mobile devices and the development of photography technology, people's demand for shooting panoramic videos is getting higher and higher. Panoramic videos can provide a more immersive viewing experience, allowing people to feel as if they were there.

[0003] A fisheye lens can capture a wide-angle view, and convert a distorted image into a panoramic image through distortion correction and projection conversion. The fisheye lens usually needs to be calibrated when used. SUMMARY

[0004] The present disclosure provides a calibration-free image processing method and device, electronic equipment and storage medium.

[0005] The present disclosure adopts the following technical solutions.

[0006] In some embodiments, the present disclosure provides a calibration-free image processing method, comprising:

[0007] obtaining a fisheye image;

[0008] determining image parameters of the fisheye image and a projection model of the fisheye image;

[0009] determining camera imaging parameters according to the image parameters of the fisheye image and the projection model;

[0010] converting the fisheye image into a panoramic image of a preset angle according to the camera imaging parameters.

[0011] In some embodiments, the present disclosure provides a calibration-free image processing device, comprising:

[0012] an image acquisition unit configured to obtain a fisheye image;

[0013] a control unit configured to determine image parameters of the fisheye image and a projection model of the fisheye image;

[0014] The control unit is further configured to determine camera imaging parameters according to the image parameters of the fisheye image and the projection model.

[0015] The control unit is further configured to convert the fisheye image into a panoramic image of a preset angle according to the camera imaging parameters.

[0016] In some embodiments, the present disclosure provides an electronic equipment, comprising at least one memory and at least one processor.

[0017] The memory is configured to store program code, and the processor is configured to invoke the program code stored in the memory to execute the method.

[0018] In some embodiments, the present disclosure provides a computer readable storage medium configured to store program code, which, when executed by a processor, causes the processor to execute the method.

[0019] The image processing method without calibration provided by the embodiments of the present disclosure comprises: acquiring a fisheye image; determining an image parameter of the fisheye image and a projection model of the fisheye image; determining a camera imaging parameter according to the image parameter and the projection model of the fisheye image; and converting the fisheye image into a panoramic image of a preset angle according to the camera imaging parameter. The present disclosure does not need calibration, reduces the operation threshold of the user, and improves the use experience. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, same or similar reference numerals can represent same or similar elements. It should be understood that the drawings are schematic, and elements and elements are not necessarily drawn in proportion.

[0021] Figure 1 is a flowchart of the image processing method without calibration of the embodiments of the present disclosure.

[0022] Figure 2 is a schematic diagram of the fisheye image taken by different aspect ratios of the embodiments of the present disclosure.

[0023] Figure 3 is a schematic diagram of the light rays passing through the fisheye lens to the imaging plane of the embodiments of the present disclosure.

[0024] Figure 4 is an effect diagram of the fisheye image converted into a panoramic image of the embodiments of the present disclosure.

[0025] Figure 5 is a schematic diagram of the fisheye image under different preview angles of the embodiments of the present disclosure.

[0026] Figure 6 is a flowchart of the image processing method without calibration of the embodiments of the present disclosure.

[0027] Figure 7 is a structural schematic diagram of an electronic device of the embodiments of the present disclosure. DETAILED DESCRIPTION

[0028] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein, but rather, the embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are only for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0029] It should be understood that each step recited in the method embodiments of the present disclosure can be executed in sequence and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present disclosure is not limited in this respect.

[0030] The term "comprising" and variations thereof as used herein are open-ended, that is, "comprising but not limited to." The term "based on" is "based, at least in part, on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments." Related terms are defined in the following description.

[0031] It should be noted that the terms "first", "second", and the like in the present disclosure are merely used to distinguish different devices, modules or units, and do not imply the order or interdependence of the functions performed by these devices, modules or units.

[0032] It should be noted that the modification "one" in the present disclosure is illustrative rather than limiting, and those skilled in the art should understand that, unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0033] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0034] The solutions provided by the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0035] Panoramic video can provide a more immersive viewing experience, allowing users to feel as if they are there. Some technologies use special panoramic cameras or stitch multiple camera views to achieve panoramic shooting, which increases the complexity and cost of the device.

[0036] The fisheye camera can capture a wide-angle view, and then convert the fisheye image into a panoramic image through distortion correction and projection conversion, thereby reducing the complexity and cost of the device. The use of the fisheye camera requires calibration, which increases the requirements on the user. The present embodiment proposes an image processing method without calibration, which can realize the function of shooting panoramic video and reduce the requirements on the user.

[0037] As shown in Figure 1 Figure 1 is a flowchart of an image processing method without calibration according to an embodiment of the present disclosure, comprising the following steps.

[0038] S11, acquiring a fisheye image.

[0039] In some embodiments, the execution end of the method according to the present embodiment can be a terminal, such as a mobile terminal, e.g. a mobile phone, a tablet, etc. A fisheye lens can be additionally added to the camera of the terminal to realize the function of shooting panoramic video by the terminal, thereby providing a more convenient and lower-cost panoramic shooting solution. The fisheye lens can use a fisheye lens with a field of view angle reaching or exceeding 180 degrees. The fisheye lens can be aligned with the camera of the terminal itself by using a clip, or a physical interface can be reserved on the mobile phone shell of the terminal, so that the fisheye lens is aligned with the camera of the terminal itself through the physical interface. The terminal itself can have multiple cameras, among which the main camera of the camera of the terminal itself usually has the highest pixel and the best adaptation, and therefore the fisheye lens can be aligned with the main camera of the terminal. The terminal can shoot a fisheye image, which can be a complete or incomplete circular image shot by the fisheye lens. When the terminal shoots the fisheye image, the light outside the terminal can be captured after passing through the fisheye lens and the camera of the terminal itself. The camera of the terminal itself can be a 4k, 6k, 8k or higher resolution camera.

[0040] S12, determining the image parameters of the fisheye image and the projection model of the fisheye image.

[0041] In some embodiments of the present disclosure, the projection model of the fisheye image is used to fit the distortion of the fisheye lens, and the projection model can be any one of an optical projection model, an equidistant projection model, an orthogonal projection model, a perspective projection model and an equal-area projection model. The projection model can be manually selected by the user.

[0042] S13, determining the camera imaging parameters according to the image parameters of the fisheye image and the projection model.

[0043] In some embodiments, the image parameters can include the center of the circle, the radius coordinate, etc. in the image. The camera imaging parameters can be the imaging parameters of the fisheye lens, such as including the focal length, the principal point, the distortion parameters, etc.

[0044] ​S14. Convert the fisheye image into a panoramic image at a preset angle based on the camera imaging parameters.

[0045] In some embodiments, a mapping relationship between the fisheye image and the panoramic image is determined based on camera imaging parameters. This mapping relationship can be used to characterize the transformation relationship between the coordinates of pixels in the panoramic image and the coordinates of pixels in the fisheye image, thereby converting the fisheye image into a panoramic image. In some embodiments, the field of view captured by the fisheye lens has a corresponding field of view angle, for example, about 180 degrees. The fisheye image may not be a 360-degree field of view image, but rather a 180-degree field of view image. The preset angle can be equal to the field of view angle of the fisheye lens.

[0046] In some embodiments of this disclosure, a fisheye lens can be additionally installed on the terminal to enable wide-angle shooting, improving the convenience of panoramic shooting and reducing costs. In some embodiments, the fisheye lens introduces distortion during shooting. In related technologies, distortion parameters are calculated through calibration, which increases user operation and places high demands on the user. By calculating the camera imaging parameters from the captured fisheye image and applying them to the panoramic image conversion process, the fisheye image can be directly converted into a panoramic image without complex camera calibration operations, thus reducing user operation and lowering the requirements for the user.

[0047] In some embodiments of this disclosure, acquiring a fisheye image includes: acquiring a target image captured by a terminal equipped with a fisheye lens using a shooting mode with a preset aspect ratio, and acquiring the fisheye image in the target image; wherein the preset aspect ratio is not greater than 4:3 and not less than 1.

[0048] In some embodiments, the terminal can be a mobile phone, and the aspect ratio of the commonly used shooting mode of the terminal is 16:9, while the fisheye image is circular, such as... Figure 2 As shown, the terminal's sensor is typically rectangular. When the terminal uses a 16:9 aspect ratio shooting mode, either the size of the fisheye image needs to be reduced, or a portion of the image on the sensor will be cropped. When the circular image presented by the fisheye lens is fully displayed in the 4:3 area of ​​the sensor, using a 16:9 aspect ratio shooting mode will result in the left and right sides of the fisheye image being cropped. If a smaller fisheye lens is used when using a 16:9 aspect ratio shooting mode, the fisheye image can be contained within the 16:9 area. Figure 2 If the image is a dotted circle (as shown in the image), then the sensor utilization rate is low, and the effective resolution of the captured fisheye image is not high. For example, if the resolution of the terminal's sensor is 4000×3000, then... Figure 2The resolution of the fisheye image corresponding to the middle dotted circle will be 2250x2250. In some embodiments of the present disclosure, a 4:3 aspect ratio is adopted, and a 4K or higher resolution video shooting mode is used, so that the pixels of the sensor of the terminal are used to a greater extent. Assuming that the resolution of the sensor of the terminal is 4000x3000, the resolution of the fisheye image will be 3000x3000, and the number of pixels is increased by 77.8% compared with the shooting mode of the 16:9 aspect ratio.

[0049] In some embodiments, after the fisheye lens is installed on the terminal, preview can be performed to preview the original image shot and to adjust the position and angle of the fisheye lens installed on the terminal, so that the fisheye lens and the camera of the terminal itself can be aligned to obtain better shooting quality. The aligned fisheye image is a complete or approximately complete circle.

[0050] In some embodiments of the present disclosure, the camera shooting mode of the terminal is traversed first, the maximum resolution at the expected shooting frame rate (for example, 30 frames per second) is selected, and then the terminal is set to shoot at the maximum resolution at the expected shooting frame rate. Frame data is obtained in the video frame callback through the camera application interface provided by the terminal. After the video frame is obtained, it is used as a target image, preview is performed, fisheye image to panorama image conversion is performed, and then video encoding can be performed for live streaming and the like.

[0051] In some embodiments of the present disclosure, obtaining a fisheye image includes: obtaining a target image, and obtaining the fisheye image in the target image; wherein the fisheye image in the target image is obtained by: extracting a fisheye effective area in the target image to obtain the fisheye image; or pre-processing the target image, and extracting a fisheye effective area in the pre-processed target image to obtain the fisheye image.

[0052] In some embodiments, the target image can be any frame image in a video frame, and the video frame can be a video frame recorded by the terminal. After the target image is obtained, preprocessing can be performed first, such as grayscale, scaling, and filtering (such as edge-preserving filtering and median filtering). Then, a circular detection algorithm is used to extract a fisheye effective area in the target image as a fisheye image. The fisheye effective area can be an area in the target image that has an image, and does not include a black area (such as Figure 2 If the fisheye effective area is detected, the fisheye image is obtained, and subsequent steps are continued, otherwise the next frame in the video frame is obtained as a target image, and the preprocessing and fisheye effective area extraction steps are repeated.

[0053] In some embodiments of the present disclosure, the fisheye image is a complete or incomplete circular image taken by a fisheye lens, and determining the image parameters of the fisheye image comprises: determining the coordinates (x, y) of the center of the circle, the radius r in the fisheye image, or determining the coordinates (x, y) of the center of the circle, the radius r and the imaging field of view angle fov in the fisheye image. In some embodiments, the fisheye image is circular, so it has a corresponding center of the circle and radius and imaging field of view angle, and the center of the circle can be expressed in pixel coordinates in the fisheye image.

[0054] In some embodiments of the present disclosure, the camera imaging parameters comprise: focal length f, principal point coordinates (c x ,c y ), distortion parameter D. In some embodiments, the fisheye lens has a corresponding focal length and principal point, the position of the principal point on the fisheye image is expressed by the principal point coordinates, and the fisheye lens will cause distortion during imaging, so it has a corresponding distortion parameter.

[0055] In some embodiments of the present disclosure, the camera imaging parameters are determined according to the image parameters of the fisheye image and the projection model, comprising: if the projection model is an orthogonal projection model, the focal length f, the principal point coordinates (c x ,c y ), and the distortion parameter D are obtained as the camera imaging parameters, c x =x, c y =y, D is zero, and f=r / sin(fov×0.5); or, if the projection model is an equidistant projection model, the focal length f, the principal point coordinates (c x ,c y ), and the distortion parameter D are obtained as the camera imaging parameters, c x =x, c y =y, D is zero, and f=2×r / fov; or, if the projection model is an equal-area projection model, the focal length f, the principal point coordinates (c x ,c y ), and the distortion parameter D are obtained as the camera imaging parameters, c x =x, c y =y, D is zero, and f=0.5×r / sin(fov×0.25); if the projection model is a stereographic projection model, the focal length f, the principal point coordinates (c x ,c y ), and the distortion parameter D are obtained as the camera imaging parameters, c x =x, c y =y, D is zero, and f=0.5×r / tan(fov×0.25); or, if the projection model is an optical model, the focal length f, the principal point coordinates (c x ,c y ), and the distortion parameter D are obtained as the camera imaging parameters, cx = x, c y = y, D is a polynomial coefficient, the polynomial coefficient is a coefficient of a fitting polynomial with the light ray incident angle as an input and the light ray exit angle as an output, f = r / tan (θ fov ), θ fov is an output result obtained by taking fov / 2 as an input of the fitting polynomial.

[0056] In some embodiments, an optical model, an equirectangular projection model, an orthographic projection model, a perspective projection model and an equal-area projection model are provided. Any one of the above projection models can be selected by a user. Camera imaging parameters such as focal length, principal point and distortion parameters are calculated according to the selected projection model. If the projection model is an equirectangular projection model, an orthographic projection model, a perspective projection model or an equal-area projection model, the position of the principal point of the fisheye lens on the fisheye image can be considered to overlap with the center of the fisheye image, and the distortion parameter D = [0, 0, 0, 0] T , and then the focal length corresponding to the projection model is calculated by using the above formula for calculating the focal length f.

[0057] In some embodiments, the present disclosure proposes a new optical model, in which a fitting polynomial is used to fit the relationship between the light ray incident angle θ and the light ray exit angle θ d of the fisheye lens, and the fitting polynomial is: wherein θ d is the light ray exit angle, θ is the light ray incident angle, n is a positive integer not less than 2, the polynomial coefficient includes {k i |i is a positive integer from 1 to n}, and the distortion parameter D can be an n x 1 matrix, and the elements in the matrix are k i . n can be selected as 3, 4, 5 or a larger positive integer. Taking n = 4 as an example, the fitting polynomial is: d θ 2 = k1 x θ + k2 x θ 3 + k3 x θ 4 + k4 x θ d , wherein θ T is the light ray exit angle, θ is the light ray incident angle, the fitting polynomial coefficient includes k1, k2, k3 and k4, and the distortion parameter D = [k1, k2, k3, k4] fov . Correspondingly, fov / 2 is input into the fitting polynomial instead of the light ray incident angle θ, and still taking n = 4 as an example, θ 2 = k1 x (fov / 2) + k2 x (fov / 2) 3 + k3 x (fov / 2) 4 + k4 x (fov / 2) . Then the camera imaging parameters such as focal length f, principal point coordinates (c x , cy ), distortion parameter D. Wherein the polynomial coefficients can be determined in advance according to the fish-eye lens fitting.

[0058] In some embodiments of the present disclosure, converting the fisheye image into a preset angle panoramic image according to the camera imaging parameters comprises: obtaining the coordinates of a panoramic pixel point on the panoramic image; determining the coordinates of the panoramic pixel point on a unit sphere; calculating the corresponding light incidence angle and the polar angle of the light projection on the imaging plane according to the coordinates of the panoramic pixel point on the unit sphere; determining the corresponding fisheye pixel point of the panoramic pixel point on the fisheye image according to the projection model, the camera imaging parameters, the corresponding light incidence angle and the polar angle of the panoramic pixel point; and generating the panoramic image according to the corresponding fisheye pixel point of each panoramic pixel point.

[0059] In some embodiments, as shown in FIG. 2, the route of the light through the fisheye lens to the imaging plane and the top view of the imaging plane are schematically shown. In the embodiments of the present disclosure, the projection format of the panoramic image can be ERP (Equirectangular projection). Using the camera imaging parameters obtained before, the one-to-one mapping relationship between the pixel points of the fisheye image and the panoramic image is calculated according to the corresponding projection model. Specifically, the pixel coordinates (u, v) of each panoramic pixel point on the panoramic image are obtained, and then the latitude and longitude on the unit sphere (the unit sphere in the Equirectangular projection) are calculated according to the pixel coordinates, and then the three-dimensional coordinates (x, y, z) of the panoramic pixel point on the unit sphere are calculated according to the latitude and longitude, wherein x and y in the three-dimensional coordinates (x, y, z) are the coordinate values of the panoramic pixel point on the imaging plane of the unit sphere in the x direction and the y direction perpendicular to each other (see the top view of the imaging plane in FIG. 2), and z in the three-dimensional coordinates (x, y, z) is the coordinate in the z direction perpendicular to the imaging plane direction on the unit sphere (see the upper part in FIG. 2), the light incidence angle θ and the polar angle φ of the light projection on the imaging plane corresponding to the panoramic pixel point are calculated according to the three-dimensional coordinates, as shown in FIG. 2, according to the trigonometric function relationship, the light incidence angle θ and the polar angle φ corresponding to the panoramic pixel point can be calculated through the three-dimensional coordinates (x, y, z), the light incidence angle θ = arctan(z / x), and the polar angle φ = arctan(y / x), after the light incidence angle and the polar angle are obtained, the corresponding fisheye pixel point of the panoramic pixel point in the fisheye image is further determined by combining the projection model and the camera imaging parameters, and then the pixel of the corresponding fisheye pixel point is mapped to the panoramic image. Figure 3 Figure 3 Figure 3 Figure 3

[0060] ​​​​​In some embodiments of the present disclosure, determining the fisheye pixel corresponding to the panoramic pixel according to the projection model, the camera imaging parameters, the light incidence angle corresponding to the panoramic pixel, and the polar angle includes: determining the fisheye pixel corresponding to the panoramic pixel according to the following formula:

[0061] u’ = H x cos(φ) + c x ,

[0062] v’ = H x sin(φ) + c y ,

[0063] where (u’, v’) is the pixel coordinate of the fisheye pixel on the fisheye image, φ is the polar angle, (c x ,c y ) is the principal point coordinate on the fisheye image, and H is the projection length of the imaging plane light.

[0064] If the projection model is an optical model, H = f x tan θ d .

[0065] If the projection model is an orthogonal projection model, H = f x sin θ.

[0066] If the projection model is an equidistant projection model, H = f x θ.

[0067] If the projection model is an equal-area projection model, H = 2 x f x sin(θ / 2).

[0068] If the projection model is a stereographic projection model, H = 2 x f x tan(θ / 2)

[0069] is the light incidence angle, and θ d is the light output angle obtained by inputting the light incidence angle θ into the fitting polynomial.

[0070] In some embodiments, as shown above, the calculation formula between the pixel coordinate (u’, v’) of the fisheye pixel on the fisheye image, the projection length H of the imaging plane light, the polar angle φ, and the principal point coordinate (c x ,c y ) is provided. For different panoramic pixels, the corresponding projection length H of the imaging plane light, the light incidence angle θ, the polar angle φ, and the principal point coordinate (c x ,c y), the corresponding data is input into the above formula, so that the fisheye pixel corresponding to the panoramic pixel can be calculated, and for different projection models, different formulas are used to calculate the projection length H of the imaging plane light. After the fisheye pixel corresponding to the panoramic pixel is obtained, the pixel of the fisheye pixel is mapped to the position of the panoramic pixel on the panoramic image. The above steps are repeated for each panoramic pixel, so that the panoramic image is obtained. As shown in Figure 4 , the fisheye image and the generated panoramic image are schematically displayed.

[0071] In some embodiments of the present disclosure, the method further comprises: in response to a preview operation on the panoramic image, determining a preview viewing angle based on the screen size of a device previewing the panoramic image, and displaying part of the panoramic image in the preview viewing angle from the panoramic image on the screen of the device.

[0072] In some embodiments, for the converted panoramic image, which contains the content of 180 degrees of horizontal and vertical field of view, if it is previewed on a device (such as a terminal where the fisheye lens is located), as shown in Figure 5 , there will be distortion, so in order to preview on the device without showing distortion, a corresponding preview viewing angle needs to be selected according to the screen (such as the aspect ratio) of the device, for example, 40 degrees horizontally and 60 degrees vertically, and the part of the panoramic image corresponding to the preview viewing angle is projected onto the screen to be displayed. At the same time, the preview direction can be changed by dragging, for example, by a gesture, so that the content of different viewing angles in the panoramic image can be seen. Figure 5 The mobile phone preview viewing angle 1 and the mobile phone preview viewing angle 2 in schematically show the preview effect of different preview viewing angles.

[0073] In some embodiments of the present disclosure, video encoding and / or live streaming technology can be used to encode the converted panoramic image into a video, so as to realize panoramic video recording or live streaming pushing.

[0074] As shown in Figure 6 , it schematically shows the method in some embodiments of the present disclosure, taking a mobile phone as an example, an fisheye lens is installed on the mobile phone, a 4K or higher high-definition circular fisheye image is shot on the mobile phone with an aspect ratio of 4:3, and fisheye preview can be performed to adjust the fisheye lens. Then, after the graph detection is passed, the fisheye image is converted into a panoramic image through camera model parameters (camera imaging parameters and projection model), and the panoramic image can be previewed. Then, it is used for video recording and / or live streaming of the panoramic image.

[0075] In some embodiments of the present disclosure, the algorithm for fitting distortion parameters and converting to generate a panoramic image does not need to perform a complex fisheye camera calibration operation, uses a terminal with a fisheye lens, and can generate a panoramic image (about 180 degrees) in real time on the terminal, achieving about 180-degree panoramic image shooting in the horizontal and vertical directions. The entire process has a fast processing speed and good user experience.

[0076] In some embodiments of the present disclosure, a calibration-free image processing device is also provided, characterized by comprising:

[0077] An image acquisition unit is configured to acquire a fisheye image, wherein the fisheye image is a complete or incomplete circular image captured by a fisheye lens.

[0078] A control unit is configured to determine an image parameter of the fisheye image and a projection model of the fisheye image.

[0079] The control unit is further configured to determine a camera imaging parameter according to the image parameter of the fisheye image and the projection model.

[0080] The control unit is further configured to convert the fisheye image into a panoramic image of a preset angle according to the camera imaging parameter.

[0081] In some embodiments, the fisheye image is acquired by:

[0082] A terminal equipped with a fisheye lens is configured to capture a target image in a shooting mode with a preset aspect ratio, and acquire the fisheye image in the target image.

[0083] In some embodiments, the fisheye image is acquired by:

[0084] In some embodiments, the fisheye image is acquired by:

[0085] In some embodiments, the fisheye image in the target image is acquired by:

[0086] In some embodiments, the fisheye image is a complete or incomplete circular image captured by a fisheye lens, and the image parameter of the fisheye image is determined by:

[0087] In some embodiments, the camera imaging parameters include focal length f, principal point coordinates (c x ,c y ), and distortion parameters D.

[0088] In some embodiments, the camera imaging parameters are determined according to the image parameters of the fisheye image and the projection model, including:

[0089] If the projection model is an orthogonal projection model, focal length f, principal point coordinates (c x ,c y ), and distortion parameters D are obtained as the camera imaging parameters, c x =x, c y =y, D is zero, and f=r / sin(fov×0.5); or,

[0090] If the projection model is an equidistant projection model, focal length f, principal point coordinates (c x ,c y ), and distortion parameters D are obtained as the camera imaging parameters, c x =x, c y =y, D is zero, and f=2×r / fov; or,

[0091] If the projection model is an equal-area projection model, focal length f, principal point coordinates (c x ,c y ), and distortion parameters D are obtained as the camera imaging parameters, c x =x, c y =y, D is zero, and f=0.5×r / sin(fov×0.25);

[0092] If the projection model is a perspective projection model, focal length f, principal point coordinates (c x ,c y ), and distortion parameters D are obtained as the camera imaging parameters, c x =x, c y =y, D is zero, and f=0.5×r / tan(fov×0.25); or,

[0093] If the projection model is an optical model, focal length f, principal point coordinates (c x ,c y ) on the pixel of the fisheye image, and distortion parameters D are obtained as the camera imaging parameters, c x =x, c y =y, D is a polynomial coefficient, the polynomial coefficient is a coefficient of a fitting polynomial with the input of the incident angle of a light ray and the output of the exit angle of the light ray, f=r / tan(θ fov ), and θ fovan output result obtained by taking fov / 2 as an input of the fitting polynomial.

[0094] In some embodiments, the fitting polynomial is: where θ d is an exit angle of a light ray, θ is an incident angle of the light ray, n is a positive integer not less than 2, and the polynomial coefficients include {k i |i is a positive integer from 1 to n}.

[0095] In some embodiments, converting the fisheye image into a preset angle panoramic image according to the camera imaging parameters includes:

[0096] obtaining a coordinate of a panoramic pixel point on the panoramic image on the panoramic image;

[0097] determining a coordinate of the panoramic pixel point on a unit sphere;

[0098] calculating a corresponding light ray incident angle and a polar angle of a projection of the light ray on an imaging plane according to the coordinate of the panoramic pixel point on the unit sphere;

[0099] determining a corresponding fisheye pixel point of the panoramic pixel point on the fisheye image according to the projection model, the camera imaging parameters, the corresponding light ray incident angle of the panoramic pixel point, and the polar angle;

[0100] generating the panoramic image according to the corresponding fisheye pixel points of the respective panoramic pixel points.

[0101] In some embodiments, determining the corresponding fisheye pixel point of the panoramic pixel point on the fisheye image according to the projection model, the camera imaging parameters, the corresponding light ray incident angle of the panoramic pixel point, and the polar angle includes:

[0102] determining the corresponding fisheye pixel point of the panoramic pixel point by using the following formula:

[0103] u’ = H × cos(φ) + c x ,

[0104] v’ = H × sin(φ) + c y ,

[0105] where (u’, v’) is a pixel coordinate of a fisheye pixel point on the fisheye image, φ is the polar angle, (c x , c y ) is a principal point coordinate on the fisheye image, and H is a projection length of an imaging plane light ray.

[0106] If the projection model is an optical model, then H = f × tan θ d .

[0107] If the projection model is an orthogonal projection model, then H = f x sin θ;

[0108] If the projection model is an equidistant projection model, then H = f x θ;

[0109] If the projection model is an equal-area projection model, then H = 2 x f x sin(θ / 2);

[0110] If the projection model is a perspective projection model, then H = 2 x f x tan(θ / 2)

[0111] θ is the light ray incidence angle, θ d is the light ray output angle obtained by inputting the light ray incidence angle into the fitting polynomial.

[0112] In some embodiments, the control unit is further configured to, in response to a preview operation on the panoramic image, determine a preview viewing angle based on a screen size of a device previewing the panoramic image, and display a portion of the panoramic image at the preview viewing angle on a screen of the device.

[0113] For the embodiments of the device, since they basically correspond to the embodiments of the method, the relevant parts are described in the part of the description of the embodiments of the method. The above-described device embodiments are merely illustrative, and the modules described as separate modules can or can not be separate. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments. Those skilled in the art can understand and implement without creative labor.

[0114] So far, the method and device of the present disclosure have been described based on the embodiments and application examples. In addition, the present disclosure also provides an electronic device and a computer readable storage medium, which are described below.

[0115] Reference is made below to Figure 7 which shows a structural schematic diagram of an electronic device (such as a terminal device or a server) 800 suitable for implementing the embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, as well as fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in the figure is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0116] The electronic device 800 can include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801 that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 802 or loaded into a random access memory (RAM) 803 from a storage device 808. Various programs and data required for the operation of the electronic device 800 are also stored in the RAM 803. The processing device 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0117] In general, the following devices can be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication devices 809 can allow the electronic device 800 to communicate wirelessly or wired with other devices to exchange data. While the electronic device 800 is shown with various devices, it should be understood that all of the illustrated devices are not required to be implemented or present. More or fewer devices can alternatively be implemented or present.

[0118] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication devices 809, or installed from the storage devices 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-described functions defined in the methods of the embodiments of the present disclosure are performed.

[0119] It should be noted that the computer-readable medium described above can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In the disclosure, the computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, in which the computer-readable program code is contained. Such a propagated data signal can take any of a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. The computer-readable signal medium can also be any computer-readable medium that is not a storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including, but not limited to, wire, cable, RF (radio frequency), or the like, or any suitable combination of the foregoing.

[0120] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0121] The computer-readable medium described above can be included in the electronic device; or can exist separately from the electronic device, and can be accessed via the electronic device.

[0122] The computer-readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method of the disclosure described above.

[0123] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0124] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0125] The units described in the embodiments of the present disclosure can be implemented by hardware, software, or a combination of hardware and software. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0126] The functions described in this specification can be implemented in part or in whole through one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0127] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more of: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0128] According to one or more embodiments of the present disclosure, a calibration-free image processing method is provided, comprising:

[0129] obtaining a fisheye image;

[0130] determining an image parameter of the fisheye image and a projection model of the fisheye image;

[0131] determining a camera imaging parameter according to the image parameter of the fisheye image and the projection model;

[0132] converting the fisheye image into a panoramic image of a preset angle according to the camera imaging parameter.

[0133] According to one or more embodiments of the present disclosure, a calibration-free image processing method is provided, and the fisheye image is obtained, comprising:

[0134] obtaining a target image by a terminal equipped with a fisheye lens in a preset aspect ratio shooting mode, and obtaining the fisheye image in the target image;

[0135] wherein the preset aspect ratio is not greater than 4:3 and not less than 1.

[0136] According to one or more embodiments of the present disclosure, a calibration-free image processing method is provided, and the fisheye image is obtained, comprising:

[0137] wherein the fisheye image in the target image is obtained by extracting a fisheye effective area in the target image to obtain the fisheye image, or by pre-processing the target image, extracting a fisheye effective area in the pre-processed target image to obtain the fisheye image.

[0138] According to one or more embodiments of the present disclosure, a calibration-free image processing method is provided, the fisheye image is a complete or incomplete circular image taken by a fisheye lens, and the image parameters of the fisheye image are determined, including: determining the center coordinates (x, y) and radius r of the fisheye image, or determining the center coordinates (x, y), radius r and imaging field of view angle fov of the fisheye image.

[0139] and / or,

[0140] The camera imaging parameters include: focal length f, principal point coordinates (c x ,c y ), and distortion parameter D.

[0141] According to one or more embodiments of the present disclosure, a calibration-free image processing method is provided, and the camera imaging parameters are determined according to the image parameters of the fisheye image and the projection model, including:

[0142] If the projection model is an orthogonal projection model, the focal length f, principal point coordinates (c x ,c y ), and distortion parameter D are obtained as the camera imaging parameters, c x =x, c y =y, D is zero, and f=r / sin(fov×0.5); or,

[0143] If the projection model is an equidistant projection model, the focal length f, principal point coordinates (c x ,c y ), and distortion parameter D are obtained as the camera imaging parameters, c x =x, c y =y, D is zero, and f=2×r / fov; or,

[0144] If the projection model is an equal-area projection model, the focal length f, principal point coordinates (c x ,c y ), and distortion parameter D are obtained as the camera imaging parameters, c x =x, c y =y, D is zero, and f=0.5×r / sin(fov×0.25);

[0145] If the projection model is a stereographic projection model, the focal length f, principal point coordinates (c x ,c y ), and distortion parameter D are obtained as the camera imaging parameters, c x =x, c y =y, D is zero, and f=0.5×r / tan(fov×0.25); or,

[0146] If the projection model is an optical model, a focal length f, a principal point in pixel coordinates (c x ,c y ) on the fisheye image, and distortion parameters D are obtained as camera imaging parameters, c x =x, c y =y, and D is a polynomial coefficient, which is a coefficient of a fitting polynomial with an input of a light ray incident angle and an output of a light ray exit angle, f=r / tan(θ fov ), and θ fov is an output result obtained by taking fov / 2 as an input of the fitting polynomial.

[0147] According to one or more embodiments of the present disclosure, a calibration-free image processing method is provided, and the fitting polynomial is: wherein θ d is a light ray exit angle, θ is a light ray incident angle, n is a positive integer not less than 2, and the polynomial coefficient includes {k i |i is a positive integer from 1 to n}.

[0148] According to one or more embodiments of the present disclosure, a calibration-free image processing method is provided, and the fisheye image is converted into a panoramic image of a preset angle according to the camera imaging parameters, including:

[0149] Obtaining coordinates of a panoramic pixel point on the panoramic image on the panoramic image;

[0150] Determining coordinates of the panoramic pixel point on a unit sphere;

[0151] Calculating a corresponding light ray incident angle and a polar angle of a light ray projection on an imaging plane according to the coordinates of the panoramic pixel point on the unit sphere;

[0152] Determining a corresponding fisheye pixel point of the panoramic pixel point on the fisheye image according to the projection model, the camera imaging parameters, the corresponding light ray incident angle of the panoramic pixel point, and the polar angle;

[0153] Generating the panoramic image according to the corresponding fisheye pixel point of each panoramic pixel point.

[0154] According to one or more embodiments of the present disclosure, a calibration-free image processing method is provided, and the fisheye pixel point of the panoramic pixel point on the fisheye image is determined according to the projection model, the camera imaging parameters, the corresponding light ray incident angle of the panoramic pixel point, and the polar angle, including:

[0155] The fisheye pixel point corresponding to the panoramic pixel point is determined by using the following formula:

[0156] u’=H×cos(φ)+cx ,

[0157] v' = H x sin(φ) + c y ,

[0158] wherein (u', v') is the pixel coordinate of the fisheye pixel point on the fisheye image, φ is the polar angle, (c x , c y ) is the principal point coordinate on the fisheye image, H is the projection length of the imaging plane light ray;

[0159] if the projection model is an optical model, H = f x tan θ d ;

[0160] if the projection model is an orthogonal projection model, H = f x sin θ;

[0161] if the projection model is an equirectangular projection model, H = f x θ;

[0162] if the projection model is an equal-area projection model, H = 2 x f x sin(θ / 2);

[0163] if the projection model is a perspective projection model, H = 2 x f x tan(θ / 2)

[0164] θ is the light ray incidence angle, θ d is the light ray output angle obtained by inputting the light ray incidence angle into the fitting polynomial.

[0165] According to one or more embodiments of the present disclosure, a calibration-free image processing method is provided, further comprising:

[0166] in response to a preview operation on the panoramic image, determining a preview viewing angle based on the screen size of a device previewing the panoramic image, and displaying part of the panoramic image in the preview viewing angle from the panoramic image on the screen of the device.

[0167] According to one or more embodiments of the present disclosure, a calibration-free image processing apparatus is provided, comprising:

[0168] an image acquisition unit configured to acquire a fisheye image;

[0169] a control unit configured to determine image parameters of the fisheye image and a projection model of the fisheye image;

[0170] the control unit is further configured to determine camera imaging parameters according to the image parameters of the fisheye image and the projection model;

[0171] the control unit is further configured to convert the fisheye image into a panoramic image of a preset angle according to the camera imaging parameters.

[0172] According to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one memory and at least one processor;

[0173] The at least one memory is configured to store program codes, and the at least one processor is configured to invoke the program codes stored in the at least one memory to execute the method according to any one of the above.

[0174] According to one or more embodiments of the present disclosure, a computer readable storage medium is provided, the computer readable storage medium is configured to store program codes, the program codes, when executed by a processor, cause the processor to execute the above method.

[0175] The above description is merely preferred embodiments of the present disclosure and a description of the applied technical principles. Those skilled in the art should understand that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by the combinations of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the technical solutions formed by replacing the above features with the technical features disclosed in the present disclosure (but not limited to) having similar functions.

[0176] In addition, although each operation is depicted in a particular order, this should not be understood as requiring the operations to be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing can be advantageous. Similarly, although several implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be combined in a single embodiment. Conversely, various features described in the context of a single embodiment can also be separated and implemented in multiple embodiments.

[0177] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A calibration-free image processing method, characterized in that, include: Obtain fisheye image; Determine the image parameters of the fisheye image and the projection model of the fisheye image; Based on the image parameters of the fisheye image and the projection model, determine the camera imaging parameters; The fisheye image is converted into a panoramic image at a preset angle based on the camera imaging parameters. The fisheye image is a complete or incomplete circular image captured by a fisheye lens. Determining the image parameters of the fisheye image includes: determining the center coordinates (x, y) and radius r of the circle in the fisheye image; or, determining the center coordinates (x, y), radius r, and imaging field of view fov of the circle in the fisheye image. The camera imaging parameters include: focal length f, principal point coordinates (c...). x ,c y ), distortion parameter D.

2. The method according to claim 1, characterized in that, The acquisition of the fisheye image includes: The target image is captured by a terminal equipped with a fisheye lens using a shooting mode with a preset aspect ratio, and the fisheye image in the target image is obtained. The preset aspect ratio is no greater than 4:3 and no less than 1.

3. The method according to claim 1, characterized in that, Acquiring a fisheye image includes: acquiring a target image, and acquiring the fisheye image within the target image; The method of obtaining the fisheye image from the target image includes: extracting the effective fisheye region from the target image to obtain the fisheye image; or, preprocessing the target image and extracting the effective fisheye region from the preprocessed target image to obtain the fisheye image.

4. The method according to claim 1, characterized in that, Based on the image parameters of the fisheye image and the projection model, the camera imaging parameters are determined, including: If the projection model is an orthogonal projection model, obtain the focal length f and the principal point coordinates (c). x ,c y The distortion parameter D is used as the camera imaging parameter, c x =x,c y =y, D is zero, f=r / sin(fov×0.5); or, If the projection model is an isometric projection model, obtain the focal length f and the principal point coordinates (c). x ,c y The distortion parameter D is used as the camera imaging parameter, c x =x,c y =y, D is zero, f=2×r / fov; or If the projection model is an equal-area projection model, obtain the focal length f and the principal point coordinates (c). x ,c y The distortion parameter D is used as the camera imaging parameter, c x =x,c y =y, D is zero, f=0.5×r / sin(fov×0.25); If the projection model is a stereoscopic projection model, obtain the focal length f and the principal point coordinates (c). x ,c y The distortion parameter D is used as the camera imaging parameter, c x =x,c y =y, D is zero, f=0.5×r / tan(fov×0.25); or, If the projection model is an optical model, then obtain the focal length f and the pixel coordinates (c) of the principal point on the fisheye image. x ,c y The distortion parameter D is used as the camera imaging parameter, c x =x,c y =y, D are polynomial coefficients, which are the coefficients of each term of the fitted polynomial with the incident angle of the light ray as input and the exit angle of the light ray as output, f=r / tan(θ) fov ), θ fov The output result is obtained by taking fov / 2 as the input of the fitted polynomial.

5. The method according to claim 4, characterized in that, The fitting polynomial is: θ d = , where θ d Let θ be the angle of incidence of the light ray, θ be the angle of incidence of the light ray, and n be a positive integer not less than 2. The polynomial coefficients include {k} i | i is a positive integer from 1 to n}.

6. The method according to claim 1, characterized in that, Converting the fisheye image into a panoramic image at a preset angle based on the camera imaging parameters includes: Obtain the coordinates of the panoramic pixels in the panoramic image; Determine the coordinates of the panoramic pixel on a unit sphere; The corresponding incident angle of light and the polar angle of the light projection on the imaging plane are calculated based on the coordinates of the panoramic pixel on the unit sphere. The fisheye pixel corresponding to the panoramic pixel on the fisheye image is determined based on the projection model, the camera imaging parameters, the light incident angle corresponding to the panoramic pixel, and the polar angle. The panoramic image is generated based on the fisheye pixels corresponding to each of the panoramic pixels.

7. The method according to claim 6, characterized in that, Determining the fisheye pixel corresponding to the panoramic pixel in the fisheye image based on the projection model, the camera imaging parameters, the incident angle of light corresponding to the panoramic pixel, and the polar angle includes: The fisheye pixel corresponding to the panoramic pixel is determined using the following formula: u'=H×cos(φ)+c x , v’=H×sin(φ)+c y , Where (u',v') are the pixel coordinates of the fisheye pixel in the fisheye image, φ is the polar angle, (c x ,c y ) represents the coordinates of the principal point on the fisheye image, and H represents the projected length of the light ray on the imaging plane; If the projection model is an optical model, then H = f × tanθ d ; If the projection model is an orthogonal projection model, then H = f × sinθ; If the projection model is an equidistant projection model, then H = f × θ; If the projection model is an equal-area projection model, then H = 2 × f × sin(θ / 2); If the projection model is a stereoscopic projection model, then H = 2 × f × tan(θ / 2). θ is the angle of incidence of the light ray. d The output angle of the light is obtained by fitting a polynomial to the incident angle of the light.

8. The method according to claim 1, characterized in that, Also includes: In response to a preview operation of the panoramic image, a preview angle is determined based on the screen size of the device previewing the panoramic image, and a portion of the panoramic image is displayed on the screen of the device from the panoramic image at the preview angle.

9. A calibration-free image processing device, characterized in that, include: Image acquisition unit, used to acquire fisheye images; A control unit is used to determine the image parameters of the fisheye image and the projection model of the fisheye image; The control unit is further configured to determine camera imaging parameters based on the image parameters of the fisheye image and the projection model; The control unit is also used to convert the fisheye image into a panoramic image at a preset angle based on the camera imaging parameters; The fisheye image is a complete or incomplete circular image captured by a fisheye lens. Determining the image parameters of the fisheye image includes: determining the center coordinates (x, y) and radius r of the circle in the fisheye image, or determining the center coordinates (x, y), radius r, and imaging field of view fov of the circle in the fisheye image. The camera imaging parameters include: focal length f, principal point coordinates (c) x ,c y ), the coefficients of each term in the composite polynomial of the distortion parameter D.

10. An electronic device, comprising: At least one memory and at least one processor; The at least one memory is used to store program code, and the at least one processor is used to call the program code stored in the at least one memory to execute the method of any one of claims 1 to 8.

11. A computer-readable storage medium for storing program code that, when executed by a processor, causes the processor to perform the method of any one of claims 1 to 8.

Images