Image processing method, device, terminal device and computer readable storage medium

Abstract

The embodiments of the present application disclose an image processing method, apparatus, terminal device and computer-readable storage medium. The method is applied to a terminal device, and the method includes: segmenting a first image to obtain a main display image and a secondary display image; obtaining the posture information of the terminal device; transforming the main display image and the secondary display image respectively according to the posture information; splicing the transformed main display image and the transformed secondary display image to obtain a target image. The above-mentioned image processing method, apparatus, terminal device and computer-readable storage medium can enrich the image processing method and improve the interactivity between users and images.

Description

Technical Field

[0001] The present application relates to the field of imaging technology, and in particular to an image processing method, apparatus, terminal device and computer-readable storage medium. Background Art

[0002] With the further development of science and technology, more and more terminal devices are equipped with powerful camera systems, which may include one or more different cameras to meet the user's needs for camera in different scenes. At present, the processing method for images taken by terminal devices is relatively simple, and the interaction between users and images is poor. Summary of the invention

[0003] The embodiments of the present application disclose an image processing method, apparatus, terminal device and computer-readable storage medium, which can enrich the image processing method and improve the interactivity between users and images.

[0004] The present application embodiment discloses an image processing method, which is applied to a terminal device, and the method includes:

[0005] Segmenting the first image to obtain a main display image and a secondary display image;

[0006] Obtaining position information of the terminal device;

[0007] transforming the main display image and the secondary display image respectively according to the posture information;

[0008] The transformed main display image and the transformed secondary display image are spliced ​​to obtain a target image.

[0009] The embodiment of the present application discloses an image processing device, which is applied to a terminal device, and the device includes:

[0010] An image segmentation module, used for segmenting the first image to obtain a main display image and a secondary display image;

[0011] A posture acquisition module, used to obtain the posture information of the terminal device;

[0012] A transformation module, used to transform the main display image and the secondary display image respectively according to the posture information;

[0013] The splicing module is used to splice the transformed main display image and the transformed secondary display image to obtain a target image.

[0014] An embodiment of the present application discloses a terminal device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements the method described above.

[0015] An embodiment of the present application discloses a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method described above is implemented.

[0016] The image processing method, apparatus, terminal device and computer-readable storage medium disclosed in the embodiments of the present application obtain a main display image and a sub-display image by segmenting a first image, obtain the posture information of the terminal device, and transform the main display image and the sub-display image respectively according to the posture information, and then splice the transformed main display image and the transformed sub-display image to obtain a target image. The first image is segmented and then spliced ​​to form a target image. The user can transform the main display image and the sub-display image by changing the posture information of the terminal device, thereby changing the target image formed by splicing, which can enrich the image processing method and improve the interactivity between the user and the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a block diagram of an image processing circuit in one embodiment;

[0019] Figure 2 is a flowchart of an image processing method in one embodiment;

[0020] Figure 3 is a schematic diagram of a main display image and a secondary display image in one embodiment;

[0021] Figure 4 is a flowchart of an image processing method in another embodiment;

[0022] Figure 5A is a schematic diagram of enlarging a sub-region in one embodiment;

[0023] Figure 5B is a schematic diagram of a main display image area and a secondary display image area in one embodiment;

[0024] Figure 6 A schematic diagram of intersecting pixel points of a secondary display image and a primary display image, and corner pixel points of the secondary display image in one embodiment;

[0025] Figure 7 A flowchart of displaying a target image in one embodiment;

[0026] Fig. 8AA schematic diagram of an interface for displaying a main display image in an embodiment;

[0027] Figure 8B A schematic diagram of an interface for displaying a target image in an embodiment;

[0028] Fig. 9 is a block diagram of an image processing device in one embodiment;

[0029] Fig.10 FIG. 4 is a structural block diagram of a terminal device in an embodiment. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0032] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, a first image may be referred to as a second image, and similarly, a second image may be referred to as a first image without departing from the scope of this application. Both the first image and the second image are images, but they are not the same image.

[0033] An embodiment of the present application provides a terminal device, wherein the terminal device includes an image processing circuit, which may be implemented using hardware and / or software components and may include various processing units that define an ISP (Image Signal Processing) pipeline. Figure 1 FIG. 1 is a block diagram of an image processing circuit in one embodiment. For ease of explanation, Figure 1 Only various aspects of the image processing technology related to the embodiments of the present application are shown.

[0034] like Figure 1As shown, the image processing circuit includes an ISP processor 140 and a control logic 150. The image data captured by the imaging device 110 is first processed by the ISP processor 140, and the ISP processor 140 analyzes the image data to capture image statistics that can be used to determine one or more control parameters of the imaging device 110. The imaging device 110 may include one or more lenses 112 and an image sensor 114. The image sensor 114 may include a color filter array (such as a Bayer filter), and the image sensor 114 may obtain light intensity and wavelength information captured by each imaging pixel, and provide a set of raw image data that can be processed by the ISP processor 140. The posture sensor 120 (such as a three-axis gyroscope, a Hall sensor, an accelerometer, etc.) can provide the collected image processing parameters (such as anti-shake parameters) to the ISP processor 140 based on the interface type of the posture sensor 120. The posture sensor 120 interface can use a SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above interfaces.

[0035] It should be noted that although Figure 1 Only one imaging device 110 is shown in the figure, but in the embodiment of the present application, at least two imaging devices 110 may be included, each imaging device 110 may correspond to one image sensor 114, or multiple imaging devices 110 may correspond to one image sensor 114, which is not limited here. The working process of each imaging device 110 may refer to the above description.

[0036] In addition, the image sensor 114 may also send raw image data to the gesture sensor 120 , and the gesture sensor 120 may provide the raw image data to the ISP processor 140 based on the interface type of the gesture sensor 120 , or the gesture sensor 120 may store the raw image data in the image memory 130 .

[0037] The ISP processor 140 processes the raw image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 140 may perform one or more image processing operations on the raw image data and collect statistical information about the image data. The image processing operations may be performed at the same or different bit depth precisions.

[0038] The ISP processor 140 may also receive image data from the image memory 130. For example, the gesture sensor 120 interface sends raw image data to the image memory 130, and the raw image data in the image memory 130 is then provided to the ISP processor 140 for processing. The image memory 130 may be a part of a memory device, a storage device, or an independent dedicated memory in an electronic device, and may include a DMA (Direct Memory Access) feature.

[0039] When receiving raw image data from the image sensor 114 interface or from the posture sensor 120 interface or from the image memory 130, the ISP processor 140 may perform one or more image processing operations, such as time domain filtering. The processed image data may be sent to the image memory 130 for further processing before being displayed. The ISP processor 140 receives the processed data from the image memory 130 and performs image data processing in the original domain and in the RGB and YCbCr color spaces on the processed data. The image data processed by the ISP processor 140 may be output to the display 160 for viewing by the user and / or further processed by a graphics engine or GPU (Graphics Processing Unit). In addition, the output of the ISP processor 140 may also be sent to the image memory 130, and the display 160 may read the image data from the image memory 130. In one embodiment, the image memory 130 may be configured to implement one or more frame buffers.

[0040] The statistical data determined by the ISP processor 140 may be sent to the control logic 150. For example, the statistical data may include image sensor 114 statistical information such as vibration frequency of a gyroscope, automatic exposure, automatic white balance, automatic focus, flicker detection, black level compensation, lens 112 shading correction, etc. The control logic 150 may include a processor and / or microcontroller that executes one or more routines (such as firmware), and the one or more routines may determine control parameters of the imaging device 110 and control parameters of the ISP processor 140 based on the received statistical data. For example, the control parameters of the imaging device 110 may include attitude sensor 120 control parameters (such as gain, integration time for exposure control, anti-shake parameters, etc.), camera flash control parameters, camera anti-shake displacement parameters, lens 112 control parameters (such as focal length for focus or zoom), or a combination of these parameters. The ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (for example, during RGB processing), and lens 112 shading correction parameters.

[0041] For example, in combination Figure 1The image processing circuit is used to illustrate the image processing method provided in the embodiment of the present application. The ISP processor 140 can obtain a first image from the imaging device 110 or the image memory 130, and segment the obtained first image to obtain a main display image and a secondary display image. The posture sensor 120 can send the collected data to the ISP processor 140, and the ISP processor 140 can obtain the posture information of the terminal device according to the data collected by the posture sensor 120, and transform the main display image and the secondary display image respectively according to the posture information. The ISP processor 140 can splice the transformed main display image and the transformed secondary display image to obtain a target image.

[0042] Optionally, after obtaining the target image, the ISP processor 140 may send the target image to the display 160 , which may display the target image, and the user may visually observe the target image through the display 160 .

[0043] like Figure 2 As shown, in one embodiment, an image processing method is provided, which can be applied to the above-mentioned terminal device, and the terminal device may include but is not limited to a mobile phone, a smart wearable device, a tablet computer, etc. The image processing method may include the following steps:

[0044] Step 210, segment the first image to obtain a main display image and a secondary display image.

[0045] In some embodiments, the first image may be an image acquired by a camera having a larger field of view (FOV), and the field of view may be used to characterize the field of view of the camera that can receive images. The larger the field of view of the camera, the larger the corresponding field of view. Optionally, the first image may be an image acquired by a camera having a field of view greater than an angle threshold, and the angle threshold may include but is not limited to 55 degrees, 60 degrees, etc.

[0046] As a specific implementation, the first image may include at least one of a wide-angle image captured by a wide-angle camera and an ultra-wide-angle image captured by an ultra-wide-angle camera. The focal length of the wide-angle camera may generally be 24-38 mm, and the field of view may generally be 60-84 degrees. The focal length of the ultra-wide-angle camera may generally be 13-20 mm, and the field of view may generally be 94-118 degrees. If it is to be noted, the wide-angle camera and the ultra-wide-angle camera may also be cameras of other models and parameters, and the first image may also be acquired by other cameras with larger field of view, which is not limited here.

[0047] The terminal device may divide the first image into a main display image and a secondary display image. The main display image may be an image of a fixed area in the first image. For example, the main display image may be an image of the central area of ​​the first image, and the secondary display image may be an image of the surrounding area of ​​the main display image in the first image. Optionally, the terminal device may divide the first image to obtain a main display image and multiple secondary display images. The main display image may be a square or rectangular image, etc., and the secondary display image may be a rectangle, a trapezoid, etc. The multiple secondary display images may respectively correspond to an image side of the main display image. If the first image has an image of the surrounding area on any image side of the main display image, the secondary display image corresponding to the image side may be divided.

[0048] Figure 3 FIG. 1 is a schematic diagram of a main display image and a secondary display image in one embodiment. Figure 3 As shown, the terminal device may segment the first image to obtain a main display image 310 and a secondary display image 302, a secondary display image 304, a secondary display image 306, and a secondary display image 308. The secondary display image 302, the secondary display image 304, the secondary display image 306, and the secondary display image 308 may correspond to the four image sides of the main display image 310, respectively, wherein the secondary display image 302 may include an image of the first image in the surrounding area of ​​the upper side of the main display image 310, the secondary display image 304 may include an image of the first image in the surrounding area of ​​the left side of the main display image 310, the secondary display image 306 may include an image of the first image in the surrounding area of ​​the lower side of the main display image 310, and the secondary display image 308 may include an image of the first image in the surrounding area of ​​the right side of the main display image 310, etc. It should be noted that Figure 3 Only one way of dividing the main display image and the auxiliary display image is shown, and the images may be divided in other ways (such as other shapes, other image areas, etc.), which are not limited here.

[0049] Step 220, obtaining the position information of the terminal device.

[0050] The posture information may include the position and / or attitude information of the terminal device. The terminal device may be provided with one or more posture sensors, such as an IMU (Inertial Measurement Unit) sensor, an acceleration sensor, etc. The posture information may be the position and / or attitude data of the terminal device at the current moment, or the change data of the position and / or attitude of the terminal device at the current moment. Data may be collected through the posture sensor, and the posture information of the terminal device may be obtained based on the data collected by the posture sensor.

[0051] As an optional implementation, the posture information acquired by the terminal device may be the position and / or posture information of the terminal device relative to an initial moment. The initial moment may be set according to actual needs. For example, the initial moment may be the moment when the terminal device captures the first image through a camera, or the moment when the terminal device initializes the display interface. Optionally, after the display interface is initialized, the terminal device may display the main display image obtained by segmentation, and use the posture information of the terminal device to adjust the displayed main display image, and finally display the target image.

[0052] The terminal device can reset the posture sensor at the initial moment, and the posture sensor can accumulate the position change and / or posture change of the terminal device from the initial moment to obtain the posture information of the terminal device in real time. For example, taking the posture sensor as an IMU sensor, the IMU can measure the angular velocity and acceleration of the terminal device in three-dimensional space from the initial moment, and then calculate the three-dimensional posture and displacement change of the terminal device. By accumulating the three-dimensional posture and displacement changes of the terminal device, the posture information of the terminal device can be obtained in real time.

[0053] Step 230, transform the main display image and the secondary display image respectively according to the position information.

[0054] In an embodiment of the present application, the transformation of the main display image and the secondary display image may be an image geometric transformation, which may include one or more of a translation transformation, a scaling transformation, a rotation transformation, and an offset transformation, but is not limited thereto. The terminal device may determine the transformation matrix corresponding to the main display image based on the posture information, and use the transformation matrix to transform the main display image to obtain the transformed main display image. Optionally, the transformation of the main display image may match the posture information of the terminal device, and the main display image may be subjected to a translation transformation and / or a scaling transformation based on the position information of the terminal device, and the main display image may be subjected to a rotation transformation and / or an offset transformation based on the posture information of the terminal device, so that the transformation of the main display image conforms to the current posture information of the terminal device.

[0055] After transforming the main display image, the terminal device can determine the transformation matrix corresponding to the secondary display image based on the transformed main display image, and then transform the secondary display image according to the transformation matrix corresponding to the secondary display image. Since the pixel points where the secondary display image intersects with the main display image will change as the main display image is transformed, the pixel points where the secondary display image intersects with the main display image can be used to determine the transformation matrix corresponding to the secondary display image, so that the pixel points where the secondary display image intersects with the main display image are still the same pixel points after transformation, thereby ensuring that the transformed secondary display image can be better spliced ​​with the main display image.

[0056] Step 240 , splicing the transformed main display image and the transformed secondary display image to obtain a target image.

[0057] In some embodiments, the terminal device may align the transformed main display image with the transformed secondary display image to obtain matching pixels in the transformed main display image and the transformed secondary display image, and fuse the matching pixels in the transformed main display image and the transformed secondary display image. Unmatched pixels may be directly spliced ​​and combined, thereby splicing the transformed main display image and the transformed secondary display image into a seamless target image.

[0058] Optionally, the terminal device may display the first image, and after splicing to obtain the target image, display the target image, so that the user can intuitively observe the changes in the first image. In some embodiments, since the first image may be an image acquired by a camera with a larger FOV, by dividing the first image into a main display image and a secondary display image, and using the posture information of the terminal device to transform and adjust the main display image and the secondary display image, and then splicing to form a new target image, the display content of the image with a large FOV can be adjusted by the posture change of the terminal device, thereby improving interactivity and enhancing the user's sense of immersion.

[0059] In an embodiment of the present application, the first image is segmented to obtain a main display image and a secondary display image, the posture information of the terminal device is obtained, and the main display image and the secondary display image are transformed respectively according to the posture information, and then the transformed main display image and the transformed secondary display image are spliced ​​to obtain a target image. The first image is segmented and then spliced ​​to form a target image. The user can transform the main display image and the secondary display image by changing the posture information of the terminal device, thereby changing the target image formed by splicing, which can enrich the image processing method and improve the interactivity between the user and the image.

[0060] like Figure 4 As shown, in one embodiment, another image processing method is provided, which can be applied to the above-mentioned terminal device, and the method may include the following steps:

[0061] Step 402: align the first image and the second image to determine an image region in the first image that matches the second image.

[0062] In one embodiment, the terminal device may be provided with at least a first camera and a second camera, wherein the field of view of the first camera may be greater than the field of view of the second camera, for example, the first camera may be a wide-angle camera or an ultra-wide-angle camera, and the second camera may be a main camera (i.e., a standard camera). The terminal device may control the first camera and the second camera to simultaneously capture images, the first image may be an image captured by the terminal device through the first camera at a first moment, the second image may be an image captured by the terminal device through the second camera at a first moment, and the first image and the second image are images captured by the first camera and the second camera at the same moment, respectively.

[0063] In some embodiments, before the terminal device captures the first image and the second image through the first camera and the second camera, it can also calibrate the first camera and the second camera through a camera calibration algorithm, correct the first camera and the second camera, and calibrate the intrinsic parameters and extrinsic parameters of the first camera and the second camera. Among them, the intrinsic parameters refer to the parameters inside the camera, which may include distortion parameters, lens optical axis offset parameters, focal length, etc., and the extrinsic parameters can be used to describe the parameters of the camera in the world coordinate system, such as the position of the camera, the rotation direction, and the positional relationship between the camera and other cameras.

[0064] The terminal device can perform distortion correction and stereo correction on the first camera according to the calibrated intrinsic and extrinsic parameters of the first camera, and perform distortion correction on the second camera according to the calibrated intrinsic and extrinsic parameters of the second camera, thereby ensuring that the normal first image and second image are obtained and improving the accuracy of subsequent alignment of the first image and the second image.

[0065] The terminal device may align the first image with the second image. The manner of aligning the first image with the second image may include but is not limited to the following manners:

[0066] Method 1: extract feature points from the first image and the second image respectively, match the feature points of the first image with the feature points of the second image, and determine the image area in the first image that matches the second image based on the matched feature point pairs in the first image and the second image. Optionally, SIFT (Scale-invariant feature transform) feature descriptors, ORB (Oriented Fast and Rotated Brief) feature descriptors, etc. can be extracted from the first image and the second image, but are not limited to these. Furthermore, algorithms such as RANSAC (Random sample consensus) can be used to screen the matched feature point pairs, filter out erroneous matching feature points, and improve the accuracy of image alignment.

[0067] Method 2: The second image can be reduced in size according to a certain reduction ratio, and the reduced second image can be used as an image search template. The image search template can be simulated to move on the first image. The sub-block covered by the image search template in the first image can be compared with the image search template, and the similarity between the image search template and the covered sub-block can be calculated. When the similarity is greater than a preset similarity threshold, the sub-block covered by the image search template in the first image can be used as the image area in the first image that matches the second image.

[0068] In one embodiment, the above-mentioned reduction ratio can be determined according to the ratio of the field of view angles between the first camera and the second camera. Furthermore, the reduction ratio can be positively correlated with the ratio of the field of view angles between the first camera and the second camera. The larger the ratio of the field of view angles between the first camera and the second camera, the greater the difference between the field of view angles of the first camera and the second camera, and a larger reduction ratio can be set, and the size change of the reduced second image relative to the original second image is larger. The smaller the ratio of the field of view angles between the first camera and the second camera, the smaller the difference between the field of view angles of the first camera and the second camera, and a smaller reduction ratio can be set, and the reduced second image is closer to the original second image.

[0069] Optionally, the second image may also be a preset reference image, or an image captured by other cameras, etc., and is not limited to images captured by a main camera of the terminal device.

[0070] Step 404: determine a main display image area and a secondary display image area in the first image according to the matched image area.

[0071] As an optional implementation, the terminal device may directly use the image area in the first image that matches the second image as the main display image area, and use other image areas in the first image except the main display image area as the secondary display image area.

[0072] As another optional implementation, since the image area in the first image that matches the second image may be an irregularly shaped image area, the terminal device may search for the inscribed quadrilateral of the matching image area, and use the vertices of the inscribed quadrilateral as the vertices of the main display image area to obtain the main display image area. The inscribed quadrilateral may include but is not limited to a square, a rectangle, etc. The image area corresponding to the inscribed quadrilateral may be used as the main display image area. By searching for the inscribed quadrilateral of the matching image area and determining the main display image area based on the inscribed quadrilateral, invalid image areas can be prevented from appearing during subsequent image segmentation, image processing efficiency can be improved, and the image stitching effect can be further guaranteed.

[0073] After determining the main display image area in the first image, the terminal device may divide the other image areas in the first image except the main display image area into a plurality of sub-areas, and determine the secondary display image area corresponding to each sub-area according to the plurality of sub-areas. The other image areas may be divided into a plurality of sub-areas corresponding to each image side according to each image side of the main display image area, so that a plurality of secondary display image areas corresponding to each image side of the main display image area may be determined.

[0074] In some embodiments, determining the secondary display image area corresponding to each sub-area based on multiple sub-areas may include: expanding the multiple sub-areas respectively according to the expansion ratio to obtain the secondary display image area corresponding to each sub-area. The expansion ratio can be set according to actual needs, such as 10%, 20%, 15%, 18%, etc., which is not limited here. Expanding the sub-area according to the expansion ratio may refer to expanding the edge where the sub-area intersects with the main display image area toward the main display image area, so that the secondary display image area obtained after the expansion has an overlapping image area with the main display image area, which can improve the stitching effect of subsequent image stitching and ensure that a seamless target image is obtained.

[0075] Figure 5A FIG. 1 is a schematic diagram of enlarging a sub-region in an embodiment. Figure 5A As shown, after determining the main display image area 510 in the first image, the terminal device can divide other image areas outside the main display image area 510 to obtain multiple sub-areas, wherein: Figure 5A Only one of the sub-areas 502 is shown in the figure, and the sub-area 502 can be enlarged according to the expansion ratio, and the edge where the sub-area 502 overlaps with the main display image area 510 can be expanded toward the main display image area 510 to obtain a secondary display image area 520 corresponding to the sub-area 502. There is an overlapping area between the secondary display image area 520 and the main display image area 510.

[0076] Figure 5B FIG. 1 is a schematic diagram of a main display image area and a secondary display image area in one embodiment. Figure 5B As shown, after determining the main display image area 510 in the first image, the terminal device can expand each divided sub-area based on the expansion ratio to obtain a secondary display image area 520, a secondary display image area 530, a secondary display image area 540 and a secondary display image area 550.

[0077] Step 406 , segmenting the first image according to the main display image area and the sub-display image area to obtain a main display image and a sub-display image.

[0078] The terminal device can divide the first image, and can use the main display image area of ​​the first image as the main display image, and use each sub-display image area in the first image as the sub-display image, so as to obtain a main display image and multiple sub-display images. Figure 5B For example, the terminal device can follow Figure 5B The main display image area 510, the sub-display image area 520, the sub-display image area 530, the sub-display image area 540 and the sub-display image area 550 are divided, then the main display image area 510 can be used as the main display image, and the sub-display image area 520, the sub-display image area 530, the sub-display image area 540 and the sub-display image area 550 can be used as sub-display images respectively.

[0079] In other embodiments, a fixed resolution image may be directly selected from the first image as the main display image, and the terminal device may select an image area of ​​fixed size and image position from the first image as the main display image. The embodiment of the present application does not limit the method of dividing the first image to obtain the main display image and the secondary display image.

[0080] Step 408, obtaining the position information of the terminal device.

[0081] In some embodiments, the terminal device can measure the angular velocity and acceleration at the current moment through the posture sensor. The angular acceleration refers to the angle change speed when the terminal device changes its posture, and the acceleration may include translation acceleration, that is, the acceleration when the terminal device translates. Based on the sensor model of the posture sensor, the translation amount of the terminal device can be calculated using the measured acceleration, and a translation matrix can be constructed based on the translation amount; the quaternion is calculated using the measured angular velocity, and a rotation matrix is ​​constructed based on the quaternion. The quaternion can be used to describe the orientation (i.e., posture) of the terminal device in three-dimensional space. The constructed translation matrix and rotation matrix can be used as the posture information of the terminal device.

[0082] As a specific implementation, the translation amount of the terminal device can be calculated according to formula (1) and formula (2):

[0083]

[0084] v j{x，y，z} =v i{x，y，z} +a {x，y，z} Δt Formula (2);

[0085] in, It can be used to represent the displacement of the terminal device in the coordinate system of the posture sensor at time i; It can be used to represent the displacement of the terminal device in the coordinate system of the posture sensor at time j; a {x，y，z}It can be used to represent the acceleration of the terminal device in the coordinate system of the posture sensor; Δt can be used to represent the time difference from time i to time j; v i{x，y，z} It can be used to represent the displacement speed of the terminal device in the coordinate system of the posture sensor at time i. j{x，y，z} It can be used to represent the displacement speed of the terminal device in the coordinate system of the posture sensor at time j.

[0086] Formula (1) can be used to calculate the translation of the terminal device on the X-axis, Y-axis and Z-axis in the coordinate system of the posture sensor at time j, where the translation on the X-axis can be The translation amount on the Y axis can be The translation in the Z axis can be It should be noted that the above-mentioned j moment may refer to the current moment, and the i moment may refer to the previous moment of the current moment. The previous moment may be determined by the frame rate of the posture sensor, and the time difference Δt from the i moment to the j moment may be 1 divided by the frame rate.

[0087] After calculating the translation of the terminal device on the X-axis, Y-axis, and Z-axis in the coordinate system of the posture sensor, the translation matrix T(p) can be constructed based on the translation:

[0088]

[0089] in, is the X-axis translation of the terminal device in the coordinate system of the posture sensor at time i, is the Y-axis translation of the terminal device in the coordinate system of the posture sensor at time i, is the Z-axis translation of the terminal device in the coordinate system of the posture sensor at time i.

[0090] As a specific implementation, the quaternion can be calculated according to formula (3):

[0091]

[0092] in, represents the quaternion calculated at time i, represents the quaternion calculated at time j, ω represents the angular velocity, and δ represents the Euler angle. In addition, the quaternion q=w+xk1+yk2+zk3, where k1, k2, k3 are imaginary parts, w is a constant parameter, and x, y, and z are the coordinates of the X-axis, Y-axis, and Z-axis of the terminal device in the coordinate system of the posture sensor.

[0093] The rotation matrix R(p) can be obtained based on the calculated quaternion:

[0094]

[0095] Step 410: transform the main display image according to the posture information.

[0096] The terminal device can transform the main display image using the calculated translation matrix and rotation matrix. The main display image can be first transformed by the translation matrix, and then the main display image can be rotated by the rotation matrix. In one embodiment, the transformation matrix P′=RP+RT corresponding to the main display image can be determined according to the calculated translation matrix and rotation matrix, where R is the rotation matrix, T is the translation matrix, and the matrix P is the homogeneous coordinate matrix of the main display image. The matrix P is translated and rotated to obtain the transformation matrix P′, and the transformation matrix P′ is used to transform the main display image to obtain the transformed main display image.

[0097] Step 412: determine a transformation matrix corresponding to the secondary display image based on the transformed primary display image, and transform the secondary display image according to the transformation matrix.

[0098] Since the secondary display image includes the image content of the first image in the surrounding image area of ​​the primary display image, there are intersecting pixels between the secondary display image and the primary display image, and the intersecting pixels can be understood as pixels that exist on the image sides of both the secondary display image and the primary display image. The intersecting pixels of the secondary display image and the primary display image will change as the primary display image changes, and the corner pixels on the outer boundary of the secondary display image will not change, and the corner pixels on the outer boundary of the secondary display image can be understood as corner pixels on the side that does not intersect with the primary display image.

[0099] Figure 6 FIG. 1 is a schematic diagram of the intersection pixel points of the secondary display image and the primary display image, and the corner pixel points of the secondary display image in one embodiment. Figure 6 As shown, p1 and p2 are the intersection pixels of the secondary display image 520 and the primary display image 510, p3 and p4 are the corner pixels on the outer boundary of the secondary display image, p1 and p2 will change with the change of the primary display image 510, and p3 and p4 will remain unchanged.

[0100] Therefore, the terminal device can determine the intersecting pixel points of the secondary display image and the main display image, the corresponding transformed pixel points in the transformed main display image, and calculate the transformation matrix corresponding to the secondary display image based on the corner pixel points of the secondary display image, the intersecting pixel points and the corresponding transformed pixel points.

[0101] Specifically, the terminal device may first determine the intersecting pixel points of the secondary display image and the main display image based on the coordinates of the pixel points on each image side of the secondary display image in the first image, and the coordinates of the pixel points on each image side of the main display image in the first image, and obtain the pixel points that exist on both the image side of the secondary display image and the image side of the main display image and have the same coordinates, namely, the intersecting pixel points. Then, based on the transformation matrix corresponding to the main display image (namely, the above-mentioned transformation matrix P′), the transformed pixel points corresponding to the intersecting pixel points of the secondary display image and the main display image are calculated, and the transformed pixel points are the transformed pixel points corresponding to the intersecting pixel points in the transformed main display image. The homography transformation matrix corresponding to the secondary display image can be calculated using the transformed pixel points corresponding to the intersecting pixel points and the corner pixel points that are not transformed in the secondary display image.

[0102] by Figure 6 For example, the terminal device can use the transformation matrix corresponding to the main display image 510 to calculate the transformed pixel points p1' and p2' corresponding to p1 and p2 of the secondary display image 520. If p3 and p4 remain unchanged, the pixel values ​​of (p1, p1'), (p2, p2'), p3 and p4 can be used to calculate the homography transformation matrix corresponding to the secondary display image 520.

[0103] After the transformation matrix corresponding to the secondary display image is calculated, each pixel in the secondary display image may be transformed according to the transformation matrix corresponding to the secondary display image to obtain the transformed secondary display image. When there are multiple secondary display images, the transformation matrix corresponding to each secondary display image may be calculated respectively in the manner described in the above embodiment, and each secondary display image may be transformed respectively according to the transformation matrix corresponding to each secondary display image, thereby obtaining multiple transformed secondary display images.

[0104] Step 414, splicing the transformed main display image and the transformed secondary display image to obtain a target image.

[0105] The terminal device may stitch the transformed main display image and the transformed multiple secondary display images to obtain a target image. For each transformed secondary display image, the transformed main display image may be aligned with the transformed secondary display image, and the overlapping area between the transformed main display image and the transformed secondary display image may be calculated. It should be noted that the method for aligning the transformed main display image with the transformed secondary display image may adopt the method for aligning the first image with the second image in the above embodiment, and the method for aligning the transformed main display image with the transformed secondary display image will not be repeated here.

[0106] In some embodiments, since the divided sub-regions are enlarged according to the expansion ratio when determining the secondary display image region corresponding to the secondary display image, there is an overlapping region between the secondary display image obtained by segmentation and the main display image. After the secondary display image is transformed, since the secondary display image and the main display image correspond to different transformation matrices, the transformed main display image and the transformed secondary display image need to be re-aligned to obtain a matching image region between the transformed main display image and the transformed secondary display image, and the matching image region is the overlapping region.

[0107] In some embodiments, after the sub-area is enlarged in the above method to obtain the secondary display image area corresponding to the sub-area, the four corner pixel points of the overlapping area between the secondary display image area and the main display image area can be recorded, and the four corner pixel points of the overlapping area can be transformed using the transformation matrix of the main display image to obtain four transformed pixel points of the four corner pixel points in the transformed main display image, thereby determining the overlapping area between the transformed main display image and the transformed secondary display image based on the four transformed pixel points.

[0108] The terminal device may fuse the overlapping area between the transformed main display image and the transformed secondary display image to obtain a target image. The method of fusing the overlapping area between the transformed main display image and each transformed secondary display image may include but is not limited to average pixel value fusion, weighted average pixel value fusion, fusion based on nonlinear method, color space fusion, etc., which are not limited here. The non-overlapping area between the transformed main display image and each transformed secondary display image can be directly spliced ​​to form a seamless target image.

[0109] In an embodiment of the present application, the main display image is transformed using the posture information of the terminal device, so that the main display image is transformed following the changes in the posture information, maintaining coordinated transformation, and further improving the interactive effect. The secondary display image is transformed based on the transformed main display image to ensure the image effect of the secondary display image after the transformation, thereby improving the accuracy of image stitching, enriching the image processing method, and improving the interactivity between the user and the image, while also improving the image visual effect of the target image.

[0110] like Figure 7 As shown, in one embodiment, the above image processing method may further include the following steps:

[0111] Step 702: display the target image.

[0112] The terminal device can initialize the display interface, and after segmenting to obtain the main display image, display the main display image. Optionally, after initializing the display interface, the image captured by the second camera can also be directly displayed. The terminal device uses the posture information to transform the main display image and the secondary display image, and can display the transformed main display image on the display interface. After splicing the transformed main display image with the secondary display image to obtain the target image, the target image can be displayed, and the user can directly observe the changes in the main display image, thereby improving interactivity.

[0113] Fig. 8A FIG. 1 is a schematic diagram of an interface for displaying a main display image in an embodiment. Fig. 8A As shown, the display interface 810 displays the main display image obtained by segmentation. After the terminal device transforms the main display image using the posture information, the transformed main display image can be displayed in the display interface 820.

[0114] Figure 8B FIG. 1 is a schematic diagram of an interface for displaying a target image in an embodiment. Figure 8B As shown in the figure, after the main display image and the secondary display image are transformed, the transformed main display image and the transformed secondary display image can be spliced ​​to obtain the target image. Since the FOV of the first image is larger, the FOV of the spliced ​​target image is also larger. Figure 8B It can also be seen intuitively that the target image can display image content with a larger FOV than the image captured by the main camera (i.e., the main display image), which can enhance the user's sense of immersion.

[0115] Step 704, when it is detected that the posture information of the terminal device has changed, the posture change information of the terminal device is obtained.

[0116] Step 706, re-transform the main display image and the secondary display image according to the posture change information, and splice the re-transformed main display image and the transformed secondary display image to obtain an updated target image, and then display the updated target image.

[0117] When the terminal device detects a change in posture information, it can obtain the posture change information of the terminal device. The posture change information can be used to indicate the changed position and posture information of the terminal device, and the target image can be updated and displayed according to the posture change information. The main display image and the secondary display image can be transformed again according to the posture change information, and the transformation of the main display image can be consistent with the posture change information. For example, when the terminal device moves to the left, the main display image can be translated to the left, and when the terminal device rotates to the right, the main display image can be rotated to the right to change the image perspective, so that the main display image can simulate the movement of the terminal device to switch the perspective. Optionally, the transformation rules of the main display image can also be set. For example, when the terminal device moves in the direction toward the user, the main display image can be enlarged and transformed, and when the terminal device moves in the direction away from the user, the main display image can be reduced and transformed, etc., which are not limited here.

[0118] The terminal device can splice the re-transformed main display image and the transformed secondary display image to obtain an updated target image, and then display the updated target image, thereby achieving the effect of updating and displaying the target image based on the posture information in real time. By changing the posture of the terminal device, the user can intuitively see different target images in the display interface, thereby improving the sense of interaction between the user and the image.

[0119] In an embodiment of the present application, the terminal device can display the target image through a display interface, and can update and display the target image in real time according to the posture of the terminal device. By changing the posture of the terminal device, the user can intuitively see different target images in the display interface, thereby improving the sense of interaction between the user and the image.

[0120] like Fig. 9 As shown, in one embodiment, an image processing device 900 is provided, which can be applied to the above-mentioned terminal device. The image processing device 900 may include an image segmentation module 910, a posture acquisition module 920, a transformation module 930 and a stitching module 940.

[0121] The image segmentation module 910 is used to segment the first image to obtain a main display image and a secondary display image.

[0122] In one embodiment, the first image includes at least one of a wide-angle image captured by a wide-angle camera and an ultra-wide-angle image captured by an ultra-wide-angle camera.

[0123] The posture acquisition module 920 is used to obtain the posture information of the terminal device.

[0124] The transformation module 930 is used to transform the main display image and the secondary display image respectively according to the posture information.

[0125] The splicing module 940 is used to splice the transformed main display image and the transformed secondary display image to obtain a target image.

[0126] In an embodiment of the present application, the first image is segmented to obtain a main display image and a secondary display image, the posture information of the terminal device is obtained, and the main display image and the secondary display image are transformed respectively according to the posture information, and then the transformed main display image and the transformed secondary display image are spliced ​​to obtain a target image. The first image is segmented and then spliced ​​to form a target image. The user can transform the main display image and the secondary display image by changing the posture information of the terminal device, thereby changing the target image formed by splicing, which can enrich the image processing method and improve the interactivity between the user and the image.

[0127] In one embodiment, the image segmentation module 910 includes an alignment unit, a region determination unit, and a segmentation unit.

[0128] An alignment unit is used to align a first image with a second image and determine an image area in the first image that matches the second image, wherein the first image is an image captured by the terminal device through a first camera at a first moment, and the second image is an image captured by the terminal device through a second camera at the first moment, and the field of view of the first camera is greater than the field of view of the second camera.

[0129] The area determination unit is used to determine the main display image area and the sub-display image area in the first image according to the matched image area.

[0130] In one embodiment, the area determination unit is further used to find an inscribed quadrilateral of the matching image area, and use the vertices of the inscribed quadrilateral as the vertices of the main display image area to obtain the main display image area, and then divide the other image areas in the first image except the main display image area into multiple sub-areas, and determine the secondary display image area corresponding to each sub-area based on the multiple sub-areas.

[0131] In one embodiment, the region determination unit is further configured to respectively expand the plurality of sub-regions according to the expansion ratio to obtain a secondary display image region corresponding to each sub-region.

[0132] The segmentation unit is used to segment the first image according to the main display image area and the sub-display image area to obtain the main display image and the sub-display image.

[0133] In one embodiment, the transform module 930 includes a first transform unit and a second transform unit.

[0134] The first transformation unit is used to transform the main display image according to the posture information.

[0135] The second transformation unit is used to determine a transformation matrix corresponding to the secondary display image based on the transformed main display image, and transform the secondary display image according to the transformation matrix.

[0136] In one embodiment, the second transformation unit is further used to determine the intersecting pixel points in the secondary display image and the main display image, and the corresponding transformation pixel points in the transformed main display image, and calculate the transformation matrix corresponding to the secondary display image according to the corner pixel points of the secondary display image, the intersecting pixel points and the corresponding transformation pixel points.

[0137] In one embodiment, the stitching module 940 is further used to align the transformed main display image with the transformed secondary display image, calculate the overlapping area between the transformed main display image and the transformed secondary display image, and merge the overlapping area between the transformed main display image and the transformed secondary display image to obtain a target image.

[0138] In an embodiment of the present application, the main display image is transformed using the posture information of the terminal device, so that the main display image is transformed following the changes in the posture information, maintaining coordinated transformation, and further improving the interactive effect. The secondary display image is transformed based on the transformed main display image to ensure the image effect of the secondary display image after the transformation, thereby improving the accuracy of image stitching, enriching the image processing method, and improving the interactivity between the user and the image, while also improving the image visual effect of the target image.

[0139] In one embodiment, the image processing device 900 includes not only the image segmentation module 910 , the posture acquisition module 920 , the transformation module 930 , and the stitching module 940 , but also a display module.

[0140] The display module is used to display the target image.

[0141] The posture acquisition module 920 is also used to obtain posture change information of the terminal device when it is detected that the posture information of the terminal device has changed.

[0142] The transformation module 930 is further used to re-transform the main display image and the secondary display image according to the posture change information.

[0143] The splicing module 940 is further used to splice the re-transformed main display image and the transformed secondary display image to obtain an updated target image.

[0144] The display module is also used to display the updated target image.

[0145] In an embodiment of the present application, the terminal device can display the target image through a display interface, and can update and display the target image in real time according to the posture of the terminal device. By changing the posture of the terminal device, the user can intuitively see different target images in the display interface, thereby improving the sense of interaction between the user and the image.

[0146] Fig.10 FIG. 1 is a block diagram of a terminal device in an embodiment. Fig.10 As shown, the terminal device 1000 may include one or more of the following components: a processor 1010, and a memory 1020 coupled to the processor 1010, wherein the memory 1020 may store one or more computer programs, and the one or more computer programs may be configured to implement the methods described in the above embodiments when executed by one or more processors 1010.

[0147] The processor 1010 may include one or more processing cores. The processor 1010 uses various interfaces and lines to connect various parts of the entire terminal device 1000, and executes various functions and processes data of the terminal device 1000 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1020, and calling data stored in the memory 1020. Optionally, the processor 1010 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1010 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 1010, but may be implemented separately through a communication chip.

[0148] The memory 1020 may include a random access memory (RAM) or a read-only memory (ROM). The memory 1020 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 1020 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may also store data created by the terminal device 1000 during use, etc.

[0149] It is understandable that the terminal device 1000 may include more or fewer structural elements than those in the above structural block diagram, for example, a power module, physical buttons, a WiFi (Wireless Fidelity) module, a speaker, a Bluetooth module, a sensor, etc., and no limitation is made here.

[0150] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein the computer program implements the method described in the above embodiment when executed by a processor.

[0151] An embodiment of the present application discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program can be executed by a processor to implement the methods described in the above embodiments.

[0152] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a ROM, etc.

[0153] As used herein, any reference to memory, storage, database, or other medium may include nonvolatile and / or volatile memory. Suitable nonvolatile memory may include ROM, programmable ROM (Programmable ROM, PROM), erasable PROM (Erasable PROM, EPROM), electrically erasable PROM (Electrically Erasable PROM, EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which is used as an external cache memory. By way of illustration and not limitation, RAM may be in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus direct RAM (RDRAM) and direct memory bus dynamic RAM (DRDRAM).

[0154] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required for the present application.

[0155] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the above-mentioned processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0156] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0157] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0158] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the above-mentioned methods of various embodiments of the present application.

[0159] The above is a detailed introduction to an image processing method, device, terminal device and computer-readable storage medium disclosed in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. At the same time, for those of ordinary skill in the art, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. An image processing method, characterized in that: Applied to a terminal device, the method comprises: Segmenting the first image to obtain a main display image and a secondary display image; the first image includes at least one of a wide-angle image captured by a wide-angle camera and an ultra-wide-angle image captured by an ultra-wide-angle camera; Obtaining position information of the terminal device; transforming the main display image and the secondary display image respectively according to the posture information; The transformed main display image and the transformed secondary display image are spliced ​​to obtain a target image.

2. The method according to claim 1, characterized in that The step of segmenting the first image to obtain a main display image and a secondary display image includes: Aligning a first image with a second image, and determining an image area in the first image that matches the second image, wherein the first image is an image acquired by the terminal device through a first camera at a first moment, and the second image is an image acquired by the terminal device through a second camera at the first moment, and the field of view of the first camera is greater than the field of view of the second camera; Determine a main display image area and a secondary display image area in the first image according to the matched image area; The first image is segmented according to the main display image area and the sub-display image area to obtain a main display image and a sub-display image.

3. The method according to claim 2, characterized in that The determining the main display image area and the secondary display image area in the first image according to the matched image area includes: Finding an inscribed quadrilateral of the matched image area, and using the vertices of the inscribed quadrilateral as vertices of the main display image area to obtain the main display image area; In the first image, other image regions except the main display image region are divided into a plurality of sub-regions, and a secondary display image region corresponding to each sub-region is determined according to the plurality of sub-regions.

4. The method according to claim 3, characterized in that The determining, according to the plurality of sub-regions, a secondary display image region corresponding to each sub-region comprises: The plurality of sub-regions are enlarged respectively according to the expansion ratio to obtain a secondary display image region corresponding to each sub-region.

5. The method according to any one of claims 1 to 4, characterized in that: The transforming the main display image and the secondary display image respectively according to the posture information includes: transforming the main display image according to the posture information; A transformation matrix corresponding to the secondary display image is determined based on the transformed primary display image, and the secondary display image is transformed according to the transformation matrix.

6. The method according to claim 5, characterized in that The determining the transformation matrix corresponding to the secondary display image based on the transformed primary display image includes: Determine the intersecting pixel points of the secondary display image and the primary display image, and the corresponding transformed pixel points in the transformed primary display image; A transformation matrix corresponding to the secondary display image is calculated according to the corner pixel points of the secondary display image, the intersecting pixel points and the corresponding transformation pixel points.

7. The method according to any one of claims 1 to 4, characterized in that: The step of splicing the transformed main display image and the transformed secondary display image to obtain a target image includes: Aligning the transformed main display image with the transformed secondary display image, and calculating an overlapping area between the transformed main display image and the transformed secondary display image; The overlapping area between the transformed main display image and the transformed secondary display image is fused and spliced ​​to obtain a target image.

8. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Displaying the target image; When it is detected that the posture information of the terminal device changes, obtaining the posture change information of the terminal device; The main display image and the secondary display image are transformed again according to the posture change information, and the re-transformed main display image and the transformed secondary display image are spliced ​​to obtain an updated target image, and then the updated target image is displayed.

9. An image processing device, characterized in that: Applied to a terminal device, the device comprises: An image segmentation module, configured to segment the first image to obtain a main display image and a secondary display image; the first image comprises at least one of a wide-angle image acquired by a wide-angle camera and an ultra-wide-angle image acquired by an ultra-wide-angle camera; A posture acquisition module, used to obtain the posture information of the terminal device; A transformation module, used to transform the main display image and the secondary display image respectively according to the posture information; The splicing module is used to splice the transformed main display image and the transformed secondary display image to obtain a target image.

10. A terminal device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images