Image processing method and device and terminal equipment

Abstract

The invention is suitable for the technical field of image processing, and provides an image processing method and device and terminal equipment, the image processing method is applied to the terminal equipment, a display screen is arranged on the terminal equipment, panoramic data are stored in the terminal equipment, and the panoramic data are data including a panoramic spherical image. The image processing method comprises the steps that if a sliding operation is detected based on the display screen, a target image area corresponding to panoramic data is adjusted according to a sliding track corresponding to the sliding operation, and the sliding track is the sliding track of a contact object on the display screen; the contact object is an object which can be in contact with the display screen and can slide on the display screen; and controlling the display screen to display an image picture corresponding to the adjusted target image area. That is, the terminal device can efficiently control itself to display an image desired to be seen by a user.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus and terminal device. Background Art

[0002] Currently, panoramic data is increasingly used in various fields, such as entertainment, tourism, and medical treatment. However, there are still some defects in the use of panoramic data, where panoramic data is data including panoramic spherical images.

[0003] For example, when a terminal device is displaying content corresponding to a panoramic spherical image, although the panoramic spherical image is an image captured based on a 360° shooting angle of view, a general display screen can only present a flat image with a small viewing angle range. When the image displayed on the display screen is not the image the user expects to see, the terminal device cannot efficiently control itself to display the image the user expects to see. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, apparatus and terminal device, which can solve the problem that the current terminal device cannot be efficiently adjusted to display the image screen that the user expects to see.

[0005] In a first aspect, the present application provides an image processing method, which is applied to a terminal device, wherein a display screen is provided on the terminal device, and panoramic data is also stored in the terminal device, wherein the panoramic data is data including a panoramic spherical image, and the image processing method comprises: if a sliding operation is detected based on the display screen, adjusting a target image area corresponding to the panoramic data according to a sliding trajectory corresponding to the sliding operation, wherein the sliding trajectory is a trajectory of a contact object sliding on the display screen, and the contact object is an object that can contact the display screen and can slide on the display screen; and controlling the display screen to display an image screen corresponding to the adjusted target image area.

[0006] In a possible implementation manner of the first aspect, adjusting the target image area corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation includes: adjusting the position of the target image area corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation.

[0007] In a possible implementation manner of the first aspect, the trajectory points on the sliding trajectory include a sliding trajectory starting point and a trajectory point behind the starting point, the trajectory point behind the starting point being: a trajectory point detected after the sliding trajectory starting point is detected, and correspondingly, adjusting the position of the target image area corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation includes: establishing a plane rectangular coordinate system corresponding to the starting point of the sliding trajectory, the origin of the plane rectangular coordinate system corresponding to the starting point of the sliding trajectory is the starting point of the sliding trajectory; determining a position vector corresponding to the trajectory point behind the starting point based on the plane rectangular coordinate system corresponding to the starting point of the sliding trajectory, the position vector corresponding to the trajectory point behind the starting point being: a vector formed from the starting point of the sliding trajectory to the trajectory point behind the starting point; and adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point behind the starting point.

[0008] In a possible implementation manner of the first aspect, adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the starting point includes: calculating Φ(i+1) according to the position vector corresponding to the trajectory point after the starting point and a specified relationship, wherein the specified relationship is: Φ(i+1)=Φ(i)+λΔtQ, i is equal to 0 or i is a positive integer greater than 0; when i is equal to 0, Φ(i) is used to represent the initial position of the target image area; when i is a positive integer greater than 0, Φ(i) is used to represent the position of the target image area after the i-th adjustment, λ is less than or greater than 1, and Δt is a first specified time length less than 1; Q is used to represent the position vector corresponding to the trajectory point on the sliding trajectory; and adjusting the position of the target image area corresponding to the panoramic data according to Φ(i+1).

[0009] In a possible implementation of the first aspect, after adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the starting point, step A is included, and step A includes: if a k-th action breakpoint is detected, establishing a plane rectangular coordinate system corresponding to the k-th action breakpoint, wherein k is a positive integer greater than or equal to one, and the k-th action breakpoint is: a trajectory point generated because the contact time between the contact object and the display screen is greater than or equal to a second specified time length; based on the plane rectangular coordinate system corresponding to the k-th action breakpoint, determining the position vector corresponding to the trajectory point after the k-th breakpoint, the position vector corresponding to the trajectory point after the k-th breakpoint is: a vector formed by the k-th action breakpoint to the trajectory point after the k-th breakpoint, and the trajectory point after the k-th breakpoint is: a trajectory point detected after the k-th action breakpoint is detected; adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the k-th breakpoint.

[0010] In a possible implementation of the first aspect, the initial value of k is equal to a specified positive integer, and the specified positive integer is a positive integer greater than or equal to one, and the step A includes: step A1, if a k-th action breakpoint is detected, then establishing a plane rectangular coordinate system corresponding to the k-th action breakpoint; step A2, based on the plane rectangular coordinate system corresponding to the k-th action breakpoint, determining the position vector corresponding to the trajectory point after the k-th breakpoint; step A3, adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the k-th breakpoint; step A4, if a k+1-th action breakpoint is detected, then setting k=k+1, and then returning to execute step A1.

[0011] In a possible implementation of the first aspect, before adjusting the position of the target image area corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation, it includes: setting a virtual camera at the center of the sphere corresponding to the panoramic data, and the position of the target image area is located in the area where the field of view of the virtual camera intersects with the panoramic data; correspondingly, adjusting the position of the target image area corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation includes: rotating the virtual camera and / or the panoramic spherical image in the panoramic data according to the sliding trajectory corresponding to the sliding operation.

[0012] In a second aspect, an embodiment of the present application provides an image processing device, which is applied to a terminal device, wherein a display screen is provided on the terminal device, and panoramic data is also stored in the terminal device, wherein the panoramic data is data including a panoramic spherical image, and the image processing device includes: an adjustment unit, which is used to adjust a target image area corresponding to the panoramic data according to a sliding trajectory corresponding to the sliding operation if a sliding operation is detected based on the display screen, wherein the sliding trajectory is a trajectory of a contact object sliding on the display screen, and the contact object is an object that can contact the display screen and can slide on the display screen; and a display control unit, which is used to control the display screen to display an image screen corresponding to the adjusted target image area.

[0013] In a third aspect, an embodiment of the present application provides a terminal device, including a display, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods described in the first aspect when executing the computer program.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the first aspects above is implemented.

[0015] In an embodiment of the present application, a display screen is provided on the terminal device, and panoramic data is also stored in the terminal device. If the terminal device detects a sliding operation based on the display screen, the target image area corresponding to the panoramic data is adjusted according to the sliding trajectory corresponding to the sliding operation, and the sliding trajectory is the trajectory of the contact object sliding on the display screen, and the contact object is: an object that can contact the display screen and can slide on the display screen; the display screen is controlled to display the image screen corresponding to the adjusted target image area. Since the contact object is generally controlled by the user, the sliding trajectory can reflect the user's inner expectations to a certain extent. The terminal device adjusts the target image area corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation, and then controls the display screen to display the image screen corresponding to the adjusted target image area, that is, the terminal device can efficiently control itself (the terminal device) to display the image screen that the user expects to see. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 is a flowchart of an image processing method provided by an embodiment of the present application;

[0018] Figure 2 is a schematic diagram of a sliding track provided by an embodiment of the present application;

[0019] Figure 3 is a schematic diagram of an image processing device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0021] It should be noted that the terms "comprises" and "has" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product or equipment comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or equipment.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0023] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it may be directly on the other element or there may be an intermediate element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be an intermediate element at the same time.

[0024] In this article, the term "and / or" is only a description of the association relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects before and after are in an "or" relationship.

[0025] Embodiment 1

[0026] Figure 1 A flow chart of an image processing method provided by an embodiment of the present application is shown, wherein the image processing method is applied to a terminal device, wherein a display screen is provided on the terminal device, and panoramic data is also stored in the terminal device. As an example but not a limitation, the terminal device is a panoramic camera or a mobile phone. The image processing method includes step S101 and step S102. The details are as follows:

[0027] Step S101: If a sliding operation is detected based on the display screen, the target image area corresponding to the panoramic data is adjusted according to a sliding trajectory corresponding to the sliding operation, wherein the sliding trajectory is a trajectory of a contact object sliding on the display screen, and the contact object is an object that can contact the display screen and slide on the display screen.

[0028] The panoramic data is data including panoramic spherical images, that is, the panoramic data includes one or at least two panoramic spherical images, and the panoramic spherical images are the result of mapping an image captured by a specified fisheye lens onto a specified spherical model.

[0029] As an example but not limitation, the sliding trajectory may be the trajectory of the contact object sliding at any position of the display screen, or the sliding trajectory may be the trajectory of the contact object sliding within a specified area of ​​the display screen, and the specified area may be a circular area. The contact object may be: a user's finger, a stylus tip or other hard object (such as a pen, a key, etc.). In addition, a virtual camera is provided at the center of the sphere corresponding to the panoramic data, that is, a virtual camera is provided at the center of the sphere of the specified spherical model, and the virtual camera is used to determine the target image area, that is, to determine the image picture (content) to be displayed on the display screen. Specifically, the field of view angle of the virtual camera forms a field of view, which intersects with the panoramic data (panoramic spherical image), and the location of the intersecting area is the target image area.

[0030] Step S102: Control the display screen to display the image corresponding to the adjusted target image area.

[0031] Specifically, the terminal device expands the adjusted target image area based on the projection method to obtain a (plane) image picture, and controls the display screen to display the (plane) image picture.

[0032] As an example but not limitation, the projection method may be an equirectangular projection method.

[0033] As an example but not a limitation, each time the terminal device adjusts the target image area, the terminal device controls the display screen to display the image corresponding to the adjusted target image area. For example, after the terminal device adjusts the target image area for the first time, the terminal device controls the display screen to display the image corresponding to the target image area after the first adjustment; after the terminal device adjusts the target image area for the second time, the terminal device controls the display screen to display the image corresponding to the target image area after the second adjustment.

[0034] Optionally, adjusting the target image region corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation includes: adjusting the area of ​​the target image region corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation.

[0035] Specifically, the area of ​​the target image region corresponding to the panoramic data is reduced or enlarged according to the sliding track corresponding to the sliding operation.

[0036] As an example but not limitation, when the length of the sliding track corresponding to the sliding operation is greater than or equal to the specified length, the area of ​​the target image area corresponding to the panoramic data is enlarged; when the sliding track corresponding to the sliding operation is less than the specified length, the area of ​​the target image area corresponding to the panoramic data is reduced.

[0037] In this way, the user can quickly adjust the area of ​​the target image region corresponding to the panoramic data through the sliding track.

[0038] Optionally, adjusting the target image area corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation includes: adjusting a position of the target image area corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation.

[0039] As an example but not limitation, the area of ​​the target image region before adjustment (position) and the area of ​​the target image region after adjustment (position) may be the same, that is, the area of ​​the target image region may remain unchanged before / after adjustment.

[0040] In this way, the user can quickly adjust the position of the target image area corresponding to the panoramic data through the sliding track. Optionally, before adjusting the position of the target image area corresponding to the panoramic data according to the sliding track corresponding to the sliding operation, it includes: setting a virtual camera at the center of the sphere corresponding to the panoramic data, and the position of the target image area is where the field of view of the virtual camera intersects with the panoramic data; correspondingly, adjusting the position of the target image area corresponding to the panoramic data according to the sliding track corresponding to the sliding operation includes: rotating the virtual camera and / or the panoramic spherical image in the panoramic data according to the sliding track corresponding to the sliding operation.

[0041] Wherein, rotating the virtual camera can change the direction of the virtual camera.

[0042] The terminal device can rotate the virtual camera and / or the panoramic spherical image in the panoramic data according to the sliding trajectory (to achieve relative rotation) to adjust the position of the target image area, so that the image displayed on the display screen will also change. In this way, the user can perceive the image corresponding to different perspectives in the panoramic data.

[0043] Optionally, the trajectory points on the sliding trajectory include a starting point of the sliding trajectory and a trajectory point after the starting point, the trajectory point after the starting point being: a trajectory point detected after the starting point of the sliding trajectory is detected, and correspondingly, adjusting the position of the target image area corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation includes: establishing a plane rectangular coordinate system corresponding to the starting point of the sliding trajectory, the origin of the plane rectangular coordinate system corresponding to the starting point of the sliding trajectory is the starting point of the sliding trajectory; determining a position vector corresponding to the trajectory point after the starting point based on the plane rectangular coordinate system corresponding to the starting point of the sliding trajectory, the position vector corresponding to the trajectory point after the starting point being: a vector formed from the starting point of the sliding trajectory to the trajectory point after the starting point; and adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the starting point.

[0044] Since the panoramic spherical image in the panoramic data belongs to a spherical image, and the image screen seen by the user through the terminal device is a plane image, when the user controls the position of the target image area through the sliding track, the user's perception of the position of the area of ​​interest in the brain and the position of the target image area that the terminal device actually needs to adjust are likely to be different. If the terminal device uses a plane rectangular coordinate system with a fixed origin during the adjustment of the position of the target image area through the sliding track, the point on the sliding track may be far away from the fixed origin, resulting in the adjusted position seriously deviating from the user's area of ​​interest. In this embodiment, the terminal device can establish a plane rectangular coordinate system with the starting point of the sliding track as the origin, and then determine the position vector corresponding to the trajectory point after the starting point based on the plane rectangular coordinate system corresponding to the starting point of the sliding track, and then adjust the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the starting point. Since the distance between the trajectory point after the starting point and the starting point of the sliding track in a sliding track is relatively close, the position of the target image area corresponding to the panoramic data is adjusted according to the position vector corresponding to the trajectory point after the starting point, which can greatly reduce or even eliminate the difference between the position of the adjusted target image area and the user's area of ​​interest, so that the terminal device can more accurately display the image screen that the user expects to see.

[0045] Specifically, the establishment of the plane rectangular coordinate system corresponding to the starting point of the sliding track includes: taking the starting point of the sliding track as the origin, taking the first specified direction as the direction of the horizontal axis, taking the second specified direction as the direction of the vertical axis, and the first specified direction and the second specified direction are perpendicular to each other, thereby establishing the plane rectangular coordinate system corresponding to the starting point of the sliding track. The position vector corresponding to the trajectory point after the starting point takes the starting point of the sliding track as the starting point and takes the trajectory point after the starting point as the end point.

[0046] For example, Figure 2 As shown, assuming that the curve OP 01 belongs to the sliding trajectory, where point O is the starting point of the sliding trajectory, P 01 Point is the trajectory point detected after point O is detected, P 01 Point belongs to the trajectory point after the starting point. Therefore, after detecting the starting point of the sliding trajectory (point O), take point O as the origin and establish the plane rectangular coordinate system corresponding to point O; if P is detected 01 Point P is determined based on the plane rectangular coordinate system corresponding to point O. 01 The vector corresponding to the point, P 01 The vector corresponding to the point starts at point O and ends at point P 01 The point is the end point.

[0047] As an example but not limitation, the position of the target image area can be represented by a vector formed by the center of the sphere corresponding to the panoramic data to the vertex of the target image area. For example, the position of the target image area can be represented by a vector formed by the center of the sphere corresponding to the panoramic data to the vertex of the upper left corner of the target image area.

[0048] In some embodiments, adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the starting point includes: calculating Φ(i+1) according to the position vector corresponding to the trajectory point after the starting point and a specified relationship, and adjusting the position of the target image area corresponding to the panoramic data according to the Φ(i+1), wherein the specified relationship is: Φ(i+1)=Φ(i)+λ△tQ, i is equal to 0 or i is a positive integer greater than 0, when i is equal to 0, Φ(i) is used to represent the initial position of the target image area (for example, the position of the target image area when no sliding operation is detected, or when a sliding operation is detected). The position of the target image area at the time when the sliding track starts is measured), when i is a positive integer greater than 0, Φ(i) is used to represent the position (vector) of the target image area after the i-th adjustment, the i-th adjustment can be the last adjustment relative to the current moment, λ is less than or greater than 1, λ is used to convert the unit of data, Δt is the first specified time length less than 1, Q is used to represent the position vector corresponding to the track point on the sliding track, for example, in order to calculate Φ(i+1) according to the position vector corresponding to the track point after the starting point and the specified relationship, the position vector corresponding to the track point after the starting point can be substituted into Q to calculate Φ(i+1). Φ(i+1) is used to represent the expected position value corresponding to the i+1th adjustment, and the expected position value is used to represent the expected position (vector) of the target image area after the i+1th adjustment. The terminal device can accurately achieve the i+1th adjustment by adjusting the position of the target image area corresponding to the panoramic data according to Φ(i+1).

[0049] For example, assuming that the last adjustment corresponding to the current moment is the second adjustment, that is, Φ(2) corresponding to the most recent adjustment to the current time is the position (vector) of the target image area after the second adjustment, λ and Δt are both pre-set data, and since the trajectory point after the starting point belongs to the trajectory point on the sliding trajectory, the position vector corresponding to the trajectory point after the starting point can be substituted into Q to calculate Φ(3), and then the position of the target image area corresponding to the panoramic data is adjusted according to Φ(3), thereby realizing the third adjustment.

[0050] In some embodiments, after adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the starting point, step A is included, and step A includes: if a k-th action breakpoint is detected, establishing a plane rectangular coordinate system corresponding to the k-th action breakpoint, wherein k is a positive integer greater than or equal to one, and the k-th action breakpoint is: a trajectory point generated because the contact time between the contact object and the display screen is greater than or equal to a second specified time length; based on the plane rectangular coordinate system corresponding to the k-th action breakpoint, determining the position vector corresponding to the trajectory point after the k-th breakpoint, the position vector corresponding to the trajectory point after the k-th breakpoint is: a vector formed by the k-th action breakpoint to the trajectory point after the k-th breakpoint, and the trajectory point after the k-th breakpoint is: a trajectory point detected after the k-th action breakpoint is detected; adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the k-th breakpoint.

[0051] In this embodiment, the terminal device can establish a plane rectangular coordinate system with the kth action breakpoint as the origin, that is, the plane rectangular coordinate system can be updated through the kth action breakpoint. Since the distance between the track point after the kth breakpoint and the kth action breakpoint in a sliding track is relatively close, the position of the target image area is adjusted according to the position vector corresponding to the track point after the kth breakpoint, which can greatly reduce or even eliminate the difference between the position of the adjusted target image area and the user's area of ​​interest, so that the terminal device can more accurately display the image screen that the user expects to see. This effectively avoids: the accuracy of the position adjustment of the target image area is reduced due to the long sliding track causing the track point to be too far from the sliding origin.

[0052] For example, Figure 2 As shown in the figure, it is assumed that point P1 is the first action breakpoint detected after the starting point of the sliding trajectory (a trajectory point generated because the contact time between the contact object and the display screen is greater than or equal to the second specified time length), that is, point P1 is the first action breakpoint, and the first action breakpoint is taken as the origin to establish a plane rectangular coordinate system corresponding to the first action breakpoint. Assume that point P1 is the first action breakpoint detected after the starting point of the sliding trajectory (a trajectory point generated because the contact time between the contact object and the display screen is greater than or equal to the second specified time length). 11 Point is the trajectory point detected after the first action breakpoint is detected, and the contact object is 11 The contact time between the display screen positions corresponding to the points is less than the second specified time length, that is, P 11 Point does not belong to the action breakpoint, but is a trajectory point after the first breakpoint. Based on the plane rectangular coordinate system corresponding to the first action breakpoint, determine P 11 The position vector corresponding to the point, the P 11 The position vector corresponding to the point is: 11 The vector formed by the point, the P 11The position vector corresponding to the point starts from the first action breakpoint and ends at the P 11 Point is the end point, according to the P 11 The position vector corresponding to the point adjusts the position of the target image area corresponding to the panoramic data, and the terminal device controls the display screen to display the image picture corresponding to the adjusted target image area.

[0053] In some embodiments, the number of trajectory points after the kth breakpoint is equal to 1.

[0054] In some embodiments, the number of trajectory points after the kth breakpoint is a positive integer greater than 1.

[0055] For example, Figure 2 As shown, point P1 is the first action breakpoint. After the terminal device establishes the plane rectangular coordinate system corresponding to the first action breakpoint, it detects point P 11 Point, contact object and P 11 The contact time between the display screen positions corresponding to the points is less than the second specified time length, P 11 The point belongs to the trajectory point after the starting point, determine P 11 The position vector corresponding to the point, and then according to P 11 The position vector corresponding to the point adjusts the position of the target image area corresponding to the panoramic data. For ease of description, the position vector corresponding to the point P is 11 The position of the target image area after the position vector corresponding to the point is adjusted is recorded as L 11 , the terminal device control display screen shows L 11 The corresponding image screen; then, the terminal device detects P 12 Point, contact object and P 12 The contact time between the display screen positions corresponding to the points is less than the second specified time length, P 12 The point belongs to the trajectory point after the starting point, determine P 12 The position vector corresponding to the point, and then according to P 12 The position vector corresponding to the point adjusts the position of the target image area corresponding to the panoramic data. For ease of description, the position vector corresponding to the point P is 12 The position of the target image area after the position vector corresponding to the point is adjusted is recorded as L 12 , the terminal device control display screen shows L 12 The corresponding image screen.

[0056] Optionally, the initial value of k is equal to a specified positive integer, the specified positive integer is a positive integer greater than or equal to one, and step A includes step A1, step A2, step A3 and step A4, wherein:

[0057] Step A1: if a k-th action breakpoint is detected, then a plane rectangular coordinate system corresponding to the k-th action breakpoint is established;

[0058] Step A2, determining the position vector corresponding to the trajectory point after the kth breakpoint based on the plane rectangular coordinate system corresponding to the kth action breakpoint;

[0059] Step A3, adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the kth breakpoint;

[0060] Step A4: If the k+1th action breakpoint is detected, set k=k+1, and then return to execute the step A1.

[0061] In this way, the terminal device can flexibly and accurately adjust the position of the target image area through multiple k-th action breakpoints.

[0062] By way of example and not limitation, Figure 2 As shown, assuming that the initial value of k is equal to 1, if the first action breakpoint P1 is detected, a plane rectangular coordinate system corresponding to the first action breakpoint is established, based on the plane rectangular coordinate system corresponding to the first action breakpoint, the position vector corresponding to the trajectory point after the first breakpoint is determined, and the position of the target image area corresponding to the panoramic data is adjusted according to the position vector corresponding to the trajectory point after the first breakpoint, and then the terminal device controls the display screen to display (according to the position vector corresponding to the trajectory point after the first breakpoint) the image picture corresponding to the adjusted position of the target image area; thereafter, if the second action breakpoint P2 is detected, a plane rectangular coordinate system corresponding to the second action breakpoint is established, based on the plane rectangular coordinate system corresponding to the second action breakpoint, the position vector corresponding to the trajectory point after the second breakpoint is determined, and the position of the target image area corresponding to the panoramic data is adjusted according to the position vector corresponding to the trajectory point after the second breakpoint, and then the terminal device controls the display screen to display (according to the position vector corresponding to the trajectory point after the second breakpoint) the image picture corresponding to the adjusted position of the target image area, and so on.

[0063] In an embodiment of the present application, a display screen is provided on the terminal device, and panoramic data is also stored in the terminal device. If the terminal device detects a sliding operation based on the display screen, the target image area corresponding to the panoramic data is adjusted according to the sliding trajectory corresponding to the sliding operation, and the sliding trajectory is the trajectory of the contact object sliding on the display screen, and the contact object is: an object that can contact the display screen and can slide on the display screen; the display screen is controlled to display the image screen corresponding to the adjusted target image area. Since the contact object is generally controlled by the user, the sliding trajectory can reflect the user's inner expectations to a certain extent. The terminal device adjusts the target image area corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation, and then controls the display screen to display the image screen corresponding to the adjusted target image area, that is, the terminal device can efficiently control itself (the terminal device) to display the image screen that the user expects to see.

[0064] Embodiment 2

[0065] Corresponding to the image processing method described in the above embodiment, an embodiment of the present application provides an image processing device, which is applied to a terminal device. The terminal device is provided with a display screen, and the terminal device also stores panoramic data. Figure 3 A schematic diagram of an image processing device provided in an embodiment of the present application is shown. The image processing device comprises: an adjustment unit 301 and a display control unit 302 .

[0066] The adjustment unit 301 is used to adjust the target image area corresponding to the panoramic data according to a sliding trajectory corresponding to the sliding operation if a sliding operation is detected based on the display screen, wherein the sliding trajectory is a trajectory of a contact object sliding on the display screen, and the contact object is an object that can contact the display screen and can slide on the display screen.

[0067] Optionally, the image processing device further includes: a setting unit.

[0068] The setting unit is used for: before the adjustment unit 301 executes the adjustment of the position of the target image area corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation, a virtual camera is set at the center of the sphere corresponding to the panoramic data, and the position of the area where the field of view of the virtual camera intersects with the panoramic data is the position of the target image area; correspondingly, when the adjustment unit 301 executes the adjustment of the position of the target image area corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation, it is specifically used for: rotating the virtual camera and / or the panoramic spherical image in the panoramic data according to the sliding trajectory corresponding to the sliding operation.

[0069] The display control unit 302 is used to control the display screen to display the image picture corresponding to the adjusted target image area.

[0070] Optionally, the trajectory points on the sliding trajectory include a starting point of the sliding trajectory and a trajectory point after the starting point, the trajectory point after the starting point being: a trajectory point detected after the starting point of the sliding trajectory is detected, and when the adjustment unit 301 performs the adjustment of the position of the target image area corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation, it is specifically used to: establish a plane rectangular coordinate system corresponding to the starting point of the sliding trajectory, the origin of the plane rectangular coordinate system corresponding to the starting point of the sliding trajectory is the starting point of the sliding trajectory; determine the position vector corresponding to the trajectory point after the starting point based on the plane rectangular coordinate system corresponding to the starting point of the sliding trajectory, the position vector corresponding to the trajectory point after the starting point is: a vector formed from the starting point of the sliding trajectory to the trajectory point after the starting point; adjust the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the starting point.

[0071] Optionally, after executing the step of adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the starting point, the adjustment unit 301 is also used to execute step A, wherein step A includes: if a k-th action breakpoint is detected, establishing a plane rectangular coordinate system corresponding to the k-th action breakpoint, wherein k is a positive integer greater than or equal to one, and the k-th action breakpoint is: a trajectory point generated because the contact time between the contact object and the display screen is greater than or equal to a second specified time length; based on the plane rectangular coordinate system corresponding to the k-th action breakpoint, determining the position vector corresponding to the trajectory point after the k-th breakpoint, the position vector corresponding to the trajectory point after the k-th breakpoint is: a vector formed by the k-th action breakpoint to the trajectory point after the k-th breakpoint, and the trajectory point after the k-th breakpoint is: a trajectory point detected after the k-th action breakpoint is detected; adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the k-th breakpoint.

[0072] It should be noted that for technical details not fully described in this embodiment, reference may be made to the image processing methods provided in the various embodiments in the above-mentioned embodiment 1.

[0073] Embodiment 3

[0074] The terminal device in this embodiment includes: at least one display screen, a processor, a memory, and a computer program stored in the memory and executable on the at least one processor, and the processor implements the steps in any of the above-mentioned image processing method embodiments when executing the computer program. The terminal device is also provided with a display screen, and the terminal device also stores panoramic data.

[0075] When the processor executes the computer program, the steps in the above-mentioned various image processing method embodiments are implemented, for example Figure 1Alternatively, when the processor executes the computer program, the functions of each unit in the above-mentioned device embodiments are realized, for example, Figure 3 The functions of the units 301 to 302 are shown.

[0076] Those skilled in the art will understand that this embodiment is merely an example of a terminal device and does not constitute a limitation on the terminal device. The terminal device may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0077] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0078] In some embodiments, the memory may be an internal storage unit of the terminal device, such as a hard disk or memory of the terminal device. In other embodiments, the memory may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device. Further, the memory may also include both an internal storage unit of the terminal device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data and other programs, such as the program code of the computer program, etc. The memory may also be used to temporarily store data that has been output or is to be output.

[0079] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0080] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0081] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0082] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0083] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0084] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0085] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0086] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0087] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0088] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The image processing method is applied to a terminal device, wherein a display screen is provided on the terminal device, and panoramic data is also stored in the terminal device, wherein the panoramic data is data including a panoramic spherical image, and the image processing method comprises: If a sliding operation is detected based on the display screen, the target image area corresponding to the panoramic data is adjusted according to a sliding trajectory corresponding to the sliding operation, wherein the sliding trajectory is a trajectory of a contact object sliding on the display screen, and the contact object is an object that can contact the display screen and slide on the display screen; The display screen is controlled to display the image frame corresponding to the adjusted target image area.

2. The image processing method according to claim 1, characterized in that: The adjusting the target image area corresponding to the panoramic data according to the sliding track corresponding to the sliding operation includes: The position of the target image area corresponding to the panoramic data is adjusted according to the sliding track corresponding to the sliding operation.

3. The image processing method according to claim 2, characterized in that: The track points on the sliding track include a sliding track starting point and a track point after the starting point, wherein the track point after the starting point is a track point detected after the sliding track starting point is detected. Correspondingly, adjusting the position of the target image area corresponding to the panoramic data according to the sliding track corresponding to the sliding operation includes: Establishing a plane rectangular coordinate system corresponding to the starting point of the sliding track, wherein the origin of the plane rectangular coordinate system corresponding to the starting point of the sliding track is the starting point of the sliding track; Determine the position vector corresponding to the trajectory point after the starting point based on the plane rectangular coordinate system corresponding to the starting point of the sliding trajectory, the position vector corresponding to the trajectory point after the starting point is: the vector formed from the starting point of the sliding trajectory to the trajectory point after the starting point; The position of the target image area corresponding to the panoramic data is adjusted according to the position vector corresponding to the trajectory point after the starting point.

4. The image processing method according to claim 3, characterized in that: The step of adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the starting point comprises: Φ(i+1) is calculated according to the position vector corresponding to the trajectory point after the starting point and the specified relationship, wherein the specified relationship is: Φ(i+1)=Φ(i)+λΔtQ, i is equal to 0 or i is a positive integer greater than 0; when i is equal to 0, Φ(i) is used to represent the initial position of the target image area; when i is a positive integer greater than 0, Φ(i) is used to represent the position of the target image area after the i-th adjustment, λ is less than or greater than 1, and Δt is a first specified time length less than 1; Q is used to represent the position vector corresponding to the trajectory point on the sliding trajectory; The position of the target image area corresponding to the panoramic data is adjusted according to the Φ(i+1).

5. The image processing method according to claim 3, characterized in that: After adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the starting point, step A is included, and step A includes: If a k-th action breakpoint is detected, a plane rectangular coordinate system corresponding to the k-th action breakpoint is established, wherein k is a positive integer greater than or equal to one, and the k-th action breakpoint is: a trajectory point generated because the contact time between the contact object and the display screen is greater than or equal to the second specified time length; Based on the plane rectangular coordinate system corresponding to the k-th action breakpoint, determine the position vector corresponding to the trajectory point after the k-th breakpoint, the position vector corresponding to the trajectory point after the k-th breakpoint is: a vector formed by the k-th action breakpoint to the trajectory point after the k-th breakpoint, the trajectory point after the k-th breakpoint is: a trajectory point detected after the k-th action breakpoint is detected; The position of the target image area corresponding to the panoramic data is adjusted according to the position vector corresponding to the trajectory point after the kth breakpoint.

6. The image processing method according to claim 5, characterized in that: The initial value of k is equal to a specified positive integer, and the specified positive integer is a positive integer greater than or equal to one. Step A includes: Step A1: if a k-th action breakpoint is detected, then a plane rectangular coordinate system corresponding to the k-th action breakpoint is established; Step A2, determining the position vector corresponding to the trajectory point after the kth breakpoint based on the plane rectangular coordinate system corresponding to the kth action breakpoint; Step A3, adjusting the position of the target image area corresponding to the panoramic data according to the position vector corresponding to the trajectory point after the kth breakpoint; Step A4: If the k+1th action breakpoint is detected, set k=k+1, and then return to execute the step A1.

7. The image processing method according to claim 2, characterized in that: Before adjusting the position of the target image area corresponding to the panoramic data according to the sliding track corresponding to the sliding operation, the method includes: A virtual camera is arranged at the center of the sphere corresponding to the panoramic data, and the position where the area where the field of view of the virtual camera intersects with the panoramic data is the position of the target image area; Correspondingly, adjusting the position of the target image area corresponding to the panoramic data according to the sliding trajectory corresponding to the sliding operation includes: rotating the virtual camera and / or the panoramic spherical image in the panoramic data according to the sliding trajectory corresponding to the sliding operation.

8. An image processing device, characterized in that: The image processing device is applied to a terminal device, the terminal device is provided with a display screen, the terminal device also stores panoramic data, the panoramic data is data including a panoramic spherical image, and the image processing device comprises: an adjusting unit, configured to adjust the target image area corresponding to the panoramic data according to a sliding trajectory corresponding to the sliding operation if a sliding operation is detected based on the display screen, wherein the sliding trajectory is a trajectory of a contact object sliding on the display screen, and the contact object is an object that can contact the display screen and slide on the display screen; The display control unit is used to control the display screen to display the image picture corresponding to the adjusted target image area.

9. A terminal device, comprising a display screen, a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images