Image processing method, device, storage medium and terminal device

Abstract

The present application discloses an image processing method, device, storage medium and terminal device, wherein the method comprises determining the state information corresponding to the target pixel point according to the posture information of the target user, and adjusting the pixel value of the target pixel point according to the state information to obtain the display pixel point corresponding to the pixel point. The present application determines the viewing angle information corresponding to the target user according to the posture information of the target user, and adjusts the pixel value of each pixel point in the target image to be played by the target device according to the viewing angle information, thereby improving the visual effect of the target image for the target user who uses the viewing angle information to watch the played image, thereby improving the visual experience of the target user.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, storage medium and terminal device. Background Art

[0002] With the advancement of technology and the improvement of residents' quality of life, the screen size of smart TVs is also increasing. Nowadays, 75-inch or even larger smart TVs are very common. However, when the screen size of smart TVs becomes larger, the viewing angle of users when watching smart TVs also becomes larger accordingly. When the user's viewing angle becomes larger, the display screen of smart TVs will have color shift, for example, low grayscale is brighter, high grayscale is darker, and so on. Summary of the invention

[0003] The technical problem to be solved by the present application is to provide an image processing method, apparatus, storage medium and terminal device in view of the deficiencies in the prior art.

[0004] In order to solve the above technical problems, the first aspect of the embodiments of the present application provides an image processing method, the method comprising:

[0005] Determine the state information corresponding to the target pixel point according to the posture information of the target user, wherein the target image is a video frame to be played by the target device; the state information includes viewing angle information, and the viewing angle information is used to reflect the positional relationship between the target user and the target device;

[0006] The pixel value of the target pixel is adjusted according to the state information to obtain a display pixel corresponding to the pixel.

[0007] The image processing method, wherein the viewing angle information includes a first angle between the user orientation of the target user and a target straight line, and a second angle between the target device and the target straight line, wherein the target straight line is a connecting line between the target user and the target pixel point of the target device, and the line of sight and the target straight line are located on the same plane.

[0008] The image processing method, wherein before determining the state information corresponding to the target pixel point according to the posture information of the target user, the method further comprises:

[0009] Detecting the target user's posture information at preset intervals;

[0010] When the detected posture information is different from the previous posture information, a step is performed to determine the state information corresponding to the target pixel point according to the posture information of the target user, wherein the previous posture information has a detection time that is before the detection time of the detected posture information and has the shortest time interval with the detection time corresponding to the detected posture information.

[0011] The image processing method, wherein the step of adjusting the pixel value of the target pixel point according to the state information to obtain a display pixel point corresponding to the pixel point specifically comprises:

[0012] Adjusting the pixel value of the target pixel point according to the state information to obtain a display pixel point corresponding to the pixel point;

[0013] According to all the obtained display pixels, a playback image corresponding to the target image is determined.

[0014] The image processing method, wherein the step of adjusting the pixel value of the target pixel point according to the state information to obtain a display pixel point corresponding to the pixel point specifically comprises:

[0015] For each target pixel in the target image, determining an adjustment curve and a correction coefficient corresponding to the target pixel based on the state information;

[0016] Based on the correction coefficient and the image block corresponding to the target pixel point, adjusting the pixel value of the target pixel point to obtain a candidate pixel point, wherein the image block is included in the target image, and the target image includes the target pixel point;

[0017] The pixel value of the candidate pixel point is adjusted based on the adjustment curve to obtain a display pixel point.

[0018] The image processing method, wherein the state information further includes environment information corresponding to the target device and a source category corresponding to the target image, and determining the correction coefficient corresponding to the target pixel point based on the state information specifically includes:

[0019] Based on the viewing angle information, determining a first correction coefficient corresponding to the target pixel point;

[0020] Based on the environmental information and the film source category, the second correction coefficient and the third correction coefficient corresponding to the target pixel are determined, and the correction coefficient set consisting of the first correction coefficient, the second correction coefficient and the third correction coefficient is used as the correction coefficient corresponding to the target pixel.

[0021] The image processing method, wherein adjusting the pixel value of the target pixel point based on the correction coefficient and the image block corresponding to the target pixel point to obtain the candidate pixel point specifically includes:

[0022] Determine a candidate pixel value corresponding to the target pixel based on the second correction coefficient, the third correction coefficient and the image block;

[0023] Based on the first correction coefficient, the pixel value of the target pixel point and the candidate pixel value are weighted to obtain the candidate pixel point corresponding to the target pixel point.

[0024] The image processing method, wherein the adjustment curve includes an R primary color adjustment curve, a G primary color adjustment curve, and a B primary color adjustment curve; the pixel value of the candidate pixel includes an R primary color value, a G primary color value, and a B primary color value; and the step of adjusting the pixel value of the candidate pixel based on the adjustment curve to obtain a display pixel specifically includes:

[0025] For each target primary color of the candidate pixel, determine a target primary color value corresponding to the target primary color according to a pixel value of the candidate pixel, and determine a display primary color value corresponding to the target primary color value based on the target primary color value and a target primary color curve corresponding to the target primary color, wherein the target primary color is included in the R primary color, the G primary color, and the B primary color, and the target primary color curve is included in the R primary color adjustment curve, the G primary color adjustment curve, and the B primary color adjustment curve;

[0026] The determined display primary color values ​​are used as target primary color values ​​of the corresponding target primary colors to obtain display pixel points.

[0027] The image processing method, wherein determining the playback image corresponding to the target image based on all the obtained display pixels specifically comprises:

[0028] Acquire display restriction conditions corresponding to the target image, wherein the display restriction conditions include display timeliness and display authenticity;

[0029] Selecting a fusion method corresponding to the target image according to the display restriction condition;

[0030] According to the fusion method, each display pixel is fused with its corresponding target pixel to obtain a playback image corresponding to the target image.

[0031] The image processing method, wherein after determining the playback image corresponding to the target image based on all the obtained display pixels, the method further comprises:

[0032] The playback image is displayed through the target device.

[0033] A second aspect of an embodiment of the present application provides an image processing device, the image processing device comprising:

[0034] A first determination module is used to determine state information corresponding to each target pixel point according to posture information of each target pixel point in a target image, and adjust a pixel value of the target pixel point according to the state information to obtain a display pixel point corresponding to the pixel point, wherein the target image is a video frame to be played by a target device; the state information includes viewing angle information, and the viewing angle information is used to reflect a positional relationship between the target user and the target device;

[0035] The second determination module is used to determine the playback image corresponding to the target image according to all the obtained display pixels.

[0036] A third aspect of an embodiment of the present application provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in any of the image processing methods described above.

[0037] A fourth aspect of the embodiments of the present application provides a terminal device, comprising: a processor, a memory, and a communication bus; the memory stores a computer-readable program that can be executed by the processor;

[0038] The communication bus realizes the connection and communication between the processor and the memory;

[0039] When the processor executes the computer-readable program, the steps in any of the above-mentioned image processing methods are implemented.

[0040] Beneficial effects: Compared with the prior art, the present application provides an image processing method, device, storage medium and terminal device, wherein the method includes determining the state information corresponding to the target pixel point according to the posture information of the target user, and adjusting the pixel value of the target pixel point according to the state information to obtain the display pixel point corresponding to the pixel point. The present application determines the viewing angle information corresponding to the target user according to the posture information of the target user, and adjusts the pixel value of each pixel point in the target image to be played by the target device according to the viewing angle information, thereby improving the visual effect of the target image for the target user who uses the viewing angle information to watch the played image, thereby improving the visual experience of the target user. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1A flowchart of the image processing method provided in this application.

[0043] Figure 2 This is an example flowchart of the image processing method provided in this application.

[0044] Figure 3 Schematic diagram of the location relationship between the target device and the target user in this application.

[0045] Figure 4 A schematic diagram of the relationship between the viewing angle information and the location information of the target user in this application.

[0046] Figure 5 This is a structural principle diagram of the image processing device provided in this application.

[0047] Figure 6 This is a schematic diagram of the structure of the terminal device provided in this application. DETAILED DESCRIPTION

[0048] The present application provides an image processing method, apparatus, storage medium and terminal device. To make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0050] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0051] In a specific implementation, the terminal device described in the embodiments of the present application includes, but is not limited to, other portable devices such as mobile phones with touch-sensitive surfaces (e.g., touch display screens and / or touch pads), laptop computers, or tablet computers. It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch display screens and / or touch pads).

[0052] In the following discussion, a terminal device including a display and a touch-sensitive surface is described. However, it should be understood that the terminal device may also include one or more other physical user interface devices such as a physical keyboard, a mouse and / or a joystick.

[0053] The terminal device supports various applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, video conferencing applications, disk burning applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, exercise support applications, photo management applications, data camera applications, digital video camera applications, web browsing applications, digital music player applications and / or digital video playback applications, etc.

[0054] Various applications that can be executed on the terminal device can use at least one common physical user interface device such as a touch-sensitive surface. The first or multiple functions of the touch-sensitive surface and the corresponding information displayed on the terminal can be adjusted and / or changed between applications and / or within the corresponding application. In this way, the common physical framework of the terminal (e.g., the touch-sensitive surface) can support various applications with user interfaces that are intuitive and transparent to the user.

[0055] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0056] After research, the inventor found that with the advancement of technology and the improvement of residents' quality of life, the screen size of smart TVs is also increasing. Nowadays, 75-inch or even larger smart TVs have become very common. However, when the screen size of smart TVs becomes larger, the viewing angle of users when watching smart TVs also becomes larger accordingly. However, the liquid crystal displays currently used in smart TVs generally have inherent defects. This inherent defect is that when the user's viewing angle becomes larger, the optical brightness curve of each pixel in the liquid crystal display no longer follows the gamma value of 2.2, but instead appears as low grayscale is brighter and high grayscale is darker, which causes the color of the display screen of the liquid crystal display to shift at a large viewing angle, thereby affecting the user's viewing effect.

[0057] For the above-mentioned problem of poor display effect at a large viewing angle, the currently commonly used solution is to divide the sub-pixels of a display device (e.g., a liquid crystal display, etc.) into two display areas, where the sub-pixel partitioning can be physical space partitioning or time partitioning; by allowing the two display areas to maintain different grayscale voltages, the liquid crystal deflection angles corresponding to the two display areas are different, thereby causing the gamma value of their optical brightness curve to change, while keeping the gamma value of the optical brightness curve of the neutralized complete pixel display close to 2.2, thereby improving the display effect. However, the above-mentioned solution requires additional hardware costs, and the aperture ratio and transmittance of the liquid crystal panel will also be affected.

[0058] In order to solve the above problem, in an embodiment of the present application, the state information corresponding to the target pixel point is determined according to the posture information of the target user, and the pixel value of the target pixel point is adjusted according to the state information to obtain the display pixel point corresponding to the pixel point. The present application determines the viewing angle information corresponding to the target user according to the posture information of the target user, and adjusts the pixel value of each pixel in the target image to be played by the target device according to the viewing angle information, thereby improving the visual effect of the target image for the target user who uses the viewing angle information to watch the played image, thereby improving the visual experience of the target user.

[0059] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0060] This implementation provides an image processing method, such as Figure 1 and 2 As shown, the method includes:

[0061] S10. For each target pixel in the target image, determine the state information corresponding to the target pixel according to the posture information of the target user.

[0062] Specifically, the target image is a video frame to be played through a target device, and the target device is a terminal device watched by a target user. In other words, the target device is used to play the target image, and the target device is within the field of view of the target user, so that the target user can watch the target image through the target device. In one implementation of this embodiment, the target device is configured with a display device (e.g., a display screen, etc.), and the target image is displayed through the display device so that the target user can watch the target image. For example, the target device is a smart TV, and the target user watches the video frame played on the smart TV. The smart TV is configured with the image processing method provided in this embodiment. When the smart TV obtains the target image to be played through the display device configured by it, the image processing method provided in this embodiment is run to obtain the playback image corresponding to the target image, and the playback image is displayed to improve the viewing effect of the target user, wherein the screen size of the display screen is larger than the preset screen size, and the preset screen size can be pre-configured, for example, 60 inches, 65 inches, etc.

[0063] The posture information is the posture information of the target user relative to the target device, wherein the posture information includes position information and user orientation, wherein the position information is used to reflect the position of the target user relative to the target device, and the user orientation is used to reflect the eye orientation of the target user. For example, in one implementation, the user orientation can be the eye orientation of the target user. In one implementation of this embodiment, the position information includes the vertical distance from the target user to the center of the display device and the horizontal distance from the target user to the center of the display device, and the process of determining the position information can obtain a scene image of the scene where the target user is located through the target device, and determine the position information of the target user based on the scene image, wherein the target device is configured with a shooting device (e.g., a camera, etc.). For example, Figure 3 As shown, assuming that the shooting device is a camera, the camera is configured in the middle of the upper edge of the display device of the target equipment, and the scene image of the scene in which the target user is located is collected by the camera, and the face image in the scene image is recognized. Based on the face image, the vertical distance L1 from the target user to the center of the display device and the horizontal distance L2 from the target user to the center of the display device can be obtained.

[0064] The state information is used to reflect the viewing state of the target image, wherein the state information includes viewing angle information, and the viewing angle information is used to reflect the positional relationship between the target user and the target device. In an implementation of this embodiment, the viewing angle information includes a first angle between the user orientation of the target user and a target straight line, and a second angle between the target device and the target straight line, wherein the target straight line is a connecting line between the target user and the target pixel point of the target device, and the sight line and the target straight line are located on the same plane.

[0065] In an implementation of this embodiment, it is assumed that the height of the eyes of the target user is within the height range of the display device, and the height of the eyes of the target user is used as a cross section of the three-dimensional space, and the viewing angle information from the target user to the rightmost edge pixel of the display device is used as an example to illustrate the process of determining the viewing angle information, wherein the rightmost edge pixel is located on the cross section. The process of determining the viewing angle information is specifically as follows:

[0066] Determining a second angle according to the position information of the target user and the device size of the display device;

[0067] The first angle is determined according to the second angle to obtain viewing angle information.

[0068] Specifically, Figure 4 As shown, the second angle includes the right triangle 0AB, excluding the right triangle 0AB, the 0B side is equal to the vertical distance L1 from the target user to the center of the display device, and the AB side is equal to the sum of the horizontal distance L2 from the target user to the center of the display device and the target distance L3, wherein the target distance L3 is half of the horizontal length of the display device. Therefore, after obtaining the location information of the target user and the device size of the display device, L1, L2 and L3 can be obtained, so according to the trigonometric function relationship, the tangent value of the second angle A4 can be known to be tan(A4)=L1 / (L2+L3), and thus, the second angle A4=arctan(L1 / (L2+L3)).

[0069] like Figure 4 It can be seen that the sum of the first angle, the second angle and the angle A3 is 90 degrees. After obtaining the second angle, only A3 needs to be obtained to determine the first angle. Figure 4It can be seen that the sum of angle A3 and angle A2 is 90 degrees, and since the angle A2 formed by the parallel line F4 of the target user plane extension line F3 and the parallel line of the normal F2 relative to the center of the display device is equivalent to the angle A1 formed by the parallel line of the user plane extension line F3 and the normal F2 relative to the center of the display device, it can be known that A3=90-A2=90-A1. Since L1 and L2 are known, A1 can be calculated according to the principle of similar triangles. Based on this, the angle A3 can be determined according to the position information, and based on the second angle A4 and the angle A3, the first angle A5=90-A3-A4 can be determined, wherein the first angle is the user orientation of the target user and the viewing angle from the target user to the rightmost edge of the display device, wherein the user orientation of the target user is a straight line passing through the user center w of the user plane where the target user is located and extending along the user orientation direction, wherein the user center w is the center point of the part between the user's eyes, and the straight line and the device plane where the display device is located have an intersection, recorded as the center positioning point w0, the user normal F1 of the user plane where the target user is located is the line connecting the user center w and the center positioning point w0, the target user plane extension line F3 is an execution passing through the user center w and perpendicular to the user normal, and the user plane extension line F3 and the user normal F1 are both located on the section. In addition, it is worth noting that for other pixels in the display device, the corresponding second angle A4 (x, y) and first angle A5 (x, y) can be obtained according to the above steps, where x and y represent the positioning coordinates of the pixel respectively.

[0070] In an implementation of this embodiment, before determining the state information corresponding to the target pixel point according to the posture information of the target user, the method further includes:

[0071] Detecting the target user's posture information at preset intervals;

[0072] When it is detected that the posture information is different from the previous posture information, a step of determining the state information corresponding to the target pixel point according to the posture information of the target user is performed.

[0073] Specifically, the preset time can be pre-set, for example, 5 minutes, 10 minutes, etc. It is understood that the target device detects the target user's posture information once every preset time during use; or, the preset time can also be determined based on the power-on operation of the target device, and the preset time is the interval between two power-on operations. It is understood that the target device executes the target user's posture information once each time the power-on operation is performed. Of course, in actual applications, the target device can also perform detection operations in real time.

[0074] The detection time of the previous posture information is before the detection time of the posture information, and in chronological order, the detection time of the previous posture information is adjacent to the detection time of the posture information, and there is no detection operation between the detection time of the previous posture information and the detection time of the posture information. In other words, the detection time of the previous posture information is before the detection time of the detected posture information, and the time interval of the detection time corresponding to the detected posture information is the shortest, and there is no posture information whose detection time is between the detection time of the previous posture information and the detection time of the posture information. Among them, the previous posture information can be stored in the terminal device used for the image processing method provided by this embodiment, and can also be stored in the cloud, etc. In one implementation of this embodiment, the target device is a smart TV, the image processing method provided by this embodiment runs in the target device, and the previous posture information is stored in the target device.

[0075] Further, the posture information includes position information and user orientation, wherein the previous posture information is different from the posture information, which means that the position information in the previous posture information is different from the position information in the posture information and / or the user orientation in the previous posture information is different from the user orientation in the posture information. For example, if the position information in the previous posture information is different from the position information in the posture information, then the previous posture information is different from the posture information; for another example, if the user orientation in the previous posture information is different from the user orientation in the posture information, then the previous posture information is different from the posture information.

[0076] S20. Adjust the pixel value of the target pixel point according to the state information to obtain a display pixel point corresponding to the pixel point.

[0077] Specifically, the target image is a video frame to be played by the target device, the played image is an adjusted video frame corresponding to the target image, and the target device plays the played image on a display device configured therewith so that the target user can watch the played image. The target pixel point is included in the target image, and when adjusting the pixel value of the target pixel point in the target image, the pixel value of some target pixel points in the target image can be adjusted, or the pixel value of each target pixel point in the target image can be adjusted. In addition, the display pixel point is a pixel point configured with an adjusted pixel value, the display pixel point corresponds to the target pixel point one by one, and the pixel coordinates corresponding to the display pixel point are the same as the pixel coordinates of the corresponding target pixel point.

[0078] In an implementation of this embodiment, adjusting the pixel value of the target pixel point according to the state information to obtain a display pixel point corresponding to the pixel point specifically includes:

[0079] For each target pixel in the target image, determining an adjustment curve and a correction coefficient corresponding to the target pixel based on the state information;

[0080] Based on the correction coefficient and the image block corresponding to the target pixel, the pixel value of the target pixel is adjusted to obtain a candidate pixel;

[0081] The pixel value of the candidate pixel is adjusted based on the adjustment curve to obtain the display pixel. Specifically, the adjustment curve and the correction coefficient are both used to adjust the pixel value of the target pixel, wherein the correction coefficient is used to correct the pixel value of the target pixel according to the neighborhood where the target pixel is located, and the adjustment curve is a three-primary color adjustment curve, and the pixel value of the target pixel is adjusted by adjusting the three-primary color values ​​of the target pixel. Wherein, when adjusting the pixel value of the target pixel based on the correction coefficient and the adjustment curve, the target pixel can be adjusted based on the correction coefficient first, and then the adjusted target pixel can be adjusted based on the adjustment curve; or the target pixel can be adjusted based on the adjustment curve first, and then the adjusted target pixel can be adjusted based on the correction coefficient, etc. In a specific implementation of the present embodiment, the target pixel is first adjusted based on the correction coefficient, and then the adjusted target pixel is adjusted based on the adjustment curve.

[0082] The image block is included in the target image, and the target image includes the target pixel. In other words, the image block is an image region in the target image that includes the target pixel. The image block can be determined based on the target pixel. For example, after obtaining the target pixel, the 3*3 neighborhood corresponding to the target pixel is obtained with the target pixel as the center to obtain the image block corresponding to the target pixel. Of course, in practical applications, in order to obtain an image block for each target pixel in the target image, the target image can be filled before obtaining the image block, for example, the periphery of the target image is filled with pixels with a 0 pixel value.

[0083] In an implementation of this embodiment, the state information further includes environment information corresponding to the target device and a source category corresponding to the target image; and determining the correction coefficient corresponding to the target pixel point based on the state information specifically includes:

[0084] Based on the viewing angle information, determining a first correction coefficient corresponding to the target pixel point;

[0085] Based on the environmental information and the film source category, the second correction coefficient and the third correction coefficient corresponding to the target pixel are determined, and the correction coefficient set consisting of the first correction coefficient, the second correction coefficient and the third correction coefficient is used as the correction coefficient corresponding to the target pixel.

[0086] Specifically, the environment information is used to reflect the playback environment of the target device, and the source category is used to reflect the resource category of the multimedia resource that provides the target image. The source category can be determined based on the multimedia resource corresponding to the target image, where the source category may include "movies", "TV series", "sports programs", "news programs" and "app interactive applications". For example, if the multimedia resource corresponding to the target image is a movie, then the source category corresponding to the target image is a movie.

[0087] The environmental information can be determined based on the ambient brightness of the environment in which the target device is located. The process of determining the environmental information can be: the ambient brightness value of the environment in which the target device is located is collected by a collection device (for example, a light sensor, etc.) configured by the target device, and the target environment category corresponding to the target device is determined based on the collected ambient brightness value, and the target environment category obtained is used as the environmental information corresponding to the target device, wherein the target environment category is included in a plurality of preset environment categories. Each of the plurality of environment categories is configured with an ambient brightness range, and the ambient brightness ranges corresponding to the respective environment categories do not overlap. Thus, when the ambient brightness value of the environment in which the target device is located is obtained, the ambient brightness range corresponding to the ambient brightness range containing the ambient brightness value of the environment in which the target device is located is selected from the plurality of environment categories, and the selected environment category is used as the target environment category. For example, several environment categories include "cinema environment", "dark home environment", "bright home environment" and "sunlight environment". When the ambient brightness value of the environment in which the target device is located is included in the ambient brightness range corresponding to the "cinema environment", the target environment category corresponding to the target device is the "cinema environment" category, and accordingly, the environmental information is the cinema environment.

[0088] In an implementation of this embodiment, the first correction coefficient is determined based on the first angle and the second angle, and the second correction coefficient and the third correction coefficient are determined based on the environmental information and the source category, wherein the first correction coefficient, the second correction coefficient and the third correction coefficient are all within the range of 0-1. In a specific implementation, the calculation formula of the first correction coefficient S0_A can be:

[0089] S0_A=(A4+A5) / 180

[0090] Wherein, S0_A represents the first correction coefficient, A4 represents the angle value of the second angle, and A5 represents the angle value of the first angle. In addition, from the calculation process of the first angle and the second angle, it can be known that the sum of the angle value of the first angle and the angle value of the second angle is less than 90, so (A4+A5) / 180 is within the range of 0-1.

[0091] The second correction coefficient and the third correction coefficient can be found in a preset correction coefficient database based on environmental information and film source categories. The correction coefficient database includes several data groups, and each of the several data groups includes two sets of data information. The first set of data information in the two data information is environmental information and film source categories, and the second set of data information is the second correction coefficient and the third correction coefficient. Therefore, after obtaining the environmental information and the film source category, the environmental information and the film source category are used as query conditions, and the second correction coefficient and the third correction coefficient can be found in the correction coefficient database. Of course, in actual applications, the correspondence between environmental information and film source categories and the second correction coefficient and the third correction coefficient can also be stored in other ways, which will not be explained here one by one. As long as the second correction coefficient and the third correction coefficient can be obtained based on the environmental information and the film source category, any method is acceptable.

[0092] In an implementation of this embodiment, after obtaining the correction coefficients including the first correction coefficient, the second correction coefficient and the third correction coefficient, the pixel value of the target pixel point can be adjusted based on the correction coefficient. Accordingly, adjusting the pixel value of the target pixel point to obtain the candidate pixel point based on the correction coefficient and the image block corresponding to the target pixel point specifically includes:

[0093] Determine a candidate pixel value corresponding to the target pixel based on the second correction coefficient, the third correction coefficient and the image block;

[0094] Based on the first correction coefficient, the pixel value of the target pixel point and the candidate pixel value are weighted to obtain the candidate pixel point corresponding to the target pixel point.

[0095] Specifically, the image block includes the target pixel point and several reference pixel points, and the candidate pixel value is determined based on the pixel value of the target pixel point and the pixel value of each reference pixel point in the several reference pixel points. The process of the candidate pixel value can first determine the fourth correction coefficient according to the second correction coefficient, and determine the fifth correction coefficient according to the third correction coefficient, wherein the sum of the second correction coefficient and the fourth correction coefficient is 1, and the sum of the third correction coefficient and the fifth correction coefficient is 1; then, four reference pixel values ​​are determined according to the image block, and the four reference pixel values ​​are weighted by the second correction coefficient, the third correction coefficient, the fourth correction coefficient and the fifth correction coefficient to obtain the candidate pixel value.

[0096] In one implementation of the present embodiment, the four reference pixel values ​​can be obtained by randomly selecting three reference pixel points from the image block, and taking the pixel values ​​corresponding to the three selected parameter pixel points and the target pixel point as reference pixel values. For example, the image block is a 3*3 neighborhood of the target pixel point, and the 9 pixel points in the target image block are respectively recorded as S0, S1, ..., S8, where S0 represents the target pixel point, and S1, S5 and S7 are selected as parameter pixel points, then the four reference pixel values ​​are the pixel values ​​of S0, S1, S5 and S7 respectively. Correspondingly, the candidate pixel value = (second correction coefficient * S1 + third correction coefficient * S7 + fourth correction coefficient * S5 + fifth correction coefficient * S0) / 4.

[0097] In one implementation of this embodiment, the four reference pixel values ​​include the pixel value of the target pixel, and each reference pixel value other than the pixel value of the target pixel may correspond to a number of pixels in the image block, and the number of pixels corresponding to each reference pixel is different. Here, each reference pixel value other than the pixel value of the target pixel includes reference pixel value A, reference pixel value B, and reference pixel value C, and the pixel points in the image block are sorted in order from top to bottom and from left to right to obtain a pixel sequence, reference pixel value A may correspond to a preset number of pixels, reference pixel value B may correspond to a preset number of pixels, and reference pixel value C corresponds to each reference pixel value other than the pixel value of the target pixel, wherein the preset number is less than the data of the pixels included in the image block, specifically, 2 times the preset number is less than the data of the pixels included in the image block; the sum of the correction parameter corresponding to the reference pixel value C and the correction parameter corresponding to the target pixel is 1, for example, the reference pixel value C is for the fourth correction parameter, the target pixel corresponds to the fifth correction parameter, and each reference pixel value is equal to the average of the pixel values ​​of all the corresponding pixels.

[0098] For example, the image block is a 3*3 neighborhood of the target pixel, the target pixel is denoted as S0, and the reference pixels other than the target pixel are sorted in order from top to bottom and from left to right and denoted as S1, ..., S8, where the pixel points corresponding to the reference pixel value A are S1, S2 and S3, the pixel points corresponding to the reference pixel value B are S6, S7 and S8, and the pixel points corresponding to the reference pixel value C are S1, ..., S8. The calculation formula for the candidate pixel value is:

[0099] P=((S0_B*(S1+S2+S3) / 3+(1-S0_B)*(S6+S7+S8) / 3)+S0_C*(S1+S2+S3+…+S8) / 8+(1-S0_C)*S0) / 2.

[0100] Among them, P represents the candidate pixel value, S0_B represents the second correction parameter, (1-S0_B) represents the third correction parameter, S0_C represents the fourth correction parameter, (1-S0_C) represents the fifth correction parameter, S0 represents the target pixel point, S1, S2, S3, ..., S8 represent the pixel points in the image block except the target pixel point.

[0101] In one implementation of this embodiment, after obtaining the candidate pixel value, based on the first correction coefficient, the pixel value of the target pixel point and the candidate pixel value are weighted, specifically: based on the first correction coefficient, the sixth correction coefficient is determined, wherein the sum of the first correction coefficient and the sixth correction coefficient is equal to 1, and then the first correction coefficient is used as the weighting coefficient corresponding to the target pixel point, and the sixth correction coefficient is used as the weighting coefficient corresponding to the candidate pixel value, and the two are weighted to obtain the target pixel value, and the pixel value of the target pixel point is replaced with the target pixel value to obtain the candidate pixel point. For example, the calculation formula of the candidate pixel value is the calculation formula in the above example, then the calculation formula of the target pixel value can be:

[0102] S9=S0_A*S0+(1-S0_A)*((S0_B*(S1+S2+S3) / 3+(1-S0_B)*(S6+S7+S8) / 3)+S0_C*(S1+S2+S3+…+S8) / 8+(1-S0_C)*S0) / 2.

[0103] Among them, S9 represents the target pixel value, S0_A represents the first correction coefficient, (1-S0_A) represents the sixth correction coefficient, S0_B represents the second correction parameter, (1-S0_B) represents the third correction parameter, S0_C represents the fourth correction parameter, (1-S0_C) represents the fifth correction parameter, S0 represents the target pixel point, S1, S2, S3, ..., S8 represent pixel points in the image block except the target pixel point.

[0104] In an implementation of this embodiment, the adjustment curve includes an R primary color adjustment curve, a G primary color adjustment curve, and a B primary color adjustment curve; the pixel value of the candidate pixel includes an R primary color value, a G primary color value, and a B primary color value; and adjusting the pixel value of the candidate pixel based on the adjustment curve to obtain a display pixel specifically includes:

[0105] For each target primary color of the candidate pixel, determine a target primary color value corresponding to the target primary color according to the pixel value of the candidate pixel, and determine a display primary color value corresponding to the target primary color value based on the target primary color value and a target primary color curve corresponding to the target primary color;

[0106] The determined display primary color values ​​are used as target primary color values ​​of the corresponding target primary colors to obtain display pixel points.

[0107] Specifically, the target primary color is included in the R primary color, the G primary color and the B primary color, and the target primary color curve is included in the R primary color adjustment curve, the G primary color adjustment curve and the B primary color adjustment curve. The adjustment curve is determined based on the viewing angle information. When the target primary color value is input into the target primary color curve, the target primary color curve will determine the display primary color value corresponding to the target primary color value. It can be understood that the target primary color curve is a relationship curve between the target primary color value and the display primary color value, and the display primary color value corresponding to the target primary color value can be determined based on the relationship curve. In an implementation of the present embodiment, the determination process of the adjustment curve can be: determine the viewing angle index value corresponding to the target pixel point according to the viewing angle information, and search the adjustment curve corresponding to the viewing angle index value in the preset adjustment curve database based on the viewing angle index value to obtain the adjustment curve corresponding to the target image. Among them, the calculation formula of the viewing angle index value can be:

[0108] A_R=A5+A_A*(A4+A5) / 3

[0109] Among them, A_R represents the viewing angle index value, A_A represents the proportional coefficient, A4 represents the second angle, and A5 represents the first angle.

[0110] In an implementation of this embodiment, the preset adjustment curve database is pre-configured, and the preset adjustment curve database includes a plurality of adjustment curves, each of the plurality of adjustment curves corresponds to a viewing angle index value, and the adjustment curve corresponds to the viewing angle index value one by one, so that the adjustment curve can be found in the preset adjustment curve database based on the viewing angle index value. The establishment process of the preset adjustment curve database can be: obtaining basic optical data under different viewing angles, wherein the basic optical data corresponding to each viewing angle can include chromaticity and brightness data of R / G / B / W monochrome fields under multiple grayscales, etc., and determining the adjustment curve based on the obtained basic optical data, wherein the adjustment curve includes an R primary color adjustment curve, a G primary color adjustment curve, and a B primary color adjustment curve. Wherein, each primary color adjustment curve in the adjustment curve includes a 256 element array, each element array corresponds to an element value, and the value range of the element value in each element array is 0-255 and the corresponding element values ​​are different from each other, wherein 0-255 includes 0 and 255. In one implementation of this embodiment, the R primary color adjustment curve of the preset adjustment curve database can be recorded as R_N[M], the G primary color adjustment curve can be recorded as G_N[M], and the B primary color adjustment curve can be recorded as B_N[M], where M represents the element value, and its value range is an integer in the range of 0-255, and N represents the viewing angle index value, and its value range is an integer in the range of 0 to 90.

[0111] S30: Determine the playback image corresponding to the target image according to all the obtained display pixels.

[0112] Specifically, the playback image is an image after the target image is processed, and the playback image includes obtaining all display pixels, and each display pixel in the playback image corresponds to a target pixel in the target image, and the pixel coordinates of the display pixel in the playback image are the same as the pixel coordinates of the corresponding target pixel in the target image, and the pixel value of the display pixel is obtained by adjusting the pixel value of the corresponding target pixel. For example, the display pixel A corresponds to the target pixel B, the pixel coordinates of the target pixel B in the target image are (100, 100), and the adjusted pixel value corresponding to the target pixel B is C, then the pixel coordinates of the display pixel A in the playback image are (100, 100), and the pixel value of the display pixel A is C.

[0113] In an implementation of this embodiment, in order to make the playback image meet the playback requirements of the multimedia resource corresponding to the target image, after the display pixel points are obtained, the fusion mode corresponding to the target image can be determined according to the display restriction conditions corresponding to the multimedia resource. Based on this, determining the playback image corresponding to the target image according to all the obtained display pixel points specifically includes:

[0114] Obtaining display restriction conditions corresponding to the target image;

[0115] Selecting a fusion method corresponding to the target image according to the display restriction condition;

[0116] According to the fusion method, each display pixel is fused with its corresponding target pixel to obtain a playback image corresponding to the target image.

[0117] Specifically, the display restriction condition is used to limit the image quality and response speed of the playback image. The display restriction condition includes display timeliness and display authenticity, wherein display timeliness is used to reflect the response speed of the playback image, and display authenticity is used to reflect the image quality of the playback image, wherein the display restriction condition can be determined according to the source category corresponding to the target image, for example, a plurality of display restriction conditions are pre-stored, and each of the plurality of source categories corresponds to a display restriction condition, and after the source category is determined, the display restriction condition corresponding to the source category can be determined.

[0118] Furthermore, the fusion method may include image fusion, filtering first and then fusion, etc., and the fusion method corresponds to the display restriction condition. After the display restriction condition is obtained, the fusion method corresponding to the display restriction condition may be selected, and the target image and the candidate image may be fused by the fusion method to obtain the playback image. For example, when the display timeliness requirement is high in the display restriction condition, the fusion method may be to weight the pixel value of the target pixel point and the corresponding display pixel point according to a preset ratio; when the display authenticity requirement is high in the display restriction condition, the fusion method may be to weight the pixel value of the target pixel point and the corresponding display pixel point according to a preset ratio to obtain the playback image, and then filter and smooth the playback image.

[0119] In an implementation of this embodiment, after determining the playback image corresponding to the target image based on all the display pixels obtained, the playback image can be played through a display device configured by the target device, so that the target user can watch the adjusted playback image, thereby improving the viewing effect of the target user. The target device is a terminal device watched by the target user, and the target user can obtain the video frame through the target device.

[0120] In summary, this embodiment provides an image processing method, which includes determining the state information corresponding to the target pixel point according to the posture information of the target user, and adjusting the pixel value of the target pixel point according to the state information to obtain the display pixel point corresponding to the pixel point. This application determines the viewing angle information corresponding to the target user according to the posture information of the target user, and adjusts the pixel value of each pixel point in the target image to be played by the target device according to the viewing angle information, thereby improving the visual effect of the target image for the target user who uses the viewing angle information to watch the played image, thereby improving the visual experience of the target user.

[0121] Based on the above image processing method, this embodiment provides an image processing device, such as Figure 5 As shown, the image processing device includes:

[0122] The first determination module 100 is used to determine the state information corresponding to each target pixel point according to the posture information of each target pixel point in the target image, and adjust the pixel value of the target pixel point according to the state information to obtain the display pixel point corresponding to the pixel point, wherein the target image is a video frame to be played by the target device; the state information includes viewing angle information, and the viewing angle information is used to reflect the position relationship between the target user and the target device;

[0123] The second determination module 200 is used to determine the playback image corresponding to the target image according to all the obtained display pixels.

[0124] In addition, it is worth noting that the working process of the image processing device provided in this embodiment is the same as the working process of the above-mentioned image processing method, which will not be repeated here. The specific description of the working process of the above-mentioned image processing method can be referred to.

[0125] Based on the above-mentioned image processing method, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the image processing method as described in the above-mentioned embodiment.

[0126] Based on the above image processing method, the present application also provides a terminal device, such as Figure 6 As shown, it includes at least one processor (processor) 20; display screen 21; and memory (memory) 22, and may also include a communications interface (Communications Interface) 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22 and the communication interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0127] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0128] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0129] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0130] In addition, the specific process of loading and executing the plurality of instructions in the storage medium and the processor in the mobile terminal has been described in detail in the method above, and will not be described one by one here.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: For each target pixel in a target image, determining state information corresponding to the target pixel according to the posture information of the target user, wherein the target image is a video frame to be played by a target device; the state information includes viewing angle information, and the viewing angle information is used to reflect the positional relationship between the target user and the target device; Adjusting the pixel value of the target pixel point according to the state information to obtain a display pixel point corresponding to the pixel point; Determine the playback image corresponding to the target image according to all the obtained display pixels; The step of adjusting the pixel value of the target pixel point according to the state information to obtain a display pixel point corresponding to the pixel point specifically includes: Determine the adjustment curve and correction coefficient corresponding to the target pixel point based on the state information; Based on the correction coefficient and the image block corresponding to the target pixel point, adjusting the pixel value of the target pixel point to obtain a candidate pixel point, wherein the image block is included in the target image, and the target image includes the target pixel point, and the image block is used to determine the neighborhood of the target pixel point; The pixel value of the candidate pixel point is adjusted based on the adjustment curve to obtain a display pixel point.

2. The image processing method according to claim 1, characterized in that: The viewing angle information includes a first angle between the user orientation of the target user and a target straight line, and a second angle between the target device and the target straight line, wherein the target straight line is a connecting line between the target user and the target pixel point of the target device, and the user orientation and the target straight line are located on the same plane.

3. The image processing method according to claim 1, characterized in that: Before determining the state information corresponding to the target pixel point according to the posture information of the target user, the method further includes: Detecting the target user's posture information at preset intervals; When the detected posture information is different from the previous posture information, a step is performed to determine the state information corresponding to the target pixel point according to the posture information of the target user, wherein the previous posture information has a detection time that is before the detection time of the detected posture information and has the shortest time interval with the detection time corresponding to the detected posture information.

4. The image processing method according to claim 1, characterized in that: The state information also includes the environment information corresponding to the target device and the source category corresponding to the target image; and determining the correction coefficient corresponding to the target pixel point based on the state information specifically includes: Based on the viewing angle information, determining a first correction coefficient corresponding to the target pixel point; Based on the environmental information and the film source category, the second correction coefficient and the third correction coefficient corresponding to the target pixel are determined, and the correction coefficient set consisting of the first correction coefficient, the second correction coefficient and the third correction coefficient is used as the correction coefficient corresponding to the target pixel.

5. The image processing method according to claim 4, characterized in that: The step of adjusting the pixel value of the target pixel to obtain a candidate pixel based on the correction coefficient and the image block corresponding to the target pixel specifically includes: Determine a candidate pixel value corresponding to the target pixel based on the second correction coefficient, the third correction coefficient and the image block; Based on the first correction coefficient, the pixel value of the target pixel point and the candidate pixel value are weighted to obtain the candidate pixel point corresponding to the target pixel point.

6. The image processing method according to claim 1, characterized in that: The adjustment curve includes an R primary color adjustment curve, a G primary color adjustment curve, and a B primary color adjustment curve; the pixel value of the candidate pixel includes an R primary color value, a G primary color value, and a B primary color value; and adjusting the pixel value of the candidate pixel based on the adjustment curve to obtain a display pixel specifically includes: For each target primary color of the candidate pixel, determine a target primary color value corresponding to the target primary color according to a pixel value of the candidate pixel, and determine a display primary color value corresponding to the target primary color value based on the target primary color value and a target primary color curve corresponding to the target primary color, wherein the target primary color is included in the R primary color, the G primary color, and the B primary color, and the target primary color curve is included in the R primary color adjustment curve, the G primary color adjustment curve, and the B primary color adjustment curve; The determined display primary color values ​​are used as target primary color values ​​of the corresponding target primary colors to obtain display pixel points.

7. The image processing method according to claim 1, characterized in that: The determining, based on all the obtained display pixels, the playback image corresponding to the target image specifically includes: Acquire display restriction conditions corresponding to the target image, wherein the display restriction conditions include display timeliness and display authenticity; Selecting a fusion method corresponding to the target image according to the display restriction condition; According to the fusion method, each display pixel is fused with its corresponding target pixel to obtain a playback image corresponding to the target image.

8. The image processing method according to claim 1, characterized in that: After determining the playback image corresponding to the target image according to all the obtained display pixels, the method further includes: The playback image is displayed through the target device.

9. An image processing device, characterized in that: The image processing device comprises: A first determination module is used to determine state information corresponding to each target pixel point according to posture information of each target pixel point in a target image, and adjust a pixel value of the target pixel point according to the state information to obtain a display pixel point corresponding to the pixel point, wherein the target image is a video frame to be played by a target device; the state information includes viewing angle information, and the viewing angle information is used to reflect a positional relationship between a target user and the target device; The step of adjusting the pixel value of the target pixel point according to the state information to obtain a display pixel point corresponding to the pixel point specifically includes: Determine the adjustment curve and correction coefficient corresponding to the target pixel point based on the state information; Based on the correction coefficient and the image block corresponding to the target pixel point, adjusting the pixel value of the target pixel point to obtain a candidate pixel point, wherein the image block is included in the target image, and the target image includes the target pixel point, and the image block is used to determine the neighborhood of the target pixel point; Adjusting the pixel value of the candidate pixel point based on the adjustment curve to obtain a display pixel point; The second determination module is used to determine the playback image corresponding to the target image according to all the obtained display pixels.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the image processing method according to any one of claims 1 to 8.

11. A terminal device, characterized in that: include: Processor, memory and communication bus; The memory stores a computer-readable program executable by the processor; The communication bus realizes the connection and communication between the processor and the memory; When the processor executes the computer-readable program, the steps in the image processing method according to any one of claims 1 to 8 are implemented.

Images