Eye information processing method, device, electronic device and storage medium

Abstract

The present invention discloses an eye information processing method, device, electronic device, and storage medium, relating to the field of computer technology. A specific embodiment of the method includes: acquiring a corresponding eye image, detecting the position information of preset eye key points, and determining the position information of the eye center point and the position information of the eye contour; obtaining the position information of multiple vertices in a preset vertex shader, and combining the position information of the eye contour to determine each target vertex within the eye contour range; adjusting the position information of each target vertex and the eye center point based on the aspect ratio of the display screen, and calculating the display change rate of each target vertex; calculating the display position of each target vertex based on the display change rate and the adjusted position information of each target vertex, and displaying the eyes based on the display position of each target vertex. This embodiment can solve the problem that the existing technology has a single method for processing eye information and cannot meet the personalized eye display needs of different users.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an eye information processing method, device, electronic device and storage medium. Background Art

[0002] With the development of intelligent technology, intelligent beauty tools are being used more and more widely. For example, intelligent beauty tools are often used in scenarios such as user live broadcasts and taking photos. Intelligent beauty tools can process user information before displaying it. For example, for the user's eyes, the eye information can be processed and then magnified for display, so as to achieve the purpose of rendering the user's eyes larger and improve the user's appearance. In the prior art, in order to achieve the purpose of rendering the user's eyes larger, a preset map is usually obtained, and based on the map, each pixel of the eye is UV offset in the fragment shader and then displayed. However, this method has a single way of processing eye information and cannot meet the personalized needs of different users' eye displays. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide an eye information processing method, device, electronic device and storage medium, which can solve the problem that the existing technology has a single processing method for eye information and cannot meet the personalized eye display needs of different users.

[0004] To achieve the above objective, according to one aspect of an embodiment of the present invention, a method for processing eye information is provided.

[0005] An eye information processing method according to an embodiment of the present invention includes: in response to an eye display instruction, collecting a corresponding eye image, detecting the position information of preset eye key points to determine the position information of the eye center point and the position information of the eye contour; obtaining the position information of multiple vertices in a preset vertex shader, and determining each target vertex within the range of the eye contour in combination with the position information of the eye contour; adjusting the position information of each target vertex and the eye center point based on the aspect ratio of the display screen to call a preset calculation model to calculate the display change rate of each target vertex; calculating the display position of each target vertex based on the display change rate of each target vertex and the adjusted position information, so as to display the eye based on the display position of each target vertex.

[0006] In one embodiment, the calling of a preset calculation model to calculate the display change rate of each target vertex includes:

[0007] Calculating the eye area radius based on the position information of the preset eye key points and the position information of the eye center point;

[0008] Calculating the distance between each target vertex and the eye center based on the adjusted position information of each target vertex and the eye center;

[0009] A preset change intensity coefficient is obtained to calculate, for each target vertex, a display change rate of the target vertex based on the change intensity coefficient, the radius, and the adjusted distance between the target vertex and the eye center point.

[0010] In yet another embodiment, calculating the display change rate of the target vertex based on the change intensity coefficient, the radius, and the adjusted distance between the target vertex and the eye center point includes:

[0011] The square value of the ratio of the distance to the radius is calculated, and the product of the square value and the change intensity coefficient is determined as the display change rate of the target vertex.

[0012] In yet another embodiment, calculating the display position of each target vertex based on the display change rate of each target vertex and the adjusted position information includes:

[0013] The display position of the target vertex is calculated based on the display change rate of the target vertex, with the adjusted position of the eye center point as the center and the extension direction from the adjusted position of the eye center point to the adjusted position of the target vertex as the change direction.

[0014] In yet another embodiment, the location information includes location coordinates, and the location coordinates include a horizontal coordinate and a vertical coordinate;

[0015] The adjusting the position information of each target vertex and the eye center point based on the aspect ratio of the display screen includes:

[0016] Based on the aspect ratio of the display screen, the horizontal coordinates of each of the target vertices and the eye center point are adjusted.

[0017] In yet another embodiment, adjusting the horizontal coordinates of each of the target vertices and the eye center point based on the aspect ratio of the display screen includes:

[0018] Invoking a preset calculation model to calculate a calculation value of the aspect ratio of the display screen;

[0019] The horizontal coordinates of each of the target vertices and the eye center point are adjusted based on the calculated value.

[0020] In yet another embodiment, before obtaining the position information of the preset vertex, the method further includes:

[0021] Obtaining the number of grid rows and grid columns preset in the vertex shader to call a preset triangle mesh construction model, and sequentially constructing the triangle mesh corresponding to the display area in a target order;

[0022] The position interval of the display area is obtained from the vertex shader to calculate the position information of each vertex in the triangle mesh.

[0023] To achieve the above objective, according to another aspect of an embodiment of the present invention, an eye information processing device is provided.

[0024] An eye information processing device according to an embodiment of the present invention includes: a response unit, configured to respond to an eye display instruction, collect a corresponding eye image, and detect position information of preset eye key points to determine position information of an eye center point and position information of an eye contour; a determination unit, configured to obtain position information of a plurality of vertices in a preset vertex shader, and determine a target vertex within the range of the eye contour in combination with the position information of the eye contour; a calculation unit, configured to adjust the position information of each target vertex and the eye center point based on the aspect ratio of the display screen, and calculate the adjusted distance between each target vertex and the eye center point to call a preset calculation model and calculate the display change rate of each target vertex; and a display unit, configured to calculate the display position of each target vertex based on the display change rate and the adjusted position information of each target vertex, so as to display the eye based on the display position of each target vertex.

[0025] In one embodiment, the computing unit is specifically configured to:

[0026] Calculating the eye area radius based on the position information of the preset eye key points and the position information of the eye center point;

[0027] Calculating the distance between each target vertex and the eye center based on the adjusted position information of each target vertex and the eye center;

[0028] A preset change intensity coefficient is obtained to calculate, for each target vertex, a display change rate of the target vertex based on the change intensity coefficient, the radius, and the adjusted distance between the target vertex and the eye center point.

[0029] In yet another embodiment, the computing unit is specifically configured to:

[0030] The square value of the ratio of the distance to the radius is calculated, and the product of the square value and the change intensity coefficient is determined as the display change rate of the target vertex.

[0031] In yet another embodiment, the display unit is specifically configured to:

[0032] The display position of the target vertex is calculated based on the display change rate of the target vertex, with the adjusted position of the eye center point as the center and the extension direction from the adjusted position of the eye center point to the adjusted position of the target vertex as the change direction.

[0033] In yet another embodiment, the location information includes location coordinates, and the location coordinates include a horizontal coordinate and a vertical coordinate;

[0034] The computing unit is specifically configured to:

[0035] Based on the aspect ratio of the display screen, the horizontal coordinates of each of the target vertices and the eye center point are adjusted.

[0036] In yet another embodiment, the computing unit is specifically configured to:

[0037] Invoking a preset calculation model to calculate a calculation value of the aspect ratio of the display screen;

[0038] The horizontal coordinates of each of the target vertices and the eye center point are adjusted based on the calculated value.

[0039] In yet another embodiment, the apparatus further comprises:

[0040] A construction unit, configured to obtain the number of grid rows and grid columns preset in the vertex shader, to call a construction model of a preset triangle grid, and to sequentially construct the triangle grid corresponding to the display area in a target order;

[0041] The calculation unit is further configured to obtain the position interval of the display area from the vertex shader to calculate the position information of each vertex in the triangle mesh.

[0042] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device is provided.

[0043] An electronic device according to an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the eye information processing method provided by the embodiment of the present invention.

[0044] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer-readable medium is provided.

[0045] A computer-readable medium according to an embodiment of the present invention stores a computer program, which, when executed by a processor, implements the eye information processing method provided by the embodiment of the present invention.

[0046] An embodiment of the above invention has the following advantages or beneficial effects: In the embodiment of the present invention, the preset vertex shader includes multiple vertices, which increases the vertex density in the vertex shader. The offset of multiple pixels of the eye can be determined based on each vertex, thereby making the eye rendering process more delicate. Therefore, after responding to the eye display instruction, the position information of the key points of the eye is detected by collecting the eye image, and the position of the eye center point and the eye contour is obtained. Then, the target vertex within the eye contour range can be determined based on the multiple vertices in the vertex shader, so that the eye rendering is performed based on the target vertex; after determining the target vertex, the position information of each target vertex and the eye center point can be adjusted based on the aspect ratio of the display screen to achieve adjustment of the eye display area, and then the display change rate of each target vertex is calculated based on the adjusted position information, so as to calculate the display position of each target vertex when displayed, that is, the UV offset of the target vertex, so as to perform eye rendering based on the display position of each target vertex and perform eye display. Thus, in the embodiment of the present invention, eye rendering is performed based on each target vertex in the vertex shader, and after adjusting the eye contour by the aspect ratio of the display screen, the UV offset of the target vertex is determined. This eliminates the need to rely on textures for eye display, allowing for more fine-grained display of the user's eyes, personalized display of the eyes for different users, and improved performance of the display device. Furthermore, in the embodiment of the present invention, UV offset calculation and eye rendering are performed for each target vertex in the vertex shader, eliminating the need to perform UV offset calculation and eye rendering for each pixel in the fragment shader. This simplifies the eye display process and improves system performance during eye display.

[0047] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present invention.

[0049] Figure 1 is a schematic diagram of a main process of an eye information processing method according to an embodiment of the present invention;

[0050] Figure 2 is a schematic diagram of an eye image according to an embodiment of the present invention;

[0051] Figure 3 is a schematic diagram of a change in the ratio of the horizontal and vertical coordinates of a target vertex according to an embodiment of the present invention;

[0052] Figure 4 is a schematic diagram of changes in the eye area according to an embodiment of the present invention;

[0053] Figure 5is another schematic diagram of an eye image according to an embodiment of the present invention;

[0054] Figure 6 is a schematic diagram of determining an eye area according to an embodiment of the present invention;

[0055] Figure 7 This is a schematic diagram of an effect of eye display according to an embodiment of the present invention;

[0056] Figure 8 A distribution diagram of each vertex constructed according to an embodiment of the present invention;

[0057] Figure 9 is a distribution diagram of a triangular mesh constructed according to an embodiment of the present invention;

[0058] Figure 10 is a schematic diagram of another main process of the eye information processing method according to an embodiment of the present invention;

[0059] Figure 11 is a schematic diagram of main units of an eye information processing device according to an embodiment of the present invention;

[0060] Figure 12 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

[0061] Figure 13 It is a schematic diagram of the structure of a computer system suitable for implementing the embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0063] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0064] An embodiment of the present invention provides an eye information processing system that can be used in scenarios where a user's eyes are displayed, such as when the user is taking photos or live streaming. Therefore, the eye information processing system includes a display screen to implement the display function. Rendering displayed images typically requires vertex shaders and fragment shaders. Therefore, in an embodiment of the present invention, vertex shaders and fragment shaders are provided in the eye information processing system.

[0065] The embodiment of the present invention provides an eye information processing method, which is executed by an eye information processing system, such as Figure 1 As shown, the method includes:

[0066] S101: In response to an eye display instruction, a corresponding eye image is collected, and position information of preset eye key points is detected to determine position information of an eye center point and position information of an eye contour.

[0067] The eye display command can be an external input or triggered by a preset trigger program. Eye display refers to displaying the user's eyes. The eye display command can be triggered independently or in conjunction with the user's facial display command. For example, the eyes are part of the user's face, so when the user triggers the facial display command, the eye display command is equivalent to triggering the eye display command.

[0068] After receiving the eye display instruction, the user's eye image can be collected first. The eye information processing system can include a device with image acquisition function (such as a camera) to directly collect eye images, or it can interact with data with devices with image acquisition function (such as mobile phones and other terminals) to collect eye images through these devices.

[0069] After collecting the eye image, the position information of the preset eye key points can be detected. The detection method is not limited. For example, the position information of the eye key points can be detected by the face key point detection algorithm. Figure 2 FIG. 1 is a schematic diagram showing the result of detecting eye key points in an eye image. The eye key points may be pre-configured, specifically points on the eye that can represent the eye area and contour.

[0070] After obtaining the position information of the key points of the eye, the positions of the eye center point and the eye contour can be determined, that is, the position information of the eye center point and the position information of the eye contour are obtained.

[0071] S102: Acquire position information of a plurality of vertices in a preset vertex shader, and determine target vertices within the range of the eye contour in combination with the position information of the eye contour.

[0072] The vertex shader includes multiple vertices, each of which is assigned corresponding position information. Since the vertex shader is used to offset the positions of various points on the display screen, multiple vertices in the vertex shader will be distributed throughout the display screen. In this step, the target vertices within the eye contour range can be determined based on the position information of multiple vertices and the position information of the eye contour. Each target vertex is within the eye contour range, indicating the area where the eye is displayed. Therefore, in order to change the style of eye display in the embodiment of the present invention, it is necessary to offset the position of the target vertex.

[0073] It should be noted that in the embodiment of the present invention, the position interval of each vertex is set in the vertex shader, and the position of each vertex is within the position interval. Therefore, in order to determine the accuracy of the target vertex, before determining the position information of the eye center point and the eye contour in step S101, it is necessary to normalize the position information of the detected preset eye key points, and then map the normalized position information to the position interval set in the vertex shader. Taking the position information as coordinates as an example, the coordinate value interval of each vertex set in the vertex shader is (-1, 1). Therefore, after the position information of the detected preset eye key points is normalized, it can be mapped to (-1, 1).

[0074] S103: Based on the aspect ratio of the display screen, adjust the position information of each target vertex and the eye center point, calculate the adjusted distance between each target vertex and the eye center point, call a preset calculation model, and calculate the display change rate of each target vertex.

[0075] The display screen represents the screen used to display the eye area. In the embodiment of the present invention, the display screen is typically a portrait display screen, i.e., the width of the display screen is smaller than the height of the display screen. To coordinate the final display effect of the eye area with the display screen, the pixels of the eye area can be compressed based on the screen resolution, that is, the eye display area is squeezed. Therefore, in this step, the position information of each target vertex and the eye center point is adjusted based on the aspect ratio of the display screen to achieve squeezing of the eye display area.

[0076] Specifically, the position information takes the position coordinates as an example. The position coordinates include horizontal and vertical coordinates. In an embodiment of the present invention, the horizontal coordinates of each target vertex and the center point of the eye can be adjusted based on the aspect ratio of the display screen to achieve squeezing of the eye display area.

[0077] For example, the step of adjusting the horizontal coordinates of each target vertex and the eye center point can be performed as follows: calling a preset calculation model to calculate the calculation value of the aspect ratio of the display screen; and adjusting the horizontal coordinates of each target vertex and the eye center point based on the calculation value.

[0078] The calculation model can be set based on demand and used to calculate the aspect ratio of the display screen to obtain a calculated value. For example, the calculation model can be used to calculate the Nth power of the aspect ratio of the display screen, and the Nth power value of the aspect ratio of the display screen is the calculated value of the aspect ratio of the display screen. Wherein, N is an integer greater than 1. The method of adjusting the horizontal coordinates of each target vertex and the center of the eye based on the calculated value can be performed as follows: the product of the calculated value and the horizontal coordinate of each target vertex is determined as the adjusted horizontal coordinate of the target vertex, and the product of the calculated value and the horizontal coordinate of the center of the eye is determined as the adjusted horizontal coordinate of the center of the eye.

[0079] Taking the Nth power of the calculation model for calculating the aspect ratio of the display screen as an example, before executing this step, the ratio of the horizontal and vertical coordinates of each target vertex and the center of the eye is consistent with the aspect ratio of the display screen, so the horizontal coordinate of each target vertex and the center of the eye is multiplied by the aspect ratio of the display screen, which is equivalent to adjusting the ratio of the horizontal and vertical coordinates of the target vertex and the center of the eye to 1, that is, adjusting the eye display area to a circular distribution. However, the effect of displaying the eye display area in a circular shape is not coordinated, so in this step, the horizontal coordinate of each target vertex and the center of the eye is multiplied by the Nth power of the aspect ratio of the display screen, that is, the eye area is compressed on the basis of the circle to form an elliptical distribution. Displaying it in an elliptical distribution can make the eye display effect more coordinated. Taking N as 2 as an example, if Figure 3 The figure shows the change of the ratio of the horizontal and vertical coordinates of the target vertex. Figure 3 (a) is a schematic diagram of the ratio of the horizontal and vertical coordinates of the target vertex before adjustment. Figure 3 (b) is a schematic diagram of the ratio of the horizontal and vertical coordinates after the horizontal coordinate of the target vertex is multiplied by the aspect ratio of the display screen for the first time. Figure 3 (c) is a schematic diagram of the ratio of the horizontal and vertical coordinates after the horizontal coordinate of the target vertex is multiplied by the aspect ratio of the display screen for the second time.

[0080] Taking N as 2 as an example, the aspect ratio of the display screen can be calculated by Formula 1, the horizontal coordinates of each target vertex after adjustment can be calculated based on Formula 2, and the horizontal coordinate of the eye center point after adjustment can be calculated based on Formula 3.

[0081] uAspectRatio=width / height (1)

[0082] curPos.x=CurPos.x*uAspectRatio 2 (2)

[0083] centerPos.x=CenterPos.x*uAspectRatio 2 (3)

[0084] In the above formula, uAspectRatio represents the aspect ratio of the display screen, width represents the width of the display screen, height represents the height of the display screen, CurPos.x represents the horizontal coordinate of the target vertex before adjustment, curPos.x represents the horizontal coordinate of the target vertex after adjustment, CenterPos.x represents the horizontal coordinate of the center point of the eye before adjustment, and centerPos.x represents the horizontal coordinate of the center point of the eye after adjustment.

[0085] In the embodiment of the present invention, adjusting the position information of each target vertex and the eye center point is equivalent to determining the shape distribution of the eye display area. On this basis, the size of the eye display area can also be adjusted. Therefore, in this step, a preset calculation model can be used to calculate the display change rate of each target vertex. The display change rate indicates the degree of change in the position information of the target vertex when it is displayed.

[0086] Specifically, the display change rate calculation method of the target vertex can be performed as follows: based on the preset position information of the eye key point and the position information of the eye center point, the radius of the eye area is calculated; based on the adjusted position information of each target vertex and the eye center point, the distance between each target vertex and the eye center point is calculated; and a preset change intensity coefficient is obtained to calculate the display change rate of the target vertex for each target vertex based on the change intensity coefficient, the radius, and the adjusted distance between the target vertex and the eye center point.

[0087] The position information of the eye key points can be used to determine the positions of various parts of the eye, and then, combined with the position information of the eye center before adjustment in step S103, the eye area radius can be calculated. In this embodiment of the present invention, the distance between the upper eyelid and the eye center can be used to determine the eye area radius. Therefore, the position of the upper eyelid can be determined based on the position information of the eye key points, and the eye area radius can be calculated, representing the distribution of various eye points in the captured eye image. Based on the adjusted position information of each target vertex and the eye center, the distance between each target vertex and the eye center is calculated to represent the distribution of each target vertex relative to the eye center after adjustment. The change intensity coefficient is pre-set and can specifically be input by the user. In this embodiment of the present invention, the display change rate of the target vertex can be calculated based on the initial state of the eye, the backlog state, and user needs, so that the eye display can meet the user's eye display needs based on the initial state.

[0088] For example, the display change rate of the target vertex may be calculated by calculating the square of the ratio of the distance to the radius, and multiplying the square by the change intensity coefficient to determine the display change rate of the target vertex.

[0089] The display change rate of the target vertex is a value between 0 and 1, so the calculation formula can be shown as Formula 4.

[0090]

[0091] In Formula 4, radius represents the radius of the eye area, d represents the distance between the target vertex and the eye center point, strength represents the change strength coefficient, and λ represents the display change rate of the target vertex.

[0092] S104: Calculating the display position of each target vertex based on the display change rate of each target vertex and the adjusted position information, so as to display the display position of each target vertex to the eyes.

[0093] Among them, after obtaining the display change rate of each target vertex, the adjusted position information of the target vertex can be converted based on the display change rate to obtain the display position of the target vertex, and then pixel rendering is performed based on the display position of each target vertex to perform eye display.

[0094] Since the display requirement of the eyes is usually to be magnified, that is, the eye area adjusted in step S103 is magnified, the display position of each target vertex in this step can be calculated as follows: with the adjusted position of the eye center point as the center, and the extension direction from the adjusted position of the eye center point to the adjusted position of the target vertex as the change direction, the display position of the target vertex is calculated based on the display change rate of the target vertex.

[0095] Taking the adjusted position of the eye center point as the center can keep the position of the eye center point unchanged, and taking the extension direction from the adjusted position of the eye center point to the adjusted position of the target vertex as the change direction, the distribution direction of the target vertex relative to the eye center point can be kept unchanged, and then the display position of the target vertex can be calculated based on the display change rate of the target vertex.

[0096] For example, the calculation formula for the display position of the target vertex may be as shown in Formula 5.

[0097] result=centerPos-(1-λ)*(curPos-centerPos) (5)

[0098] In Formula 5, result represents the display position of the target vertex, λ represents the display change rate of the target vertex, centerPos represents the position information of the eye center point after adjustment, and curPos represents the position information of the target vertex after adjustment.

[0099] Through the above process, the eye area can be enlarged to achieve a large eye display effect after eye rendering. Figure 4 The diagram shows the changes in the eye area. Figure 4 Point B is the center point of the eye, curve 1 represents the eye area before executing this step, and curve 2 represents the eye area after executing this step. Figure 5-7 This is a schematic diagram of the eye display in actual scenes, where: Figure 5 is the initial eye image collected in step S101, Figure 6 Schematic diagram of the calculated eye area. Figure 7 This is a schematic diagram of the effect after the eye display in this step.

[0100] In the embodiment of the present invention, eye rendering is performed based on each target vertex in the vertex shader, and after adjusting the eye contour by the aspect ratio of the display screen, the UV offset of the target vertex is determined. There is no need to rely on maps for eye display, and the user's eyes can be displayed in a more fine-grained manner. It can also achieve personalized display of the eyes of different users and improve the performance of the display device.

[0101] It should be noted that in the prior art, the fragment shader can be used to perform offset calculations on each point in the display screen. However, as the resolution of mobile phones is generally 1280*720, traversing each point for calculation will lead to a decrease in overall display performance and an increase in time consumption. In the embodiment of the present invention, a high-density triangular mesh is constructed in the vertex shader, and the position offset of each point in the eye area is calculated using the mesh in the vertex shader, and then the pixel offset is calculated in combination with the pixel difference between the fragment shader to perform eye rendering and display. Specifically, before executing step S102, it may also include: obtaining the preset number of grid rows and grid columns in the vertex shader to call the preset triangular mesh construction model, and sequentially constructing the triangular mesh corresponding to the display area in the target order; obtaining the position interval of the display area from the vertex shader to calculate the position information of each vertex in the triangular mesh.

[0102] Specifically, the number of grid rows and grid columns can be preset in the vertex shader. With M as the column and L as the row, the number of vertex grids is M*N. The total number of vertices in the constructed triangle grid can be calculated. The calculation formula is shown in Formula 6.

[0103] POINTS_VERTEX_NUM=(L*2-2)*3*(M-1) (6)

[0104] In Formula 6, POINTS_VERTEX_NUM represents the total number of vertices in the triangle mesh.

[0105] Specifically, building a model can be achieved through a mesh building code, and the target order can be set based on demand. In the embodiment of the present invention, the target order is counterclockwise as an example, that is, a triangular mesh is automatically constructed according to the rule of constructing a triangular mesh counterclockwise, and the vertex index of each triangular mesh is output. The POINTS_VERTEX data structure gradually increases the vertex index. Through a loop iteration, the obtained vertex index order can be stored in POINTS_VERTEX. If the mesh is set to M=5 and L=3, the vertex index order of the triangle is: 0,1,3; 1,4,3; 1,2,4; 2,5,4; 3,4,6; 4,7,6; 4,5,7; 5,8,7; 6,7,9; 7,10,9; 7,810; 8,11,10; 9,10,12; 10,13,12; 10,11,13; 11,14,13. The above index order is the mesh vertex order automatically constructed by the loop algorithm. To increase the mesh density, you only need to modify the values of M and L.

[0106] Taking position information as coordinates as an example, the vertex shader includes the position information of the four outermost vertices of the display area, which can be divided into (-1, -1), (-1, 1), (1, -1), (1, 1), so that the position interval of each vertex can be determined, and then the vertex coordinate value can be determined for each vertex. The main calculation formula is: Where x represents the horizontal coordinate of the vertex, y represents the vertical coordinate of the vertex, L is the number of grid rows, M is the number of grid columns, p and q are temporary variables when traversing M and L respectively. M*L points are distributed geometrically between -1≤x≤1 and -1≤y≤1, and the coordinates of each vertex are automatically generated.

[0107] like Figure 8 As shown in the figure, the distribution diagram of each vertex is constructed by the above method, and each vertex is drawn by the coordinates of each vertex. Figure 9 The figure shows the distribution diagram of the triangle mesh constructed by the above method. The above process can automatically realize the construction of the triangle mesh in the vertex shader, and then determine the position information of each vertex, eliminating the need for manual implementation, avoiding wasted time and improving the efficiency of mesh construction.

[0108] The following combination Figure 1 The embodiment shown in FIG. 1 is used to specifically illustrate the eye information processing method in the embodiment of the present invention. Figure 10 As shown, the method includes:

[0109] S1001: In response to an eye display instruction, a corresponding eye image is collected, and position information of preset eye key points is detected to determine the position information of the eye center point, the position information of the eye contour, and the eye area radius.

[0110] S1002: Acquire position information of a plurality of vertices in a preset vertex shader, and determine target vertices within the range of the eye contour in combination with the position information of the eye contour.

[0111] S1003: Acquire the aspect ratio of the display screen, and adjust the horizontal coordinates of each target vertex and the eye center point based on the aspect ratio of the display screen.

[0112] S1004: Calculate the distance between each target vertex and the eye center point based on the adjusted position information of each target vertex and the eye center point.

[0113] S1005: Obtain a preset change intensity coefficient to calculate the square value of the ratio between the distance and the radius for each target vertex, and multiply the square value by the change intensity coefficient to determine the display change rate of the target vertex.

[0114] S1006: Calculate the display position of each target vertex based on the display change rate of each target vertex and the adjusted position information.

[0115] S1007: Displaying the eyes based on the display position of each target vertex.

[0116] It should be noted that the data processing principle of the embodiment of the present invention is the same as Figure 1 The corresponding data processing principles of the illustrated embodiment are the same and will not be described again.

[0117] In the embodiment of the present invention, eye rendering is performed based on each target vertex in the vertex shader, and after adjusting the eye contour by the aspect ratio of the display screen, the UV offset of the target vertex is determined. There is no need to rely on maps for eye display, and the user's eyes can be displayed in a more fine-grained manner. It can also achieve personalized display of the eyes of different users and improve the performance of the display device.

[0118] In order to solve the problems existing in the prior art, an embodiment of the present invention provides an eye information processing device 1100, such as Figure 11 As shown, the apparatus 1100 includes:

[0119] The response unit 1101 is configured to respond to the eye display instruction, collect the corresponding eye image, detect the position information of the preset eye key points, and determine the position information of the eye center point and the position information of the eye contour;

[0120] A determining unit 1102 is configured to obtain position information of a plurality of vertices in a preset vertex shader, and determine a target vertex within the eye contour range in combination with the position information of the eye contour;

[0121] a calculation unit 1103 configured to adjust the position information of each target vertex and the eye center based on the aspect ratio of the display screen, calculate the adjusted distance between each target vertex and the eye center, and invoke a preset calculation model to calculate a display change rate of each target vertex;

[0122] The display unit 1104 is configured to calculate the display position of each target vertex based on the display change rate of each target vertex and the adjusted position information, so as to display the eye based on the display position of each target vertex.

[0123] It should be understood that the manner in which the embodiments of the present invention are implemented is different from the manner in which the embodiments of the present invention are implemented. Figure 1 The methods of the embodiments shown are the same and will not be described again here.

[0124] In one embodiment, the computing unit 1103 is specifically configured to:

[0125] Calculating the eye area radius based on the position information of the preset eye key points and the position information of the eye center point;

[0126] Calculating the distance between each target vertex and the eye center based on the adjusted position information of each target vertex and the eye center;

[0127] A preset change intensity coefficient is obtained to calculate, for each target vertex, a display change rate of the target vertex based on the change intensity coefficient, the radius, and the adjusted distance between the target vertex and the eye center point.

[0128] In yet another embodiment, the computing unit 1103 is specifically configured to:

[0129] The square value of the ratio of the distance to the radius is calculated, and the product of the square value and the change intensity coefficient is determined as the display change rate of the target vertex.

[0130] In yet another embodiment, the display unit 1104 is specifically configured to:

[0131] The display position of the target vertex is calculated based on the display change rate of the target vertex, with the adjusted position of the eye center point as the center and the extension direction from the adjusted position of the eye center point to the adjusted position of the target vertex as the change direction.

[0132] In yet another embodiment, the location information includes location coordinates, and the location coordinates include a horizontal coordinate and a vertical coordinate;

[0133] The calculation unit 1103 is specifically configured to:

[0134] Based on the aspect ratio of the display screen, the horizontal coordinates of each of the target vertices and the eye center point are adjusted.

[0135] In yet another embodiment, the computing unit 1103 is specifically configured to:

[0136] Invoking a preset calculation model to calculate a calculation value of the aspect ratio of the display screen;

[0137] The device 1100 further comprises: adjusting the horizontal coordinates of each of the target vertices and the eye center based on the calculated value.

[0138] A construction unit, configured to obtain the number of grid rows and grid columns preset in the vertex shader, to call a construction model of a preset triangle grid, and to sequentially construct the triangle grid corresponding to the display area in a target order;

[0139] The calculation unit 1103 is further configured to obtain the position interval of the display area from the vertex shader to calculate the position information of each vertex in the triangle mesh.

[0140] It should be understood that the manner in which the embodiments of the present invention are implemented is different from the manner in which the embodiments of the present invention are implemented. Figure 1 Figure 10 The methods of the embodiments shown are the same and will not be described again here.

[0141] In the embodiment of the present invention, eye rendering is performed based on each target vertex in the vertex shader, and after adjusting the eye contour by the aspect ratio of the display screen, the UV offset of the target vertex is determined. There is no need to rely on maps for eye display, and the user's eyes can be displayed in a more fine-grained manner. It can also achieve personalized display of the eyes of different users and improve the performance of the display device.

[0142] According to an embodiment of the present invention, an electronic device and a readable storage medium are further provided.

[0143] An electronic device according to an embodiment of the present invention includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the eye information processing method provided by the embodiment of the present invention.

[0144] Figure 12 An exemplary system architecture 1200 is shown, to which an eye information processing method or an eye information processing apparatus according to an embodiment of the present invention may be applied.

[0145] like Figure 12As shown, system architecture 1200 may include terminal devices 1201, 1202, and 1203, a network 1204, and a server 1205. Network 1204 is used to provide a medium for communication links between terminal devices 1201, 1202, and 1203 and server 1205. Network 1204 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0146] Users can use terminal devices 1201, 1202, and 1203 to interact with server 1205 via network 1204 to receive or send messages, etc. Various client applications can be installed on terminal devices 1201, 1202, and 1203.

[0147] The terminal devices 1201 , 1202 , and 1203 may be, but are not limited to, smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0148] The server 1205 may be a server that provides various services. The server may analyze and process received data such as product information query requests, and feed back processing results (such as product information—just an example) to the terminal device.

[0149] It should be noted that the eye information processing method provided in the embodiment of the present invention is generally executed by the server 1205 , and accordingly, the eye information processing device is generally set in the server 1205 .

[0150] It should be understood that Figure 12 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0151] Reference below Figure 13 , which shows a schematic structural diagram of a computer system 1300 suitable for implementing an embodiment of the present invention. Figure 13 The computer system shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0152] like Figure 13 As shown, the computer system 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage unit 1308 into a random access memory (RAM) 1303. Various programs and data required for the operation of the system 1300 are also stored in the RAM 1303. The CPU 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0153] The following components are connected to the I / O interface 1305: an input section 1306 including a keyboard, a mouse, and the like; an output section 1307 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and speakers; a storage section 1308 including a hard disk; and a communication section 1309 including a network interface card such as a LAN card or a modem. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as needed. Removable media 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1310 as needed, so that computer programs read therefrom can be installed into the storage section 1308 as needed.

[0154] In particular, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1309, and / or installed from a removable medium 1311. When the computer program is executed by the central processing unit (CPU) 1301, the above-mentioned functions defined in the system of the present invention are performed.

[0155] It should be noted that the computer-readable medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0156] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a unit, program segment, or a part of code, and the above-mentioned unit, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0157] The units involved in the embodiments of the present invention may be implemented in software or hardware. The units described may also be provided in a processor. For example, it may be described as follows: a processor includes a response unit, a determination unit, a calculation unit, and a display unit. The names of these units do not, in some cases, constitute limitations on the units themselves. For example, the response unit may also be described as a "unit for command response function."

[0158] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiments, or may exist independently and not incorporated into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to perform the eye information processing method provided by the present invention.

[0159] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for processing eye information, characterized in that: include: In response to the eye display instruction, the corresponding eye image is collected, and the position information of the preset eye key points is detected to determine the position information of the eye center point and the position information of the eye contour; Obtaining position information of a plurality of vertices in a preset vertex shader, and combining the position information of the eye contour with the position information of the eye contour to determine each target vertex within the range of the eye contour; Based on the aspect ratio of the display screen, adjusting the position information of each target vertex and the eye center point to call a preset calculation model to calculate the display change rate of each target vertex; Calculating the display position of each target vertex based on the display change rate of each target vertex and the adjusted position information, so as to display the eye based on the display position of each target vertex; The calling of a preset calculation model to calculate the display change rate of each target vertex includes: Calculating the eye area radius based on the position information of the preset eye key points and the position information of the eye center point; Calculating the distance between each target vertex and the eye center based on the adjusted position information of each target vertex and the eye center; A preset change intensity coefficient is obtained to calculate the square value of the ratio between the distance and the radius for each target vertex, and the product of the square value and the change intensity coefficient is determined as the display change rate of the target vertex.

2. The method according to claim 1, characterized in that Calculating the display position of each target vertex based on the display change rate of each target vertex and the adjusted position information includes: The display position of the target vertex is calculated based on the display change rate of the target vertex, with the adjusted position of the eye center point as the center and the extension direction from the adjusted position of the eye center point to the adjusted position of the target vertex as the change direction.

3. The method according to claim 1, characterized in that The position information includes position coordinates, and the position coordinates include a horizontal coordinate and a vertical coordinate; The adjusting the position information of each target vertex and the eye center point based on the aspect ratio of the display screen includes: Based on the aspect ratio of the display screen, the horizontal coordinates of each of the target vertices and the eye center point are adjusted.

4. The method according to claim 3, characterized in that Adjusting the horizontal coordinates of each of the target vertices and the eye center point based on the aspect ratio of the display screen includes: Invoking a preset calculation model to calculate a calculation value of the aspect ratio of the display screen; The horizontal coordinates of each of the target vertices and the eye center point are adjusted based on the calculated value.

5. The method according to claim 1, wherein Before obtaining the position information of the preset vertex, the method further includes: Obtaining the number of grid rows and grid columns preset in the vertex shader to call a preset triangle mesh construction model, and sequentially constructing the triangle mesh corresponding to the display area in a target order; The position interval of the display area is obtained from the vertex shader to calculate the position information of each vertex in the triangle mesh.

6. An eye information processing device, characterized in that: include: A response unit, configured to respond to an eye display instruction, collect a corresponding eye image, detect position information of preset eye key points, and determine position information of an eye center point and position information of an eye contour; a determining unit, configured to obtain position information of a plurality of vertices in a preset vertex shader, and determine a target vertex within the range of the eye contour in combination with the position information of the eye contour; a calculation unit, configured to adjust the position information of each target vertex and the eye center based on the aspect ratio of the display screen, calculate the adjusted distance between each target vertex and the eye center, and invoke a preset calculation model to calculate the display change rate of each target vertex; a display unit, configured to calculate a display position of each target vertex based on a display change rate of each target vertex and the adjusted position information, so as to display the eye based on the display position of each target vertex; The computing unit is specifically configured to: Calculating the eye area radius based on the position information of the preset eye key points and the position information of the eye center point; Calculating the distance between each target vertex and the eye center based on the adjusted position information of each target vertex and the eye center; A preset change intensity coefficient is obtained to calculate the square value of the ratio between the distance and the radius for each target vertex, and the product of the square value and the change intensity coefficient is determined as the display change rate of the target vertex.

7. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images