A mobile beauty optimization method, storage medium, electronic device and system

Abstract

The present invention discloses a mobile end beauty optimization method, storage medium, electronic device and system, and relates to the field of image processing. The method comprises obtaining beauty material and live broadcast screen, and positioning the key points of facial features of the beauty material and the face in the live broadcast screen; performing triangular meshing on the key points of facial features of the beauty material to generate triangular meshes; performing affine transformation based on the key points of facial features and triangular meshes of the face in the beauty material, and the key points of facial features of the face in the live broadcast screen based on a multi-threaded parallel processing method; adjusting the image size of the affine transformation based on the performance of the beauty equipment, and displaying the beauty material on the live broadcast screen; fusing the face of the beauty material with the face of the live broadcast screen to complete the beauty processing of the face in the live broadcast screen. The present invention can effectively ensure the beauty processing effect of the live broadcast screen.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a mobile beauty optimization method, storage medium, electronic device and system. Background Art

[0002] With the rapid development of mobile smart device technology, the functions of smart mobile devices have become more diverse, and watching live broadcasts based on smart mobile devices has become more convenient. As a result, more and more young people like to spend their free time watching live broadcasts and enrich their spare time, which in turn promotes the vigorous development of the live broadcast industry.

[0003] In order to present a perfect self to the audience in the live broadcast room, the anchor will turn on the beauty function during the live broadcast to beautify the face in the live broadcast screen, thereby improving the live viewing experience of the audience in the live broadcast room. However, due to the performance differences of the live broadcast equipment, the beauty function may fail or the beauty effect may be poor during the activation of the beauty function, which in turn affects the live viewing experience of the audience in the live broadcast room. Summary of the invention

[0004] In view of the defects existing in the prior art, the purpose of the present invention is to provide a mobile beauty optimization method, storage medium, electronic device and system, which can effectively ensure the beauty processing effect of live broadcast pictures.

[0005] To achieve the above objectives, the present invention provides a mobile terminal beauty optimization method, comprising the following steps:

[0006] Obtain beauty materials and live broadcast images, and locate the key points of the facial features of the faces in the beauty materials and live broadcast images;

[0007] Perform triangulation on the key points of the facial features of the beauty material to generate a triangular mesh;

[0008] Based on the facial features and triangular mesh of the makeup material, as well as the facial features of the live broadcast screen, affine transformation is performed using multi-threaded parallel processing.

[0009] Adjust the image size of the affine transformation based on the performance of the beauty equipment, and display the beauty material on the live broadcast screen;

[0010] Merge the faces in the beauty materials with the faces in the live broadcast screen to complete the beauty processing of the faces in the live broadcast screen.

[0011] On the basis of the above technical solution, the beauty material face is fused with the live broadcast screen face, and the fusion is specifically performed through the following formula:

[0012] R=(S*(1-a)+D*a)α

[0013] Among them, R represents the result of the fusion of the beauty material face and the live broadcast screen face, S represents the face image in the live broadcast screen, a represents the beauty intensity parameter, D represents the result of the beauty material displayed on the live broadcast screen, and α represents the fusion result adjustment factor.

[0014] On the basis of the above technical solution, the affine transformation is performed based on the key points of facial features and triangular mesh of the makeup material and the key points of facial features of the live broadcast screen in a multi-threaded parallel processing manner. The specific steps include:

[0015] Create a queue for storing triangular meshes and create multiple threads for affine transformation.

[0016] The generated triangular meshes are stored in a queue to form a triangular mesh queue;

[0017] Multiple threads obtain triangular meshes from the queue in turn, and perform affine transformation based on the key points of the facial features of the makeup material faces and the live broadcast screen faces.

[0018] Based on the above technical solution, the multiple threads sequentially obtain triangular meshes from the queue, and the specific steps include:

[0019] In order from the beginning to the end, each thread obtains the triangle mesh from the triangle mesh queue in turn;

[0020] For the triangle mesh obtained by the current thread, determine whether the triangle mesh obtained by the current thread intersects with the triangle mesh that is undergoing affine transformation:

[0021] If yes, the current thread places the obtained triangle mesh at the end of the triangle mesh queue and obtains another triangle mesh from the triangle mesh queue;

[0022] If not, no action will be taken.

[0023] Based on the above technical solutions,

[0024] The determining whether the triangular mesh obtained by the current thread intersects with the triangular mesh undergoing affine transformation, wherein whether the triangular meshes intersect is determined based on whether rectangles containing the triangular meshes intersect;

[0025] When the rectangle where the triangle mesh obtained by the current thread is located intersects with the rectangle where the triangle mesh undergoing affine transformation is located, the triangle mesh obtained by the current thread intersects with the triangle mesh undergoing affine transformation, otherwise, they do not intersect.

[0026] Based on the above technical solution, the steps for determining whether the rectangles intersect are as follows:

[0027] Get rectangle A and rectangle B to be judged whether they intersect, and calculate the distance between the center points of rectangle A and rectangle B in the X-axis direction, as well as the distance between the center points of rectangle A and rectangle B in the Y-axis direction. The calculation formula is:

[0028] L1=abs((x0+x1) / 2–(x2+x3) / 2)β

[0029] L2=abs((y0+y1) / 2–(y2+y3) / 2)β

[0030] Among them, L1 represents the distance between the center points of rectangle A and rectangle B in the X-axis direction, abs represents the operation function for calculating the absolute value, x0 represents the horizontal coordinate of the lower left corner vertex of rectangle A, x1 represents the horizontal coordinate of the lower right corner vertex of rectangle A, x2 represents the horizontal coordinate of the lower left corner vertex of rectangle B, x3 represents the horizontal coordinate of the lower right corner vertex of rectangle B, L2 represents the distance between the center points of rectangle A and rectangle B in the Y-axis direction, y0 represents the vertical coordinate of the lower left corner vertex of rectangle A, y1 represents the vertical coordinate of the upper left corner vertex of rectangle A, y2 represents the vertical coordinate of the lower left corner vertex of rectangle B, y3 represents the vertical coordinate of the upper left corner vertex of rectangle B, and β represents the distance calculation correction factor.

[0031] Calculate the side length of rectangle A in the X-axis direction and the side length of rectangle B in the X-axis direction. The calculation formula is:

[0032] L3=abs(x0-x1)

[0033] L4=abs(x2-x3)

[0034] Wherein, L3 represents the side length of rectangle A in the X-axis direction, and L4 represents the side length of rectangle B in the X-axis direction;

[0035] Calculate the side length of rectangle A in the Y-axis direction and the side length of rectangle B in the Y-axis direction. The calculation formula is:

[0036] L5=abs(y0-y1)

[0037] L6=abs(y2-y3)

[0038] Wherein, L5 represents the side length of rectangle A in the Y-axis direction, and L6 represents the side length of rectangle B in the Y-axis direction;

[0039] Determine whether rectangle A and rectangle B intersect. If and only if L1>(L3+L4) / 2 and L2>(L5+L6) / 2, rectangle A and rectangle B do not intersect. Otherwise, rectangle A and rectangle B intersect.

[0040] On the basis of the above technical solution, the image size of the affine transformation is adjusted based on the performance of the beauty equipment, specifically,

[0041] When performing affine transformation, the time consumed for affine transformation of each frame of the live broadcast screen is counted:

[0042] When the affine transformation time of the consecutive preset number of frames is greater than the preset time, if the affine transformation is not in the low quality mode, the affine transformation is switched to the low quality mode; if the affine transformation is in the low quality mode, no processing is performed;

[0043] When the affine transformation time of the preset number of frames is less than the set time, if the affine transformation is in low quality mode, the affine transformation is switched to high quality mode; if the affine transformation is in high quality mode, no processing is performed;

[0044] The affine transformation includes a low-quality mode and a high-quality mode. The low-quality mode reduces the affine transformed image by a preset multiple, and the high-quality mode enlarges the affine transformed image by a preset multiple.

[0045] The present invention provides a readable storage medium on which a computer program is stored, and the program implements the steps of the above-mentioned method when executed by a processor.

[0046] An electronic device provided by the present invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the steps of the above-mentioned method are implemented when the processor executes the program.

[0047] The present invention provides a mobile terminal beauty optimization system, comprising:

[0048] An acquisition module is used to acquire beauty materials and live broadcast images, and locate the key points of the facial features of the faces in the beauty materials and live broadcast images;

[0049] A generation module is used to perform triangulation on the key points of the facial features of the beauty material and generate a triangulated mesh;

[0050] Affine transformation module, which is used to perform affine transformation based on the key points of facial features and triangular mesh of the makeup material and the key points of facial features of the live broadcast screen based on multi-threaded parallel processing;

[0051] An adjustment module, which is used to adjust the image size of the affine transformation based on the performance of the beauty equipment and display the beauty material on the live broadcast screen;

[0052] The fusion module is used to fuse the faces in the beauty material with the faces in the live broadcast screen to complete the beauty processing of the faces in the live broadcast screen.

[0053] Compared with the prior art, the advantages of the present invention are: when performing affine transformation in the beauty processing process, the image size of the affine transformation is adjusted based on the performance of the beauty equipment to ensure the smooth progress of the affine transformation, meet the beauty requirements of devices with different performances, and effectively ensure the beauty processing effect of the live broadcast picture. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 This is a flow chart of a mobile makeup optimization method according to an embodiment of the present invention;

[0056] Figure 2 Schematic diagram of the structure of a mobile beauty optimization system in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The embodiment of the present invention provides a mobile end beauty optimization method. When performing affine transformation in the beauty processing process, the image size of the affine transformation is adjusted based on the performance of the beauty device to ensure the smooth progress of the affine transformation, meet the beauty requirements of devices with different performances, and effectively ensure the beauty processing effect of the live broadcast screen. The embodiment of the present invention also provides a readable storage medium, an electronic device, and a mobile end beauty optimization system accordingly.

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0059] See also Figure 1 As shown, a mobile makeup optimization method provided by an embodiment of the present invention specifically includes the following steps:

[0060] S1: Obtain beauty materials and live broadcast images, and locate the key points of facial features of the faces in the beauty materials and the live broadcast images.

[0061] Beauty processing in image beautification processing refers to making the face look like a makeup effect without makeup. For a face, the parts that need to be made up are fixed, and the types of makeup effects for the parts are also fixed. Therefore, by displaying the makeup effect image of a certain part of the face on the face and integrating it with the face, the beauty effect processing of a certain part of the face can be achieved.

[0062] Since there are multiple parts of a face that need to be made up, there are also multiple corresponding makeup effect pictures. The multiple makeup effect pictures are arranged according to the face style, and the resulting image is the beauty makeup material. For the face in the live broadcast screen, when making up, multiple beauty makeup styles can be selected for the face in the live broadcast screen, and the corresponding beauty makeup materials are also multiple. The embodiment of the present invention is explained by selecting one beauty makeup style, and the selected beauty makeup material is also one.

[0063] In the embodiment of the present invention, the graphic structure of the beauty makeup material corresponds to the structure of the human face, so the human face also exists in the beauty makeup material, and therefore the key points of the facial features of the human face in the beauty makeup material can be located.

[0064] S2: Triangulate the key points of the facial features of the beauty material to generate a triangular mesh. For an image, it can be split into multiple triangles, that is, triangulate the image. The present invention uses the key points of the facial features of the human face as reference points to triangulate the beauty material, and multiple triangular meshes can be obtained.

[0065] In one possible implementation, in order to reduce the time consumption of the entire beauty processing process, the key points of the facial features of the beauty material can be located and the triangulated mesh subdivision of the key points of the facial features can be performed in advance. The key points of the facial features of the beauty material can be located and the triangulated mesh subdivision of the key points of the facial features can be performed in the server, and then an application package is generated and placed in the mobile device performing the beauty processing. The mobile beauty device can directly call it when performing the beauty processing.

[0066] S3: Based on the key points of facial features and triangular mesh of the makeup material, and the key points of facial features of the live screen, affine transformation is performed based on multi-threaded parallel processing. Before the makeup material and live screen are merged, it is necessary to determine the correspondence between the face of the live screen and the face of the makeup material. Affine transformation is to find the correspondence between the face parts of the live screen and the face parts of the makeup material, and determine the correspondence between each part of the face of the live screen and the triangular mesh of the makeup material.

[0067] S4: Adjust the image size of the affine transformation based on the performance of the beauty equipment, and display the beauty material on the live broadcast screen.

[0068] In the embodiment of the present invention, for an image, the larger it is, the more details it can show, and vice versa, it cannot show more details. The more details, the higher the quality of the final beauty processing effect. The more beauty details, accordingly, if you want to get a high-quality beauty effect, the higher the performance requirements for the hardware device in the beauty processing process. When performing affine transformation, the present invention adjusts the image size of the affine transformation based on the performance of the beauty device. If the performance of the beauty device is good, the image size of the affine transformation is enlarged. If the performance of the beauty device is poor, the image size of the affine transformation is reduced, so that the mobile terminal beauty optimization method of the embodiment of the present invention can run on hardware devices with different performances, effectively ensuring the stable beauty processing of the live screen.

[0069] S5: The makeup material face is integrated with the live broadcast screen face to complete the makeup processing of the face in the live broadcast screen.

[0070] In the embodiment of the present invention, the makeup material face is fused with the live broadcast screen face, and the fusion is specifically performed by the following formula:

[0071] R=(S*(1-a)+D*a)α

[0072] Among them, R represents the result of the fusion of the beauty material face and the live broadcast screen face, S represents the face image in the live broadcast screen, a represents the beauty intensity parameter, D represents the result of the beauty material displayed on the live broadcast screen, and α represents the fusion result adjustment factor. After the fusion of the beauty material face and the live broadcast screen face, the brightness of the image will change, so the fusion result can be further adjusted by the fusion result adjustment factor.

[0073] For the above fusion formula, when there is no fusion result adjustment factor, the result of the fusion of the beauty material face and the live screen face is obtained by adding the adjusted beauty material presentation effect D*a and the adjusted original face image S*(1-a). Since the final fusion effect is obtained by adding the two adjusted images, the brightness and contrast of the image presented by the final fusion result may be poor. Therefore, the value of R is adjusted by the fusion result adjustment factor, that is, the final fusion result is adjusted by the fusion result adjustment factor. When α increases, the value of R increases. At this time, the brightness, contrast and other values ​​of the image presented by the final fusion result increase, improving the display effect of the fused image. On the contrary, when α decreases, the display effect of the fused image decreases.

[0074] The principle of the above formula is explained below:

[0075] In the field of face beautification processing, the beautification intensity is adjustable. The higher the beautification intensity, the stronger the final beautification effect. The lower the beautification intensity, the weaker the final beautification effect. As the beautification effect increases, the original face picture presentation effect weakens, and the face picture presentation effect after beautification increases. Correspondingly, the beauty intensity in the face beauty effect is also adjustable, and the beauty intensity can be controlled by a parameter. The larger the beauty intensity parameter, the stronger the final beauty effect. The present invention merges the beauty material face with the live broadcast picture face, thereby realizing the beauty processing of the face in the live broadcast picture. Therefore, the beauty effect of the present invention is affected by both the original face image in the live broadcast picture and the beauty material.

[0076] That is, in the present invention, the beauty effect is affected by the values ​​of D and S. The present invention controls the value of S by 1-a and controls the value of D by a. When it is necessary to increase the beauty effect, it is necessary to increase the value of D and reduce the value of S. Therefore, when increasing the beauty effect, by increasing the value of a, the value of D*a increases, that is, the effect of the beauty material on the live screen increases, the value of 1-a decreases, and the value of S*(1-a) decreases, that is, the effect of the original face image on the live screen decreases, and the influence of the original face image on the beauty effect is reduced while the influence of the beauty material on the beauty effect is increased, thereby achieving the enhancement of the beauty effect of the face on the live screen presented in the end; when it is necessary to weaken the beauty effect, it is necessary to reduce the value of D and increase the value of S. Therefore, when increasing the beauty effect, by reducing the value of a, the value of D*a decreases, that is, the effect of the beauty material on the live screen decreases, the value of 1-a decreases, and the value of S*(1-a) increases, that is, the effect of the original face image on the live screen is enhanced, and the influence of the original face image on the beauty effect is increased while the influence of the beauty material on the beauty effect is reduced, thereby achieving the weakening of the beauty effect of the face on the live screen presented in the end.

[0077] In the embodiment of the present invention, according to the key points of facial features and triangular mesh of the makeup material and the key points of facial features of the live broadcast screen, affine transformation is performed based on a multi-threaded parallel processing method, and the specific steps include:

[0078] S301: Create a queue for storing triangular meshes, and create multiple threads for performing affine transformation.

[0079] S302: The generated triangular mesh is stored in a queue to form a triangular mesh queue;

[0080] S303: Multiple threads sequentially obtain triangular meshes from the queue, and perform affine transformation based on the key points of the facial features of the makeup material face and the live broadcast screen face.

[0081] When performing an affine transformation, it is necessary to correspond each triangular mesh of the makeup material face to the part of the face in the live broadcast screen. The embodiment of the present invention creates multiple threads, each thread is used to correspond the triangular mesh to the part of the face in the live broadcast screen, and multiple threads are performed simultaneously, thereby effectively improving the processing efficiency of the affine transformation and reducing the time consumption of the entire makeup processing effect. The number of threads is flexibly adjusted according to actual conditions. The triangular mesh is placed in a queue, and multiple threads obtain the triangular mesh from the queue at the same time. When the triangular mesh obtained by the thread is processed, the triangular mesh is obtained from the queue again and processed, and so on. The faster the processing thread, the more triangular meshes it obtains, avoiding the processing method of each thread sharing the task equally.

[0082] Furthermore, multiple threads sequentially obtain triangular meshes from the queue. The specific steps include:

[0083] S3031: each thread sequentially obtains a triangle mesh from the triangle mesh queue in order from the beginning to the end;

[0084] S3032: For the triangular mesh obtained by the current thread, determine whether the triangular mesh obtained by the current thread intersects with the triangular mesh that is undergoing affine transformation:

[0085] If yes, the current thread places the obtained triangle mesh at the end of the triangle mesh queue and obtains another triangle mesh from the triangle mesh queue;

[0086] If not, no action will be taken.

[0087] After using multiple threads to perform affine transformation, the final beauty effect will have a probabilistic flickering phenomenon. The reason for the flickering is that the triangular mesh obtained by the current thread intersects with the triangular mesh undergoing affine transformation. Therefore, in order to solve the flickering problem, after the current thread obtains the triangular mesh, a judgment is made as to whether the triangular mesh obtained by the current thread intersects with the triangular mesh undergoing affine transformation. If so, the triangular mesh obtained by the current thread is placed at the end of the triangular mesh queue, and another triangular mesh is obtained from the triangular mesh queue, and a judgment is made as to whether they intersect here, until the triangular mesh obtained by the current thread does not intersect with the triangular mesh undergoing affine transformation, thereby effectively solving the flickering problem.

[0088] In an embodiment of the present invention, it is determined whether the triangle mesh obtained by the current thread intersects with the triangle mesh that is undergoing affine transformation, wherein whether the triangle meshes intersect is determined based on whether the rectangles where the triangle meshes are located intersect; when the rectangle where the triangle mesh obtained by the current thread is located intersects with the rectangle where the triangle mesh that is undergoing affine transformation is located, the triangle mesh obtained by the current thread intersects with the triangle mesh that is undergoing affine transformation, otherwise, they do not intersect. The rectangle where the triangle mesh is located refers to a rectangle formed by taking one side of the triangle mesh as the bottom edge, taking the bottom edge as the bottom edge of the rectangle, and taking the distance from the bottom edge to the vertex of the triangle mesh as the width of the rectangle, and the triangle mesh is located in the formed rectangle.

[0089] In the embodiment of the present invention, the steps of determining whether the rectangles intersect are as follows:

[0090] (1) Obtain rectangle A and rectangle B to be judged as to whether they intersect, and calculate the distance between the center points of rectangle A and rectangle B in the X-axis direction, as well as the distance between the center points of rectangle A and rectangle B in the Y-axis direction. The calculation formula is:

[0091] L1=abs((x0+x1) / 2–(x2+x3) / 2)β

[0092] L2=abs((y0+y1) / 2–(y2+y3) / 2)β

[0093] Wherein, L1 represents the distance between the center points of rectangle A and rectangle B in the X-axis direction, abs represents the operation function for finding the absolute value, x0 represents the abscissa of the lower left corner vertex of rectangle A, x1 represents the abscissa of the lower right corner vertex of rectangle A, x2 represents the abscissa of the lower left corner vertex of rectangle B, x3 represents the abscissa of the lower right corner vertex of rectangle B, L2 represents the distance between the center points of rectangle A and rectangle B in the Y-axis direction, y0 represents the ordinate of the lower left corner vertex of rectangle A, y1 represents the ordinate of the upper left corner vertex of rectangle A, y2 represents the ordinate of the lower left corner vertex of rectangle B, y3 represents the ordinate of the upper left corner vertex of rectangle B, β represents the distance calculation correction factor, and the distance between the calculated coordinate points is corrected by the distance calculation correction factor to prevent calculation errors. When the distance result between the coordinate points does not need to be corrected, β takes the value of 1.

[0094] Affected by the image resolution, the position coordinate accuracy of pixels in images with different resolutions is different. The higher the resolution, the higher the position coordinate accuracy of pixels, and the lower the resolution, the lower the position coordinate accuracy of pixels. Even in the same image, the resolution of different parts is different. Therefore, when calculating the distance between the center points of rectangle A and rectangle B in the Y-axis direction, if the resolutions of the areas where rectangle A and rectangle B are located are different, the position coordinate accuracy of the corresponding points in rectangle A and rectangle B is also different, resulting in an error in the calculated distance between the center points of rectangle A and rectangle B in the Y-axis direction. Therefore, according to experience, when the calculated distance is smaller than the correct distance due to different resolutions, the value of β is greater than 1, and the smaller the calculated distance is than the correct distance, the larger the value of β is, and the calculated distance value is adjusted so that the calculated distance value is close to the correct distance value; when the calculated distance is greater than the correct distance due to different resolutions, the value of β is less than 1, and the larger the calculated distance is than the correct distance, the smaller the value of β is, and the calculated distance value is adjusted so that the calculated distance value is close to the correct distance value; when the resolutions of the areas where rectangles A and B are located are the same, the calculated distance value is equal to the correct distance value, and there is no calculation error, so the value of β is 1 at this time.

[0095] The principle of the above formula is explained below:

[0096] In a rectangular coordinate system, the difference between the horizontal coordinates of two coordinate points is the distance between the two coordinate points in the X-axis direction. Therefore, in the embodiment of the present invention, the distance between the center point of rectangle A and the center point of rectangle B in the X-axis direction is calculated, that is, the difference between the horizontal coordinate of the center point of rectangle A and the horizontal coordinate of the center point of rectangle B is calculated. A rectangle includes four vertices (upper left vertex, lower left vertex, upper right vertex and lower right vertex). Therefore, the abscissa of the center point of the rectangle can be calculated according to the abscissa of the upper left vertex and the abscissa of the upper right vertex of the rectangle, or the abscissa of the center point of the rectangle can be calculated according to the abscissa of the lower left vertex and the abscissa of the lower right vertex of the rectangle. Therefore, for the abscissa of the center point of rectangle A, it can be calculated based on the abscissa of the lower left vertex of rectangle A and the abscissa of the lower right vertex of rectangle A, and the center point of rectangle A is the center point of the lower left vertex of rectangle A and the lower right vertex of rectangle A. The solution formula for the center point is the sum of the two points divided by 2. Substitute the abscissa of the lower left vertex of rectangle A and the abscissa of the lower right vertex of rectangle A to obtain (x0+x1) / 2, that is, the abscissa of the center point of rectangle A is calculated; correspondingly Therefore, for the horizontal coordinate of the center point of rectangle B, it can be calculated according to the horizontal coordinates of the lower left corner vertex of rectangle B and the lower right corner vertex of rectangle B. Substitute the horizontal coordinates of the lower left corner vertex of rectangle B and the lower right corner vertex of rectangle B into the center point solution formula to obtain (x2+x3) / 2, that is, solve for the horizontal coordinate of the center point of rectangle B, and subtract the horizontal coordinates of the center point of rectangle A from the horizontal coordinates of the center point of rectangle B to obtain (x0+x1) / 2–(x2+x3) / 2. Since the distance is generally a positive value, the calculation result of (x0+x1) / 2–(x2+x3) / 2 needs to be calculated by taking the absolute value, that is, abs((x0+x1) / 2–(x2+x3) / 2), so as to finally obtain the distance between the center point of rectangle A and the center point of rectangle B in the X-axis direction.

[0097] In a rectangular coordinate system, the difference between the ordinates of two coordinate points is the distance between the two coordinate points in the Y-axis direction. Therefore, in the embodiment of the present invention, the distance between the center point of rectangle A and the center point of rectangle B in the Y-axis direction is calculated by calculating the difference between the ordinates of the center point of rectangle A and the center point of rectangle B. Therefore, the ordinate of the center point of the rectangle can be calculated based on the ordinates of the upper left vertex and the lower left vertex of the rectangle, or the ordinate of the center point of the rectangle can be calculated based on the ordinates of the upper right vertex and the lower right vertex of the rectangle. Therefore, for the ordinate of the center point of rectangle A, the ordinate of the upper left vertex of rectangle A and the ordinate of the lower left vertex of rectangle A can be used, and the center point of rectangle A is the center point of the upper left vertex of rectangle A and the lower left vertex of rectangle A. The solution formula for the center point is the sum of the two points divided by 2. Substituting the ordinates of the upper left vertex of rectangle A and the lower left vertex of rectangle A, we get (y0+y1) / 2, that is, the ordinate of the center point of rectangle A is calculated; correspondingly, for the ordinate of the center point of rectangle B, we can According to the ordinates of the upper left corner vertex of rectangle B and the lower left corner vertex of rectangle B, substitute the ordinates of the upper left corner vertex of rectangle B and the lower left corner vertex of rectangle B into the center point solution formula to obtain (y2+y3) / 2), that is, the ordinate of the center point of rectangle B is solved, and the ordinate of the center point of rectangle A is subtracted from the ordinate of the center point of rectangle B to obtain (y0+y1) / 2–(y2+y3) / 2. Since the distance is generally a positive value, the calculation result of (y0+y1) / 2–(y2+y3) / 2 needs to be calculated by taking the absolute value, that is, abs((y0+y1) / 2–(y2+y3) / 2), so as to finally obtain the distance between the center point of rectangle A and the center point of rectangle B in the Y-axis direction.

[0098] (2) Calculate the side length of rectangle A in the X-axis direction and the side length of rectangle B in the X-axis direction. The calculation formula is:

[0099] L3=abs(x0-x1)

[0100] L4=abs(x2-x3)

[0101] Wherein, L3 represents the side length of rectangle A in the X-axis direction, and L4 represents the side length of rectangle B in the X-axis direction;

[0102] The principle of the above formula is explained below:

[0103] For a rectangle, the side length in the x-axis direction is equal to the difference between the horizontal coordinates of the lower left vertex of the rectangle and the horizontal coordinates of the lower right vertex of the rectangle, or the difference between the horizontal coordinates of the upper left vertex of the rectangle and the horizontal coordinates of the upper right vertex of the rectangle. Therefore, the side length of rectangle A in the x-axis direction can be calculated according to the difference between the horizontal coordinates of the lower left vertex of rectangle A and the horizontal coordinates of the lower right vertex of rectangle A, or the difference between the horizontal coordinates of the upper left vertex of rectangle A and the horizontal coordinates of the upper right vertex of rectangle A. In the embodiment of the present invention, the side length of rectangle A in the x-axis direction is solved according to the difference between the horizontal coordinates of the lower left vertex of rectangle A and the horizontal coordinates of the lower right vertex of rectangle A, and x0-x1 is calculated to obtain the difference between the horizontal coordinates of the lower left vertex of rectangle A and the horizontal coordinates of the lower right vertex of rectangle A. Since the side length is generally a positive value, the calculation result of x0-x1 needs to be calculated in absolute value, that is, abs(x0-x1), so as to obtain the side length of rectangle A in the x-axis direction.

[0104] Correspondingly, the side length of rectangle B in the X-axis direction can be calculated based on the difference between the horizontal coordinates of the lower left vertex of rectangle B and the horizontal coordinates of the lower right vertex of rectangle B, or the difference between the horizontal coordinates of the upper left vertex of rectangle B and the horizontal coordinates of the upper right vertex of rectangle B. In the embodiment of the present invention, the side length of rectangle B in the X-axis direction is solved based on the difference between the horizontal coordinates of the lower left vertex of rectangle B and the horizontal coordinates of the lower right vertex of rectangle B, and x2-x3 is calculated to obtain the difference between the horizontal coordinates of the lower left vertex of rectangle B and the horizontal coordinates of the lower right vertex of rectangle B. Since the side length is generally a positive value, the calculation result of x2-x3 needs to be calculated by taking the absolute value, that is, abs(x2-x3), so as to obtain the side length of rectangle B in the X-axis direction.

[0105] (3) Calculate the side length of rectangle A in the Y-axis direction and the side length of rectangle B in the Y-axis direction. The calculation formula is:

[0106] L5=abs(y0-y1)

[0107] L6=abs(y2-y3)

[0108] Wherein, L5 represents the side length of rectangle A in the Y-axis direction, and L6 represents the side length of rectangle B in the Y-axis direction.

[0109] The principle of the above formula is explained in detail below:

[0110] For a rectangle, the side length in the Y-axis direction is equal to the difference between the ordinates of the lower left vertex of the rectangle and the ordinates of the upper left vertex of the rectangle, or the difference between the ordinates of the lower right vertex of the rectangle and the ordinates of the upper right vertex of the rectangle. Therefore, the side length of rectangle A in the Y-axis direction can be calculated according to the difference between the ordinates of the lower left vertex of rectangle A and the ordinates of the upper left vertex of rectangle A, or the difference between the ordinates of the lower right vertex of rectangle A and the ordinates of the upper right vertex of rectangle A. In the embodiment of the present invention, the side length of rectangle A in the Y-axis direction is solved according to the difference between the ordinates of the lower left vertex of rectangle A and the ordinates of the upper left vertex of rectangle A, y2-y3 is calculated, and the difference between the ordinates of the lower left vertex of rectangle A and the ordinates of the upper left vertex of rectangle A is obtained. Since the side length is generally a positive value, the calculation result of y2-y3 needs to be calculated in absolute value, that is, abs(y2-y3), so as to obtain the side length of rectangle A in the Y-axis direction.

[0111] Correspondingly, the side length of rectangle B in the Y-axis direction can be calculated based on the difference between the ordinates of the lower left vertex of rectangle B and the upper left vertex of rectangle B, or the difference between the ordinates of the lower right vertex of rectangle B and the upper right vertex of rectangle B. In the embodiment of the present invention, the side length of rectangle B in the Y-axis direction is solved based on the difference between the ordinates of the lower left vertex of rectangle B and the upper left vertex of rectangle B, and y2-y3 is calculated to obtain the difference between the ordinates of the lower left vertex of rectangle B and the upper left vertex of rectangle B. Since the side length is generally a positive value, the calculation result of y2-y3 needs to be calculated in absolute value, that is, abs(y2-y3), so as to obtain the side length of rectangle B in the Y-axis direction.

[0112] (4) Determine whether rectangle A and rectangle B intersect. If and only if L1>(L3+L4) / 2 and L2>(L5+L6) / 2, rectangle A and rectangle B do not intersect. Otherwise, rectangle A and rectangle B intersect.

[0113] For the judgment of whether rectangle A and rectangle B intersect in the X-axis direction, if rectangle A is located on the left side of rectangle B, and the left side of rectangle A is about to touch but not the right side of rectangle B, then the distance between the center point of rectangle A and the center point of rectangle B is the minimum judgment distance for whether rectangle A and rectangle B intersect in the X-axis direction. Therefore, the judgment of whether rectangle A and rectangle B intersect in the X-axis direction is based on the distance. If it is less than or equal to the distance, rectangle A and rectangle B intersect in the X-axis direction. For the judgment of whether rectangle A and rectangle B intersect in the Y-axis direction, if rectangle A is located below rectangle B, and the upper side of rectangle A is about to touch but not the lower side of rectangle B, then the distance between the center point of rectangle A and the center point of rectangle B is the minimum judgment distance for whether rectangle A and rectangle B intersect in the Y-axis direction. Therefore, the judgment of whether rectangle A and rectangle B intersect in the Y-axis direction is based on the distance. If it is less than or equal to the distance, rectangle A and rectangle B intersect in the Y-axis direction.

[0114] Therefore, when judging whether rectangle A and rectangle B intersect in the X-axis direction, the distance between the center point of rectangle A and the center point of rectangle B is calculated when the left side of rectangle A is about to touch the right side of rectangle B but is not in contact. This distance is equal to 1 / 2 of the side length of rectangle A in the X-axis direction and 1 / 2 of the side length of rectangle B in the X-axis direction, that is, (L3+L4) / 2. When the distance between the center point of rectangle A and the center point of rectangle B in the X-axis direction is less than or equal to (L3+L4) / 2, rectangle A and rectangle B intersect in the X-axis direction.

[0115] When judging whether rectangle A and rectangle B intersect in the Y-axis direction, the distance between the center point of rectangle A and the center point of rectangle B is calculated when the upper side of rectangle A and the lower side of rectangle B are about to touch but not touch. The distance is equal to 1 / 2 of the length of the side of rectangle A in the Y-axis direction and 1 / 2 of the length of the side of rectangle B in the Y-axis direction, that is, (L5+L6) / 2. When the distance between the center point of rectangle A and the center point of rectangle B in the Y-axis direction is less than or equal to (L5+L6) / 2, rectangle A and rectangle B intersect in the Y-axis direction. If rectangle A and rectangle B do not intersect, it is necessary to satisfy that rectangle A and rectangle B do not intersect in the X-axis direction, and rectangle A and rectangle B do not intersect in the Y-axis direction, so it is necessary to satisfy L1>(L3+L4) / 2, and L2>(L5+L6) / 2.

[0116] In the embodiment of the present invention, the image size of the affine transformation is adjusted based on the performance of the beauty equipment, specifically,

[0117] When performing affine transformation, the time consumed for affine transformation of each frame of the live broadcast screen is counted:

[0118] When the affine transformation time of the consecutive preset number of frames is greater than the preset time, if the affine transformation is not in the low quality mode, the affine transformation is switched to the low quality mode; if the affine transformation is in the low quality mode, no processing is performed;

[0119] When the affine transformation time of the consecutive preset number of frames is less than the set time, if the affine transformation is in low quality mode, the affine transformation is switched to high quality mode, and if the affine transformation is in high quality mode, no processing is performed; wherein, the affine transformation includes a low quality mode and a high quality mode, the low quality mode is to reduce the affine transformed image by a preset multiple, and the high quality mode is to enlarge the affine transformed image by a preset multiple. The preset number of frames in the embodiment of the present invention can be 15 frames, and the preset time, the set time and the preset multiple can be flexibly determined according to needs.

[0120] During the beauty makeup process, the size of the face in the beauty makeup material and the size of the face in the live broadcast screen may be different. The present invention performs beauty makeup on the face in the live broadcast screen. Therefore, when the size of the face in the beauty makeup material and the size of the face in the live broadcast screen are different, it is necessary to adjust the size of the face in the beauty makeup material to make it the same as the size of the face in the live broadcast screen. In the affine transformation, for example, for a certain part C above the human eye, part C corresponds to the triangular mesh D in the beauty makeup material, then it is necessary to adjust the size of the triangular mesh D to make it the same as the size of part C.

[0121] The larger the image size, the more details it displays and the greater the performance consumption of the hardware device. Therefore, when the performance of the hardware device is better, the size of the triangular mesh D is increased so that it can display more details. Then, when the triangular mesh D is adjusted to the same size as part C, it can still retain the details and will not discard the details. Therefore, the beauty presentation effect obtained at this time is better; the smaller the image size, the fewer details it displays and the lower the performance consumption of the hardware device. Therefore, when the performance of the hardware device is poor, the size of the triangular mesh D is reduced so that it discards some details. Then, when the triangular mesh D is adjusted to the same size as part C, the details will still be discarded. Therefore, the beauty presentation effect obtained at this time is poor.

[0122] The mobile makeup optimization method of an embodiment of the present invention adjusts the image size of the affine transformation based on the performance of the beauty device when performing affine transformation in the beauty processing process, thereby ensuring the smooth progress of the affine transformation, meeting the beauty requirements of devices with different performances, and effectively ensuring the beauty processing effect of the live broadcast picture.

[0123] An embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the mobile beauty optimization method described in the above embodiment are implemented.

[0124] The embodiment of the present invention also provides a readable storage medium, on which a computer program is stored, and when the program is executed by the processor, the steps of the mobile end beauty optimization method described above are implemented. The storage medium may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0125] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0126] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] See also Figure 2 As shown, a mobile beauty optimization system provided by the present invention includes an acquisition module, a generation module, an affine transformation module, an adjustment module and a fusion module.

[0128] The acquisition module is used to acquire beauty materials and live broadcast images, and to locate the key points of facial features of the beauty materials and the faces in the live broadcast images; the generation module is used to triangulate the key points of facial features of the beauty materials and generate triangular meshes; the affine transformation module is used to perform affine transformation based on the key points of facial features and triangular meshes of the faces in the beauty materials, as well as the key points of facial features of the faces in the live broadcast images, based on a multi-threaded parallel processing method; the adjustment module is used to adjust the image size of the affine transformation based on the performance of the beauty equipment, and display the beauty materials on the live broadcast images; the fusion module is used to fuse the faces of the beauty materials with the faces of the live broadcast images, and complete the beauty processing of the faces in the live broadcast images.

[0129] It should be noted that, in this application, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0130] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

[0131] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

Claims

1. A mobile terminal beauty optimization method, characterized in that: The following steps are involved: Obtain beauty materials and live broadcast images, and locate the key points of the facial features of the faces in the beauty materials and live broadcast images; Perform triangulation on the key points of the facial features of the beauty material to generate a triangular mesh; Based on the facial features and triangular mesh of the makeup material, as well as the facial features of the live broadcast screen, affine transformation is performed using multi-threaded parallel processing. Adjust the image size of the affine transformation based on the performance of the beauty equipment, and display the beauty material on the live broadcast screen; Merge the makeup material face with the live broadcast face to complete the makeup processing of the face in the live broadcast screen; The affine transformation is performed based on the key points of facial features and the triangular mesh of the makeup material and the key points of facial features of the live broadcast screen in a multi-threaded parallel processing manner, and the specific steps include: Create a queue for storing triangular meshes and create multiple threads for affine transformation. The generated triangular meshes are stored in a queue to form a triangular mesh queue; Multiple threads sequentially obtain triangular meshes from the queue and perform affine transformation based on the key points of the facial features of the makeup material and the live broadcast screen. The multiple threads sequentially obtain triangular meshes from the queue, and the specific steps include: In order from the beginning to the end, each thread obtains the triangle mesh from the triangle mesh queue in turn; For the triangle mesh obtained by the current thread, determine whether the triangle mesh obtained by the current thread intersects with the triangle mesh that is undergoing affine transformation: If they intersect, the triangle mesh obtained by the current thread is placed at the end of the triangle mesh queue, and another triangle mesh is obtained from the triangle mesh queue, and the intersection is judged again until the triangle mesh obtained by the current thread does not intersect with the triangle mesh undergoing affine transformation.

2. A mobile terminal beauty optimization method as claimed in claim 1, characterized in that: The beauty material face is fused with the live broadcast screen face, specifically by the following formula: in, It represents the result of merging the makeup material face with the live broadcast face. Represents the face image in the live broadcast screen. Indicates the makeup intensity parameter, Indicates the result of beauty materials being displayed on the live screen. Indicates the fusion result adjustment factor.

3. A mobile terminal beauty optimization method according to claim 1, characterized in that: The determining whether the triangular mesh obtained by the current thread intersects with the triangular mesh undergoing affine transformation, wherein whether the triangular meshes intersect is determined based on whether rectangles containing the triangular meshes intersect; When the rectangle where the triangle mesh obtained by the current thread is located intersects with the rectangle where the triangle mesh undergoing affine transformation is located, the triangle mesh obtained by the current thread intersects with the triangle mesh undergoing affine transformation, otherwise, they do not intersect.

4. A mobile terminal beauty optimization method as claimed in claim 3, characterized in that: The steps to determine whether the rectangles intersect are: Get rectangle A and rectangle B to be judged whether they intersect, and calculate the distance between the center points of rectangle A and rectangle B in the X-axis direction, as well as the distance between the center points of rectangle A and rectangle B in the Y-axis direction. The calculation formula is: L1 = abs( (x0+x1) / 2 – (x2+x3) / 2)β L2 = abs( (y0+y1) / 2 – (y2+y3) / 2)β Wherein, L1 represents the distance between the center points of rectangle A and rectangle B in the X-axis direction, abs represents the operation function for finding the absolute value, x0 represents the abscissa of the lower left corner vertex of rectangle A, x1 represents the abscissa of the lower right corner vertex of rectangle A, x2 represents the abscissa of the lower left corner vertex of rectangle B, x3 represents the abscissa of the lower right corner vertex of rectangle B, L2 represents the distance between the center points of rectangle A and rectangle B in the Y-axis direction, y0 represents the ordinate of the lower left corner vertex of rectangle A, y1 represents the ordinate of the upper left corner vertex of rectangle A, y2 represents the ordinate of the lower left corner vertex of rectangle B, y3 represents the ordinate of the upper left corner vertex of rectangle B, and β represents the distance calculation correction factor; Calculate the side length of rectangle A in the X-axis direction and the side length of rectangle B in the X-axis direction. The calculation formula is: L3 = abs(x0-x1) L4 = abs(x2-x3) Wherein, L3 represents the side length of rectangle A in the X-axis direction, and L4 represents the side length of rectangle B in the X-axis direction; Calculate the side length of rectangle A in the Y-axis direction and the side length of rectangle B in the Y-axis direction. The calculation formula is: L5 = abs(y0-y1) L6 = abs(y2-y3) Wherein, L5 represents the side length of rectangle A in the Y-axis direction, and L6 represents the side length of rectangle B in the Y-axis direction; Determine whether rectangle A and rectangle B intersect. If and only if L1 > (L3 + L4) / 2 and L2 > (L5 + L6) / 2, rectangle A and rectangle B do not intersect. Otherwise, rectangle A and rectangle B intersect.

5. A mobile terminal beauty optimization method as claimed in claim 1, characterized in that: The image size of the affine transformation is adjusted based on the performance of the beauty equipment, specifically, When performing affine transformation, the time consumed for affine transformation of each frame of the live broadcast screen is counted: When the affine transformation time of the consecutive preset number of frames is greater than the preset time, if the affine transformation is not in the low quality mode, the affine transformation is switched to the low quality mode; if the affine transformation is in the low quality mode, no processing is performed; When the affine transformation time of the preset number of frames is less than the set time, if the affine transformation is in low quality mode, the affine transformation is switched to high quality mode; if the affine transformation is in high quality mode, no processing is performed; The affine transformation includes a low-quality mode and a high-quality mode. The low-quality mode reduces the affine transformed image by a preset multiple, and the high-quality mode enlarges the affine transformed image by a preset multiple.

6. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A mobile beauty optimization system, characterized in that: include: An acquisition module is used to acquire beauty materials and live broadcast images, and locate the key points of the facial features of the faces in the beauty materials and live broadcast images; A generation module is used to perform triangulation on the key points of the facial features of the beauty material and generate a triangulated mesh; Affine transformation module, which is used to perform affine transformation based on the key points of facial features and triangular mesh of the makeup material and the key points of facial features of the live broadcast screen based on multi-threaded parallel processing; An adjustment module, which is used to adjust the image size of the affine transformation based on the performance of the beauty equipment and display the beauty material on the live broadcast screen; The fusion module is used to fuse the makeup material face with the live broadcast screen face to complete the makeup processing of the face in the live broadcast screen; The affine transformation is performed based on the key points of facial features and the triangular mesh of the makeup material and the key points of facial features of the live broadcast screen in a multi-threaded parallel processing manner, and the specific steps include: Create a queue for storing triangular meshes and create multiple threads for affine transformation. The generated triangular meshes are stored in a queue to form a triangular mesh queue; Multiple threads sequentially obtain triangular meshes from the queue and perform affine transformation based on the key points of the facial features of the makeup material and the live broadcast screen. The multiple threads sequentially obtain triangular meshes from the queue, and the specific steps include: In order from the beginning to the end, each thread obtains the triangle mesh from the triangle mesh queue in turn; For the triangle mesh obtained by the current thread, determine whether the triangle mesh obtained by the current thread intersects with the triangle mesh that is undergoing affine transformation: If they intersect, the triangle mesh obtained by the current thread is placed at the end of the triangle mesh queue, and another triangle mesh is obtained from the triangle mesh queue, and the intersection is judged again until the triangle mesh obtained by the current thread does not intersect with the triangle mesh undergoing affine transformation.

Images