Image processing system and image processing method

Abstract

The invention discloses an image processing system and an image processing method. The image processing method comprises the following steps: receiving an image by the object detection module, and detecting the image based on an object detection algorithm; in response to the detected human faces, obtaining the range of each human face based on the shape of the polygon; a weight calculation module is used for executing the steps of obtaining the weight of a current pixel based on the coordinate information of the current pixel in the pixels and the position information of each human face on the pixels of the area covered by the range of all the human faces; and in response to the fact that there is an unselected pixel in the pixels, selecting the unselected pixel in the pixels as the current pixel, and returning to the step of obtaining the weight of the current pixel.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a technology for adjusting images based on face regions. Background Art

[0002] Figure 1 This is a diagram of a face image. Please refer to Figure 1 When the existing technology is used to enhance the color and brightness of the HSI color gamut of the face (face 101 and face 102) in image 100, the transition area from the center of the face to the surrounding area is often not smooth. Especially when multiple faces overlap, obvious color and brightness discontinuities are easily observed. When the image is played on a TV, obvious linear cutting lines (such as cutting line 103) will appear at the edges of the face, and obvious brightness and color discontinuities will appear on both sides of the cutting line. Summary of the Invention

[0003] In view of this, some embodiments of the present invention provide an image processing system and an image processing method to improve the problems of the prior art.

[0004] One embodiment of the present invention provides an image processing system, comprising an object detection module and a weight calculation module; the object detection module is configured to perform: (a1) receiving an image and detecting the image based on an object detection algorithm; and (a2) in response to detecting at least one face, obtaining the range of each face based on the shape of a polygon; and the weight calculation module is configured to perform, for multiple pixels in an area covered by the range of all faces: (b1) obtaining the weight of the current pixel based on the coordinate information of the current pixel among the aforementioned pixels and the position information of each face; and (b2) in response to there being an unselected pixel among the pixels, selecting one of the unselected pixels as the current pixel and returning to step (b1).

[0005] An embodiment of the present invention provides an image processing method, comprising: (a) an object detection module executing: (a1) receiving an image and detecting the image based on an object detection algorithm; and (a2) in response to detecting at least one face, obtaining the range of each face based on the shape of a polygon; (b) a weight calculation module executing, on multiple pixels in an area covered by the range of all faces: (b1) obtaining the weight of the current pixel based on the coordinate information of the current pixel among the aforementioned pixels and the position information of each face; and (b2) in response to there being an unselected pixel among the pixels, selecting one of the unselected pixels as the current pixel and returning to step (b1).

[0006] Based on the above, the image processing system and image processing method provided by some embodiments of the present invention can effectively reduce the area of ​​non-face areas and the area of ​​face overlapping areas by using polygonal shapes to obtain the range of each face, thereby alleviating the side effects brought by the non-face parts marked as face areas; by obtaining the weight of the corresponding pixel based on the coordinate information of each pixel in the area covered by the range of all human faces and the position information of each face, the subsequent stage can be provided with brightness and chromaticity adjustment based on the weight of the corresponding pixel. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 It is a schematic diagram of a face image.

[0008] Figure 2 is a block diagram of an image processing system according to some embodiments of the present invention.

[0009] Figure 3 FIG. 4 is a schematic diagram of the operation of an object detection module according to some embodiments of the present invention.

[0010] Figure 4 FIG. 1 is a schematic diagram of generating a face range according to some embodiments of the present invention.

[0011] Figures 5A to 5D is a schematic diagram illustrating the operation of an image processing system according to some embodiments of the present invention.

[0012] Figure 6 is a block diagram of a display system according to some embodiments of the present invention.

[0013] Figure 7 is a schematic structural diagram of an electronic device according to some embodiments of the present invention.

[0014] Figure 8 is a flow chart of an image processing method according to some embodiments of the present invention.

[0015] Figure 9 FIG. 4 is a flowchart of generating a face range according to some embodiments of the present invention.

[0016] Figure 10 is a flow chart of an image processing method according to some embodiments of the present invention.

[0017] Figure 11 is a flow chart of an image processing method according to some embodiments of the present invention.

[0018] Figure 12 is a flow chart of an image processing method according to some embodiments of the present invention.

[0019] Figure 13is a flow chart of an image processing method according to some embodiments of the present invention.

[0020] Figure 14 is a flow chart of an image processing method according to some embodiments of the present invention.

[0021] Figure 15 is a flow chart of an image processing method according to some embodiments of the present invention.

[0022] Figure 16 is a flow chart of an image processing method according to some embodiments of the present invention.

[0023] Explanation of symbols

[0024] 100,203,300: Image

[0025] 101, 102, 503: Face

[0026] 103: Cutting Line

[0027] 200: Image processing system

[0028] 201: Object detection module

[0029] 202: Weight calculation module

[0030] 301, 302, 400: rectangular frame

[0031] 401, 402, 403, 404: straight line

[0032] 4001: Top

[0033] 4002: Below

[0034] 4003-4006: Triangle

[0035] m1,m2,m3,m4: slope

[0036] d1, d2, d3, d4: intercepts

[0037] 405,501,502,504: Octagonal box

[0038] 505,506: Range

[0039] 600: Display system

[0040] 601: Main chip

[0041] 6011: Drive system

[0042] 602: High-definition multimedia interface signal source

[0043] 603: USB signal source

[0044] 604: Display panel

[0045] 700: Electronic devices

[0046] 701: Processing unit

[0047] 702: Internal memory

[0048] 703: Non-volatile memory

[0049] S801~S808,S901~S905,S1001,S1101~S1107,S1201~S1206,S1301~S1305,S1401,S1501,S1601: Steps DETAILED DESCRIPTION

[0050] The foregoing and other technical contents, features, and technical effects of the present invention will be clearly presented in the following detailed description of the embodiments with reference to the accompanying drawings. Anything that does not affect the technical effects and objectives that can be achieved by the present invention shall still fall within the scope of the technical contents disclosed in the present invention.

[0051] Figure 2 is a block diagram of an image processing system according to some embodiments of the present invention. Figure 5A FIG is a schematic diagram of an image processing system according to some embodiments of the present invention. Figure 2 、 Figure 5A Image processing system 200 includes an object detection module 201 and a weight calculation module 202. Object detection module 201 is configured to receive image 203 and detect image 203 based on an object detection algorithm. The object detection algorithm may be, for example, Faster R-CNN, SSD, or YOLO. When faces are detected in image 203, object detection module 201 is configured to determine the extent of each face based on a polygonal shape.

[0052] by Figure 5A For example, in some embodiments, the polygon is an octagon. Object detection module 201 detects face 101 and face 102 in image 300. In response to detecting face 101 and face 102, object detection module 201 obtains the ranges of face 101 and face 102 based on the shape of the octagon, where the ranges of face 101 and face 102 are the ranges contained within octagonal frame 501 and octagonal frame 502, respectively. Weight calculation module 202 is configured to calculate a weight for each pixel included in the area covered by the ranges of all detected faces (e.g., the range of face 101 and the range of face 102).

[0053] The following describes in detail the image processing methods of some embodiments of the present invention and how the modules of the image processing system 200 operate in coordination with each other with reference to the accompanying drawings.

[0054] Figure 8 is a flow chart of an image processing method according to some embodiments of the present invention. Figure 2 、 Figure 5A as well as Figure 8 ,exist Figure 8 In an embodiment of the present invention, the image processing method includes steps S801 to S808. In step S801, the object detection module 201 receives an image (e.g., image 203) and detects the aforementioned image based on an object detection algorithm to detect whether there is a face in the image. In step S802, if the object detection module 201 determines that no face is detected, it enters step S803 to exit the current program to process the next image received by the object detection module 201. If the object detection module 201 detects at least one face in the image, it enters step S804. In step S804, in response to detecting at least one face, the object detection module 201 generates a polygon (e.g., the aforementioned face) based on the polygon. Figure 5A In step S805, for each unselected current pixel in the area covered by all face ranges, the weight calculation module 202 calculates the weight of the current pixel based on the coordinate information of the current pixel and the position information of each face.

[0055] In step S806, the weight calculation module 202 determines whether there are any unselected pixels in the area covered by the range of all faces that have not been selected for weight calculation. If so, the process proceeds to step S808. If not, the process proceeds to step S807 to exit the current process so that the remaining modules can use the weights of the pixels in the area covered by the range of all faces to process the pixels in the area covered by the range of all faces (for example, perform brightness adjustment or chromaticity adjustment). In step S808, in response to the presence of unselected pixels in the area covered by the range of all faces, the weight calculation module 202 selects one of the unselected pixels in the area covered by the range of all faces as the current pixel and returns to step S805 to execute steps S805 to S808 again.

[0056] Figure 3 FIG. 4 is a schematic diagram of the operation of an object detection module according to some embodiments of the present invention. Figure 4 FIG. 1 is a schematic diagram of generating a face range according to some embodiments of the present invention. Figure 9 FIG is a flowchart of generating a face range according to some embodiments of the present invention. Figures 2 to 4 and Figure 9 .exist Figure 9 In the embodiment of the present invention, the polygon is an octagon, and the image processing method includes executing steps S901 to S905 by the object detection module 201. In step S901, a rectangular frame of each face in the received image is obtained based on the object detection algorithm. Figure 3 For example, the object detection module 201 detects the face 101 and the face 102 in the image 300 , and obtains the rectangular frame 301 of the face 101 and the rectangular frame 302 of the face 102 based on the object detection algorithm.

[0057] After step S901, the object detection module 201 sequentially processes the rectangular frames of the detected faces to obtain the range of each face. In step S902, based on the four intercepts and four slopes corresponding to a current face among the at least one detected face, the four triangles at the four corners of the rectangular frame of the current face are deleted to obtain an octagonal frame. The object detection module 201 uses the range contained in the octagonal frame as the range of the current face. In step S903, it is determined whether there are any unselected faces among the at least one detected face. If not, the process proceeds to step S905 and exits the current process. If so, the process proceeds to step 904. In step 904, in response to the fact that there are unselected faces among the at least one detected face, one of the unselected faces is selected as the current face and the process returns to step S902 to execute steps S902 to S905 again.

[0058] by Figure 4 For example, the rectangular frame of the current face obtained by object detection module 201 is rectangular frame 400 (rectangular frame 400 can be rectangular frame 301 or rectangular frame 302). The four intercepts corresponding to the current face are d1, d2, d3, and d4, and the four slopes are m1, m2, m3, and m4. Based on intercept d1 and slope m1, object detection module 201 can obtain line 401 on the plane of image 300. Similarly, based on intercepts d2, d3, and d4 and the corresponding slopes m2, m3, and m4, object detection module 201 can obtain lines 402-404 on the plane of image 300. Based on the intersections of lines 401-404 and rectangular frame 400, object detection module 201 can obtain four triangles at the four corners of rectangular frame 400 of the current face: triangles 4003-4006. The object detection module 201 deletes the four triangles 4003 to 4006 at the four corners of the rectangular frame 400 of the current face to obtain an octagonal frame 405. Figure 5A As shown, after processing the rectangular frame 301 and the rectangular frame 302 , the octagonal frame 501 and the octagonal frame 502 can be obtained.

[0059] In some embodiments of the present invention, the object detection module 201 obtains the intercepts d1, d2, d3, and d4 based on the four ratios r1, r2, r3, and r4 and the length d of one side (the upper side 4001) of the rectangular frame 400 of the current face, where d1 = r1d, d2 = r2d, d3 = r3d, and d4 = r4d.

[0060] In some embodiments of the present invention, the object detection module 201 multiplies the length d of the upper and lower sides (upper side 4001 and lower side 4002) of the rectangular frame 400 by a predetermined ratio r (the length of the upper side 4001 and the length of the lower side 4002 are the same) to obtain the intercepts d1, d2, d3, and d4 (where d1=d2=d3=d4=rd). The object detection module 201 sets the slopes m1, m2, m3, and m4 with a predetermined slope absolute value. For example, the predetermined slope absolute value is m, since in Figure 4 In the illustrated system, the slope of line 401 is negative, so object detection module 201 sets m1 = -m. The slope of line 404 is positive, so object detection module 201 sets m4 = m. For the same reason, object detection module 201 sets m2 = m and m3 = -m. In some embodiments of the present invention, the preset ratio r is 0.25, and the preset slope absolute value m is 0.25.

[0061] It is worth noting that based on the above process, other polygonal frames can also be obtained. The present invention is not limited to using an octagonal frame to frame the face. The user can set an appropriate polygon according to the use requirements.

[0062] Figure 10 is a flow chart of an image processing method according to some embodiments of the present invention. Figure 8 as well as Figure 10 ,exist Figure 10 In an embodiment, the coordinate information of the current pixel includes uv coordinate information, and the position information of each face includes uv coordinate information of the uv center and uv range information, wherein the uv center of the face is the center of the face under the uv coordinate, and the uv range information includes a uv range value, which represents the range of the face under the uv coordinate system. The aforementioned uv coordinates are a texture coordinate system. Since the texture coordinates are a two-dimensional coordinate system, each point in the texture coordinate system includes a first coordinate value and a second coordinate value. The uv coordinate information included in the coordinate information of the current pixel is the texture coordinate information of the current pixel. In this embodiment, when the object detection module 201 detects at least one face in the image, the object detection module 201 also detects the uv coordinate information and uv range information of the uv center of each face based on the object detection algorithm.

[0063] Step S805 includes the weight calculation module 202 executing step S1001. In step S1001, based on the UV coordinate information of the UV center of each face and the UV coordinate information of the current pixel, the UV distance from the UV center of each face to the current pixel is calculated to obtain the UV distance from the UV center of each face to the current pixel. Finally, based on the UV distances of all faces, the weight of the current pixel is obtained.

[0064] Figure 11 is a flow chart of an image processing method according to some embodiments of the present invention. Figure 8 、 Figures 10 and 11 , take over Figure 10 In an embodiment, when the object detection module 201 detects at least one face in the image, the object detection module 201 also detects the UV range information of each face based on the object detection algorithm and stores it in a memory for subsequent use. That is, in this embodiment, after the object detection module 201 detects the UV range information of each face based on the object detection algorithm, a UV range information is stored in the memory for each detected face. In this embodiment, step S1001 includes the weight calculation module 202 executing steps S1101 to S1107. In step S1101, based on the UV coordinate information of the UV center of a current face among the faces and the UV coordinate information of the current pixel, the UV distance from the UV center of the current face to the current pixel is calculated.

[0065] In step S1102, it is determined whether the UV distance from the current pixel to the UV center of the current face is greater than the UV range value in the UV range information of the current face. If so, step S1103 is executed; if not, step S1104 is executed. In step S1103, the ratio corresponding to the current face is set to a preset ratio. In other words, the weight calculation module 202 sets the ratio corresponding to the current face to the preset ratio.

[0066] In step S1104, in response to the UV distance from the current pixel to the UV center of the current face being less than or equal to the UV range value in the UV range information, the ratio corresponding to the current face is set to: the preset ratio multiplied by the UV distance and then divided by the UV range value of the current face. In step S1105, it is determined whether there are unselected faces among the faces. If so, step S1106 is executed; if not, step S1107 is executed. In step S1106, in response to the presence of unselected faces among the faces, an unselected face among the faces is selected as the current face and the process returns to step S1101 to continue executing steps S1101 to S1107. In step S1107, in response to the presence of all faces being selected, the smallest ratio among all faces is selected as the weight of the current pixel. In some embodiments of the present invention, the preset ratio is 64.

[0067] Figure 12 is a flow chart of an image processing method according to some embodiments of the present invention. Figure 8 、 Figure 10 as well as Figure 12 , take over Figure 10 In an embodiment, when the object detection module 201 detects at least one face in the image, the object detection module 201 also detects the uv range information of each face based on the object detection algorithm and stores it in a memory for subsequent use. That is to say, in this embodiment, after the object detection module 201 detects the uv range information of each face based on the object detection algorithm, a uv range information is stored in the memory for each detected face. In this embodiment, step S1001 includes the weight calculation module 202 executing steps S1201 to S1206. In step S1201, based on the uv coordinate information of the uv center of a current face among the faces and the uv coordinate information of the current pixel, the uv distance from the uv center of the current face to the current pixel is calculated.

[0068] In step S1202, determine whether the uv distance from the current pixel to the uv center of the current face is greater than the uv range value in the uv range information of the current face. If so, execute step S1204, if not, execute step S1203. In step S1203, in response to the uv distance from the current pixel to the uv center of the current face being less than or equal to the uv range value in the uv range information of the current face, set the ratio corresponding to the current face to be: the preset ratio multiplied by the uv distance and then divided by the uv range value of the current face, and put the ratio corresponding to the current face into the candidate weight set. That is, let uvdistance represent the uv distance from the current pixel to the uv center of the current face, uvrange_currentface represent the uv range value of the current face, and preratio represent the preset ratio, then the ratio of the current face is set to

[0069]

[0070] In step S1204, a determination is made as to whether the faces contain an unselected face. If so, step S1205 is executed; if not, step S1206 is executed. In step S1205, in response to the presence of an unselected face in the faces, an unselected face is selected as the current face and the process returns to step S1201 to continue executing steps S1201-S1206. In step S1206, in response to the presence of all selected faces, all elements of the candidate weight set are averaged to obtain an average ratio, and the average ratio is set as the weight of the current pixel.

[0071] Figure 13 is a flow chart of an image processing method according to some embodiments of the present invention. Figure 11 、 Figure 12 as well as Figure 13 The UV coordinate information of the UV center of the current face includes a first coordinate value and a second coordinate value, and the UV coordinate information of the current pixel includes a first coordinate value and a second coordinate value. When the object detection module 201 detects at least one face in the image, the object detection module 201 also detects the first coordinate value and the second coordinate value of the UV coordinate of the UV center of each face based on the object detection algorithm. The steps of calculating the UV distance from the UV center of the current face to the current pixel in steps S1101 and S1201 include steps S1301 to S1305.

[0072] In step S1301, the absolute value of the difference between the first coordinate value of the current pixel and the first coordinate value of the UV center of the current face is calculated to obtain a first absolute value. In step S1302, the absolute value of the difference between the second coordinate value of the current pixel and the second coordinate value of the UV center of the current face is calculated to obtain a second absolute value. In step S1303, it is determined whether the second absolute value is greater than the first absolute value. If so, step S1304 is executed; if not, step S1305 is executed. In step S1304, in response to the second absolute value being greater than the first absolute value, the UV distance from the UV center of the current face to the current pixel is set to: the first absolute value multiplied by one-half plus the second absolute value. In step S1305, in response to the second absolute value being less than or equal to the first absolute value, the UV distance from the UV center of the current face to the current pixel is set to: the second absolute value multiplied by one-half plus the first absolute value.

[0073] If f u Indicates the first coordinate value of the current pixel, expressed as f v Indicates the second coordinate value of the current pixel, in center u Indicates the first coordinate value of the uv center of the current face, in center v Indicates the second coordinate value of the uv center of the current face, in t u Indicates the absolute value of the difference between the first coordinate value of the current pixel and the first coordinate value of the uv center of the current face, expressed in t v The absolute value of the difference between the second coordinate value of the current pixel and the second coordinate value of the uv center of the current face is represented by uvdistance, and the uv distance from the uv center of the current face to the current pixel is represented by uvdistance. Then steps S1301 to S1305 can be represented by the following C++ virtual code:

[0074] t u =abs(f u -center u );

[0075] t v =abs(f v -center v );

[0076] uvdistance=t v >t u ? (t v +t u / 2):(t u +t v / 2);

[0077] Among them, abs() is a function provided by C++ language to calculate absolute value.

[0078] Figure 14 is a flow chart of an image processing method according to some embodiments of the present invention. Figure 8 as well as Figure 14 , the polygon is an octagon, the coordinate information of the current pixel includes pixel coordinate information, the position information of each face includes the upper left corner coordinate information and the lower right corner coordinate information of the rectangular frame, four intercepts, four slopes and a region range information. Among them, the object detection module 201 obtains a rectangular frame, the upper left corner coordinate information and the lower right corner coordinate information of the rectangular frame and the region range information for each face in the received image based on the object detection algorithm. The region range information includes a region range value, which indicates the range of the area covered by the face. The pixel coordinate information included in the coordinate information of the current pixel includes the pixel coordinates of the current pixel, the upper left corner coordinate information of the rectangular frame of each face includes the pixel coordinates of the upper left corner of the rectangular frame, and the lower right corner coordinate information of the rectangular frame of each face includes the pixel coordinates of the lower right corner of the rectangular frame, wherein the pixel coordinates are coordinates in units of pixels in the image. The pixel coordinates of each point in the received image include a first coordinate value and a second coordinate value. The four intercepts and slopes of each face are used to delete the four triangles at the four corners of the rectangular frame of the face to obtain an octagonal frame. In this embodiment, the weight calculation module 202 stores a preset edge range for each face.

[0079] Step S805 includes the weight calculation module 202 executing step S1401. In step S1401, the weight calculation module 202 obtains at least one ratio of the current pixel to the at least one face based on the upper left corner coordinate information and the lower right corner coordinate information of the rectangular frame in the location information of each face, the four intercepts and the four slopes, the area range information, the edge range of each of the at least one face, and the pixel coordinate information of the current pixel.

[0080] Figure 5B to Figure 5C is a schematic diagram of the operation of an image processing system according to some embodiments of the present invention. Figure 5B to Figure 5C as well as Figure 14 , take over Figure 14In an embodiment, the object detection module 201 detects N faces in the image 203, and the weight calculation module 202 performs the following operations: (1) storing the first coordinate value and the second coordinate value of the pixel coordinate of the upper left corner of the rectangular box of each face detected by the object detection module 201 in the array starti[] and the array startj[], respectively; (2) storing the first coordinate value and the second coordinate value of the pixel coordinate of the lower right corner of the rectangular box of each face detected by the object detection module 201 in the array endi[] and the array endj[], respectively; (3) storing the intercept and the slope (e.g., the intercept and the slope) of the upper left corner of the rectangular box of each face detected by the object detection module 201 in the array endi[] and the array endj[], respectively. Figure 4 d1 and m1 in the array are stored in the array jLU[] and the array jLUm[] respectively, and the intercept and slope (e.g. Figure 4 d2 and m2 in the array are stored in the array jLD[] and the array jLDm[] respectively, and the intercept and slope (e.g. Figure 4 The intercept and slope of the lower right corner of the rectangular box corresponding to each face detected by the object detection module 201 (e.g. Figure 4 d3 and m3 in the image are stored in the array jRD[] and the array jRDm[] respectively. (4) The region range value contained in the region range information of each face detected by the object detection module 201 is stored in the array AI_region_range[]. In this embodiment, the weight calculation module 202 stores an edge range for each face. The edge range corresponding to each face is a preset value and is stored in the array AI_edge_range[]. Since the aforementioned data are stored in an array, the corresponding value can be obtained by the index value of the corresponding face. For example, jLD[k] represents the intercept of the lower left corner of the rectangular box corresponding to the face with index value k, and AI_edge_range[k] represents the edge range of the face with index value k. In this embodiment, since the object detection module 201 detects N faces in the image 203, the range of k is 0 to N-1.

[0081] In this embodiment, weight calculation module 202 sets the array aiRatio6face[] to store the ratio of the current pixel to each face, with xi representing the first coordinate value of the current pixel and xj representing the second coordinate value of the current pixel. It also sets variables zdist and finalRatio for calculation purposes. The ratio of the current pixel to the face with index k can be obtained using the following C++ pseudocode:

[0082] zdist=0;

[0083] z=starti[k]-xi;zdist=Max(zdist,z);

[0084] z=xi-endi[k];zdist=Max(zdist,z);

[0085] z=startj[k]-xj;zdist=Max(zdist,z);

[0086] z=xj-endj[k];zdist=Max(zdist,z);

[0087] z=startj[k]+jLU[k]-xj;

[0088] z=starti[k]+(z / pow(2,1+jLUm[k]))-xi;

[0089] zdist=Max(zdist,z);

[0090] z=startj[k]+jRU[k]-xj;

[0091] z=xi-endi[k]+(z / pow(2,1+jRUm[k]));

[0092] zdist=Max(zdist,z);

[0093] z=xj-endj[k]+jLD[k];

[0094] z=starti+(z / pow(2,1+jLDm[k]))-xi;

[0095] zdist=Max(zdist,z);

[0096] z=xj-endj[k]+jRD[k];

[0097] z=xi-endi[k]+(z / pow(2,1+jRDm[k]));

[0098] zdist=Max(zdist,z);

[0099] if zdist<=AI_edge_range[k]:

[0100] if zdist == 0:

[0101] finalRatio=0;

[0102] else:

[0103] finalRatio=64*(zdist / AI_edge_range[k]);

[0104] else:

[0105] finalRatio=64;

[0106] aiRatio6face[k]=finalRatio;

[0107] Among them, Max() is a function provided by C++ language, which takes the larger value of the parameters and returns it. pow() is a function provided by C++ language, which takes the second parameter as the exponent of the first parameter and returns the result of the exponential operation. For example, pow(a,b) will return a b . Let k = 0, 1, 2...N-1 in sequence, and execute the above virtual code to get the ratio of the current pixel to each face. From the above C++ virtual code, we can know that if the current pixel is in the octagonal frame, then zdist = 0, and the ratio of the current pixel to the face is 0. If the current pixel is not in the octagonal frame, then zdist is not 0. When zdist is not 0, the ratio of the current pixel to the face will be set according to the ratio of zdist and AI_edge_range[k]. Figures 5B to 5D The setting of finalRatio is further described. The face 503 is a face detected by the object detection module 201. The object detection module 201 obtains the octagonal frame 504 of the face 503 based on the above steps. Figure 5C ,exist Figure 5CIn the example, the edge range corresponding to face 503 is set to 0. Based on the aforementioned C++ virtual code, the ratios of pixels within range 505 enclosed by octagonal frame 504 to face 503 are all calculated to be 0 (because zdist is 0 within octagonal frame 504). The ratios of pixels in range 505, represented by black, to face 503 are calculated to be 0 by the aforementioned C++ virtual code. The ratios of pixels outside octagonal frame 504 to face 503 are all calculated to be 64 by the aforementioned C++ virtual code.

[0108] See also Figure 5D ,exist Figure 5D In the example, the edge range corresponding to the face 503 is set to a non-zero positive number (for example, 32). At this time, based on the aforementioned C++ virtual code, the ratios of the pixels within the range 505 enclosed by the octagonal frame 504 corresponding to the face 503 are all calculated to be 0 (because zdist is 0 within the octagonal frame 504). When zdist is not 0 but less than or equal to the edge range, the ratio of the current pixel corresponding to the face will be set to a value less than 64 but greater than 0 based on 64*(zdist / AI_edge_range[k]). Figure 5D In FIG. 5 , range 506 indicates a range where finalRatio is not 64. Within range 506 , values ​​from 0 to 64 are represented by grayscale. Values ​​closer to white indicate values ​​closer to 64 for finalRatio, and values ​​closer to black indicate values ​​closer to 0 for finalRatio.

[0109] Figure 15 is a flow chart of an image processing method according to some embodiments of the present invention. Figure 8 、 Figure 14 as well as Figure 15 , take over Figure 14 In the embodiment of the present invention, in step S805, after executing step S1401, the weight calculation module 202 executes step S1501 to obtain the weight of the current pixel. In step S1501, the smallest minimum ratio is selected from at least one ratio corresponding to at least one face of the current pixel (stored in the array aiRatio6face[]) as the weight of the current pixel.

[0110] Figure 16 is a flow chart of an image processing method according to some embodiments of the present invention. Figure 8 、 Figure 14 as well as Figure 16 , take over Figure 14In the embodiment of the present invention, in step S805, after step S1401, weight calculation module 202 executes step S1601 to obtain a weight for the current pixel. In step S1601, at least one ratio corresponding to at least one face of the current pixel (stored in array aiRatio6face[]) is averaged to obtain an average ratio. Weight calculation module 202 then sets the average ratio as the weight for the current pixel.

[0111] Figure 6 is a block diagram of a display system according to some embodiments of the present invention. Figure 6 In the embodiment, the display system 600 includes a main chip 601, a High Definition Multimedia Interface (HDMI) signal source 602, a USB signal source 603, and a display panel 604. The main chip 601 includes an image processing system 200 and a driver system 6011. The main chip 601 receives video data from the HDMI signal source 602 or the USB signal source 603. The image processing system 200 uses each frame of the video data as an image 203. If a face is detected, the image 203 is processed based on the obtained weights and then transmitted to the driver system 6011 to drive the display panel 604 for display.

[0112] Figure 7 Schematic diagram of the electronic device structure according to some embodiments of the present invention. In this embodiment, the architecture of the main chip 601 is as follows Figure 7 The electronic device 700 includes a processing unit 701, an internal memory 702, and a non-volatile memory 703. The internal memory 702 is, for example, a random-access memory (RAM). The processing unit 701 is a processor. The internal memory 702 and the non-volatile memory 703 are used to store programs, which may include program codes, and the program codes include computer operating instructions. The processing unit 701 reads the corresponding computer program from the non-volatile memory 703 into the internal memory 702 and then runs it, so as to implement the object detection module 201 and the weight calculation module 202 at the logical level.

[0113] Based on the above, the image processing system and image processing method provided by some embodiments of the present invention can effectively reduce the area of ​​non-face areas and the area of ​​face overlapping areas by using polygonal shapes to obtain the range of each face, thereby alleviating the side effects brought by the non-face parts marked as face areas; by obtaining the weight of the corresponding pixel based on the coordinate information of each pixel in the area covered by the range of all human faces and the position information of each face, the subsequent stage can be provided with brightness and chromaticity adjustment based on the weight of the corresponding pixel.

Claims

1. An image processing system comprising an object detection module and a weight calculation module, wherein the object detection module is configured to perform: Step (a1) receives an image and detects the image based on an object detection algorithm; and Step (a2) in response to detecting at least one face, obtaining a range of each of the at least one face based on a shape of a polygon; and The weight calculation module is configured to perform, on a plurality of pixels in an area covered by the range of all the at least one face: Step (b1) obtaining a weight of a current pixel among the plurality of pixels based on coordinate information of the current pixel and position information of each of the at least one human face; and In response to there being an unselected pixel among the plurality of pixels, step (b2) selects one of the plurality of pixels that has not been selected as the current pixel and returns to step (b1).

2. The image processing system of claim 1 , wherein the polygon is an octagon, and step (a2) comprises the following steps to obtain the range of each of the at least one face: Step (a21) obtaining a rectangular frame for each of the at least one human face based on the object detection algorithm; Step (a22) is to delete four triangles at the four corners of the rectangular frame of a current face among the at least one face, based on the four intercepts and four slopes of the current face, to obtain an octagonal frame, and use the range contained in the octagonal frame as the range of the current face; and In response to the at least one face having an unselected face, step (a23) selects one of the at least one faces that has not been selected as the current face and returns to step (a22). 3 . The image processing system of claim 2 , wherein the object detection module is configured to obtain the plurality of intercepts based on four ratios and a length of a side of the rectangular frame of the current face.

4. The image processing system of claim 1 , wherein the coordinate information of the current pixel includes UV coordinate information, the position information of each of the at least one face includes UV coordinate information of a UV center and UV range information, and step (b1) comprises: Step (b11) calculates a UV distance from the UV center of each of the at least one face to the current pixel based on the UV coordinate information of the UV center of each of the at least one face and the UV coordinate information of the current pixel; and obtains a weight of the current pixel based on the UV distances and the UV range information of all the at least one face.

5. The image processing system according to claim 4, wherein: The uv range information includes a uv range value, and step (b11) includes: Step (b111) calculating the UV distance from the UV center of a current face among the at least one face to the current pixel based on the UV coordinate information of the UV center of the current face and the UV coordinate information of the current pixel; Step (b112) in response to the UV distance from the current pixel to the UV center of the current face being greater than the UV range of the current face, setting a ratio corresponding to the current face to a predetermined ratio; In response to a UV distance from the current pixel to the UV center of the current face being less than or equal to the UV range value of the current face, setting the ratio corresponding to the current face to be: the predetermined ratio multiplied by the UV distance and then divided by the UV range value of the current face; as well as In response to there being an unselected face among the at least one face, step (b113) selects one of the at least one faces that has not been selected as the current face and returns to step (b111); in response to all of the at least one face being selected, selects the smallest ratio among all of the at least one face as the weight of the current pixel.

6. The image processing system according to claim 4, wherein: The uv range information includes a uv range value, and step (b11) includes: Step (b111) calculating the UV distance from the UV center of a current face among the at least one face to the current pixel based on the UV coordinate information of the UV center of the current face and the UV coordinate information of the current pixel; In step (b112), in response to the UV distance from the current pixel to the UV center of the current face being greater than the UV range value of the current face, step (b113) is executed; in response to the UV distance from the current pixel to the UV center of the current face being less than or equal to the UV range value of the current face, a ratio corresponding to the current face is set as: a preset ratio multiplied by the UV distance and then divided by the UV range value of the current face, and the ratio corresponding to the current face is placed in a candidate weight set; as well as In response to there being an unselected face among the at least one face, step (b113) selects one of the at least one faces that has not been selected as the current face and returns to step (b111); in response to the at least one face being selected, averages all elements of the candidate weight set to obtain an average ratio, and sets the average ratio as the weight of the current pixel.

7. The image processing system according to claim 5 or claim 6, wherein: The UV coordinate information of the UV center of the current face includes a first coordinate value and a second coordinate value, and the UV coordinate information of the current pixel includes a first coordinate value and a second coordinate value. The step of calculating the UV distance from the UV center of the current face to the current pixel includes: Calculating an absolute value of a difference between the first coordinate value of the current pixel and the first coordinate value of the uv center of the current face to obtain a first absolute value; Calculating an absolute value of a difference between the second coordinate value of the current pixel and the second coordinate value of the uv center of the current face to obtain a second absolute value; and In response to the second absolute value being greater than the first absolute value, the UV distance from the UV center of the current face to the current pixel is set to: the first absolute value multiplied by one half plus the second absolute value; in response to the second absolute value being less than or equal to the first absolute value, the UV distance from the UV center of the current face to the current pixel is set to: the second absolute value multiplied by one half plus the first absolute value.

8. The image processing system of claim 1 , wherein the polygon is an octagon, an edge range is preset for each of the at least one human face, the coordinate information of the current pixel includes pixel coordinate information, the position information of each of the at least one human face includes upper left corner coordinate information and lower right corner coordinate information of a rectangular frame, four intercepts, four slopes, and area range information, and step (b1) comprises: Step (b11) obtains at least one ratio of the current pixel to the at least one face based on the upper left corner coordinate information and the lower right corner coordinate information of the rectangular frame of the position information of each at least one face, the four intercepts, the four slopes and the area range information, the edge range of each at least one face and the pixel coordinate information of the current pixel.

9. The image processing system of claim 8, wherein step (b1) comprises: Step (b12) selects a minimum ratio from the at least one ratio of the current pixel corresponding to the at least one face as the weight of the current pixel.

10. An image processing method, comprising: Step (a) is performed by an object detection module: Step (a1) receiving an image and detecting the image based on an object detection algorithm; and Step (a2) in response to detecting at least one human face, obtaining a range of each of the at least one human face based on a shape of a polygon; as well as Step (b) is performed by a weight calculation module on a plurality of pixels in an area covered by the range of the at least one face: Step (b1) obtaining a weight of a current pixel among the plurality of pixels based on coordinate information of the current pixel and position information of each of the at least one human face; and In response to there being an unselected pixel among the plurality of pixels, step (b2) selects one of the plurality of pixels that has not been selected as the current pixel and returns to step (b1).

Images