Image processing method and device

Abstract

The present invention provides an image processing method and device, comprising: obtaining positional information of a facial region in an image to be processed; determining positional information of a first region in the image to be processed where a person resides based on the positional information of the facial region; and performing scaling processing on pixel data of the first region based on the positional information of the first region. The method of an embodiment of the present invention processes the person region in the image before displaying the image, thereby improving the image display quality.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method and device. Background Art

[0002] With the advancement of science and technology, televisions have evolved from traditional black and white to color, and then from color to today's LCD TVs. As LCD panel manufacturing technology continues to improve, the number of pixels per unit area of ​​the panel increases, leading to the emergence of larger LCD TVs. For example, large-screen LCD TVs with a resolution of 3840×2160 are gaining popularity among consumers.

[0003] In the related art, for large-size 16:9 LCD TVs, if the format of the received image signal does not match the size of the TV, the image signal needs to be enlarged according to a certain ratio when displayed on the TV. Generally, the image will be stretched in the horizontal direction, causing the displayed picture to look unnatural, especially scenes with characters familiar to the user, which will appear fatter. Summary of the Invention

[0004] The present invention provides an image processing method and device to improve image display effects.

[0005] In a first aspect, the present invention provides an image processing method, comprising:

[0006] Obtaining the location information of the face area in the image to be processed;

[0007] Determining, based on the position information of the face region, position information of a first region where a person is located in the image to be processed;

[0008] Scaling is performed on the pixel data of the first area according to the position information of the first area.

[0009] In a possible implementation, determining the position of the first region where the person is located in the image to be processed according to the position of the face region includes:

[0010] Calculating the width of the face area according to the position information of the face area;

[0011] Determining the width of the person's body based on the width of the face area and a preset ratio of the face to the body;

[0012] The position information of the first area is determined according to the position information of the face area and the width of the person's body.

[0013] In a possible implementation, after determining the width of the character's body, the method further includes:

[0014] determining the height of the character's body;

[0015] Determining the position information of the first area according to the position information of the face area and the width of the person's body includes:

[0016] The position information of the first area is determined according to the position information of the face area, the width of the person's body, and the height of the person's body.

[0017] In a possible implementation, if the image to be processed is an image obtained by scaling the original image, scaling the pixel data of the first area according to the position information of the first area includes:

[0018] determining a scaling ratio of the first area according to the aspect ratio of the image to be processed and the aspect ratio of the original image;

[0019] Scaling processing is performed on the pixel data of the first area according to the position information of the first area and the scaling ratio of the first area.

[0020] In a possible implementation manner, after scaling the pixel data of the first area, the method further includes:

[0021] performing scaling processing on edge pixel data of a second region adjacent to the first region in the image to be processed, to obtain the processed image to be processed; the edge pixel data is pixel data in the second region adjacent to the first region;

[0022] The processed image to be processed is displayed.

[0023] In a possible implementation, the scaling process on edge pixel data of a second area adjacent to the first area in the image to be processed includes:

[0024] If the second area includes at least two sub-areas, respectively obtaining a scaling ratio of edge pixel data of each sub-area;

[0025] The edge pixel data of each sub-region is scaled according to the scaling ratio of the edge pixel data of each sub-region.

[0026] In a possible implementation, obtaining the position information of the face region in the image to be processed includes:

[0027] According to the pre-trained machine learning model, the image to be processed is identified to obtain the position information of the face area in the image to be processed; the input parameter of the machine learning model is the image to be processed, and the output parameter of the machine learning model is the position information of the face area.

[0028] In a second aspect, the present invention provides an image processing device, comprising:

[0029] An acquisition module is used to obtain the position information of the face area in the image to be processed;

[0030] a determination module, configured to determine position information of a first region where a person is located in the image to be processed based on the position information of the face region;

[0031] A processing module is used to perform scaling processing on the pixel data of the first area according to the position information of the first area.

[0032] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any one of the methods described in the first aspect.

[0033] In a fourth aspect, an embodiment of the present invention provides an electronic device, including:

[0034] processor; and

[0035] a memory for storing executable instructions of the processor;

[0036] The processor is configured to perform the method according to any one of the first aspects by executing the executable instructions.

[0037] The image processing method and device provided by the embodiments of the present invention obtain the position information of the face area in the image to be processed; determine the position information of the first area where the person is located in the image to be processed based on the position information of the face area; and scale the pixel data of the first area based on the position information of the first area. In the above scheme, by scaling the area where the person is located in the image, the display effect of the person in the image is better, avoiding the problem in the prior art that the person in the image displays unnaturally and the effect is poor after the image is stretched. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0039] Figure 1 1 is a flow chart of an embodiment of an image processing method provided by the present invention;

[0040] Figure 2 This is a schematic diagram of the principle of an embodiment provided by the present invention;

[0041] Figure 3 It is a schematic diagram of the principle of another embodiment provided by the present invention;

[0042] Figure 4 This is a schematic diagram of the principle of another embodiment provided by the present invention;

[0043] Figure 5 is a structural diagram of an embodiment of an image processing device provided by the present invention;

[0044] Figure 6 It is a structural diagram of an embodiment of an electronic device provided by the present invention.

[0045] The above drawings illustrate specific embodiments of the present disclosure, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present disclosure in any way, but rather to illustrate the concepts of the present disclosure to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0046] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0047] The terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0048] First, the application scenarios involved in the present invention are introduced:

[0049] The image processing method provided by the embodiment of the present invention is applied to scenarios where the original image's scale format is different from the scale format when it is displayed, to improve the image display effect. The method can be applied to electronic devices such as televisions, smartphones, and tablet computers.

[0050] The method of the embodiment of the present invention achieves better display effects of the characters in the image by scaling the area where the characters are located in the image, thereby avoiding the problem in the prior art that the characters in the image appear unnatural and the effect is poor after the image is stretched.

[0051] The technical solution of the present invention is described in detail below with reference to specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0052] Figure 1 FIG. 1 is a flow chart of an embodiment of an image processing method provided by the present invention. Figure 1 As shown, the method provided in this embodiment includes:

[0053] Step 101: Obtain location information of a face region in an image to be processed.

[0054] Specifically, if the ratio format of the original image is different from the ratio format when displayed, for example, the ratio format of the original image is 4:3 in aspect ratio, and the aspect ratio of the display is 16:9, the image will be stretched when displayed, the characters in the image will be deformed, and the display effect will be poor.

[0055] In embodiments of the present invention, a facial region in an image to be processed can be identified and its location information obtained. This location information can include the coordinates of the pixels in the facial region, particularly the coordinates of the boundary pixels of the facial region. The region containing the person in the image to be processed can then be processed to achieve a display effect similar to that of the person in the original image.

[0056] Furthermore, step 101 can be implemented in the following manner:

[0057] According to the pre-trained machine learning model, the image to be processed is identified to obtain the position information of the face area in the image to be processed; the input parameter of the machine learning model is the image to be processed, and the output parameter of the machine learning model is the position information of the face area.

[0058] Furthermore, before the image to be processed is recognized based on the pre-trained machine learning model, the method further includes:

[0059] The machine learning model is trained according to the training data to obtain a trained machine learning model; the training data includes multiple images and position information of facial areas in the images.

[0060] Among them, the machine learning model can be established through algorithms such as deep learning algorithms and obtained after training with training data.

[0061] Furthermore, in other embodiments of the present invention, the area where the person is located can also be directly obtained through a machine learning model.

[0062] Step 102: Determine the position information of a first region where a person is located in the image to be processed based on the position information of the face region.

[0063] Specifically, after obtaining the position information of the face area, the position information of the first area where the person is located can be determined. For example, the ratio of the face to the person's body can be obtained in advance, such as the ratio of the width, so that the width of the person's body can be obtained based on the width of the face area, thereby determining the first area where the person is located.

[0064] Furthermore, step 102 can be implemented in the following manner:

[0065] Calculating the width of the face area according to the position information of the face area;

[0066] Determining the width of the person's body based on the width of the face area and a preset ratio of the face to the body;

[0067] The position information of the first area is determined according to the position information of the face area and the width of the person's body.

[0068] Specifically, based on the position information of the facial region, i.e., pixel coordinate information, the width of the facial region is calculated. The width of the facial region can be expressed in pixels. Then, based on a preset ratio of the face to the body, the width of the person's body is determined. The width of the person's body can also be expressed in pixels. Furthermore, based on the position information of the facial region and the width of the person's body, the position information of the first region is determined.

[0069] Exemplarily, Pixelnum1 represents the width of the face area, K represents the preset ratio of the face to the body, and the width of the person's body Pixelnum2 = Pixelnum1 / K. Figure 2 The area between the middle dotted lines is the first area where the person is located, and the resolution of the image to be processed is 3840×2160.

[0070] Furthermore, in other embodiments of the present invention, the height of the person's body can be determined based on the height ratio of the face to the body, and then the height Pixelnum3 of the first area where the person is located can be determined, that is, the first area only includes the person, such as Figure 3 shown.

[0071] Step 103: Perform scaling processing on the pixel data of the first area according to the position information of the first area.

[0072] Specifically, if the image to be processed is an image obtained by magnifying the original image, the first area where the character is located may be reduced so that the character can be restored to the display effect before the magnification.

[0073] For example, if the original image has a 4:3 aspect ratio and the display has a 16:9 aspect ratio, the image will be stretched when displayed. That is, the image to be processed is the stretched original image, and the characters in the image will be deformed, resulting in poor display quality. In the above solution, the character area in the image to be processed is scaled so that the characters maintain the display quality of the original image, and then the processed image is displayed.

[0074] Furthermore, after processing the first area where the person is located, other areas in the image to be processed may be processed so that the proportional format of the image when displayed is consistent with the proportional format of the display.

[0075] The method of this embodiment obtains the position information of the face area in the image to be processed; determines the position information of the first area where the person is located in the image to be processed based on the position information of the face area; and scales the pixel data of the first area based on the position information of the first area. In the above scheme, by scaling the area where the person is located in the image, the display effect of the person in the image is better, avoiding the problem in the prior art that the person in the image displays unnaturally and the effect is poor after the image is stretched.

[0076] On the basis of the above embodiment, further, if the image to be processed is an image obtained by scaling the original image, step 103 can be implemented by the following steps:

[0077] determining a scaling ratio of the first area according to the aspect ratio of the image to be processed and the aspect ratio of the original image;

[0078] Scaling processing is performed on the pixel data of the first area according to the position information of the first area and the scaling ratio of the first area.

[0079] Specifically, the scaling ratio of the first area where the character is located is determined according to the aspect ratio of the image to be processed and the aspect ratio of the original image. For example, if the aspect ratio of the image to be processed is 16:9 and the aspect ratio of the original image is 4:3, the scaling ratio of the first area is

[0080] In one embodiment of the present invention, Figure 2 As shown, the image to be processed is an image after the original image is magnified, and only the pixel data of the first area can be reduced in the width direction (Pixelnum2) according to the scaling ratio.

[0081] In another embodiment of the present invention, Figure 3 As shown, the pixel data of the first area can be scaled in the width direction and the length direction (Pixelnum3) according to the scaling ratio.

[0082] In other embodiments of the present invention, the image to be processed may be an image obtained by reducing the original image, and the pixel data of the first area may be amplified according to the scaling ratio.

[0083] Furthermore, in order to improve the display effect of the image, it is necessary to smooth the adjacent pixel data of the character area, that is, the following operations may be performed after step 103:

[0084] performing scaling processing on edge pixel data of a second region adjacent to the first region in the image to be processed, to obtain the processed image to be processed; the edge pixel data is pixel data in the second region adjacent to the first region;

[0085] The processed image to be processed is displayed.

[0086] Specifically, such as Figure 4 As shown, assuming that after the first area is reduced in width, there will be a blank area between the first area and the second area, then the pixels of the blank area need to be compensated, that is, by scaling the edge pixel data of the second area adjacent to the first area in the processed image, such as enlarging the edge pixel data, and filling the blank area with the edge pixel data of the second area, where the edge pixel data refers to Figure 4 The pixel data between the two dotted lines is shown as X, L L and L R are the left and right boundaries of the first area where the character is located, L L and L R The pixel data between is the pixel data of the first area where the character is located.

[0087] Furthermore, scaling is performed on edge pixel data of a second region adjacent to the first region in the image to be processed, which can be specifically implemented in the following manner:

[0088] If the second area includes at least two sub-areas, respectively obtaining a scaling ratio of edge pixel data of each sub-area;

[0089] The edge pixel data of each sub-region is scaled according to the scaling ratio of the edge pixel data of each sub-region.

[0090] Specifically, for example Figure 4In the embodiment, the second area includes two areas, a left area and a right area, and the scaling ratios of the edge pixel data of the left and right areas are respectively obtained, and scaling processing is performed on the edge pixel data of the left and right areas.

[0091] In actual applications, the width of the edge pixel data on the left and right sides may occupy different pixels, and the scaling ratio may also be different.

[0092] In other embodiments of the present application, Figure 3 As shown, if the first area is also scaled in the height direction, the second area can also include pixel data of multiple areas above, below, left and right. The edge pixel data of each area is scaled separately, and then the processed image is displayed.

[0093] For example, if X represents the number of pixels of the edge pixel data of the second region, k represents the ratio of X to Pixelnum2, and assuming that the reduction ratio of the first region in the width direction is 3 / 4, the scaling ratio y of the edge pixel data of the second region can be determined according to the following formula:

[0094]

[0095] If k is 1 / 2, then y is 5 / 4.

[0096] In this embodiment, the area where the person is located in the image is scaled so that the display effect of the person in the image is better, and the other areas in the image except the person are scaled so that the person and other areas in the image are smoothed so that the image display effect is better.

[0097] Figure 5 This is a structural diagram of an embodiment of an image processing device provided by the present invention, as shown in FIG. Figure 5 As shown, the image processing device of this embodiment includes:

[0098] An acquisition module 501 is used to acquire the position information of the face area in the image to be processed;

[0099] A determination module 502 is configured to determine position information of a first region where a person is located in the image to be processed based on the position information of the face region;

[0100] The processing module 503 is configured to perform scaling processing on the pixel data of the first area according to the position information of the first area.

[0101] In a possible implementation, the determining module 502 is specifically configured to:

[0102] Calculating the width of the face area according to the position information of the face area;

[0103] Determining the width of the person's body based on the width of the face area and a preset ratio of the face to the body;

[0104] The position information of the first area is determined according to the position information of the face area and the width of the person's body.

[0105] In a possible implementation, the determining module 502 is specifically configured to:

[0106] determining the height of the character's body;

[0107] The position information of the first area is determined according to the position information of the face area, the width of the person's body, and the height of the person's body.

[0108] In a possible implementation, if the image to be processed is an image obtained by scaling the original image, the processing module 503 is specifically configured to:

[0109] determining a scaling ratio of the first area according to the aspect ratio of the image to be processed and the aspect ratio of the original image;

[0110] Scaling processing is performed on the pixel data of the first area according to the position information of the first area and the scaling ratio of the first area.

[0111] In a possible implementation, the processing module 503 is specifically configured to:

[0112] performing scaling processing on edge pixel data of a second region adjacent to the first region in the image to be processed, to obtain the processed image to be processed; the edge pixel data is pixel data in the second region adjacent to the first region;

[0113] The processed image to be processed is displayed.

[0114] In a possible implementation, the processing module 503 is specifically configured to:

[0115] If the second area includes at least two sub-areas, respectively obtaining a scaling ratio of edge pixel data of each sub-area;

[0116] The edge pixel data of each sub-region is scaled according to the scaling ratio of the edge pixel data of each sub-region.

[0117] In a possible implementation, the acquisition module 501 is specifically configured to:

[0118] According to the pre-trained machine learning model, the image to be processed is identified to obtain the position information of the face area in the image to be processed; the input parameter of the machine learning model is the image to be processed, and the output parameter of the machine learning model is the position information of the face area.

[0119] The device of this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0120] Figure 6 The electronic device embodiment provided by the present invention is shown in FIG. Figure 6 As shown, the electronic device includes:

[0121] A processor 601 and a memory 602 for storing executable instructions of the processor 601 .

[0122] Optionally, it may further include: a display screen 603 for displaying image data.

[0123] The above components may communicate via one or more buses.

[0124] Among them, the processor 601 is configured to execute the corresponding method in the aforementioned method embodiment by executing the executable instructions. The specific implementation process can be found in the aforementioned method embodiment and will not be repeated here.

[0125] The electronic device in the embodiment of the present invention may be a television, a smart phone, a tablet computer, etc.

[0126] A computer-readable storage medium is also provided in an embodiment of the present invention, on which a computer program is stored. When the computer program is executed by a processor, the corresponding method in the aforementioned method embodiment is implemented. The specific implementation process can be referred to the aforementioned method embodiment. The implementation principle and technical effects are similar and will not be repeated here.

[0127] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0128] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: include: Obtaining the location information of the face area in the image to be processed; Determining, based on the position information of the face region, position information of a first region where a person is located in the image to be processed; If the image to be processed is an image obtained by scaling the original image, determining the scaling ratio of the first area according to the aspect ratio of the image to be processed and the aspect ratio of the original image; The pixel data of the first area is scaled according to the position information of the first area and the scaling ratio of the first area, so that the character maintains the display effect of the original image.

2. The method according to claim 1, characterized in that The step of determining the position of the first region where the person is located in the image to be processed according to the position of the face region includes: Calculating the width of the face area according to the position information of the face area; Determining the width of the person's body based on the width of the face area and a preset ratio of the face to the body; The position information of the first area is determined according to the position information of the face area and the width of the person's body.

3. The method according to claim 2, characterized in that After determining the width of the character's body, the method further includes: determining the height of the character's body; Determining the position information of the first area according to the position information of the face area and the width of the person's body includes: The position information of the first area is determined according to the position information of the face area, the width of the person's body, and the height of the person's body.

4. The method according to any one of claims 1 to 3, characterized in that After scaling the pixel data of the first area, the method further includes: performing scaling processing on edge pixel data of a second region adjacent to the first region in the image to be processed, to obtain the processed image to be processed; the edge pixel data is pixel data in the second region adjacent to the first region; The processed image to be processed is displayed.

5. The method according to claim 4, characterized in that The scaling process of edge pixel data of a second area adjacent to the first area in the image to be processed includes: If the second area includes at least two sub-areas, respectively obtaining a scaling ratio of edge pixel data of each sub-area; The edge pixel data of each sub-region is scaled according to the scaling ratio of the edge pixel data of each sub-region.

6. The method according to any one of claims 1 to 3, characterized in that The step of obtaining the position information of the face region in the image to be processed includes: According to the pre-trained machine learning model, the image to be processed is identified to obtain the position information of the face area in the image to be processed; the input parameter of the machine learning model is the image to be processed, and the output parameter of the machine learning model is the position information of the face area.

7. An image processing device, characterized in that include: An acquisition module is used to obtain the position information of the face area in the image to be processed; a determination module, configured to determine position information of a first region where a person is located in the image to be processed based on the position information of the face region; The processing module is configured to determine the scaling ratio of the first area according to the aspect ratio of the image to be processed and the aspect ratio of the original image if the image to be processed is an image after scaling processing of the original image; and scale the pixel data of the first area according to the position information of the first area and the scaling ratio of the first area so that the character maintains the display effect of the original image.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to perform the method according to any one of claims 1 to 6 by executing the executable instructions.

Images