Object centering method and object centering device

Abstract

The invention discloses an object centering method and an object centering device. According to the method, a processor reads at least one instruction stored in a memory to execute the object centering method. The object centering method comprises the following steps: performing foreground object segmentation according to a depth image and a face image to generate a foreground object image; judging a relation between the foreground object image and a default threshold value to generate a judgment result; and performing object centering on the foreground object image according to the judgment result to generate an object centering image, or performing object centering on the face image according to the judgment result to generate an object centering image.

Description

Technical Field

[0001] This application relates to a method and apparatus for centering objects, and more particularly to a method and apparatus for centering objects in a foreground object image or a face image based on a determination of a foreground object. Background Technology

[0002] In the field of image processing, object centering techniques rely on the results of object detection. For example, face centering techniques require face detection in an image and, based on the results, placing the face in the center of the image.

[0003] However, the sheer variety of object types makes it extremely difficult to collect and train on such a diverse range of objects, resulting in object detection results that cannot cover all object types. Incorrect object detection results will affect object centering. Summary of the Invention

[0004] In view of the shortcomings of the prior art, one of the objectives of this application is (but not limited to) to provide an object centering method and an object centering device to improve the shortcomings of the prior art.

[0005] In some embodiments, a processor reads at least one instruction stored in memory to execute an object centering method, the object centering method comprising the following steps: segmenting a foreground object based on a depth image and a face image to generate a foreground object image; determining the relationship between the foreground object image and a default threshold to generate a determination result; and centering the foreground object image according to the determination result to generate an object-centered image, or centering the face image according to the determination result to generate an object-centered image.

[0006] In some embodiments, the object centering device includes memory and a processor. The memory is used to store at least one instruction. The processor is used to read at least one instruction to perform the following steps: segmenting a foreground object based on a depth image and a face image to generate a foreground object image; determining the relationship between the foreground object image and a default threshold to generate a determination result; and centering the foreground object image according to the determination result to generate an object-centered image, or centering the face image according to the determination result to generate an object-centered image.

[0007] The technical means embodied in the embodiments of this application can improve at least one of the shortcomings of the prior art. For example, there are many types of objects, and such a large variety of object types is very difficult to collect and train, making it difficult for the object detection results to cover all types of objects, thus leading to erroneous object detection results. The object centering method and object centering device of this application can center the foreground object image or face image according to the judgment result of the foreground object, thereby solving the problem that the object detection results are erroneous and affect the object centering.

[0008] The features, implementation, and effects of this application are described in detail below with reference to the accompanying drawings, showing preferred embodiments. Attached Figure Description

[0009] Figure 1 A schematic diagram of an object centering device is provided based on some embodiments of this application. Figure 2 A flowchart of an object centering method is provided based on some embodiments of this application. Figure 3 A flowchart for generating a foreground object image is provided according to some embodiments of this application; Figure 4 A schematic diagram illustrating an input image according to some embodiments of this application; Figure 5 This is a schematic diagram illustrating an object masking image of a foreground object according to some embodiments of this application; Figure 6 A flowchart of a block correction and image magnification process is provided based on some embodiments of this application. Figure 7 A schematic diagram illustrating a block modification based on some embodiments of this application is provided; Figure 8 A schematic diagram illustrating an image magnification according to some embodiments of this application is provided; Figure 9 A flowchart for generating a depth image is drawn according to some embodiments of this application; and Figure 10 A flowchart for generating a human face image is provided according to some embodiments of this application. Detailed Implementation

[0010] All terms used herein have their ordinary meanings. The definitions of the terms above in commonly used dictionaries, and any examples of the use of any term discussed herein, are merely illustrative and should not be construed as limiting the scope or meaning of this application. Similarly, this application is not limited to the various embodiments shown in this specification.

[0011] As used herein, "coupled" or "connected" can refer to two or more components making direct physical or electrical contact with each other, or indirectly making direct physical or electrical contact with each other, or to two or more components operating or moving together. As used herein, the term "circuit" can refer to a device in which at least one transistor and / or at least one active or passive component are connected in a certain manner to process signals.

[0012] As used herein, the term "and / or" includes any combination of one or more of the listed related items. The terms "first," "second," and "third," etc., are used herein to describe and distinguish the individual components. Therefore, a first component may also be referred to as a second component without departing from the intent of this application. For ease of understanding, similar components in the various figures will be designated with the same reference numerals.

[0013] To address the issue of erroneous object detection results affecting object centering, this application proposes an object centering method and an object centering device, detailed below.

[0014] Figure 1 A schematic diagram of an object centering device 100 is provided according to some embodiments of this application. As shown, the object centering device 100 includes a processor 110 and a memory 120. The memory 120 is used to store at least one instruction. The processor 110 is used to read at least one instruction to execute the object centering process. For ease of understanding of the operation of the object centering device 100, please also refer to... Figure 2 , Figure 2 This is a flowchart of a method 200 for centering objects.

[0015] In step 210, foreground object segmentation is performed based on the depth image and the face image to generate a foreground object image. For example, please refer to... Figure 3 In step 310, a depth image is input. In step 320, a face image is input. Subsequently, in step 330, an average face depth is obtained based on the depth image and the face image. For example, the formula is as follows: …Formula 1 As shown in Formula 1, For average face depth, This refers to the region of interest (ROI) for a face. This is a depth image.

[0016] In step 340, binarization is performed based on the average face depth. For example, the formula is as follows: …Formula 2 As shown in Formula 2, depth image Between 0 and 255, Formula 2 uses the average face depth. This serves as a threshold to obtain an object-masking image that is closer to the image capture device than the human face. Please see. Figure 4 The input image 400 includes a face 410 and a foreground object 420. Please refer to... Figure 5According to Formula 2, this application can obtain an object masking image 510 that is closer to the image capture device than the face 410, and the object masking image 510 corresponds to the foreground object 420.

[0017] In step 350, a foreground object image is generated. For example, this application may use connected component analysis to analyze the object masking image 510 to obtain a frame 520, where the frame 520 corresponds to the object within the input image 400, which is the foreground object 420.

[0018] Please refer back to this. Figure 2 In step 220, the relationship between the foreground object image and a default threshold is determined to generate a determination result. In step 230, the foreground object image is centered according to the determination result to generate a centered object image, or the face image is centered according to the determination result to generate a centered object image. For example, the determination formula is as follows: …Formula 3 As shown in Formula 3 The region of interest for an object is determined by the pixel ratio of the foreground object. If the pixel ratio of the foreground object exceeds a preset threshold of 0.03, the foreground object is centered to generate a centered image. If the threshold is less than the default threshold of 0.03, object centering is performed on the face to generate a face-centered image. However, this application is not limited to the above embodiments, which are merely illustrative of one implementation method of this application. In other embodiments, the preset threshold can be any other suitable value, depending on the actual needs.

[0019] In some embodiments, the steps of centering an object image to generate a centered object image are described in the following examples. Figure 6 In step 610, bounding box correction is performed on the object image (such as a foreground object image or a face image) to generate corrected bounding boxes. For example, see [link to example]. Figure 7 This application obtains an object image (such as an object frame) 720 based on object 710. Since the aspect ratio of the object image 720 does not match the input image 740, the object image 720 needs to be frame corrected to generate a corrected frame 730. In this way, if the corrected frame 730 is subsequently enlarged, the aspect ratio of the corrected frame 730 will match the input image 740 or the subsequent output image.

[0020] …Formula 4 …Formula 5 …Formula 6 …Formula 7 Please refer to Formula 4. The corrected width is based on the original object's height. or width Therefore, see Formula 5. The corrected height is based on the original object's height. or width Therefore, see Formula 6. The corrected x-coordinate is based on the original object's x-coordinate. Width of the original object And the corrected width Therefore, see Formula 7. The corrected y-coordinate is based on the original object's y-coordinate. The height of the original object And the corrected high And thus.

[0021] In step 620, the corrected bounding box is enlarged to produce a centered image of the object. For example, please refer to... Figure 8 This application performs image magnification on the correction box 810 to generate a centered object image 820. In some embodiments, this application may employ bilinear interpolation for image magnification.

[0022] In some embodiments, directly enlarging the object during object centering may affect the user's visual experience. Therefore, this application provides a progressive centering method, the formula for which is as follows: …Formula 8 As shown in Formula 8, Center the image of the object. To correct the box, The previous box represents the progressive parameter K, which can be adjusted by the user. This application can use Formula 8 to progressively center the object to avoid affecting the user's visual experience.

[0023] In some embodiments, this application can generate an object depth image based on an object centering image and a depth image. For example, the formula is as follows: …Formula 9 …Formula 10 As shown in Formula 9, this application centers the image of the object. Filtering is performed to produce depth of field. As shown in Formula 10, this application is based on depth of field. Depth images Centered image of the object To generate object depth images .

[0024] Please see Figure 4 as well as Figure 9 In step 910, the input image 400 is reduced to produce a reduced image. For example, this application may use bilinear interpolation to reduce the input image 400, and the width and height of the reduced image may be 256*160. However, this application is not limited to the above embodiments, which are only used to illustrate one implementation method of this application. In other embodiments, the width and height of the reduced image may use other suitable values, depending on the actual needs.

[0025] In step 920, depth estimation is performed on the reduced image. In step 930, a depth image is generated. For example, this application may employ a neural network-like system to perform depth estimation. In step 940, this application may normalize the depth image. For example, this application may employ a depth estimation neural network-like system to perform normalization, as shown in the following formula: …Formula 11 Please refer to Formula 11. For depth images, the depth value output by the neural network can be 0 to FLOAT_MAX, and is normalized to be between 0 and 255 by Equation 11.

[0026] Please see Figure 4 as well as Figure 10 In step 1010, the input image 400 is reduced in size to generate a reduced image. For example, this application may use bilinear interpolation to reduce the input image 400, and the width and height of the reduced image may be 320*180, with the output being the face position (x, y, width, height) in the image. In step 1020, face detection is performed on the reduced image. In step 1030, a face image is generated. For example, this application may use a neural network-like system to perform face detection. However, this application is not limited to the above embodiments, which are only used to illustrate one implementation method of this application. In other embodiments, the width and height of the reduced image may use other suitable values, depending on the actual needs.

[0027] It should be noted that this application is not intended to... Figures 1 to 10 The embodiments shown are limited and are merely illustrative of one implementation of this application to facilitate understanding of the technology. The patent scope of this application shall be determined by the scope of the invention application. Modifications and refinements made by those skilled in the art to the embodiments of this application without departing from the spirit of this application still fall within the scope of the invention application.

[0028] The technical means embodied in the embodiments of this application can improve at least one of the shortcomings of the prior art. The object centering method and object centering device of this application can center the foreground object image or face image according to the judgment result of the foreground object, thereby solving the problem that the object detection result is incorrect and affects the object centering.

[0029] Although the embodiments of this application are described above, these embodiments are not intended to limit this application. Those skilled in the art can make changes to the technical features of this application based on the express or implied content of this application. All such changes may fall within the scope of patent protection sought by this application. In other words, the scope of patent protection of this application shall be determined by the claims of this specification.

[0030] [Symbol Explanation] 100: Object centering device 110: Processor 120: Memory 200: Object Centering Method 210~230: Steps 310~350: Steps 400: Input Image 410: Face 420: Foreground object 510: Object masking image 520: Screen frame 610~620: Steps 710: Object 720: Object Image 730: Correction box 740: Input Image 810: Correction box 820: Centered object image 910~940: Steps 1010~1030: Steps

Claims

1. A method for centering an object, wherein a processor reads at least one instruction stored in a memory to execute the object centering method, characterized in that, Include: A foreground object image is generated by segmenting a foreground object based on a depth image and a face image. Determine the relationship between the foreground object image and a default threshold to generate a judgment result; as well as Based on the judgment result, the foreground object image is centered to generate an object-centered image, or the face image is centered to generate an object-centered image.

2. The object centering method as described in claim 1, characterized in that, The steps of centering the foreground object image based on the judgment result to generate a centered object image include: If a proportion of the foreground object image is greater than the preset threshold, the foreground object image is centered to generate a centered object image.

3. The object centering method as described in claim 1, characterized in that, The steps for centering the face image based on the judgment result to generate a centered image of the object include: If a proportion of the foreground object image is less than the preset threshold, the face image is centered to generate a centered object image.

4. The object centering method as described in claim 1, characterized in that, Also includes: A depth estimation is performed on an input image to generate a depth image.

5. The object centering method as described in claim 4, characterized in that, The steps of performing this depth estimation on the input image to generate the depth image include: The input image is reduced in size to produce a reduced image; and The depth estimation is performed on the reduced image to generate the depth image.

6. The object centering method as described in claim 1, characterized in that, Also includes: Perform face detection on an input image to generate the face image.

7. The object centering method as described in claim 6, characterized in that, The steps of performing face detection on the input image to generate the face image include: The input image is reduced in size to produce a reduced image; and The face detection is performed on the reduced image to generate the face image.

8. The object centering method as described in claim 1, characterized in that, Also includes: A depth image of an object is generated based on the object's centered image and its depth image.

9. The object centering method as described in claim 1, characterized in that, The steps of centering the foreground object image based on the judgment result to generate a centered object image include: Apply a box correction to the foreground object image to produce a correction box; as well as The corrected box is magnified to produce a centered image of the object. The step of centering the object in the face image based on the judgment result to generate a centered object image includes: Apply a bounding box correction to the face image to generate a correction bounding box; as well as The corrected box is magnified to produce a centered image of the object.

10. An object centering device, characterized in that, Include: A memory location for storing at least one instruction; and A processor for reading the at least one instruction to perform the following steps: A foreground object image is generated by segmenting a foreground object based on a depth image and a face image. Determine the relationship between the foreground object image and a default threshold to generate a judgment result; as well as Based on the judgment result, the foreground object image is centered to generate an object-centered image, or the face image is centered to generate an object-centered image.

Images