Human image capture method, device, equipment and storage medium

Abstract

The present application relates to the field of image processing, and proposes a method, apparatus, device, and storage medium for capturing a human figure image. The method comprises: performing feature segmentation on an image to be captured to obtain a human figure region in the image to be captured; calculating a clarity score for the human figure region, and calculating a part score for each body part in the human figure region; determining a posture score for the image to be captured based on the part score for each body part in the human figure region; determining a human figure score for the image to be captured based on the clarity score and the posture score; and determining a target captured image based on the human figure score. Determining a clarity score and a posture score based on segmented body parts can effectively improve the accuracy of the human figure score, thereby facilitating the selection of higher quality images as captured human figure images and facilitating human figure recognition.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, apparatus, device and storage medium for capturing human-shaped images. Background Art

[0002] In fields such as intelligent surveillance, in order to efficiently collect accurate monitoring information, surveillance images are usually scored for human shape. Based on the results of the human shape scoring, images with higher human shape scores are selected as captured images, which is conducive to completing the pedestrian re-identification (ReID) task based on the captured images.

[0003] Common metrics for human figure scoring include figure clarity, figure pose, and figure lighting. By calculating and scoring each of these image metrics, the resulting figure score is obtained. However, when a camera captures an image containing a figure, the figure may be in motion, resulting in a blurry figure and a clearer background. This influence of the background image can lead to inaccurate figure scores. Furthermore, when calculating pose scores, the pose scoring model is typically a classification model, which is inaccurate for motion-blurred and incomplete figures, making it difficult to select high-quality human figures. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a method, apparatus, device and storage medium for capturing a human image to solve the problem in the prior art that the calculated human image scoring result is inaccurate, which is not conducive to selecting human images with better quality.

[0005] A first aspect of an embodiment of the present application provides a method for capturing a human image, the method comprising:

[0006] Performing feature segmentation on the image to be captured to obtain a human-shaped area in the image to be captured;

[0007] Calculating a clarity score of the human-shaped region and calculating a part score of each body part in the human-shaped region;

[0008] determining a posture score of the image to be captured based on the body part scores of the body parts in the human-shaped area;

[0009] Determining a human figure score of the image to be captured based on the clarity score and the posture score;

[0010] A target captured image is determined according to the human shape score.

[0011] In combination with the first aspect, in a first possible implementation of the first aspect, obtaining the human-shaped region in the image to be captured includes:

[0012] Extracting detail features of the image to be captured through a detail branch network, and extracting semantic features of the image to be captured through a semantic branch network;

[0013] The detail features and the semantic features are aggregated through an aggregation layer to determine a segmentation result of the human-shaped area in the image to be captured.

[0014] In combination with the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, aggregating the detail features and the semantic features through an aggregation layer includes:

[0015] Downsampling the feature image of the detail feature extracted by the detail branch network and directly aggregating it with the feature image of the semantic feature extracted by the semantic branch network;

[0016] Alternatively, the feature image of the detail features extracted by the detail branch network is directly aggregated with the feature image of the semantic features extracted by the semantic branch network after upsampling;

[0017] Alternatively, the feature image of the detail features extracted by the detail branch network is downsampled and directly aggregated with the feature image of the semantic features extracted by the upsampled semantic branch network.

[0018] In combination with the first aspect, in a third possible implementation of the first aspect, calculating the score of each part in the human-shaped region includes:

[0019] Determine a distribution matrix corresponding to the human-shaped area;

[0020] Determine a probability distribution heat map corresponding to the human-shaped area, and determine a probability matrix corresponding to the body part based on the probability distribution heat map;

[0021] An integration process is performed based on the distribution matrix and the probability matrix to determine the part score of each body part in the human-shaped area.

[0022] In combination with the third possible implementation manner of the first aspect, performing integration processing according to the distribution matrix and the probability matrix to determine the part score of each body part of the human-shaped region includes:

[0023] Performing integration processing based on the distribution matrix and the probability matrix to determine a similarity score of each body part of the human-shaped region;

[0024] A part score of the body part is determined according to the basic score of the body part and the similarity score.

[0025] In combination with the first aspect, in a fifth possible implementation of the first aspect, determining the posture score of the image to be captured based on the part scores of each body part in the human-shaped area includes:

[0026] determining a weight corresponding to the body part;

[0027] The posture score of the image to be captured is determined according to the part scores of the body parts in the human-shaped area and the weights corresponding to the body parts.

[0028] In combination with the first aspect, the first possible implementation manner of the first aspect, the second possible implementation manner of the first aspect, the third possible implementation manner of the first aspect, the fourth possible implementation manner of the first aspect, or the fifth possible implementation manner of the first aspect, in a sixth possible implementation manner of the first aspect, determining the target captured image based on the human figure score includes:

[0029] Comparing the human shape score with a preset score threshold, and setting the captured image with the human shape score greater than the score threshold as the target captured image;

[0030] Alternatively, the captured image with the highest human figure score is set as the target captured image.

[0031] A second aspect of the embodiments of the present application provides a human-shaped image capturing device, the device comprising:

[0032] A segmentation unit, configured to perform feature segmentation on the image to be captured, and obtain a human-shaped area in the image to be captured;

[0033] a first score calculation unit, configured to calculate a clarity score of the human-shaped region and a part score of each body part in the human-shaped region;

[0034] a second score calculation unit, configured to determine a posture score of the image to be captured based on the body part scores of the body parts in the human-shaped area;

[0035] a human figure score determining unit, configured to determine a human figure score of the image to be captured based on the clarity score and the posture score;

[0036] An image selection unit is used to determine a target captured image according to the human figure score.

[0037] A third aspect of an embodiment of the present application provides a human image capture device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first aspects when executing the computer program.

[0038] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of the first aspects are implemented.

[0039] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the present application obtains a human-shaped area in the image to be captured by performing feature segmentation on the image to be captured, and calculates a clarity score based on the segmented human-shaped area, thereby better ensuring the clarity of the portrait in the selected image; and determines a posture score based on the scores of each body part in the human-shaped area, so that a blurred or incomplete human figure can obtain a more accurate posture score, and a more accurate human figure score is obtained based on the clarity score and the posture score, which is conducive to accurately selecting the image with the highest quality as the target captured image, and provides convenience for human figure image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0041] Figure 1 This is a schematic diagram of the implementation process of a human image capture method provided in an embodiment of the present application;

[0042] Figure 2 This is a schematic diagram of a network structure for obtaining a humanoid area using a method provided in an embodiment of the present application;

[0043] Figure 3 This is a probability distribution heat map of various parts of the body provided in an embodiment of the present application;

[0044] Figure 4 This is a flow chart of determining a posture score according to an embodiment of the present application;

[0045] Figure 5 Schematic diagram of a human image capture device provided in an embodiment of the present application;

[0046] Figure 6 Schematic diagram of a humanoid image capture device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0048] In order to illustrate the technical solution described in this application, specific embodiments are provided below.

[0049] In application scenarios such as intelligent monitoring or intelligent photography, images can be scored for human figures. Based on the results of the human figure scoring, images with better quality can be determined as captured images, so as to facilitate subsequent human figure recognition and other tasks based on the captured images. Since the human figures in the shooting scene are often in motion, the human figures in the captured images may be relatively blurred while the background is relatively clear. Affected by the background, the accuracy of the calculated clarity score is not high. In addition, when calculating the posture score, the posture scoring model used is usually a classification model. The classification model is often not accurate enough for the classification of blurred human figures, incomplete human figures, and human figures from different perspectives (front, back, side), resulting in the low accuracy of the calculated human figure score, which affects the quality of the captured human figure image.

[0050] In order to overcome the above problems, Figure 1 As shown, the embodiment of the present application proposes a method for capturing a human image, the method comprising:

[0051] In S101 , feature segmentation is performed on the image to be captured to obtain a human-shaped region in the image to be captured.

[0052] The image to be captured described in the embodiments of the present application can be a preview image captured by a camera, or an image to be processed captured by a camera. The images to be captured include two or more. By scoring the human figures of two or more images to be captured, an image with better quality can be found among the two or more human figures and used as the captured human figure image. Among them, better quality can be manifested in better clarity of the human figure in the human figure image, more comprehensive posture of the human figure in the human figure image, and other aspects for evaluation.

[0053] When performing feature segmentation on the image to be captured, it may include segmenting the portrait area and the non-human area (background area) in the image to be captured, or it may also include segmenting the body parts in the human area. For example, the segmented body parts may include the hair part, the face part, the upper limb torso part, the lower limb part and the shoe part, etc. (but not limited to this). Segmenting the hair part and the face part as separate body parts can more reliably extract human feature information, which is convenient for subsequent recognition processing. Among them, the lower limb part is the part below the torso and above the shoes. The shoe part is the part of the shoe worn or the part of the foot without shoes.

[0054] When the image to be captured is segmented, the image to be captured can be input into a trained neural network, and the neural network can be used to identify the area where the body parts included in the image to be captured are located. For example, the neural network can be Figure 2 The network structure shown is based on the improvement of BiSeNetV2. In this network structure, the backbone network includes two branches, namely the detail branch network and the semantic branch network. Among them, the size of the feature image of the detail branch network is larger than that of the semantic branch network, and the depth of the detail branch network is shallower than that of the semantic branch network. The size of the feature image of the semantic branch network is smaller than that of the detail branch network, and the depth of the semantic branch network is deeper than that of the detail branch network.

[0055] The feature images extracted by the detail branch network are larger in size, while those extracted by the semantic branch network are smaller in size. For example, when the model input size is 96*96, the semantic branch network can downsample by a factor of 16, extracting a feature image of 6*6. Due to the significant loss of spatial information in a 3*3 feature image, downsampling by a factor of 32 is unnecessary. Accordingly, the detail branch network can downsample by a factor of 4, obtaining a feature image of 24*24.

[0056] The size of the feature image extracted by 16-fold downsampling of the semantic branch network is smaller, so global information broadcasting is no longer required and context embedding processing is not required.

[0057] The feature images extracted by the semantic branch network and the feature branch network can be aggregated at the aggregation layer. This can include upsampling smaller semantic feature images and then directly aggregating them with detail feature images, downsampling larger detail feature images and then directly aggregating them with semantic feature images, or upsampling the semantic feature images and downsampling the detail feature images separately to ensure that the aggregated feature images are of the same size. For example, a 96*96 image to be captured can be downsampled 4x by the detail branch network to produce a first feature image of 24*24. Downsampling 16x by the semantic branch network can extract a second feature image of 6*6. The first feature image can be downsampled to 6*6 or 12*12, or the second feature image can be upsampled to 12*12 or 24*24 before being fused. In a possible implementation, the first feature image can be downsampled while the second feature image is upsampled to obtain feature images of uniform size for aggregation, such as a 12*12 feature image. Assuming that the divided body parts include five parts, a 12*12*6 feature image can be obtained, where 6 represents the categories included in the image, for example, five body parts and a background image.

[0058] In S102 , a clarity score of the human-shaped region is calculated, and a part score of each body part in the human-shaped region is calculated.

[0059] After segmenting the image to be captured, the human-shaped area and the non-human-shaped area (background area) included in the image to be captured can be obtained. The clarity of the obtained human-shaped area can be calculated to obtain a clarity score for the image to be captured. In other words, the clarity score of the image to be captured can be directly calculated based on the clarity of the human-shaped area. Since this solution only needs to calculate the clarity of the human-shaped area (human-shaped area), it can effectively eliminate the influence of the clarity of the non-human-shaped area, thereby improving the accuracy of the clarity score.

[0060] Among them, the method for calculating the clarity of the image to be captured may include a grayscale variance algorithm based on a human-shaped area, a Laplace operator method, an entropy function, etc.

[0061] When determining the posture score of a humanoid region, scores can be calculated for each part of the humanoid region based on the segmented humanoid region, thereby effectively robust to the effects of blurred or incomplete portraits or portrait shooting angles on the posture score.

[0062] like Figure 4 This is a flow chart of determining the posture score provided in the embodiment of the present application. Figure 4 As shown, when calculating the posture score of the image to be captured, it is necessary to predetermine the probability distribution heat map corresponding to each body part, that is, to determine the heat map obtained by statistically analyzing the probability of the body part appearing at different positions in the image.

[0063] The purpose of the pose score is to identify people. The person re-identification model is trained using the market-1501 dataset. This model performs better for people whose poses are close to those in the market-1501 dataset, but performs worse for people whose poses differ significantly from those in the training set. Therefore, the criterion for evaluating a person's pose can be the degree of similarity to the average person in the market-1501 training set. The closer the person is to the average person in the market-1501 dataset, the higher the pose score; otherwise, the lower the score.

[0064] In order to obtain the average human figure in the market-1501 dataset, we can use the body part segmentation model to perform semantic segmentation on a predetermined number of human figures in the dataset. Each human figure is divided into five parts. Then, statistics are performed on each part to obtain the following: Figure 3 The probability distribution heatmap for each body part shown is a statistical plot of the distribution probability of each body part at different locations. Based on the resulting probability distribution heatmap, a probability matrix of predetermined sizes corresponding to each body part can be derived. For example, in the probability matrix for hair, the size of each element represents the probability that the area is hair. To facilitate comparison and calculation, the probability matrix can be a matrix where all values ​​sum to 1.

[0065] When calculating the scores of the body parts of a human figure in the image to be captured, it is also necessary to determine the distribution matrix corresponding to the body parts in the image to be captured. For example, the human figure area includes the hair part, the face part, the upper torso part, the lower limb part, and the shoe part. The distribution matrix of the hair part, the distribution matrix of the face part, the distribution matrix of the upper torso part, the distribution matrix of the lower limb part, and the distribution matrix of the shoe part are determined respectively. For example, according to the segmentation results of the image to be captured, for the hair part, if the part is hair, the corresponding matrix element is 1, and if the part is not hair, the corresponding matrix element is 0. The size of the distribution matrix is ​​consistent with the size of the probability matrix, for example, both can be set to a size of 32*32.

[0066] Based on the determined probability matrix and distribution matrix, the distribution matrix and probability matrix can be integrated to obtain the partial scores of each body part in the humanoid region. The integration process can be performed by calculating the product of the corresponding elements in the two matrices by element-by-element multiplication. All products are added together to obtain the corresponding part score S = H * P, where H is the distribution matrix and P is the probability matrix.

[0067] For example, for pixel A in the human-shaped area, the score of the pixel in the distribution matrix is ​​determined to be h1, and the score of the pixel in the probability matrix is ​​p1. The score of the pixel can be expressed as h1*p1. If the number of pixels in the divided part is N, the score corresponding to the part is

[0068] In S103 , the posture score of the image to be captured is determined based on the body part scores of the body parts in the human-shaped area.

[0069] After determining the corresponding part scores for each body part in the image to be captured, the weights corresponding to each part can be determined based on their different areas and importance, taking into account the different areas and importance of each part. For example, the weights for hair, face, upper torso, lower limbs, and shoes can be determined to be 10, 10, 25, 15, and 15, respectively. Assuming that the calculated part scores for hair, face, upper torso, lower limbs, and shoes are A, B, C, D, and E, respectively, the weighted score for the part of the image to be captured can be expressed as: 10%*A+10%*B+25%*C+15%*D+15%*E.

[0070] In a possible implementation, a basic score for each body part may be set, for example, the basic score may be set to 5. When the part exists in the image to be captured, the basic score corresponding to the part may be obtained as 5 points.

[0071] The pose score of the image to be captured can be obtained by summing the weighted score and the basic score. For example, if the weighted score is 40 points and the sum of the basic scores is 15 points, the pose score can be determined to be 55 points.

[0072] To facilitate evaluation and comparison, the posture scores can be scaled. For example, if the range of the posture scores determined according to the above definition is 0-70, the range can be scaled to obtain a posture score in the range of 0-100. For example, a posture score of 70 will be 100 after scaling.

[0073] In S104 , a human figure score of the image to be captured is determined according to the clarity score and the posture score.

[0074] After determining the clarity score and posture score of the image to be captured, the human figure score of the image to be captured can be calculated based on the weights corresponding to the clarity and posture.

[0075] Considering that an incomplete human figure is of lower quality than a complete but blurry one, the pose weight can be set to be greater than the clarity weight. For example, the pose weight can be set to 0.6 and the clarity weight to 0.4. If the pose score is x and the clarity score is y, then the human figure score can be expressed as: 0.6x + 0.4y.

[0076] In S105 , a target captured image is determined according to the human figure score.

[0077] After determining the human shape scores of two or more images to be captured, the images to be captured can be compared based on the determined human shape scores, and the image to be captured whose human shape score is greater than a predetermined score threshold can be selected as the target captured image. In a possible implementation, if there are multiple images to be captured that are greater than the predetermined score threshold, or if there are no images to be captured that are greater than the predetermined score threshold, the image to be captured with the highest human shape score can be selected as the target captured image. The captured image includes human shape information and can be used for human shape recognition or analysis.

[0078] Since the present application determines the clarity score based on the area of ​​the segmented human figure, the accuracy of the clarity score can be improved, and the part score is calculated based on the distribution matrix of each body part and the probability matrix obtained based on the statistics of the data set, and the posture score is determined according to the part score, it can effectively adapt to the score calculation of human figure information in incomplete images, blurred images or images from different perspectives, so that images that are more conducive to human figure recognition can be screened according to the human figure score, thereby improving the accuracy of human figure recognition.

[0079] In order to verify the accuracy of the human image capturing method described in the embodiment of the present application, a test data set can be constructed for verification analysis.

[0080] When constructing a test dataset, a predetermined number of human images can be selected, for example, 1,000 human images, which can be divided into groups of N human images, for example, 10 human images per group, which can be divided into 100 groups. The 100 groups of images are evenly distributed to the testers, for example, to 25 testers, with each tester receiving 4 groups of images.

[0081] The clarity of each group of images can be sorted according to the current clarity algorithm. When comparing clarity, the comparison is made according to the following rules:

[0082] 1) Only consider the clarity of the human figure, without considering the influence of the background.

[0083] 2) Only the clarity dimension is sorted, without considering dimensions such as human figure integrity and lighting.

[0084] 3) Compare the two human figures to see which one is more blurred based on the size of the blurred block and the blur strength of the blurred block (loss of edge information and details).

[0085] 4) For human figures of the same clarity, the more edge information and details, the higher the clarity.

[0086] 5) To sort the clarity of each group of figures, simply add a prefix from 0 to 9 to the current file name. For example, if the original file name is 38_59_pg_1628826582.jpg, and it ranks fifth in clarity within the group, it should be changed to 4_38_59_pg_1628826582.jpg.

[0087] 6) If the clarity of the figures in two images is indistinguishable, the clarity is allowed to be the same. For example, a group of 10 figures has prefixes of 0, 1, 1, 2, 3, 4, 5, 5, 6, and 7 after sorting.

[0088] During the test, each group of 10 people, after sorting, was paired up in pairs, for a total of 10*9 / 2 = 45 pairs. Using the existing full-image clarity algorithm and the segmentation-based clarity score calculation method of this application, the accuracy rates achieved were 0.58 and 0.795, respectively. The clarity score calculation method described in this application significantly improves the accuracy of clarity score calculation.

[0089] Similarly, the present application determines the posture score based on the part score of the human figure area, and the posture score determined can effectively meet expectations. For example, in the examples of posture scores determined by the present application based on the part score, including images with only a head, half-truncated images, occluded images, images with postures that do not conform to the standard human figure, images with poor distribution of the human figure in the picture, and images with good postures, the posture scores determined gradually increase through calculation, which can effectively meet the requirements of human figure recognition based on the posture score.

[0090] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0091] Figure 5 A schematic diagram of a human image capturing device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the device includes:

[0092] The segmentation unit 501 is used to perform feature segmentation on the image to be captured to obtain a human-shaped area in the image to be captured;

[0093] A first score calculation unit 502 is configured to calculate a clarity score of the human-shaped region and a part score of each body part in the human-shaped region;

[0094] A second score calculation unit 503 is configured to determine a posture score of the image to be captured based on the body part scores of the body parts in the human-shaped area;

[0095] A human figure score determination unit 504 is configured to determine a human figure score of the image to be captured based on the clarity score and the posture score;

[0096] The image selection unit 505 is configured to determine a target captured image according to the human figure score.

[0097] Figure 5 The human-shaped image capturing device shown in FIG. Figure 1 The human image capture method shown corresponds to.

[0098] Figure 6 Schematic diagram of a human image capture device provided by an embodiment of the present application. Figure 6 As shown, the human image capture device 6 of this embodiment includes: a processor 60, a memory 61, and a computer program 62, such as a human image recognition program, stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, it implements the steps of the aforementioned human image capture method embodiments. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of the various modules / units in the aforementioned device embodiments.

[0099] Exemplarily, the computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 62 in the humanoid image capture device 6.

[0100] The human image capturing device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that Figure 6 It is only an example of the humanoid image capture device 6 and does not constitute a limitation of the humanoid image capture device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the humanoid image capture device may also include input and output devices, network access devices, buses, etc.

[0101] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0102] The memory 61 can be an internal storage unit of the humanoid image capture device 6, such as a hard disk or memory of the humanoid image capture device 6. The memory 61 can also be an external storage device of the humanoid image capture device 6, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the humanoid image capture device 6. Furthermore, the memory 61 can also include both the internal storage unit and the external storage device of the humanoid image capture device 6. The memory 61 is used to store the computer program and other programs and data required by the humanoid image capture device. The memory 61 can also be used to temporarily store data that has been output or is to be output.

[0103] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0104] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0105] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0106] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0107] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0108] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0109] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0110] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for capturing a human image, characterized in that: The method comprises: Performing feature segmentation on the image to be captured to obtain a human-shaped area in the image to be captured; Calculating a clarity score of the human-shaped region and calculating a part score of each body part in the human-shaped region, including: determining a distribution matrix corresponding to each body part in the human-shaped region; performing semantic segmentation on a predetermined number of human figures in a data set using a body part segmentation model, performing statistics on each part, determining a probability distribution heat map corresponding to each body part, and determining a probability matrix corresponding to each body part based on the probability distribution heat map; performing integration processing based on the distribution matrix and the probability matrix to determine a part score for each body part in the human-shaped region; determining a posture score of the image to be captured based on the body part scores of the body parts in the human-shaped area; Determining a human figure score of the image to be captured based on the clarity score and the posture score; A target captured image is determined according to the human shape score.

2. The method according to claim 1, characterized in that Obtaining a human-shaped area in the image to be captured includes: Extracting detail features of the image to be captured through a detail branch network, and extracting semantic features of the image to be captured through a semantic branch network; The detail features and the semantic features are aggregated through an aggregation layer to determine a segmentation result of the human-shaped area in the image to be captured.

3. The method according to claim 2, characterized in that Aggregating the detail features and the semantic features through an aggregation layer includes: Downsampling the feature image of the detail feature extracted by the detail branch network and directly aggregating it with the feature image of the semantic feature extracted by the semantic branch network; Alternatively, the feature image of the detail features extracted by the detail branch network is directly aggregated with the feature image of the semantic features extracted by the semantic branch network after upsampling; Alternatively, the feature image of the detail features extracted by the detail branch network is downsampled and directly aggregated with the feature image of the semantic features extracted by the upsampled semantic branch network.

4. The method according to claim 1, wherein Performing integration processing according to the distribution matrix and the probability matrix to determine the part score of each body part of the human-shaped region includes: Performing integration processing based on the distribution matrix and the probability matrix to determine a similarity score of each body part of the human-shaped region; A part score of the body part is determined according to the basic score of the body part and the similarity score.

5. The method according to claim 1, wherein Determining the posture score of the image to be captured based on the body part scores of the body parts in the human-shaped area includes: determining a weight corresponding to the body part; The posture score of the image to be captured is determined according to the part scores of the body parts in the human-shaped area and the weights corresponding to the body parts.

6. The method according to any one of claims 1 to 5, characterized in that Determining a target captured image according to the human figure score includes: Comparing the human shape score with a preset score threshold, and setting the captured image with the human shape score greater than the score threshold as the target captured image; Alternatively, the captured image with the highest human figure score is set as the target captured image.

7. A human-shaped image capturing device, characterized in that: The device comprises: A segmentation unit, configured to perform feature segmentation on the image to be captured, and obtain a human-shaped area in the image to be captured; a first score calculation unit, configured to calculate a clarity score of the human-shaped region and a part score of each body part in the human-shaped region, comprising: determining a distribution matrix corresponding to each body part in the human-shaped region; performing semantic segmentation on a predetermined number of human figures in a data set using a body part segmentation model, performing statistics on each part, determining a probability distribution heat map corresponding to each body part, and determining a probability matrix corresponding to each body part based on the probability distribution heat map; and performing integration processing based on the distribution matrix and the probability matrix to determine a part score for each body part in the human-shaped region; a second score calculation unit, configured to determine a posture score of the image to be captured based on the body part scores of the body parts in the human-shaped area; a human figure score determining unit, configured to determine a human figure score of the image to be captured based on the clarity score and the posture score; An image selection unit is used to determine a target captured image according to the human figure score.

8. A humanoid image capturing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images