Image auditing method and device, equipment and medium

By calculating the non-overlapping degree of the clothing enclosed area in the image and the coordinates of the key points of the human body, the problem of insufficient accuracy of the dress degree evaluation in the image audit in the prior art is solved, and more accurate and continuous image audit results are achieved, and the audit is improved. efficiency.

CN119006889APending Publication Date: 2024-11-22BEIJING QIYI CENTURY SCI & TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410988621.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The prior art has insufficient accuracy in the evaluation of the dress degree of characters in image review and cannot respond in fine-grained manner, resulting in insufficient accuracy in the audit results.

Method used

By obtaining the set of coordinate points of the clothing enclosing area and the set of coordinate points of the human body, non-overlapping degrees are calculated to represent the audit results of the image, providing a more accurate and continuous evaluation of the degree of dress.

Benefits of technology

It improves the accuracy and efficiency of image review, meets the precise evaluation needs of dressing in actual applications, and provides finer-grained evaluation results compared with discrete classification methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006889A_ABST
    Figure CN119006889A_ABST
Patent Text Reader

Abstract

The embodiment of the invention relates to an image auditing method and device, equipment and a medium. The method comprises the steps of obtaining a to-be-processed image; acquiring a clothing surrounding area coordinate point set and a human body key point coordinate set corresponding to the to-be-processed image; determining a non-overlapping degree based on the clothing surrounding area coordinate point set and the human body key point coordinate set; and determining an image auditing result of the to-be-processed image based on the non-overlapping degree. According to the embodiment of the invention, the accurate non-overlapping degree is calculated through the combination of the clothing surrounding area coordinate point set and the human body key point coordinate set so as to represent the auditing result of the image, the dressing degree of the image can be accurately audited and evaluated, and the image auditing efficiency and effect are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of multimedia processing technology, and in particular to an image review method, device, equipment and medium. Background Art

[0002] Generally, in image review, content filtering, and multimedia data analysis, reviewing and evaluating the clothing of people in images is a very important task, which is of great significance to social media platforms, content review agencies, and image analysis systems. By reviewing and evaluating the clothing of people, the content standards of the platform can be better maintained and the content health can be improved.

[0003] Most related technologies use discrete classification to evaluate images. For example, the degree of clothing is simply divided into two categories: "exposing" and "not revealing". This method cannot reflect the clothing situation in a fine-grained manner, resulting in poor accuracy in image review and evaluation. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides an image review method, device, equipment and medium.

[0005] An embodiment of the present disclosure provides an image review method, which includes: obtaining an image to be processed; obtaining a clothing enclosing area coordinate point set and a human body key point coordinate set corresponding to the image to be processed; determining a non-overlapping degree based on the clothing enclosing area coordinate point set and the human body key point coordinate set; and determining an image review result of the image to be processed based on the non-overlapping degree.

[0006] Optionally, obtaining the set of coordinate points of the clothing enclosing area corresponding to the image to be processed includes: obtaining the maximum circumscribed detection area of ​​clothing corresponding to the image to be processed; obtaining the target pixel coordinates corresponding to the maximum circumscribed detection area of ​​clothing; inputting the target pixel coordinates and the image to be processed into a general segmentation model for segmentation to obtain the set of coordinate points of the clothing enclosing area.

[0007] Optionally, obtaining the human body key point coordinate set corresponding to the image to be processed includes: obtaining a human body image area corresponding to the image to be processed; performing human body key point detection on the human body image area based on a preset human body key point detection algorithm to obtain the human body key point coordinate set.

[0008] Optionally, determining the non-overlapping degree based on the clothing enclosed area coordinate point set and the human body key point coordinate set includes: acquiring target human body key point coordinates of each human body sub-area based on the human body key point coordinate set; determining reference clothing enclosed area coordinate points of each human body sub-area from the clothing enclosed area coordinate point set based on the target human body key point coordinates; determining clothing coverage distance and clothing uncovered distance of each human body sub-area based on the target human body key point coordinates and the reference clothing enclosed area coordinate points of each human body sub-area; determining candidate non-overlapping degrees of each human body sub-area based on the clothing coverage distance and the clothing uncovered distance of each human body sub-area; and obtaining the non-overlapping degree based on the candidate non-overlapping degrees of each human body sub-area.

[0009] Optionally, determining the candidate non-overlapping degrees of each human body sub-region based on the clothing-covered distance and the clothing-uncovered distance of each human body sub-region includes: determining the total distance of each human body sub-region based on the clothing-covered distance and the clothing-uncovered distance of each human body sub-region; and determining the ratio of the clothing-uncovered distance of each human body sub-region to the total distance as the candidate non-overlapping degree of each human body sub-region.

[0010] Optionally, obtaining the non-overlapping degree based on the candidate non-overlapping degrees of the respective human body sub-regions includes: comparing the candidate non-overlapping degrees of the respective human body sub-regions to determine a maximum candidate non-overlapping degree as the non-overlapping degree.

[0011] Optionally, obtaining the non-overlapping degree based on the candidate non-overlapping degrees of the respective human body sub-regions includes: obtaining preset weight information of each human body sub-region; performing calculations based on the weight information of each human body sub-region and the candidate non-overlapping degree of each of the human body sub-regions to obtain multiple target non-overlapping degrees; and comparing the multiple target non-overlapping degrees to determine the maximum target non-overlapping degree as the non-overlapping degree.

[0012] The disclosed embodiment also provides an image review device, which includes: a first acquisition module, used to acquire an image to be processed; a second acquisition module, used to acquire a set of coordinate points of a clothing enclosed area corresponding to the image to be processed; a third acquisition module, used to acquire a set of coordinate points of a human body key points corresponding to the image to be processed; and a calculation module, used to perform calculation based on the set of coordinate points of the clothing enclosed area and the set of coordinate points of the human body key points to obtain the non-overlap degree of the image to be processed.

[0013] An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the image review method provided by the embodiment of the present disclosure.

[0014] The embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the image review method provided by the embodiment of the present disclosure.

[0015] The above technical solution provided by the embodiment of the present disclosure can accurately evaluate the degree of dressing of the image by combining the coordinate point set of the clothing enclosed area and the coordinate set of the key points of the human body to calculate the accurate non-overlapping degree to represent the audit result of the image, thereby improving the efficiency and effect of image audit. Compared with the prior art method of simply dividing clothing into two categories by discrete classification, the above method can provide a more accurate and continuous non-overlapping degree to represent the evaluation result of the degree of dressing of the image, meeting the demand for accurate evaluation of the degree of dressing of the image in practical applications.

[0016] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 A flowchart of an image review method provided by an embodiment of the present disclosure;

[0020] Figure 2 An example diagram of key points of a human body provided in an embodiment of the present disclosure;

[0021] Figure 3a An example diagram of overlapping key points of a human body and clothing provided in an embodiment of the present disclosure;

[0022] Figure 3b Another example diagram of overlapping key points of a human body and clothing provided in an embodiment of the present disclosure;

[0023] Figure 3c Another example diagram of overlapping key points of a human body and clothing provided in an embodiment of the present disclosure;

[0024] Figure 4 A flowchart of an image review method provided by an embodiment of the present disclosure;

[0025] Figure 5 A schematic diagram of an image review process provided by an embodiment of the present disclosure;

[0026] Figure 6 A schematic diagram of the structure of an image review device provided by an embodiment of the present disclosure;

[0027] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0030] In existing image review and evaluation scenarios, most image clothing degree detection methods use discrete classification to evaluate, such as simply dividing clothing degrees into "exposing" and "non-exposing" categories, or performing graded evaluations through limited clothing levels (such as level 1, level 2, level 3, etc.). Although these traditional methods have achieved basic clothing degree judgment, they are obviously insufficient in precision and continuity, and cannot meet the needs of accurate clothing degree evaluation in practical applications.

[0031] Therefore, it can be seen that the above-mentioned discrete classification method can only discretely classify the degree of dressing. This simple discrete method cannot reflect more fine-grained exposure conditions; in addition, the evaluation method based on discrete levels cannot provide more accurate and continuous exposure values, which limits the application value of the detection results. To this end, the embodiment of the present disclosure obtains the set of coordinate points of the clothing enclosed area and the set of coordinates of the key points of the human body and calculates the accurate non-overlapping degree to represent the audit results of the image, which can accurately audit and evaluate the degree of dressing of the image and improve the efficiency and effect of image audit. The following is a detailed explanation:

[0032] Figure 1The following is a flow chart of an image review method provided by an embodiment of the present disclosure. The method can be applied to electronic devices, such as computers, mobile phones, tablet computers, televisions, servers, etc., without limitation. Figure 1 As shown, the method mainly includes the following steps S102 to S210:

[0033] Step S102, obtaining an image to be processed.

[0034] In the embodiments of the present disclosure, there are many ways to obtain an image to be processed. As an example, a video to be processed is obtained, frames are extracted from the video to be processed to obtain each video frame, and each video frame is preprocessed such as resizing and color normalization as the image to be processed; wherein each video frame can be all video frames, interval video frames, etc., or can be a key video frame filtered out according to a preset filtering strategy, for example, video frames including human bodies are filtered out as key video frames, and the specific settings are selected according to the needs of the application scenario.

[0035] As another example, an input image is obtained, and the input image is preprocessed such as resizing and color normalization as the image to be processed. The above two methods are only examples of methods for obtaining the image to be processed, and the embodiments of the present disclosure do not specifically limit the methods for obtaining the image to be processed.

[0036] Step S104, obtaining a set of coordinate points of a clothing enclosing area and a set of coordinate points of a human body corresponding to the image to be processed.

[0037] In the embodiments of the present disclosure, there are many ways to obtain a set of coordinate points of the clothing enclosing area corresponding to the image to be processed. As an example, the image to be processed is detected based on a pre-trained clothing detection model, the maximum circumscribed detection area of ​​the clothing is obtained, the target pixel coordinates corresponding to the maximum circumscribed detection area of ​​the clothing are obtained, and the target pixel coordinates and the image to be processed are input into a general segmentation model for segmentation to obtain a set of coordinate points of the clothing enclosing area; as another example, the image to be processed is input into a pre-trained image segmentation model to obtain a set of coordinate points of the clothing enclosing area; wherein the image segmentation model is obtained by training in advance according to annotated image samples and convolutional neural networks, for example, the image segmentation model includes a fully connected layer and a normalization layer, the image to be processed is processed by the fully connected layer and then input into the normalization layer to obtain the specific position of the clothing, that is, the set of coordinate points of the clothing enclosing area. The above two methods are only examples of methods for obtaining a set of coordinate points of the clothing enclosing area corresponding to the image to be processed, and the embodiments of the present disclosure do not specifically limit the methods for obtaining a set of coordinate points of the clothing enclosing area corresponding to the image to be processed.

[0038] In the embodiment of the present disclosure, the human body key point coordinate set refers to the pixel coordinates of the human body key points in the image to be processed. It can be understood that the human body key points are usually as follows: Figure 2 The 1-12 key points shown are used to detect the key points of the human body in the human image area, and the coordinate set of the key points of the human body can be understood as Figure 2 The coordinates of one or more key points among the key points 1-12 shown.

[0039] In the embodiments of the present disclosure, there are many ways to obtain a set of human key point coordinates corresponding to the image to be processed. As an example, the image to be processed is detected based on a pre-trained human detection model to obtain a human image area, and human key point detection is performed on the human image area based on a preset human key point detection algorithm to obtain a preliminary key point detection result; the preliminary key point detection result is adjusted based on the positional relationship between the human image area and the image to be processed to obtain a set of human key point coordinates; as another example, feature recognition is performed on the image to be processed based on a pre-trained human key point detection model to identify key point features, and key point coordinates corresponding to the key point features are obtained as a set of human key point coordinates. The above two methods are only examples of methods for obtaining a set of human key point coordinates corresponding to the image to be processed, and the embodiments of the present disclosure do not specifically limit the methods for obtaining a set of human key point coordinates corresponding to the image to be processed.

[0040] Step S106, determining the non-overlapping degree based on the coordinate point set of the clothing enclosing area and the coordinate point set of the human body key points.

[0041] In an embodiment of the present disclosure, after obtaining a clothing enclosing area coordinate point set and a human body key point coordinate set, a non-overlapping degree is determined based on the clothing enclosing area coordinate point set and the human body key point coordinate set. As an example, target human body key point coordinates of each human body sub-area are obtained based on the human body key point coordinate set, reference clothing enclosing area coordinate points of each human body sub-area are determined from the clothing enclosing area coordinate point set based on the target human body key point coordinates, clothing coverage distance and clothing uncovered distance of each human body sub-area are determined based on the target human body key point coordinates and the reference clothing enclosing area coordinate points of each human body sub-area, candidate non-overlapping degrees of each human body sub-area are determined based on the clothing coverage distance and clothing uncovered distance of each human body sub-area, and non-overlapping degrees are obtained based on the candidate non-overlapping degrees of each human body sub-area; as another example, the clothing enclosing area coordinate point set and the human body key point coordinate set are processed based on a preset audit algorithm or model to obtain a non-overlapping degree.

[0042] Step S108, determining the image review result of the image to be processed based on the non-overlapping degree.

[0043] In the disclosed embodiment, the non-overlapping degree indicates the degree of clothing non-coverage. A higher non-overlapping degree indicates a higher degree of clothing non-coverage, that is, the clothing covers less area of ​​the human body. Thus, the image review result of the image to be processed can be determined based on the non-overlapping degree.

[0044] In the embodiments of the present disclosure, the image to be processed can be further processed according to its non-overlapping degree. For example, the non-overlapping degree of the image to be processed is compared with a preset first threshold. When the non-overlapping degree of the image to be processed is greater than the first threshold, the image to be processed is deleted. For another example, the non-overlapping degree of the image to be processed is compared with a preset second threshold, wherein the second threshold is less than the first threshold. When the non-overlapping degree of the image to be processed is greater than the second threshold, the target area of ​​the image to be processed is blocked or other processing is performed. The specific setting is selected according to the application scenario to meet the usage requirements; wherein the first threshold and the second threshold are selected and set according to the needs of the application scenario.

[0045] In the above embodiment, compared with the prior art which uses discrete classification to simply divide clothing into two categories, a more accurate and continuous non-overlapping degree representation of the clothing degree assessment result of the image can be provided, thereby meeting the demand for accurate assessment of the clothing degree of the image in practical applications.

[0046] In some embodiments, obtaining a set of coordinate points of a clothing enclosing area corresponding to an image to be processed includes: obtaining the maximum circumscribed detection area of ​​clothing of the image to be processed, obtaining the target pixel coordinates corresponding to the maximum circumscribed detection area of ​​clothing, inputting the target pixel coordinates and the image to be processed into a general segmentation model for segmentation, and obtaining a set of coordinate points of a clothing enclosing area.

[0047] In the embodiment of the present disclosure, the maximum clothing circumscribed detection area refers to a clothing image area included in the image to be processed, and the maximum clothing circumscribed detection area can be represented by a set of pixel coordinates corresponding to the clothing image area.

[0048] In the embodiments of the present disclosure, there are many ways to obtain the maximum external detection area of ​​clothing corresponding to the image to be processed. As an example, the image to be processed is detected based on a pre-trained clothing detection model to obtain the maximum external detection area of ​​clothing; wherein the clothing detection model is pre-trained, specifically, image samples are obtained, each sample has a label of the maximum external detection area of ​​clothing, and a neural network such as convolution is trained based on the image samples to obtain a pre-trained clothing detection model; as another example, features are extracted from the image based on a preset image feature algorithm, and the extracted image features are classified based on a preset classification algorithm, and the image area corresponding to the image features of the service whose classification results are obtained as the maximum external detection area of ​​clothing. The above two methods are only examples of obtaining the maximum external detection area of ​​clothing corresponding to the image to be processed, and the embodiments of the present disclosure do not specifically limit the method of obtaining the maximum external detection area of ​​clothing corresponding to the image to be processed.

[0049] In the disclosed embodiment, the maximum circumscribed detection area of ​​clothing can be determined by four target pixel coordinates. The target pixel coordinates corresponding to the maximum circumscribed detection area of ​​clothing are used as prompt words and input into a general segmentation model such as the segmented arbitrary model SAM (Segment Anything Model) for segmentation with the image to be processed to obtain a set of coordinate points of the clothing enclosed area, that is, to obtain the mask result corresponding to the clothing, for example, "1" is the clothing area, "0" is the non-clothing area, and the coordinate points corresponding to the connection points of the clothing area and the non-clothing area are obtained to obtain a set of coordinate points of the clothing enclosed area.

[0050] Among them, by pre-training on a large-scale data set to obtain a general segmentation model, objects in the image can be flexibly segmented under given prompts. In the embodiment of the present disclosure, the target pixel coordinates and the image to be processed are input into the general segmentation model for segmentation, and the set of coordinate points of the clothing enclosing area is obtained. It can be understood as performing clothing cutout processing on the image corresponding to the target pixel coordinates in the image to be processed. Specifically, the target pixel coordinates are encoded, the image to be processed is encoded, and then the two encoding results are fused to obtain the mask code, and the mask code is decoded to obtain the set of coordinate points of the clothing enclosing area.

[0051] Therefore, based on the general segmentation model, the coordinate point set of the clothing enclosed area can be quickly obtained, thereby improving the efficiency and effect of subsequent image review.

[0052] In some embodiments, obtaining a set of human body key point coordinates corresponding to the image to be processed includes: obtaining a human body image area corresponding to the image to be processed, performing human body key point detection on the human body image area based on a preset human body key point detection algorithm, and obtaining a set of human body key point coordinates.

[0053] In the embodiment of the present disclosure, the human body image region refers to the human body image region included in the image to be processed, and the human body image region can be represented by a set of pixel coordinates corresponding to the human body image region.

[0054] In the embodiments of the present disclosure, there are many ways to obtain the human body image area corresponding to the image to be processed. As an example, the image to be processed is detected based on a pre-trained human body detection model to obtain the human body image area; wherein the human body detection model is pre-trained, specifically, image samples are obtained, each sample has a human body image area label, and a neural network such as convolution is trained based on the image samples to obtain a pre-trained human body detection model; as another example, features are extracted from the image based on a preset image feature algorithm, and the extracted image features are classified based on a preset classification algorithm, and the image area corresponding to the image features of the human body whose classification result is obtained as the human body image area. The above two methods are only examples of obtaining the human body image area corresponding to the image to be processed, and the embodiments of the present disclosure do not specifically limit the methods of obtaining the human body image area corresponding to the image to be processed.

[0055] Furthermore, based on a preset human key point detection algorithm, human key point detection is performed on the human image area to obtain preliminary key point detection results, and the preliminary key point detection results are adjusted based on the positional relationship between the human image area and the image to be processed to obtain a set of human key point coordinates.

[0056] Among them, the human key point detection algorithm can be trained by collecting image samples and annotating them, so that the human image area can be input into the human key point detection, and the preliminary key point detection results can be obtained, that is, the key point coordinates of each key point in the human image area. It can be understood that when performing non-overlapping calculations, each coordinate point is the pixel coordinate in the coordinate system of the corresponding size of the image to be processed. Therefore, it is necessary to adjust the preliminary key point detection results according to the positional relationship between the human image area and the image to be processed to obtain a set of human key point coordinates, thereby ensuring the accuracy of subsequent calculation results.

[0057] Therefore, based on the human body key point detection algorithm, the coordinate set of the human body key points in the image to be processed is accurately obtained, thereby improving the efficiency and effect of subsequent image review.

[0058] In some embodiments, the non-overlapping degree is determined based on a clothing enclosing area coordinate point set and a human body key point coordinate set, including: obtaining target human body key point coordinates of each human body sub-area based on the human body key point coordinate set, determining reference clothing enclosing area coordinate points of each human body sub-area from the clothing enclosing area coordinate point set based on the target human body key point coordinates, determining clothing coverage distance and clothing uncovered distance of each human body sub-area based on the target human body key point coordinates and the reference clothing enclosing area coordinate points of each human body sub-area, determining candidate non-overlapping degrees of each human body sub-area based on the clothing coverage distance and clothing uncovered distance of each human body sub-area, and obtaining non-overlapping degrees based on the candidate non-overlapping degrees of each human body sub-area.

[0059] In the embodiment of the present disclosure, the image to be processed includes various human body sub-regions, and the non-overlapping degrees of different human body sub-regions are usually different, for example Figure 2 The human key points "2", "4" and "6" are used as a human sub-region, the human key points "1", "3" and "4" are used as the first human sub-region, the human key points "2" and "8" are used as a human sub-region, and the human key points "1" and "7" are used as a human sub-region, and so on. The specific division of each human sub-region can also be selected according to the actual application needs.

[0060] Therefore, it can be understood that the coordinates of the target human body key points determined in different human body sub-regions are different, and the coordinate points of the reference clothing enclosing areas of the human body sub-regions are also different. As an example, Figure 3a As shown, based on the human body key point coordinate set, the coordinate points corresponding to the target human body key points "2", "4" and "6" can be determined. Figure 3a The "X" in the figure is the coordinate point of the reference clothing enclosing area. The clothing coverage distance is calculated based on the coordinate points corresponding to the target human key points "2", "4" and "6" and the coordinate point "X" of the reference clothing enclosing area. Figure 3a The dashed line "A" and the distance not covered by the garment are shown as Figure 3a The solid line “B” shown, the non-overlapping degree of the human body sub-region can be determined according to the clothing covered distance and the clothing uncovered distance, for example Figure 3a , calculate the total distance between the dotted line "A" and the solid line "B", and divide the dotted line "A" by the total distance as the non-overlap degree of the human body sub-region, based on Figure 3a As shown, the non-overlapping degree is greater than 0 and less than 1.

[0061] As another example, Figure 3b As shown, based on the human body key point coordinate set, the coordinate points corresponding to the target human body key points "2" and "8" can be determined. Figure 3b The human body sub-region composed of the target human body key points "2" and "8" does not contain the coordinate points of the reference clothing enclosed region. The clothing coverage distance is calculated based on the coordinate points corresponding to the target human body key points "2" and "8". Figure 3b For the solid line “C” shown, the clothing uncovered distance is 0. The non-overlapping degree of the human body sub-region can be determined according to the clothing covered distance and the clothing uncovered distance. For example Figure 3b , calculate the total distance between the solid line "C" corresponding to the clothing coverage distance and the clothing uncovered distance 0, that is, the total distance is the clothing coverage distance, and divide the clothing uncovered distance 0 by the total distance as the non-overlap degree of the human body sub-area, based on Figure 3bAs shown, the non-overlap degree is 0.

[0062] As another example, Figure 3c As shown in FIG. 1 , based on the human body key point coordinate set, the coordinate points corresponding to the target human body key points “1”, “3” and “5” are determined. The human body sub-region composed of the coordinate points corresponding to the target human body key points “1”, “3” and “5” does not include the coordinate points of the reference clothing enclosed region. Thus, according to the coordinate points corresponding to the target human body key points “1”, “3” and “5”, the distance covered by clothing is obtained as shown in FIG. Figure 3c The dashed line “D” shown, according to the clothing covered distance and clothing uncovered distance, can determine the non-overlapping degree of the human body sub-region, for example Figure 3c , calculate the total distance between the clothing uncovered distance dotted line "D" and the clothing covered distance corresponding to 0, that is, the total distance is the clothing uncovered distance, and divide the dotted line "D" by the total distance as the non-overlap degree of the human body sub-region, based on Figure 3a As shown, the non-overlap degree is 1.

[0063] It is understandable that there may be one or more human sub-regions in the image to be processed. When there is one human sub-region, the non-overlapping degree of the human sub-region is directly used as the non-overlapping degree of the image to be processed. When multiple human sub-regions are stored, the non-overlapping degree of the human sub-region is calculated to obtain the total non-overlapping degree of the image to be processed. As an example, the non-overlapping degrees of multiple human sub-regions are compared as candidate non-overlapping degrees to determine the maximum human sub-region non-overlapping degree as the final non-overlapping degree of the image to be processed. Figure 3a and Figure 3b For example, you can Figure 3a The non-overlapping degree of the corresponding human body sub-region is taken as the final non-overlapping degree of the image.

[0064] As another example, the preset weight information of each human body sub-region is obtained, and calculation is performed based on the weight information of each human body sub-region and the candidate non-overlapping degree of each human body sub-region to obtain multiple target non-overlapping degrees, and the multiple target non-overlapping degrees are compared to determine the maximum target non-overlapping degree as the non-overlapping degree.

[0065] Among them, the weight information of each human body sub-region can be set according to the gender, age and other attributes of the person in the image, and the non-overlap degree of each human body sub-region can be adjusted through the weight information to further meet the image review requirements in different scenarios.

[0066] Therefore, we can make full use of different algorithms to conduct comprehensive evaluation of images, provide more accurate and continuous non-overlapping degree representation of the dressing degree evaluation results of images, and meet the needs of accurate evaluation of the dressing degree of images in practical applications.

[0067] Figure 4The following is a flow chart of an image review method provided by an embodiment of the present disclosure. The method mainly includes the following steps S402 to S408:

[0068] Step S402, obtaining an image to be processed.

[0069] Step S404, obtaining the maximum clothing circumscribed detection area corresponding to the image to be processed, obtaining the target pixel coordinates corresponding to the maximum clothing circumscribed detection area, inputting the target pixel coordinates and the image to be processed into a general segmentation model for segmentation, and obtaining a set of coordinate points of the clothing enclosing area.

[0070] Step S406, obtaining the human body image area corresponding to the image to be processed, performing human body key point detection on the human body image area based on a preset human body key point detection algorithm to obtain preliminary key point detection results, adjusting the preliminary key point detection results based on the positional relationship between the human body image area and the image to be processed, and obtaining a set of human body key point coordinates.

[0071] Step S408, obtaining target human key point coordinates of each human sub-region based on the human key point coordinate set, and determining reference clothing enclosing area coordinate points of each human sub-region from the clothing enclosing area coordinate point set based on the target human key point coordinates.

[0072] Step S410, determining the clothing coverage distance and clothing uncovered distance of each human body sub-region based on the target human body key point coordinates of each human body sub-region and the reference clothing enclosed area coordinate points, determining the candidate non-overlapping degree of each human body sub-region based on the clothing coverage distance and clothing uncovered distance of each human body sub-region, and obtaining non-overlapping based on the candidate non-overlapping degree of each human body sub-region.

[0073] Specifically, a clothing detection model is used to identify clothing parts in the image to be processed, and the maximum circumscribed detection area of ​​the clothing is output. The clothing in the maximum circumscribed detection area of ​​the clothing is finely segmented using a general segmentation model to obtain a set of coordinate points of the clothing enclosing area (i.e., a clothing mask). Then, a human key point detection algorithm is used to obtain the positions of important key points of the human body, i.e., a set of human key point coordinates. Finally, the human key point coordinate set and the set of coordinate points of the clothing enclosing area are combined to calculate the dressing conditions of various parts of the human body, and finally a non-overlapping degree is output; wherein the non-overlapping degree is a continuous value greater than or equal to 0 and less than or equal to 1, which is used to represent the overall degree of dressing.

[0074] Exemplarily, a video to be processed is obtained, frame extraction is performed on the video to be processed to obtain each video frame, and each video frame is screened according to whether it includes a human body to obtain a key video frame, and the key video frame is resized according to a subsequent processing model or algorithm, and the resized key video frame is subjected to color normalization and other processing as the image to be processed, thereby further improving the accuracy of subsequent processing.

[0075] Furthermore, the image to be processed is detected by a pre-trained clothing detection model to obtain the maximum circumscribed detection area of ​​the clothing (i.e., a set of pixel coordinates), and the maximum circumscribed detection area of ​​the clothing is used as a prompt word and input into the general segmentation model together with the image to be processed for segmentation to obtain a set of coordinate points of the clothing enclosing area. That is, the general segmentation model performs clothing cutout processing on the image area corresponding to the maximum circumscribed detection area of ​​the clothing according to the prompt corresponding to the maximum circumscribed detection area of ​​the clothing to obtain a set of coordinate points of the clothing enclosing area.

[0076] Furthermore, the image to be processed is detected by a pre-trained human body detection model to obtain the human body image area, and the image area corresponding to the human body image area is subjected to key point detection by a preset human body key point detection algorithm, so that preliminary key point detection results (i.e., the coordinates of each preliminary key point) can be obtained and converted into a set of human body key point coordinates (i.e., the human body key point coordinates consistent with the image to be processed) according to the positional relationship between the human body image area and the image to be processed (coordinate transformation matrix).

[0077] Furthermore, the target human body key point coordinates corresponding to different human body sub-regions can be determined according to the human body key point coordinate set, and the reference clothing enclosing area coordinate points of the boundary between clothing and the human body can be determined according to the clothing enclosing area coordinate point set, so that the non-overlapping degrees corresponding to different human body sub-regions can be obtained according to the target human body key point coordinates and the reference clothing enclosing area coordinate points. Generally, the larger the non-overlapping degree, the larger the area of ​​the human body outside the clothing. Therefore, the audit values ​​of multiple human body sub-regions can be compared to determine the maximum human body sub-region non-overlapping degree as the final non-overlapping degree of the image to be processed.

[0078] It can also be understood that different genders and ages of human body regions may have different reviews of the degree of clothing, and the weight information of the human body sub-regions can be adjusted according to target attributes such as gender and age, so that the weight information is combined with the non-overlapping degree of the human body sub-regions for calculation and sorting, and the adjusted maximum non-overlapping degree of the human body region is used as the final non-overlapping degree of the image to be processed.

[0079] The disclosed embodiment achieves a more accurate continuous value assessment of a person's degree of clothing by comprehensively applying the advantages of a clothing detection model, a general segmentation model, and a human key point detection algorithm.

[0080] As an example scenario, Figure 5 As shown, step 5.1 inputs an image; step 5.2 preprocesses the image and outputs the image to be processed; step 5.3 performs clothing detection on the image to be processed to obtain the maximum external detection area of ​​the clothing; step 5.4 segments the maximum external detection area of ​​the clothing to obtain a set of coordinate points of the clothing enclosed area (i.e., obtains a clothing mask); step 5.5 performs human key point detection on the image to be processed to obtain a set of coordinate points of the human key points; step 5.6 obtains a continuous non-overlapping degree by analyzing the spatial relationship between the key points and the overlap of the coordinate point set of the clothing enclosed area.

[0081] Specifically, the input image is resized and color normalized to adapt to the input requirements of the subsequent model; a pre-trained clothing detection model is used to perform clothing detection on the preprocessed image, and the maximum external detection area of ​​the clothing is output; the maximum external detection area of ​​the clothing is input as a prompt to the general segmentation model to obtain a set of coordinate points of the clothing enclosed area; a human key point detection algorithm is used to obtain important key points of the human body; the human key points and the set of coordinate points of the clothing enclosed area are combined, and the continuous value of the degree of dressing is calculated and the continuous value (non-overlapping degree) is output by analyzing the spatial relationship between the key points and the overlap of the set of coordinate points of the clothing enclosed area; the non-overlapping degree is a continuous value greater than or equal to 1 and less than or equal to 1, which is used to represent the overall degree of dressing of the person in the image.

[0082] In summary, the image audit method provided by the embodiment of the present disclosure comprehensively evaluates the image by making full use of the advantages of different algorithms. Based on the obtained non-overlapping degree, it can reflect more fine-grained dressing conditions and improve the application value of the audit results by providing more accurate and continuous dressing degree values.

[0083] Corresponding to the aforementioned image review method, the present disclosure further provides an image review device, Figure 6 This is a schematic diagram of the structure of an image review device provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and is applied to electronic devices. The device includes:

[0084] The first acquisition module 602 is used to acquire the image to be processed.

[0085] The second acquisition module 604 is used to acquire a set of coordinate points of a clothing enclosing area corresponding to the image to be processed.

[0086] The third acquisition module 606 is used to acquire a set of human body key point coordinates corresponding to the image to be processed.

[0087] The calculation module 608 is used to determine the non-overlapping degree based on the coordinate point set of the clothing enclosed area and the coordinate point set of the human body key points.

[0088] The determination module 610 is used to determine the image review result of the image to be processed based on the non-overlapping degree.

[0089] The above-mentioned device provided by the embodiment of the present disclosure can accurately evaluate the degree of dressing of the image by combining the coordinate point set of the clothing enclosed area and the coordinate set of the key points of the human body to calculate the accurate non-overlapping degree to represent the audit result of the image, thereby improving the efficiency and effect of image audit. Compared with the prior art method of simply dividing clothing into two categories by discrete classification, the above-mentioned method can provide a more accurate and continuous non-overlapping degree to represent the evaluation result of the degree of dressing of the image, meeting the demand for accurate evaluation of the degree of dressing of the image in practical applications.

[0090] In some embodiments, the second acquisition module 604 is specifically used to: detect the image to be processed based on a pre-trained clothing detection model to obtain the maximum external detection area of ​​the clothing; obtain the target pixel coordinates corresponding to the maximum external detection area of ​​the clothing; input the target pixel coordinates and the image to be processed into a general segmentation model for segmentation to obtain a set of coordinate points of the clothing enclosed area.

[0091] In some embodiments, the third acquisition module 606 is specifically used to: detect the image to be processed based on a pre-trained human body detection model to obtain the human body image area; perform human body key point detection on the human body image area based on a preset human body key point detection algorithm to obtain the human body key point coordinate set.

[0092] In some embodiments, the calculation module 608 includes: a first determination unit, used to obtain the target human body key point coordinates of each human body sub-region based on the human body key point coordinate set; a second determination unit, used to determine the reference clothing enclosing area coordinate points of each human body sub-region from the clothing enclosing area coordinate point set based on the target human body key point coordinates; a first calculation unit, used to determine the clothing coverage distance and clothing uncovered distance of each human body sub-region based on the target human body key point coordinates and the reference clothing enclosing area coordinate points of each human body sub-region; a second calculation unit, used to determine the candidate non-overlapping degree of each human body sub-region based on the clothing coverage distance and the clothing uncovered distance of each human body sub-region; and a processing unit, used to obtain the non-overlapping degree based on the candidate non-overlapping degree of each human body sub-region.

[0093] In some embodiments, the second calculation unit is specifically used to: determine the total distance of each human body sub-region based on the clothing coverage distance and the clothing uncovered distance of each human body sub-region; determine the ratio of the clothing uncovered distance of each human body sub-region to the total distance as the candidate non-overlapping degree of each human body sub-region.

[0094] In some implementations, the processing unit is specifically configured to: compare the candidate non-overlapping degrees of the various human body sub-regions to determine a maximum candidate non-overlapping degree as the non-overlapping degree.

[0095] In some embodiments, the second calculation unit is specifically used to: obtain preset weight information of each human body sub-region; calculate based on the weight information of each human body sub-region and the candidate non-overlapping degree of each of the human body sub-region to obtain multiple target non-overlapping degrees; compare the multiple target non-overlapping degrees to determine the maximum target non-overlapping degree as the non-overlapping degree.

[0096] The image review device provided in the embodiments of the present disclosure can execute the image review method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0097] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device embodiment can refer to the corresponding process in the method embodiment, and will not be repeated here.

[0098] An embodiment of the present disclosure provides an electronic device, which includes: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to implement the steps of any method in the present disclosure.

[0099] Reference below Figure 7 , which shows a schematic diagram of the structure of an electronic device 700 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0100] like Figure 7As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0101] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0102] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0103] In addition to the above-mentioned methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, cause the processor to execute the image review method provided by the embodiments of the present disclosure. The computer program product may be written in any combination of one or more programming languages ​​to write program codes for executing the operations of the embodiments of the present disclosure, the programming languages ​​including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on a user computing device, partially on a user device, as a separate software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0104] In addition, the embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the image review method provided by the embodiment of the present disclosure.

[0105] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0106] The present disclosure embodiment also provides a computer program product, including a computer program / instruction, which implements the image review method in the present disclosure embodiment when executed by a processor.

[0107] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0108] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0109] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0110] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0111] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0112] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image review method, characterized in that: The method comprises: Get the image to be processed; Obtaining a set of coordinate points of a clothing enclosing area and a set of coordinate points of a human body corresponding to the image to be processed; Determine the non-overlapping degree based on the clothing enclosed area coordinate point set and the human body key point coordinate set; An image review result of the image to be processed is determined based on the non-overlapping degree.

2. The method according to claim 1, characterized in that The step of obtaining a set of coordinate points of a clothing enclosing area corresponding to the image to be processed includes: Obtain the maximum external detection area of ​​clothing corresponding to the image to be processed; Obtaining the target pixel coordinates corresponding to the maximum circumscribed detection area of ​​the clothing; The target pixel coordinates and the image to be processed are input into a general segmentation model for segmentation to obtain a set of coordinate points of the clothing enclosing area.

3. The method according to claim 1, characterized in that The step of obtaining a set of human body key point coordinates corresponding to the image to be processed includes: Obtaining a human body image region corresponding to the image to be processed; Based on a preset human key point detection algorithm, human key point detection is performed on the human image area to obtain the human key point coordinate set.

4. The method according to claim 1, characterized in that: The determining of the non-overlapping degree based on the clothing enclosed area coordinate point set and the human body key point coordinate set includes: Acquire target human body key point coordinates of each human body sub-region based on the human body key point coordinate set; Determine reference clothing enclosing area coordinate points of each human body sub-area from the clothing enclosing area coordinate point set based on the target human body key point coordinates; Determine the clothing coverage distance and clothing uncovered distance of each human body sub-region based on the target human body key point coordinates of each human body sub-region and the reference clothing enclosed region coordinate points; Determine candidate non-overlapping degrees of each human body sub-region based on the clothing coverage distance and the clothing uncovered distance of each human body sub-region; The non-overlapping degree is obtained based on the candidate non-overlapping degrees of each human body sub-region.

5. The method according to claim 4, characterized in that The determining of the candidate non-overlapping degrees of the respective human body sub-regions based on the clothing coverage distances and the clothing uncovered distances of the respective human body sub-regions comprises: Determine the total distance of each human body sub-region based on the clothing covered distance and the clothing uncovered distance of each human body sub-region; A ratio of the clothing uncovered distance of each human body sub-region to the total distance is determined as a candidate non-overlapping degree of each human body sub-region.

6. The method according to claim 4, characterized in that The obtaining the non-overlapping degree based on the candidate non-overlapping degrees of each human body sub-region includes: The candidate non-overlapping degrees of the human body sub-regions are compared to determine the maximum candidate non-overlapping degree as the non-overlapping degree.

7. The method according to claim 4, characterized in that The obtaining the non-overlapping degree based on the candidate non-overlapping degrees of each human body sub-region includes: Obtaining preset weight information of each human body sub-region; Calculating based on the weight information of each human body sub-region and the candidate non-overlapping degree of each human body sub-region to obtain multiple target non-overlapping degrees; The multiple target non-overlapping degrees are compared to determine the maximum target non-overlapping degree as the non-overlapping degree.

8. An image review device, characterized in that: The device comprises: A first acquisition module, used for acquiring an image to be processed; A second acquisition module is used to acquire a set of coordinate points of a clothing enclosing area corresponding to the image to be processed; A third acquisition module is used to obtain a set of human body key point coordinates corresponding to the image to be processed; A calculation module, used for determining the non-overlapping degree based on the coordinate point set of the clothing enclosed area and the coordinate point set of the human body key points; A determination module is used to determine the image review result of the image to be processed based on the non-overlapping degree.

9. An electronic device, characterized in that: The electronic device comprises: a storage device having a computer program stored thereon; A processing device is used to execute the computer program in the storage device to implement the steps of the image review method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the image review method described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Personnel illegal wearing detection method, device and equipment and storage medium

    CN115223204A

  • Image classification method, image classification model training method, device and equipment

    CN115393643A

  • Method for defining garment type of vehicle occupant

    CN117546213A

  • Method and apparatus for garment color identification of figure images, and electronic device

    WO2019096174A1