Method, device, electronic device and storage medium for determining human body occlusion degree

Abstract

The present application provides a method, device, electronic device, and storage medium for determining the degree of human body occlusion, belonging to the technical field of data processing. The method comprises extracting an image to be processed from a target video; generating a corresponding human body region frame for each object in the image to be processed; determining a plurality of key points corresponding to each object in the image to be processed, wherein the plurality of key points corresponding to each object are used to indicate a plurality of key parts of the human body of each object; and determining the degree of occlusion of each object based on the number of key points of other objects contained in the human body region frame of each object. The present application aims to accurately determine the degree of occlusion of the human body.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of data processing, and specifically, to a method, device, electronic device, and storage medium for determining the degree of human body occlusion. Background Art

[0002] In the processing scenario of film and television data, the completeness of the human characters in the picture affects the accuracy of subsequent processing. For example, in the advertising business derived from film and television data, purchase links for the same clothing worn by the characters in the film and television can be pushed to users. However, due to the different postures and positions of the characters, the clothing will be presented differently in different shots. When different characters in the picture are very close or there is occlusion between multiple characters, it will introduce a lot of noise to the subsequent clothing recognition and matching processes.

[0003] Therefore, before conducting a secondary analysis of the characters in the film and television images, it is necessary to determine whether the characters in the image are blocked. By promptly processing the blocked characters, the noise in the subsequent processing process can be effectively reduced and the accuracy of the subsequent processing process can be improved.

[0004] Currently, there are two methods to determine whether there is human body occlusion in film and television images: manual screening and classification algorithms. However, manual screening is inefficient, and classification algorithms cannot quantify the occlusion of the human body, resulting in low judgment accuracy. Summary of the Invention

[0005] The embodiments of the present application provide a method, device, electronic device, and storage medium for determining the degree of human body occlusion, aiming to accurately determine the degree of human body occlusion.

[0006] In a first aspect, an embodiment of the present application provides a method for determining a degree of human body occlusion, the method comprising:

[0007] Extract the image to be processed from the target video;

[0008] Generating a corresponding human body region frame for each object in the image to be processed;

[0009] Determining a plurality of key points corresponding to each object in the image to be processed, respectively, wherein the plurality of key points corresponding to each object are used to indicate a plurality of key parts of a human body of each object;

[0010] The occlusion degree of each object is determined according to the number of key points of other objects contained in the human body region frame of each object.

[0011] Optionally, generating a corresponding human body region frame for each object in the image to be processed includes:

[0012] Presetting a plurality of anchor frames of different sizes on the feature map of the image to be processed;

[0013] Correcting the positions of the multiple anchor frames of different sizes based on the real human body region, and determining a first confidence level that each anchor frame contains an object;

[0014] Determine an intersection-over-union ratio between any two anchor frames among the plurality of anchor frames whose first confidence score is greater than a first threshold;

[0015] When the intersection-over-union ratio between any two anchor frames is greater than a second threshold, retain the anchor frame with a higher first confidence among the any two anchor frames, so that each object corresponds to a unique anchor frame as a human body region frame.

[0016] Optionally, correcting the positions of the multiple anchor frames of different sizes based on the real human body region includes:

[0017] For any anchor box, determine the offset between the anchor box and any real human body area;

[0018] The position of each anchor frame is corrected according to the preset position of each anchor frame and the minimum offset.

[0019] Optionally, determining a plurality of key points corresponding to each object in the image to be processed includes:

[0020] generating a human body feature map for each object in the image to be processed according to the image to be processed;

[0021] Generating a heat map corresponding to each key point of each object according to the human body feature map of each object;

[0022] Determining, based on the thermal value in the thermal map corresponding to each key point of each object, a coordinate position and a second confidence level of each key point of each object, wherein the second confidence level is used to characterize the reliability of each key point in the image to be processed;

[0023] Key points whose second confidence level is less than a second threshold are eliminated.

[0024] Optionally, after determining the occlusion degree of each object based on the number of key points of other objects contained in the human body region frame of each object, the method further includes:

[0025] determining the posture of the object according to a positional relationship between a plurality of key points of the object whose occlusion degree is less than an occlusion threshold;

[0026] When the posture of the object is a frontal posture, matching the clothing image of the object with product images in the clothing product library;

[0027] A purchase link for clothing identical to and / or similar to the clothing image is displayed in the target video.

[0028] Optionally, displaying a purchase link for clothing identical and / or similar to the clothing image in the target video includes:

[0029] A pop-up window is displayed within a predetermined time period starting from the frame where the image to be processed in the target video is located. The pop-up window includes a purchase link for clothing that is identical and / or similar to the clothing image.

[0030] Optionally, displaying a purchase link for clothing identical and / or similar to the clothing image in the target video includes:

[0031] A bullet comment is displayed in the frame where the image to be processed in the target video is located, and the bullet comment includes purchase links for clothing that is the same as and / or similar to the clothing image.

[0032] In a second aspect, an embodiment of the present application provides a device for determining a degree of human body obstruction, the device comprising:

[0033] An extraction module, used to extract the image to be processed from the target video;

[0034] A human body region frame determination module, configured to generate a human body region frame corresponding to each object in the image to be processed;

[0035] A key point determination module is used to respectively determine a plurality of key points corresponding to each object in the image to be processed, wherein the plurality of key points corresponding to each object are used to indicate a plurality of key parts of the human body of each object;

[0036] The occlusion degree determination module is used to determine the occlusion degree of each object according to the number of key points of other objects contained in the human body region frame of each object.

[0037] In a third aspect, an embodiment of the present application provides an electronic device comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the method for determining the degree of human body occlusion as described in the first aspect of the embodiment.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for determining the degree of human body occlusion as described in the first aspect of the embodiment is implemented.

[0039] Beneficial effects:

[0040] First, the image to be processed is extracted from the target video. Then, a corresponding human body region frame is generated for each object in the image to be processed. Then, multiple key points corresponding to each object in the image to be processed are determined respectively. The multiple key points represent multiple key parts of the human body of each object. The degree of occlusion of each object is determined according to the number of key points of other objects contained in the human body region frame of each object.

[0041] This method determines the degree of occlusion by determining the number of key points of other objects contained in the human body region frame of an object. When there are multiple people occluding each other in the picture, the various occlusion situations of the multiple people can be quantified. Compared with manually determining the occlusion degree of the object, the efficiency and screening quality are improved, and compared with the classification algorithm, the accuracy of the occlusion degree can be improved. At the same time, based on the more accurate occlusion degree of the person in the picture, a representative image of the person that better meets the actual business requirements can be selected in the target video, thereby improving the accuracy of the subsequent business algorithm results. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0043] Figure 1 This is a flowchart of the steps of the method for determining the degree of human body occlusion proposed in one embodiment of the present application;

[0044] Figure 2 This is a schematic diagram of the process of generating key points proposed in one embodiment of the present application;

[0045] Figure 3 This is a schematic diagram of the distribution of multiple key points of each object provided by an embodiment of the present application;

[0046] Figure 4 A schematic diagram of an occlusion scenario provided by an embodiment of the present application;

[0047] Figure 5 An execution flow chart of a method for determining a degree of human body occlusion provided in an embodiment of the present application;

[0048] Figure 6 This is a functional module diagram of a device for determining the degree of human body occlusion provided in one embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0050] In businesses derived from film and television data, it is necessary to extract representative images of characters from the film and television screen for data processing. For example, in advertising businesses that push purchase links for the same clothing worn by characters in film and television, if multiple characters are close to each other in a film and television screen, such as in hugging scenes, the characters may overlap or be blocked in the screen. The overlapping or blocked characters will bring a lot of noise to the recognition and matching of clothing in subsequent businesses, which may easily lead to inaccurate processing results or even processing failure.

[0051] Therefore, it is necessary to determine the occlusion status of the characters in the picture in advance, and then determine whether to continue to process the severely occluded characters according to different business needs. The existing methods for determining whether there is occlusion of the human body in film and television pictures include manual screening and classification algorithms. However, manual screening is inefficient, and the quality of the determination results is unstable due to the different subjective judgment standards of each person; the classification algorithm cannot quantify the occlusion status of the human body, and there is a problem of low judgment accuracy, which in turn affects the processing accuracy of the entire business.

[0052] In order to more accurately determine whether a person is obscured, an embodiment of the present application provides a method for determining the degree of human body obstruction.

[0053] Reference Figure 1 , shows a flowchart of a method for determining a degree of human body obstruction in an embodiment of the present application, and the method may specifically include the following steps:

[0054] S101: Extracting an image to be processed from a target video.

[0055] In actual applications, different extraction methods can be used according to different application businesses. For example, in the clothing recommendation business, multiple images of people wearing different clothes can be selected, or images of popular clothes can be selected, or images of classic scenes can be selected according to the plot. This embodiment does not limit the extraction method of the image to be processed.

[0056] S102: Generate a corresponding human body region frame for each object in the image to be processed.

[0057] Specifically, the image to be processed is first scaled to a fixed size, and then a convolutional neural network is used to extract features of the image to be processed to generate a feature map of the image to be processed; then, multiple anchor frames of different sizes are preset on the feature map of the image to be processed. For example, in order to include as many human objects of different sizes in the feature map as possible, 9 anchor frames of different sizes can be preset for each pixel point on the feature map. For example, if the feature map is 100x100 in length and width, and 9 anchor frames of different sizes are preset for each pixel point, then the feature map will have a total of 100x100x9 anchor frames.

[0058] However, not every anchor frame contains a human object, and even if some anchor frames contain human objects, they are not complete. Therefore, it is necessary to determine the first confidence level of each anchor frame of different sizes that contains an object, and also to determine the offset between multiple anchor frames of different sizes and the real human object.

[0059] In the actual implementation process, the image to be processed can be input into the trained human body region frame determination model, which is a model trained with images carrying manually labeled frames. During the training process, the human body region frame determination model can generate human body prediction frames. Each human body prediction frame corresponds to a first confidence level, which is used to characterize the reliability of the human object contained in the human body prediction frame. At the same time, the human body region frame determination model can also continuously correct the offset size of the human body prediction frame according to the position of the manually labeled frame, so that the position of the human body prediction frame is close to the position of the manually labeled frame. Since the coordinate position of each anchor frame is known, each anchor frame after position correction can be obtained according to the prior position and offset corresponding to each anchor frame.

[0060] Since each anchor frame carries its corresponding first confidence, the size of the first confidence and the first threshold is judged. When the first confidence is less than the first threshold, it can be indicated that the reliability of the human object contained in this anchor frame is low, and the anchor frame with the first confidence less than the first threshold can be eliminated.

[0061] For anchor frames with a first confidence greater than or equal to a first threshold, due to the different sizes of the anchor frames, multiple anchor frames may contain the same object at the same time. In order to generate only one human region frame for a person object, this embodiment uses a non-maximum suppression (NMS) algorithm to calculate the intersection over union (IOU) of all anchor frames with a first confidence greater than or equal to the first threshold in the feature map, and calculates the IOU between any two anchor frames respectively. When the IOU between any two anchor frames is greater than a second threshold, it indicates that the two anchor frames contain the same object. Therefore, among the two anchor frames with an IOU greater than the second threshold, the anchor frame with a higher first confidence is retained. By traversing the IOUs of any two anchor frames with a first confidence greater than or equal to the first threshold, each object can have only one anchor frame as the human region frame.

[0062] This embodiment adopts an anchor frame-based target detection method in the process of determining the human body region frame. In other embodiments, other methods may be used to determine the human body region frame of each object.

[0063] S103: Determine a plurality of key points corresponding to each object in the image to be processed, respectively. The plurality of key points corresponding to each object are used to indicate a plurality of key parts of the human body of each object.

[0064] Reference Figure 2 , shows a schematic diagram of the process of generating key points provided by an embodiment of the present application. In a feasible implementation, when determining multiple key parts of the human body of each object, first, a human body feature map of each object in the image to be processed is generated based on the image to be processed.

[0065] like Figure 2 In actual implementation, image recognition can be performed on both the people and objects in the processed image, and then a feature map corresponding to each person and each object can be generated separately. Then, the key points corresponding to each key part of the human body of the person object are determined, such as person 1 and person 2. However, key points cannot be generated for the objects in the image.

[0066] In order to improve processing efficiency, only the human body position of each object in the image to be processed can be extracted, that is, only the feature map is generated for the human object, reducing the interference of other non-object graphics and reducing the computational cost; in this embodiment, the image information within the human body area frame of each object can be extracted, and then feature extraction is performed through a convolutional neural network to obtain a human body feature map of each object, and then, for each object's human body feature map, multiple key points corresponding to each object are determined respectively.

[0067] In a feasible implementation, when determining multiple human key points of each object, a heat map corresponding to each key point can be generated based on the human feature map. The process of generating the heat map is actually feature matching in the spatial dimension. The convolution kernel "slides" on the feature map plane, paying more attention to and utilizing local information. Therefore, the calculation of each key point is independent, that is, a convolution kernel is used to perform calculations on the feature map in sequence to see whether each local area meets the characteristics of the key point.

[0068] Specifically, each type of coordinate is represented by a probability graph. Each pixel position in the image corresponds to a thermal value, which is used to indicate the probability that the point belongs to the corresponding key point. The closer the pixel is to the key point, the closer the thermal value is to 1, and the farther the pixel is from the key point, the closer the thermal value is to 0. From a visualization perspective, the closer the pixel is to the key point, the brighter the color.

[0069] Reference Figure 3 , a distribution diagram of multiple key points of each object is shown. The number of key points of each object is fixed. In this embodiment, the number of key points of each object is 22. The 22 key points can describe 22 key parts of the human body, including: top of the head, left ear, right ear, left eye, right eye, nose, left shoulder, right shoulder, left elbow, left wrist, right elbow, right wrist, left hip, right hip, left knee, left ankle, right knee, right ankle, left chest, right chest and navel.

[0070] After generating a heat map corresponding to each key point of each object, the coordinate position and the second confidence level of each key point of each object are determined based on the heat value in the heat map corresponding to each key point of each object. The second confidence level can characterize the reliability of the existence of each key point in the image to be processed.

[0071] For example, in the heat map corresponding to any key point, the coordinate position with the highest heat value and the coordinate position with the second highest heat value are weighted averaged to obtain the coordinate position and the second confidence of the key point, and then the key points with the second confidence less than the second threshold are eliminated. For example, if only the upper body of an object appears, the second confidence of the key points of the lower body is less than the second threshold. After eliminating the key points of the lower body that are less than the second threshold, the remaining key points can represent the body area presented by the object in the image to be processed.

[0072] In the actual implementation process, when determining key points based on heat maps, the image to be processed can be input into a trained human key point algorithm model, and the human key point algorithm model can output the key points of each object in the image to be processed; in addition to determining key points based on heat maps, other methods can also be used in other embodiments to determine multiple key points of each object.

[0073] S104: Determine the occlusion degree of each object according to the number of key points of other objects contained in the human body region frame of each object.

[0074] In this embodiment, the number of key points of other objects contained in the human body region frame of an object can be counted. For each additional key point of another object, the occlusion degree of the object is increased by one point. For example, there are 2 key points of object 2 and 2 key points of object 3 in the human body region frame of object 1, then the occlusion degree of object 1 is 4 points. In this embodiment, the occlusion degree is specifically defined as a score corresponding to the number of key points. In other embodiments, the occlusion degree can also be quantified into other data forms. For example, the proportion of the key points of other objects in the human body region frame of object 1 can be calculated.

[0075] In a feasible embodiment, an occlusion threshold is set. When the occlusion degree of any object is greater than the occlusion threshold, it indicates that the object is mostly occluded and is not suitable for subsequent business processing. Therefore, the person area frames corresponding to the objects with occlusion degrees greater than the occlusion threshold can be eliminated, and only the person area frames corresponding to the objects with occlusion degrees less than or equal to the occlusion threshold are retained. For example, in the clothing recommendation business, most of the occluded people are used as objects for clothing recognition, and it is easy to obtain recognition results with large errors. By screening out objects with high occlusion degrees and retaining objects with high integrity in the image for clothing recognition, resources for subsequent processing can be saved and the processing accuracy of the entire subsequent business can be improved.

[0076] Reference Figure 4 , shows a schematic diagram of an occlusion scene provided by an embodiment of the present application, Figure 4 The solid line frame is the human body area frame of character A, and the dotted line frame is the human body area frame of character B. Although it can be seen intuitively that characters A and B are blocking each other because they are standing very close to each other, the method provided in this embodiment can quantify the degree of occlusion and obtain a more accurate occlusion degree judgment result. Specifically, in the human body area frame of character A (solid line frame), there are 13 key points of character B; in the human body area frame of character B (dotted line frame), there are 7 key points of character A. The occlusion degree of character A is 13 points, and the occlusion degree of character B is 7 points.

[0077] In other implementations, the occlusion degree of the object may be graded by setting multiple different occlusion thresholds, and different processing processes may be performed on objects at different occlusion levels. Specifically, different occlusion thresholds may be set according to different subsequent application services.

[0078] Reference Figure 5 , shows an execution flow chart of the method for determining the degree of human body occlusion provided by an embodiment of the present application, such as Figure 5 In a feasible implementation, when determining the degree of occlusion of a human body in an image, first, video frame extraction is performed to determine an image to be processed from a target video, and then it is detected whether there is a human body in the image to be processed. If there is no human body in the image to be processed, a new image to be processed is extracted from the target video again; if there is a human body in the image to be processed, a human body region frame corresponding to each human body is determined, and multiple key points corresponding to each human body are determined; then, based on the number of key points of other human bodies contained in each human body region frame, the degree of occlusion of each human body is determined, that is, the occlusion score of each human body is obtained; finally, the occlusion score of the human body is compared to see whether it is greater than an occlusion threshold. If so, the human body region frame corresponding to the human body is removed, otherwise, the human body region frame is retained.

[0079] In a feasible implementation, in the clothing recommendation service, after determining the degree of occlusion of each object, the posture of the object can be determined based on the positional relationship between multiple key points of the object whose occlusion degree is less than or equal to the occlusion threshold. For example, when the object has only the left ear key point, the distance between the left shoulder and right shoulder key points is less than the first calibration value, and the distance between the left hip and right hip key points is less than the second calibration value, the object is characterized as being in a side-standing posture.

[0080] For clothing recommendation services, the most representative clothing images need to be selected for matching. Therefore, when the subject's posture is frontal, the clothing presentation effect is best. The clothing image of the subject in the frontal posture can be matched with the product images in the clothing product library, and then the purchase links of clothing that are the same and / or similar to the clothing image can be displayed in the target video.

[0081] When displaying purchase links for clothing that is identical and / or similar to the clothing image in the target video, a pop-up window can be displayed within a calibrated time period starting from the frame where the image to be processed is located in the target video, for example, the pop-up window can be continuously displayed for one minute, and purchase links for clothing that are identical and / or similar to the clothing image can be displayed through the pop-up window, or a barrage can be displayed in the frame where the image to be processed is located in the target video, and purchase links for clothing that are identical and / or similar to the clothing image can be presented through the barrage.

[0082] In actual implementation, clothing can also be recommended based on the body area represented by the key points of the object. For example, by locating all the key points of the upper body, the clothing image of the top can be extracted separately for matching, and the purchase link of the clothing can be displayed.

[0083] The occlusion degree determination method provided in this embodiment quantifies the occlusion degree of the object in the image, which significantly improves the efficiency and screening quality compared to manually determining the occlusion degree of the object, and can improve the accuracy of the occlusion degree compared to the classification algorithm; after determining the occlusion degree of the object, an occlusion threshold can be set according to actual business needs, and subsequent business processing can be performed on objects with an occlusion degree less than the occlusion threshold, which can improve the success rate and quality of the processing results and reduce the noise and resource consumption caused by objects with a larger occlusion degree in subsequent processing.

[0084] Reference Figure 6 , shows a functional module diagram of a device for determining a degree of human body obstruction in an embodiment of the present application, the device comprising:

[0085] The extraction module 100 is used to extract the image to be processed in the target video;

[0086] A human body region frame determination module 200 is configured to generate a human body region frame corresponding to each object in the image to be processed;

[0087] A key point determination module 300 is configured to respectively determine a plurality of key points corresponding to each object in the image to be processed, wherein the plurality of key points corresponding to each object are used to indicate a plurality of key parts of the human body of each object;

[0088] The occlusion degree determination module 400 is configured to determine the occlusion degree of each object according to the number of key points of other objects contained in the human body region frame of each object.

[0089] Optionally, the human body region frame determination module includes:

[0090] A preset anchor frame unit, configured to preset a plurality of anchor frames of different sizes on the feature map of the image to be processed;

[0091] a first determining unit, configured to correct positions of the plurality of anchor frames of different sizes based on a real human body region, and determine a first confidence level that each anchor frame contains an object;

[0092] The human body region frame determination unit is configured to determine an intersection-and-union ratio (IoU) between any two anchor frames among multiple anchor frames whose first confidence is greater than a first threshold; when the IoU between any two anchor frames is greater than a second threshold, retain the anchor frame with a higher first confidence among the any two anchor frames, so that each object corresponds to a unique anchor frame as a human body region frame.

[0093] Optionally, the first determining unit includes:

[0094] An offset determination subunit, configured to determine, for any anchor frame, an offset between the anchor frame and any real human body region;

[0095] The correction subunit is configured to correct the position of each anchor frame according to the preset position of each anchor frame and the minimum offset.

[0096] Optionally, the key point determination module includes:

[0097] A feature map generating unit, configured to generate a human body feature map of each object in the image to be processed based on the image to be processed;

[0098] a heat map generating unit, configured to generate a heat map corresponding to each key point of each object according to the human body feature map of each object;

[0099] The second determination unit is used to determine the coordinate position and the second confidence of each key point of each object according to the thermal value in the thermal map corresponding to each key point of each object, wherein the second confidence is used to characterize the reliability of each key point in the image to be processed; and eliminate the key points whose second confidence is less than the second threshold.

[0100] Optionally, the device further comprises:

[0101] a posture determination module, configured to determine the posture of the object based on a positional relationship between a plurality of key points of the object whose occlusion degree is less than an occlusion threshold;

[0102] a matching module, configured to match the clothing image of the object with product images in a clothing product library when the posture of the object is a frontal posture;

[0103] The purchase link display module is used to display purchase links of clothing that is the same as and / or similar to the clothing image in the target video.

[0104] Optionally, the purchase link display module includes:

[0105] The first display unit is configured to display a pop-up window within a predetermined time period starting from a frame where the image to be processed in the target video is located, wherein the pop-up window includes a purchase link for clothing that is identical to and / or similar to the clothing image.

[0106] Optionally, the purchase link display module includes:

[0107] The second display unit is configured to display a bullet screen in a frame where the image to be processed in the target video is located, wherein the bullet screen includes purchase links for clothing that is identical to and / or similar to the clothing image.

[0108] An embodiment of the present application also provides an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the method for determining the degree of human body occlusion as described in the embodiment.

[0109] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for determining the degree of human body occlusion described in the embodiment is implemented.

[0110] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0111] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0112] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0113] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal device. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0115] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0116] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0117] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for determining the degree of human body occlusion, characterized in that: The method comprises: Extract the image to be processed from the target video; Generating a corresponding human body region frame for each object in the image to be processed; Determining a plurality of key points corresponding to each object in the image to be processed, respectively, wherein the plurality of key points corresponding to each object are used to indicate a plurality of key parts of a human body of each object; Determining an occlusion degree of each object according to the number of key points of other objects contained in the human body region frame of each object; the occlusion degree is a score corresponding to the number of key points of other objects contained in the human body region frame of each object, or a proportion of the key points of other objects in the human body region frame of each object; determining the posture of the object according to a positional relationship between a plurality of key points of the object whose occlusion degree is less than an occlusion threshold; When the posture of the object is a frontal posture, matching the clothing image of the object with product images in the clothing product library; A purchase link for clothing identical to and / or similar to the clothing image is displayed in the target video.

2. The method according to claim 1, characterized in that Generating a corresponding human body region frame for each object in the image to be processed, including: Presetting a plurality of anchor frames of different sizes on the feature map of the image to be processed; Correcting the positions of the multiple anchor frames of different sizes based on the real human body region, and determining a first confidence level that each anchor frame contains an object; Determine an intersection-over-union ratio between any two anchor frames among the plurality of anchor frames whose first confidence score is greater than a first threshold; When the intersection-over-union ratio between any two anchor frames is greater than a second threshold, retain the anchor frame with a higher first confidence among the any two anchor frames, so that each object corresponds to a unique anchor frame as a human body region frame.

3. The method according to claim 2, characterized in that Correcting the positions of the multiple anchor frames of different sizes based on the real human body region includes: For any anchor box, determine the offset between the anchor box and any real human body area; The position of each anchor frame is corrected according to the preset position of each anchor frame and the minimum offset.

4. The method according to any one of claims 1 to 3, characterized in that: Determining a plurality of key points corresponding to each object in the image to be processed, including: generating a human body feature map for each object in the image to be processed according to the image to be processed; Generating a heat map corresponding to each key point of each object according to the human body feature map of each object; Determining, based on the thermal value in the thermal map corresponding to each key point of each object, a coordinate position and a second confidence level of each key point of each object, wherein the second confidence level is used to characterize the reliability of each key point in the image to be processed; Key points whose second confidence level is less than a second threshold are eliminated.

5. The method according to claim 1, wherein Display purchase links for clothing identical and / or similar to the clothing image in the target video, including: A pop-up window is displayed within a predetermined time period starting from the frame where the image to be processed in the target video is located. The pop-up window includes a purchase link for clothing that is identical and / or similar to the clothing image.

6. The method according to claim 1, wherein Display purchase links for clothing identical and / or similar to the clothing image in the target video, including: A bullet comment is displayed in the frame where the image to be processed in the target video is located, and the bullet comment includes purchase links for clothing that is the same as and / or similar to the clothing image.

7. A device for determining the degree of human body occlusion, characterized in that: The device comprises: An extraction module, used to extract the image to be processed from the target video; A human body region frame determination module, configured to generate a human body region frame corresponding to each object in the image to be processed; A key point determination module is used to respectively determine a plurality of key points corresponding to each object in the image to be processed, wherein the plurality of key points corresponding to each object are used to indicate a plurality of key parts of the human body of each object; an occlusion degree determination module, configured to determine an occlusion degree of each object based on the number of key points of other objects contained in the human body region frame of each object; the occlusion degree being a score corresponding to the number of key points of other objects contained in the human body region frame of each object, or a ratio of the number of key points of other objects contained in the human body region frame of each object; a posture determination module, configured to determine the posture of the object based on a positional relationship between a plurality of key points of the object whose occlusion degree is less than an occlusion threshold; a matching module, configured to match the clothing image of the object with product images in a clothing product library when the posture of the object is a frontal posture; The purchase link display module is used to display purchase links of clothing that is the same as and / or similar to the clothing image in the target video.

8. An electronic device, characterized in that: include: processor; A memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the method for determining the degree of human body occlusion as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for determining the degree of human body occlusion according to any one of claims 1 to 6 is implemented.

Images