Outer contour point marking method, device, electronic device and readable storage medium

Abstract

The embodiments of the present application provide a method, device, electronic device and readable storage medium for marking outer contour points, which relate to the field of image processing technology. The method comprises obtaining an image to be processed, wherein the image to be processed includes a target area, and the target area represents the area where the object to be marked is located. The three-dimensional model parameters of the object to be marked in the target area are obtained, and a three-dimensional model is constructed based on the three-dimensional model parameters, wherein the three-dimensional model includes a three-dimensional outer contour point candidate area and a three-dimensional outer contour vertex. Then, the three-dimensional outer contour point candidate area is projected onto the object to be marked to obtain a two-dimensional outer contour point candidate area of ​​the object to be marked. Based on the convex hull algorithm, the target outer contour point is screened from the two-dimensional outer contour point candidate area. Finally, the target outer contour point is marked on the object to be marked to obtain the outer contour point marking result. In this way, the two-dimensional human outer contour can be marked quickly and accurately, so as to obtain a large amount of training data, thereby improving the accuracy of the training results.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an outer contour point marking method, device, electronic device and readable storage medium. Background Art

[0002] In the field of image processing technology, the outer contour of the human body is composed of multiple two-dimensional outer contour points with clear semantics. By training the model to detect the outer contour points of the human body, it can be applied to projects such as body shaping.

[0003] Currently, there is no public or similar dataset in the industry that includes human body contour points. Existing datasets mainly rely on manual annotation, which has a small amount of data, slow annotation speed and high labor costs. However, model training requires a large amount of training data, and the amount of training data seriously affects the training results.

[0004] How to quickly and accurately achieve the annotation of the two-dimensional human body contour in order to obtain a large amount of training data is a problem that needs to be solved urgently. Summary of the invention

[0005] In view of this, the present application provides an outer contour point marking method, device, electronic device and readable storage medium to improve the above-mentioned problems.

[0006] In a first aspect, the present invention provides a method for marking outer contour points, which is applied to an electronic device, and the method comprises:

[0007] Acquire an image to be processed, wherein the image to be processed includes a target area, and the target area represents an area where an object to be labeled is located;

[0008] Acquire three-dimensional model parameters of the object to be annotated in the target area, and construct a three-dimensional model based on the three-dimensional model parameters, wherein the three-dimensional model includes a three-dimensional outer contour point candidate area and three-dimensional outer contour vertices;

[0009] Projecting the three-dimensional outer contour point candidate region onto the object to be labeled to obtain a two-dimensional outer contour point candidate region of the object to be labeled;

[0010] Based on the convex hull algorithm, the target outer contour point is obtained by screening from the two-dimensional outer contour point candidate area;

[0011] The target outer contour points are marked on the object to be marked to obtain an outer contour point marking result.

[0012] In an optional embodiment, the three-dimensional outer contour vertex includes a plurality of outer contour vertices, and before executing the step of projecting the three-dimensional outer contour point candidate area onto the object to be labeled, the method further includes:

[0013] Obtaining the camera position, and calculating the distance and angle between each outer contour vertex in the three-dimensional outer contour and the camera position;

[0014] The outer contour vertex with the smallest distance from the camera position when the angles are the same is obtained, and the outer contour vertex is used as a visible outer contour vertex, and other outer contour points are used as invisible outer contour points.

[0015] In an optional embodiment, the electronic device includes a preset three-dimensional model template and a pre-trained neural network model; the object to be labeled is a human body; and the step of obtaining the three-dimensional model parameters of the object to be labeled in the target area includes:

[0016] Acquire the face region of the object to be labeled in the target region, and perform gender recognition on the face region using a binary classification face gender algorithm to obtain a gender recognition result;

[0017] Based on the gender recognition result, obtaining a preset three-dimensional model template;

[0018] The three-dimensional model parameters of the object to be labeled are extracted using a pre-trained neural network model.

[0019] In an optional implementation, the step of constructing a three-dimensional model based on the three-dimensional model parameters includes:

[0020] Modeling is performed based on the three-dimensional model parameters and the three-dimensional model template to obtain initial three-dimensional model parameters of the object to be labeled in the target area;

[0021] Acquire two-dimensional key points of the object to be marked in the target area;

[0022] Based on the SMPLify algorithm, the initial three-dimensional model parameters are optimized using the two-dimensional key points to obtain a three-dimensional model of the object to be labeled in the target area.

[0023] In an optional implementation, the step of obtaining the three-dimensional model parameters of the object to be labeled in the target area includes:

[0024] In response to the input operation, the three-dimensional model parameters of the object to be labeled in the target area are obtained.

[0025] In an optional embodiment, there are multiple 3D outer contour vertices, each of which includes a vertex number, and the step of screening the target outer contour point from the 2D outer contour point candidate area based on the 3D outer contour vertices includes:

[0026] Based on the convex hull algorithm, calculating the extreme value of each candidate area of ​​the two-dimensional outer contour point;

[0027] Based on all the extreme values, determining the outer contour points of each of the two-dimensional outer contour point candidate regions;

[0028] The orientation of the object to be labeled is determined, and each vertex number is marked on each outer contour point according to the orientation of the object to be labeled to obtain a target outer contour point.

[0029] In an optional implementation, the step of determining the outer contour points of each of the two-dimensional outer contour point candidate regions based on all the extreme values ​​comprises:

[0030] Based on all the extreme values, determining the initial outer contour point of each of the two-dimensional outer contour point candidate regions;

[0031] Acquire a mask image of the object to be labeled;

[0032] Calculating the distance between the initial outer contour point of each of the two-dimensional outer contour point candidate regions and the mask image;

[0033] For each of the two-dimensional outer contour point candidate regions, the closest initial outer contour is selected from all initial outer contour points in the two-dimensional outer contour point candidate region, and the closest initial outer contour is used as the outer contour point of the two-dimensional outer contour point candidate region.

[0034] In a second aspect, the present invention provides an outer contour point marking device, which is applied to an electronic device, and the device comprises:

[0035] An acquisition module, used for acquiring an image to be processed, wherein the image to be processed includes a target area, and the target area represents an area where an object to be labeled is located;

[0036] The acquisition module is further used to acquire the three-dimensional model parameters of the object to be annotated in the target area, and construct a three-dimensional model based on the three-dimensional model parameters, wherein the three-dimensional model includes a three-dimensional outer contour point candidate area and a three-dimensional outer contour vertex;

[0037] A projection module, used to project the three-dimensional outer contour point candidate area to the object to be labeled, so as to obtain the two-dimensional outer contour point candidate area of ​​the object to be labeled;

[0038] A screening module, used for screening target outer contour points from the two-dimensional outer contour point candidate area based on the convex hull algorithm;

[0039] The marking module is used to mark the target outer contour points on the object to be marked to obtain the outer contour point marking results.

[0040] In a third aspect, the present invention provides an electronic device, comprising a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the outer contour point marking method described in any one of the aforementioned implementations.

[0041] In a fourth aspect, the present invention provides a readable storage medium, wherein the readable storage medium stores a computer program, and when the computer program is executed, the steps of the outer contour point marking method described in any one of 1-7 are implemented.

[0042] The embodiment of the present application provides a method, device, electronic device and readable storage medium for labeling outer contour points, by acquiring an image to be processed, the image to be processed includes a target area, the target area represents the area where the object to be labeled is located, obtaining the three-dimensional model parameters of the object to be labeled in the target area, and constructing a three-dimensional model based on the three-dimensional model parameters, wherein the three-dimensional model includes a three-dimensional outer contour point candidate area and a three-dimensional outer contour vertex. Then the three-dimensional outer contour point candidate area is projected to the object to be labeled to obtain a two-dimensional outer contour point candidate area of ​​the object to be labeled. Based on the convex hull algorithm, the target outer contour point is screened from the two-dimensional outer contour point candidate area. Finally, the target outer contour point is labeled on the object to be labeled to obtain the outer contour point labeling result. In this way, the labeling of the two-dimensional human outer contour can be achieved quickly and accurately, so as to obtain a large amount of training data, thereby improving the accuracy of the training results.

[0043] In order to make the above-mentioned objects, features and advantages of the present application more obvious and understandable, some examples are given below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A structural block diagram of an electronic device provided in an embodiment of the present application.

[0046] Figure 2 A schematic flow chart of a method for marking outer contour points provided in an embodiment of the present application.

[0047] Figure 3 A schematic diagram of a standard three-dimensional outer contour point candidate area of ​​a human body provided in an embodiment of the present application.

[0048] Figure 4 This is one of the schematic diagrams of a reconstructed three-dimensional model of an object to be labeled provided in an embodiment of the present application.

[0049] Figure 5 The second schematic diagram of a reconstructed three-dimensional model of an object to be labeled provided in an embodiment of the present application.

[0050] Figure 6 This is one of the outer contour point marking result diagrams provided in an embodiment of the present application.

[0051] Figure 7 A mask image of an object to be labeled is provided in an embodiment of the present application.

[0052] Figure 8 This is a second diagram of an outer contour point marking result provided in an embodiment of the present application.

[0053] Fig. 9 A functional module block diagram of the outer contour point marking device provided in an embodiment of the present application.

[0054] Icons: 100 - electronic device; 110 - memory; 120 - processor; 130 - outer contour point marking device; 131 - acquisition module; 132 - projection module; 133 - screening module; 134 - marking module; 140 - communication unit. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0056] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0057] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0058] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0059] It should be noted that, in the absence of conflict, the features in the embodiments of the present application may be combined with each other.

[0060] As introduced in the background technology, in the field of image processing technology, the outer contour of the human body is composed of multiple two-dimensional outer contour points with clear semantics. By training the model to detect the outer contour points of the human body, it can be applied to projects such as body shaping.

[0061] Currently, there is no public or similar dataset in the industry that includes human body contour points. Existing datasets mainly rely on manual annotation, which has a small amount of data, slow annotation speed and high labor costs. However, model training requires a large amount of training data, and the amount of training data seriously affects the training results.

[0062] How to quickly and accurately achieve the annotation of the two-dimensional human body contour in order to obtain a large amount of training data is a problem that needs to be solved urgently.

[0063] In view of this, the embodiments of the present application provide a method, device, electronic device and readable storage medium for marking outer contour points, by establishing a three-dimensional human body parameter model of the human body to be marked in the image, projecting the three-dimensional human body parameter model onto a two-dimensional plane, obtaining the target outer contour points through screening and optimization, and finally marking the target outer contour points at the corresponding positions to obtain the marking results. The marking of the two-dimensional human outer contour can be achieved quickly and accurately, so as to obtain a large amount of training data, thereby improving the accuracy of the training results. The above scheme is described in detail below.

[0064] The defects existing in the solutions in the above-mentioned prior art are the results obtained by the applicant after practice and careful research. Therefore, the process of discovering the above-mentioned problems and the solutions proposed in the embodiments of the present application below for the above-mentioned problems should be the contributions made by the applicant to the present application during the application process.

[0065] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and key points in the embodiments can be combined with each other.

[0066] Please refer to Figure 1 , Figure 1 The present invention provides a structural block diagram of an electronic device 100 provided in an embodiment of the present application. The device may include a processor 120, a memory 110, an outer contour point marking device 130 and a communication unit 140. The memory 110 stores machine-readable instructions executable by the processor 120. When the electronic device 100 is running, the processor 120 and the memory 110 communicate through a bus, and the processor 120 executes the machine-readable instructions and executes the outer contour point marking method.

[0067] The memory 110 , the processor 120 , and the communication unit 140 are electrically connected to each other directly or indirectly to achieve signal transmission or interaction.

[0068] For example, these elements may be electrically connected to each other via one or more communication buses or signal lines. The outer contour point marking device 130 includes at least one software function module that can be stored in the memory 110 in the form of software or firmware. The processor 120 is used to execute the executable module stored in the memory 110, such as the software function module or computer program included in the outer contour point marking device 130.

[0069] Among them, the memory 110 can be, but is not limited to, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0070] The processor 120 may be an integrated circuit chip with signal processing capability. The processor 120 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), and the like.

[0071] It can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. It can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application. The general processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0072] In the embodiment of the present application, the memory 110 is used to store the program, and the processor 120 is used to execute the program after receiving the execution instruction. The process definition method disclosed in any implementation of the embodiment of the present application can be applied to the processor 120 or implemented by the processor 120.

[0073] The communication unit 140 is used to establish a communication connection between the electronic device 100 and other electronic devices through a network, and to send and receive data through the network.

[0074] In some embodiments, the network may be any type of wired or wireless network, or a combination thereof. By way of example only, the network may include a wired network, a wireless network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, or a near field communication (NFC) network, or any combination thereof.

[0075] In the embodiment of the present application, the electronic device 100 may be, but is not limited to, a smart phone, a personal computer, a tablet computer, or other device with processing capabilities.

[0076] Understandably, Figure 1 The structure shown is for illustration only. The electronic device 100 may also have Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 The components shown may be implemented in hardware, software or a combination thereof.

[0077] The following is based on Figure 1 The structural diagram of the electronic device 100 shown describes in detail the steps of the outer contour point marking method, device, electronic device and readable storage medium provided in the embodiment of the present application.

[0078] Please refer to Figure 2 , Figure 2 A schematic diagram of the flow chart of the outer contour point marking method provided in an embodiment of the present application.

[0079] Step S1, obtaining an image to be processed, wherein the image to be processed includes a target area, and the target area represents an area where an object to be labeled is located.

[0080] Step S2, obtaining the 3D model parameters of the object to be annotated in the target area, and constructing a 3D model based on the 3D model parameters, wherein the 3D model includes a 3D outer contour point candidate area and 3D outer contour vertices.

[0081] Step S3, projecting the three-dimensional outer contour point candidate area onto the object to be labeled to obtain the two-dimensional outer contour point candidate area of ​​the object to be labeled.

[0082] Step S4, based on the convex hull algorithm, filter the target outer contour points from the two-dimensional outer contour point candidate area.

[0083] Step S5, marking the target outer contour points on the object to be marked, and obtaining the outer contour point marking result.

[0084] The three-dimensional outer contour point candidate region may include multiple candidate outer contour points, and the target outer contour point may be screened from the multiple candidate outer contour points included in the three-dimensional outer contour point candidate region.

[0085] Optionally, before step S1, the embodiment of the present application may also pre-define annotation to obtain a standard 3D outer contour point candidate region of the human body. For example, the region corresponding to the outer contour points included in the SMPL human body template model may be annotated. The standard 3D outer contour point candidate region may be as follows: Figure 3 As shown, Figure 3 A schematic diagram of a standard three-dimensional outer contour point candidate area of ​​a human body provided in an embodiment of the present application.

[0086] It should be noted that due to the large differences between male and female SMPL human body templates, in order to improve accuracy, the embodiment of the present application can mark the male and female SMPL human body templates separately. The figure shows the standard three-dimensional outer contour point candidate area pre-defined based on the female SMPL human body model. The standard three-dimensional outer contour point candidate area for men is similar to that for women and is not shown here.

[0087] In the subsequent actual processing, the candidate areas of the 3D outer contour points are different according to the different behaviors and postures of the human body. At the same time, the object to be labeled can be a human body, an animal or any object. The following is a detailed description of the above steps with the human body as the object to be labeled.

[0088] In the embodiment of the present application, a three-dimensional human body parameter model of the human body to be annotated in the image is established, the three-dimensional human body parameter model is projected onto a two-dimensional plane, and the target outer contour points are obtained through screening and optimization, and finally the target outer contour points are annotated at the corresponding positions to obtain the annotation results. The annotation of the two-dimensional human outer contour can be realized quickly and accurately, so as to obtain a large amount of training data, thereby improving the accuracy of the training results.

[0089] In actual image processing, the portraits in the images often present different actions and body postures, so some contour points will be blocked by the body. In order to make the parameters of the established three-dimensional model of the object to be annotated more accurate, the contour points blocked by the portrait can also be pre-calculated in the embodiment of the present application. Therefore, as an optional implementation, the three-dimensional outer contour vertex includes multiple outer contour vertices. In step S3, before the step of projecting the three-dimensional outer contour point candidate area to the object to be annotated, the method also includes:

[0090] Get the camera position and calculate the distance and angle between each outer contour vertex in the three-dimensional outer contour and the camera position.

[0091] When the angles are the same, the outer contour vertex with the smallest distance from the camera position is obtained, and the outer contour vertex is used as the visible outer contour vertex, and the other outer contour points are used as invisible outer contour points.

[0092] The method of obtaining the camera position can be referred to in the prior art, and the camera position is obtained by calculating according to different rotation matrices, which will not be described in detail here.

[0093] In this way, by screening the outer contour vertices at the same angle from the camera position according to the distance from the camera position, the outer contour vertex with the smallest distance is used as the visible outer contour point, and the other outer contour points are used as invisible outer contour points, so as to adapt to different postures of the human body, so that the subsequent projection of the three-dimensional outer contour point candidate area to the target outer contour point obtained by the object to be labeled is more accurate.

[0094] As an optional implementation, step S1, obtaining the three-dimensional model parameters of the object to be annotated in the target area can be achieved by:

[0095] In response to the input operation, the three-dimensional model parameters of the object to be labeled in the target area are obtained.

[0096] That is to say, the candidate area of ​​the 3D outer contour points and the 3D outer contour vertices of the object to be labeled can be manually labeled, and the labeled 3D model parameters are input into the electronic device after manual labeling. The electronic device responds to the input operation and obtains the 3D model parameters of the object to be labeled in the target area.

[0097] As another optional implementation, the electronic device includes a pre-set three-dimensional model template and a pre-trained neural network model, and the object to be labeled is a human body. Step S1, obtaining the three-dimensional model parameters of the object to be labeled in the target area can be achieved by:

[0098] The face region of the object to be labeled is obtained in the target area, and the gender of the face region is identified using a binary classification face gender algorithm to obtain a gender recognition result.

[0099] Based on the gender recognition result, a pre-set 3D model template is obtained.

[0100] The pre-trained neural network model is used to extract the 3D model parameters of the object to be labeled.

[0101] Since the SMPL human body models of different genders are different, the standard 3D outer contour point candidate areas for annotation based on the SMPL human body models of different human bodies are also different. Therefore, it is necessary to obtain the corresponding 3D model template according to the gender of the human body to be annotated.

[0102] When the human body to be labeled is identified as a male, a 3D model template of a male is obtained, and when the human body to be labeled is identified as a female, a 3D model template of a female is obtained.

[0103] In this way, the embodiment of the present application can perform three-dimensional human body reconstruction on the human body to be annotated in the two-dimensional image through the above method, and finally obtain the three-dimensional model parameters of the object to be annotated in the target area.

[0104] Furthermore, the three-dimensional model parameters obtained by performing three-dimensional human body reconstruction on the human body to be annotated in the two-dimensional image based on the above method may be different from the actual human body, such as Figure 4 As shown, Figure 4 This is one of the schematic diagrams of a reconstructed 3D model of an object to be annotated provided in an embodiment of the present application. Although the movements of the modeled 3D human body are similar to those of the actual human body, the head direction of the modeled 3D human body is different from that of the actual human body, and the right leg of the modeled 3D human body is also different from that of the actual human body.

[0105] Therefore, in an optional implementation, the modeled three-dimensional human body may be further optimized based on the following steps:

[0106] Modeling is performed based on the 3D model parameters and the 3D model template to obtain the initial 3D model parameters of the object to be labeled in the target area.

[0107] Get the two-dimensional key points of the object to be marked in the target area.

[0108] Based on the SMPLify algorithm, the initial 3D model parameters are optimized using 2D key points to obtain the 3D model of the object to be labeled in the target area.

[0109] The two-dimensional key points can be obtained based on the openpose model, which includes 25 2D key points.

[0110] Then, the projection error of the above two-dimensional key points is used as the optimization target, and the initial three-dimensional model parameters are optimized based on the SMPLify algorithm until the projection error reaches the preset threshold. The optimization is completed and the three-dimensional model parameters of the object to be labeled in the target area are obtained. The three-dimensional human body model constructed by the optimized three-dimensional model parameters is as follows Figure 5 As shown, Figure 5 This is a second schematic diagram of a reconstructed three-dimensional model of an object to be labeled provided in an embodiment of the present application. Figure 5 and Figure 4 It can be seen that Figure 5 The 3D human body model in the image is consistent with the actual human body, and the head and legs are also consistent. This makes the subsequent 2D human body contour points obtained based on the 3D model parameter projection more accurate.

[0111] In an optional implementation, there are multiple 3D outer contour vertices, each of which includes a vertex number. Step S4, based on the 3D outer contour vertices, screening the target outer contour point from the 2D outer contour point candidate area can be implemented by the following method:

[0112] Based on the convex hull algorithm, the extremum of each candidate region of the two-dimensional outer contour point is calculated.

[0113] Based on all extreme values, the outer contour points of each two-dimensional outer contour point candidate region are determined.

[0114] The orientation of the object to be labeled is determined, and each vertex number is marked on each outer contour point according to the orientation of the object to be labeled to obtain the target outer contour point.

[0115] The orientation of the object to be annotated may be determined by the angle between the normal vectors of the object to be annotated before and after rotation, for example, the object to be annotated may be toward the right, left, front, or back.

[0116] Then, according to the orientation of the object to be labeled, the vertex labels of each 3D outer contour vertex are matched with each outer contour point to obtain the target outer contour point. Assuming that there are 65 vertex labels in total, the outer contour point labeling result image after the target outer contour point is labeled on the object to be labeled can be as follows: Figure 6 shown.

[0117] Depend on Figure 6 It can be seen that, in some cases, the outer contour points of each two-dimensional outer contour point candidate area determined directly based on the extreme value are slightly different from the outer contour points of the actual object to be labeled, and there is a problem of inaccuracy.

[0118] Therefore, as an optional implementation, the outer contour points can be optimized in the following manner to obtain more accurate target outer contour points:

[0119] Based on all extreme values, the initial outer contour points of each two-dimensional outer contour point candidate region are determined.

[0120] Get the mask image of the object to be labeled.

[0121] Calculate the distance between the initial outer contour point of each 2D outer contour point candidate region and the mask image.

[0122] For each two-dimensional outer contour point candidate region, the closest initial outer contour is selected from all initial outer contour points of the two-dimensional outer contour point candidate region, and the closest initial outer contour is used as the outer contour point of the two-dimensional outer contour point candidate region.

[0123] Likewise Figure 6 Taking the object to be labeled as shown in FIG. 1 as an example, the mask image of the object to be labeled can be as follows Figure 7 shown.

[0124] By selecting the initial outer contour point closest to the mask image from the initial outer contour points of each two-dimensional outer contour point candidate area, the initial outer contour with the closest distance is used as the outer contour point of the two-dimensional outer contour point candidate area, thereby obtaining a more accurate target outer contour point.

[0125] like Figure 8 It can be seen from the image shown that the target outer contour points obtained after optimization based on the above method are more consistent with the outer contour of the object to be marked.

[0126] Based on the same invention concept, please refer to Fig. 9 , Fig. 9 This is a functional module block diagram of the outer contour point marking device provided in the embodiment of the present application. Figure 2 The outer contour point marking device 130 corresponding to the outer contour point marking method shown in the figure comprises:

[0127] The acquisition module 131 is used to acquire the image to be processed, where the image to be processed includes a target area, and the target area represents the area where the object to be labeled is located.

[0128] The acquisition module 131 is further used to acquire the 3D model parameters of the object to be labeled in the target area, and construct a 3D model based on the 3D model parameters, wherein the 3D model includes a 3D outer contour point candidate area and 3D outer contour vertices.

[0129] The projection module 132 is used to project the three-dimensional outer contour point candidate area to the object to be labeled to obtain the two-dimensional outer contour point candidate area of ​​the object to be labeled.

[0130] The screening module 133 is used for the convex hull algorithm to screen the target outer contour points from the two-dimensional outer contour point candidate area.

[0131] The marking module 134 is used to mark the target outer contour points on the object to be marked to obtain the outer contour point marking result.

[0132] Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned outer contour point marking method, device, electronic device and readable storage medium in the embodiment of the present application, the implementation principle of the device can refer to the implementation principle of the method, and the repeated parts will not be repeated.

[0133] An embodiment of the present application also provides a readable storage medium, in which a computer program is stored. When the computer program is executed, the steps of the above-mentioned outer contour point marking method are implemented.

[0134] In summary, the embodiments of the present application provide a method, device, electronic device 100 and readable storage medium for labeling outer contour points, by acquiring an image to be processed, the image to be processed includes a target area, the target area represents the area where the object to be labeled is located, the three-dimensional model parameters of the object to be labeled in the target area are acquired, and a three-dimensional model is constructed based on the three-dimensional model parameters, wherein the three-dimensional model includes a three-dimensional outer contour point candidate area and a three-dimensional outer contour vertex. Then the three-dimensional outer contour point candidate area is projected to the object to be labeled to obtain a two-dimensional outer contour point candidate area of ​​the object to be labeled. Based on the convex hull algorithm, the target outer contour point is screened from the two-dimensional outer contour point candidate area. Finally, the target outer contour point is labeled on the object to be labeled to obtain the outer contour point labeling result. In this way, the labeling of the two-dimensional human outer contour can be achieved quickly and accurately, so as to obtain a large amount of training data, thereby improving the accuracy of the training results.

[0135] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for marking outer contour points, characterized in that: Applied to electronic devices, the method comprises: there are multiple vertices of the three-dimensional outer contour, and each of the three-dimensional outer contour vertices comprises a vertex number; Acquire an image to be processed, wherein the image to be processed includes a target area, and the target area represents an area where an object to be labeled is located; Using a pre-trained neural network model to obtain three-dimensional model parameters of the object to be annotated in the target area, and constructing a three-dimensional model based on the three-dimensional model parameters, wherein the three-dimensional model includes a three-dimensional outer contour point candidate area and a three-dimensional outer contour vertex; Projecting the three-dimensional outer contour point candidate region onto the object to be labeled to obtain a two-dimensional outer contour point candidate region of the object to be labeled; Based on the convex hull algorithm, calculating the extreme value of each candidate area of ​​the two-dimensional outer contour point; Based on all the extreme values, determining the outer contour points of each of the two-dimensional outer contour point candidate regions; Determine the orientation of the object to be labeled, and mark each vertex number on each outer contour point according to the orientation of the object to be labeled to obtain a target outer contour point; Marking the target outer contour points on the object to be marked to obtain an outer contour point marking result; The step of using the pre-trained neural network model to obtain the three-dimensional model parameters of the object to be labeled in the target area includes: Acquire the face region of the object to be labeled in the target region, and perform gender recognition on the face region using a binary classification face gender algorithm to obtain a gender recognition result; Based on the gender recognition result, obtaining a preset three-dimensional model template; The three-dimensional model parameters of the object to be labeled are extracted using a pre-trained neural network model.

2. The outer contour point marking method according to claim 1, characterized in that: The three-dimensional outer contour vertex includes a plurality of outer contour vertices. Before executing the step of projecting the three-dimensional outer contour point candidate area onto the object to be labeled, the method further includes: Obtaining the camera position, and calculating the distance and angle between each outer contour vertex in the three-dimensional outer contour and the camera position; The outer contour vertex with the smallest distance from the camera position when the angles are the same is obtained, and the outer contour vertex is used as a visible outer contour vertex, and other outer contour points are used as invisible outer contour points.

3. The outer contour point marking method according to claim 1, characterized in that: The step of constructing a three-dimensional model based on the three-dimensional model parameters comprises: Modeling is performed based on the three-dimensional model parameters and the three-dimensional model template to obtain initial three-dimensional model parameters of the object to be labeled in the target area; Acquire two-dimensional key points of the object to be marked in the target area; Based on the SMPLify algorithm, the initial three-dimensional model parameters are optimized using the two-dimensional key points to obtain a three-dimensional model of the object to be labeled in the target area.

4. The outer contour point marking method according to claim 1, characterized in that: The step of obtaining the three-dimensional model parameters of the object to be labeled in the target area comprises: In response to the input operation, the three-dimensional model parameters of the object to be labeled in the target area are obtained.

5. The outer contour point marking method according to claim 1, characterized in that: The step of determining the outer contour points of each of the two-dimensional outer contour point candidate regions based on all the extreme values ​​comprises: Based on all the extreme values, determining the initial outer contour point of each of the two-dimensional outer contour point candidate regions; Acquire a mask image of the object to be labeled; Calculating the distance between the initial outer contour point of each of the two-dimensional outer contour point candidate regions and the mask image; For each of the two-dimensional outer contour point candidate regions, the closest initial outer contour is selected from all initial outer contour points in the two-dimensional outer contour point candidate region, and the closest initial outer contour is used as the outer contour point of the two-dimensional outer contour point candidate region.

6. An outer contour point marking device, characterized in that: Applied to electronic equipment, the device comprises: a plurality of vertices of a three-dimensional outer contour, each of the vertices of the three-dimensional outer contour comprises a vertex number; An acquisition module, used for acquiring an image to be processed, wherein the image to be processed includes a target area, and the target area represents an area where an object to be labeled is located; The acquisition module is further used to acquire the three-dimensional model parameters of the object to be annotated in the target area by using the pre-trained neural network model and construct a three-dimensional model based on the three-dimensional model parameters, wherein the three-dimensional model includes a three-dimensional outer contour point candidate area and a three-dimensional outer contour vertex; A projection module, used to project the three-dimensional outer contour point candidate area to the object to be labeled, so as to obtain the two-dimensional outer contour point candidate area of ​​the object to be labeled; A screening module is used to calculate the extreme value of each of the two-dimensional outer contour point candidate areas based on a convex hull algorithm; determine the outer contour points of each of the two-dimensional outer contour point candidate areas based on all the extreme values; determine the orientation of the object to be labeled, and mark each vertex number on each of the outer contour points according to the orientation of the object to be labeled to obtain a target outer contour point; A marking module, used for marking the target outer contour points on the object to be marked to obtain an outer contour point marking result; The acquisition module is specifically used for: Acquire the face region of the object to be labeled in the target region, and perform gender recognition on the face region using a binary classification face gender algorithm to obtain a gender recognition result; Based on the gender recognition result, obtaining a preset three-dimensional model template; The three-dimensional model parameters of the object to be labeled are extracted using a pre-trained neural network model.

7. An electronic device, characterized in that: The electronic device includes a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the outer contour point marking method described in any one of claims 1-5.

8. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed, the steps of the outer contour point marking method described in any one of claims 1 to 5 are implemented.

Images