Method and device for generating training data for infant mouth and nose occlusion analysis model

Abstract

The present invention relates to the field of infant care technology, solves the problem that realistic simulated infant mouth and nose area occlusion image data cannot be generated for model training in the prior art, and provides a method and device for generating training data for infant mouth and nose occlusion analysis model. The method comprises: obtaining the initial position information of the infant mouth and nose area preset in the pre-collected infant face image to be occluded; based on the initial position information, performing contour feature extraction and smoothing processing on the infant mouth and nose area, and determining the contour position information of the target area after smoothing; obtaining an occlusion image set, combining the contour position information, and selecting an occlusion area with the same shape as the target area from the occlusion image set according to preset rules; superimposing the preset texture information on the occlusion area, and outputting the infant face image after the infant mouth and nose area is occluded as training data. The present invention can generate realistic and detailed simulated mouth and nose occlusion images.

Description

Technical Field

[0001] The present invention relates to the technical field of infant care, and in particular to a method and device for generating training data for an infant mouth and nose occlusion analysis model. Background Art

[0002] In the infant sleep care scenario, ensuring the infant's airway is unobstructed is one of the key issues. Since infants may inadvertently cover their faces with quilts, towels or other objects during sleep, it is very easy to cause the risk of suffocation. Therefore, monitoring the infant's face occlusion through image processing technology can alert the caregiver at the first time. However, when developing a model for judging infant face occlusion, there is often a practical problem, that is, the lack of sufficient training data on infant face occlusion. Because it involves infant safety, it is not feasible to directly collect a large amount of real occlusion data. Therefore, using image processing technology to generate simulated infant face occlusion image data has become an effective means to solve this data shortage problem. These synthetic data can not only be used to enrich the training data set, but also help improve the accuracy and robustness of the occlusion detection model.

[0003] Existing technologies face multiple technical problems when generating simulated infant face occlusion image data. When generating occluded images, it is also necessary to maintain the natural transition between the infant skin features and the occluders to avoid unreasonable splicing traces or artifacts that affect the training effect of the model. Finally, existing image synthesis methods are usually difficult to adjust the degree and position of occlusion according to the complexity of the actual scene, which limits the generalization ability of the model in different occlusion situations.

[0004] The existing Chinese patent CN107689069A discloses an automatic image synthesis method based on the recognition of facial occlusions, including: the occluders obtained include hands, glasses, hair, hats and scarves, etc., so as to make it more diverse and close to the synthetic data set of the real world; then the quality of the acquired data is evaluated. Since obtaining clear occluders is very necessary for the success of the subsequent stages of the synthesis algorithm, an interactive iterative segmentation algorithm is used to segment the hand occluders from the original image; for each unobstructed face in the database, a suitable occluder is first retrieved from the initial occluder group, the occluder is rescaled according to the facial matching posture and direction, the facial size, the color of the face and the occluder is matched, the lighting conditions and quality are improved, and finally the occluder is placed in the selected area of ​​the face. Although the above patent also discloses placing the occluder in the selected area of ​​the face to synthesize data, the interactive iterative segmentation algorithm is easily affected by subjectivity during the manual adjustment process, and it is difficult to maintain consistency in large-scale data sets. At the same time, it only occludes specific areas of the face, and cannot meet the special needs of occluding the mouth and nose area of ​​infants and young children in infant care scenarios.

[0005] Therefore, how to generate realistic simulated infant mouth and nose area occluded image data for model training is an urgent problem to be solved. Summary of the invention

[0006] In view of this, the present invention provides a method and device for generating training data for an infant mouth and nose occlusion analysis model, so as to solve the problem in the prior art that realistic simulated infant mouth and nose area occlusion image data cannot be generated for model training.

[0007] The technical solution adopted by the present invention is:

[0008] In a first aspect, the present invention provides a method for generating training data for an infant mouth and nose occlusion analysis model, the method comprising:

[0009] S1: Acquire initial position information of a preset infant mouth and nose region in a pre-collected infant face image to be occluded;

[0010] S2: extracting contour features and performing smoothing on the mouth and nose area of ​​the infant based on the initial position information, and determining contour position information of the target area after the smoothing process;

[0011] S3: obtaining a pre-collected occlusion image set, combining the contour position information, and selecting an occlusion area having the same shape as the target area from the occlusion image set according to a preset rule;

[0012] S4: superimposing the preset texture information onto the occluded area, and outputting the infant's facial image after the mouth and nose area of ​​the infant is occluded as training data.

[0013] Preferably, the S2 includes:

[0014] S21: extracting contour features of the mouth and nose area of ​​the infant based on the initial position information to obtain contour point position information of the mouth and nose area;

[0015] S22: Processing the contour point position information to determine vertex position information of each vertex of the minimum circumscribed polygon of the mouth and nose area;

[0016] S23: according to the vertex position information, taking the area surrounded by the minimum circumscribed polygon as the target area;

[0017] S24: Extract features of the target area to determine the contour position information.

[0018] Preferably, the S24 includes:

[0019] S241: Calculate the target area to determine the minimum circumscribed rectangle of the minimum circumscribed polygon;

[0020] S242: selecting a point closest to the left vertex from among the vertices as an initial vertex based on the left vertex position information of the minimum circumscribed rectangle and the vertex position information;

[0021] S243: The initial vertex is taken as the first point, and other vertices other than the initial vertex are arranged in a clockwise manner, and position information corresponding to each vertex is output as the contour position information.

[0022] Preferably, S3 includes:

[0023] S31: Obtain the area of ​​each occluder image in the occluder image set, and compare the area of ​​each occluder image with the calculated target area;

[0024] S32: When the area of ​​the occluder image is larger than the area of ​​the target region, extracting the corresponding occluder image as the target image;

[0025] S33: Calculating geometric features of the target area, and selecting a plurality of mapping points in the target image based on the geometric feature information obtained by calculation;

[0026] S34: Connect the mapping points to obtain a target rectangular frame, and determine an occlusion area with the same shape as the target area in the target rectangular frame according to the contour position information.

[0027] Preferably, the S33 includes:

[0028] S331: Obtain the minimum circumscribed rectangular frame of the minimum circumscribed polygon of the target area, and calculate the coordinates of each vertex of the minimum circumscribed rectangular frame;

[0029] S332: performing difference calculation on the coordinates of each vertex to determine the width and length of the minimum circumscribed rectangular frame;

[0030] S333: Calculate the slope of each vertex coordinate to determine the slope of each side of the minimum circumscribed rectangular frame, and use the width, length and slope as the geometric feature information;

[0031] S334: Select any point in the target image as a mapping point of the upper left corner vertex of the minimum circumscribed rectangular box, and determine multiple other mapping points corresponding to the vertices of the minimum circumscribed rectangular box in combination with the geometric feature information.

[0032] Preferably, the S4 includes:

[0033] S41: replacing corresponding vertices in the original infant face image with mapping points carrying the texture information in the target image according to the contour position information;

[0034] S42: Based on the replaced mapping point position information, a line scanning method is used to replace each pixel point in the target area one by one through each pixel point in the occluded area, and the replaced image is output as an infant facial image after the infant's mouth and nose area is occluded.

[0035] Preferably, before S1, it also includes: obtaining the whitened area position information of the infant's facial image after processing by the user on a preset graphic software interface, and using the whitened area position information as the initial position information; the infant's mouth and nose area at least includes the infant's mouth and nose.

[0036] In a second aspect, the present invention provides a device for generating training data for an infant mouth and nose occlusion analysis model, the device comprising:

[0037] A data collection module, used to obtain initial position information of a preset infant mouth and nose region in a pre-collected infant face image to be occluded;

[0038] A feature extraction and smoothing processing module, used to extract contour features and perform smoothing processing on the mouth and nose area of ​​the infant according to the initial position information, and determine the contour position information of the target area after the smoothing processing;

[0039] An occlusion region extraction module is used to obtain a pre-collected occlusion image set, and select an occlusion region having the same shape as the target region from the occlusion image set according to a preset rule in combination with the contour position information;

[0040] The occlusion processing module is used to superimpose preset texture information on the occlusion area, and output the infant's facial image after the occlusion processing of the infant's mouth and nose area as training data.

[0041] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of the first aspect in the above-mentioned embodiment.

[0042] In a fourth aspect, an embodiment of the present invention further provides a storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method of the first aspect in the above-mentioned embodiment is implemented.

[0043] In summary, the beneficial effects of the present invention are as follows:

[0044] The method and device for generating training data for the infant mouth and nose occlusion analysis model provided by the present invention include: obtaining the initial position information of the infant mouth and nose area preset in the pre-collected infant face image to be occluded; based on the initial position information, extracting the contour feature and smoothing the infant mouth and nose area, and determining the contour position information of the target area after the smoothing; obtaining the pre-collected occlusion image set, combining the contour position information, and selecting the occlusion area with the same shape as the target area from the occlusion image set according to the preset rules; superimposing the preset texture information on the occlusion area, and outputting the infant face image after the occlusion of the infant mouth and nose area as training data. The present invention combines the initial position information and contour features of the infant mouth and nose area for precise smoothing, ensures that the shape of the occlusion area matches the natural contour of the infant face, and then selects a suitable occlusion from the pre-collected occlusion image set for superposition, generates a realistic and detailed simulated mouth and nose occlusion image, improves the authenticity of the data, and helps the model better deal with the recognition and processing problems under occlusion during training. In addition, through the superposition of preset rules and texture information, the generated images are closer to the real scenes in shape, texture, and occlusion effects, which is conducive to improving the generalization ability of the model and its ability to cope with complex occlusion scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the drawings required for use in the embodiment of the present invention will be briefly introduced below. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work, and these are all within the protection scope of the present invention.

[0046] Figure 1 This is a schematic diagram of the overall working process of the method for generating training data for an infant mouth and nose occlusion analysis model in Example 1 of the present invention;

[0047] Figure 2 This is a schematic diagram of the process of extracting contour features and smoothing the mouth and nose area of ​​an infant in Example 1 of the present invention;

[0048] Figure 3 It is a schematic diagram of the process of extracting features from the target area in Embodiment 1 of the present invention;

[0049] Figure 4 Schematic diagram of a process of selecting an occlusion region with the same shape as the target region from an occlusion image set in Embodiment 1 of the present invention;

[0050] Figure 5 It is a schematic diagram of a process of selecting multiple mapping points in a target image in Embodiment 1 of the present invention;

[0051] Figure 6It is a schematic diagram of a process of superimposing preset texture information onto the occluded area in Embodiment 1 of the present invention;

[0052] Figure 7 This is a structural block diagram of a device for generating training data for an infant mouth and nose occlusion analysis model in Example 2 of the present invention;

[0053] Figure 8 It is a schematic diagram of the structure of the electronic device in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described in conjunction with the drawings in the embodiment of the present invention. It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. In the description of the present invention, it should be understood that the orientation or position relationship indicated by the terms "center", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such a process, method, article or device. In the absence of further restrictions, the elements defined by the phrase "comprising..." do not exclude the existence of other identical elements in the process, method, article or device comprising the elements. If there is no conflict, the embodiments of the present invention and the various features in the embodiments can be combined with each other, all within the protection scope of the present invention.

[0055] Example 1

[0056] See also Figure 1 Embodiment 1 of the present invention discloses a method for generating training data for an infant mouth and nose occlusion analysis model, the method comprising:

[0057] S1: Acquire initial position information of a preset infant mouth and nose region in a pre-collected infant face image to be occluded;

[0058] In one embodiment, before S1, the method includes:

[0059] The position information of the whitened area after the user processes the infant's facial image on a preset graphic software interface is obtained, and the position information of the whitened area is used as the initial position information; the infant's mouth and nose area at least includes the infant's mouth and the infant's nose.

[0060] Specifically, first, the user manually paints the mouth and nose area in the infant's facial image through a graphical user interface software program, and the graphical user interface software program includes at least a command line, a web interface, and a mobile application interface. When painting, the user can accurately mark and select the mouth and nose position of the infant. The whitening process can help to quickly obtain the outline of the mouth and nose area, and input these outline information as initial position information into the subsequent image processing process to obtain the initial position information of the infant's mouth and nose area preset in the pre-collected infant facial image to be occluded; wherein, the initial position information of the infant's mouth and nose area usually includes the regional boundaries of the nose and mouth. This user-interactive painting method not only improves the accuracy of the initial position information, but also can flexibly respond to changes in facial shapes and postures of different infants, and provide a more accurate regional positioning basis for subsequent mouth and nose occlusion processing, ensuring that users mark key areas according to actual needs, thereby improving the accuracy of occlusion superposition and the realism of image generation.

[0061] S2: extracting contour features and performing smoothing on the mouth and nose area of ​​the infant based on the initial position information, and determining contour position information of the target area after the smoothing process;

[0062] Specifically, the contour feature extraction and smoothing of the mouth and nose area of ​​infants and young children based on the initial position information is to accurately capture the shape and boundaries of the mouth and nose area of ​​infants and young children. First, the edge and contour features of the area are identified based on the initial position information, and the specific boundary shape of the area is obtained by extracting these contour points; next, the smoothing process can remove noise or irregular parts in the contour, making the boundary smoother and continuous, reducing unnecessary details, and ensuring that the regional contour is more natural in subsequent image processing steps. Finally, this process determines the contour position information of the target area after smoothing, providing a reliable basis for occlusion, texture overlay or other subsequent operations.

[0063] In one embodiment, before S2, the method further includes:

[0064] S201: performing contour filling processing on the mouth and nose area image of the infant painted by the user according to the initial position information, and outputting a filled first target area image;

[0065] Specifically, first, according to the initial position information provided by the user, the outline of the painted mouth and nose area is filled. The specific operation is to convert the painted area marked by the user into a binary image, the painted area is displayed in white, and the rest is black. Then, the morphological dilation algorithm is used to dilate the binary image with a structural element of a preset size. This process aims to fill the gaps in the user's painted area and generate a complete and connected target area. The first target area image output after processing can more accurately represent the position of the infant's mouth and nose, providing a more stable foundation for subsequent processing.

[0066] S202: performing edge detection and Gaussian blur processing on the first target area image, and outputting a second target area image after the Gaussian blur processing;

[0067] Specifically, the first target area image is further processed using edge detection and Gaussian blur technology. First, Gaussian blur is applied to reduce the noise in the image, which helps to improve the accuracy of subsequent edge detection. Gaussian blur can make the details of the image smoother and reduce unnecessary high-frequency noise. Next, the Canny edge detection algorithm is used to extract the edge information of the image. The Canny algorithm calculates the gradient of the image and identifies the area with obvious brightness changes, thereby extracting the contour of the mouth and nose area. The second target area image output by this processing provides clear edge information and can effectively capture the natural boundaries of the infant's face.

[0068] S203: performing color segmentation and morphological operations on the second target region image, and determining the position information of the mouth and nose region after region correction as new initial position information.

[0069] Specifically, the second target area image is subjected to color segmentation and morphological operations to determine the corrected position information of the mouth and nose area. First, the image is converted into the HSV color space through color segmentation technology to more accurately identify the color difference between the mouth and nose area and foreign objects, such as food and snot. The interference area is extracted according to the preset color threshold. Then, morphological operations such as opening and closing operations are applied to clean up the image, remove small noise, and repair holes, thereby strengthening the main mouth and nose area. Finally, combined with the processed area information, the boundary of the mouth and nose area is redefined, and the corrected position information of the area is output as the new initial position information. The purpose of this step is to ensure that the positioning of the mouth and nose area is more accurate, thereby providing a reliable basis for subsequent occlusion processing.

[0070] In one embodiment, see Figure 2 , said S2 comprises:

[0071] S21: extracting contour features of the mouth and nose region of the infant based on the initial position information to obtain contour point position information of the mouth and nose region;

[0072] Specifically, contour feature extraction is performed based on the initial white-painted area manually marked by the user. First, the boundaries in the white-painted area are identified, and the contours related to the mouth and nose area are extracted from it. Extracting contours means that the system can automatically detect changes in the boundaries and record the positions of these boundary points as feature points. This step provides key contour position information for subsequent geometric analysis, ensuring that subsequent processing can accurately lock the boundaries of the mouth and nose area.

[0073] S22: Processing the contour point position information to determine vertex position information of each vertex of the minimum circumscribed polygon of the mouth and nose area;

[0074] Specifically, after extracting the contour points, these contour points are analyzed and the minimum circumscribed polygon that can surround these contour points is calculated. The method for calculating the minimum circumscribed polygon can use a geometric algorithm, such as a convex hull algorithm, to determine the vertex set of the minimum polygon that can cover all contour points. By recording the position information of each vertex of the polygon, an accurate geometric framework is provided for subsequent image processing to ensure that all key areas are included.

[0075] S23: according to the vertex position information, taking the area surrounded by the minimum circumscribed polygon as the target area;

[0076] Specifically, when the vertex positions of the minimum circumscribed polygon are determined, the area enclosed by the entire polygon is marked as the target area, which includes the pixels painted by the user and other unpainted pixels. This means that after this processing, the painted area of ​​the mouth and nose of infants and young children is transformed into a recognizable geometric shape for more refined image processing. This not only ensures that the mouth and nose area is fully covered, but also avoids the problem of missing boundary points.

[0077] S24: Extract features of the target area to determine the contour position information.

[0078] Specifically, the process of extracting features of the target area and determining the contour position information is achieved by analyzing the shape features in the image and extracting the boundary contour points of the target area. Through this feature extraction, the external shape of the target area is identified, and the precise position information of these contour points is further determined. This position information not only provides an accurate geometric basis for subsequent processing, but also ensures that it can accurately match the shape of the target area when applying texture, occlusion or other image processing operations.

[0079] In one embodiment, see Figure 3 , the S24 comprises:

[0080] S241: Calculate the target area to determine the minimum circumscribed rectangle of the minimum circumscribed polygon;

[0081] Specifically, the target area is first geometrically calculated to find the minimum circumscribed polygon that can surround the target area. In order to simplify the processing and facilitate subsequent steps, the minimum circumscribed rectangle that can completely contain this polygon is further calculated. This rectangle provides a regularized bounding box that can accurately cover the entire target area to ensure that no part is missed. In this way, subsequent processing can be based on rectangles, which is simpler and more efficient.

[0082] S242: selecting a point closest to the left vertex from among the vertices as an initial vertex based on the left vertex position information of the minimum circumscribed rectangle and the vertex position information;

[0083] Specifically, after determining the minimum enclosing rectangle, the upper left corner vertex of the rectangle is used as the reference point. Then, all vertices of the minimum enclosing polygon are traversed, and the distance between each vertex and the left vertex of the rectangle is calculated. The polygon vertex closest to the left vertex is found and marked as the initial vertex. The purpose of this step is to provide a starting point so that the subsequent vertex arrangement can be carried out in an orderly manner.

[0084] S243: The initial vertex is taken as the first point, and other vertices other than the initial vertex are arranged in a clockwise manner, and position information corresponding to each vertex is output as the contour position information.

[0085] Specifically, the previously selected initial vertex is used as the first vertex, and then the remaining polygon vertices are processed. In order to maintain the consistency of the vertex order, the system will arrange other vertices in a clockwise direction. Through this arrangement, the system can ensure that the order of contour points remains regular, which is convenient for subsequent image processing steps such as texture overlay, and all arranged vertex positions are output as the final contour position information. These vertex position information is an accurate description of the geometric boundaries of the target area, providing the basis for subsequent image processing. Based on these precise contour points, further processing such as texture overlay and area filling is performed to ensure that the final generated image meets the requirements in shape and content.

[0086] S3: obtaining a pre-collected occlusion image set, combining the contour position information, and selecting an occlusion area having the same shape as the target area from the occlusion image set according to a preset rule;

[0087] In one embodiment, see Figure 4 , said S3 comprises:

[0088] S31: Obtain the area of ​​each occluder image in the occluder image set, and compare the area of ​​each occluder image with the calculated target area;

[0089] Specifically, first, the area of ​​the target area is calculated based on the contour information of the target area. Then, the area of ​​each frame of the occluder image is extracted from the occluder image set. In the process of calculating the target area area and the occluder image area, it is first necessary to calculate the area of ​​the polygon based on the vertex coordinates of the minimum circumscribed polygon of the target area through a geometric algorithm. This is usually done using a polygon area formula, such as a two-dimensional coordinate formula using the vertex coordinates of the minimum circumscribed polygon. As for the occluder image, it is processed as an independent shape, the outer contour of the occluder is extracted and its area is calculated. Then, the area of ​​the occluder is compared with the area of ​​the target area to ensure that the selected occluder is large enough to cover the target area. If the occluder area is small, it may not be completely occluded. Such images will be excluded. Only occluders with an area greater than or equal to the target area will be selected for subsequent synthesis processing.

[0090] S32: When the area of ​​the occluder image is larger than the area of ​​the target region, extracting the corresponding occluder image as the target image;

[0091] Specifically, once it is confirmed that the area of ​​the occluder image is greater than or equal to the area of ​​the target area, the occluder image is extracted as an image for simulating occlusion. This step ensures that the selected occluder is large enough to cover the mouth and nose area of ​​the infant.

[0092] S33: Calculating geometric features of the target area, and selecting a plurality of mapping points in the target image based on the geometric feature information obtained by calculation;

[0093] Specifically, the geometric features of the target area are calculated so that multiple mapping points corresponding to the target area are selected in the occluder image. First, the geometric features of the area are determined through the previously obtained contour information of the target area, including width, length, and slopes of each side; these geometric features reflect the shape and direction of the target area. Next, an initial mapping point is selected in the occluder image, usually the vertex in the upper left corner of the rectangular box. Based on the geometric features of the target area, such as width, length, and slope, other mapping points corresponding to the vertices of the rectangular box of the target area are calculated through mathematical formulas. Through these mapping points, the part that matches the target area can be accurately found in the occluder image, laying the foundation for the next step of occlusion processing.

[0094] In one embodiment, see Figure 5 , the S33 comprises:

[0095] S331: Obtain the minimum circumscribed rectangular frame of the minimum circumscribed polygon of the target area, and calculate the coordinates of each vertex of the minimum circumscribed rectangular frame;

[0096] Specifically, first obtain the minimum circumscribed polygon of the target area. The minimum circumscribed polygon refers to the smallest polygon that can surround the entire target area. Then, based on this polygon, further calculate its minimum circumscribed rectangular frame, that is, the smallest rectangle that can completely contain the polygon. Through geometric calculation, determine the coordinates of the four vertices of the rectangular frame (usually pt1, pt2, pt3, pt4). These vertex coordinates will become an important basis for subsequent geometric feature calculation and mapping point selection.

[0097] S332: performing difference calculation on the coordinates of each vertex to determine the width and length of the minimum circumscribed rectangular frame;

[0098] Specifically, after determining the coordinates of the vertices of the rectangular box, the difference between the coordinates of each vertex is calculated to determine the width and length of the rectangular box. The width (W) and length (H) of the rectangle are obtained by calculating the horizontal and vertical distances between two adjacent points. This step is achieved through the basic Euclidean distance calculation formula. For example, the width is calculated by the coordinate difference between vertices pt1 and pt2, pt3 and pt4, and the length is calculated by the difference between vertices pt1 and pt3, pt2 and pt4.

[0099] S333: Calculate the slope of each vertex coordinate to determine the slope of each side of the minimum circumscribed rectangular frame, and use the width, length and slope as the geometric feature information;

[0100] Specifically, the slope of each edge is then calculated. The slope calculation can be achieved using the two-point slope formula (y2-y1) / (x2-x1). The slopes k1, k2, k3, and k4 of each edge are related to the direction and shape of the rectangular frame. By combining the width, length, and slope, the complete geometric feature information of the target area is obtained, which will be used for subsequent mapping point determination.

[0101] S334: Select any point in the target image as a mapping point of the upper left corner vertex of the minimum circumscribed rectangular box, and determine multiple other mapping points corresponding to the vertices of the minimum circumscribed rectangular box in combination with the geometric feature information.

[0102] Specifically, firstly, an arbitrary point is selected in the occluder image as the mapping point of the upper left corner vertex of the minimum circumscribed rectangular box. This initial mapping point is used to determine the mapping point positions of other vertices. Then, using the width, length and slope information calculated previously, other mapping points corresponding to the vertices pt2, pt3 and pt4 of the target area rectangular box are calculated in sequence; through these mapping points, the position of the rectangular box that matches the shape of the target area can be accurately determined in the occluder image.

[0103] S34: Connect the mapping points to obtain a target rectangular frame, and determine an occlusion area with the same shape as the target area in the target rectangular frame according to the contour position information.

[0104] Specifically, after all the mapping points are obtained, these points are connected to form a new rectangular frame, called the target rectangular frame. This target rectangular frame matches the shape of the original target area. Finally, by combining the contour position information of the target area, an occlusion area that is completely consistent with the shape of the target area is selected in this rectangular frame. The purpose of this step is to find the part that matches the infant's mouth and nose area in the occlusion image, so as to achieve occlusion of the infant's mouth and nose.

[0105] S4: superimposing the preset texture information onto the occluded area, and outputting the infant's facial image after the mouth and nose area of ​​the infant is occluded as training data.

[0106] In one embodiment, see Figure 6 , said S4 comprises:

[0107] S41: replacing corresponding vertices in the original infant face image with mapping points carrying the texture information in the target image according to the contour position information;

[0108] Specifically, first, the previously calculated contour position information is used to determine the required texture area in the target image, which should cover the mouth and nose area in the infant's face image; by mapping the texture mapping points in the target image to the corresponding positions in the original infant's face image, the texture information is accurately replaced with the corresponding vertices in the original image. This step ensures that the texture information can be accurately aligned with the occluded area, making the occlusion effect more natural and realistic.

[0109] S42: Based on the replaced mapping point position information, a line scanning method is used to replace each pixel point in the target area one by one through each pixel point in the occluded area, and the replaced image is output as an infant facial image after the infant's mouth and nose area is occluded.

[0110] Specifically, a line scanning method is used to replace the target area one by one at the pixel level. Specifically, the position information of the replaced mapping points is used as a reference to perform line-by-line scanning in the occluded area of ​​the target image. During the scanning process, the corresponding pixels in the original infant facial image are replaced with texture pixels in the target image one by one. This method ensures that every detail of the texture information is accurately applied to the occluded area, achieving a realistic occlusion effect. Finally, the output image shows the processed infant facial image, and the mouth and nose areas are occluded by the preset texture, so that it has the realism required for the training model.

[0111] Example 2

[0112] See also Figure 7 Embodiment 2 of the present invention further provides a device for generating training data for an infant mouth and nose occlusion analysis model, the device comprising:

[0113] A data collection module, used to obtain initial position information of a preset infant mouth and nose region in a pre-collected infant face image to be occluded;

[0114] A feature extraction and smoothing processing module, used to extract contour features and perform smoothing processing on the mouth and nose area of ​​the infant according to the initial position information, and determine the contour position information of the target area after the smoothing processing;

[0115] An occlusion region extraction module is used to obtain a pre-collected occlusion image set, and select an occlusion region having the same shape as the target region from the occlusion image set according to a preset rule in combination with the contour position information;

[0116] The occlusion processing module is used to superimpose preset texture information on the occlusion area, and output the infant's facial image after the occlusion processing of the infant's mouth and nose area as training data.

[0117] Specifically, a device for generating training data for an infant mouth and nose occlusion analysis model provided by an embodiment of the present invention is used, and the device includes: a data collection module, used to obtain initial position information of a preset infant mouth and nose area in a pre-collected infant facial image to be occluded; a feature extraction and smoothing processing module, used to perform contour feature extraction and smoothing processing on the infant mouth and nose area based on the initial position information, and determine the contour position information of the target area after smoothing; an occlusion area extraction module, used to obtain a pre-collected occlusion image set, and select an occlusion area with the same shape as the target area from the occlusion image set according to preset rules in combination with the contour position information; an occlusion processing module, used to superimpose preset texture information on the occlusion area, and output the infant facial image after the occlusion processing of the infant mouth and nose area as training data. This device combines the initial position information and contour features of the infant's mouth and nose area for precise smoothing, ensuring that the shape of the occluded area matches the natural contour of the infant's face, and then selects suitable occluders from the pre-collected occluder image set for superposition, generating realistic and detailed simulated mouth and nose occlusion images, improving the authenticity of the data and helping the model to better cope with recognition and processing problems under occlusion during training. In addition, through the superposition of preset rules and texture information, the generated image is closer to the real scene in shape, texture, and occlusion effect, which is conducive to improving the model's generalization ability and ability to cope with complex occlusion scenarios.

[0118] Example 3

[0119] In addition, combined Figure 1 The method for generating training data for the infant mouth and nose occlusion analysis model described in Example 1 of the present invention can be implemented by an electronic device. Figure 8 A schematic diagram of the hardware structure of an electronic device provided in Embodiment 3 of the present invention is shown.

[0120] The electronic device may include a processor and a memory storing computer program instructions.

[0121] Specifically, the processor may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0122] The memory may include a large capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory may include a removable or non-removable (or fixed) medium. Where appropriate, the memory may be inside or outside the data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0123] The processor implements any one of the methods for generating training data for an infant mouth and nose occlusion analysis model in the above-mentioned embodiments by reading and executing computer program instructions stored in the memory.

[0124] In one example, the electronic device may further include a communication interface and a bus. Figure 8 As shown, the processor, memory, and communication interface are connected via a bus and communicate with each other.

[0125] The communication interface is mainly used to implement communication between the modules, devices, units and / or equipment in the embodiments of the present invention.

[0126] The bus includes hardware, software or both, and the components of the device are coupled to each other. For example, but not limitation, the bus may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industrial standard architecture (EISA) bus, a front-side bus (FSB), a hypertransport (HT) interconnection, an industrial standard architecture (ISA) bus, an infinite bandwidth interconnection, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus or other suitable bus or a combination of two or more of these. Where appropriate, the bus may include one or more buses. Although the embodiments of the present invention describe and illustrate a specific bus, the present invention contemplates any suitable bus or interconnection.

[0127] Example 4

[0128] In addition, in combination with the method for generating training data for the infant mouth and nose occlusion analysis model in the above embodiment 1, embodiment 4 of the present invention can also provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any method for generating training data for the infant mouth and nose occlusion analysis model in the above embodiments is implemented.

[0129] In summary, the embodiments of the present invention provide a method and device for generating training data for an infant mouth and nose occlusion analysis model.

[0130] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0131] The functional blocks shown in the structural block diagram described above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" may include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0132] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant location, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0133] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0134] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A method for generating training data for an infant mouth and nose occlusion analysis model, characterized in that: The method comprises: S1: Acquire initial position information of a preset infant mouth and nose region in a pre-collected infant face image to be occluded; S2: extracting contour features and performing smoothing on the mouth and nose area of ​​the infant based on the initial position information, and determining contour position information of the target area after the smoothing process; S3: obtaining a pre-collected occlusion image set, combining the contour position information, and selecting an occlusion area having the same shape as the target area from the occlusion image set according to a preset rule; S4: superimposing the preset texture information to the occluded area, and outputting the infant's facial image after the mouth and nose area of ​​the infant is occluded as training data; The S3 includes: S31: Obtain the area of ​​each occluder image in the occluder image set, and compare the area of ​​each occluder image with the calculated target area; S32: When the area of ​​the occluder image is larger than the area of ​​the target region, extracting the corresponding occluder image as the target image; S33: Calculating geometric features of the target area, and selecting a plurality of mapping points in the target image based on the geometric feature information obtained by calculation; S34: Connect the mapping points to obtain a target rectangular frame, and determine an occlusion area with the same shape as the target area in the target rectangular frame according to the contour position information.

2. The method for generating training data for an infant mouth and nose occlusion analysis model according to claim 1, characterized in that: The S2 includes: S21: extracting contour features of the mouth and nose area of ​​the infant based on the initial position information to obtain contour point position information of the mouth and nose area; S22: Processing the contour point position information to determine vertex position information of each vertex of the minimum circumscribed polygon of the mouth and nose area; S23: according to the vertex position information, taking the area surrounded by the minimum circumscribed polygon as the target area; S24: Extract features of the target area to determine the contour position information.

3. The method for generating training data for an infant mouth and nose occlusion analysis model according to claim 1, characterized in that: The S24 includes: S241: Calculate the target area to determine the minimum circumscribed rectangle of the minimum circumscribed polygon; S242: selecting a point closest to the left vertex from among the vertices as an initial vertex based on the left vertex position information of the minimum circumscribed rectangle and the vertex position information; S243: The initial vertex is taken as the first point, and other vertices other than the initial vertex are arranged in a clockwise manner, and position information corresponding to each vertex is output as the contour position information.

4. The method for generating training data for an infant mouth and nose occlusion analysis model according to claim 1, characterized in that: The S33 includes: S331: Obtain the minimum circumscribed rectangular frame of the minimum circumscribed polygon of the target area, and calculate the coordinates of each vertex of the minimum circumscribed rectangular frame; S332: performing difference calculation on the coordinates of each vertex to determine the width and length of the minimum circumscribed rectangular frame; S333: Calculate the slope of each vertex coordinate to determine the slope of each side of the minimum circumscribed rectangular frame, and use the width, length and slope as the geometric feature information; S334: Select any point in the target image as a mapping point of the upper left corner vertex of the minimum circumscribed rectangular box, and determine multiple other mapping points corresponding to the vertices of the minimum circumscribed rectangular box in combination with the geometric feature information.

5. The method for generating training data for an infant mouth and nose occlusion analysis model according to claim 4, characterized in that: The S4 includes: S41: replacing corresponding vertices in the original infant face image with mapping points carrying the texture information in the target image according to the contour position information; S42: Based on the replaced mapping point position information, a line scanning method is used to replace each pixel point in the target area one by one through each pixel point in the occluded area, and the replaced image is output as an infant facial image after the infant's mouth and nose area is occluded.

6. The method for generating training data for an infant mouth and nose occlusion analysis model according to claim 1, characterized in that: Before S1, it also includes: obtaining the whitened area position information of the infant's facial image after the user processes it on a preset graphic software interface, and using the whitened area position information as initial position information; the infant's mouth and nose area at least includes the infant's mouth and nose.

7. A device for generating training data for an infant mouth and nose occlusion analysis model, characterized in that: The device comprises: A data collection module, used to obtain initial position information of a preset infant mouth and nose region in a pre-collected infant face image to be occluded; A feature extraction and smoothing processing module, used to extract contour features and perform smoothing processing on the mouth and nose area of ​​the infant according to the initial position information, and determine the contour position information of the target area after the smoothing processing; An occlusion region extraction module is used to obtain a pre-collected occlusion image set, and select an occlusion region having the same shape as the target region from the occlusion image set according to a preset rule in combination with the contour position information; An occlusion processing module, used to superimpose preset texture information on the occlusion area, and output an infant face image after the occlusion processing of the infant's mouth and nose area as training data; The obtaining of the pre-collected occlusion image set, combining the contour position information, and selecting an occlusion area having the same shape as the target area from the occlusion image set according to a preset rule comprises: Obtain the image area of ​​each occluder in the occluder image set, and compare the image area of ​​each occluder with the calculated target area; When the area of ​​the occluder image is larger than the area of ​​the target region, extracting the corresponding occluder image as the target image; Calculating geometric features of the target area, and selecting a plurality of mapping points in the target image based on the geometric feature information obtained by the calculation; The mapping points are connected to obtain a target rectangular frame, and according to the contour position information, an occlusion area having the same shape as the target area is determined in the target rectangular frame.

8. An electronic device, characterized in that: include: At least one processor, at least one memory and computer program instructions stored in the memory, when the computer program instructions are executed by the processor, implement the method according to any one of claims 1 to 6.

9. A storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images