Infant facial image capture and enhancement method, device, equipment and storage medium

Abstract

The present invention relates to the field of image processing technology, solves the problem in the prior art that when capturing and enhancing infant facial images in infant facial recognition scenarios, multiple types of low-quality images appear, and provides an infant facial image capture and enhancement method, device, equipment and storage medium. The method comprises: acquiring multiple frames of real-time images in the scene where the infant is located; performing motion field analysis on each frame of real-time image, extracting multiple frames of first target images in which the horizontal average optical flow velocity in each of the images meets the preset requirements; performing image enhancement on each frame of the first target image, and outputting each frame of the second target image; performing facial analysis on each frame of the second target image, and outputting an image in which the infant's face is located in a preset area in the second target image. The present invention avoids the occurrence of multiple types of low-quality images when capturing and enhancing infant facial images, and improves the accuracy of infant facial recognition.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for capturing and enhancing infant facial images. Background Art

[0002] With the in-depth research on artificial intelligence, image recognition technology is constantly being implemented. In face image recognition technology, the quality of face images is an important basis for image recognition, which directly affects the accuracy and effectiveness of the face image recognition system. In particular, face image recognition requires face images with higher visual quality to ensure the accuracy and effectiveness of face image recognition.

[0003] At present, in the face recognition scene for infants and young children, the quality of the captured infant and young children's face images is not high due to many factors such as external environmental factors, infant and young children's activities, camera parameters and camera placement. For example, when the infant and young children are running fast, the captured infant and young children's face images will be blurred due to the movement of the infant and young children; when the infant and young children are crawling and other activities, the camera is in a fixed placement, and the captured infant and young children's face images may deviate from the center of the picture; when the camera is in insufficient light, it will capture images with high noise and dark light, and when the external light is too strong, it will capture overexposed images. The above images are collectively referred to as low-quality images. Due to the different influencing factors of low-quality images, the data distribution of low-quality images is very different. Even if the complexity of the image enhancement network is increased, it is difficult to use a single image enhancement network to enhance low-quality images with different influencing factors. The image enhancement effect is not good. Moreover, the use of an overly complex image enhancement network will lead to an increase in computing resources and reduce the speed of image enhancement, thereby reducing the recognition speed of infants' faces. At the same time, the increase in a large amount of computing resources is not conducive to deployment in embedded infant care equipment.

[0004] Therefore, how to avoid the occurrence of various types of low-quality images when capturing and enhancing infant facial images in infant facial recognition scenarios is an urgent problem to be solved. Summary of the invention

[0005] In view of this, the present invention provides a method, device, equipment and storage medium for capturing and enhancing infant facial images, so as to solve the problem in the prior art that various types of low-quality images appear when capturing and enhancing infant facial images in infant facial recognition scenarios.

[0006] The technical solution adopted by the present invention is:

[0007] In a first aspect, the present invention provides a method for capturing and enhancing infant facial images.

[0008] The method comprises:

[0009] S1: Acquire real-time facial video data of an infant in a scene where the infant is located, and decompose the video data into multiple frames of real-time images;

[0010] S2: Performing motion field analysis on each frame of real-time image obtained by decomposing the real-time video data, and extracting a plurality of frames of first target images whose horizontal average optical flow velocity in each of the images meets the preset requirements;

[0011] S3: performing image enhancement on each frame of the first target image, and outputting each frame of the second target image for infant face recognition;

[0012] S4: Performing facial analysis on each frame of the second target image, and outputting an image in which the infant's face is located in a preset area of ​​the second target image.

[0013] Preferably, the S2 includes:

[0014] S21: downsampling each frame of the real-time image to a first resolution in sequence, and selecting an original image from each downsampled first resolution image;

[0015] S22: Acquire multiple target pixel points in the original image, and find multiple LK optical flow matching points corresponding to the target pixel points in the next frame image of the target frame image;

[0016] S23: determining the position information of each target pixel point and the position information of the matching point, and obtaining the horizontal average optical flow velocity of the original image according to the position information of the target pixel point and the position information of the matching point;

[0017] S24: repeating steps S21 to S23 for the remaining images in each of the images until the horizontal average optical flow speed of all frame images is obtained;

[0018] S25: Obtain a preset optical flow velocity threshold, restore the image with the smallest average horizontal optical flow velocity in all frame images and not greater than the optical flow velocity threshold to the original resolution before downsampling, and extract it as the first target image.

[0019] Preferably, S3 includes:

[0020] S31: Selecting the first target image and downsampling it to a second resolution to obtain a sub-image;

[0021] S32: inputting the sub-image into a CRF network to obtain multiple frames of images with different exposures corresponding to the sub-image;

[0022] S33: performing weighted processing on the images with different exposures according to preset weighted parameters to obtain weighted processed images;

[0023] S34: obtaining a first mapping matrix according to the first position information of each pixel in the weighted image and the second position information of each pixel in the sub-image;

[0024] S35: Smoothly fill the first mapping matrix to obtain a second mapping matrix corresponding to the original resolution;

[0025] S36: performing a linear transformation on the first target image according to the second mapping matrix to obtain the second target image of each frame.

[0026] Preferably, the S33 includes:

[0027] S331: Obtaining the grayscale value corresponding to each pixel in the images with different exposures;

[0028] S332: performing weighted averaging processing on the grayscale values ​​according to preset weighting parameters, taking the average value after processing as the new grayscale value of each pixel point, and obtaining a weighted processed image.

[0029] Preferably, the S35 includes:

[0030] S351: Acquire first image width information corresponding to the original resolution and second image width information corresponding to the second resolution;

[0031] S352: Acquire third position information of each element in the first mapping matrix, and map each element in the first mapping matrix to a new mapping matrix according to the first image width information and the second image width information;

[0032] S353: Assign values ​​to each element in the new mapping matrix to obtain the second mapping matrix.

[0033] Preferably, the S353 includes:

[0034] S3531: Obtain fourth position information of each element in the new mapping matrix;

[0035] S3532: according to the third position information and the fourth position information, if the element positions in the new mapping matrix are the same as the element positions in the first mapping matrix, assigning the elements in the new matrix to the corresponding element values ​​in the first mapping matrix;

[0036] S3533: Obtain the remaining unassigned elements, select the target element closest to the unassigned element, and assign the element value of the target element to the unassigned element.

[0037] Preferably, the S4 includes:

[0038] S41: inputting each of the second target images into a preset infant face detection model to obtain infant face position information in the second target image;

[0039] S42: Obtaining the center point position information of the infant's face based on the infant's face position information;

[0040] S43: Based on the image side length information of the second target image and the center point position information, when the center point of the infant's face is within a preset range of the second target image, output an image corresponding to the center point of the infant's face.

[0041] In a second aspect, the present invention provides a device for capturing and enhancing infant facial images, the device comprising:

[0042] An image acquisition module, used to acquire real-time facial video data of infants and young children in the scene where the infants and young children are located, and decompose the video data into multiple frames of real-time images;

[0043] A motion field analysis module is used to perform motion field analysis on each frame of real-time image obtained by decomposing the real-time video data, and extract a plurality of frames of first target images in which the average lateral optical flow velocity of each of the images meets the preset requirements;

[0044] An image enhancement module, used for performing image enhancement on each frame of the first target image, and outputting each frame of the second target image for infant face recognition;

[0045] The face analysis module is used to perform face analysis on each frame of the second target image and output an image in which the face of the infant is located in a preset area in the second target image.

[0046] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of the first aspect in the above-mentioned embodiment.

[0047] In a fourth aspect, an embodiment of the present invention further provides a storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method of the first aspect in the above-mentioned embodiment is implemented.

[0048] In summary, the beneficial effects of the present invention are as follows:

[0049] The present invention provides a method, device, equipment and storage medium for capturing and enhancing infant facial images. The method comprises: obtaining real-time infant facial video data in a scene where the infant is located, and decomposing the video data into multiple frames of real-time images; performing motion field analysis on each frame of real-time image obtained by decomposing the real-time video data, and extracting multiple frames of first target images in which the horizontal average optical flow velocity in each of the images meets preset requirements; performing image enhancement on each frame of the first target image, and outputting each frame of second target image for infant facial recognition; performing facial analysis on each frame of the second target image, and outputting an image in which the infant's face is located in a preset area in the second target image. By calculating the average optical flow of the lateral motion of the image, since the smaller the lateral optical flow, the smoother the image motion and the lower the possibility of image blur, the image with the smallest lateral optical flow and meeting the threshold requirement is extracted, avoiding image blur and ensuring image clarity; by identifying the position information of the infant's face, the infant's face is ensured to be located in the center area of ​​the image, thereby avoiding the problem of the captured image not meeting the user's requirements due to the offset of the infant's face in the image; through image enhancement operations, overexposure or too dark images are avoided; and by performing corresponding image processing on various types of low-quality images, the accuracy of infant face recognition is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the drawings required for use in the embodiment of the present invention will be briefly introduced below. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work, and these are all within the protection scope of the present invention.

[0051] Figure 1 This is a schematic diagram of the overall working process of the infant face image capture and enhancement method in Example 1 of the present invention;

[0052] Figure 2 This is a schematic diagram of a process for extracting a first target image in Embodiment 1 of the present invention;

[0053] Figure 3 This is a schematic diagram of a process for enhancing an image in Embodiment 1 of the present invention;

[0054] Figure 4 A schematic diagram of a process for extracting a weighted average image in Embodiment 1 of the present invention;

[0055] Figure 5 This is a schematic diagram of a process for determining a second mapping matrix in Embodiment 1 of the present invention;

[0056] Figure 6 This is a schematic diagram of a process of assigning values ​​to elements in the second mapping matrix in Embodiment 1 of the present invention;

[0057] Figure 7 This is a schematic diagram of the process of performing facial analysis in Embodiment 1 of the present invention;

[0058] Figure 8 is a structural block diagram of an infant facial image capture and enhancement device in Embodiment 2 of the present invention;

[0059] Fig. 9 It is a schematic diagram of the structure of the electronic device in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described in conjunction with the drawings in the embodiment of the present invention. It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. In the description of the present invention, it should be understood that the orientation or position relationship indicated by the terms "center", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such a process, method, article or device. In the absence of further restrictions, the elements defined by the phrase "comprising..." do not exclude the existence of other identical elements in the process, method, article or device comprising the elements. If there is no conflict, the embodiments of the present invention and the various features in the embodiments can be combined with each other, all within the protection scope of the present invention.

[0061] Example 1

[0062] See also Figure 1 Embodiment 1 of the present invention discloses a method for capturing and enhancing infant facial images, the method comprising:

[0063] S1: Acquire real-time facial video data of an infant in a scene where the infant is located, and decompose the video data into multiple frames of real-time images;

[0064] Specifically, real-time video data collected by a camera in an infant face recognition scenario is obtained, and the video data is decomposed into N frames of images, where N is a positive integer; since in face image recognition technology, face image quality is an important basis for image recognition and directly affects the accuracy and effectiveness of the face image recognition system, the N frames of images are processed to obtain face images with higher visual quality to ensure the accuracy and effectiveness of infant face image recognition.

[0065] S2: Performing motion field analysis on each frame of real-time image obtained by decomposing the real-time video data, and extracting a plurality of frames of first target images whose horizontal average optical flow velocity in each of the images meets the preset requirements;

[0066] Specifically, the N frames of images are acquired, and motion field analysis is performed on each frame of image to obtain the average optical flow velocity of the lateral motion of each frame of image, wherein the smaller the average optical flow velocity of the lateral motion is, the smoother the motion of each pixel in the image is, and the image is less likely to be blurred. The image whose lateral average optical flow velocity in each of the N frames of image meets the preset requirements is extracted as the first target image, thereby avoiding image blur and improving the clarity of the final generated image.

[0067] In one embodiment, see Figure 2 , said S2 comprises:

[0068] S21: downsampling each frame of the real-time image to a first resolution in sequence, and selecting an original image from each of the downsampled first resolution images;

[0069] Specifically, the input N frames of images are sent to a sampling pool, and the images in the sampling pool are downsampled to a first resolution in turn. Taking the first resolution of 360*240 as an example, the i-th frame image is selected as the original image from each frame image at a resolution of 360*240, wherein i is a positive integer and i=<N-1, and the downsampling method adopts a method of linearly extracting row and column images.

[0070] S22: Acquire multiple target pixel points in the original image, and find multiple LK optical flow matching points corresponding to the target pixel points in the next frame image of the target frame image;

[0071] Specifically, multiple target pixels are uniformly selected in the i-th frame image. Here, 100 target pixels are selected as an example. The coordinates of these 100 target pixels are output to obtain a target pixel set Pi, and the LK optical flow matching points with the 100 target points in the i-th frame image are calculated in the i+1-th frame image, and all LK optical flow matching points are output to obtain a matching point set Pi+1, where m<=100. The LK optical flow method is based on the following assumptions: Constant brightness: The brightness value (pixel grayscale value) of a pixel is constant as time changes. This is the basic setting of the optical flow method and must be satisfied by all optical flow methods. Small motion: The change in time will not cause a drastic change in position. In this way, the grayscale value change caused by the position change between adjacent frames can be used to obtain the partial derivative of grayscale with respect to position. All optical flow methods must satisfy this. Spatial consistency: That is, the adjacent pixels in the previous frame are also adjacent in the next frame. This is a unique assumption of the LK optical flow method. Because in order to obtain the speed in the x and y directions, it is necessary to establish multiple equations for simultaneous solution. The spatial consistency assumption can use n neighboring pixels to establish n equations. The optical flow method does not need to calculate the descriptors of feature points like ORB, SIFT, and SURF. It uses the distance between descriptors to determine whether feature points match, thereby simplifying the workflow of finding matching points and improving work efficiency.

[0072] S23: determining the position information of each target pixel point and the position information of the matching point, and obtaining the horizontal average optical flow velocity of the original image according to the position information of the target pixel point and the position information of the matching point;

[0073] Specifically, the coordinate position information of all pixels in the target pixel point set and the matching point set in the camera coordinate system is obtained to obtain the target pixel point coordinate position information set Pi(p1(x1,y1),p2(x2,y2),…,p100(x100,y100)) and the matching point coordinate position information set Pi+1(p1(x1,y1),p2(x2,y2),…pm(xm,ym)), where p1, p2…p100 represent the names of points from 1 to 100 respectively; x1, x2…x100 represent the coordinate positions of points from 1 to 100 respectively. The horizontal coordinates, y1, y2…y100, represent the vertical coordinates from 1 to 100 points respectively, and pm(xm,ym) represents the horizontal and vertical coordinates of the mth matching point. According to the above-mentioned target pixel point coordinate position information set and the matching point coordinate position information set, the horizontal average optical flow speed of the i-th frame image is calculated to be avg_flow(x_i)=((Pi+1(p1(x1))-Pi(p1(x1)))+(Pi+1(p(x2))Pi(p(x2)))+...+(Pi+1(pm(x)-Pi(pm(x1)))) / m.

[0074] S24: repeating steps S21 to S23 for the remaining images in each of the images until the horizontal average optical flow speed of all frame images is obtained;

[0075] Specifically, steps S21 to S23 are repeated for the first N-2 frames of the N frames of the sampling pool, so as to obtain the horizontal average optical flow speed corresponding to all N-1 frames of images, and all the horizontal average optical flow speeds are output to obtain the optical flow speed sequence set as follows:

[0076] Flow_avg(avg_flow(x_1),avg_flow(x_2),…,avg_flow(x_N-1)),

[0077] Among them, avg_flow(x_1),avg_flow(x_2),…,avg_flow(x_N-1) respectively represent the horizontal average optical flow speed corresponding to the first frame to the N-1th frame image.

[0078] S25: Obtain a preset optical flow velocity threshold, restore the image with the smallest average horizontal optical flow velocity in all frame images and not greater than the optical flow velocity threshold to the original resolution before downsampling, and extract it as the first target image.

[0079] Specifically, a preset optical flow velocity threshold is obtained, wherein the optical flow velocity threshold can be flexibly set according to different actual application scenarios, and the above-mentioned optical flow velocity sequence set is obtained, each optical flow velocity in the sequence set is sorted, and the corresponding image with the smallest optical flow velocity after sorting and the optical flow velocity less than or equal to the optical flow velocity threshold is extracted and restored to the original resolution before downsampling, and the restored image is extracted as the first target image; and when the optical flow velocities in the optical flow velocity sequence set do not satisfy the rule of being less than or equal to the optical flow velocity threshold, N frames of images are discarded, and the subsequent real-time video data is decomposed and input into the sampling pool to re-extract the first target image.

[0080] S3: performing image enhancement on each frame of the first target image, and outputting each frame of the second target image for infant face recognition;

[0081] Specifically, due to the change in external light intensity when the camera collects video data, excessive light will cause the generated image to be overexposed, and too dark light conditions will cause the generated image to be too dark. Overexposed and too dark images are not conducive to the recognition of infant faces. Therefore, each of the second target images is obtained, and according to the preset image enhancement rules, the enhanced corrected image is output for infant face recognition. The first target image is analyzed and processed for image enhancement, and an HDR corrected image is output, which avoids the situation that the image is overexposed and too dark, and further improves the accuracy of infant face recognition.

[0082] In one embodiment, see Figure 3 , said S3 comprises:

[0083] S31: Selecting the first target image and downsampling it to a second resolution to obtain a sub-image;

[0084] S32: inputting the sub-image into a CRF network to obtain multiple frames of images with different exposures corresponding to the sub-image;

[0085] S33: performing weighted processing on the images with different exposures according to preset weighted parameters to obtain weighted processed images;

[0086] Specifically, a frame of image is selected from each of the first target images extracted and downsampled to the second resolution. The downsampling method adopts a method of linearly extracting row and column images. Taking the second resolution of 640*480 as an example, a sub-image S1 with a resolution of 640*480 is obtained; the CRF network refers to inputting a captured image, generating a mapping function CRF to obtain images with different exposures, inputting a sub-image with a resolution of 640*480 into the CRF network, and obtaining multiple frames of images with different exposures corresponding to the sub-image; after obtaining the weighting parameters preset by the user, weighted processing is performed on each of the images with different exposures to obtain a weighted average image S2. By performing weighted average filtering on images with different exposures, the noise on the image surface is effectively reduced.

[0087] In one embodiment, see Figure 4 , the S33 comprises:

[0088] S331: Obtaining the grayscale value corresponding to each pixel in the images with different exposures;

[0089] S332: performing weighted averaging processing on the grayscale values ​​according to preset weighting parameters, taking the average value after processing as the new grayscale value of each pixel point, and obtaining a weighted processed image.

[0090] Specifically, each pixel in the image with different exposures is obtained as the pixel to be processed. For the current pixel to be processed, a template is selected, which is composed of several pixels adjacent to it, and the gray value of the original pixel is replaced by the average of the template. The so-called weighted average, using this term, means multiplying pixels by different coefficients, that is, some pixels have greater weights than other pixels, thus providing greater importance to each pixel in the mean calculation.

[0091] S34: obtaining a first mapping matrix according to the first position information of each pixel in the weighted image and the second position information of each pixel in the sub-image;

[0092] Specifically, the first position information and the second position information are obtained by acquiring the coordinate position information of each pixel in the weighted average image S2 and each pixel in the sub-image S1 in the camera coordinate system, and the first mapping matrix M1 is obtained by dividing the first position information by the second position information.

[0093] S35: Smoothly fill the first mapping matrix to obtain a second mapping matrix corresponding to the original resolution;

[0094] Specifically, the first mapping matrix M1 is obtained, and the first mapping matrix is ​​smoothly filled. The smooth filling method adopts a block processing and a linear fitting method to obtain a second mapping matrix corresponding to the original resolution.

[0095] In one embodiment, see Figure 5 , the S35 comprises:

[0096] S351: Acquire first image width information corresponding to the original resolution and second image width information corresponding to the second resolution;

[0097] S352: Acquire third position information of each element in the first mapping matrix, and map each element in the first mapping matrix to a new mapping matrix according to the first image width information and the second image width information;

[0098] S353: Assign values ​​to each element in the new mapping matrix to obtain the second mapping matrix.

[0099] In one embodiment, see Figure 6 , the S353 includes:

[0100] S3531: Obtain fourth position information of each element in the new mapping matrix;

[0101] S3532: according to the third position information and the fourth position information, if the element positions in the new mapping matrix are the same as the element positions in the first mapping matrix, assigning the elements in the new matrix to the corresponding element values ​​in the first mapping matrix;

[0102] Specifically, the coordinate position information of all elements in the new mapping matrix obtained in the above step S352 is obtained. If the element positions in the new mapping matrix are the same as the element positions in the first mapping matrix, the elements in the new matrix are assigned the corresponding element values ​​in the first mapping matrix.

[0103] S3533: Obtain the remaining unassigned elements, select the target element closest to the unassigned element, and assign the element value of the target element to the unassigned element.

[0104] Specifically, the elements in M2 whose values ​​have not been assigned are assigned in the following way: a circle with the element M2(x,y) as the center and a radius of r pixels (for example, r is 6), which contains the assigned elements M2(x1,y1), M2(x2,y2), M2(x3,y3), …, M2(xp,yp), select the element values ​​corresponding to the p elements M2(xi,yi) closest to M2(x,y) as the elements of M2(x,y).

[0105] S36: performing a linear transformation on the first target image according to the second mapping matrix to obtain the second target image of each frame.

[0106] Specifically, in the HDR domain, the pixel value of the image is linearly related to the light intensity, so a linear function can be used to express the relationship between the HDR image and multiple frames of differently exposed images. Pk is linearly transformed to obtain the HDR corrected image P_new = Pk.*M2, where .* represents the corresponding multiplication of matrix elements, and the corrected image P_new is output as the enhanced image for infant face recognition. By performing corresponding image processing on various types of low-quality images, the accuracy of infant face recognition is improved.

[0107] S4: Performing facial analysis on each frame of the second target image, and outputting an image in which the infant's face is located in a preset area of ​​the second target image.

[0108] Specifically, due to the young age of infants and young children, for some infants and young children who are able to crawl or walk, the camera in a fixed position cannot well capture the facial image of the infant and young child in the center area of ​​the image. The extracted second target image is analyzed and processed to filter out non-target images in which the infant and young child's face is not in the center area of ​​the image, and only the images in which the infant and young child's face is in the center area of ​​the image in each of the second target images are retained. On the one hand, it ensures that the infant and young child's face is located in the center of the image, thereby avoiding the problem of missing infant and young child facial information in the image due to infant and young child activities, and improving the accuracy of infant and young child facial recognition; on the other hand, only images that are important for facial recognition are retained, and non-target images are discarded, which also reduces subsequent work processes and improves work efficiency.

[0109] In one embodiment, see Figure 7 , said S4 comprises:

[0110] S41: inputting each of the second target images into a preset infant face detection model to obtain infant face position information in the second target image;

[0111] Specifically, multiple frames of extracted second target images are obtained, and each of the second target images is input into a preset infant face detection model. The construction process of the infant face detection model is as follows: a large number of images of infants at various growth stages publicly available in society are collected for analysis, mainly focusing on the daily activity images of younger infants, pre-marking the faces of infants in these daily activity images, and using the marked images as training data for the deep learning model to construct and train a neural network detection algorithm model based on the YoloV6s structure, namely, an infant face detection model. Each of the second target images is analyzed and processed by the above-mentioned infant face detection model to obtain the coordinate position information of the infant's face in the camera coordinate system of the camera, which is recorded as the infant face position information.

[0112] S42: Obtaining the center point position information of the infant's face based on the infant's face position information;

[0113] Specifically, the facial position information of the infant is obtained, and the minimum circumscribed rectangular frame of the infant's face is marked with a virtual straight line, the upper left corner vertex coordinates of the rectangular frame are (x1, y1); the lower right corner vertex coordinates are (x2, y2), the width of the rectangular frame is w, and the height is h. Then, the center point coordinates of the rectangular frame can be calculated to be (x, y, w, h), where x = (x1+x2) / 2; y = (y1+y2) / 2, and the center point coordinates of the rectangular frame are output as the center point position information of the infant's face.

[0114] S43: Based on the image side length information of the second target image and the center point position information, when the center point of the infant's face is within a preset range of the second target image, output an image corresponding to the center point of the infant's face.

[0115] Specifically, since the second target image is an image restored to its original resolution, the original width img_width and the original height img_height of the image at the original resolution are obtained, and the img_width and img_height are used as the image side length information to obtain the above-mentioned center point position information (x, y, w, h), and determine whether the center point of the infant's face is within the preset range of the second target image: if x+w / 2>=1 / 3*img_width, x+w / 2<=2 / 3*img_width and y+h / 2>=1 / 3*img_height, y+h / 2<=2 / 3*img_height, it is considered that the infant's face is in the central area of ​​the image, and the corresponding image is extracted; if the above conditions are not met, the corresponding image is discarded and no extraction is performed.

[0116] Example 2

[0117] See also Figure 8 Embodiment 2 of the present invention further provides a device for capturing and enhancing infant facial images, the device comprising:

[0118] An image acquisition module, used to acquire real-time facial video data of infants and young children in the scene where the infants and young children are located, and decompose the video data into multiple frames of real-time images;

[0119] A motion field analysis module is used to perform motion field analysis on each frame of real-time image obtained by decomposing the real-time video data, and extract a plurality of frames of first target images in which the average lateral optical flow velocity of each of the images meets the preset requirements;

[0120] An image enhancement module, used for performing image enhancement on each frame of the first target image, and outputting each frame of the second target image for infant face recognition;

[0121] The face analysis module is used to perform face analysis on each frame of the second target image and output an image in which the face of the infant is located in a preset area in the second target image.

[0122] Specifically, the infant facial image capture and enhancement device provided by the embodiment of the present invention includes: an image acquisition module, which is used to acquire real-time infant facial video data in the scene where the infant is located, and decompose the video data into multiple frames of real-time images; a motion field analysis module, which is used to perform motion field analysis on each frame of real-time image obtained by decomposing the real-time video data, and extract multiple frames of first target images in which the horizontal average optical flow velocity in each of the images meets preset requirements; an image enhancement module, which is used to perform image enhancement on each frame of the first target image, and output each frame of second target image for infant facial recognition; a facial analysis module, which is used to perform facial analysis on each frame of the second target image, and output an image in which the infant's face is located in a preset area in the second target image. By calculating the average optical flow of the lateral motion of the image, since the smaller the lateral optical flow, the smoother the image motion and the lower the possibility of image blur, the image with the smallest lateral optical flow and meeting the threshold requirement is extracted, avoiding image blur and ensuring image clarity; by identifying the position information of the infant's face, the infant's face is ensured to be located in the center area of ​​the image, thereby avoiding the problem of the captured image not meeting the user's requirements due to the offset of the infant's face in the image; through image enhancement operations, overexposure or too dark images are avoided; and by performing corresponding image processing on various types of low-quality images, the accuracy of infant face recognition is improved.

[0123] Example 3

[0124] In addition, combined Figure 1 The infant facial image capture and enhancement method of the embodiment 1 of the present invention described above can be implemented by an electronic device. Fig. 9 A schematic diagram of the hardware structure of an electronic device provided in Embodiment 3 of the present invention is shown.

[0125] The electronic device may include a processor and a memory storing computer program instructions.

[0126] Specifically, the processor may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present invention.

[0127] The memory may include a large capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory may include a removable or non-removable (or fixed) medium. Where appropriate, the memory may be inside or outside a data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0128] The processor implements any one of the infant facial image capture and enhancement methods in the above embodiments by reading and executing computer program instructions stored in the memory.

[0129] In one example, the electronic device may further include a communication interface and a bus. Fig. 9 As shown, the processor, memory, and communication interface are connected via a bus and communicate with each other.

[0130] The communication interface is mainly used to implement communication between the modules, devices, units and / or equipment in the embodiments of the present invention.

[0131] The bus includes hardware, software or both, and the parts of the device are coupled to each other. For example, but not limitation, the bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In suitable cases, the bus may include one or more buses. Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

[0132] Example 4

[0133] In addition, in combination with the infant facial image capture and enhancement method in the above embodiment 1, embodiment 4 of the present invention can also provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any of the infant facial image capture and enhancement methods in the above embodiments is implemented.

[0134] In summary, the embodiments of the present invention provide a method, apparatus, device and storage medium for capturing and enhancing infant facial images.

[0135] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0136] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0137] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0138] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A method for capturing and enhancing infant facial images, characterized in that: The method comprises: S1: Acquire real-time facial video data of an infant in a scene where the infant is located, and decompose the video data into multiple frames of real-time images; S2: Performing motion field analysis on each frame of real-time image obtained by decomposing the real-time video data, and extracting a plurality of frames of first target images whose horizontal average optical flow velocity in each of the images meets the preset requirements; S3: performing image enhancement on each frame of the first target image, and outputting each frame of the second target image for infant face recognition; S4: performing facial analysis on each frame of the second target image, and outputting an image in which the infant's face is located in a preset area of ​​the second target image; The S2 includes: S21: downsampling each frame of the real-time image to a first resolution in sequence, and selecting an original image from each downsampled first resolution image; S22: Acquire multiple target pixel points in the original image, and find multiple LK optical flow matching points corresponding to the target pixel points in the next frame image of the original image; S23: determining the position information of each target pixel point and the position information of the matching point, and obtaining the horizontal average optical flow velocity of the original image according to the position information of the target pixel point and the position information of the matching point; S24: repeating steps S21 to S23 for the remaining images in each of the images until the horizontal average optical flow speed of all frame images is obtained; S25: Obtain a preset optical flow velocity threshold, restore the image with the smallest average horizontal optical flow velocity in all frame images and not greater than the optical flow velocity threshold to the original resolution before downsampling, and extract it as the first target image.

2. The method for capturing and enhancing infant facial images according to claim 1, characterized in that: The S3 includes: S31: Selecting the first target image and downsampling it to a second resolution to obtain a sub-image; S32: inputting the sub-image into a CRF network to obtain multiple frames of images with different exposures corresponding to the sub-image; S33: performing weighted processing on the images with different exposures according to preset weighted parameters to obtain weighted processed images; S34: obtaining a first mapping matrix according to the first position information of each pixel in the weighted image and the second position information of each pixel in the sub-image; S35: Smoothly fill the first mapping matrix to obtain a second mapping matrix corresponding to the original resolution; S36: performing a linear transformation on the first target image according to the second mapping matrix to obtain the second target image of each frame.

3. The method for capturing and enhancing infant facial images according to claim 2, characterized in that: The S33 includes: S331: Obtaining the grayscale value corresponding to each pixel in the images with different exposures; S332: performing weighted averaging processing on the grayscale values ​​according to preset weighting parameters, taking the average value after processing as the new grayscale value of each pixel point, and obtaining a weighted processed image.

4. The method for capturing and enhancing infant facial images according to claim 2, characterized in that: The S35 includes: S351: Acquire first image width information corresponding to the original resolution and second image width information corresponding to the second resolution; S352: Acquire third position information of each element in the first mapping matrix, and map each element in the first mapping matrix to a new mapping matrix according to the first image width information and the second image width information; S353: Assign values ​​to each element in the new mapping matrix to obtain the second mapping matrix.

5. The method for capturing and enhancing infant facial images according to claim 4, characterized in that: The S353 includes: S3531: Obtain fourth position information of each element in the new mapping matrix; S3532: according to the third position information and the fourth position information, if the element positions in the new mapping matrix are the same as the element positions in the first mapping matrix, assigning the elements in the new matrix to the corresponding element values ​​in the first mapping matrix; S3533: Obtain the remaining unassigned elements, select the target element closest to the unassigned element, and assign the element value of the target element to the unassigned element.

6. The method for capturing and enhancing infant facial images according to any one of claims 1 to 5, characterized in that: The S4 includes: S41: inputting each of the second target images into a preset infant face detection model to obtain infant face position information in the second target image; S42: Obtaining the center point position information of the infant's face based on the infant's face position information; S43: Based on the image side length information of the second target image and the center point position information, when the center point of the infant's face is within a preset range of the second target image, output an image corresponding to the center point of the infant's face.

7. A device for capturing and enhancing infant facial images, characterized in that: The device comprises: An image acquisition module, used to acquire real-time facial video data of infants and young children in the scene where the infants and young children are located, and decompose the video data into multiple frames of real-time images; A motion field analysis module is used to perform motion field analysis on each frame of real-time image obtained by decomposing the real-time video data, and extract a plurality of frames of first target images in which the average lateral optical flow velocity of each of the images meets the preset requirements; An image enhancement module, used for performing image enhancement on each frame of the first target image, and outputting each frame of the second target image for infant face recognition; A face analysis module, used for performing face analysis on each frame of the second target image, and outputting an image in which the face of the infant is located in a preset area of ​​the second target image; The step of performing motion field analysis on each frame of real-time image obtained by decomposing the real-time video data and extracting a plurality of frames of first target images in which the horizontal average optical flow velocity of each frame meets the preset requirements comprises: Downsampling each frame of the real-time image to a first resolution in sequence, and selecting an original image from each downsampled first-resolution image; Acquire multiple target pixel points in the original image, and find multiple LK optical flow matching points corresponding to the target pixel points in the next frame image of the original image; Determine the position information of each target pixel point and the position information of the matching point, and obtain the horizontal average optical flow velocity of the original image based on the position information of the target pixel point and the position information of the matching point; Repeating the calculation of the horizontal average optical flow velocity for the remaining images in each of the images until the horizontal average optical flow velocity of all frame images is obtained; A preset optical flow velocity threshold is obtained, and an image having the smallest average lateral optical flow velocity in all frame images and not greater than the optical flow velocity threshold is restored to the original resolution before downsampling and extracted as the first target image.

8. An electronic device, characterized in that: include: At least one processor, at least one memory and computer program instructions stored in the memory, when the computer program instructions are executed by the processor, implement the method according to any one of claims 1 to 6.

9. A storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images