Infant image classification electronic album optimization method and device, equipment and medium

Abstract

The present application belongs to the technical field of video clip, solves the technical problem that the user stickiness is reduced due to the fact that the aesthetic needs of the user difference cannot be met by using the automatically generated electronic album, and provides a kind of classification electronic album optimization method, device, equipment and medium of baby image, which comprises the following steps: obtaining the operation information of the user on the electronic album of each action category, using the operation information to optimize the preset classification rule in advance, so as to obtain the target classification rule closer to the user's aesthetic concept; using the optimized target classification rule to group each frame image of the newly stored target video with classification; obtaining the new target image group of each action category, through this way, the interest degree of the user on each frame image in the existing electronic album can be improved, and more action category corresponding electronic album can be refined, so as to obtain the recognition of the user on the electronic album, and improve the stickiness of the user to the product.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video clips, in particular to a classification electronic album optimization method, device and equipment for baby images and a medium. BACKGROUND

[0002] With the deepening of urbanization, young parents cannot take care of babies and infants by themselves, so they are taken care of by the elderly, which makes young parents unable to better pay attention to the growth of babies and infants. If this problem is not solved, young parents choose to use AI devices to assist in the care of babies and infants. In order to better serve customers, the AI device will classify each frame of image of the recorded baby video according to the preset rule to establish an electronic album, so as to establish the wonderful moments in the growth process of the baby, increase user stickiness and care effect.

[0003] However, the existing technology, the electronic album automatically generated by the existing classification rule of the AI intelligent care device has rough classification, and the baby motion images of the electronic album of different users have no difference, which not only cannot meet the aesthetic difference needs between users, but also causes negative evaluation of the electronic album by users, resulting in a decrease in user interest in the product and a loss of user stickiness, which is not conducive to the promotion of the AI intelligent care device. SUMMARY

[0004] Therefore, the embodiments of the present application provide a classification electronic album optimization method, device, equipment and medium for baby images, which solves the technical problem that the automatically generated electronic album cannot meet the aesthetic needs of different users, resulting in a decrease in user stickiness.

[0005] The technical solution adopted by the present application is:

[0006] The present application provides a classification electronic album optimization method for baby images, which comprises:

[0007] S1: obtaining a target video to be processed and operation information of each electronic album corresponding to each type of image automatically generated by a user, the operation information comprising first operation information for the album itself and second operation information for each image in the album;

[0008] S2: optimizing the preset classification rule of the image according to the operation information corresponding to each electronic album to obtain a target classification rule;

[0009] S3: classifying each frame of image of the target video by using the target classification rule to obtain a target image group corresponding to each image category one by one;

[0010] S4: generating each target electronic album corresponding to each target image group according to each target image group and each electronic album and each image in each electronic album.

[0011] Preferably, the S1 comprises:

[0012] S11: acquiring an image screening rule and a basic video to be processed;

[0013] S12: screening each frame image of the basic video by using the image screening rule to obtain each target image;

[0014] S13: generating the target video according to the shooting time sequence of each target image.

[0015] Preferably, the S12 comprises:

[0016] S121: acquiring a first coincidence threshold between adjacent images, a second coincidence threshold between two frame images at least one frame image apart, and a continuous image number threshold meeting similarity requirements;

[0017] S122: segmenting the basic video according to the first coincidence threshold and the second coincidence threshold to obtain each basic video segment;

[0018] S123: comparing the total number of images of each basic video segment with the continuous image threshold, and outputting each target video segment, the target video segment being the basic video segment with the total number of images greater than or equal to the continuous image number threshold;

[0019] Wherein, each target image is each frame image of each target video segment.

[0020] Preferably, the S2 comprises:

[0021] S21: acquiring a vector difference between each feature vector corresponding to each image in each electronic album before and after user operation corresponding to the operation information and the preset classification rule;

[0022] S22: adjusting each feature vector corresponding to the preset classification rule according to each vector difference to obtain the target classification rule corresponding to each electronic album.

[0023] Preferably, the S3 comprises:

[0024] S31: acquiring first imaging information of infant facial imaging and second imaging information of infant mouth imaging in each frame image;

[0025] S32: filtering out a group of smiley face basic images meeting requirements according to the image proportion of each of the first imaging information and the target classification rule corresponding to the smiley face image;

[0026] S33: filtering each frame image of the group of smiley face basic images according to the image proportion and the image roundness of the second imaging information, to obtain the target image group meeting the smiley face category requirement;

[0027] The smiley face category at least includes one of the following: laughing, smiling and fake smiling, and the image roundness is a radius ratio of a maximum circumscribed circle to a maximum inscribed circle of an imaging shape of a mouth of the infant.

[0028] Preferably, the S4 comprises:

[0029] S41: obtaining a target scoring rule for scoring each image of each category of images and an image score threshold;

[0030] S42: scoring each image of each target image group and each image of each electronic album by using each target scoring rule, to obtain a score value of each image;

[0031] S43: comparing the score value of each image with the image score threshold, to obtain each target image meeting the requirement in each target image group and each electronic album;

[0032] S44: generating each target electronic album according to each target image corresponding to each target image group and each electronic album.

[0033] Preferably, the S41 comprises:

[0034] S411: obtaining a first time length of an image in a first state, a second time length of the image in a second state and a time length threshold corresponding to the image in the first state, wherein the first state is a display state corresponding to that the image is not enlarged or reduced, and the second state is a display state corresponding to that the image is enlarged or reduced;

[0035] S412: adjusting a preset scoring rule of the image according to the first time length and the second time length corresponding to the enlarged region of each image, to obtain the target scoring rule;

[0036] The first time length corresponding to the preset scoring rule is less than or equal to the time length threshold.

[0037] The application further provides an infant image classification electronic album optimization device, comprising:

[0038] The data acquisition module is used for acquiring a target video to be processed and operation information of a user on each electronic album corresponding to each type of image automatically generated, wherein the operation information comprises first operation information on the album itself and second operation information on each image in the album.

[0039] The data processing module is used for optimizing a preset classification rule of the image according to the operation information corresponding to each electronic album, to obtain a target classification rule.

[0040] The image grouping module is used for classifying each frame image of the target video by using the target classification rule, to obtain a target image group corresponding to each image category.

[0041] The album generation module is used for generating each target electronic album corresponding to each target image group according to each target image group and each electronic album and the images already in each electronic album.

[0042] The application further provides an electronic device, comprising at least one processor, at least one memory and computer program instructions stored in the memory, when the computer program instructions are executed by the processor, the method described in any one of the above is realized.

[0043] The application further provides a medium, which stores computer program instructions, when the computer program instructions are executed by a processor, the method described in any one of the above is realized.

[0044] In summary, the beneficial effects of the application are as follows:

[0045] The application provides an image classification electronic album optimization method, device, equipment and medium for infants, which acquires operation information of a user on each action category electronic album, optimizes a preset classification rule according to the operation information, to obtain a target classification rule closer to the user's aesthetic concept, classifies each frame image of a new stored target video with the classification according to the optimized target classification rule, obtains a target image group of each new action category, and through the method, the interest of the user on each frame image in the existing electronic album is improved, more action category corresponding electronic albums are refined, the user's recognition of the electronic album is obtained, and the user's stickiness to the product is improved. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments of the application. For those skilled in the art, other drawings can also be obtained according to these drawings without creative labor, and these are within the protection scope of the application.

[0047] Figure 1A flowchart of an image classification electronic album optimization method for infant images in embodiment 1 of the present application;

[0048] Figure 2 A flowchart of a target video acquisition method in embodiment 1 of the present application;

[0049] Figure 3 A flowchart of an image classification rule optimization method in embodiment 1 of the present application;

[0050] Figure 4 A flowchart of an image grouping method in embodiment 1 of the present application;

[0051] Figure 5 A flowchart of a target electronic album generation method in embodiment 1 of the present application;

[0052] Figure 6 A structural diagram of an image classification electronic album optimization device for infant images in embodiment 2 of the present application;

[0053] Figure 7 A structural diagram of an electronic device in embodiment 3 of the present application. DETAILED DESCRIPTION

[0054] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in detail with reference to the drawings in the embodiments of the present application. It should be noted that, in this document, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or sequence between these entities or operations. In the description of the present application, it should be understood that the orientations or positional relationships indicated by terms such as center, upper, lower, front, rear, left, right, vertical, horizontal, top, bottom, inner, and outer are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. Moreover, the terms “include”, “contain” or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the elements defined by the statement “include” do not exclude the presence of other identical elements in the process, method, article or device that includes the elements. If there is no conflict, the embodiments of the present application and the various features in the embodiments can be combined with each other, and are all within the protection scope of the present application.

[0055] Embodiment 1

[0056] With the process of urbanization, young parents gradually recognize the use of AI devices for baby care. In order to better serve customers, in the AI smart care device, the images captured or the frames of the recorded video are classified by preset image classification rules to obtain different categories of image groups to establish corresponding electronic albums for users to watch. The categories of electronic albums include, but are not limited to, albums corresponding to actions such as laughing, crying, crying, making a ghost face, being cute, and being surprised. However, the electronic albums generated directly by the preset classification rules often have image classification errors or do not meet the individual differences of users, resulting in poor effects of the electronic albums. This will gradually exhaust the user's interest in the product, which is not conducive to the promotion of AI smart care.

[0057] See Figure 1 , Figure 1 The flowchart of the method for optimizing the classification of baby images in Embodiment 1 of the present application is shown. The method comprises:

[0058] S1: obtaining a target video to be processed and user operation information on each electronic album corresponding to each type of image automatically generated, the operation information including first operation information on the album itself and second operation information on each image in the album;

[0059] Specifically, in the process of 24-hour baby care by an AI caretaker (the AI caretaker is an intelligent camera configured with a program capable of identifying various activity states of a baby), video data generated in the monitoring process is recorded in a loop according to a preset time length, in order to save the wonderful actions of the baby, the preset snapshot rule is often used to take snapshots or analyze each frame of image in the recorded video data, so as to screen out images meeting the requirements, then the captured images and / or screened images are classified according to action categories, and a corresponding electronic album is generated for users to view; for example, the snapshot rule or the screening rule is that when the imaging area of the baby in the image accounts for more than a preset value of the whole image, the snapshot or screening is performed to obtain a target image; then each target image is classified according to action categories, including but not limited to recognizing limb actions or micro expressions; it should be noted that the preset snapshot rule or the screening rule includes but is not limited to detecting face information, the centering degree of the baby image, and the coincidence degree of multiple consecutive images, which are not limited herein. The first operation information includes but is not limited to user adjustment of the name of the automatically generated electronic album, adjustment of the name of each image, deletion of images in the electronic album, movement of images between different electronic albums (including but not limited to moving images in electronic album A to electronic album B, adding images not selected in the basic video to the corresponding electronic album, and establishing a new electronic album by using the moved images), and the like, and the second operation information includes but is not limited to cropping images, splicing multiple images, picture adjustment (brightness adjustment, saturation adjustment, hue adjustment, etc.), and adding decoration elements (such as animal shapes, cartoon shapes, words, pendants, and beautiful pictures) to images.

[0060] In an embodiment, referring to Figure 2 , the S1 includes:

[0061] S11: acquiring an image screening rule and a basic video to be processed;

[0062] S12: screening each frame of image of the basic video by using the image screening rule to obtain each target image;

[0063] S13: generating the target video according to the shooting time sequence of each target image.

[0064] Specifically, the image screening rule is used to screen each frame image of the latest stored base video to obtain a target image meeting the requirements, and the image screening rule includes, but is not limited to, the coincidence degree between adjacent images, the number of consecutive images meeting the coincidence degree requirements, the image file size, the baby imaging size, the baby image position and the baby imaging integrity, etc. For example, the file size of the target image is greater than or equal to 512k, the coincidence degree between the adjacent two images is greater than 80%, the number of images between the two frames of images corresponding to the coincidence degree greater than 50% is not less than 5, the proportion of the baby image area in the whole image is greater than 1 / 2, the baby image needs to contain all parts of the baby's body, or at least contain the area above the baby's legs. The image screening rule is used to screen each frame image of the base video to obtain each frame image meeting the requirements to form a target image.

[0065] In an embodiment, the S12 comprises:

[0066] S121: acquiring a first coincidence degree threshold between adjacent images, a second coincidence degree threshold between two frames of images at least one frame of image apart and a consecutive image number threshold meeting the similarity requirements;

[0067] S122: segmenting the base video according to the first coincidence degree threshold and the second coincidence degree threshold to obtain each base video segment;

[0068] S123: comparing the total number of images of each base video segment with the consecutive image threshold, and outputting each target video segment, wherein the target video segment is the base video segment with the total number of images greater than or equal to the consecutive image number threshold;

[0069] Wherein, each target image is each frame image of each target video segment.

[0070] Specifically, by continuously comparing the coincidence degrees between each frame of image, the coincidence degree comparison includes the comparison between two adjacent frames of image, and the coincidence degree between the first frame of image and the current frame of image of each frame of image with the coincidence degree meeting the requirement, for example: the video frame rate is 20 frames / s, each basic video is 30s, when the coincidence degree between the 9th frame of image and the 10th frame of image is greater than the first coincidence degree threshold (meeting the coincidence degree requirement), when the coincidence degree between the 1st frame of image and the 10th frame of image is less than the second coincidence degree threshold (not meeting the coincidence degree requirement), the first coincidence degree and the second coincidence degree corresponding to the 2nd frame of image to the 9th frame of image all meet the requirement, at this time, the 1st frame of image is the first frame of image, and the current frame of image is the 2nd frame of image to the 10th frame of image in sequence; because the coincidence degree between the 1st frame of image and the 10th frame of image is less than the second coincidence degree threshold (not meeting the coincidence degree requirement), so the 10th frame of image will be used as the first frame of image in the next stage. By this way, the basic video is divided into multiple video ends, and then each basic video segment is screened by using the continuous image threshold, each video segment with stable picture is obtained as a target video segment, so as to complete the screening of each frame of image of the basic video, and each target image is obtained. By this way, the images with poor quality caused by too fast picture change in the basic video can be deleted, and the data processing amount of the image classification in the later stage is reduced.

[0071] S2: optimizing a preset classification rule of the image according to the operation information corresponding to each electronic album to obtain a target classification rule;

[0072] Specifically, a large number of images of various actions of infants are obtained, the action categories corresponding to each image are labeled to form a sample set for model training, the images of various actions of infants in the sample set are used to train the model to obtain a preset classification rule corresponding to each action of infants. When each frame of image of a target video is sent into the action classification model, each frame of image will be output according to the category according to each preset classification rule, so as to establish a corresponding electronic album according to each action category. The user can operate the images in the electronic album according to his own individual differences, and the preset classification rule of the image is optimized according to the operation information of the user, so that the classification rule of the image is more in line with the individual differences of different users, for example: all the images with hand information in the face area in the electronic album of the smiling face of the infant are screened out to establish a new electronic album, and the smiling face images with too large face size are deleted, and the preset classification rule is optimized according to the image information corresponding to the operation information before and after, and the smiling face images corresponding to the above features are output by using the optimized target classification rule when classifying the smiling face images.

[0073] In an embodiment, referring to Figure 3 , the S2 includes:

[0074] S21: obtaining vector difference values between each feature vector corresponding to each image in each electronic album before and after user operation and the preset classification rule corresponding to the operation information;

[0075] Specifically, the feature information of the watch image is represented by a multi-dimensional feature vector, such as a 256-dimensional feature vector or a 512-dimensional feature vector. The multi-dimensional feature vector of each image in the electronic album before and after user operation is obtained, the vector difference value of the multi-dimensional feature vector is calculated, and the preset classification rule for image classification is obtained. It should be noted that the feature information corresponding to the preset classification rule corresponds to the feature information of each type of action, that is, the vector value of the multi-dimensional vector is used to determine the category to which the image belongs.

[0076] S22: adjusting each feature vector corresponding to the preset classification rule according to each vector difference value to obtain the target classification rule corresponding to each electronic album.

[0077] Specifically, the value of the feature vector corresponding to the preset classification rule is adjusted according to each vector difference value, including but not limited to adjusting the vector value of the feature vector corresponding to the preset classification rule by using the average value of each feature vector corresponding to each vector dimension of each frame image in the adjusted electronic album. For example, in the electronic album of smiling faces, the user adjusts the brightness of image A, then the average value of the feature vectors representing brightness of all images in the electronic album (including the image A after adjusting the brightness) is calculated, and the difference value between the feature vector representing brightness before adjustment and the average value of the feature vector representing brightness after adjustment is calculated. The difference value is used to adjust the feature value of the feature vector representing brightness in the preset classification rule of the smiling face to obtain the target classification rule of the new feature vector representing brightness corresponding to the classification rule of the smiling face. In this way, the classification rule of each action category is continuously optimized, and an electronic album meeting the needs of different users is established.

[0078] In an embodiment, the S2 includes:

[0079] First step: obtaining album name information corresponding to the first operation information and feature vectors representing image information of each image in the album;

[0080] Specifically, the album name information includes the name of a newly created electronic album, the original name and the existing name of an existing album whose name has been adjusted, and the feature vectors include feature vectors before image adjustment and feature vectors after image adjustment.

[0081] Second step, compare each album name with each action category name, if there is an action category corresponding to the electronic album name, use the feature vector of each image in the album to optimize the preset classification rule of the action category, and obtain the target classification rule of the action; if there is no existing action category matching the album name, use the feature vector of each frame image of the album to establish the preset classification rule of the new action category corresponding to the album name; by this method, the action classification result can be continuously optimized, and at the same time, since the baby grows rapidly, this optimization method can adapt to individual differences for classification.

[0082] S3: classifying each frame image of the target video by using the target classification rule to obtain a target image group corresponding to each image category;

[0083] Specifically, each frame image of the target video is classified by using the target classification rule to obtain each group of images corresponding to each electronic album category, denoted as a first image group; and each group of images not belonging to any existing electronic album, denoted as a second image group, the first image group, the second image group, and each group of images corresponding to each electronic album are denoted as each target image group; it should be noted that the first image group and each image in the electronic album corresponding to the action category of the first image group are the same target image group; the target classification rule will generate an image group other than the existing electronic album, thereby realizing the subdivision of the electronic album, so as to realize the differentiated customization of the electronic album of the user and increase the user stickiness.

[0084] In an embodiment, the S3 further includes:

[0085] First step: obtaining target image information of a target object in each image;

[0086] Second step: determining age information of the target object according to size information of the target in the target image information and size information of a reference in background image information, and / or pattern information on the target object in the target image information;

[0087] Third step: screening each frame image according to the age information to obtain the target image group. Specifically, after completing the classification of each frame image according to the action category, the imaging size of the target object is compared with the imaging size of other reference objects in the environment, the reference objects include: sofa, baby bed, vehicle, table, television, etc.; and / or the age of the target object is determined according to the pattern information on the target object and the pattern information in the background image information, if the pattern is a cartoon pattern, it is determined as a baby, thereby screening out baby images.

[0088] In an embodiment, please refer to Figure 4 , the S3 includes:

[0089] S31: obtaining first imaging information of baby face imaging and second imaging information of baby mouth imaging in each frame image;

[0090] S32: screening a required smiley base image group according to image proportion of each first imaging information and the target classification rule corresponding to the smiley image;

[0091] S33: screening each frame image of the smiley base image group according to image proportion and image roundness of the second imaging information, to obtain the target image group meeting the smiley category requirement;

[0092] Wherein, the smiley category at least includes one of the following: laughing, smiling and fake smiling, and the image roundness is the radius ratio of the maximum circumscribed circle and the maximum inscribed circle of the imaging shape of the baby mouth.

[0093] Specifically, in the process of establishing the electronic album of smiley, first, each frame image of the target video is screened according to the target classification rule corresponding to the smiley image, to obtain a smiley base image group corresponding to the smiley action, and then first imaging information of baby face imaging and second imaging information of baby mouth imaging in each frame image of the smiley base image group are obtained; wherein, the first imaging information includes but is not limited to the proportion of the face image size to the whole image, the distance between points corresponding to the geometric shape of the face image, side face, front face, etc., and the second imaging information includes but is not limited to the proportion of the mouth image size to the face, the geometric shape of the mouth, and the roundness of the mouth, etc., the mouth roundness can be calculated by the formula Q = r / R, wherein r is the maximum inscribed circle radius of the geometric shape of the mouth, R is the maximum circumscribed circle radius of the geometric shape of the mouth, and Q is the roundness, when Q tends to 1, the smiley is a laughing smiley, when Q tends to 0.5, the smiley is a smiling smiley, the corresponding value range can be set to distinguish between smiling and laughing, and the value range can be automatically adjusted by the response operation of the user to smiling and / or laughing, such as putting the smiling image into the electronic album of laughing; each frame image of the smiley squeeze image group is screened by combining the proportion of the mouth image size to the face with the roundness, to obtain the target image group of the smiley action.

[0094] S4: generating each target electronic album corresponding to each target image group according to each target image group and each electronic album and the existing images in each electronic album.

[0095] In an embodiment, please refer to Figure 5 , the S4 includes:

[0096] S41: obtaining a target scoring rule for scoring each image of each type of image and an image scoring threshold;

[0097] In an embodiment, the S41 includes:

[0098] S411: Obtain a first duration when the image is in a first state, a second duration when the image is in a second state, and a duration threshold corresponding to the first state, wherein the first state is a corresponding display state in which the image is not enlarged or reduced, and the second state is a corresponding display state in which the image is enlarged or reduced;

[0099] S412: Adjust a preset scoring rule of the image according to the first duration and the second duration corresponding to the enlarged region of each image, to obtain the target scoring rule;

[0100] The first duration corresponding to the preset scoring rule is less than or equal to the duration threshold.

[0101] Specifically, the operation of the user viewing the image is divided into a viewing stage and a viewing end stage. The user can directly view the images of the album, or can view the images of the album after local enlargement. The image state corresponding to direct viewing is recorded as the first state, and the image state of viewing after image enlargement is recorded as the second state. The longer the user viewing time is, the higher the score of the image is. A first scoring weight is set for the first state, and a second scoring weight is set for the second state. The first scoring weight is less than the second scoring weight. The preset scoring rule is adjusted according to the features of the image viewed by the user. For example, the user enlarges and views the eye position of the baby in image A in the smiley electronic album. The feature information of the eye is extracted, and a higher score is given. The user does not enlarge image B, and the first duration is extremely short. The feature information of each part in image B is extracted, and a lower score is given. It should be noted that when the first duration is greater than the threshold, it is considered that the user has left. At this time, there is no continuous viewing of the image, that is, the effective duration of directly viewing the image is less than or equal to the duration threshold. The target image liked by the user is screened through the scoring method, the interest of the user in the electronic album is increased, and the popularization of the AI care machine is facilitated.

[0102] S42: Score each image in each target image group and each electronic album by using each target scoring rule, to obtain a score value of each image;

[0103] S43: Compare the score value of each image with the image score threshold, to obtain each target image in each target image group and each electronic album that meets the requirement;

[0104] S44: Generate each target electronic album according to each target image corresponding to each target image group and each electronic album.

[0105] In an embodiment, the S44 includes:

[0106] S441: Obtain a target image quantity;

[0107] Specifically, the target image quantity includes all images in the electronic album corresponding to the action category of the target image group.

[0108] S442: If the target image quantity is greater than the first quantity threshold and less than the second quantity threshold, a transition electronic album is generated according to the target image group and each target image of the corresponding electronic album;

[0109] S443: If the existing image quantity is greater than the second quantity threshold, the transition electronic album is generated by deleting images according to the arrangement order of each target image at a preset interval image quantity.

[0110] S443: The transition electronic album is optimized according to the similarity between each image in the transition electronic album to obtain the target electronic album.

[0111] Specifically, the first quantity threshold and the second quantity threshold are set for the image quantity contained in the electronic album. When the image quantity of the electronic album is between the first quantity threshold and the second quantity threshold, images with a similarity meeting a requirement are deleted. It should be noted that when the images with a similarity meeting a requirement in the album are deleted, the image quantity after deletion is not less than the first quantity threshold. Therefore, multiple similarity thresholds are set for the similarity of the images. When more images need to be deleted, the similarity threshold is lower. When fewer images need to be deleted, the similarity threshold is higher. Preferably, a dynamic similarity threshold is set, that is, the similarity threshold requirement is increased as the number of images to be deleted decreases. When the image quantity is greater than the second quantity threshold, the two ways of direct deletion and deletion according to similarity are combined for image deletion. The direct deletion way is to delete by a preset interval image quantity, for example, if the album has 70 images and the second quantity threshold is 50, at least 20 images need to be deleted, and the preset interval image quantity is 5 images per interval, then there are 56 images left after deletion. At least 6 images are deleted by similarity. In this way, the image quantity of the electronic album is ensured to be within a reasonable range, the user does not need to spend too much time watching the album, the user interest is avoided from being lost due to too long watching time, and the user stickiness is improved.

[0112] It should be noted that when the image quantity is less than the first quantity threshold, the target electronic album is directly generated without similarity screening, so as to avoid too few images in the electronic album affecting the user viewing experience.

[0113] The classification electronic album optimization method for baby images in the embodiment can obtain operation information of the user on the electronic album of each action category, optimize the preset classification rule in advance according to the operation information, and obtain a target classification rule that is closer to the aesthetic concept of the user; the target classification rule after optimization is used to group each frame image of a new stored target video with classification; and a new target image group of each action category is obtained, so that the interest degree of the user on each frame image in the existing electronic album can be improved, more electronic albums corresponding to the action categories can be refined, the user's approval of the electronic album is obtained, and the stickiness of the user to the product is improved.

[0114] Embodiment 2

[0115] The application further provides a classification electronic album optimization device for baby images, which is used for optimizing the classification electronic album of the baby images, and the device comprises: Figure 6 , and the device comprises:

[0116] The data acquisition module is used for acquiring a target video to be processed and operation information of the user on each electronic album corresponding to each type of image automatically generated, wherein the operation information comprises first operation information on the album itself and second operation information on each image in the album.

[0117] The data processing module is used for optimizing a preset classification rule of the image according to the operation information corresponding to each electronic album, and obtaining a target classification rule.

[0118] The image grouping module is used for classifying each frame image of the target video by using the target classification rule, and obtaining a target image group corresponding to each image category.

[0119] The album generation module is used for generating each target electronic album corresponding to each target image group according to each target image group, each electronic album and each image in each electronic album.

[0120] The classification electronic album optimization device for baby images in the embodiment can obtain operation information of the user on the electronic album of each action category, optimize the preset classification rule in advance according to the operation information, and obtain a target classification rule that is closer to the aesthetic concept of the user; the target classification rule after optimization is used to group each frame image of a new stored target video with classification; and a new target image group of each action category is obtained, so that the interest degree of the user on each frame image in the existing electronic album can be improved, more electronic albums corresponding to the action categories can be refined, the user's approval of the electronic album is obtained, and the stickiness of the user to the product is improved.

[0121] In an embodiment, the data acquisition module comprises:

[0122] The video acquisition unit is used for acquiring an image screening rule and a basic video to be processed.

[0123] An image screening unit screens each frame image of the base video according to the image screening rule to obtain each target image.

[0124] A video generation unit generates the target video according to a shooting time sequence of each target image.

[0125] In an embodiment, the image screening unit comprises:

[0126] An image screening parameter unit obtains a first coincidence threshold between adjacent images, a second coincidence threshold between two images at least one frame apart, and a continuous image quantity threshold that meets a similarity requirement;

[0127] A video segmentation unit segments the base video according to the first coincidence threshold and the second coincidence threshold to obtain each base video segment;

[0128] A target video unit compares the total number of images of each base video segment with the continuous image threshold, and outputs each target video segment, wherein the target video segment is the base video segment whose total number of images is greater than or equal to the continuous image quantity threshold;

[0129] Each target image is each frame image of each target video segment.

[0130] In an embodiment, the data processing module comprises:

[0131] An operation parameter acquisition unit obtains a vector difference between each feature vector corresponding to each image in each electronic album before and after user operation corresponding to the operation information, and the preset classification rule;

[0132] A classification rule optimization unit adjusts each feature vector corresponding to the preset classification rule according to each vector difference to obtain the target classification rule corresponding to each electronic album.

[0133] In an embodiment, the image grouping module comprises:

[0134] An infant imaging information unit obtains first imaging information of infant face imaging and second imaging information of infant mouth imaging in each frame image;

[0135] A smiling face screening unit screens a smiling face base image group that meets the requirement according to an image proportion of each first imaging information and the target classification rule corresponding to the smiling face image.

[0136] The target image screening unit screens each frame image of the smiley face basic image group according to an image proportion and an image circularity of the second imaging information, and obtains the target image group meeting the smiley face category requirement;

[0137] The smiley face category at least includes one of the following: laughing, smiling, and fake smiling, and the image circularity is a radius ratio of a maximum circumscribed circle to a maximum inscribed circle of an imaging shape of a mouth of the infant.

[0138] In an embodiment, the album generation module comprises:

[0139] The score parameter acquisition unit acquires a target score rule and an image score threshold for scoring each image of each category of image;

[0140] The image score value unit scores each image of each target image group and each electronic album by using each target score rule, and obtains a score value of each image;

[0141] The target image generation unit compares the score value of each image with the image score threshold, and obtains each target image meeting the requirement in each target image group and each electronic album;

[0142] The electronic album generation unit generates each target electronic album according to each target image corresponding to each target image group and each electronic album.

[0143] In an embodiment, the score parameter acquisition unit comprises:

[0144] The viewing time length unit acquires a first time length of an image in a first state, a second time length of the image in a second state, and a time threshold corresponding to the image in the first state, wherein the first state is a display state of the image in a non-magnified or non-reduced state, and the second state is a display state of the image in a magnified or reduced state;

[0145] The score rule optimization unit adjusts a preset score rule of the image according to the first time length and the second time length corresponding to the magnified area of each image, and obtains the target score rule;

[0146] The first time length of the preset score rule is less than or equal to the time threshold.

[0147] The classification electronic album optimization device for baby images of the embodiment obtains operation information of the user on the electronic album of each action category, optimizes the preset classification rule in advance according to the operation information, so as to obtain a target classification rule closer to the aesthetic concept of the user, and groups each frame image of the newly stored target video with classification according to the optimized target classification rule, so as to obtain new target image groups of each action category, thereby improving the interest degree of the user on each frame image in the existing electronic album, refining more electronic albums corresponding to the action categories, and obtaining the recognition of the user on the electronic album, and improving the stickiness of the user to the product.

[0148] Embodiment 3

[0149] The present application provides an electronic device and a medium, please refer to Figure 7 , comprising at least one processor, at least one memory and computer program instructions stored in the memory.

[0150] Specifically, the above-mentioned processor can include a central processing unit (CPU), or a specific integrated circuit (Application Specific Integrated Circuit, ASIC), or can be configured to implement one or more integrated circuits of the embodiments of the present application, and the electronic device at least includes one of the following: a camera, a mobile device with a camera, and a wearable device with a camera.

[0151] The memory can include a mass storage for data or instructions. By way of example and not limitation, the memory can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Where appropriate, the memory can include removable or non-removable (or fixed) media. Where appropriate, the memory can be internal or external to the data processing device. In certain embodiments, the memory is a non-volatile solid-state memory. In certain embodiments, the memory includes read-only memory (ROM). Where appropriate, this ROM can be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0152] The processor reads and executes the computer program instructions stored in the memory to implement the classification electronic album optimization method for baby images in any one of the above-mentioned embodiment modes.

[0153] In one example, the electronic device can further include a communication interface and a bus. The processor, the memory, and the communication interface are connected through the bus and complete communication with each other.

[0154] The communication interface is mainly used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application.

[0155] The bus includes hardware, software or both to couple the components of the electronic device to each other. By way of example, and not limitation, the bus can include an accelerated graphics port (AGP) or other graphics bus, an enhanced industry standard architecture (EISA) bus, a front-side bus (FSB), a hypertransport (HT) interconnect, an industry standard architecture (ISA) bus, an infiniband interconnect, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standards association local (VLB) bus, or another suitable bus or combination of two or more of these. Where appropriate, the bus can include one or more buses. Although the present embodiments describe and show a particular bus, the present application contemplates any suitable bus or interconnect.

[0156] To sum up, the embodiments of the present application provide a kind of classification electronic album optimization method, device, equipment and medium of baby image, the operation information of each action category electronic album of user is acquired, with the operation information, the pre-set classification rule is optimized, to obtain the target classification rule more close to the user aesthetic concept;With the target classification rule of optimization to the frame image of each class of target video stored newly with classification is grouped;New target image group of each action is obtained, by this way, the interest of user to each frame image in existing electronic album can be improved, and more action categories corresponding electronic album can be refined, so as to obtain the approval of user to electronic album, improve the stickiness of user to product.

[0157] It needs to be clear that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of simplicity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after understanding the spirit of the present application.

[0158] The functional blocks shown in the structural block diagrams described above can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, functional cards, and the like. When implemented in software, the elements of the present application are program or code segments that are used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link. A "machine-readable medium" includes any medium that can store or transfer information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, and the like. The code segments can be downloaded via computer networks such as the Internet, intranets, and the like.

[0159] Finally, it should be noted that the above-described embodiments are merely intended to illustrate the technical solutions of the present application, and are not intended to limit the present application; even though the present application has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the above-described embodiments, or make equivalent replacements for some or all of the technical features; and these modifications or replacements do not cause the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for optimizing electronic photo albums by classifying images of infants and toddlers, characterized in that, The method includes: S1: Obtain the target video to be processed and the user's operation information on each electronic album corresponding to the automatically generated images. The operation information includes first operation information on the album itself and second operation information on each image in the album. S2: Optimize the preset classification rules of the images according to the operation information corresponding to each of the electronic photo albums to obtain the target classification rules; S3: Classify each frame of the target video using the target classification rules to obtain a target image group that corresponds one-to-one with each image category; S4: Generate each target electronic album corresponding to each target image group based on each target image group, each electronic album and the images already in each electronic album; Wherein, S2 includes: S21: Obtain the vector difference between the feature vectors of each image in each of the electronic albums corresponding to the operation information before and after the user operation, and the preset classification rule; S22: Adjust each feature vector corresponding to the preset classification rule according to the vector difference to obtain the target classification rule corresponding to each electronic album; Wherein, S4 includes: S41: Obtain the target scoring rules and image scoring thresholds for each image of each type; S42: Using the target scoring rules, score each image in each target image group and each image in each electronic album to obtain the score value of each image; S43: Compare the score value of each image with the image score threshold to obtain each target image group and each target image in each electronic album that meets the requirements; S44: Generate each target electronic album based on each target image corresponding to each target image group and each electronic album.

2. The method for optimizing electronic photo albums of infant and toddler images according to claim 1, characterized in that, S1 includes: S11: Obtain image filtering rules and the underlying video to be processed; S12: Filter each frame of the basic video using the image filtering rules to obtain each of the target images; S13: Generate the target video according to the shooting sequence of each target image.

3. The method for optimizing electronic photo albums of infant and toddler images according to claim 2, characterized in that, S12 includes: S121: Obtain the first overlap threshold between adjacent images, the second overlap threshold between two frames of images with at least one frame interval, and the threshold for the number of consecutive images with the required similarity. S122: The basic video is segmented according to the first overlap threshold and the second overlap threshold to obtain each basic video segment; S123: Compare the total number of images in each of the basic video segments with the consecutive image number threshold, and output each target video segment, wherein the target video segment is the basic video segment whose total number of images is greater than or equal to the consecutive image number threshold; Wherein, each of the target images is a frame image of each of the target video segments.

4. The method for optimizing electronic photo albums of infant and toddler images according to any one of claims 1 to 3, characterized in that, S3 includes: S31: Obtain the first imaging information of the infant's face and the second imaging information of the infant's mouth in each frame image; S32: Based on the image proportion of each of the first imaging information and the target classification rule corresponding to the smiley face image, select a set of basic smiley face images that meet the requirements; S33: Based on the image proportion and image roundness of the second imaging information, each frame of the smiley face base image group is filtered to obtain the target image group that meets the requirements of the smiley face category; The smiley face category includes at least one of the following: laughing, smiling, and fake smiling, and the image roundness is the ratio of the radius of the largest circumcircle to the radius of the largest incircle of the image shape of the infant's mouth.

5. The method for optimizing electronic photo albums of infant and toddler images according to claim 1, characterized in that, S41 includes: S411: Obtain the first duration of the image in the first state, the second duration of the image in the second state, and the duration threshold corresponding to the image in the first state, wherein the first state is the display state corresponding to the image not being enlarged or reduced, and the second state is the display state corresponding to the image being enlarged or reduced. S412: Based on the first duration and the second duration corresponding to the magnified area of ​​each image, adjust the preset scoring rules of the image to obtain the target scoring rules; Specifically, the first duration corresponding to the preset scoring rule is adjusted to be less than or equal to the duration threshold.

6. A device for classifying and optimizing electronic photo albums of infants and toddlers, characterized in that, include: Data acquisition module: used to acquire the target video to be processed and the operation information of the user on each electronic album corresponding to the automatically generated images. The operation information includes first operation information on the album itself and second operation information on each image in the album. The data processing module is used to optimize the preset classification rules of images based on the operation information corresponding to each of the electronic albums to obtain the target classification rules. Specifically, it is used to: obtain the vector differences between the feature vectors of each image in each of the electronic albums corresponding to the operation information before and after the user operation, and the preset classification rules; adjust the feature vectors corresponding to the preset classification rules based on the vector differences to obtain the target classification rules corresponding to each of the electronic albums. Image grouping module: used to classify each frame of the target video using the target classification rules to obtain target image groups that correspond one-to-one with each image category; Album generation module: used to generate target electronic albums corresponding to each target image group based on each target image group, each electronic album, and the images already existing in each electronic album. Specifically, it is used to: obtain target scoring rules and image scoring thresholds for scoring each image of various types of images; use each target scoring rule to score each image in each target image group and each electronic album to obtain the score value of each image; compare the score value of each image with the image scoring threshold to obtain each target image in each target image group and each electronic album that meets the requirements; and generate each target electronic album based on each target image corresponding to each target image group and each electronic album.

7. An electronic device, characterized in that, include: At least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method as described in any one of claims 1-5.

8. A medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by a processor, the method as described in any one of claims 1-5 is implemented.

Images