A method and device for automatically matching props for taking photos
Through the mobile terminal automatically matches props that match the scene during shooting, the problem of users needing to manually select big heads in the existing technology is solved, achieving a richer shooting experience and diversified decorative effects.
Patent Information
- Application Number
- CN201810824043.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2018-07-25
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2039-02-04
AI Technical Summary
Existing mobile phone photography big head posts require users to manually choose, and cannot be combined with the user's expressions and actions. The expression form is limited, which cannot meet the diversity of users' decorative pictures when taking photos.
When the mobile terminal starts the camera, it automatically selects prop information matching the shooting scene from the preset prop library, and superimposes it with the user image information to generate synthetic images to realize automatic matching of props and decorate.
It increases the interactiveness and surprise of the shooting process, reduces the hassle of manual selection by users, provides richer choices of decorative pictures, and improves the playability of taking photos.
Smart Images

Figure CN110766602B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of mobile terminals, and in particular to a method and device for automatically matching props to take photos. Background Art
[0002] Smartphone photo stickers are widely used by female users. Adding a photo sticker to a selfie and forwarding it to Moments can greatly increase the fun and cuteness. Forwarding to Moments can resonate with friends and trigger more topics. That is, when taking photos, users have the need to add, decorate, and beautify the content of the photo.
[0003] Currently, the photo stickers on mobile phones are mainly selected by users when previewing photos. The photo stickers track the position of the face and combine the face and the photo sticker into a composite photo when taking photos.
[0004] However, the current photo booth solutions have obvious shortcomings. For example, the photo booth needs to be manually selected by the user (when the user changes the photo booth, he needs to select from the candidate library), the photo booth cannot be combined with the user's expression (the user's happy or angry photo booth will not change). The props of the photo booth are mainly hats, ears, eyebrows, beards, blush, etc., and the expression forms are limited.
[0005] In addition to manually taking photos of your own portrait, current technology does not automatically select matching props to decorate the picture based on hand gestures and body movements. Summary of the invention
[0006] The technical problem solved by the solution provided by the embodiment of the present invention is that it cannot satisfy the diversity of users' decoration of pictures when taking photos.
[0007] A method for photographing by automatically matching props according to an embodiment of the present invention includes:
[0008] After the mobile terminal starts the camera and enters the state of automatically decorating the screen with props, it obtains the shooting scene including the user image information and the environment information;
[0009] The mobile terminal selects prop information matching the shooting scene from a preset prop library according to the shooting scene;
[0010] The mobile terminal performs superposition processing on the user image information and the prop information to generate a composite image to be shot.
[0011] A photographing device for automatically matching props according to an embodiment of the present invention includes:
[0012] An acquisition module, used to acquire a shooting scene including user image information and environmental information after the camera is started and the screen enters an automatic decoration prop state;
[0013] A selection module, used to select prop information matching the shooting scene from a preset prop library according to the shooting scene;
[0014] A generation module is used to superimpose the user image information and the prop information to generate a composite image to be shot.
[0015] According to an embodiment of the present invention, a device for automatically matching props for taking photos is provided, and the device includes: a processor, and a memory coupled to the processor; the memory stores a program for automatically matching props for taking photos that can be run on the processor, and when the program for automatically matching props for taking photos is executed by the processor, the steps of the method for automatically matching props for taking photos provided by the embodiment of the present invention are implemented.
[0016] A computer storage medium provided according to an embodiment of the present invention stores a program for automatically matching props for taking photos, and when the program for automatically matching props for taking photos is executed by a processor, the steps of the method for automatically matching props for taking photos provided according to an embodiment of the present invention are implemented.
[0017] According to the solution provided by the embodiment of the present invention, the user starts the mobile phone camera and selects to enter the state of automatic decoration of the screen with props. In the camera preview screen, the user makes various gestures, the camera obtains the preview picture, analyzes the human body posture in the screen, and automatically loads the props matching the current scene on the screen. The props are automatically matched, and the user needs to make a certain gesture to match the appropriate props. If the user wants to match the props he likes, he needs to make realistic movements and gestures, which increases the interactivity of the user's shooting process. The automatically matched props are likely to exceed the user's imagination, increase the user's sense of surprise, and avoid the trouble of manual selection by the user. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 This is a flow chart of a method for automatically matching props for taking photos provided by an embodiment of the present invention;
[0019] Figure 2 Schematic diagram of a photographing device for automatically matching props provided by an embodiment of the present invention;
[0020] Figure 3 is a block diagram of a system for automatically matching props for taking photos provided by an embodiment of the present invention;
[0021] Figure 4 It is a flow chart of a method for automatically matching props for taking photos provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0022] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described below are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0023] Figure 1 is a flow chart of a method for automatically matching props for taking photos provided by an embodiment of the present invention. Figure 1 As shown, the method may include:
[0024] Step S101: after the mobile terminal starts the camera and enters the state of automatic screen decoration props, it obtains a shooting scene including user image information and environmental information.
[0025] In one embodiment, the user image information includes any one or a combination of the following: user facial features, user gesture information, user body posture information, and user voice features; the environmental information includes: shooting time, shooting location, shooting season, and shooting weather; wherein the shooting time includes any one of the following: morning, noon, afternoon, evening, and late night; the shooting season includes any one of the following: spring, summer, autumn, and winter; the shooting weather includes any one of the following: sunny, windy, cloudy, foggy, rainy, snowy, frost, thunder, and hail.
[0026] Step S102: the mobile terminal selects prop information matching the shooting scene from a preset prop library according to the shooting scene.
[0027] In one embodiment, the mobile terminal selects prop information matching the shooting scene from a preset prop library according to the shooting scene, including: the mobile terminal extracts key shooting information in the shooting scene by analyzing the user image information and the environment information in the shooting scene respectively; the mobile terminal selects prop information matching the key shooting information from a preset prop library according to the key shooting information; wherein the prop library stores a shooting prop relationship table showing the correspondence between the key shooting information and the prop information. The prop information includes any one or combination of the following: any applicable prop information such as animal prop information, plant prop information, food prop information, wearable prop information, and game tool information.
[0028] Step S103: the mobile terminal superimposes the user image information and the prop information to generate a composite image to be shot.
[0029] In another embodiment, after the mobile terminal generates the composite image to be shot, it also includes: when the mobile terminal receives the user's shooting instruction, it generates a shooting source composite image and receives the user's operation instruction; when the operation instruction is an image saving instruction, the mobile terminal uses the shooting source composite image as the shooting target composite image and saves the shooting target composite image; when the operation instruction is an image editing instruction, the mobile terminal starts the image prop editing function, and obtains and saves the shooting target composite image by editing and adjusting the props in the shooting source composite image. Specifically, the mobile terminal edits and adjusts the props in the shooting source composite image, including any one or a combination of the following: adding props in the shooting source composite image; deleting props in the shooting source composite image; adjusting the position of props in the shooting source composite image; adjusting the size of props in the shooting source composite image; adjusting the angle of props in the shooting source composite image; and adjusting the color of props in the shooting source composite image.
[0030] Figure 2 is a schematic diagram of a photographing device for automatically matching props provided by an embodiment of the present invention, such as Figure 2 As shown, the device may include: an acquisition module 201, a selection module 202 and a generation module 203.
[0031] The acquisition module 201 is used to acquire a shooting scene including user image information and environmental information after the camera is started and the screen enters the automatic prop decoration state.
[0032] In one embodiment, the user image information acquired by the acquisition module 201 includes any one or a combination of the following: user facial features, user gesture information, user body posture information, and user voice features, etc.; the environmental information includes: shooting time, shooting location, shooting season, and shooting weather, etc.; wherein the shooting time includes any one of the following: morning, noon, afternoon, evening, and late night, etc.; the shooting season includes any one of the following: spring, summer, autumn, and winter, etc.; the shooting weather includes any one of the following: sunny, windy, cloudy, foggy, rainy, snowy, frost, thunder, and hail, etc.
[0033] In one embodiment, the acquisition module 201 may acquire the user's facial features, such as facial expressions, eye sight and other features, by a facial feature recognition method. For example, a neural network face recognition method, a support vector machine face recognition method, etc. The user gesture information may be acquired by a human body feature recognition method to respectively acquire the user gesture information and the user's human body posture information. The user voice features may be acquired by a voice collection device.
[0034] The acquisition module 201 can also read the time, location, weather, season and other data of the shooting through local or network channels to obtain the corresponding environmental information.
[0035] The selection module 202 is used to select prop information matching the shooting scene from a preset prop library according to the shooting scene. In one embodiment, the prop information includes any one or a combination of the following: animal prop information, plant prop information, food prop information, wearable prop information, and game tool information.
[0036] In one embodiment, the selection module 202 includes: an extraction unit, which is used to extract the key shooting information in the shooting scene by analyzing the user image information and the environmental information in the shooting scene respectively; a selection unit, which is used to select the prop information matching the key shooting information from the preset prop library according to the key shooting information; wherein the prop library stores a shooting prop relationship table of the corresponding relationship between the key shooting information and the prop information. For example, in a scene where a person is walking in a park with flowers, the more obvious or prominent information extracted from the user image information and the environmental information is used as the key shooting information, such as the facial information and eye information in the user image information at this time cannot be obtained, so the key shooting information includes the posture of walking, the time in the morning, the location park, etc., then the props such as the wreath and the puppy are selected from the prop library, and the headdress wreath is matched on the head of the female user, and the puppy is matched at the feet, etc. For example, in a scene of a person lying on the grass, the key shooting information extracted from the user's image information and environmental information includes a smile on the face, closed eyes, lying posture, etc., and props such as food or sports awards are selected from the prop library.
[0037] In one embodiment, if the user's posture changes, the user image information analysis is not performed, and the user image information obtained from the motion trajectory 1-2 seconds before the user's action stops is used as the key information for shooting.
[0038] The generating module 203 is used to perform superposition processing on the user image information and the prop information to generate a composite image to be shot.
[0039] An embodiment of the present invention provides a device for automatically matching props for taking photos, the device comprising: a processor, and a memory coupled to the processor; the memory stores a program for automatically matching props for taking photos that can be run on the processor, and when the program for automatically matching props for taking photos is executed by the processor, the steps of the method for automatically matching props for taking photos provided by the embodiment of the present invention are implemented.
[0040] A computer storage medium provided by an embodiment of the present invention stores a program for automatically matching props for taking photos. When the program for automatically matching props for taking photos is executed by a processor, the steps of the method for automatically matching props for taking photos provided by an embodiment of the present invention are implemented.
[0041] Figure 3 is a system block diagram of a photographing device for automatically matching props provided by an embodiment of the present invention. Figure 3 As shown, the system may include a camera unit 301 , a user posture analysis unit 302 , an environment data acquisition unit 303 , a data analysis and matching unit 304 , a prop library 305 and a prop superposition unit 306 .
[0042] Camera unit 301: obtains a photo preview image and takes a photo.
[0043] User posture analysis unit 302: used to analyze the image in the preview image to see whether there is an image of a face, hand or whole body, and provide key information such as the size, position, angle, etc. of the recognized face, hand and whole body in the preview image, obtain the motion trajectory of the user 1-2 seconds before the action stops, and obtain the key data for defining the user posture, as shown in Table 1, to prepare for the next step of prop matching.
[0044] Environmental data acquisition unit 303: obtains data such as time, location, weather, season, etc. when shooting from local or network channels, determines the scene when the user shoots, and prepares for the next step of prop matching, as shown in Table 2.
[0045] Data analysis and matching unit 304: Based on the data obtained by the user posture analysis system and the data obtained by the environmental data acquisition system, the data analysis and matching system selects and determines appropriate props from the prop library to match the current shooting scene.
[0046] Prop library 305: used to store various props, each prop is matched with the keywords in Table 1 and Table 2, such as Table 3, which provides a data analysis matching system to call props. In one embodiment, the prop library can be stored locally or on a cloud server (not shown in the figure).
[0047] Prop superposition unit 306: The user posture analysis system obtains data and scales the props according to the human body proportions. The key point positions of the props are matched with the key point positions of the human body. For example, if the matching prop is a mace, the handle of the mace needs to be connected to the user's waving hand, and the angle of the handle matches the user's hand holding state.
[0048] In order to accurately and clearly analyze the user's posture, the user posture analysis system of the embodiment of the present invention is performed according to a multi-dimensional analysis method. The important dimensions come from the preview image analysis and are closely related to people, as shown in Table 1.
[0049] Dimension 1-1, human trunk and limb posture;
[0050] Dimension 1-2, head posture and binocular sight;
[0051] Dimensions 1-3, facial expressions;
[0052] Dimensions 1-4, human hand movements;
[0053] Dimensions 1-5, scenes around the human body.
[0054] Table 1: Image and video analysis data table
[0055]
[0056] Auxiliary dimensions, come from environmental data collection, as shown in Table 2.
[0057] Dimension 2-1, time;
[0058] Dimension 2-2, location;
[0059] Dimension 2-3, weather;
[0060] Dimensions 2-4, indoors and outdoors.
[0061] Table 2: Environmental collection data table
[0062]
[0063] Table 3: Prop library and dimension attributes Table 1
[0064]
[0065] The principle of automatic prop matching is as follows: the user posture analysis system analyzes the following from the preview image: Dimension 1-1: walking or running; Dimension 1-4: hand holding; the data obtained by the environmental data collection system is: Dimension 2-4: street. The data analysis and matching system searches for the prop with the highest matching degree from the prop library based on the known dimension data. The "dog walking" prop (with the keywords "walking, hand holding, street") is matched in the preview image, as shown in Table 3.
[0066] Automatic selection of props
[0067] The user posture analysis system analyzes the keywords "walking" and "holding" from the preview image. The environmental data collection system collects data keywords such as "rainy" and "street". The data analysis and matching system searches for keywords from the prop library and the dimension attribute table. If there is a match, the position is set to 1, and if there is no match, it is set to 0, as shown in Table 4.
[0068] Table 4: Prop library and dimension attributes Table 2
[0069]
[0070] The data analysis matching system calculates the row data selected by the props. The larger the value, the higher the matching degree.
[0071] Prop n = (dimension 1-1) + (dimension 1-2) + (dimension 1-3) + (dimension 1-4) + (dimension 1-5) + ... + (dimension 2-1) + (dimension 2-2) + (dimension 2-3) + (dimension 2-4) + ...
[0072] Prop 1=1+0+0+0+0 +0+0+0+1=2
[0073] Prop 2=0+0+0+1+0 +0+0+0+0=1
[0074] Prop 3=0+0+0+1+0 +0+0+0+0=1
[0075] Item 4=1+0+0+1+0 +0+0+1+1=4
[0076] According to the calculation results, the data analysis and matching system selects prop 4 (umbrella, the maximum calculation data is 4). If the user enters the prop editing state, the data analysis and matching system recommends prop 4 (umbrella) and prop 1 (dog walking, the second largest calculation result is 2).
[0077] Furthermore, voice collection data may be added to obtain keywords that the user communicates when taking photos, such as the content of the user's chat, the user's laughter, or the user's shouting, etc.
[0078] Figure 4 is a flow chart of a method for automatically matching props for taking photos provided by an embodiment of the present invention. Figure 4 As shown, the method may include:
[0079] Step 1: Initialize the mobile phone camera;
[0080] Step 2: The mobile phone camera enters the state of automatically adding props preview images;
[0081] Step 3: The user strikes a pose and makes a move;
[0082] Step 4: The user posture analysis system analyzes and obtains the key data of the user's current posture from the camera preview image and the user's last action; the environmental data collection system collects environmental data;
[0083] Step 5: The data analysis and matching system selects props that match the current user posture and scene from the prop library based on the data from the user posture analysis system and the environmental data collection system;
[0084] Step 6: Is the prop matching completed?
[0085] If the props are matched, go to step 7, otherwise return to step 3.
[0086] The prop overlay system obtains props from the prop allocation system, obtains the prop overlay position, size, and angle requirements from the image recognition system, and matches the props to the appropriate position.
[0087] Step 7: The user issues a photo-taking command, and the mobile phone camera generates a photo of the content in the preview image;
[0088] Step 8: Are you satisfied with the props match?
[0089] If the user is not satisfied with the matched props, he can go to step 9 to edit the props, adjust the size and position of the props, or reselect the props from the prop recommendation library, otherwise the process ends.
[0090] Step 9: The data analysis and matching system records the user's selection behavior in the prop editing state. When there are multiple candidate props, the props most commonly used by the user are recommended according to user habits.
[0091] In the preview interface, if the facial expression, gestures, body, etc. change, the preview interface will continuously update the matching props and prop size and position to adapt to the new preview image content.
[0092] After taking a photo, if the user is not satisfied with the automatically matched props, the user can return the photo to the editing state, select a favorite prop from several props that match the scene, or adjust the position, angle, etc. of the props, and then save the photo again.
[0093] In addition to mobile phone cameras, the function of the embodiment of the present invention can also be extended to pads, digital cameras and other fields. It can also be extended to video prop matching.
[0094] For example, when the mobile phone camera is turned on and a flower wreath is placed on the head of a female user, the user can be matched with a headdress. When the user is eating, the user can be matched with related food. When the user is writing or working, the user can be matched with a stack of exercise books or official documents. When the user is reading, the user can be matched with a dog reading a book attentively.
[0095] The solution provided by the embodiment of the present invention has the following beneficial effects:
[0096] 1. When taking photos with your mobile phone, add relevant props and match the props function through artificial intelligence, which greatly improves the playability of taking photos.
[0097] 2. Apply the facial prop library to automatically match props that match the current expression for various facial expressions, such as anger, joy, contemplation, open mouth, closed eyes, etc. This can bring surprises to users and also reduce the trouble of users manually selecting headshots.
[0098] 3. The props in the prop library can be open sourced from the Internet and designed by users. It can keep up with market hotspots, supplement a large number of popular comic props, and form good interaction with users.
[0099] Although the present invention is described in detail above, the present invention is not limited thereto, and those skilled in the art can make various modifications based on the principles of the present invention. Therefore, all modifications made based on the principles of the present invention should be understood to fall within the protection scope of the present invention.
Claims
1. A method for automatically matching props for taking photos, characterized in that: include: After the mobile terminal starts the camera and enters the state of automatic screen decoration props, it obtains a shooting scene including user image information and environmental information, wherein the user image information includes any one or a combination of the following: user facial features, user gesture information, user body posture information, and user voice features; The environmental information includes: shooting time, shooting location, shooting season and shooting weather; The mobile terminal extracts key shooting information in the shooting scene by analyzing the user image information and the environment information in the shooting scene respectively; The mobile terminal selects prop information matching the key shooting information from a preset prop library according to the key shooting information; The mobile terminal performs superposition processing on the user image information and the prop information to generate a composite image to be shot.
2. The method according to claim 1, characterized in that The item information includes any one or a combination of the following: animal item information, plant item information, food item information, wearable item information, and game tool information.
3. The method according to claim 2, characterized in that The preset props library stores a shooting props relationship table showing the corresponding relationship between key shooting information and props information.
4. The method according to claim 1, characterized in that: After the mobile terminal generates the composite image to be captured, the method further includes: When the mobile terminal receives a shooting instruction from a user, it generates a composite image of shooting sources and receives an operation instruction from the user; When the operation instruction is an image saving instruction, the mobile terminal uses the shooting source composite image as the shooting destination composite image and saves the shooting destination composite image; When the operation instruction is an image editing instruction, the mobile terminal starts an image prop editing function, and obtains and saves a shooting target composite image by editing and adjusting the props in the shooting source composite image.
5. The method according to claim 4, characterized in that The mobile terminal edits and adjusts the props in the composite image of the shooting source by any one or a combination of the following: Adding a prop to the composite image of the shooting source; Deleting the props in the composite image of the shooting source; Performing position adjustment processing on the props in the composite image of the shooting source; Resizing the props in the composite image of the shooting source; Performing angle adjustment processing on the props in the composite image of the shooting source; as well as Performing color adjustment processing on the props in the composite image of the shooting source.
6. A photographing device for automatically matching props, characterized in that: include: An acquisition module is used to acquire a shooting scene including user image information and environmental information after the camera is started and the screen automatically decorates with props, wherein the user image information includes any one or a combination of the following: user facial features, user gesture information, user body posture information, and user voice features; The environmental information includes: shooting time, shooting location, shooting season and shooting weather; A selection module, configured to extract key shooting information in the shooting scene by analyzing the user image information and the environment information in the shooting scene respectively; and select prop information matching the key shooting information from a preset prop library according to the key shooting information; A generation module is used to superimpose the user image information and the prop information to generate a composite image to be shot.
7. The device according to claim 6, characterized in that The item information includes any one or a combination of the following: animal item information, plant item information, food item information, wearable item information, and game tool information.
8. The device according to claim 7, characterized in that The prop library stores a shooting props relationship table showing the corresponding relationship between key shooting information and props information.
9. A device for automatically matching props for taking photos, characterized in that: The device includes: a processor, and a memory coupled to the processor; the memory stores a program for automatically matching props for taking photos that can be run on the processor, and when the program for automatically matching props for taking photos is executed by the processor, the steps of the method for automatically matching props for taking photos as described in any one of claims 1 to 5 are implemented.
10. A computer storage medium, characterized in that: The computer storage medium stores a program for automatically matching props for taking photos, and when the program for automatically matching props for taking photos is executed by a processor, the steps of the method for automatically matching props for taking photos as claimed in any one of claims 1 to 5 are implemented.
