A method and device for controlling the composition of photographed images and a computer-readable storage medium

Through the judgment of the triggering conditions of composition recommendation network and cameras, intelligent switching of cameras in multiple camera devices is achieved, solving the problem of single composition and aesthetic in the existing technology, and improving the composition aesthetics and user experience.

CN114025099BActive Publication Date: 2024-11-26NUBIA TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111414228.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-25
Publication Date
2024-11-26
Estimated Expiration
2041-11-25

AI Technical Summary

Technical Problem

In the prior art, the composition method based on subject object detection is single in multiple shooting scenes and does not conform to the aesthetic mechanism, so it cannot provide an intelligent and humanized composition control scheme for shooting images.

Method used

By obtaining the preview image and inputting it to the composition recommendation network, obtaining the camera list and its trigger conditions, and determining whether the recommended composition meets the trigger conditions. If it is satisfied, switch the camera. Otherwise, retain the current camera and generate a moving option, and use the multi-camera configuration to enhance the aesthetics of the composition.

Benefits of technology

It realizes a humanized image composition scheme, makes full use of multi-camera conditions, and improves the composition aesthetics and user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114025099B_ABST
    Figure CN114025099B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, device and computer-readable storage medium for controlling the composition of captured images, wherein the method comprises: obtaining a preview image, and inputting the preview image into a preset composition recommendation network to obtain a recommended composition; obtaining a current camera list and a trigger condition list corresponding to each camera in the camera list; judging whether the recommended composition corresponding to each camera in the camera list satisfies the trigger condition corresponding to the camera in the trigger condition list; if so, generating a switching option for the camera in the preview area, and if not, retaining the current camera and generating a moving option corresponding to the recommended composition. A humanized image capture composition scheme is implemented, which makes full use of the multi-camera configuration conditions of the current device, greatly improves the aesthetic level of the composition, and enhances the user's shooting composition experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of mobile communications, and in particular to a method and device for controlling the composition of a captured image, and a computer-readable storage medium. Background Art

[0002] In the prior art, with the continuous development of smart terminal devices, users have higher and higher requirements for the shooting experience of mobile phones and other devices. In particular, in a variety of shooting scenes, such as people, animals, scenery, architecture, food, etc., providing users with intelligent and effective composition recommendations can lower the threshold for users to use mobile devices to shoot, and greatly improve the user experience.

[0003] To solve the above technical problems, the commonly used composition solutions are generally based on subject target detection, such as center-entry composition, thirds composition, etc. The composition usage scenarios are single and the composition methods are single. At the same time, this type of composition recommendation method based on subject target detection does not conform to the aesthetic mechanism.

[0004] Therefore, in the prior art, there is an urgent need for a more intelligent, humane and aesthetic image composition control solution. Summary of the invention

[0005] In order to solve the above technical defects in the prior art, the present invention proposes a method for controlling the composition of a captured image, the method comprising:

[0006] A preview image is obtained, and the preview image is input into a preset composition recommendation network to obtain a recommended composition.

[0007] Get the current camera list and the trigger condition list corresponding to each camera in the camera list.

[0008] It is determined whether the recommended composition corresponding to each camera in the camera list satisfies the trigger condition corresponding to the camera in the trigger condition list.

[0009] If there is a camera that meets the trigger condition, a switching option for the camera is generated in the preview area; if there is no camera that meets the trigger condition, the current camera is retained and a moving option corresponding to the recommended composition is generated.

[0010] Optionally, the obtaining of the preview image and inputting the preview image into a preset composition recommendation network to obtain a recommended composition may include:

[0011] Get a preset number of image datasets.

[0012] Each image in the image data set is cropped according to a preset cropping strategy, wherein the cropping strategy includes one or more of aspect ratio setting, area ratio setting, and position region setting.

[0013] Optionally, the obtaining of the preview image and inputting the preview image into a preset composition recommendation network to obtain a recommended composition may further include:

[0014] Each cropped image is input into a preset composition aesthetic scoring network to obtain a composition aesthetic score of the cropped image, and the composition aesthetic score is used as label data of the cropped image.

[0015] Each cropped image and the corresponding label data are combined into an updated image data set.

[0016] Optionally, the obtaining of the preview image and inputting the preview image into a preset composition recommendation network to obtain a recommended composition may further include:

[0017] A composition recommendation network based on composition aesthetic score is constructed, and the composition recommendation network is trained using a training set obtained by dividing the updated image dataset.

[0018] When the loss of the composition recommendation network in the test set divided from the updated image data set is less than a preset threshold, the training is stopped, and the composition recommendation network and the corresponding model parameters are solidified.

[0019] Optionally, the determining whether the recommended composition corresponding to each camera in the camera list satisfies a trigger condition corresponding to the camera in the trigger condition list includes:

[0020] A first threshold condition corresponding to the area of ​​the recommended composition is determined according to the type of the camera.

[0021] If the recommended composition corresponding to the camera is greater than the first threshold condition, it is determined that the camera meets the trigger condition.

[0022] Optionally, the determining whether the recommended composition corresponding to each camera in the camera list satisfies the trigger condition corresponding to the camera in the trigger condition list further includes:

[0023] The corresponding position condition is determined according to the type of the camera and the regional position of the recommended composition.

[0024] If the recommended composition corresponding to the camera meets the position condition, it is determined that the camera meets the trigger condition.

[0025] Optionally, the determining whether the recommended composition corresponding to each camera in the camera list satisfies the trigger condition corresponding to the camera in the trigger condition list further includes:

[0026] Corresponding neighborhood conditions are determined according to the type of the camera, the regional position of the recommended composition, and the preview center area.

[0027] If the area position of the recommended composition corresponding to the camera belongs to the neighborhood area of ​​the preview center area, it is determined that the camera meets the trigger condition.

[0028] Optionally, the determining whether the recommended composition corresponding to each camera in the camera list satisfies the trigger condition corresponding to the camera in the trigger condition list further includes:

[0029] A second threshold condition corresponding to the area of ​​the recommended composition is determined according to the type of the camera.

[0030] If the recommended composition corresponding to the camera is less than the second threshold condition, it is determined that the camera meets the trigger condition.

[0031] The present invention also proposes a device for controlling the composition of captured images, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the method for controlling the composition of captured images as described in any one of the above items are implemented.

[0032] The present invention also proposes a computer-readable storage medium, on which a shooting image composition control program is stored. When the shooting image composition control program is executed by a processor, the steps of the shooting image composition control method as described in any one of the above items are implemented.

[0033] The method, device and computer-readable storage medium for controlling the composition of captured images of the present invention are implemented by obtaining a preview image and inputting the preview image into a preset composition recommendation network to obtain a recommended composition; obtaining a current camera list and a trigger condition list corresponding to each camera in the camera list; determining whether the recommended composition corresponding to each camera in the camera list satisfies the trigger condition corresponding to the camera in the trigger condition list; if there is a camera that satisfies the trigger condition, a switching option for the camera is generated in the preview area; if there is no camera that satisfies the trigger condition, the current camera is retained and a moving option corresponding to the recommended composition is generated. A humanized image capture composition scheme is implemented, which makes full use of the multi-camera configuration conditions of the current device, greatly improves the aesthetic level of the composition, and enhances the user's shooting composition experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0035] Figure 1 It is a hardware structure diagram of a mobile terminal involved in the present invention;

[0036] Figure 2 is a communication network system architecture diagram provided by an embodiment of the present invention;

[0037] Figure 3 is a flow chart of a first embodiment of a method for controlling the composition of captured images according to the present invention;

[0038] Figure 4 is a flow chart of a second embodiment of a method for controlling the composition of captured images according to the present invention;

[0039] Figure 5 is a flow chart of a third embodiment of a method for controlling the composition of a captured image according to the present invention;

[0040] Figure 6 is a flow chart of a fourth embodiment of a method for controlling the composition of a captured image according to the present invention;

[0041] Figure 7 is a flow chart of a fifth embodiment of a method for controlling the composition of a captured image according to the present invention;

[0042] Figure 8 is a flow chart of a sixth embodiment of a method for controlling the composition of captured images according to the present invention;

[0043] Fig. 9 is a flow chart of a seventh embodiment of a method for controlling the composition of captured images according to the present invention;

[0044] Fig.104 is a flow chart of an eighth embodiment of the method for controlling the composition of captured images according to the present invention. DETAILED DESCRIPTION

[0045] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0046] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0047] The terminal may be implemented in various forms. For example, the terminal described in the present invention may include mobile terminals such as mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0048] The following description will be made by taking a mobile terminal as an example, and those skilled in the art will appreciate that, in addition to components specifically used for mobile purposes, the configuration according to the embodiments of the present invention can also be applied to fixed-type terminals.

[0049] See also Figure 1 , which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present invention, the mobile terminal 100 may include: RF (Radio Frequency, radio frequency) unit 101, WiFi module 102, audio output unit 103, A / V (audio / video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111 and other components. Those skilled in the art can understand that Figure 1 The structure of the mobile terminal shown in the figure does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0050] Combine the following Figure 1 The following is a detailed introduction to the various components of the mobile terminal:

[0051] The radio frequency unit 101 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving the downlink information of the base station, it is sent to the processor 110 for processing; in addition, the uplink data is sent to the base station. Generally, the radio frequency unit 101 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 101 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution) and TDD-LTE (Time Division Duplexing-Long Term Evolution), etc.

[0052] WiFi is a short-range wireless transmission technology. Mobile terminals can help users send and receive emails, browse web pages, and access streaming media through WiFi module 102. It provides users with wireless broadband Internet access. Figure 1 The WiFi module 102 is shown, but it is understandable that it is not an essential component of the mobile terminal and can be omitted as needed without changing the essence of the invention.

[0053] The audio output unit 103 can convert the audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, etc. Moreover, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0054] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processor (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 106. The image frame processed by the graphics processor 1041 can be stored in the memory 109 (or other storage medium) or sent via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a telephone call mode, a recording mode, a voice recognition mode, and other operating modes, and can process such sound into audio data. The processed audio (voice) data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 can implement various types of noise elimination (or suppression) algorithms to eliminate (or suppress) noise or interference generated in the process of receiving and sending audio signals.

[0055] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can also be configured on the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0056] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0057] The user input unit 107 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile terminal. Specifically, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1071 or near the touch panel 1071 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 110, and can receive and execute the command sent by the processor 110. In addition, the touch panel 1071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Specifically, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc., which are not specifically limited here.

[0058] Furthermore, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of the touch event. Then, the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of the touch event. Figure 1 In the figure, the touch panel 1071 and the display panel 1061 are used as two independent components to implement the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to implement the input and output functions of the mobile terminal, which is not limited here.

[0059] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

[0060] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 109 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0061] The processor 110 is the control center of the mobile terminal. It uses various interfaces and lines to connect various parts of the entire mobile terminal. It executes various functions of the mobile terminal and processes data by running or executing software programs and / or modules stored in the memory 109, and calling data stored in the memory 109, so as to monitor the mobile terminal as a whole. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 110.

[0062] The mobile terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0063] although Figure 1 Not shown, the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0064] To facilitate understanding of the embodiments of the present invention, the communication network system on which the mobile terminal of the present invention is based is described below.

[0065] See also Figure 2 , Figure 2 A communication network system architecture diagram is provided for an embodiment of the present invention. The communication network system is an LTE system of universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are sequentially connected for communication.

[0066] Specifically, UE201 may be the above-mentioned mobile terminal 100, which will not be described in detail here.

[0067] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 , etc. Among them, eNodeB 2021 can be connected to other eNodeBs 2022 through a backhaul (eg, an X2 interface), and eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 can provide UE 201 with access to EPC 203 .

[0068] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, PGW (PDN Gate Way) 2035 and PCRF (Policy and Charging Rules Function) 2036, etc. Among them, MME2031 is a control node that processes signaling between UE201 and EPC203, and provides bearer and connection management. HSS2032 is used to provide some registers to manage functions such as home location register (not shown in the figure), and save some user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, which selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0069] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem) or other IP services.

[0070] Although the above introduction takes the LTE system as an example, those skilled in the art should know that the present invention is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA and future new network systems, etc., which are not limited here.

[0071] Based on the above mobile terminal hardware structure and communication network system, various embodiments of the method of the present invention are proposed.

[0072] Embodiment 1

[0073] Figure 3 1 is a flow chart of a first embodiment of a method for controlling the composition of a captured image according to the present invention. A method for controlling the composition of a captured image, the method comprising:

[0074] S1. Obtain a preview image, and input the preview image into a preset composition recommendation network to obtain a recommended composition.

[0075] S2. Obtain a current camera list and a trigger condition list corresponding to each camera in the camera list.

[0076] S3. Determine whether the recommended composition corresponding to each camera in the camera list satisfies the trigger condition corresponding to the camera in the trigger condition list.

[0077] S4. If there is a camera that meets the trigger condition, a switching option of the camera is generated in the preview area; if there is no camera that meets the trigger condition, the current camera is retained and a moving option corresponding to the recommended composition is generated.

[0078] In this embodiment, it is taken into account that the current composition recommendation method is generally based on the preview of the camera currently selected by the user, and does not take into account the influence of camera selection and switching on the composition under the premise of multiple cameras; and only based on the camera currently selected by the user for composition recommendation, sometimes it is not possible to obtain a good composition recommendation effect; for example, when the user chooses a standard camera to shoot plants, if the current composition recommendation area meets the established conditions, it is recommended that the user choose a macro camera to shoot, which can obtain better composition and shooting effects.

[0079] Specifically: Step 1, obtain the preview image image_preview; Step 2, preprocess the preview image according to the established requirements to obtain the preprocessed preview image image_preprocess and the preprocessed image size: width_preprocess, height_preprocess; Step 3, input the preprocessed preview image image_preprocess into the composition proposal network CPN (Composition Proposal Network); Step 4, obtain the output CPN_out of the composition recommendation network CPN: CPN_out = {x, y, width, height, score}, where: x, y are the horizontal and vertical coordinates of the starting and ending points (generally the upper left point) of the current recommended composition area, width, height are the width and height of the current recommended composition area, and score is the aesthetic score of the current recommended composition area.

[0080] Specifically: Step 5, get the preset camera list: Lenss = {lens1, lens2, ..., lens n}; Step 6, obtain the trigger condition list corresponding to each camera: Trig = {trig1, trig2, ..., trig n}; Where: cond j To trigger the lens i The trigger conditions that need to be met; Step 7, for each lens in the camera list Lenss i ; Step 7.1, get its trigger condition list trig i ; Step 7.2, determine whether the current composition recommendation meets the current trigger conditions; Step 7.3, if it does, prompt the user to switch to the corresponding camera to shoot in the specified area in the preview; Step 8, if the trigger conditions of any camera are not met, keep using the current camera, and according to the current composition recommendation, prompt the user terminal device to move in the required direction, etc.

[0081] It can be seen that in this embodiment, a composition recommendation network based on aesthetic scoring is proposed, so that composition recommendation is not limited to a single scene or a single composition method. The composition recommendation method combined with multiple cameras proposed in this embodiment improves the rationality of composition recommendation, increases the utilization rate of multiple cameras, and improves image capture quality.

[0082] The beneficial effect of this embodiment is that a recommended composition is obtained by obtaining a preview image and inputting the preview image into a preset composition recommendation network; obtaining the current camera list and the trigger condition list corresponding to each camera in the camera list; judging whether the recommended composition corresponding to each camera in the camera list satisfies the trigger condition corresponding to the camera in the trigger condition list; if there is a camera that satisfies the trigger condition, a switching option for the camera is generated in the preview area; if there is no camera that satisfies the trigger condition, the current camera is retained and a moving option corresponding to the recommended composition is generated. A humanized image shooting composition scheme is realized, which makes full use of the multi-camera configuration conditions of the current device, greatly improves the aesthetic degree of the composition, and enhances the user's shooting composition experience.

[0083] Embodiment 2

[0084] Figure 4 : is a flow chart of a second embodiment of the method for controlling the composition of captured images of the present invention. Based on the above embodiment, the method of obtaining a preview image and inputting the preview image into a preset composition recommendation network to obtain a recommended composition includes:

[0085] S01. Obtain a preset number of image data sets.

[0086] S02. Cropping each image in the image data set according to a preset cropping strategy, wherein the cropping strategy includes one or more of aspect ratio setting, area ratio setting, and position region setting.

[0087] Optionally, in this embodiment, a predetermined number of image data sets IAMGE_DATASET are obtained; IAMGE_DATASET = {iamge1, iamge2, ..., iamge m1}.

[0088] Optionally, in this embodiment, for each image in the image dataset, iamge i , and cut according to the established strategy: The established strategies include but are not limited to: an established aspect ratio, such as 1:1, 2:3, 3:2, etc., an established area ratio, such as 0.8, 0.5, etc. of the preview area, an established location area, such as a center area, etc.

[0089] The beneficial effect of this embodiment is that by obtaining a preset number of image data sets and cropping each image in the image data set according to a preset cropping strategy, wherein the cropping strategy includes one or more of aspect ratio setting, area ratio setting, and position region setting, a humanized image composition scheme is implemented, which makes full use of the multi-camera configuration conditions of the current equipment, greatly improves the aesthetics of the composition, and enhances the user's shooting composition experience.

[0090] Embodiment 3

[0091] Figure 5 : is a flowchart of a third embodiment of the method for controlling the composition of captured images of the present invention. Based on the above embodiment, the method of obtaining a preview image and inputting the preview image into a preset composition recommendation network to obtain a recommended composition also includes:

[0092] S03. Input each cropped image into a preset composition aesthetic scoring network to obtain a composition aesthetic score of the cropped image, and use the composition aesthetic score as label data of the cropped image.

[0093] S04. Combining each cropped image and the corresponding label data into an updated image data set.

[0094] Optionally, in this embodiment, each image is imaged i Each crop image of crop j, input the existing aesthetic scoring network to obtain each crop image crop j The aesthetic score of the cropped image j Label data label j :

[0095] Optionally, in this embodiment, each image is imaged i Each crop image of crop j and its label data to form a new image data set IAMGE_CROP_DATASET; IAMGE_DATASET = {image_data1, image_data2, ..., image_data m2}, where: image_data i ={image i ,image_crop i ,image_label i}.

[0096] The beneficial effect of this embodiment is that by inputting each cropped image into a preset composition aesthetic scoring network, the composition aesthetic score of the cropped image is obtained, and the composition aesthetic score is used as the label data of the cropped image; each cropped image and the corresponding label data form an updated image data set. A humanized image composition solution is realized, which makes full use of the multi-camera configuration conditions of the current equipment, greatly improves the composition aesthetics, and enhances the user's shooting composition experience.

[0097] Embodiment 4

[0098] Figure 6 : is a flowchart of a fourth embodiment of the method for controlling the composition of captured images of the present invention. Based on the above embodiment, the method of obtaining a preview image and inputting the preview image into a preset composition recommendation network to obtain a recommended composition also includes:

[0099] S05. Construct a composition recommendation network based on composition aesthetic score, and train the composition recommendation network using a training set obtained by dividing the updated image dataset.

[0100] S06. When the loss of the composition recommendation network in the test set obtained by dividing the updated image data set is less than a preset threshold, the training is stopped, and the composition recommendation network and the corresponding model parameters are solidified.

[0101] Optionally, in this embodiment, the image data set IAMGE_CROP_DATASET is divided into image subsets including but not limited to a training set IAMGE_CROP_DATASET_TRAIN and a test set IAMGE_CROP_DATASET_TEST. A composition recommendation network based on aesthetic scoring is constructed; the aesthetic scoring network is input to each data image_data in the given data set. i Image in i , the output is the aesthetic score of each sub-image under the established cropping strategy.

[0102] Optionally, in this embodiment, the composition recommendation network is trained using the training set in the data set so that the network output net_out and the label data image_label i The loss between them decreases, and the network model parameters are optimized by back propagation: When the loss of the composition recommendation network in the test set is less than a predetermined threshold loss_TH, the training is stopped. Finally, the network model structure and network model parameters are solidified.

[0103] Optionally, in this embodiment, the method for using the composition recommendation network includes: obtaining a solidified model structure file and a model parameter file of the composition recommendation network; restoring the network model using the solidified model structure file and the model parameter file of the composition recommendation network; obtaining the network model input; obtaining the network model output net_out; obtaining the sub-image with the highest aesthetic score in the network model output as the composition recommendation output CPN_out of the composition recommendation network CPN.

[0104] The beneficial effect of this embodiment is that, by constructing a composition recommendation network based on composition aesthetic score, the composition recommendation network is trained with the training set obtained by dividing the updated image data set; when the loss of the composition recommendation network in the test set obtained by dividing the updated image data set is less than a preset threshold, the training is stopped, and the composition recommendation network and the corresponding model parameters are solidified. A humanized image composition scheme is realized, which makes full use of the multi-camera configuration conditions of the current equipment, greatly improves the aesthetic degree of composition, and enhances the user's shooting composition experience.

[0105] Embodiment 5

[0106] Figure 7 is a flow chart of a fifth embodiment of the method for controlling the composition of captured images of the present invention. Based on the above embodiment, the step of determining whether the recommended composition corresponding to each camera in the camera list satisfies the trigger condition corresponding to the camera in the trigger condition list includes:

[0107] S31. Determine a first threshold condition corresponding to the area of ​​the recommended composition according to the type of the camera.

[0108] S32: If the recommended composition corresponding to the camera is greater than the first threshold condition, determine that the camera meets the trigger condition.

[0109] Optionally, in this embodiment, a preset camera list is obtained: Lenss = {lens1, lens2, ..., lens n}; Get the trigger condition list corresponding to each camera: Trig = {trig1, trig2, ..., trig n}; Where: cond j To trigger the lens i The trigger conditions that need to be met.

[0110] Optionally, in this embodiment, lens i For a macro camera, the corresponding trigger condition may be: cond1 is that the area of ​​the current recommended composition area is greater than a predetermined first threshold:

[0111] The beneficial effect of this embodiment is that the first threshold condition corresponding to the area of ​​the recommended composition is determined by the type of the camera; if the recommended composition corresponding to the camera is greater than the first threshold condition, it is determined that the camera meets the trigger condition. A humanized image composition solution is implemented, which makes full use of the multi-camera configuration conditions of the current equipment, greatly improves the aesthetics of the composition, and enhances the user's shooting composition experience.

[0112] Embodiment 6

[0113] Figure 8 is a flow chart of a sixth embodiment of the method for controlling the composition of captured images of the present invention. Based on the above embodiment, the step of determining whether the recommended composition corresponding to each camera in the camera list satisfies the trigger condition corresponding to the camera in the trigger condition list further includes:

[0114] S33: Determine corresponding location conditions according to the type of the camera and the area location of the recommended composition.

[0115] S34: If the recommended composition corresponding to the camera meets the position condition, determine that the camera meets the trigger condition.

[0116] The beneficial effect of this embodiment is that the corresponding position condition is determined by the type of the camera and the regional position of the recommended composition; if the recommended composition corresponding to the camera meets the position condition, it is determined that the camera meets the trigger condition. A humanized image shooting composition solution is realized, which makes full use of the multi-camera configuration conditions of the current equipment, greatly improves the aesthetic degree of the composition, and enhances the user's shooting composition experience.

[0117] Embodiment 7

[0118] Fig. 9 is a flow chart of a seventh embodiment of the method for controlling the composition of captured images of the present invention. Based on the above embodiment, the step of determining whether the recommended composition corresponding to each camera in the camera list satisfies the trigger condition corresponding to the camera in the trigger condition list further includes:

[0119] S35. Determine corresponding neighborhood conditions according to the type of the camera, the regional position of the recommended composition, and the preview center area.

[0120] S36: If the area position of the recommended composition corresponding to the camera belongs to the neighborhood area of ​​the preview center area, it is determined that the camera meets the trigger condition.

[0121] Optionally, in this embodiment, cond3 is that the current recommended composition area is located within a predetermined neighborhood range of the preview center area.

[0122] in:

[0123]

[0124]

[0125]

[0126]

[0127] The beneficial effect of this embodiment is that the corresponding neighborhood condition is determined by the type of the camera, the regional position of the recommended composition, and the preview center area; if the regional position of the recommended composition corresponding to the camera belongs to the neighborhood area of ​​the preview center area, it is determined that the camera meets the trigger condition. A humanized image shooting composition solution is realized, which makes full use of the multi-camera configuration conditions of the current equipment, greatly improves the aesthetic degree of the composition, and enhances the user's shooting composition experience.

[0128] Embodiment 8

[0129] Fig.10is a flow chart of an eighth embodiment of the method for controlling the composition of captured images of the present invention. Based on the above embodiment, the step of determining whether the recommended composition corresponding to each camera in the camera list satisfies the trigger condition corresponding to the camera in the trigger condition list further includes:

[0130] S37. Determine a second threshold condition corresponding to the area of ​​the recommended composition according to the type of the camera.

[0131] S38: If the recommended composition corresponding to the camera is less than the second threshold condition, determine that the camera meets the trigger condition.

[0132] Optionally, in this embodiment, cond2 is that the area of ​​the currently recommended composition region is smaller than a predetermined second threshold:

[0133] The beneficial effect of this embodiment is that the second threshold condition corresponding to the area of ​​the recommended composition is determined by the type of the camera; if the recommended composition corresponding to the camera is less than the second threshold condition, it is determined that the camera meets the trigger condition. A humanized image composition solution is implemented, which makes full use of the multi-camera configuration conditions of the current device, greatly improves the aesthetics of the composition, and enhances the user's shooting composition experience.

[0134] Embodiment 9

[0135] Based on the above embodiments, the present invention also proposes a device for controlling the composition of photographed images, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method for controlling the composition of photographed images as described in any one of the above items.

[0136] It should be noted that the above-mentioned device embodiment and method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, and the technical features in the method embodiment are correspondingly applicable in the device embodiment, which will not be repeated here.

[0137] Embodiment 10

[0138] Based on the above embodiments, the present invention also proposes a computer-readable storage medium, which stores a shooting image composition control program. When the shooting image composition control program is executed by a processor, the steps of the shooting image composition control method as described in any one of the above items are implemented.

[0139] It should be noted that the above-mentioned medium embodiment and method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, and the technical features in the method embodiment are correspondingly applicable in the medium embodiment, which will not be repeated here.

[0140] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0141] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0142] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0143] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A method for controlling the composition of a captured image, characterized in that: The method comprises: Acquire a preview image, and input the preview image into a preset composition recommendation network to obtain a recommended composition; Obtain the current camera list and the trigger condition list corresponding to each camera in the camera list; Determine whether the recommended composition corresponding to each camera in the camera list satisfies the trigger condition corresponding to the camera in the trigger condition list; If there is a camera that meets the trigger condition, a switching option of the camera is generated in the preview area; if there is no camera that meets the trigger condition, the current camera is retained and a moving option corresponding to the recommended composition is generated; The determining whether the recommended composition corresponding to each camera in the camera list satisfies the trigger condition corresponding to the camera in the trigger condition list includes: Determining a first threshold condition corresponding to the area of ​​the recommended composition according to the type of the camera; If the recommended composition corresponding to the camera is greater than the first threshold condition, determining that the camera meets the trigger condition; Determining corresponding position conditions according to the type of the camera and the regional position of the recommended composition; If the recommended composition corresponding to the camera meets the position condition, determining that the camera meets the trigger condition; Determining corresponding neighborhood conditions according to the type of the camera, the regional position of the recommended composition, and the preview center area; If the area position of the recommended composition corresponding to the camera belongs to the neighborhood area of ​​the preview center area, it is determined that the camera meets the trigger condition; Determining a second threshold condition corresponding to the area of ​​the recommended composition according to the type of the camera; If the recommended composition corresponding to the camera is less than the second threshold condition, it is determined that the camera meets the trigger condition.

2. The method for controlling the composition of captured images according to claim 1, characterized in that: The method of obtaining a preview image and inputting the preview image into a preset composition recommendation network to obtain a recommended composition includes: Obtain a preset number of image data sets; Each image in the image data set is cropped according to a preset cropping strategy, wherein the cropping strategy includes one or more of aspect ratio setting, area ratio setting, and position region setting.

3. The method for controlling the composition of captured images according to claim 2, characterized in that: The step of obtaining a preview image and inputting the preview image into a preset composition recommendation network to obtain a recommended composition may also include: Inputting each cropped image into a preset composition aesthetic scoring network to obtain a composition aesthetic score of the cropped image, and using the composition aesthetic score as label data of the cropped image; Each cropped image and the corresponding label data are combined into an updated image data set.

4. The method for controlling the composition of captured images according to claim 3, characterized in that: The step of obtaining a preview image and inputting the preview image into a preset composition recommendation network to obtain a recommended composition may also include: Constructing a composition recommendation network based on composition aesthetic score, and training the composition recommendation network with a training set obtained by dividing the updated image dataset; When the loss of the composition recommendation network in the test set divided from the updated image data set is less than a preset threshold, the training is stopped, and the composition recommendation network and the corresponding model parameters are solidified.

5. A device for controlling the composition of a photographed image, characterized in that: The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the method for controlling the composition of captured images as described in any one of claims 1 to 4 are implemented.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a captured image composition control program, and when the captured image composition control program is executed by the processor, the steps of the captured image composition control method according to any one of claims 1 to 4 are implemented.